Many manufacturers have started talking about Pixel Binning, a technique that is supposed to improve image quality, but what exactly is it and how does it work? We explain
The rate at which smartphone cameras have been getting better has been slowing down over the last few years. While lens apertures and overall image quality have been changing over the years, the biggest limitation in making the “next big leap” when it comes to smartphone photography is the imaging sensor. Flagship cameras have all stuck to the 12-megapixel resolution and while the Huawei P20 Pro (Review) did use a denser, larger sensor, it is an outlier when it comes to smartphones. When it comes to overall image quality, there is only so far software can go, and after a point, you do need to improve the hardware. While not many companies would want to make the effort of using large-format sensors, Pixel Binning is the last hurrah before we hit the limit of how far we can push existing hardware choices.
Understanding the Sensor
Unlike what we understand a pixel to be when we look at an image, the “pixels” on a sensor has a red, green or blue filter on top of it, allowing only the light that passes through these filters to be registered. These filtered pixels are laid out in alternate patterns, with green pixels being two times that of the red and blue pixels in any given horizontal or vertical line. The adjacent red green and blue pixels are combined into a single RGB pixel using a process called ‘demosaicing.’ The binning process takes place during the demosaicing stage, even before the JPG is generated.
What is Pixel Binning
The obvious question is, what is Pixel Binning? The technique is essentially an Image Signal Processor level implementation where the demosaicing process combines information from four pixels into 1. For simplicity sake, let's just say that pixel binning takes pixels in a 2×2 grid and combines them into one. The information captured by each of the individual pixels are now combined into one large Super Pixel. The unfortunate result of this is that the effective resolution of the resulting image is 1/4th the resolution of the sensor. The combination of data collected by 4 adjacent pixels into one “super pixel” is called pixel binning and is helping smartphone manufacturers squeeze the last bit of performance from existing sensor designs. Depending on how the sensor is configured, manufacturers could program is to perform 2×2 pixel binning, 2×1 binning (horizontal pixels) or 1×2 binning (vertical pixels). The latter two techniques are not being used in consumer imaging devices.
Benefits of Pixel Binning
Pixel Binning alleviates one big problem that plagues almost every smartphone camera; low light photography. When shooting in low light, the sensor sensitivity has to be boosted significantly so that it can see in the dark. This degrades the signal to noise ratio significantly per pixel, resulting in not just noise, but also lowered dynamic range, poor colour reproduction and at times, even poor exposure. By combining raw data from four pixels into one, the effective output is a higher amount of light captured, while reducing the noise generated. Without actually artificially boosting the “signal” (amount of light perceived by the sensor, usually done by increasing the ISO), the resulting image is much brighter when Pixel binning is used simply because of the combination of four pixel-data into one. In basic terms, this means that in low light, you won’t have to boost the ISO too high to get a well-exposed image. Along with that, the resulting images will also have lower amounts of noise. Additionally, the benefits of pixel binning can even be noticed in good light, with slightly better dynamic range and smoother gradation between colours, but it isn't a remarkable difference and does not necessarily justify the resolution trade-off. Here are some samples of what pixel binning can achieve, vs. a non-pixel binned photo. All photos were shot on the LG G7+ThinQ in auto mode, HDR off and pixel binning turned on/off manually.
Pixel Binning turned off
Pixel Binning turned On
Pixel Binning turned off
Pixel Binning turned On
The above samples have been resized for web, but you can see the original, straight out of camera image samples in our Flickr Gallery
Pixel binning is definitely a very smart way of getting the most out of a sensor, but there are limitations even to this implementation.
Pixel Binning Limitation #1
Because the resolution is reduced by 1/4th, the image sensor used needs to be of a high enough megapixel count. Huawei got this very right with the P20 Pro, where they used a 40-megapixel sensor to generate a 10-megapixel image which set a benchmark for the entire industry. But for sensors with a lower resolution like the LG G7+ ThinQ which has a 16-megapixel sensor, the resulting image when binned is just 4 megapixels, resulting in a lower level of details captured. However, the images are brighter, making “more visible to the eye”. While you do get to see more "illumination" in the photo, the detail from objects that have been shot is significantly lower due to the reduced resolution.
Pixel Binning Limitation #2
The other downside to the use of pixel binning is the lack of RAW output. Since RAW output is the encapsulation of the raw sensor data, a pixel-binned version cannot exist. If the JPG algorithms being used by the smartphone manufacturer are aggressive in favour of reduced file size, the compression would result in even more detail degradation, to the point where photos might look like oil paintings. If the manufacturer allows it, you can either record a RAW image file (without pixel binning) or a pixel-binned JPG.
Current Smartphones using Pixel Binning
There are a number of smartphones available today in the market that employs pixel binning. We already know the Huawei P20 Pro does it, along with the LG G7+ ThinQ (Review), but there are a few budget-friendly smartphones that do the same. For example, the Xiaomi Redmi Y2 (Review) uses pixel binning for the front facing 16-megapixel sensor, the Xiaomi Mi A2 (Review) uses it for both front and back cameras. There are more smartphones out there using the technology, but some of them choose to not talk much about this feature. Sony also recently announced a new 48-megapixel sensor, the IMX586, that is designed specifically keeping pixel binning in mind. Unfortunately, no smartphone company has yet announced the use of this sensor in their smartphone, but we do expect it to make an appearance very soon.
In Conclusion
Pixel Binning trades sensor resolution for better illumination in an image and truly comes in handy when shooting in low light. While you do lose 1/4th of the sensor resolution, you end up with images that are social media worthy. While it is a remarkable stop-gap solution to the low light problem, there is still no substitute for a larger sensor, one with only a slightly higher resolution, but with no need for binning. For example, if the sensor on the Huawei P20 Pro had a resolution of 16 or 20 megapixels (instead of 40) while maintaining its 1/2" size, it would have been an even better performer than it is today. Given the space constraints within a smartphone, the use of large format sensors becoming a standard practice is very unlikely.
Follow Us
Swapnil Mathur
Swapnil was Digit's resident camera nerd, (un)official product photographer and the Reviews Editor. Swapnil has moved-on to newer challenges. For any communication related to his stories, please mail us using the email id given here. View Full Profile
