What is pixel binning?  Smartphone sensor technology explained

What is pixel binning? Smartphone sensor technology explained

November 9, 2022

The cameras in your smartphone have come a long way in just a few years. Advances in sensor technology combined with the use of artificial intelligence algorithms have allowed smartphone cameras to produce images with more accurate colors, reduced noise, and more fine detail.

At the heart of this technology is a process called pixel binning. Even the most technically savvy photographers may find themselves asking, “What is pixel binning?” It’s a relatively new concept in sensor technology and something to consider when upgrading to your next smartphone.

What is pixel binning?

Pixel grouping is a process by which a sensor groups – or “boxes” – adjacent pixels together to make one larger pixel in order to reduce the amount of noise. The most common matrix, for example, is 2×2 binning (some smartphone manufacturers use 3×3 and 4×4 binning), which combines four pixels into one. In this group, the sensor effectively reduces the number of pixels by a quarter. This means that you also reduce the resolution of the image and the amount of information that the sensor can record.

Why would you want your sensor to do this? By processing less data, “blocked” images have lower noise levels. It is ideal when you are using the small sensor of your smartphone in low light conditions.

In short, pixel grouping is a process by which smartphone manufacturers can increase image quality without increasing sensor size. To answer the question “what is pixel grouping” more comprehensively, it is useful to understand the nature and design of the image sensor.

What is pixel binning?

Understand sensor size

It may seem like general basic knowledge, but we often forget how much a camera’s sensor size affects image quality. If you have two cameras with the same number of pixels, a camera with a larger sensor size will produce higher quality photos. This is because individual pixels on a larger sensor will be larger than those on a physically smaller sensor. For example, individual pixels on a full-frame sensor will be larger than those on a 1-inch type sensor.

It’s helpful to think of sensor pixels as light buckets – the larger the bucket, the more light it can collect, and thus more detail from a scene can be recorded.

When a pixel collects light, the sensor converts it into a digital image signal. Therefore, the more light a pixel can collect, the stronger the image signal it can produce.

Since smartphones have sensors that are too small physically to fit in their slim design, their sensors have tiny pixels. Smaller pixels that capture less light tend to produce images with high levels of noise and poor color rendering, especially in low light. This is why manufacturers use a pixel binning process to artificially increase the size of each pixel, allowing the sensor to record more light.

With pixel binning, how many pixels are actually in my smartphone’s camera?

This is a very good question. Since pixel grouping means merging a group of smaller pixels together into one larger pixel, you are effectively reducing the number of pixels. And phone manufacturers that support pixel binning are not always clear about this when explaining the number of pixels in their phones. This is because consumers usually make purchases based on a higher pixel count and don’t always understand the nuances of why sometimes fewer and bigger pixels are better.

iPhone14Pro

iPhone 14 Pro. Credit: Amy Davis.

For example, it was announced recently iPhone 14 Pro Supports pixel binning. On paper, the iPhone 14 Pro’s 48MP camera is the highest resolution camera ever in an iPhone. However, when you group its pixels into a 2 x 2 matrix, as we talked about above, it reduces the number of pixels by a quarter. Therefore, its camera is finally capable of producing a 12MP image with pixel binning.

Ideally you want a smartphone that gives you the option to turn pixel binning on and off because if you are using it in ideal lighting conditions you are not using your sensor to its full potential if pixel binning is enabled. Fortunately, most smartphones now allow you to turn pixel binning on and off. But usually the default is to turn it on. So be sure to turn it off before taking what you hope will be a high-resolution photo.

Why don’t manufacturers make fewer pixels?

You may ask yourself, what is the use of pixel binning? Why don’t smartphone manufacturers make sensors with fewer but bigger pixels? It’s a very good question! There are two main reasons.

First, it’s a good idea to have the option to shoot at a higher resolution if conditions require it. For example, if you’re walking through Yosemite on a bright summer’s day and have a great view of El Capitan in all its glory, wouldn’t you like to take a rich, detailed photo that uses the full potential of your sensor? However, equally, if you stay in that spot until sunset to capture the submerged cliff in low light, you may want to take a photo at a lower resolution that produces less noise.

And secondly, the importance of pixels to consumers. For most people who are upgrading their phones, they’ll be more attracted to Samsung’s 108MP sensor that grabs the headlines than the longer description of the better signal-to-noise ratio that can be achieved when capturing pixel binned photos at a quarter of that resolution. It’s like an audio clip of a politician: The quickest and most effective sale is often the most effective.

How about a Bayer filter in pixel grouping?

While we think of pixel clustering as a relatively new technical advance, astronomers and astrophotographers have already used it for decades. When digital photography emerged in the early 2000s, photographers used monochrome cameras for astrophotography that had pixel binning options. They were combining images captured using red, green, and blue filters covering each pixel, making the production of the combined image easy.

However, modern digital cameras use a more complex setup. All modern digital cameras are also monochrome cameras but use color filter arrays to produce images with the accurate color tones we expect. Until recently, nearly every camera used what’s called a Bayer filter array – a kind of checkerboard pattern to arrange RGB color filters on a 2×2 pixel grid. This grid consists of two green filters sitting diagonally apart, and red and blue filters on top of the other two pixels.

Normally, a camera that works with a Bayer filter matrix will compute the sum of all this color data into a single value, but with pixel binning, this doesn’t work. Manufacturers needed a way to store each color individually.

To this end, manufacturers have designed a so-called quad-bayer matrix in which one color is assigned to each 2×2 group of pixels. Four of those are then grouped together similar to the original Bayer filter combination of 2x green, 1x blue, and 1x red. The diagram below from a Sony semiconductor module shows how this works.

Left: standard Bayer matrix, right: four-layer matrix

Left: Standard Bayer Group, Right: Bayer Quad Group

The quad-bayer matrix (which Apple refers to when it uses the term ‘quad-pixels’) not only allows smartphone manufacturers to preserve color data in the pixel binning process but also enables them to offer other innovative features such as HDR Photo mode.

Quad Bayer array can be used in two ways, either for high resolution or high sensitivity

Quad Bayer array can be used in two ways, either for high resolution or high sensitivity

When do you use pixel binning

Pixel binning is very useful when shooting in low light conditions. This is the time when you will experience the most noise. But if we’re being honest, how often do you really need to shoot at the highest resolution with your smartphone?

On cloudy days or even sunny days with a lot of contrast, you may find that shooting at lower resolutions but larger pixels helps you produce better images with a full range of color tones.

It should also be noted that pixel binning does not support raw capture. So, if you’re looking to shoot raw files for more flexibility in post-production, a smartphone camera can only output raw files that aren’t stored in pixels.

What phones use pixel binning?

More and more smartphones are using pixel binning every year across all manufacturers. As a rule, a quick way to find out if a device offers pixel binning is if the camera has a really high pixel count. 108 mega pixels Samsung Galaxy S21 Ultrafor example, uses pixel binning, as does the main 50MP sensor in Google Pixel 7 Pro Telephoto Camera (50 MP to 12.5 MP) and Telephoto Camera (48 MP to 12 MP).

Google Pixel 7 Pro camera strap, Photo: Joshua Waller / AP

Google Pixel 7 Pro camera strap, Photo: Joshua Waller / AP

And it’s not just rear cameras. Many manufacturers now offer pixel binning in front-end selfie cameras as well. Which makes sense when you think about it. You’ll most likely take a selfie in one of two places – on vacation or out at night with friends. In the latter low-light conditions, the pixelated image will be clearer and the colors better.

What are common pixel matrices?

The 4-in-1 (2×2) matrix is ​​the most common. You’ll often see these 48MP cameras squeezed into 12MP or 64MP to 16MP cameras. But some brands, such as Samsung, use a 9-in-1 matrix, assembling a 108MP camera to produce 12MP photos.

Some brands – including Samsung – are testing 200-megapixel sensors that use 16-cell pixel grouping to produce a 12.5-megapixel image.

questions and answers

Below we’ve answered the most common questions people ask about pixel grouping.

What is the advantage of pixel grouping?

The benefit of pixel grouping is that you can merge a 2 x 2 pixel group of 4 pixels into one larger pixel. This results in clearer images with less noise in low light conditions.

What are the disadvantages of pixel binning?

The downside of pixel binning is that you have a lower resolution image. So while there is less noise in your image, there will also likely be less detail. Sony A7S series chooses 12MP sensor as the quality of 4K output stacked pixel sensor is not as good as the original resolution sensor. However, it will be difficult for you to find out.

Do DSLRs and mirrorless cameras use pixel binning?

Camera manufacturers can incorporate pixel binning into mirrorless and DSLR cameras, but that hasn’t been done yet. The closest technology to date is Pentax’s Pixel Shift High Resolution mode. It does not increase the pixel count of an image but it increases the quality by combining multiple images.

If fewer and more pixels is better, why do manufacturers make sensors with a higher pixel count?

If a 12MP sensor like the one on the Sony A7S or even the Nikon D300s from last year is proven capable, then a 12MP sensor will serve you just fine. Provided you want to look at your photos on a digital screen, that is. The issue is printing. If you want to print your photos larger than A4 or A3, you need more resolution

When 12MP images are displayed on a smartphone screen, the resolution of these screens is often not enough to see the entire image in full resolution. However, if you want to view an image from a 108MP camera like Samsung’s print camera, the print size will be much larger. A 12-megapixel photo with a resolution of 4621 x 2596 printed at a resolution of 300 pixels per inch is 39.12 x 21.98 cm. An image with a resolution of 108 megapixels will be 13865 x 7789 and a print of 117.39 x 65.95 cm.

There are also considerations when scaling. If you print a photo from the iPhone 11 Pro versus a photo from the iPhone 14 Pro, the higher resolution captures much greater detail and the image width dimensions of the latter nearly double.

The higher resolution and larger physical sensor give you more flexibility to make adjustments to your photos in post-production. Noise from using high ISO settings is also random. Combining the signal from four pixels to create a single imaging pixel is likely to produce a better quality image. The sensors in smartphones are small, so the pixels are small. This makes high ISO noise more of an issue than it would be with a larger sensor.

Featured image credit: Steve Johnson via Unsplash.


Related articles:

Best camera phones in 2022

iPhone 14 Pro vs Samsung S22 Ultra: Camera Comparison

John Bentley: Pixels vs. Noise: Are Bigger Pixels Better?


Follow AP on FacebookAnd the TwitterAnd the InstagramAnd the Youtube.

SubscriptionSubscription


#pixel #binning #Smartphone #sensor #technology #explained

Leave a Comment

Your email address will not be published. Required fields are marked *