Image Sensors and Cell phone Camera Performance

The article below demonstrates the importance of semiconductor and software developments on cell phone cameras. Semiconductor chip image sensor size, back side illumination (BSI), image signal processing, and gyroscope data processing improved image stabilization. 

Ron
Insightful, timely, and accurate semiconductor consulting.
Semiconductor information and news at - http://www.maltiel-consulting.com/

Your Smartphone Camera Should Suck. Here’s Why It Doesn’t

JOSH VALCARCEL/WIRED

SMARTPHONE CAMERAS ARE great, or at least close enough to great that you don’t notice the difference. We’ve reached the point where you’ve got to work pretty hard to find a phone with a mediocre camera, and when you do, it is an anachronism to be mocked and derided—and passed over for a phone with a better one.

It wasn’t always this way, of course. There was a time, not too long ago, when smartphone cameras sucked. They took genuinely bad photos that were underexposed or overexposed or grainy or … well, you remember. And if you don’t, consider yourself lucky. It’s taken a few years, but nowadays people take a great camera for granted. Thank companies like Nokia, which started pushing that envelope in 2007, and Apple, which gave the iPhone 4 the first camera that made people go, “Daaaaaaaamn.”

How did this happen? When you consider things like sensor size, pixel density, controls, and optics, smartphone cameras should be pretty lousy. Compared to a DSLR, they still are. But the camera in your pocket is crazy good considering the limitations manufacturers work under. And the advancements keep coming. As we look to the future, the cameras in our phones are only going to get better.

The Limits of Size

No matter what kind of camera you’re talking about, there’s a universal truth: the bigger the image sensor, the better the image. A bigger sensor will capture more detail with wider dynamic range (the detail in dark and light areas), offer superior low-light performance, and focus more sharply on moving objects. However, with few exceptions, smartphone cameras have tiny sensors

The vast majority of top-tier smartphones use Sony sensors for their main cameras and Samsung sensors for their front-facing selfie cameras. And every phone on DxOMark’s list of the 10 smartphones with the best image quality has a sensor size between 1/2.3 and 1/3 inches. In terms of surface area, the one-inch sensor in a nice point-and-shoot like Sony’s RX100 is more than six times bigger than any of the top smartphone camera sensors, while the sensor in a consumer DSLR is around 19 times bigger. Drop the cash for a pro-grade DSLR and the sensor is 50 times the size of that puny thing in your iPhone 6S.

This means smartphone sensors struggle to harness light—glance at a smartphone photo taken in low light and compare it to one shot with a DSLR. It’s no contest. But a little clever engineering has made smartphones better than they ought to be. “Backside illumination” or BSI, moves some wiring to the back of the sensor, maximizing the surface area upon which photons can hit the photosites. Another trick is using a 4 megapixel sensor with a 1/3-inch image sensor. Yes, this decreased overall resolution, but also pixel density, making them better performers in the dark.

But these are workarounds. Why not just use a bigger sensor to begin with? Because there are a lot of challenges to packing one into something so small as a phone, not the least of which is heat. “With a larger sensor that takes up more real estate, this leaves less room for heat dissipation,” says Dan Unger, a Panasonic spokesman. “Add to this the current demands of the larger sensor, and heat can be a real challenge to manage.”

You can get around that, as Panasonic did, by making the phone noticeably thicker. That improves thermal management. And it’s fine for a phone designed largely for photography, but it’s not an ideal solution. If smartphones are going to use bigger senses but not fill your pocket like a Stephenson novel, there’s a lot of work to do—especially when you consider bigger sensors cost more. And if shoot video, whoa does that generate some heat.

“When you add the movie into the overall equation, then you’re putting a lot more demand on the camera in terms of dealing with heat generation,” says Chuck Westfall, a technical adviser at Canon. “The larger the sensor, the greater the heat-generation possibilities become. In a small space like a compact camera, it works against you a little bit. It’s much more so for video than it is for still imaging.”

Imaging Is a Process

The sensor simply senses light and converts it into an electrical signal. To use an analogy, it buys the groceries. Someone else cooks dinner. So while a high-quality sensor helps, it’s hardly the most important component. The lens is important, of course, but the biggest difference between a great camera and a good camera is the image signal processor—the secret sauce to any smartphone camera’s features and performance.
Hung says that the image sensor isn’t the only thing feeding information into the ISPs. A modern smartphone has several sensors at its disposal. “The gyroscope has evolved in terms of image stabilization,” he says. “A lot of the ISPs now can take the input from the gyroscope (and) combine that input with the image sensor to provide image stabilization. It’s a new kind of digital stabilization system.”

Apple and Samsung use their own image signal processors for the iPhone and Galaxy phones, respectively. However, many high-end Android handsets use the integrated image signal processors in Qualcomm’s Snapdragon system-on-a-chip, which keeps camera features relatively consistent from phone to phone. As good as it is, the company says the next-gen processor arriving early in 2016 will improve noise reduction, artifact correction, autofocus, and color reproduction.

Optics Will Stay Simple

The molded plastic lens elements in many cameras have reached the point where they’re essentially perfect. They’re also cheap. Oh sure, critics argue they don’t have an optical zoom. There’s a reason for that. Optical zooms have moving parts, which runs counter to the phone industry’s slimmer uber alles mentality. You want an optical zoom? You’ll have to accept fatter phones.

“I think most users care more about a good-looking phone and image quality than perhaps that extra bit of functionality,” Hung says. “The people that care will get lens accessories to do those things. I don’t see those more advanced things being built into many phones.”

However, some patented technologies could hasten the arrival of optical zooms. Pretty much every point-and-shoot has an optical zoom as great as 5x with lenses housed entirely within the the camera. Hell, Canon’s patented a 45X zoom folded-optics lens, but has no plans to use it anytime soon.

“The overwhelming objective on a smartphone is to keep the physical size of the device to a minimum in terms of thickness,” Westfall says. “There is going to be a limitation no matter what in terms of the quality of the lens that they’re able to put in there. Not just in terms of resolution, but in terms of focal length range and aperture as well.”

The Future of Smartphone Cameras

Earlier this year, Apple bought the image-sensor companyLinX, which uses an array of lenses to enable Lytro-like refocusing, create 3-D depth maps, and improve image quality in low light. The Light L16 camera, which uses 16 lenses and sensors to recreate the surface area and low-light capabilities of a DSLR sensor, is also due next year. The company’s founders hinted that the L16’s multi-sensor technology could show up in smartphones before long.

But as good as they are, smartphone cameras probably won’t ever match the quality of a DSLR. And they probably won’t have to. For all but the most serious photographer, the ease of a smartphone camera, and the plethora of apps that can make a crappy photo look good, is plenty. And the two approaches are complimentary. Although the smartphones have decimated the point-and-shoot segment, sales of DSLR and other high-end rigs remain strong.

As long as that’s the case—as long as DSLR cameras take a better picture, you can bet the companies making sensors and processors for smartphone cameras will continue pushing the technology further.