The article
below demonstrates the importance of semiconductor and software developments on
cell phone cameras. Semiconductor chip image sensor size, back side
illumination (BSI), image signal processing, and gyroscope data processing
improved image stabilization.
RonInsightful, timely, and accurate semiconductor consulting.Semiconductor information and news at - http://www.maltiel-consulting.com/
JOSH VALCARCEL/WIRED
SMARTPHONE CAMERAS ARE great, or at least close enough to great that you don’t notice the
difference. We’ve reached the point where you’ve got to work pretty hard to
find a phone with a mediocre camera, and when you do, it is an anachronism to
be mocked and derided—and passed over for a phone with a better one.
It wasn’t always this way, of course. There was a time,
not too long ago, when smartphone cameras sucked. They took genuinely bad
photos that were underexposed or overexposed or grainy or … well, you remember.
And if you don’t, consider yourself lucky. It’s taken a few years, but nowadays
people take a great camera for granted. Thank companies like Nokia, which
started pushing that envelope in 2007, and Apple, which gave the iPhone
4 the first camera that made people go, “Daaaaaaaamn.”
How did this happen? When you consider things like sensor
size, pixel density, controls, and optics, smartphone cameras should
be pretty lousy. Compared to a DSLR, they still are. But the camera
in your pocket is crazy good considering the limitations manufacturers work
under. And the advancements keep coming. As we look to the future, the cameras
in our phones are only going to get better.
The
Limits of Size
No matter what kind of camera you’re talking about,
there’s a universal truth: the bigger the image sensor, the better the image. A
bigger sensor will capture more detail with wider dynamic range (the detail in
dark and light areas), offer superior low-light performance, and focus more
sharply on moving objects. However, with few exceptions, smartphone cameras
have tiny sensors
The vast majority of top-tier smartphones use Sony
sensors for their main cameras and Samsung sensors for their front-facing
selfie cameras. And every phone on DxOMark’s list of the 10 smartphones
with the best image quality has a sensor size between 1/2.3 and 1/3 inches. In terms of
surface area, the one-inch sensor in a nice point-and-shoot like Sony’s RX100 is
more than six times bigger than any of the top smartphone camera sensors, while
the sensor in a consumer DSLR is around 19 times bigger. Drop the cash for a
pro-grade DSLR and the sensor is 50 times the size of that puny thing in your
iPhone 6S.
This means smartphone sensors struggle to harness
light—glance at a smartphone photo taken in low light and compare it to one
shot with a DSLR. It’s no contest. But a little clever engineering has made
smartphones better than they ought to be. “Backside illumination” or BSI, moves
some wiring to the back of the sensor, maximizing the surface area upon which
photons can hit the photosites. Another trick is using a 4 megapixel sensor with
a 1/3-inch image sensor. Yes, this decreased overall resolution, but also pixel
density, making them better performers in the dark.
But these are workarounds. Why not just use a bigger
sensor to begin with? Because there are a lot of challenges to packing one into
something so small as a phone, not the least of which is heat. “With a larger
sensor that takes up more real estate, this leaves less room for heat
dissipation,” says Dan Unger, a Panasonic spokesman. “Add to this the current
demands of the larger sensor, and heat can be a real challenge to manage.”
You can get around that, as Panasonic did, by making the
phone noticeably thicker. That improves thermal management. And it’s fine for a
phone designed largely for photography, but it’s not an ideal solution. If
smartphones are going to use bigger senses but not fill your pocket like a
Stephenson novel, there’s a lot of work to do—especially when you consider
bigger sensors cost more. And if shoot video, whoa does that generate some
heat.
“When you add the movie into the overall equation, then
you’re putting a lot more demand on the camera in terms of dealing with heat
generation,” says Chuck Westfall, a technical adviser at Canon. “The larger the
sensor, the greater the heat-generation possibilities become. In a small space
like a compact camera, it works against you a little bit. It’s much more so for
video than it is for still imaging.”
Imaging
Is a Process
The sensor simply senses light and converts it into an
electrical signal. To use an analogy, it buys the groceries. Someone else cooks
dinner. So while a high-quality sensor helps, it’s hardly the most important
component. The lens is important, of course, but the biggest difference between
a great camera and a good camera is the image signal processor—the secret sauce
to any smartphone camera’s features and performance.
Hung says that the image sensor isn’t the only thing feeding information into
the ISPs. A modern smartphone has several sensors at its disposal. “The
gyroscope has evolved in terms of image stabilization,” he says. “A lot of the
ISPs now can take the input from the gyroscope (and) combine that input with
the image sensor to provide image stabilization. It’s a new kind of digital
stabilization system.”
Apple and Samsung use their own image signal processors
for the iPhone and Galaxy phones, respectively. However, many high-end Android
handsets use the integrated image signal processors in Qualcomm’s Snapdragon
system-on-a-chip, which keeps camera features relatively consistent from phone
to phone. As good as it is, the company says the next-gen processor arriving
early in 2016 will improve noise reduction, artifact correction, autofocus, and
color reproduction.
Optics
Will Stay Simple
The molded plastic lens elements in many cameras have
reached the point where they’re essentially perfect. They’re also cheap. Oh
sure, critics argue they don’t have an optical zoom. There’s a reason for that.
Optical zooms have moving parts, which runs counter to the phone industry’s
slimmer uber alles mentality. You want an optical zoom? You’ll have to accept
fatter phones.
“I think most users care more about a good-looking phone
and image quality than perhaps that extra bit of functionality,” Hung says.
“The people that care will get lens accessories to do those things. I don’t see
those more advanced things being built into many phones.”
However, some patented technologies could hasten the
arrival of optical zooms. Pretty much every point-and-shoot has an optical zoom
as great as 5x with lenses housed entirely within the the camera. Hell, Canon’s
patented a 45X zoom folded-optics lens, but has no plans to use it
anytime soon.
“The overwhelming objective on a smartphone is to keep
the physical size of the device to a minimum in terms of thickness,” Westfall
says. “There is going to be a limitation no matter what in terms of the quality
of the lens that they’re able to put in there. Not just in terms of resolution,
but in terms of focal length range and aperture as well.”
The
Future of Smartphone Cameras
Earlier this year, Apple bought the image-sensor
companyLinX, which uses an array of lenses to enable
Lytro-like refocusing, create 3-D depth maps, and improve image quality in low
light. The Light
L16 camera, which uses 16 lenses and sensors to recreate the surface
area and low-light capabilities of a DSLR sensor, is also due next year. The
company’s founders hinted that the
L16’s multi-sensor technology could show up in smartphones before
long.
But as good as they are, smartphone cameras probably
won’t ever match the quality of a DSLR. And they probably won’t have to. For
all but the most serious photographer, the ease of a smartphone camera, and the
plethora of apps that can make a crappy photo look good, is plenty. And the two
approaches are complimentary. Although the smartphones have decimated the
point-and-shoot segment, sales of DSLR and other high-end rigs remain strong.
As long as that’s the case—as long as DSLR cameras take a
better picture, you can bet the companies making sensors and processors for
smartphone cameras will continue pushing the technology further.
