A Brief Interview With John Hamilton

Kodak obviously wants its inventors and researchers to be doing their primary scientific work and not answering idle questions from blokes like me, but thanks to Bruce Graham at Kodak I was able to put a few questions directly to Dr. John F. Hamilton, Jr., of the Photographic Science and Technology Center at Kodak Research Labs, one of the inventors responsible for the just-announced partial-panchromatic array.

A graduate of Cornell University, John Hamilton received his Ph.D. in mathematics at Indiana University. In 1974, he accepted a position at the Kodak Research Laboratories where he applied mathematics to various problems in graphic arts (printing), medical imaging, clinical diagnostic imaging, and electronic digital imaging. He is a Research Fellow, a recipient of the Eastman Innovation Award (2003), a recipient of the Rochester Intellectual Property Law Association (RIPLA) Distinguished Inventor of the Year Award (2005), and a member of Kodak’s Distinguished Inventors Gallery with 41 patents in the area of digital image processing. Currently, he is developing novel image processing algorithms for Kodak's digital camera business, Kodak's sensor business, and related applications.

The Online Photographer: Dr. Hamilton, to what extent is Kodak's new sensor filtration array really a new idea? Doesn't the Bayer standard already allow for panchromatic pixels or undyed photosites?

John Hamilton: The Bayer patent shows a 3-channel sensor, but the new sensor has four. Other patents show four channels, but not the family of patterns that interest us.

T.O.P.: My readers are mainly interested not in the high-volume applications that probably greatly interest Kodak, but in high-quality applications. Is this something we are likely to see in high-end DSLRs and digital medium-format backs?

John Hamilton: I have no idea what to expect regarding product offerings. We are in research and that is a business decision.

T.O.P.: I speculated that one thing that made this new type of array possible is the greatly increased number of pixels in modern sensors. Is this one of the factors that enables this new array design?

John Hamilton: Yes.

T.O.P.: If the panchromatic pixels permit one stop more light sensitivity, doesn't that also mean that they'll saturate one stop earlier too? Wouldn't that affect highlight resolution?

John Hamilton: Yes, the panchromatic pixels would saturate before the others. The exposure controls should be set to suit the panchromatic pixels.

T.O.P.: One of my readers expressed surprise that the new array maintains the old G-G-B-R ratio in the colored photosites. Why is it important to still have double the number of green pixels?

John Hamilton: Many cameras have video ASICs that require Bayer RGB input. The new filters were designed to accommodate that fact.

T.O.P.: Would the new array lend itself better to B&W-only implementations?

John Hamilton: I would expect a B&W sensor to be entirely panchromatic pixels, such as the one used in the Kodak Professional DCS760M from about five years ago.

T.O.P.: The human eye puts an emphasis on luminance information for the sake of image detail. Is the new sensor likely to increase the level of real detail in digital images?

John Hamilton: Not really. The panchromatic pixels function just like the green pixels of the Bayer pattern except that they are photographically faster. However, under low light conditions, the new patterns will outperform Bayer because of improved signal-to-noise.

T.O.P.: I appreciate that part of what will make this new array practical is that new interpolation algorithms will have to be devised for it, and some of that work is still in the future. But knowing what you know, do you anticipate that the likely problems or advantages will make the new array best suited for certain applications as opposed to others?

John Hamilton: The new filter patterns were designed with low-light conditions in mind, but it's too soon to say where they work best. Under well-controlled lighting conditions, such as in a studio, I would expect the new filters and a Bayer filter to be roughly equivalent.

T.O.P.: One last question—so how come the new array isn't named after its inventors, like the Bayer Array was named after its inventor, Kodak's Dr. Bryce Bayer, in 1976?

John Hamilton: We are just the tip of the iceberg. Many sensor and algorithm people are involved in bringing this technology forward.

T.O.P.: Thank you, Dr. Hamilton.

John Hamilton: Thank you.

____________

As a follow-up, I should add that I did press both Bruce and John about likely applications and possible products, but I was brought around to their view that the story for now is the success of the technical research—it's really too early to tell how the idea will be implemented and in what kinds of products it will be most successful. Kodak can make sensors for its own products, or make sensors as OEM products for other cameramakers, or design products to be fabricated by others, or license its intellectual property (IP) for products designed and built by other companies. How any given research will "filter" through to the market is dependent on many people and tiers of decisions in other departments of Kodak and at many other companies, so the inventors can't really speculate about such things.

One little tidbit that doesn't quite come through in the foregoing is that Kodak is working with a G-G-B-R color arrangement so far only because that's what established technology demands, but the inventors are actively experimenting with other color pixel combinations. Obviously, the biggest hurdle to overcome is that the new array only yields half as much color information directly. A Bayer array requires interpolation, but the new array requires more interpolation. It's at least possible that the most successful implementation of the new array (I still think it needs a name...) won't be RGB at all.

My thanks to Bruce Graham and John Hamilton at Kodak for their generous cooperation.

___________

Mike

Comments

[Randolph Fritz]

"Many cameras have video ASICs that require Bayer RGB input. The new filters were designed to accommodate that fact."

A-hem! I think they've got applications in mind, if they are thinking of interoperation with specific components.

[Magumi]

One stop improvement does not seem so much given the fact that Canon improves by one or two stops with every generation of its high end cameras without changing the array.

Oh Mike, please do not fall prey to the (pun intended/not intended) plague. You writing is excellent as it is and you don't need this writer's crutch to express what you meant.

[Robin Harrison]

I think sensor design is still very immature, and manufacturers have so far focussed on reducing prices (and hence replicating existing technology) in sensors of reasonable quality. What makes it so shocking that this particular idea has taken so long to emerge is that it emulates nature! The human eye works on a luminance/colour basis!

How about combining this idea with other technologies?

Combine it with the Foveon sensor in a chessboard fashion - alternate pixels are panchromatic for sensitivity (ooh - just like the eye's rods) and filtered on three layers for colour (ooh - just like the eye's cones).

Combine it with the Fuji 'S' concept of two 'sites' per 'pixel'. Let the big, sensitive site by panchromatic, and let the smaller sites take care of the Bayer Array. At low sensitivities, the sensor wouldn't behave any differently to existing sensors. At high sensitivities, one would have a far cleaner luminance layer, at the expense of some colour information (gee whiz - just like the human eye...again). Nice ISO 200/400 colour images, and smooth ISO 3200/6400 B&W images from the same sensor. Dreamy.

Dr. Hamilton's comment on the RGB ASICs is interesting. My understanding is that one requires three axes to describe colour (e.g. R, G, B). BUT "Lab color" is described by luminance (L) and two colour components (a and b). Hence only two coloured filters should be required in the implementation above. These could be arranged in a chessboard fashion, leading to far simpler interpolation techniques than the Bayer array allows.

These ideas are simple extensions of what already exists. But something tells me the optimum solution will be something more...organic. A quasi-random (but known to the image processor) distribution of various luminance and colour cells of various sizes would be closer to what nature has developed. And what's good enough for nature....

Oh, by the way - you read all of this stuff first here, right, Mr. Fuji and Mr. Sony. It's all copyright me, 2007, okay?

[Martin Doonan]

In response to Robin's comments on mixing Pano & Foveon sensors - I think: I'd rather have it the other way: big RGBs (less colour noise) and small panos (good sensitivity anyway). Luminance noise is easier to deal with in processing.
Why not combine the 2 ideas and have a 2-colour Foveon-type sensor site?

[David Bennett]

Nice post - I don't know whether I would have had access to this kind of thing were it not for TOP.

[Bruce McL]

It's kind of a relief to read about this development. It gives hope that increasing processing power in cameras and increasing pixel density will be used to make better images. That would be a big change from the present, where increasing pixel density leads to worse images, and more processing power is used to try to hide the newly created problems.

I like the idea of using light and dark pixels to increase dynamic range. But I admit I'm no expert. To me any change in this area, if it encourages still more changes, is good.

[Luc Novovitch]

Interesting story, and being noticed: Rob Ralbraith (http://www.robgalbraith.com) has a nice plug for The Online Photographer today.

[georg]

Thanks for the interview! It is refreshing to hear such things, not intermingled with marketing speak, and 'yeah we can do everything better'-answers.