Written by Ctein
Please bear with me, Faithful Readers...this is a photography column. It'll just take me a bit to get us there.
I am irritated by the success that corporate/marketing droids have had in co-opting the term "A.I." That stands for "artificial intelligence," and has, in the past, commonly meant some kind of general intelligence that deals with the real world in all its perplexity. The ultimate goal may be (trans)human level, but it's gradable down—you could have parrot-level AI, or cat-level or even goldfish-level (currently, we've reached the some-but-not-all-insects level of AI).
Now it's been degraded to refer to any system that uses one of the deep learning networks (there are many different flavors of those) to learn how to do a sophisticated task. They're popping up all over, and they do amazing things. But Artificial Intelligence? Nah, I don't think so.
(What are those promoters going to do when we get genuine AI? Call it AIFRTTH—Artificial Intelligence, For Real This Time, Honestly? Maybe that's the first question we should ask real AI, if we ever get it.)
I think there's a better descriptor for what these programs do: Artificial Common Sense. I'm not being flippant. I think it's right and significant, and it lets us take these programs seriously. Believe me, we should be taking them seriously.
Common sense is something that we've thought of as being a fairly sophisticated function, a way of distilling down the otherwise unfathomable complexity and intricacy of the world into something we can manage almost intuitively. "Sophisticated" isn't the same as intelligent; dogs and cats exhibit a great deal of common sense, but down at their level of mentation.
At the human level, common sense isn't necessarily either smart or correct. Everyone here has likely had the experience of getting into an argument with somebody who, backed into a losing corner, utters those fateful words—"but, it's just common sense that...." What then comes out of their mouths is invariably wrong. Common sense is not, in fact, good at dealing with the esoteric. But it's pretty damned good at dealing with everyday life and figuring out what's important and what's ignorable.
At the forefront
Which brings me to photography software. Consider the sorts of manipulation and filter programs we use routinely, like noise reduction, enlargement, and sharpening routines. As humans, we look at a photograph and we can easily figure out what is noise and what is detail, because we just know what the world looks like. Sure, there are the edge cases, where the subject detail is getting so similar to the grain/noise in the photograph that we're not sure if the feature we're looking at is real or an artifact of the media. But by and large, we know the difference. When we sharpen or enlarge a photograph, we want the real detail improved and enhanced, not the grain and the artifacts. When we run noise reduction, we don't want it to degrade the real image detail and tonality. We know and can see the difference because, well, we just know. 'Cause we're smart, sophisticated real-world data processing machines.
Until recently, we could imagine that this kind of discrimination was also a sophisticated function, something that would require a pretty high level of real AI to achieve. Sure, we've got algorithms that do a fair job of brute-force faking it, but it's pretty obvious that they're faking it with frequency filters and threshold discriminators and the like. They don't really understand a photograph, not the way we do.
Now we've got software, based on some deeply learnt neural networks, that appears to be doing the real thing, making the same kind of common sense decisions about a photograph that we make when we look at it. These programs don't work by some programmer handing them a finished set of rules and equations. Instead, they throw problems and solutions at them and say, "Okay, now figure out how to get from A to B." The more cases the networks have to chew on, the cleverer the solutions they invent. After chewing on a few million comparison photographs, they've got some pretty clever ideas.
At the forefront is Topaz Labs. For several years now they've been turning out, in my opinion, the best Photoshop filters that money can buy. Yeah, that's sometimes a matter of taste. For example, I've got at least a half-dozen different noise-reduction programs, because no two of them work the same way and each works better on some kinds of images and noise than the others.
In the past year, Topaz Labs have leapt ahead of everyone else by harnessing these buzzword-AI-tools to develop standalone programs and plug-ins that are beyond good. They are unbelievably good. That's not hyperbole—I look at some of the results and I simply don't know how it's possible for a "dumb" program to achieve them.
This column I'm going to talk about Topaz Sharpen AI, a standalone program currently in version 1.4.0 and on sale for $59.99. (These programs are frequently updated, not to fix bugs or add features, but to incorporate the very latest cleverness that the neural net has figured out as it chews on ever more real-world cases.)
Sharpen AI is already scary clever. The photograph below (figure 1) is the very first photograph I threw at Sharpen AI. I did not cherry-pick it out of hundreds of trials. It sold me on the software.
Figure 1
(Note: Even if you click through to the enlarged versions of the figures, you're not seeing them at 100% and there may be compression artifacts—the limitations of the host publishing software. I've put a full-resolution version up on my website here with no compression artifacts, for any who want to pixel-peep.)
This is a photograph I'd normally throw away—I was on a moving boat and it was late in the day. It's the best of a pretty poor sequence, if I'm being honest. I figured it would be a good test case. The first program I tried was Photoshop's Shake Reduction filter. This algorithm is supposed to analyze the photograph, calculate what's causing the blur, and undo it. It's called deconvolution. The tricky part is coming up with exactly the right point blur function and applying it without generating artifacts. As you can see in figure 2, Shake Reduction still isn't very good at that. In fact, I've yet to find any but the simplest and most minimal cases of subject blur or camera shake where it produced an artistically-usable result.
Figure 2
Then I handed figure 1 off to Sharpen AI, selected the "Stabilize" processing mode and let it rip at its auto-default settings. Two minutes later, I got back figure 3.
Whoooaaa...
Yup, two minutes. Sharpen AI is not only clever but a hard worker. On a Retina iMac with a 4GHz i7 processor, it takes that long using seven threads to process a 20-MP image. That is a heckuvva lot of gigaflops, on the order of 50,000 calculations per pixel. Not your everyday sharpening routine, nosiree. In fact, just updating the preview within the application can take as much as 30 seconds. Not surprisingly I have "auto update" turned off; otherwise every time I moved one of the adjustment sliders the program would want to take another 30 seconds to update the preview. (Almost always, Sharpen's automatic settings produce the best results, so I'm not sure why I bother. I guess I'm just a fiddler.)
Once you've gotten past being awed at the overall improvement, take a close look at the thin twigs and branches along the lower sides and the bottom of the picture. Sharpen AI has brought them back into an amazing focus without any ringing artifacts and while leaving the background largely untouched.
If you look closely at the edges of the birds and the nest in the sky areas, you can see some small artifacts—thin vertical lines that extend a few pixels away from the subjects. These are much more evident on the screen than they would be in a print. In fact, I printed out the full image on 17x22" paper and they were completely invisible. The birds and the nest looked perfectly sharp with no evidence that any enhancements had been run on the original photograph.
Other modes
Sharpen AI has three processing modes—Sharpen, Stabilize, and Focus. You've seen what Stabilize can do. Sharpen does more ordinary sharpening and is recommended for photographs that are otherwise in-focus and un-blurred. The effect is modest, but it's clean. So far, I haven't found an out-of-focus photograph where Focus improves things. Unlike Sharpening, which is understated, Focus seems to go way overboard. Perhaps I haven't found the right image to apply it to, but so far it's been useless.
On every photograph I've tried, Stabilize does the best job, better than Sharpen even on photographs that are in focus and don't have any visible camera shake or subject movement. I haven't reached the point of applying it to every photograph, even the ones that don't look like they need any sharpening, but I'm starting to wonder if maybe I should try that.
Sharpen AI will process all kinds of original files—TIFF, JPEG, PSD, and even RAW. I don't recommend using it on RAW, though, if you like the rendition of your usual RAW converter. Me, I like what Adobe Camera RAW does and Sharpen's raw converter doesn't produce the same results, in terms of tone, color or geometric correction. I'm not saying it's bad, but it's different, and even though Sharpen will output a new raw file, the new one won't render the same as the original. My recommendation is to make your usual corrections in your preferred RAW converter, generate a 16-bit TIFF from that and put Sharpen to work on the TIFF.
In past years, I've deleted photographs that were artistically appealing but unacceptably blurry, knowing that I'd never be able to pull prints from them that met my technical and aesthetic standards. I'm thinking I may have made a mistake.
Topaz Sharpen AI—recommended without reservation.
Next column (in a couple of weeks) I'll talk about Topaz Labs' Gigapixel, which is absolutely, positively the best upsampling program ever. It does all those image-improvement miracles that other upsampling programs claim to do... and don't.
Ctein
Ctein, pronounced "kuh-TINE," rhymes with fine, is one of the most experienced and accomplished photo-writers alive. He was TOP's Technical Editor before leaving for a new career as a science fiction novelist. He has written two books of photo-tech, Digital Restoration from Start to Finish and Post-Exposure. This is his 343rd column for TOP; older columns can be found under the "Ctein" Category in the right-hand sidebar.
Original contents copyright 2019 by Ctein. All Rights Reserved. Links in this post may be to our affiliates; sales through affiliate links may benefit this site.
Please help support The Online Photographer through Patreon
(To see all the comments, click on the "Comments" link below.)
Featured Comments from:
Aashish Sharma: "I haven't even started reading the article, but I'm writing to let you know how delighted I am to see Ctein back. I have always enjoyed his writing on everything and missed it while he was away. An article by him is even more welcome as it is my birthday! Thanks for writing, Ctein. I'll now go back and read it."
Michael Elenko: "Thank you for this post. I've had this pano HDR I created eight years ago. It was in the mountains, and the mosquitoes were the worst, even getting in my eyeballs. So I could not provide enough stability for the brackets to line up right, plus dusk was coming on, so my shutter speed was too slow for effectiveness. But the shot was special, just too mushy for my standards and a plethora of tried tools. Until I used Sharpen AI at the settings recommend by Ctein. Wowie zowie, this finally looks right. Thank you, I've waited for this."
Joe Holmes: "I tried a demo copy of Sharpen AI on a couple of my images and immediately paid for the software. I find that it can work miracles on certain images, though there are times when it gets everything wrong, when it simply cannot handle an image and leaves crazy artifacts. And then there are times when it does a brilliant job on certain portions of an image but creates bizarre and ugly artifacts on other parts, like the surface of a river or clouds. In those cases I can make the best use of it by bringing a Sharpen AI image into Photoshop as a layer and applying a layer mask to apply the best parts on top of the original image. Even with that extra work, it's worth it, because the results are way better than what I can do with other tools."
Marcelo Guarini: "First, thanks for the initial part of this post, about artificial intelligence. I'm a professor of Electrical Engineering at a university and I am tired of repeating the same to my colleagues, especially those who work in the so-called field of artificial intelligence. I will show them your comments so they can see that I am not the only one with the same opinion. Thanks for the term 'artificial common sense.' Artificial intelligence is too high-flown a term for what deep learning is.
"I have been using Topaz Labs Gigapixel from its inception, and have been very surprised by its excellent results. As you say, 'positively the best upsampling program ever.' Almost every time I open the program to use it, there is an upgrade available. I have been a beholder of its evolution through cumulative training and refinements. Today its results are much better than at the beginning; you can easily see the improvements. It is just a very cool tool. I will try Sharpen AI and will follow your indications regarding the different modes and auto-update.
"Thanks much for your post, and happy to see your reviews again."
Soeren Engelbrecht: "'Since it is often hard to distinguish common sense from equally common nonsense, professional advice is useful.' —Leslie Lamport."
Tom R. Halfhill: "I'm equally uncomfortable with the buzzphrase 'artificial intelligence'—and I write about microprocessors for a living. (I'm a senior technology analyst for Microprocessor Report, the leading newsletter in this field since 1987.) I prefer 'machine learning,' which I think is more accurate and less grandiose, but the 'AI' label is unavoidable because it has become so widespread. Essentially, it's sophisticated pattern matching and sorting. But at a high level, sometimes we don't understand how these neural networks achieve their results. We turn them loose for training on a huge data set, let them chew on it, then evaluate their performance. It's almost a black box, although programmers can delve deeply into the structure to see what's happening. Of course, we don't fully understand how the human brain works, either, so even 'natural intelligence' is mysterious.
"The Topaz Sharpen AI tool is indeed impressive. What I need is a dust-and-scratch removal tool that's equally good. I've scanned tens of thousands of old family photos that need more fixing than I can handle. The automated tools I've tried often can't tell a dust spot from a pearl earring or a scratch from a crease in clothing."
Ken Lunders: "Whoa!! I downloaded Topaz Labs Sharpen AI for a 30-day trial. Well, the trial has ended after less than hour. I'm now $59.99 poorer, but 100% happy with the purchase. I'm delighted to see Ctein providing content here once again."
eliburakian: "Damn, I just spent $300 on the whole Topaz AI suite after reading this and trying out the software. Thanks a lot, now I have some explaining to do to the spouse."
