is a blog about design, technology and culture written by Khoi Vinh, and has been more or less continuously published since December 2000 in New York City. Khoi is currently Principal Designer at Adobe, Design Chair at Wildcard and co-founder of Kidpost. Previously, Khoi was co-founder and CEO of Mixel (acquired by Etsy, Inc.), Design Director of The New York Times Online, and co-founder of the design studio Behavior, LLC. He is the author of “Ordering Disorder: Grid Principles for Web Design,” and was named one of Fast Company’s “fifty most influential designers in America.” Khoi lives in Crown Heights, Brooklyn with his wife and three children. Refer to the advertising and sponsorship page for inquiries.+
For years we’ve groaned every time a character in a movie commands a computer to “Enhance!” a low resolution image, and then watched as an implausibly clear, high resolution replacement appears before our eyes. For me, this has always been one of the worst kinds of lazy storytelling; it always suggests a fundamental lack of understanding of how digital imaging works on the part of the moviemakers.
Well as it turns out, the work of scientists over at the Max Planck Institute for Intelligent Systems in Germany may ultimately give technologically clueless film directors from the 1980s and 1990s the last laugh.
In a project called “EnhanceNet: Single Image Super-Resolution through Automated Texture Synthesis,” they’ve developed a shockingly high quality method of constructing surprisingly believable versions of low-quality originals. Here’s one example that starts with this image of a bird:
The picture is downsampled, reducing the data to this pixelated state:
That image is then processed with their “ENet-PAT” method and results in this:
I’m not sure this approach can resolve a grainy image of a face into something instantly recognizable, which is usually what one sees in films, but this example is stunningly effective nevertheless.
To give some context, the resampling techniques most of us are familiar with from Photoshop and other image editors generate new pixels and details strictly from what’s available in a given low-resolution image. That results in glaringly unconvincing results that are usually either overly smooth or pock-marked with unsightly image artifacts. By contrast ENet-PAT uses machine learning techniques to teach a neural network how to best guess what details should be added to an image. Once trained, the system can produce reliably believable results like these. Scary.
Read more at tuebingen.mpg.de.+