Camera noise is the bane of high ISO photography and photographers have collectively spent an innumerable amount of time developing techniques for minimising the noise they have in their low light shots. While DSLRs still manage to do alright, smartphone photography, in particular, suffers from this problem, with low light performance being pitiable at best. Nvidia, it seems has come out with a solution to the noise problem, one that could have an impact on photography as significant as the invention of the Back-Side Illuminated sensor.
Nvidia has unveiled an AI called Noise2Noise that can automatically analyse images and remove all noise, grain and even watermark from them. What’s even more impressive is that the AI can clean up the images without being fed any “clean” images, and in fact, only relies on corrupt images to perform its duties. The new AI was developed jointly by researchers at Nvidia, MIT and Aalto University. In their research paper, the team says that “it is possible to recover signals under complex corruptions without observing clean signals, at performance levels equal or close to using clean target data,” which they show through a series of images published along with the paper.
The application of the algorithm, for now, is projected for use in further neural network training, where a clean image is not possible to acquire. Examples laid out are of course low light photography and even MRI scans, where a short burst generates a noisy image. The AI can clean up the noise and grain, restoring the photo or an MRI scan to a state where it looks like it was a clean image, to begin with. Other real-world applications laid out in the paper also include astrophotography, where low-light noisy images are all you get straight out of the camera and require a significant amount of work to clean.
The paper also focuses on the speed at which the AI can create clean images, showing that it is nearly at par in terms of performance with AI algorithms that have been trained using the “clean image-corrupt image” data sets. The team used Nvidia’s Tesla P100 and Titan V GPUs for their study.
It is too soon to expect a workable, commercial tool to be available to the masses for use in daily lives. Also given that the researchers had to use multiple high-end GPUs simultaneously to run the AI, it is unlikely that the technology is even consumer friendly at the moment. However, this research paper shows a proof of concept that is very promising and could change the face of low-light photography forever.