Super-resolution, or upscaling of photographs, has typically relied on linear filters that draw data from nearby pixels, and RAISR is out to change that.
Image upscaling has been around for a while, but a lot has been said about the quality of super-resolution photographs. Google’s latest AI algorithm addresses just that. RAISR, or Rapid and Accurate Image Super-Resolution, uses machine learning to make the image upscaling process more versatile and variable. This, Google claims, will make upscaled high resolution images include more details, better colour depth and superior end result on overall terms, in comparison with traditional methods.
The typical methods
As explained by Peyman Milanfar, Research Scientist at Google, RAISR’s approach towards upsampling (image upscaling) is what makes it different from typical upsampling methods. The most commonly used super-resolution method was by using static, or linear filters to add more pixels to the concerned image. This practically ‘copies’ the neighbouring pixel to recreate substance and significantly increase the resolution of the image. Over time, this is what has typically been used to derive high resolution outputs of low res photographs and images.
The problems
With such upscaling techniques, although it was possible to amplify the resolution of the image, the high res versions lacked details, sharpness and precision, although colour data was retained reasonably well. It is this that made the usual techniques impractical for usage in industrial applications, and branches such as medicine and precision engineering mostly relied on high resolution cameras for suitable image output. Even though the aforementioned upsampling method is fast, obtaining sharper results were considerably slower.
RAISR’s machine learning upsampling
With RAISR, Milanfar states, there are two methods that can be used to get high resolution image outputs. The fundamental process includes applying selective filters to the pixels of a low res and a high res copy of the image, to give rise to a super-resolution photograph with comparable level of details retained.
Typical upscaling: Lack of vivid details in upscaled photograph (right)
The level of details are maintained by the machine learning algorithm, which comparatively analyses the low res and high res copies of the image, following which it detects the scenes and patterns in the frame and adds more details to the new pixels in the frame. The most effective output derived from RAISR initially uses a cheap upsampler to upscale a low resolution photograph, and then the machine learning algorithm compares the two image copies (the original low resolution and the derived high resolution) to identify the linear filters and new pixel areas.
RAISR upsampling: Equal details and sharpness is retained in the upscaled photograph (right)
It needs to be added that while this method is somewhat slower, this also makes use of hardware-based upsampling of images, or deriving super-resolution outputs from the camera hardware itself. RAISR’s filters used in the upsampling method improve on brightness, colour data, textures, stroke directions and image sharpness. RAISR can also be used to tackle interference, or Moire, in photographs with altered quality.
Going forward
As Milanfar puts it, “Today, the use of machine learning, in tandem with decades of advances in imaging technology, has enabled progress in image processing that yields many potential benefits. For example, in addition to improving digital “pinch to zoom” on your phone, one could capture, save, or transmit images at lower resolution and super-resolve on demand without any visible degradation in quality, all while utilizing less of mobile data and storage plans.”
What we wait for right now is to see the technology in everyday usage, which will automatically allow smartphone cameras to grow in relevance and semi-professional use cases. RAISR holds the potential to significantly alter the realm of high resolution imagery, and all that remains to be seen is the acceptance of this technology by companies around the world.
Follow Us
Souvik Das
The one that switches between BMWs and Harbour Line Second Class. View Full Profile
