Google is trying to overcome one of the biggest problems plaguing smartphone photography using machine learning; low light photography
Users of the Google Pixel 3 and Pixel 3XL smartphones are about to get an update to the camera app which will bring Night Sight, an exciting low-light photography feature to the smartphones. Google gave us access to the update one whole night early and we managed to take a few sample shots to see what Night Sight can and cannot do. Here’s what you can expect from the feature
What is Night Sight
When the third generation of Google’s Pixel smartphones were announced, Google once again showed off that it has achieved through software. The focus was to emphasize Google’s belief that software could compensate for lack of a certain type of hardware. One particular area where smartphones suffer is low light photography due to their minuscule sensors. Since increasing sensor size is not feasible, Google decided to put machine learning to use and created Night Sight, a mode which according to the on-stage demo, was capable of creating perfectly lit photos out of landscapes completely dark. This is done using a combination of burst mode photography and oodles of machine learning, but honestly, in this case, the results matter more than the method used to achieve it.
Below are some photos shot using the regular mode on the Google Pixel 3XL followed by the same photo being shot using the Night Sight Mode. The photos below have been resized for web, but you can see a larger collection of all the original photos by going to our Flickr Gallery here.
Test Scene 1: Normal Mode
Test Scene 1: Night Sight Mode
Test Scene 2: Normal Mode
Test Scene 2: Night Sight Mode
Test Scene 3: Normal Mode
Test Scene 3: Night Sight Mode
Test Scene 4: Normal Mode
Test Scene 4: Night Sight Mode
Observation #1
While shooting photos in Night Sight, it was interesting to discover that the shot duration was anywhere between 3-5 seconds. This isn’t because the Pixel is shooting a 3 second or 5 second exposure, but more because it is shooting a large number of frames at varying shutter speeds and then combining the best parts of each into the resulting photo. While hand-holding the phone for long wasn’t a problem in most cases, there were two photos that showed blur due to hand shake. For the most part though, this should not be an issue for most users.
Observation #2
The scene needs to be very poorly lit in order for Night Sight to really do its thing. If the algorithm determines that the scene has enough light, it will generate a photo which will look very close to the photo shot in the regular mode. Thankfully, every time you will find yourself in poorly lit situations, the camera will suggest switching to Night Sight. If the suggestion doesn’t pop up, you probably don’t need to use it.
Observation #3
Photos shot in Night Sight mode are visibly soft and lack critical sharpness in most cases. There were two photos in our test sample where the subject of the photo was adequately sharp, but in the case of the two people we shot, the photo lacks sharpness. The softness isn’t due to noise reduction (since there’s very little of it being applied in the first place), but probably due to the way the photos are merged. Thankfully, this is fixable through a software update, just like Apple’s alleged “beautygate.”
Observation #4
If your subject (or you) are in motion, the resulting photo will not have the desired visual quality. Since there is plenty of merging and aligning of multiple frames involved, if there is noticeable movement between frames, the mode won’t work properly and will exhibit ghosting.
Observation #5
Interestingly, the colour reproduction, white balance and contrast (local contrast included) is spot on in every photo. The shots look like they were created using long exposure and not using software manipulation and that is very commendable.
In Conclusion
Google has time and time again shown that it had the ability to defy odds using its machine learning algorithms. We first saw it with the HDR algorithms on the original Pixel, followed by the Portrait Mode on the Pixel 2 and now, Night Sight on the Pixel 3. The mode is an excellent proof of concept and works very well 8 out of 10 times, but there is definitely room for improvement and hopefully, Google will continue to improve the feature in noticeable ways.
Follow Us
Swapnil Mathur
Swapnil was Digit's resident camera nerd, (un)official product photographer and the Reviews Editor. Swapnil has moved-on to newer challenges. For any communication related to his stories, please mail us using the email id given here. View Full Profile
