Google introduced its latest Pixel devices, the Pixel 4 and 4XL, back in October. The phones came with a 90Hz display, dual cameras and new technology that enabled faster face unlock and air gestures. One of the standout features, when it came to the cameras, was the improved Night Sight that enabled impressive astrophotography. For those wondering how the feature works, the search giant has provided more information in a blog post.
The post provides a detailed look at the feature, the efforts that went into creating it, and the challenges faced. The engineers explain the aspects of shooting low light images of the sky, including exposure times, fixing dark current and hot pixels, helping with composing the scene through the display in dark environments, getting autofocus and post-processing images of the sky right, and more.
One of the many challenges involved in shooting a better low-light landscape is that of exposure. The amount of light that enters a camera sensor determines how bright an image gets. To do so, cameras need to capture an image for a longer timeframe, resulting in motion blur or star trails in terms of the night sky. Think of clicking car taillights at night and inadvertently moving the camera that results in a trail of light. Similarly, the movement of the stars can leave a trail on an image. The team overcame this by shooting 15 frames at shorter 16-second exposures.
The post also details other interesting aspects such as the efforts taken to mitigate hot pixels, small bright dots that may appear on an image taken in low light. The team also used machine learning and trained convolutional neural networks to identify the skies in an image and process them accurately to avoid over brightening the night sky. This was important because overexposed skies can tend to look like images shot in the day. This was also done to improve the contrast in the images and to perform “sky-specific noise reduction” to reduce noise (graininess in images).
Those interested in understanding the other aspects in detail can head to the blog post. The team has also posted nifty tips and tricks for shooting better night-time photos with the Pixel 3, 3a, and 4.