Till now, smartphone cameras were able to capture images in portrait mode, but the processing prowess of the chipsets wasn’t simply enough to capture and process videos with depth sensing. During one of the sessions with Indian journalists last month, Google’s Marc Levoy, who heads the Computational photography team for Pixel phones, had explained why it’s not possible to do live depth sensing for an elongated duration which is required to have portrait mode effect on videos. But today, with the announcement of Snapdragon 855, Qualcomm has enabled just that. 4K HDR video recording at 60fps with portrait mode! Judd Heape, Sr. Director, Product Management for Computer Vision at Qualcomm explained that the Snapdragon 855 comes with first ever ISP with computer vision. Previously, the Spectra ISP used to capture the image colors, but it was the DSP which used to do the depth sensing. Now, with the Spectra 380 ISP, Qualcomm has integrated “numerous hardware accelerated computer vision (CV) capabilities, enabling the world’s first announced CV-ISP to provide cutting-edge computational photography and video capture features while at the same time offering up to 4x power savings.” In simple words, the ISP is powerful enough to do the depth sensing by itself without relying on the DSP. The CV-ISP includes hardware-based depth sensing which enables video capture, object classification, and object segmentation all in real-time in 4K HDR at 60fps. Earlier this year, Apple had demonstrated similar capabilities with iOS 12 and TrueDepth camera (and possibly at lower resolutions than 4K HDR), but we are yet to see the real-life implementation of the same. In addition to 4K HDR video recording with Portrait Mode, the Spectra 380 ISP will support video recording using HRD10+, an industry-first yet again. Then there is an added support for HEIF image format with hardware acceleration support for capture and playback. A lot to look forward to in the cameras of premium smartphones launching in 2019. Disclosure: Qualcomm sponsored the correspondent’s flights and hotel for the event in Hawaii.
