Smartphone photography has changed the way people look at imaging and optics. Today, you don't need to carry a bulky camera with different types of lenses every time you travel. If you have a good smartphone camera, it will take care of things very well. Now, to improve smartphone cameras even further, OEMs are doing a lot behind closed doors. This doesn't just include partnering with legacy camera brands and slapping bigger camera sensors on smartphones. It requires complex R&D, a lot of technical challenges, and quite a bit of trial and error.
To understand what goes around in camera development, we had a small chat with Vivo's product manager for camera development, Keshav Chugh on the sidelines of Vivo X90 series pre-launch briefing about Vivo’s camera development. Having a background in photography, Chugh takes care of Vivo's camera development, which includes coming up with new features, what hardware to use, how to tune the software according to different markets, and much more. Here's how the conversation went:
1. To begin with, give us a fair understanding of what goes behind the camera development on Vivo smartphones?
So every time we start with a new device, there is a three-step system that is usually followed in order to tune the cameras. The first step is coming up with a problem statement. A problem statement is what we want to achieve with a camera, what our focus will be, for any given product.
Then come the optics part, where we select what hardware to use for what device. The optics stage is also where Zeiss steps in. They are the experts in terms of optics and imaging, and they help us tune the hardware according to different markets.
The third step is computation, where machine learning and AI play a big part. This is where after having everything in place, we look at the software and how we can use our algorithms to enhance the camera experience. This not only includes the camera app or controls, but also how AI enhances the images that are captured by the camera. How much light to let in, how to tune the colours, edge detection in portrait, all the post processing is decided and tuned in this stage.
2. Can you tell us more about the Vivo V2 chip and what the company's goal with the ISP chip is?
Yes, so apart from handling all the camera processing that goes on in Vivo (and iQOO) flagships, the V2 chip also acts as a Deep Learning Accelerator (DLA) which facilitates and handles all the AI algorithms. The V2 is a scheduling chip that arranges, controls, and optimises workloads in a production process. It basically acts like a manager to help solve problems and implement new camera features.
Coming to the development, we have to first figure out what problem we are trying to solve. Then, we go about how to develop the ISP chip. Then, our teams who are developing it come into action, the third parties that help us also come in at this stage. Ultimately, our goal is not to market V2 or anything that Vivo does, the end goal is that the problem statement should be solved. For example, if I'm telling you that the Vivo X90 Pro has the best night camera in the market, I should have the results to back it up.
Further, with the amount of AI involved in making features like Zeiss Cine Flare or our portrait modes along with other things, we need hardware support to run so many algorithms.
3. Tell us more about Vivo's partnership with Zeiss
With Zeiss, we have two types of collaborations – one is co-engineering, one is collaboration. The co-engineering part is where they work with us product-by-product. For example, how they helped us with the camera development in the Vivo X90 series and other previous X-series flagships. That is product engineering, where they are with us every step of the way in the camera development.
Next is collaboration, which is a more long-term thing. Here, we are currently doing R&D on what we'll bring in the future. This involves a lot of experimentation and testing. There is a specific lab at Zeiss in Germany, only for development of Vivo products. So here also, we work on a problem statement basis. For example, how do we bring Zeiss Cine Flare to smartphones is a problem statement. Now, for the user it is just pointing and shooting to bring cine flare in their images. But to implement it is very difficult, since we have to replicate that natural cine flare using AI.
This includes detecting what the scene is, where the light is coming from, what is the subject, what is the light source, what is the white balance in the scene, how intense is the light, and things like that. On the basis of that, the camera has to make calculations around the scene, simulation of scenes, then there are different kinds of cine flares. There also, we have to make calculations as to what size of flare do I need, on what location do I need it based on the light, colour of the flare based on the lighting scenario and white balance. So for all this, there are a lot of AI applications involved. It's not just slapping on a filter with a png sticker, just for the sake of it.
This is another reason why our partnership with Zeiss has been successful. Because if we were to just slap on filters, a big legacy brand like Zeiss wouldn't want to associate with us. It is because they are happy with the final product, we can confidently use their name.
4. Who is making the process for the V2? Does Vivo have any plans to license these ISP chips?
So the silicon manufacturing is done by silicon manufacturers, but the design and architecture is proprietary to us.
Coming to licensing, there are no plans to license this. That is not the idea behind the Vivo V-series chips. This is something that we have invested a lot of money and resources in, and it is our IP.
5. How has Zeiss responded to the positive response that the new Vivo smartphones are getting for cameras?
So Zeiss is with us at every step, even when we are tuning the sensor. This is why we are calling it a Zeiss Main Camera, even though it uses Sony's sensor, hardware wise. But in the optimisation and tuning of the sensor, Zeiss plays a huge part. Even in terms of colours, Zeiss is with us. We are one of the few OEMs who are doing multiple colour sciences. We have Vivo Vivid Colour, and Zeiss Natural Colour. With the Vivo X90 series, Zeiss is also helping us professionally calibrate the display also. Because if our screen is not calibrated, then taking images in Zeiss Natural Colour won't make sense.
So the partnership has evolved beyond the cameras and towards phone development holistically. Also, the Zeiss Natural Colour display setting isn't just a software that Zeiss is pushing, we are individually calibrating the displays.
6. What was the biggest challenge while developing the cameras for the Vivo X90 series?
The toughest thing I'd say would be the tuning for the 1-inch type sensor. Now, we can do very simple tuning or we can do very complex tuning. Very simple tuning is done by some brands and very complex tuning is when you localise the phones. Every country has a different demand. With India being the most important market for us, tuning our cameras according to the Indian customer, be it in terms of skin tones, portraits, or in terms of front or back cameras, or in terms of understanding their needs, how they are different from other users, is very important for us.
Now, to get all of that right and convert it into an actual product that not only works, but is the best at what it does, is the toughest part. It also takes a lot of time. For example, I have to run hundreds and thousands of simulations to find the perfect parameters. Then, we put it to a test version, where we do a real test. Then we repeat this process until we are happy with the final output. That is when we think a product is ready to go to the markets.
7. What is your favourite camera feature in the Vivo X90 series?
In terms of camera, I'd say I like the portrait modes a lot. Of course, Cine Flare is a new one, but overall all the Zeiss portraits are my favourite. So I was a commercial photographer before this and I love street photography. In street photography, 50mm is the perfect focal length. Pair this with f/1.6 aperture, and it becomes an incredible thing for street photography. This kind of (f/1.6) aperture is very big. It lets in a lot of light and this time we have a 30% bigger sensor, so portraits have improved drastically over the previous generation.
8. One camera feature you want on the next Vivo flagship?
Now there is no single feature that I can think of. As product managers, we want to put as many features in our products as we can. However, to think of just one thing, I'd love to have a 35mm lens on my smartphone, but that is not feasible. While I am a product manager, I'm also a photographer so my demands are like that.
This is not feasible because even though we are catering to professionals also, there is still a balance to be maintained. That balance is that you have to make a great camera, while keeping it a great smartphone. It can't be a good smartphone with a bad camera or a good camera setup on a bad smartphone.
9. So you must have seen Xiaomi's concept phone with a replaceable lens. Is there something similar coming from Vivo or is Vivo working on replaceable lens on smartphones?
No. See, all brands have concepts and they never make it to the market. The idea behind these is that OEMs are constantly trying to figure out what the future of the smartphone industry will be.
Now, this is a very good thing for the industry since it opens up new ideas. And in the future, if we want to figure out how smartphones can fully replace conventional cameras, then these kind of concepts are important. Even we have a lot of concept phones – some we bring forth, some we don't. Nevertheless, these kinds of concepts are a good thing, not just for the brands, but for the whole industry altogether.
10. Do you think replaceable lenses on smartphones is a good idea?
As a photographer, yes. But as a smartphone user, no. Because right now the technology doesn't exist. While the idea is good, the tech isn't still up to the mark. My personal opinion is that if you have a 5-inch thick lens on your smartphone, you might as well carry a camera only and have a bigger sensor. Secondly, if my phone always needs a lens attached to the camera, that means without the lens my camera is useless. As I said, it has to be a good smartphone. So, if I get a smartphone camera which allows me to put different lenses, I'd have to carry multiple lenses at least at some point. Now, if I'm carrying so much bulk, why not get a proper camera?
Follow Us
