Does your phone have artificial intelligence (AI)? The latest buzzword in smartphones, AI is being touted as the next big thing for photography. It’s being used by marketers to claim that the latest devices can do incredible things for zero effort. In practice, AI is merely the latest way to talk about technology that has been in cameras for years. Remember that ‘auto mode’ on your camera you spent years trying to move away from? Well, it’s back and more tempting, more accessible and more impressive than before.

Essentially software that makes decisions on your behalf, AI increasingly identifies objects and scenes in your compositions and tweaks settings accordingly. From faking ‘bokeh’ blur in portrait mode to all-new and powerful ‘night mode’, AI is changing photography.

Here’s everything you need to know about AI and photography, how to use it to your benefit, and what to be wary of.
‍

1 Understanding AI and photography

AI is concerned with teaching computers what to do in an effort to automate complicated or time-consuming tasks.

"AI is concerned with teaching computers what to do in an effort to automate complicated or time-consuming tasks."

It’s not new; auto-mode, red eye-removal and ‘pet’ mode have been around for decades. In cameras AI comes in various guises, from speech recognition (‘Hey Siri, take a photo’) and image recognition (some phones take a photo if you wave at the self camera) to what’s best known as computational photography.

All that really means is the inclusion of software that enhances the photos you capture using machine learning, computer vision and neural networks. So what AI on cameras is really about is ever more advanced algorithms, which are not new, but are vastly improving automatic settings for better point-and-shoot photos.

‍

Author’s tip:

‍The current revolution in AI is driven by a leap in processing power in the latest smartphones. You’ll often hear the makers boast about how AI helps their cameras improve colour, contrast and light range – in the form of a newly powerful ‘night mode’ – as well as simulated depth of field (for fake ‘bokeh’) and automatic retouching. 

2 How auto-focus works

Auto modes on cameras take care of aperture, shutter speed and ISO by making decisions based on light levels. A key parameter is auto-focus, which has improved in recent years with the arrival on smartphones sporting lenses whose focus can be adjustable.

While smartphones used to use algorithms based on a slow method of measuring contrast and making adjustments, nowadays most now use phase-detection. A tech from the DSLR world, phase-detection auto-focus is when light from two apertures on opposite sides of the lens are compared and a calculation about focus is made instantly.

‍

Author’s tip:

Some of the latest flagship smartphones use Light Detection and Ranging (LiDAR) lasers and scanners to calculate exact distances and depths in the surrounding environment. It’s essentially used to improve autofocusing speed in low light. 

‍

3 AI and photo archives

Will AI change the way we take photos? Sure, but it seems destined to have just as big an effect on photo editing. On the upside, that means a lot less photo editing and tagging will be necessary in future.

Look in your smartphone’s camera roll and you’ll notice that it already presents an automatically created folder of selfies you’ve taken. How does it do that? Image recognition and intelligent object selection, that’s how, which are forms of computer vision. As image recognition improves expect to be able to interrogate your old photo archives for all images featuring, say, mountains, trees or the night sky.
‍

However, it’s not just about curating. A more streamlined approach to post-processing is likely, too, all of which will be incredibly time-saving for photographers. An AI that is always looking at your archives and making its own decisions about how to improve them is always going to be interesting to photographers.
‍

Author’s tip:

AI is fast becoming the differential between smartphones and their cameras. It’s therefore not much of a leap to say that the future of cameras – including DSLR and mirrorless models – is going to be about the cleverest software, not the core hardware. For example, now big brand cameras can record 8K video there are no real improvements that can be made in terms of resolution – we’ve hit a wall.

4 Time-of-flight cameras

There’s a new smartphone camera technology in town that allows the user more control over how the subject of a picture is displayed. Called time-of-flight (ToF), it uses infrared light to measure distances to separate objects to create a depth map so that the background of an image can be identified by the smartphone’s AI software. It’s then possible to create a fake ‘bokeh’ effect by defocusing that background.

You’ll often find this tech within ‘portrait mode’ on some smartphones, but ‘bokeh’ is just the beginning. Since ToF cameras can separate different objects, in conjunction with other AI techniques they’re also capable of recognising an individual in a crowd and separating them by de-focusing their surroundings. We’re already seeing this feature on flagship phones.

‍

Author’s tip:

‍Another recent advance led by AI is ‘night video’. While ‘night mode’ has been all the rage for smartphone cameras in the last few years, video at night remains disappointing. Expect that to change, with the very latest smartphones based on Qualcomm’s Snapdragon 888 chipset (which contains a Neural Processing Unit) boasting ‘night video’ modes that work by applying machine learning algorithms to noisy video frames.

5 A dark side to AI?

AI comes with a dark side called augmented reality (AR). Increasingly powerful algorithms are being used to automatically enhance and augment photos like never before, with the end result being photos that you can’t trust are actually ‘real’. From something fairly innocent like placing the Moon in an otherwise empty landscape to ‘deep fake’ videos that are AI-manipulated and distorted so cleverly that they are indistinguishable from a ‘real’ photo.

‍

However, AI is ironically now able to help with such issues, helping the likes of Microsoft Video Authenticator and Adobe Photoshop’s ‘face aware liquify’ feature to backwards-engineer photos to detect and reveal AI-powered distortions made to a photo or video.
‍

Author’s tip:

‍Searching for shadows in your photographs is an excellent way of enhancing your subject, but equally an awkwardly-placed shadow can ruin a composition. An academic paper published in 2020 by Google Research and the University of California Berkeley reveals an upcoming AI technique that can remove unwanted shadows from photographs even in dodgy lighting conditions.