University of Toronto Magazine University of Toronto Magazine
Illustration of Big Ben split between three photo frames

Computer Vision

Want to know where an unidentified picture was taken? A computer program being developed at U of T can help

Imagine a friend has put together a photo album, but hasn’t labelled the pictures. As you flip through the images, you’re left guessing where they were taken. You might make some fairly accurate estimates – but a new U of T computer program would probably do much better.

Computer science professor Aaron Hertzmann, PhD student Evangelos Kalogerakis and others have developed an algorithm that can analyze a series of photos and determine where they were taken. The program – the first of its kind, Hertzmann says – isn’t designed to identify individual images, although it can make a rough guess at one-off photos. Rather, its power lies in its ability to identify a whole series of images, if it knows the sequence in which they were taken.

Hertzmann’s program exploits the enormous image database of the popular photo-sharing website, On Flickr, people have the option of “tagging” photos to indicate when and where they were taken. Hertzmann’s program uses these tags – mini-summaries of the photo-taking habits of thousands of Flickr users – to determine where photos were taken.

“If you take a picture of some city street, it could be anywhere in the world,” Hertzmann says. “But if half an hour later you take a picture of Big Ben or the Eiffel Tower, then it becomes much clearer where that first picture was taken.”

On a recent holiday, Hertzmann photographed the Odeon of Herodes Atticus, an ancient Greek amphitheatre in Athens. A day later he photographed a nearly featureless seascape off Santorini, with just a sliver of coastline showing. Taken individually, the program does just fine with the Odeon photo because many people have posted similar shots to Flickr; it puts a dot on a world map at Athens. But the seascape could be anywhere. Without any additional information, the program puts dots all over the world’s seas. Feed the program both photos – along with the time frame in which they were shot – and suddenly it recognizes the seascape as being from the Aegean and puts a dot near Santorini. (The program knows not only that Santorini is close to Athens, but also that it’s a popular target for Flickr photographers who have recently visited Athens.)

Hertzmann imagines many practical uses for the program. Tourism workers and urban planners could benefit from a detailed knowledge of the routes that people take between – and within – cities. And health-care professionals could use it to help fight epidemics, especially if the photo-taking data is combined with information such as air travel routes and cellphone usage.

Recent Posts

Photo of front campus field and Convocation Hall with flower emoji illustrations floating above

Clearing the Air

U of T wants to drastically cut carbon emissions by 2050. It’s enlisting on-campus ingenuity for help

Abstract illustration showing a red-coloured body and face, with small black and white pieces flowing from inside body out of the mouth, and the U.S. Capitol Building dangling on puppet strings from one hand

The Extremism Machine

Online disinformation poses a danger to society. Researchers at U of T’s Citizen Lab are tracking it – and trying to figure out how to stop it

Prof. Mark V. Campbell with a beige background and red lighting

Charting Hip Hop’s Course

Professor Mark V. Campbell grew up during the early years of rap music. Now, he is helping preserve Canadian hip-hop culture for future generations

Leave a Reply

Your email address will not be published. Required fields are marked *

  1. One Response to “ Computer Vision ”

  2. Anton N. says:

    Another blow to privacy! What's next? Using Google's enormous "Street View" database to determine where individuals live, based on the photos they've taken?