# Scientists Have Invented a Software That Can ‘See’ Several Minutes Into The Future

Most of us are used to technology predicting the words we want to type. Soon this kind of software will be able to look even further into the future to predict actions we’ve barely considered ourselves.

This isn’t some creepy Minority Report style of precognition. Not yet at least. But it could see us get robot butlers that hand us a decent coffee before we even ask for it.

A process developed by computer scientists from the University of Bonn in Germany has successfully pushed the boundaries on how computers can accurately anticipate human behaviours minutes ahead.

Current generations of anticipatory software are only interested in knowing what we’ll do in the next few seconds.

“We want to predict the timing and duration of activities minutes or even hours before they happen,” says research team leader Jürgen Gall.

The goal was to see if a program could list a sequence of actions up to five minutes into the future based on watching the first few steps of an activity.

Specifically, they trained software to guess what a chef would do next by showing it a number of videos of people making breakfast or a salad.

You can see some examples below.

They then showed the program a completely new video of another person preparing a similar meal, and watched how it guessed upcoming steps and their respective duration.

For you or I, this isn’t exactly a challenge. If you see somebody grab a bowl and some cereal, you can be pretty sure they’ll go for the milk next, which will take 5 seconds, and maybe then cross the kitchen for a spoon.

But this type of reasoning isn’t a walk in the park for computers.

The team tested two approaches using different types of artificial neural network: one that anticipated future actions and reflected before anticipating again, and another that built a matrix in one hit before crunching the probabilities.

As you’d expect, the deeper they looked into the future, the more mistakes they made.

“Accuracy was over 40 percent for short forecasting periods, but then declined the more the algorithm needed to look into the future,” says Gall.

The reflective approach did a little better than the matrix method when looking at the next 20 seconds, but the two different neural networks were equally matched when looking beyond 40 seconds.

At the extreme end, the scientists discovered their trained program could correctly predict an action and its duration 3 minutes in the future roughly 15 percent of the time.

That might not sound impressive, but it does establish solid ground for future artificial intelligence that could potentially develop super-human foresight.

The team will present their results at this year’s IEEE Conference on Computer Vision and Pattern Recognition in Salt Lake City, which we hope will generate some interest in predictive software.

There’s still plenty of ground to cover, especially since the end goal would be to have programs build up enough experience on their own without a need to be trained.

Forget predicting crimes before they happen: we have a long way to go before we need to worry about that kind of future. Our best algorithms struggle to predict which criminals will reoffend, let along guess who will break the law in the first place.

What it could do is help us improve the intelligence of self-driving vehicles, or help us out around the home by switching on a stove the moment we pull out a lasagne.

Maybe it will finally give us a version of Clippy the office assistant who would predict at a glance that you really don’t want any help writing that letter.

0 points
Upvote Downvote