What’s the best way to overcome limitations of current AI systems, such as their lack of generalization, inefficiency in learning from minimal data, and their reliance on linguistic reasoning? This question takes center stage in this DLD26 conversation between Ulrike Hoffmann-Burchardi (UBS), Llion Jones (Sakana AI) and Raphaël Millière (University of Oxford).
Since the 1950s, AI has oscillated between rule-based machine learning and data-driven statistical approaches, Millière notes. “Fast forward to today: we’ve learned many decades later how to finally use machines to learn from data on their own.” Powering the current generative AI boom is the transformer architecture, “the culmination of this statistical approach”, he says.
But even today’s powerful AI systems suffer from a “failure of broad generalization”, Millière points out: they are unable to truly understand the world they operate in.
So what comes next? Llion Jones advocates for taking inspiration from nature and open-ended exploration. “It’s clear that the brain is able to do things that current AI cannot”, he says. “Perhaps if we look to the brain for inspiration, for things that it’s doing that the current models aren’t, that maybe we can make progress like that.”
Watch the video to explore this session in detail.
