Compared to LLMs, human children require far, far less data to become proficient users of a natural language.
This yawning divide between children and machines is called the data efficiency gap. And it raises a tantalizing question for cognitive scientists and a challenge for the architects of AI models: How is it that kids can still outperform the most linguistically sophisticated machines ever built?
Kids outlearn AI—and we still don’t know why, by Elise Cutts.
Given the massive computational scale and data volume used to train LLMs, it seems like a brute-force approach, when compared to the apparently much more efficient way that human children pick their first language.