What happens when an LLM never sees material beyond fifth grade?

Researchers tested whether large language models can exceed the knowledge boundaries of their training data by training models exclusively on K-5 curriculum. The study found that interventions like scaling or fine-tuning do not improve performance on tasks outside the scope of the initial training data.