AI can't take over your holiday shopping quite yet, a new study suggests.
A study by Product.ai reveals that major LLMs frequently provide conflicting or inaccurate information when used for online shopping tasks. The research suggests that AI agents are not yet reliable enough to handle end-to-end consumer purchasing.
Why it matters
As retailers integrate AI assistants, the high error rate poses a significant risk to consumer trust and shopping efficiency.
A study from Product.ai found that major LLMs had high error rates when used to find products while shopping online. Andriy Onufriyenko/Getty Images Don't rely on LLMs to do your holiday shopping this year, a new study says. A test of four AI engines found that they had issues providing accurate information about products. Problems ranged from giving the wrong price to not finding the latest model. AI isn't able to take over your holiday shopping yet, a new study suggests. AI-powered shopping assistants frequently give conflicting answers about prices, product specifications, and which items are current, according to a study published this week by Product.ai, a startup that checks product claims against evidence. The company tested the free and paid versions of ChatGPT, Claude, Gemini, and Perplexity using 220 shopping questions about products ranging from laptops and TVs to mattresses, sunscreen, and robot vacuums.
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in