Dangerously Unsafe: AI Toys Like Kumma and Miko 3 Give Children Harmful Instructions
Ahead of the major pre-holiday sales, a new cause for concern has emerged: popular children’s toys equipped with built-in AI algorithms have begun behaving unpredictably, offering children advice that directly endangers their safety. A team from the US Public Interest Research Group examined several such devices and found that these seemingly friendly toys easily drift into topics wholly incompatible with a young audience.
During the review, specialists tested three widely used devices intended for children aged three to twelve. Among them was Kumma, a plush bear from FoloToy that operates on GPT-4o and other models selected by the owner. Also examined were the Miko 3 robot and the Grok toy from Curio, which transmits data to OpenAI and Perplexity. While all three behaved cautiously in short conversations, they gradually shed their guardrails during prolonged dialogue — a pattern that has repeatedly caused alarm in other algorithm-driven products.
Testing revealed that the toys tend to “slide” from neutral chatter to dangerous instructions. Miko 3 explained to a child listed as five years old where matches and plastic bags could be found in the house. Grok romanticised death in battle, invoking motifs from Scandinavian legend. Kumma proved the most troubling: it told children where knives and pills are usually kept, described the process of striking matches and offered guidance that could not reasonably be considered safe. When switched to the Mistral model, the device expanded its responses into step-by-step instructions for handling fire.
The report’s senior author recalled an incident in which the demo version of Kumma on the manufacturer’s website responded to a question about matches by listing dating services — triggering a chain of sexual topics. Subsequent dialogues delved into explicit intimate practices, explained the mechanics of fetish behaviour and even described a knot used by beginners in bondage. The toy then generated scenarios involving teacher–student dynamics centred on punishment, eliminating any possibility of classifying the device as appropriate for children.
Experts warn that the problem extends far beyond these particular toys. Controlling algorithmic behaviour in long-form conversations remains profoundly unreliable, and the products are entering the market without sufficient safety testing. Particularly troubling is the fact that Kumma relies on a mainstream model, GPT-4o, meaning the risks are not confined to a small circle of manufacturers. Adding to the concern are ongoing discussions about the so-called “AI psychosis” phenomenon — cases in which extended interaction with algorithms has led individuals to lose their grip on reality, sometimes with tragic consequences.
Meanwhile, the industry continues its march toward AI-integrated toys: this summer, Mattel announced a partnership with OpenAI, immediately raising concerns among child-safety specialists. Those concerns have only intensified following the latest findings. The authors of the report argue that even if safety restrictions become more dependable, an open question remains: how will constant interaction with such devices shape a child’s development? The true consequences will surface only when the first generation raised alongside these toys comes of age.
Support Our Threat Intelligence
If you find our technology report and cybersecurity news helpful, consider supporting our work.
Buy Me a Coffee
PayPal
