A software engineer has detailed his quest to develop an ‘anti-Turing test‘ to prove that Facebook‘s personal assistant service ‘M’ is powered exclusively by humans, rather than AI as the company claims.
Facebook has touted M as a comprehensive personal assistant to rival Apple’s Siri and Microsoft’s Cortana — with the caveat that M is powered by humans, not simply AI. But now Arik Sosman, software engineer at BitGo, has taken to Medium to claim that M’s artificially intelligent element is smaller than advertised.
“M’s capabilities far exceed those of any competing AI,” Sosman writes. “Where some AIs would be hard-pressed to tell you the weather conditions for more than one location, M will tell you the forecast for every point on your route, and also provide you with convenient gas station suggestions, account for traffic, and provide you with options for food and entertainment at your destination.”
“When communicating with M, it insists it’s an AI,” he continues. “And that it lives right inside Messenger. But it’s non-instantaneous, and the sheer unlimited complexity of tasks it can handle suggest otherwise.”
Sosman used a number of examples to illustrate his thesis — which he called “much harder” than a traditional Turing test as “it’s much easier for humans to pretend to be an AI than for an AI to pretend to be human”.
Sosman asked M to perform a set of complicated tasks that “no other AI could pull off” — asking it for directions and then altering his request slightly. The AI was able to perform his request — but not without a few spelling mistakes and other errors that Sosman believes proves M is powered by humans. And when M called a landline, it was operated by a distinctly human woman.
“Thus, here we are. We have definitive prove that M is powered by humans,” wrote Sosman. “The next question is: Is it only humans, or is there at least some AI-driven component behind it?”
Facebook insists that M is powered by AI, but that it is trained and supervised by human beings — thus explaining the human errors experienced by Sosman.
The personal assistant service is part of an ambitious long-term project within the Messenger team to build a complex AI system that can complete tasks without human supervision. It is expected that M will become more and more automated over time.