Experts have been saying for awhile now that we just have to wait, and AI tools like ChatGPT will evolve enough to be good enough to pass as human.
While many have been laughing at its failures, experts say GPT-4 is fooling a good 50% they’ve put in front of it.
University of California San Diego researchers claim in their new paper that GPT-4 is the first large language model (LLM) to fully pass the Turing test – and to fool more than half of the people interacting with it, too.
The researchers involved performed a simple experiment, asking 500 people to have a five-minute text conversation with a human or a chatbot built on GPT-4.
Then, they asked them if they thought they’d been talking to a person or AI.
54% of the subjects believed they’d been speaking to humans when they’d actually been interacting with OpenAI’s ChatGPT-4.
The Turing Test was theorized by computer science pioneer Alan Turing back in the 1950s. In his original test, Turing theorized three “players.”
One was a human interrogator, another was a witness of indeterminate humanity or machine-ness, and the last was a human observer.
The UC San Diego team tweaked the original formula by eliminating the third human observer, but added a few extra witnesses.
In addition to the GPT-4 bot, they had the participants chat with another human, a GPT-3.5, and a 1960s chatbot called ELIZA.
The study authors hypothesized that participants would be able to easily identify ELIZA, but would only be successful with the newer models about half of the time – and they were right.
While just 22% of the participants thought ELIZA was a human, 50% believed GPT3.5 was real, compared with the 54% who bought GPT-4’s ability to mimic humanity.
The paper is still under review, but has received an endorsement from Ethereum cofounder Vitalik Buterin, who says this research “counts as GPT-4 passing the Turing test.”
Once the paper has gone through a full peer review, we might all be able to jump on the bandwagon.
Or light our pitchforks.
Either way.
If you enjoyed that story, check out what happened when a guy gave ChatGPT $100 to make as money as possible, and it turned out exactly how you would expect.