Each year the Stanford University Institute for Human-centered Artificial Intelligence (HAI) issues a robust report on the state of AI. This is what you need to know.
Don’t have time to read 502 pages of the 2024 AI Index? The topline takeaways are below.
1. AI beats humans on some tasks, but not on all. AI has surpassed human performance in image classification, visual reasoning, and English comprehension. It still needs to conquer competition-level mathematics, visual commonsense reasoning and planning.
2. Industry continues to dominate frontier AI research. 51 machine learning models were produced by industry in 2023. Academia, once the mainstay of AI research and development, produced 15. Another 21 were produced from industry-academia collaborations.
3. Frontier models get way more expensive. Training state-of-the-art AI models has become extraordinarily expensive. OpenAI’s GPT-4 cost $78 million to train, Google’s Gemini Ultra $191 million.
4. The United States leads China, the EU, and the U.K. as the leading source of top AI models. 61 AI models came from the US, 21 from the European Union, and 15 from China.
5. Robust and standardised evaluations for LLM responsibility are seriously lacking. Responsible AI reporting needs to be standardised. OpenAI, Google, and Anthropic all use different responsible AI benchmarks, inhibiting the comparison of risks and limitations between models.
Training state-of-the-art AI models has become extraordinarily expensive. OpenAI’s GPT-4 cost $78 million to train, Google’s Gemini Ultra $191 million.
AI Index 2024
6. Generative AI investment skyrockets. Funding for generative AI increased 8x, totaling $25.2 billion in 2023. OpenAI, Anthropic, Hugging Face, and Inflection raised significant capital.
7. The data is in: AI makes workers more productive and leads to higher quality work. Research found that AI enables workers to complete tasks more quickly, and improve the quality of their work, and bridge the skill gap between low- and high-skilled workers.
8. Scientific progress accelerates even further, thanks to AI. Science-related AI applications like AlphaDev (making algorithmic sorting more efficient) and GNoME (facilitating materials discovery) are pushing science forward rapidly.
9. The number of AI regulations in the United States sharply increases. AI-related regulations grew by 56.3% in 2023. The US has 25 AI-related regulations, an increase of 24 since 2016.
10. People across the globe are more cognisant of AI’s potential impact—and more nervous. 52% of respondents to a global Ipsos survey expressed nervousness toward AI products a ndservices, a 13 per cent increase from 2022.
The 2024 AI Index has expanded its scope to cover public perceptions of AI, geopolitical dynamics, and impacts on science and medicine.
Ray Perrault and Jack Clark are the co-directors of the AI Index. They published a statement stating that the increases in awareness and investment into AI are driving change in organisations across the globe.
“Although global private investment in AI decreased for the second consecutive year, investment in generative AI skyrocketed. More Fortune 500 earnings calls mentioned AI than ever before, and new studies show that AI tangibly boosts worker productivity,” Perrault and Clark’s statement reads.
They caution that both the public and governments around the globe have concerns, however.
“On the policymaking front, global mentions of AI in legislative proceedings have never been higher. U.S. regulators passed more AI-related regulations in 2023 than ever before. Still, many expressed concerns about AI’s ability to generate deepfakes and impact elections. The public became more aware of AI, and studies suggest that they responded with nervousness.”
Look back on the week that was with hand-picked articles from Australia and around the world. Sign up to the Forbes Australia newsletter here or become a member here.
