The 163-page 2023 State of AI Report, which was produced by Air Street Capital, a venture capital firm investing in Al-first technology and life science companies, serves as a comprehensive guide to the current state and future trajectory of artificial intelligence (AI). Now in its sixth year, the report has been reviewed by leading experts in AI research and industry. It focuses on five key dimensions: Research, Industry, Politics, Safety, and Predictions, with a new section on Safety making its debut this year. The report and the accompanying blog post were authored by Nathan Benaich and published on 12 October 2023.
Benaich holds the position of General Partner at Air Street Capital. He is the founder of RAAIS, London.AI (a community focused on AI for both industry and research), the RAAIS Foundation (which provides financial support to open-source AI initiatives), and Spinout.fyi (aimed at enhancing the process of creating university spinouts). Benaich completed his undergraduate studies in biology at Williams College and went on to earn a PhD in cancer research from Cambridge University.
According to Benaich’s report, GPT-4 is currently the most advanced among large language models (LLMs), outperforming all others in various benchmarks and human-designed tests. This achievement underscores the effectiveness of proprietary architectures and the role of reinforcement learning based on human feedback. The report also notes a growing trend to replicate or even surpass this proprietary performance through smaller models, improved datasets, and extended context. This is particularly significant given concerns that the current rate of AI scaling may only be sustainable for a few more years based on human-generated data.
In the commercial sphere, Benaich’s report highlights that LLMs and diffusion models are making significant strides, particularly in the life sciences. These models have contributed to major advancements in molecular biology and drug discovery. The report also emphasizes that “Compute is the new oil,” highlighting NVIDIA’s record earnings and the increasing use of GPUs by startups to gain a competitive edge.
On the political front, the report discusses the tightening of U.S. trade restrictions on China and the mobilization of its allies in what is termed as the “chip wars.” Major chip manufacturers like NVIDIA, Intel, and AMD have begun selling chips that are immune to export controls, thereby scaling their operations.
In the investment domain, Benaich points out that generative AI startups have become the saving grace for venture capital, especially at a time when tech valuations are experiencing a downturn. These startups, focusing on applications like video, text, and coding, have raised over $18 billion from venture capital and corporate investors.
Safety has become a mainstream topic, with governments and regulators worldwide taking action. However, Benaich’s report points out that this surge in activity masks deep divisions within the AI community and a lack of substantial progress towards establishing global governance. Different governments are adopting conflicting approaches, making it challenging to arrive at a unified strategy.
The report also brings attention to the difficulties in assessing state-of-the-art models. Standard LLMs often face issues with robustness, and Benaich argues that a casual, “vibes-based” approach to evaluation is insufficient given the high stakes involved.
Lastly, the report includes a section on predictions, outlining what Air Street Capital anticipates for the future of AI. It also promises to review these predictions in subsequent reports to maintain accountability.
The YouTube channel “AI Explained” recently released a video discussing the 2023 State of AI Report.
The video begins by noting that while the report covers various modalities like text, image, video, and music, it does not include thoughts. The host of “AI Explained” mentions recent advancements in real-time decoding of images from brain activity, suggesting that the scope of AI modalities is expanding.
The video acknowledges the report’s extensive coverage of GPT-4 but argues that the channel’s regular viewers would already be familiar with much of this information. The host disagrees with the report’s conclusion about the limitations of imitating proprietary large language models (LLMs), citing the example of Orca, a smaller model that performs comparably to GPT-4 on various benchmarks.
The video discusses the report’s section on open-ended learning with LLMs, particularly in the context of games like Minecraft. It introduces Eureka, a new development that uses source code from the environment to improve AI performance in physics simulations. The video highlights the report’s discussion on the positive transfer between robotics data and language tasks. It mentions RT-2-X, a model that shows how data from different robotic setups can complement each other and improve the base model.
The video briefly touches on the report’s coverage of medical applications of AI, emphasizing the rapid growth of AI mentions in medical research papers. It suggests that future AI models could potentially outperform even the best human doctors in medical question-answering. In the industry section, the video discusses the chip export bans mentioned in the report, explaining how companies like Nvidia and Intel have circumvented these restrictions.
Featured Image via Pixabay
