As we are nearing the end of 2024, it’s time to revisit the predictions I laid out at the beginning of this year again. The year was even more interesting than the last one and there is a lot to talk about. Let’s evaluate how my forecasts stood against the reality of 2024. (If you just want to see my predictions jump here)
AI was an even bigger topic this year than last year, and I am happy to say that most of my predictions came true. Image/video/music/3D generative models indeed got a lot better and made a mark this year and I was also right that there will be no popular LLMs with over 1 trillion parameters this year. However, I was wrong about the speed at which LLMs will be integrated into assistants like Alexa or Siri. While there are plans to do so and some functionality is already there, it is not where I imagined it to be at the end of 2024. The main reasons for this delay are cost, scale, and reliability. I am especially proud of this part of my predictions:
State space models like RWKV will also become more relevant for specific use cases and most models will at least support image input if not more modalities. RL Models like Alphafold will push scientific discovery even faster in 2024.
SSMs, multimodality, and RL models all played a big role this year in the areas I expected. We had GPT4-Omni as a truly multimodal model, Jumbo and Mamba as new SSMs, and multiple Alpha models from AlphaGeometry to AlphaProteo over the year. Open-source models also had a great year, with Llama 3, Mistral, and Deepseek.
Open-source models will stay a few months behind closed-source models, and even further in areas like integration, but offer more customizability. Custom AI hardware like the AI Pin will not become widespread, but smartphones will adapt to AI by including more sensors and I/O options
I think it is fair to say that open source will always be a bit behind the top models like Claude 3.5 or GPT-4o. AI gadgets also had a big year, but in my opinion not a single one of them took off. New smartphones are also more centered around AI than ever before. I also had a few misses like my prediction that talking to your PC would be one of the main interfaces at the end of the year. I think I would be closer here if Microsoft shipped the version of Copilot that they showed a while ago, but that was delayed which means I am wrong here.
I made on more precise prediction about the GAIA benchmark.
No system in 2024 will outperform Humans on the new GAIA benchmark, but they are going to double their performance on it. This will mostly be accomplished by improving reasoning, planning, and tool use with improved fine-tuning and new training strategies.
We are on the mark right now with exactly double the scores from last year using the predicted improvements.
The next topic is AI chips and hardware where I was mostly right about demand and Nvidia’s lead. I also predicted the rise of custom chips like Groq, or Samba. My prediction that half of the global compute would be used for AI at the of the year was a bit of an overestimation. I think depending on how broad your definition of AI is, I could argue here but I will just take the loss and move on to the next prediction.
VR Hardware will continue to improve, and we will finally see the first useful everyday AR glasses towards the end of 2024. Quantum computers will become part of some of the cloud providers and will be offered as specialized hardware just like GPUs (Note: This part was written before the AWS Event announcement). They will become more relevant for many industries as the number of Qbits grows. We will also see more variety in chips as they become more specialized to save energy. Brain-computer interfaces will finally be used in humans for actual medical applications.
As you can see in the note, part of the prediction was already true before I released the last blog post and the rest became true after a while. At least if we exclude the prediction of everyday AR glasses. I have a history of overestimating the speed of smart glasses and I am a bit disappointed that we still have to wait for them. We saw the Orion glasses by Meta a few months ago which already are very close to my idea of true AR glasses, but they are sadly not available yet. Brain-computer interfaces were used in actual patients and we saw a lot of new chips specially designed for AI in some form.
I made some general predictions about humanoid robots that were not really specific to 2024 and my opinion on that did not change:
I expect an initial hype around them and adoption in some areas. However, towards the end of the decade they will be replaced with special-purpose robots and humanoid robots will be limited to areas where a human form factor is needed
The overall amount of robots did increase and we sadly saw them most prominently as weapons in Ukraine where both ground robots and small drones were used.
My energy predictions included a commercial fusion reactor which exists now. I also said that nuclear energy and solar would get better and more popular. Nuclear energy had a comeback story this year thanks to the energy demands in AI. Solar is growing faster every year and is surpassing expectations, especially in China.
I made the claim that room-temperature superconductors would be found this year or next if they exist, and while we have no conclusive proof that we found them yet, I still believe that we will, with the help of all the new material science AIs, find them very soon.
Biology and medicine are poised to make significant leaps, powered by AI systems like Alphafold and similar technologies. Cancer and other deadly diseases will become increasingly treatable and aging will become a target for many in the field. The public opinion that aging is natural and cannot/should not be stopped will not change this year but maybe in 2025. Prostheses will become more practical and will be connected directly to nerves and bones. This will make them in some areas better than human parts, but touch and precision will continue to be way worse. We will also see progress in artificial organs grown in animals or completely made in a lab.
These predictions are a bit mixed. Deepmind won the Nobel price for Alphafold and we got not only Alphafold 3 and AlphaProteo, but many more AI models related to biology and medicine. We got cancer vaccines and Prostheses also got better. Growing organs is making progress, but is not as far as I hoped.
Transportation in 2024 will change slightly. EVs will become more popular and cheaper but will not reach the level of adaptation that they have in China. Self-driving cars will stay in big cities as taxi replacements and will not be generally available until 2025. […]
Other infrastructures like the Internet will continue to stay behind the demand for the next few years. The main driver of the increased need for bandwidth will be high-quality video streaming while the main need for speed will arise from interactive systems like cloud-based AI assistants.
EVs did become more popular but are severely limited by import duties which keep prices high outside of China. Internet infrastructure is needed desperately, not just because of AI inference as I predicted, but mostly because of AI training in the US which needs to connect data centers. I did not anticipate the possibility of training across data centers.
Climate change and unstable governments will lead to an increase in refugees worldwide and social unrest will increase. We will see the first effects of AI-induced Job losses. The political debate will become more heated and some important elections like the US election will be fully determined by large-scale AI-based operations that use Fake news, Deepfakes, and online bots to control the public opinion.
My final prediction is sadly the one that is the closest to reality. Wars and economic problems lead to refugees around the world. Economies that are not profiting from AI are struggling and radical parties are gaining support around the world. The US election is filled with misinformation campaigns, both domestic and from outside. We can see the effect of a wave of AI-generated lies that are too much to fight and manipulating voters is as possible as never before. (At the time of this writing the result is not out).
All in all the year was as fast as expected. Some areas were a bit slower and some a bit faster. AI did not scale as fast but made progress in other directions. Outside of technology, this was not a good year. Wars continued and escalated, soft fascism is on the rise globally and we broke the 1.5-degree threshold. This year was a test for international order and we failed. So let’s hope for the next year.
My Predictions
As always we start with AI. Since this will be a bit more comprehensive I will make a list of the things that I think will be most important for AI in 2025
Test time compute (reasoning models): After the announcement of o1, it took less than two months for other labs to develop similar reasoning models. The advantages of these systems are clear: models become better at planning and reasoning, shifting costs from pretraining to inference, introducing new levers to scale, and enabling selective use of compute for complex tasks. Deepseek was among the first to release an alternative, and many others will follow. Reasoning will not stay limited to generating chains of thoughts in advance. More complex forms of test time compute will emerge like Coconut or different forms of search. They will become more and more complex and better. The true potential of these models will shine in 2025 when they become open source and are combined with custom ASICs like Cerebras for fast and cheap inference. While o1 is expensive and takes up to 2 min to think, open source alternatives could run on up to 2000tok/s and cheaper enabling reasoning in seconds. This is also the reason why the demand for custom inference chips is growing. More companies will develop their own chips for this purpose. OpenAI is working on this but will not have their own chips in 2025. The growing demand for reasoning models will drive the demand for inference computing. I really want to emphasize how crucial it is to get inference faster and cheaper. The demand could easily surpass a trillion tokens per hour.
The trend of cheaper and cheaper APIs will slow down in 2025 and most improvements in efficiency will go towards the growing demand for tokens. Companies will rise and fall with the ability to serve fast and cheap reasoning.
Pretraining and scaling: As I already said last year, scaling will not happen as fast as all these companies claim. Two years after GPT 4 we will see trillion parameter models again, but not much bigger. Nothing bigger than 5 trillion will appear in 2025 and the demand will be really low because of the pricing and speed. The relevant sizes will be <7B for edge devices, 20-70B for cheap and fast models that can also be self-hosted, and 70B- 700B for the top models. Everything above that is not suited for mass usage. Reasoning models will stay between 7B and 300B. Everything below is not really powerful enough to make full use of test-time compute and everything above is too slow and expensive. Data for pretraining will not change significantly but fine-tuning and post-training will change a lot with synthetic data. The number of training tokens could exceed 20 trillion tokens for a single model next year. Models will learn from other models, reasoning models will get distilled into smaller models, and so on. B100 and B200 will dominate the training hardware and lead to faster training speeds of bigger models, enabling laps to experiment more and iterate faster on existing models. Here is a list of Models to expect: The next GPT generation from OpenAI, Claude 4 family, Llama 4 family, Gemini 2 (Probably released before I finish this blog post)(Update: It was), and reasoning models from all relevant labs. Models will get more general and form foundation models that combine modalities. We saw this trend in 2024 already with models like Fugatto that are not trained for a single task but form a foundation for all audio tasks. Foundation models for video will exist in 2025 and models that combine multiple modalities will appear more often. We will see some small changes in the architectures, like different tokenization, loss functions, or different integration of modalities, and changes to better work with reasoning and test time compute. There is a small chance that we will see some more radical changes.
Agents: Agents were already a popular term in 2024 but were lacking in every aspect. They need reasoning models which did not exist for most of 2024 and they need a lot of scaffolding around the models which takes time. In 2025 we have the necessary capabilities in the models and hardware that is fast enough. A lot of work in 2024 will pay off in 2025 and lead to some impressive systems that will be able to easily perform long-horizon tasks like playing Minecraft, writing a book, doing research, and working as assistants. At the end of 2025, they will appear like something that I would call weak AGI. A system that is not able to do everything a human can but is as general as a human with capabilities that a human would not have and a big overlap. AI APIs like Anthropics Model Context Protocol will become more important as they speed up the interaction between Agents and tools without them relying on GUIs.
AI overall will improve dramatically in 2025. Not a single lab will get a clear lead and Google, OpenAI, Anthropic, and others will shine in slightly different areas without a clear winner. OpenAI will probably be the first to “weak AGI” but at the highest cost, and others will follow in early 2026 for less money. Open Source will stay behind by around half a year in most areas unless certain teams like Meta AI or Qwen are not allowed or do not want to continue open sourcing. Some benchmark predictions: ARC AGI will be beaten(85%) (Update: o3 was announced after I wrote this and already beat it), Frontier math will get close to 50% by the end of 2025, and most other existing Benchmarks will be saturated; Including the GAIA benchmark that I talked about last year. They will develop into three product fields. One is for consumers focusing on tool use, multimodality, and memory, and the other is for commercial use, which will focus on using cheap and fast LLMs for automatization and different workflows. The last one is AI for research. This field will make the most use of reasoning models and will use the most expensive Models and simulations to advance science in 2025. If the answer has scientific value, it is worth spending thousands of dollars on a single question.
RL and science: Outside of the generative AI we have a growing number of specialized AI systems that solve core problems in science. Deepmind is at the forefront of this trend and will continue releasing models like AlphaProteo that revolutionize research. As I mentioned last year, I think if a superconductor exists, we will find it this year. Material science is one of the fields that is taken over by AI and the same goes for parts of physics, math, and biology. 2025 can become a breakthrough year in science. End-to-end vaccine creation could become possible in early 2026. Reasoning models will start to help develop breakthroughs in math and computer science like faster algorithms, new proofs, and new hypotheses.
Hardware: There are multiple trends that will develop. AI servers for hobbyists like Tinybox will become quite popular as open-source models get better and the need for privacy increases. Training will mostly happen on Nvidia AI chips like B100 and the next-gen. Moving the training to other platforms is too complex and expensive which is why Nvidia will stay in the lead here. For inference, the Hardware will become more diverse. Custom ASICs like Cerebras will grow, and most big companies like Meta, OpenAI, Amazon, etc will try to develop their own chips for their models. The biggest demand is, as I already said, fast and cheap inference for reasoning. Google is unique here as they already have TPUs (Trillium) for training and inference which will give them a huge advantage in 2025. I do not think that there will be v7 TPU for most of 2025, but some announcements about it will come out towards the end of the year. This next generation will likely have a special version optimized for inference (even more than v6e). There are Microsoft AI PCs which will stay a gimmick as laptops will have to rely on AI servers anyway for any significant task. Nvidia will try to develop their own inference-optimised chip, or at least increase the memory of their current B200 cards to support faster inference. A software trend that could help here is the transition to lower precision training and inference which gives massive speedups and saves in memory demand.
Robots: 2025 will be the year of humanoid robots. Some will start mass production and they will appear more and more in different industries. They will make huge leaps in autonomy, going from needing manual programming per task to being able to be controlled by voice and demonstration. The software side will improve rapidly using AI generated simulations to train in. World models will serve as training grounds for RL. Some companies will start to market towards consumers but the demand will be very low, because of the price and some general dislike for machines. Instead specialized robots will be very popular. They will change industries like agriculture or public service, like collecting fruits from fields, collecting trash bins in the city, or patients from their rooms. A lot of this will start in 2025. In most cases as single tests but more broadly towards the end of the decade. Robots will quickly be part of every part of live and work.
Self-driving cars are slowly getting better. Waymo will expand its areas and continue to grow. Tesla will not have full self-driving ready in 2025, at least not for the existing cars, but there is a small chance that Elon will use his influence on the government to get permission to sell it as such, leading to a big increase in Tesla-related accidents.
Energy: Solar will continue to surpass expectations and grow the fastest. Fission reactors will return as the need for reliable power for AI increases. Fusion will have its first commercial reactor built but no energy will enter the grid from a fusion reactor in 2025. CO2 emissions will peak in 2025 again and will start to go down at the end of the decade. The global energy consumption will grow by over 3% in 2025, fueled by AI and electric cars.
Quantum computing: Quantum computer will continue to rapidly grow the number of stable Qbits and start being used for material science. They will however not reach the point where they start breaking encriptions in 2025. More applications for quantum computers will be developed now that they are available. I do not expect Quantum computers to become relevant for machine learning as there are no known ways to gain speedups over current hardware given the current models and training methods.
VR/AR: 2025 will be a big year for AR. Multiple AR glasses will come out and mark the beginning of the next big product category. VR glasses will have a slower year with just incremental improvements such as a new Quest, a new VR device from Valve, and the upcoming Samsung headset. The bigger focus will be on the software side, both for VR and AR. Glasses are the perfect form factor for AI assistants. Android XR will use Gemini starting in 2025, Meta will use Llama 4 in the next generation of their glasses, and Apple will stay behind in terms of AI assistants in glasses.
Longevity: 2024 already showed a lot of progress in the research side of longevity, but also the popularity started growing. Movements like “Don’t Die” gained fans and some news outlets started talking about the possibility. This will speed up in 2025. The idea that it is possible to stop aging using modern medical knowledge will become more mainstream. This will lead to more public discussion about the feasibility and consequences. I can see longevity becoming a key component of the class struggle in the coming decade. The difference in life expectancy between rich and poor will grow rapidly over the next 10 years. It is hard to make short-term predictions about this topic since the result of most human experiments will only appear after a few years. I believe that the first person to reach 200 is already alive and probably already older than 50 and I made a very similar prediction 2 years ago.
Geopolitics: I have a very hard time predicting global politics this year. Trump and other extremists are unpredictable and add to an already uncertain development fueled by the exponential development of Technology. What I can say is that wars will continue and become more and more dominated by technology. We already saw the first drones and robots fighting in Ukraine and this technology will develop rapidly. By the end of this decade, autonomous drones will be the main form of warfare for a developed country. Global influence will shift, with China growing as a global superpower. Silent warfare in the form of cyberattacks and attacks on infrastructure will increase worldwide. The consequences of climate change will devastate the global south and create anger. A bigger escalation is very likely in the next 4 years. Some more precise predictions for 2025: More attacks on undersea cables or satellites, and Russia will gain more parts of Ukraine with the war slowing down. The situation in the middle east around Israel will escalate more. The US will enter at least one conflict zone, with a small chance of sending infantry. There will be at least one high-profile political assassination (a president or similar) and the number of democratic countries will go down.
It is getting harder and harder to make predictions as we speed up as a species.
The interplay between technological advancement and geopolitical instability creates a precarious balance, where the tools of progress can simultaneously become instruments of disruption. I fear that we have reached a point where humanity cannot keep up with their own development and we run into a state of escalation and die a heat death as a society. We need to strengthen our intellectual foundation and create better education systems to prevent a future where we are just apes with access to nukes. We have to give up control to AI at some point when the complexity of the world becomes too much for our brains or become the Übermensch described by Nitschke and develop ourselves to the point that we can keep up.
I did not wrote a lot this year and I hope to change that next year. I finished my degree this year and hope to start many new projects in 2025. You are welcome to write your predictions in the comments.
