In dieser Episode reden Florian und Ich über Apples Einstieg in Mixed Reality und wohin sich die VR und AR Industie entwickelt. Außerdem reden wir über Patentregeln für Generative KI , und DeepMinds neue KI.
Apple finally announced their upcoming VR headset which will focus on productivity and Cinematic entertainment. The 4K displays are powered by their M2 chip. This requires an external energy source and makes the headset with 3500$ very expensive. The Headset focuses on Mixed reality experiences similar to the new Meta Quest 3, but unlike the Quest, it will not be released until next year. It is probably a good starting point for Apple to build their new Product platform, but if you are not in desperate need of a high-resolution headset it is perhaps not a good choice for you.
Meta announced their new Meta Quest 3 headset. It is the successor to the Quest 2, the most popular VR headset of all time. The price went up a bit, the processing power and form factor improved as did the visuals. especially passthrough is better with color passthrough. Eye tracking is not included. Together with the upcoming Apple entrance into the VR space, this will give the XR World a new push forward.
2022 was an eventful year with lots of ups and downs. While the global economy is struggling, and problems like climate change and social instability continue to grow, there have also been some significant technological and scientific breakthroughs.
The most prominent developments probably happened in deep learning with the appearance of generative models that are able to generate human-level music, art, dialog, and code. In this context, I want to talk about two specific papers that shaped the field this year and most likely next year. The paper “Denoising Diffusion Probabilistic Models” which is the basis for Dall-E 2, Stable diffusion, and many other generative models, and the chinchilla paper from Deepmind, which demonstrated the importance of high-quality training data over model size. This will likely shape the design and cost of future models, including the anticipated release of OpenAI’s GPT-4 in 2023, which is expected to outperform humans in many text-based tasks. The improvements are not only driven by Moore’s law and architectural improvements but also the money spent to train and develop these systems increases. This is expected as the potential is more and more recognized and the value these systems provide is ever-increasing.
But not just GPT-4. AI will continue to disrupt various industries such as search and creative writing and spark public debate about its impact, even more than is happening right now. It will also lead to the production of high-quality media with fewer people and resources thanks to AI’s assistance. In the field of 3D generation, I expect to see similar progress in 2023, bringing us closer to the quality of 2D generation.
Fusion, the process of combining atomic nuclei to release a large amount of energy, has made significant strides in recent years. This is largely due to the incorporation of machine learning and advancements in various fields such as materials science and engineering. Recently, the U.S. Department of Energy announced that they were able to achieve a positive net outcome from a fusion reaction, which is a major milestone in the pursuit of unlimited clean energy. While I expect to see continued progress in this field, it is unlikely that we will see a commercial fusion reactor within the next two years. However, the upcoming start of the Iter project, an international collaboration to build a fusion reactor, may refuel interest and drive further developments in this promising area.
The James Webb Space Telescope (JWST) is an important milestone in the field of astronomy because it is designed to be the most powerful and advanced space telescope ever built. It started to operate this year. It is a collaboration between NASA, the European Space Agency (ESA), and the Canadian Space Agency (CSA). One of the main goals of the JWST is to study the early universe and the formation and evolution of galaxies. It will be able to observe some of the most distant objects in the universe, including the first stars and galaxies that formed after the Big Bang. In addition to studying the early universe, the JWST will also be able to observe exoplanets (planets outside of our solar system) and potentially search for signs of life on these planets. It will have the ability to study the atmospheres of exoplanets and look for biomarkers, such as oxygen and methane, which could indicate the presence of life. The JWST is also expected to make important contributions to our understanding of planetary science, by studying the atmospheres and surfaces of planets in our own solar system and beyond.
The hardware industry has faced challenges this year due to manufacturing bottlenecks. Despite the continuation of Moore’s law and the development of new alternatives to silicon, it has been difficult to obtain chips at this time. The industry is restructuring in order to better handle future demand for hardware. Specialized hardware, such as AI processors and quantum computers, are seeing rapid development. According to IBM’s roadmap, we can expect to see quantum computers with over 1000 Qbits in the upcoming year. GPUs will become more important with the rise of AI. However, these advancements in hardware technology also come with the need for careful consideration and planning in terms of production and distribution. Ensuring a stable and efficient supply chain will be crucial in meeting the increasing demand for these specialized hardware components.
Virtual Reality (VR) technology has experienced a difficult period in recent years due to overhyping of its potential. While some people may have expected VR to revolutionize the way we interact with and experience the world, it has yet to reach the level of ubiquity and practicality that was promised by Meta. But the year 2023 is shaping up to be a promising one for the VR hardware market, with multiple new headsets, such as the Quest 3, and maybe even an Apple Headset, set to be released. These new products will likely offer improved graphics, more intuitive controls, and a wider range of content and experiences. While it may not fully realize the vision of a “Metaverse”, VR is still likely to be a great entertainment product for many people
2023 will be a critical year for AR. It will be the first time that we can build affordable Hardware in a small form factor. Chips like the Snapdragon AR2 Gen 1 implement Wifi 7 and low energy usage and will make it possible to build Smart glasses. Depending on the availability and price of the chips and other components I expect glasses from many different companies with even more capabilities than Oppo air Glass 2.
One of the most exciting developments in computer interfaces is the emergence of brain-computer interfaces (BCIs). These allow for direct communication between the brain and a computer, enabling the possibility of controlling devices with thought alone. While companies like Neuralink are claiming to begin human trials next year, non-invasive BCIs present a much lower barrier to entry and are being actively developed by startups such as Synchron, which has received significant funding. AI will also help the field by decoding brain signals. It is likely that we will see at least one viral video showcasing the capabilities of these non-invasive BCIs, similar to the viral video of a monkey playing pong using a BCI that was released last year. The potential applications for BCIs are vast and diverse, ranging from medical and therapeutic uses to gaming and everyday tasks. As these technologies continue to evolve, it is exciting to consider the possibilities for the future of human-computer interaction.
Researchers from biotech and other fields were able to develop an mRNA vaccine for COVID-19 in less than a year. The same technology was also used to create a universal flu vaccine and a vaccine for malaria. The combination of biology and AI has yielded promising results in the development of treatments for various viruses and illnesses. For example, a team led by Chris Jones of the Institute of Cancer Research used AI tools to identify a new drug combination to fight diffuse intrinsic pontine glioma, a type of incurable childhood brain cancer. The proposed combination extended survival in mice by 14% and has been tested in a small group of children. Additionally, Dr. Luis A. Diaz Jr. of Memorial Sloan Kettering Cancer Center published a paper in the New England Journal of Medicine describing a treatment that resulted in complete remission in all 18 rectal cancer patients who took the drug. Overall, the progress in the field is accelerating thanks to advancements in AI, such as Alphafold 2, which are designed to find and develop treatments for various diseases. If this continues we will be able to beat cancer in the next few years, which leads to the next field.
I predict that every person under 60 has the potential to live forever, as I mentioned in my post about longevity escape velocity. The field of aging research has made significant progress in recent years and is more confident than ever in its understanding of the aging process and life itself. For example, researchers at the Weizmann Institute of Science in Israel were able to create fully synthetic mouse embryos in a bioreactor using stem cells cultured in a Petri dish, without the use of an egg or sperm. These embryos developed normally, starting to elongate on day three and developing a beating heart by day eight. This marked a major advancement in the study of how stem cells form different organs and how mutations can cause developmental diseases. This is a promising step toward the end goal: Achieving complete control over all biological processes in the body.
While this was a slow year in some aspects, major progress was made in most fields, and 2023 will be even faster. We are at the knee of an exponential blowup and we are not ready for what is coming. While I am still worried about how society will react and adapt, I am excited for 2023 and the rest of the decade.
I decided to split the metaverse blog post into a mini-series since the topic is so broad, that when I tried to put everything into one post I simply failed.
We start with the currently most relevant part: VR Hardware.
VR is one of the two technologies that will be the platforms for the metaverse soon. Arguably not the most important one, but the one that will be available first.
2023 will be a big year for VR. We will see some new VR devices from Meta, Apple, Pico, and others. Some of these new devices will tackle the most important problems for VR hardware.
The problem with existing VR devices, like the meta quest, is that you cannot use them for extended periods, and it is not a pleasant experience at all. They are too heavy, and they cause eye strain. The movement in VR leads to nausea and the ways to interact with VR are limited. On top of that, the viewing itself is far from reality.
Some of these problems will be fixed this year. Each new headset is lighter than the last one and Apple’s VR headset is supposed to have a way higher resolution than most currently available headsets thanks to apple silicon. Eye tracking is coming in meta’s next headset and in many others, which will help with performance and resolution, and will give us new ways to interact.
Some other problems like contrast, adaptive depth, distortion, and field of view are harder to fix and will take some time, but mark Zuckerberg recently showed some prototypes that tackle some of these problems too.
Most of these solutions require huge amounts of computation power, especially higher resolutions. Standalone Headsets will not be able to perform fast enough, at least not for the next year. I think apple is most likely to be able to bring a good visual experience to a standalone headset thanks to apple silicon, but their first model which is expected to launch in January 2023 will not be able to fix all the existing visual problems. Even PC-VR is still limited by data rates of cables and wireless transmission. We need at least Wi-Fi 6 to reach a point where wireless transmission is viable for realistic-looking VR experiences.
The problem with nausea will become less bad with improved visuals but as long as we use a controller to move the problem persists. I do not think omnidirectional treadmills are the way to go. they are too expensive, and most people do not want to waste that much space, money, and energy in their free time. Some applications use teleporting or walking in place to move, and many other solutions are currently being tested. While treadmills are not likely going to be a standard accessory, full-body tracking will be. The difference in emersion with full body tracking is huge and it gives VR another important input tool. Cheap full-body tracking solutions like slimeVR will become better and better and will give us realistic bodies in VR. The already mentioned eye tracking is another step of emersion that will be important for social VR. Being able to look someone in the eyes and read their mimic is a core element of human interaction and we are sensitive to strange facial movements. But eye tracking can do even more. It improves performance by limiting the resolution in areas that we are not looking at and it serves as an input device for VR. We can look at objects and control elements and the software will be able to extrapolate what we want to touch or click, which will remove frustrating moments like not being able to click the right button because of unprecise hand tracking. This brings me to my last point: hand tracking. It is arguably part of full-body tracking, but it is so important since hands are our primary way to interact with VR. Realistic and precise hand tracking is one of the most important aspects of emersion.
These videos show some of the key papers for hand tracking, published in the last two years. These papers are the foundation of meta’s hand tracking and will most likely continue to improve in the coming year.
If we look at the current development of headsets in the market it looks pretty good.
And the number of Headsets that are used every month for gaming is a good indicator for this upcoming billion-dollar entertainment industry.
I think we will see an even greater wave of people getting into VR in the next 2 years. Not just for gaming, but with apple joining the market, we will also see increases in areas like education and industry.
In the end, I want to take a short look into the far future of VR and virtual reality. I am talking about 5-10 years, probably after a technological singularity. The final goal of VR is full dive. The ability to simulate all 5 senses directly within the brain and to intercept all outputs from our brain to paralyze our body and redirect all movement into virtual reality. I will not talk about the implications for society that is a topic for another time, but from a pure hardware perspective, this is extremely challenging. While reading the output of the brain is an area where we are currently making a lot of progress, intercepting the signal to prevent our body from moving is not possible right now without a lot of medical expertise and long-lasting effects. Sending signals for all senses directly into the brain is even harder since every brain is different. I do not think we will be able to do this without an AGI, but if in the far future a machine overlord decides to put us all in our own matrix it will hopefully be heaven and not hell.