-
Learning to Grow Pretrained Models for Efficient Transformer Training
A new research paper proposes a method to accelerate the training of large-scale transformers, called the Linear Growth Operator (LiGO). By utilizing the parameters of smaller, pre-trained models to initialize larger models, LiGO can save up to 50% of the computational cost of training from scratch while achieving better performance. This approach could have important… — read more
-
ChatGPT’s biggest update jet
OpenAI announced that they will introduce plugins to ChatGPT. Two of them developed by OpenAi themself allow the model to search the web for information and run generated python code. Other third-party plugins like Wolfram allow the model to use other APIs to perform certain tasks. the future capabilities of a model enhanced this way… — read more
-
GitHub announced Copilot X
After Copilot became inferior to GPT-4, they finally announced a set of new functionalities based on GPT-4, like Generated pull requests, answering questions about code or documentation, and helping with coding. — read more
-
Google opens Bard
Google’s GPT alternative Bard is now available in the US and UK. Early testers already speak out in favor of Bing which also launched image generation this week. Bard is based on LaMDA, an older Language model that is not as capable as GPT-4. — read more
-
Nvidia goes big in AI
Right now the GTC 2023 is going on and Nvidia showed off some of their newest steps in AI including this amazing Intro. They introduced cuLitho, a new tool to optimize the design of processors. This was a complicated process that took weeks to calculate and can now be done in a few hours. Speeding… — read more
-
From GPT-4 to Proto-AGI
Deutsche Version Artificial General Intelligence (AGI) is the ultimate goal of many AI researchers and enthusiasts. It refers to the ability of a machine to perform any intellectual task that a human can do, such as reasoning, learning, creativity, and generalization. However, we are still far from achieving AGI with our current AI systems. One… — read more
-
GPTs are GPTs: How Large Language Models Could Transform the U.S. Labor Market
A new study by OpenAI and the University of Pennsylvania investigates the potential impact of Generative Pre-trained Transformer (GPT) models on the U.S. labor market. The paper, titled “GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models,” assesses occupations based on their correspondence with GPT capabilities, using both… — read more
-
FlexGen Enables High-Throughput Inference of Large Language Models on Single GPUs
FlexGen is a new generation engine that enables high-throughput inference of large language models on a single commodity GPU. It uses a linear programming optimizer to efficiently store and access tensors and compresses weights and attention cache to 4 bits. FlexGen achieves significantly higher throughput than state-of-the-art offloading systems, reaching a generation throughput of 1… — read more
-
New Speech Recognition Model by AssemblyAi
AssembyAi added a new speech recognition model to their products. Conformer-1 is “a state-of-the-art speech recognition model trained on 650K hours of audio data that achieves near human-level performance and robustness across a variety of data.” It combines convolutional networks with transformers to archive never seen scores on various recognition tasks. — read more
-
Microsoft presents its copilot for Office
Today Microsoft showed off how they integrated AI tools, including GPT-4, into their office products. You can ask Copilot to build excel tables, PowerPoints, and Emails or ask it about meetings, or lets it summarise documents and chats. Although currently only available to a select few companies, Copilot is set to become widely available over… — read more
-
GPT-4 is here
OpenAI presented its new GPT model today. GPT-4 has a context window of 32K tokens and outperforms humans and previous models like GPT-3.5 in almost all language tasks. It is also multimodal and supports images as inputs. Read more here or watch the presentation here. OpenAI just released GPT-4, a game-changer in AI language models.… — read more
-
Google is publishing its new AI tools and APIs
In a new blog post, Google presents their Generative AI App Builder, PaLM API, and MakerSuite which works similarly to OpenAI’s playground. This announcement is happening shortly before the Microsoft presentation on Thursday. Similar to how they did it with their Bard presentation just before the Bing chat announcement. — read more
-
MathPrompter: Mathematical Reasoning using Large Language Models
Microsoft published a new paper in which they present the language model MathPrompter which uses the Zero-shot chain-of-thought prompting technique to generate multiple Algebraic expressions or Python functions to solve the same math problem in different ways and thereby raise the confidence level in the output results. This led to a score of 92.5 on… — read more
-
Jan Leike Shares His Ideas on Alignment
Jan Leike, the alignment team lead at OpenAI, has published a blog post where he presents possible solutions for AI aligning to group preferences. He proposes a simulated deliberative democracy, where groups of randomly selected people agree on a stand on a specific topic. The decision is then used to fine-tune the AI. — read more
-
GPT-4 Next Week
In a small german information event today, four Microsoft employees talked about the potential of LLMs and mentioned that they are going to release GPT-4 next week. They implied that GPT-4 will be able to work with video data, which implies a multimodal model comparable to PaLM-E. Read more here. — read more
-
Large Language Models: An Overview
Large Language Models (LLMs) are machine learning-based tools that are able to predict the next word in a given sequence of words. In this post, I want to clarify what they can and cannot do, how they work, what their limitations will be in the future, and how they came to be. History With the… — read more
-
Google crushes Speech recognition
Google released the Universal Speech Model (USM), which can transcribe over 300 languages. It outperforms the state-of-the-art model Whisper in the 18 languages that Whisper supports. This is part of Google’s plan to support the 1000 most spoken languages. The model is with 2B parameters slightly bigger than Whisper and was pre-trained mostly on unlabeled… — read more
-
Google presents PaLM-E. An Embodied Multimodal Language Model
PaLM-E has 562B parameters which make it one of the largest models today. It combines sensory data from a robot with text and image data. It is based on PaLM and was fine-tuned on input & scene representations for different sensor modalities. These kinds of more general models are the way to more powerful and… — read more