Liquid AI has announced the availability of its "new generation" of generative AI models that deliver "state-of-the-art performance at every scale while maintaining a smaller memory footprint and more efficient inference." Liquid AI refers to its large language models as Liquid Foundation Models (LFMs).
Something quite interesting about these new LFMs is the fact they're not using transformer architectures like ChatGPT (the T stands for transformer). This design choice allows LFMs to be more memory-efficient.
The current array of LFMs includes LFM 1.3B, LFM 3B, and LFM 40B MoE. The numbers represent the number of parameters, the more parameters there are the smarter the models get but also require more powerful computing resources. The MoE refers to a mixture of experts where the model is split up with each sub-network, or expert, specializing in different parts of the input, often resulting in a better output.
In a graphic shared by Liquid AI (at the top of this article) which shows the number of parameters on one axis and the MMLU-Pro benchmark performance on the other axis, the new LFM models outperform other models of a similar size, including Google's Gemma models, Mistral's models, Microsoft's Phi models, and Meta's Llama models. The LFM 40B MoE model outperforms the Llama 3.1 170B model just slightly which is an enormous feat.
Liquid AI said that its goal is to explore ways to build foundation models beyond Generative Pre-trained Transformers (GPTs). Currently, the LFMs are good at general and expert knowledge, mathematics and logical reasoning, efficient and effective long-context tasks, and multilingualism - they know English, Spanish, French, German, Chinese, Arabic, Japanese, and Korean.
Right now, they also have some limitations. Today, they're not good at zero-shot code tasks, precise numerical calculations, time-sensitive information, counting r's in the word "Strawberry", and human preference optimization techniques haven't been applied extensively to them yet.
When OpenAI started, they wanted to be open about the advancements they were making but are now a secretive company on the cusp of becoming fully for-profit. Liquid AI said that it will openly publish its findings and methods through scientific and technical reports. As it has invested a lot of time and resources on these models, it said they will not be open-sourced yet.
The new LFMs can be used today on Liquid Playground, Lambda (Chat UI and API), Perplexity Labs, and soon on Cerebras Inference. The LFM stack is being optimized for NVIDIA, AMD, Qualcomm, Cerebras, and Apple hardware.
Source: Liquid AI
1 Comment - Add comment