ZML Launches Free AI Inference Software to Enhance Performance Across Multiple Chips

On November 4, French startup ZML unveiled its LLMD software, designed to optimize AI inference across a range of chips, including Nvidia, AMD, Google’s TPU, Apple Metal, and Intel Arc. This launch aims to break down existing barriers in AI technology and provide more options for enterprises.

Understanding ZML's Innovative Approach to AI Inference

ZML, co-founded by Steeve Morin and endorsed by Yann LeCun, focuses on making AI inference more efficient. Morin stated, "The idea is to give people back the power to create their own system and achieve real efficiency gains that allow [AI] to be disseminated.” The software enables various open-source large language models to run at peak performance on different hardware.

As the demand for AI technology grows, the optimization of inference—the process of executing prompts—has become increasingly important. Morin pointed out that the current landscape is plagued by software and architectural barriers that often lead to vendor lock-in. ZML's solution could disrupt the market by allowing users to select from a mix of chips, potentially reducing costs and energy consumption.

Market Impact and Competition in AI Inference

The introduction of ZML/LLMD comes amid what Morin describes as the "inference gold rush," where companies are investing heavily in improving AI inference capabilities. ZML faces competition from companies like Baseten, valued at $13 billion, and Inferact, creators of the open-source project vLLM. Other competitors include RadixArk, which is behind SGLang.

Despite this competition, Morin remains optimistic about ZML's prospects. He emphasized the startup's agility, attributing it to its lean team of 20 and significant venture funding amounting to $20 million from firms like 20VC and Kima Ventures. He noted, "We have reached the point where we are co-designing silicon,” indicating ZML's commitment to advancing AI hardware.

Future Plans and Strategic Goals for ZML

Although ZML/LLMD is currently a free product aimed at gathering user insights, Morin has not ruled out monetization in the future. He stated, “I’d rather measure and [then generate revenue] where it is most effective without hindering my growth stupidly because I have been too greedy from the get-go.” The startup plans to continue innovating, with more releases on the horizon.

Morin believes that ZML's approach can support emerging AI chipmakers, particularly those from Europe, such as Axelera and Kalray. He expressed confidence that ZML can collaborate on groundbreaking technologies that have not been developed elsewhere.