How We Cut LFM3’s First-Token Latency from 710 ms to 200 ms on Dragonwing IQ9
We brought six Liquid AI models to Qualcomm’s dual-NPU board for a hackathon in San Francisco. The biggest speedup came when LFM3-VL answered a question about an image.

On November 13 in San Francisco, the teams at Liquid AI and Qualcomm’s hackathon knew what would happen on demo day. The network would be off. If their app needed to see, listen or speak, one Dragonwing IQ9 on the table had to do the work.
Argus, the winning team, built an operating-room assistant around that constraint. In their demo, LFM3-Omni watched for surgical instruments and listened for spoken updates. Both inputs stayed on the board. For that application, running offline was part of the design.
Our job started before the hackathon. We used the ixian Hexagon SDK to prepare six of Liquid AI’s LFM3 and d1 models for the IQ9, published the packages on Hugging Face, then joined the teams in the room to help them get running.

Six models, ready on day one
Teams could start with LFM3-1.7B for text, VL-2B for images and video, or Audio-2B for speech. Omni-2B combined those inputs in one model. The two d1 models served a different need: scoring the possible answers to a question in a single pass.
Each package comes in two versions. The default uses both Hexagon NPUs. A team already using one NPU for perception can run the model on the other.

What a robot loses while it waits
A robot has 30 seconds to finish a task. How much difference could a fraction of a second make? During his talk, ZETIC co-founder Seongjun Kim showed an experiment from Wang, Clark and Song’s Chronos Policy paper. The robot finished in time on nine of ten runs. When the researchers introduced delays ranging from zero to 200 milliseconds, it finished in time on just four.
Watch the two runs below. The experiment gave us a reason to care about the wait before a model responds. Our IQ9 tests measure that wait directly; how it affects a particular robot depends on the rest of its control loop.
Read the Chronos Policy paper.
Getting to the first token
Give LFM3-VL a photo and ask a question. Before it can produce even the first word, it has to encode the image and process the prompt. That was our largest measured gain on the IQ9. The first token arrived in 200 milliseconds, down from 710 with the fastest llama.cpp configuration we tested on the same board.
| Workload | llama.cpp | ZETIC | Speedup |
|---|---|---|---|
| VL · image + question | 710 ms | 200 ms | 3.6× |
| Omni · image + audio | 930 ms | 310 ms | 3.0× |
| Omni · image | 620 ms | 230 ms | 2.7× |
| Audio · 30-second clip | 1,430 ms | 570 ms | 2.5× |
| Text · 2K-token prompt | 1.00 s | 0.66 s | 1.5× |
Images were 512 × 512 RGB. These times end at the first output token.
Once the answer starts
A fast first token gets a conversation going. Keeping it going depends on how quickly the rest of the answer arrives. On the same IQ9, Omni streamed 62 tokens per second, up from 43. For spoken replies, it generated 2.75 seconds of audio per second of compute.
| Workload | llama.cpp | ZETIC | Speedup |
|---|---|---|---|
| LFM3-1.7B | 41 | 60 | 1.5× |
| LFM3-VL-2B | 39 | 59 | 1.5× |
| LFM3-Audio-2B | 38 | 57 | 1.5× |
| LFM3-Omni-2B | 43 | 62 | 1.4× |
| LFM3-1.7B · 64K context | 20.9 | 26.6 | 1.3× |
| Omni · speech synthesis | 1.06× real time | 2.75× real time | 2.6× |
The last column compares the two runtimes. Real time describes the duration of audio generated per second of compute.
Making two NPUs work together
Splitting a model across two processors only pays off if they spend more time computing than waiting for each other. For LFM3-1.7B, splitting each layer by matrix columns let both NPUs process the prompt at once. In our own engine, that brought the prompt step from 681 to 374 milliseconds.
For generation, each NPU computed half the output rows. Throughput rose from 46.1 to 60.5 tokens per second. At long context, the bigger issue was moving the KV cache. Keeping it on the NPU that used it brought generation at 64K context from 17.7 to 26.6 tokens per second. These are successive engineering comparisons within our runtime.

Checking the answers
We checked each package against its reference before release and repeated the IQ9 release checks on November 11. For LFM3-1.7B, perplexity stayed within 0.5% of the reference, and greedy decoding returned the same tokens on the tested prompts.
VL and Omni answered all five test image questions correctly. Audio and Omni matched the reference transcripts on four speech clips. For generated speech, the word error rate after transcription ranged from 0 to 0.02. The model cards document the inputs behind those checks.
See it run
Watch LFM3-Omni-2B handle text, vision and audio on the Dragonwing IQ9. The side-by-side recording lets you follow when each answer starts and how it streams.
How we measured
We used the Qualcomm Dragonwing IQ-9075 EVK (QCS9075) running Ubuntu 24.04 with 36 GB of LPDDR5. The tables compare the fastest llama.cpp setup we tested on its Hexagon NPUs with our two-NPU ixian Hexagon SDK 0.2.x packages. The language model weights were 4-bit and the vision and audio encoders used FP16.
The d1 packages are available, but the runtimes handle their questions differently: llama.cpp runs one question per pass, while our engine batches questions about an input. We left them out of the speed comparison. These tests measured latency and throughput; energy and inference memory use were outside their scope.
Run it on your board
Download the six IQ9 packages from Hugging Face and install the ixian Hexagon SDK (0.2.1 or later). Start with the included command-line tool, or integrate a model into your application through the C++ API.
Thanks to Viviana Márquez and the Liquid AI team, Heath Blandford and the Qualcomm team, and everyone who brought a demo to San Francisco and ran it with the network off.
