Ternlight: A 7 MB Embedding Model That Runs Entirely in the Browser

What Ternlight Is

Ternlight is a hobby project that answers a specific question: can you ship a useful embedding model in a web browser without external API calls?

The answer appears to be yes. The author distilled a sentence encoder from MiniLM using ternary quantization-aware training, wrote a Rust inference engine from scratch, compiled it to WASM with SIMD support, and packaged the result as an npm module. Text goes in, a 384-dimensional vector comes out, and cosine similarity between vectors tells you how semantically related two texts are - regardless of shared keywords.

The numbers:

• 7 MB for the base model (@ternlight/base), 5 MB for the mini variant (@ternlight/mini)
• ~5ms per embedding on the base model, ~2.5ms on mini
• 0.84 Spearman fidelity to the MiniLM teacher model
• Entirely client-side after initial load - no network traffic, no API keys, no per-request costs

The demo indexes 2,000 React docs pages and runs semantic search-as-you-type against them locally in the browser.

Last verified: July 7, 2026.

How It Works

Ternlight's size comes from ternary quantization - representing weights as {-1, 0, +1} instead of full floating point. This is not post-training quantization where you take a trained model and compress it. The entire distillation process is quantization-aware from the start, so the ternary weights are learned rather than fitted after the fact.

The author explains in the HN thread:

"It's entirely the QAT. The whole distillation process is quantization-aware from the start, so the ternary weights are learned rather than fitted after the fact. The only post-training quantization I applied was int4 on the embedding layer, and I ran a small ablation there to find the sweet spot between size and quality."

The inference engine is Rust compiled to WASM SIMD, which is why it runs at millisecond latencies on modern browsers. After the initial model load (which can be cached), there is no network dependency - the entire embedding computation happens on the client CPU.

What HN is Saying

The discussion (260+ points, 57 comments) is mostly enthusiastic, with a few practical concerns.

On use cases: Developers are already finding applications. One commenter reports: "We've just used it to embed the entire Django doc + our private knowledge base, allowing us to search in the 2 sources instantly!" Another is exploring semantic search over OpenStreetMap tags - "Do you think your work could help us let users type 'pancake' and get 'crepe' without writing an explicit dictionary entry?"

On performance variability: One user reports only 35 embeddings/second on an i5-4570 in Firefox, versus the claimed 400/second. Browser and hardware matter. The author notes testing was done on Apple Silicon and that there are known issues on some configurations.

On quality benchmarks: Commenters are asking for more comparative benchmarks. The author notes that MiniLM (Ternlight's teacher) scores around 56 on MTEB average, while gte-small scores around 61. Head-to-head comparisons are on the roadmap, as is distilling from gte-small as teacher for better quality.

On the fan noise: Multiple commenters mention that the initial embedding phase (indexing the document corpus) spins up the CPU enough to start fans. One suggests adding a button to trigger the demo rather than auto-running on page load. This is a real consideration for UX - the model runs entirely client-side, which means the client pays the compute cost.

On standardization: A commenter points to Chrome's built-in LLM API as a potential future standard: "What we need is a W3C LLM API like the one Chrome already offers." Browser-native AI primitives could eventually subsume tools like Ternlight, but we are not there yet.

Where Browser-Side Embeddings Make Sense

Ternlight is not competing with OpenAI's ada-002 or Cohere's embed-v3 on quality. It is competing on deployment model. The tradeoffs favor browser-side embeddings when:

1. Privacy is non-negotiable. If user queries cannot leave the device - legal docs, medical records, personal notes - client-side embedding eliminates the data leak surface entirely.

2. Latency matters more than quality. Search-as-you-type UX requires sub-50ms round trips. Even fast APIs add network latency that client-side inference does not.

3. Offline is a requirement. After the initial 7 MB download (which caches), Ternlight works with no network connection. Progressive web apps, field tools, and airplane-mode scenarios all benefit.

4. Per-request cost is a problem. Embedding APIs charge per token or per request. Client-side inference has a fixed cost (the download) and zero marginal cost per query. For high-volume internal tools or consumer apps with many users, this inverts the economics.

5. You control the corpus and can pre-embed. The 30-second embedding time for the React docs demo is a one-time cost. If you can pre-embed your documents server-side and ship the vectors to the client, users only pay query latency, not indexing time.

The flip side: if you need multilingual support, high-quality cross-lingual retrieval, or the best possible MTEB scores, you probably want a larger model served from an API. Ternlight's 0.84 fidelity to MiniLM is good for a 7 MB model, but MiniLM itself is not frontier quality.

Technical Integration

Installation is straightforward:

npm install @ternlight/base
# or
npm install @ternlight/mini

The API is minimal:

import { embed, similar } from '@ternlight/base';

// Generate embedding for a query
const queryVector = await embed("how do I reset my password");

// Find similar documents from pre-computed embeddings
const results = similar(queryVector, documentVectors, { topK: 5 });

For production use, the author recommends pre-computing document embeddings server-side and shipping them to the client, so users only pay query embedding latency. The GitHub repo includes the full training pipeline under MIT license.

Why This Matters

The broader trend is AI inference moving to the edge. Ternlight is a proof point for embeddings: a 7 MB model that runs useful semantic search entirely in the browser, with no API dependencies, at millisecond latencies.

This does not replace server-side embedding pipelines for most production systems. But it opens a category of applications where the deployment model - not the model quality - is the primary constraint. Privacy-first search, offline-capable apps, and high-volume consumer tools all fit the pattern.

The interesting question is whether ternary quantization-aware training can scale to larger models and more capable tasks. If the quality-per-byte curve keeps improving, browser-side AI becomes viable for more than just embeddings.

Sources

• Ternlight Demo
• Ternlight GitHub Repository
• Hacker News Discussion
• Chrome Built-in AI API

FAQ

How does Ternlight compare to transformers.js?

Ternlight is a single-purpose embedding model optimized for size and speed. Transformers.js is a general framework for running multiple model types in the browser. Ternlight is smaller and faster for embeddings specifically, but transformers.js offers more flexibility if you need multiple model types.

Can Ternlight handle languages other than English?

The current model is trained primarily on English text. The author notes that multilingual support is not a current strength. For cross-lingual search, you would need a multilingual teacher model, which is on the roadmap.

Is the 5ms latency realistic for my hardware?

The benchmarks were measured on Apple Silicon. Older Intel CPUs and some browser configurations show significantly worse performance. Test on your target hardware before committing to the architecture.

Can I pre-compute embeddings on the server and ship them to the client?

Yes - this is the recommended approach for production. Run indexing server-side once, ship the vectors to the client, and users only pay query embedding latency. The model runs identically in Node and browsers.