$500 RL Fine-Tune of a 9B Open Model Beat GPT-5.6 Sol and Claude Opus 4.8 on Catalog Review

Last updated: July 28, 2026

FermiSense published a detailed case study on July 27 showing that a $500 GRPO fine-tune of Qwen 3.5 9B Instruct beats every frontier model they tested on automotive catalog review. The trained specialist hit 97% accuracy, compared to 93% for GPT-5.6 Sol and 91% for Claude Opus 4.8 on the same task, at roughly 1/68th the cost per listing.

The result is not a fluke on an easy benchmark. It is a structured evaluation against five frontier models on a real production workflow: matching vehicle descriptions to the correct part across a catalog of 50,000+ parts spanning multiple automakers. And the margin is large enough that the cost-quality tradeoff curve favors the specialist at every volume.

What the benchmark actually measures#

Catalog review is a standing problem in e-commerce and supply chain: given a free-text listing description (e.g. "brake pad set, ceramic, front, fits Toyota Camry 2022-2025"), the system must find the correct entry in a product taxonomy, verify the claimed brand against a registry, pull the attribute schema for that category, and commit a structured decision. Each episode involves multiple tool calls: search taxonomy, lookup brand, get attribute schema, commit verdict.

The frontier models were tested with optimized prompts on 200 stratified validation episodes, with identical tools, images, scorer, and turn budget. The five models tested were GPT-5.5, GPT-5.6 Sol, Gemini 3.1 Pro, Claude Opus 4.8, and Claude Fable 5.

All five plateaued within a tenth of a point of each other. The best frontier configuration scored 76.9% of the maximum achievable score. The fine-tuned specialist scored 87.3% - a 13.5% relative improvement.

The training setup#

FermiSense used Qwen 3.5 9B Instruct as the base model and applied GRPO (Group Relative Policy Optimization), the same RL method DeepSeek R1 popularized. The training data was 1,000 labeled examples. The compute budget was approximately $500 in GPU time on a single 80GB GPU across roughly 2,500 RL steps.

The base model started at 64.2% accuracy on the task. After 2,500 steps it reached 97%.

The key enabler is the verifier. Catalog review has a ground-truth answer for every listing, so the reward signal is deterministic and free. GRPO eliminates the value model that PPO requires - instead of training a separate network to estimate expected rewards, it samples multiple responses per prompt and uses the group mean reward as the baseline. With a good verifier, the training stack collapses to: a policy model, a reference model for the KL leash, and a verifier function.

Cost comparison#

FermiSense published a per-1,000-listings cost comparison that makes the economic case concrete:

The specialist is 40x cheaper than the least expensive frontier option and roughly 68x cheaper than the most expensive. At scale, those ratios translate to meaningful infrastructure budget differences.

The cost advantage comes from two factors. First, inference on a 9B parameter model is dramatically cheaper than running a frontier model with hundreds of billions of active parameters. Second, the specialist makes fewer errors, which means fewer re-runs and less human review overhead.

What this means for developers#

This result is part of a growing pattern. Over the past year, we have seen GRPO fine-tuning produce specialist models that outperform frontier generalists on narrow, verifiable tasks across code generation, math, structured extraction, and now catalog review. The common thread: if you can write a verifier, you can train a better model than any API-accessible frontier for that specific task.

The practical implication is that the "open model vs frontier API" decision is increasingly a false binary. The right question is: is this a task with a verifiable output? If yes, a small open model with RL post-training will likely beat the frontier at a fraction of the cost. If the task is open-ended creative work or research without clear correctness criteria, APIs still win.

FermiSense also surveyed the broader landscape and found similar results at other companies. Ramp used RL to beat frontier models at spreadsheet search. Intercom's Fin Apex resolves more support issues at lower cost than GPT-4o. Checkr classified criminal-record entries more accurately than GPT-4 on the hardest cases. The pattern holds across domains.

How to try this yourself#

The recipe is straightforward for teams with GPU access. Start with a base model from the Qwen 3.5 or Llama 4 family at the 7B-9B size. Collect a few hundred to a few thousand labeled examples where the correct output is verifiable. Write a verifier function that scores outputs against ground truth. Use Hugging Face's TRL library with GRPOTrainer - the same tooling covered in our GRPO explainer.

A single A100 80GB can handle a 9B model with group size 8. The training time for 2,500 steps is a few hours. Total compute cost: well under $1,000 on spot instances.

For teams without GPU infrastructure, the alternative path is prompt engineering against a small local model with tool-use scaffolding. Our Qwen 3.5 local guide covers running these models on commodity hardware.

FAQ#

How much did the FermiSense RL fine-tune cost?#

About $500 in GPU time. FermiSense trained on a single 80GB GPU for roughly 2,500 GRPO steps, using 1,000 labeled examples.

Which model was fine-tuned, and with what method?#

FermiSense fine-tuned Qwen 3.5 9B Instruct with GRPO (Group Relative Policy Optimization), the RL method DeepSeek R1 popularized. GRPO drops the separate value model PPO needs and uses the group mean reward of several sampled responses as the baseline, so with a good verifier the stack is a policy model, a reference model and a verifier function.

How did the fine-tuned 9B model compare to frontier models?#

The specialist reached 97% accuracy on automotive catalog review, against 93% for GPT-5.6 Sol and 91% for Claude Opus 4.8. On the full scored evaluation, the best frontier configuration reached 76.9% of the maximum score and the specialist 87.3%, at $0.50 per 1,000 listings versus roughly $20 to $34 for frontier models.

When does an RL fine-tune beat a frontier API?#

When the task has a verifiable output. If you can write a verifier that scores answers against ground truth, a small open model with RL post-training will likely beat the frontier at a fraction of the cost. For open-ended creative work or research without clear correctness criteria, APIs still win.

Sources#

• FermiSense case study: When Machines Take the Wheel (July 27, 2026)
• HN discussion (250 points): item 49078454
• Qwen 3.5 9B Instruct on Hugging Face: huggingface.co/Qwen
• TRL GRPOTrainer documentation: huggingface.co/docs/trl/grpo_trainer

Continue Reading#

• DeepSeek R1, PPO, and GRPO Explained for Devs - the mechanic: how GRPO works under the hood, why it removes the value model, and a minimal training script using TRL
• Qwen 3.6 27B: Dense Coder Evaluation - how the Qwen 3.5/3.6 family performs on code generation, structured reasoning, and tool calling
• The AI Affordability Crisis: When Agent Costs Scale Faster Than Value - the economic side of the cost-per-task equation and why specialist models change the calculus
• Agent Fleet Economics: Fable 5 vs Sonnet 5 at Scale - cost modeling for production agent deployments, including the crossover point where smaller fine-tuned models win
• OpenAI Is Winding Down Fine-Tuning: Where to Move Your Fine-Tunes - which hosted APIs still run RL and LoRA fine-tuning, and which let you download the weights
• Local Qwen is a Different Tool, Not a Worse Opus - why measuring open models against APIs on general benchmarks misses what they are good at