I Tested Three Vision Models on Catalog Images: OCR Was the Easy Part

Converting a visual product catalog into structured, queryable data sounds simple on paper:

1. Send the page image to an OCR or Vision-Language Model (VLM).
2. Extract product codes and prices into JSON.
3. Ingest the rows into a production database.

Our hands-on benchmark with Mistral OCR, DeepSeek V4 Flash Vision Exp, and Qwen3-VL-32B-Instruct proved why that mental model breaks down in real life.

The hard problem was never character recognition. The real bottleneck was relationship preservation: linking the right 5-digit SKU to the correct variant, distinguishing campaign offers from list prices, and resolving multi-product shared-price blocks without human intervention.

Here is what we learned building an extraction pipeline for 138-page PDF catalogs, why raw OCR metrics are misleading, and how we architected a human-in-the-loop workflow that prevents business incidents.

---

1. Input Constraints: You Can't Upscale Missing Information

Our catalog consisted of 138 image-only pages, each embedded as a full-page JPEG capped at 150 DPI.

Our first architectural decision was straightforward: rendering the PDF at 300+ DPI was a waste of compute. Upscaling low-resolution assets doesn't create new signal; it just burns CPU cycles and expands token payloads.

Instead, our pipeline extracts the raw JPEG byte stream directly and feeds the original images to model endpoints.

The real challenge lay in the layout patterns:

• Vertical product codes placed over low-contrast photography.
• Campaign prices and list prices printed side by side.
• Shared price blocks applying to multiple product codes.
• Editorial pages with zero products that still trigger false positives.

If a pipeline transcribes every digit correctly but pairs a campaign price with the wrong SKU, it has generated corrupt catalog data.

---

2. The Benchmark: 3 Models on Complex Layouts

We evaluated three candidates across the same catalog fixtures:

• Mistral OCR (mistral-ocr-latest): Document layout and text extraction.
• DeepSeek V4 Flash Vision Exp: Vision extraction via OpenRouter ($0.00269/page actual).
• Qwen3-VL-32B-Instruct: Structured vision extraction via OpenRouter ($0.00024/page actual).

All requests ran with deterministic settings (temperature=0). We captured raw payloads, latency, normalized JSON schemas, and per-page cost.

The Baseline Test: Page 94 (Shared Price Fixture)

We tested all three candidates against a tricky baseline: Page 94, featuring three SKUs (41367, 41368, 41369) sharing a single campaign price (C$339.00) and list price (C$570.00).

*Note on Qwen3-VL below: the 0% score was caused by an evaluator schema mismatch, not an inference failure.

The takeaway: Every model caught the campaign price. But isolating rotated product codes and mapping them back to a shared parent price failed across automated runs. DeepSeek hallucinated nearby numbers (41347, 41348, 41349), while Mistral misclassified the list price as a second campaign price.

---

3. The Evaluator Is Part of the System Under Test

Our biggest engineering takeaway wasn't model accuracy—it was an evaluator contract bug.

During manual verification, Qwen3-VL had successfully identified product codes and prices. However, our automated test harness reported a 0% match rate because the scorer expected a JSON field named code, while Qwen returned product_code.

A single key mismatch turned a working extraction into a synthetic 0% metric on the dashboard.

Rule of thumb: In LLM/VLM evaluation pipelines, parser contracts are production code:

• Standardize on a single canonical schema across all model adapters.
• Preserve raw API payloads to debug evaluator bugs vs. model hallucinations.
• Score entity relationships (Price ↔ SKU mapping), not just raw token overlap.

---

4. Operational Guardrails: Assistance Over Full Automation

Because a wrong price or mismatched SKU causes actual financial errors downstream, we rejected a 100% zero-touch ingestion pipeline. Instead, we established operating confidence bands:

• ≥ 98% Exact Match: Eligible for automated ingestion.
• 85% – 97% Match: Fast-track human review (side-by-side verification).
• < 85% Match: Defer to manual entry if correction overhead exceeds drafting from scratch.

We selected Qwen3-VL-32B-Instruct for our pipeline due to its exceptional cost-to-performance ratio ($0.00024/page) and clean structured output, backing it with a human-in-the-loop review interface:

1. VLM extracts structured data into staging tables.
2. Reviewers inspect extracted fields side-by-side with the original catalog page.
3. Automated validation runs math sanity checks (e.g., verifying (List Price - Campaign Price) / List Price matches the printed badge discount within 1%).
4. Only human-approved rows are promoted to the active application database.

---

Key Takeaways for Engineers

1. OCR detects tokens; Vision extracts relationships. If your system cannot map a discount to its target SKU, high OCR precision is meaningless.
2. Respect the source ceiling. Don't waste latency upscaling low-DPI catalog scans—extract raw JPEG bytes directly.
3. Discount reconciliation is your best validation test. Cross-referencing percentage badges with arithmetic catches price swaps before human review.
4. Human-in-the-loop is a product feature, not technical debt. Building efficient side-by-side audit tools turns an imperfect 90% model into a 10x multiplier for operations.