Top AI Product

Every RAG pipeline chokes on the same question: is this PDF real text or a scan? Guess wrong and you either burn GPU cycles OCR-ing plain text or feed garbage to your LLM. pdf-inspector, Firecrawl’s open-source Rust library, answers it in 10-50 milliseconds. It sits at 8.6k stars, +1,699 today — the fastest-growing AI repo on GitHub trending.

What it actually does

It classifies a PDF as text-based, scanned, image-based, or mixed by reading internal structure — font encodings, text operators, image coverage. No rendering, no ML models, single dependency. Then it extracts position-aware text and clean Markdown, handling tables and multi-column layouts. On a 200-PDF benchmark it processed the entire corpus in 0.47 seconds and topped competing engines on reading order.

This is the engine inside Fire-PDF, Firecrawl’s parsing pipeline: text pages skip the GPU entirely, only scans hit OCR. Result: 3.5-5.7x faster parsing, at a fraction of the cost.

Drop it into your pipeline

It’s a library, not a service — pip, npm, cargo, even browser WASM. Typical flow: classify, extract text-based PDFs locally in ~150ms, route scans to OCR. MIT licensed.

After web crawling, this is Firecrawl’s second weapon in the feed-data-to-LLMs stack. The upstream of every agent is getting commoditized — fast.

You Might Also Like

• Pageindex Just hit Github Trending and it Might Make you Rethink rag Entirely
• Heretic Just hit Github Trending and the ai World has Opinions
• Pentagi Just hit 1 on Github Trending and Yeah its Worth the Hype
• Openfang Just Dropped and its Already the Hottest Agent os on Github
• Shuo sub 500ms Voice Agent 600 Lines of Python That Make Voice ai Feel Instant