Ask HN: What is the best method for turning a scanned book as a PDF into text?

201

107

I like reading philosophy, particularly from the authors rather than a secondhand account.

However I often run into that these come as scanned documents, Discourses on Livy and Politics Among Nations for example.

I would greatly benefit from turning these into text. I can snipping tool pages and put them in ChatGPT and it turns out perfect. If I used classic methods, it often screws up words. My final goal is to turn these into audiobooks, (or even just make it easier to copypaste for my personal notes)

Given the state of AI, I'm wondering what my options are. I don't mind paying.

https://products.aspose.ai/pdf/searchable

then

https://products.aspose.app/pdf/conversion/pdf-to-txt

Makes the layout of the space for the text

23 hours agoby epirogov

I did this very recently for a 19th century book in German with occasionally some Greek. The method that produces the highest level of accuracy I've found is to use ImageMagick to extract each page as a image, then send each image file to Claude Sonnet (encoded as base64) with a simple user prompt like "Transcribe the complete text from this image verbatim with no additional commentary or explanations". The whole thing is completed in under an hour & the result is near perfect and certainly much better than from standard OCR softwares.

4 days agoby aragonite

I made a high-quality scan of PAIP (Paradigms of Artificial Intelligence Programming), and worked on OCR'ing and incorporating that into an admittedly imperfect git repo of Markdown files. I used Scantailor to deskew and do other adjustments before applying Tesseract, via OCRmyPDF. I wrote notes for some of my process over at https://github.com/norvig/paip-lisp/releases/tag/v1.2 .

I'd also tried ocrit, which uses Apple's Vision framework for OCR, with some success - https://github.com/insidegui/ocrit

It's an ongoing, iterative process. I'll watch this thread with interest.

Some recent threads that might be helpful:

* https://news.ycombinator.com/item?id=42443022 - Show HN: Adventures in OCR

* https://news.ycombinator.com/item?id=43045801 - Benchmarking vision-language models on OCR in dynamic video environments - driscoll42 posted some stats from research

* https://news.ycombinator.com/item?id=43043671 - OCR4all

(Meaning, I have these browser tabs open, I haven't fully digested them yet)

5 days agoby pronoiac

Copyright issues aside (e.g. if your thing is public domain), the galaxy-brain approach is to upload your raw scanned PDF to the Internet Archive (archive.org), fill in the appropriate metadata, wait about 24 hours for their post-upload format-conversion tasks to run automatically, and then download the size-optimized and OCR-ized PDF from them.

I've done this with a few documents from the French and Spanish national archives, which were originally provided as enormous non-OCRed PDFs but shrank to 10% the size (or less) after passage through archive.org and incidentally became full-text-searchable.

4 days agoby quuxplusone

I have tried a bunch of things. This is what worked best for me: Surya [0]. It can run fully local on your laptop. I also tried EasyOCR [1], which is also quite good. I haven't tried this myself, but I will look at Paddle [2] if the previous two don't float your boat.

All of these are OSS, and you don't need to pay a dime to anyone.

[0]: https://github.com/VikParuchuri/surya

[1]: https://github.com/JaidedAI/EasyOCR

[2]: https://github.com/PaddlePaddle/Paddle

4 days agoby __rito__

For classic books like those you mentioned, Project Gutenberg has text versions along with pdfs/epubs/etc.

For instance, Discourses on Livy:

https://www.gutenberg.org/cache/epub/10827/pg10827-images.ht...

https://www.gutenberg.org/ebooks/10827

4 days agoby Electricniko

My understanding is that Gemini OCR is now considered state of the art and a material step forward in OCR accuracy

5 days agoby lquist

I have not seen this answer so I’ll chime in:

There is a lot of enthusiasm around language models for OCR and I have found that generally they work well, however I have had much better results, especially if there are tables etc., by sending the raw page to the llm along with the ocrd page, and asking it transcribe from the image and validate words/character sequences against the ocr.

This largely solves for numbers and things being jumbled or hallucinated.

I recently tested llamaparse after trying it a year prior and was very impressed. You may be able to do your project on the free tier, and it will do a lot of this for you.

4 days agoby 7thpower

We do this in our Text to speech app (Read4Me): https://apps.apple.com/us/app/read4me-talk-browser-pdf-doc/i...

You can scan a book and listen (also copy and paste the text extracted to other apps).

If you are looking to do this on large scale in your own UI, I would recommend either of Google solutions:

1. Google Cloud Vision API (https://cloud.google.com/vision?hl=en)

2. Using Gemini API OCR capabilities.(Start here: https://aistudio.google.com/prompts/new_chat)

5 days agoby kumarm

I haven’t seen anyone else mention this tool yet, but I’ve found great accuracy and flexibility with [OCRmyPDF](https://github.com/ocrmypdf/OCRmyPDF). It (usually) detects and fixes page rotation, and works quite well on slanted text or A | B pages in regards to copying and formatting. I believe it uses tesseract in the background, but using it is very simple and it has the just works factor.

4 days agoby The_SamminAter

It’s surprising to me that no one has mentioned llamaparse. My team has been using it for a while and is quite satisfied. If other people think that other services are better then I’d be interested in hearing why.

4 days agoby arctangos

hey, i recommend checking out the previous HN threads [1] on why LLMs shouldn’t be used in production-grade OCR, especially if accuracy is super important (as in the audiobook case)

we wrote the post and created Pulse [2] for these exact use cases, feel free to reach out for more info!

[1]: https://news.ycombinator.com/item?id=42966958 [2]: https://runpulse.com

4 days agoby ritvikpandey21

For years I have been printing PDFs off on regular paper and then binding them into books. 1. Print it at work when no one is looking. 2. Get two rigid boards and squeeze the stack of paper together. I customarily use two wooden armrests that originally came from a garden-furniture lounger. 3. Squeeze the paper with just a 1/4-inch showing. 4. Use wood glue and with your finger working like a toothbrush, work the glue into the pages at the gluing end. 5. Get a 14-inch X 4-inch strip of canvas. I use cutoff painter's canvas. 6. Hang all this by the boards and put glue also on top of the canvas strip. 7. When it dries, remove the boards and glue down the sides. You have a strong, bound book out of those printed pages.

5 days agoby tyronehed

Docling is great for PDFs https://github.com/DS4SD/docling but if the input is really only images (in PDF) than cloud AI based solutions (like latest models from Google) may be better.

5 days agoby therealmarv

I made a site recently that works pretty well for this for a lot of sample scanned PDFs I tried, you might get good results:

https://fixmydocuments.com/

I also made a simple iOS app that basically just uses the built in OCR functionality on iPhones and automatically applies it to all the pages of a PDF. It won’t preserve formatting, but it’s quite accurate in terms of OCR:

https://apps.apple.com/us/app/super-pdf-ocr/id6479674248

4 days agoby eigenvalue

Wow I asked a similar question a few days, glad to see this is getting some traction! Archive.org OCR is new to me! Very interesting. I am working on a tool to do OCR, translate and layout recovery (like .. all the apps nowadays), but focussed on running locally/processing thousands of pages/scans. Let me know if you want to collab!

3 days agoby thenthenthen

More of a comment on your goal and something I’ve been thinking about with non-fiction books recently.

I’ve begun thinking I dislike the rigid format of non -fiction books. Few ideas bulked out and iterated, with lots of context and examples. Takes me ages to get through and I have very little free time. Cliffs notes are awful because you need some iteration and emotional connection to make the content stick.

I’d love a version of a book that has variable rates of summarization and is navigable around points and themes, so I can hop about while ensuring I don’t miss a key fact or insight buried somewhere

3 days agoby navaed01

There was a thread here recently about OCR4All ( I haven't used any of these tools recently, but I'm keeping track because I might be doing that soon ).

https://news.ycombinator.com/item?id=43043671

https://www.ocr4all.org/

4 days agoby RobGR

I recently used AWS Textract and had good results. There are accuracy benchmarks out there, I wish I saved the links, but I recall Gemini 2.0 and Textract towards the top in terms of accuracy. I also read that an LLM could extrapolate/conjure up cropped text therefore my idea would be to combine traditional OcR with LLM to determine conflicts.

4 days agoby jonnycoder

Google Cloud Document AI is amazing, I love it https://cloud.google.com/document-ai?hl=en

It can correctly read many other languages than English if that is something you need. Previously I tried others and there were many errors in conversion. This does it well.

5 days agoby srameshc

I had to scan some historic books and papers for a research project and tried a couple desktop apps and python libraries. Still the Android scan to document worked better, so I just photographed all pages which was faster, then wrote an unsupervised python script to loop through the images in the camera folder and send them to Google Document AI for OCR, added some metadata and merged them. This outperformed the other even commercial solutions I tried by far and was dirt cheap. If you're interested I'll clean up the code and put it on github.

4 days agoby mreiner

The article below compares 5 tools for a slightly different but close enough use case (more complex pdf than just running text). It concludes:

"Surprisingly, ChatGPT-4o gave the best Markdown output overall. Asking a multimodal LLM to simply convert a document to Markdown might be the best option if slow processing speed and token cost are not a problem."

https://ai.gopubby.com/benchmarking-pdf-to-markdown-document...

4 days agoby PeterStuer

I was actually just working on a project like this to digitize an old manuscript. I used a PDF scanning app (there are plenty, I used Naps32, simple but it works). And then I piped the images into a `tesseract-ocr`. This will extract the text from the image but it won't deal with formatting or obvious typos. For that you're going to want to feed the text into an LLM with some prompt telling the model to correct errors, fix formatting, and provide clean text. Smaller local models (<70b parameters) do not work very well on this task for big documents, but I found ChatGPT's reasoning model does a fine job. My goal is to find a model that can run locally with similar performance.

4 days agoby briga

Recommend the ABBY Finereader PDF app (their OCR engine is used by lots of other providers) - fantastic scanning and export to EPUB or other formats - used for ~300 different book across multitudes of languages for translation using AI.

4 days agoby czcar

The smart LLM's are great at this (Gemini Flash seems accurate and cheap), but they can't be trusted not to engage in unexpected censorship, typically skipping parts they find objectionable without reliably telling you that that's what they did. That's annoying enough if you're dealing with, e.g., names that happen to spell something awkward, but it's a big problem if you're scanning medical notes or something else where the awkward text is legitimately needed.

Anyone have success with prompting them to "just give me the text verbatim?"

4 days agoby waldrews

I've found there's a big difference in OCR accuracy where it comes to handwriting. For printed text, I've used tesseract, but it seems to miss a lot for handwriting. In my experience, google cloud vision is far more accurate at transcribing handwriting. Haven't tried other cloud based tools, so I couldn't tell you if it's better, but I would say that overall, the cloud based ones seem to be much better at handwriting or oddly formed text, but that for basic typeset printed text, open source apps like tesseract do well.

4 days agoby geebee

This might be of interest: https://www.expectedparrot.com/content/johnjhorton/grant-let... It uses a collection of models to extract the text from a handwritten letter by US Grant. Probably overkill for something nicely printed.