🔒 Extracting mixed data from a PDF

[surreal zealot]

I'm trying to make a study resource from a bunch of PDFs with math questions. I want to extract all the questions from these PDFs into one big .json file, where each question will have its ID number (which is already in the PDF) and the question text. a lot of the questions have images to go with them and/or have math equations that are just images in the PDF and not actual text. How can I easily extract these questions with mixed text and images?

[patent shoal]

@surreal zealot

Python help channel opened

Remember to:

• Ask your Python question, not if you can ask or if there's an expert who can help.
• Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
• Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.

[wanton dragon]

There are a few libraries that do text extraction but pymupdf is usually regarded as one of the best for text extraction. You can read the PDF document using it then extract the text using Page.get_text which supports outputting to a few formats such as text or HTML.

[surreal zealot]

will that work with questions that have images, too?

[wanton dragon]

If the question text is itself in the image, nope, you'd need OCR to be able to extract that.
Otherwise, if it's just the image itself, it should be able to extract it.

[surreal zealot]

does pymupdf have OCR?

[wanton dragon]

It does support it, though it requires Tesseract for it to work.

[sweet cave]

If the number of total PDF pages isn't very large, you can try the image capabilities of the free tier of gpt-4o (provide the PDF pages as images and ask it to transcribe the questions as, say, LaTeX). Any other way of doing OCR on equations is probably going to be hellish.

[patent shoal]

Python help channel closed for inactivity

This help channel has been closed. Feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.