Extract text using X and Y coordinates

[kind wadi]

Hello everyone, sorry for bothering you.
I'm having a problem, please I need some help.
I'm using a library called ng2-pdf-viewer in order to view my pdf documents within my web application.
My requirement is to upload a PDF file and extract text from specific areas selected by the user.
Looking for the internet I found a way to draw a rectangle and store all the coordinates inside it, but these rectangles are only visual as they don't interact with the PDF.
So my question is:
Is there a way to extract the text inside these rectangles?
Since I have saved the X and Y coordinates of these rectangles (Second image).

Please help

[twilit orchid]

Don't know that library, but a quick glance at its README gave me this

[opaque hollow]

my gut feeling thinks this is not possible. As far as I know there are no metadata that tells where the text starts and ends based on coordinates

[rigid tusk]

Also keep in mind that some pdfs are just images.

[kind wadi]

Thanks but I already use that method and doesn't work as I expected

[kind wadi]

Thanks.
Yesterday I was looking for a way to do it but it's as you say, there is no metadata inside the coordinate.
One way that i'm trying to do is to take a screenshot inside the rectangle and pass that screenshot to some library which can extract the text from the image.

it's a bit tricky but I think it will work

[opaque hollow]

usually those PDF renderer generates a canvas. Then you can crop your canvas and send it to an OCR library

[azure geyser]

If the user is selecting text maybe use getSelection().toString()?

[kind wadi]

That's exactly what I will try to do

I found a library called tesseract.js that works as OCR

[kind wadi]

Yes you can.
But the thing is
I need a way to save those highlights and print them again when the user reloads the document.

Something like this
https://kbatyrbayev.github.io/ng-pdf-highlighter/

[kind wadi]

In fact this way works pretty well but I can't extract text inside the rectangles, that's my main problem

[azure geyser]

you can save the text selections and highlight them the next time the document loads, assuming the document hasn't changed.

[kind wadi]

How can I highlight them again?
ng2-pdf-viewer doesn't have any method to highlight text (as far as I know).

[azure geyser]

you would need to serialize the selection range, save it in localstorage or database, then deserialize and restore the selection when the page is reloaded

You might have luck with this lib https://github.com/timdown/rangy/wiki/Serializer-Module#serializer-module

[kind wadi]

I didn't know about this technique, I'm going to take a look at it.
One question, can you deserialize multiple highlights? Or is it limited to only one serialization at a time

[azure geyser]

you can have as many text selections as you want

keep in mind that this only selects text, if the user clicks those selections go away, so you probably want to do some DOM transform to wrap selected text in a <mark> tag or something

[kind wadi]

The problem is
ng2-pdf-viewer wrap all the text in the following way:

<span>This</span><span>is</span><span>my</span><span>PDF</span><span>document</span>

[azure geyser]

that's fine, selections can span multiple elements

[kind wadi]

Sorry for being so redundant
But how can I position those selection in the HTML in order to wrap them?

[azure geyser]

when you use the Selection/Range API you can extract the contents of the selection, iterate over them, and transform the dom

here's an example https://codepen.io/brave3d/pen/dyyVwKo

[kind wadi]

Ohh I see
This is exactly what I'm looking for

I will do some tests and read the documentation of this range library
Thank you, your information helps me a lot