#Image capturing

119 messages Β· Page 1 of 1 (latest)

past bramble
#

I would like to have a button called "capture image" on my frontend (react) that allows the user to capture any portion of their screen and send the image data to my program.

I am aiming to have a flow like this:

  1. User clicks capture
  2. User selects portion of screen
  3. Preview is displayed on frontend
  4. User clicks "submit"
  5. Backend receives image

Is this (steps 2 & 3) possible with Tauri? If so, how?
Also, I would want to send that image data to the backend, how would I send that? (step 5)

rare meadow
#

All of it is possible, but you'll pretty much guaranteed have to do nearly all of it in the Rust end and use OS specific crates to accomplish it

past bramble
rare meadow
#

Yes, that's the only way I can think of doing it. And you cant use the regular IPC for it because it's too slow, so you gotta look at data streaming

past bramble
rare meadow
#

Yes

past bramble
#

dang

#

can't i just tell the user to wait? πŸ˜‚

#

seems like this is getting more complex than i thought

rare meadow
#

Events and commands use JSON to communicate and the current form is too slow. Version 2 will be faster

You can definitely do that

past bramble
#

do you have an estimate for the amount of time it would take? i am guessing users are taking up to 1000x1000 pixel images at max

rare meadow
#

Anything outside of your own window is always gonna require using something native and is always a bit tricky. Sometimes things may also fail because windows may be protected from screenshotting (hence why OBS has a special mode of capturing video to bypass such protection)

#

I have gotten 1mbps transfer rate, but that was a pretty extreme scenario

past bramble
#

daang

#

oh well

#

i might just resort to users uploading their own images πŸ˜‚

#

alright, i'll see what i can do first though

past bramble
rare meadow
past bramble
#

I think i'll just go with the regular ipc for now, if its too slow then i'll change

#

thank you

#

okay, now just time to find a window selection tool

past bramble
rare meadow
#

Yea it wouldn't surprise me if you're gonna need to develop this yourself from scratch

#

Even using the official Windows crate https://crates.io/crates/windows is going to require you to know a bit about how the C++ version functions because it's basically just bindings to those API's

#

And then you gotta do that process once per platform you wanna support

#

So you know, glhf πŸ˜…

past bramble
#

and luckily i have a windows laptop...

past bramble
#

one issue: i have no idea where to start.....

rare meadow
#

That should give you an idea of what to do. "Just" translate it to Rust

past bramble
#

"just" is a tad bit of an understatement πŸ˜‚

past bramble
past bramble
#

Okay. this is pretty dang complex πŸ™ƒ

past bramble
#

gotta find bindings that actually work and include all the things i need

past bramble
#

woohoo! fun!

thread '<unnamed>' panicked at 'Uncaught exception <NSException: 0x142a0a1e0>', /Users/alan/.cargo/registry/src/github.com-1ecc6299db9ec823/cacao-0.3.2/src/appkit/window/mod.rs:80:30

no trace, no actual error 😦

#

Anyway, am I allowed to make another NSApplication on the same thread as tauri?

median lagoon
#

highly unlikely

past bramble
#

alrighty

past bramble
#

at this point im thinking of just making a seperate executable so i dont have to deal with these threading issues πŸ˜‚

median lagoon
#

the fun stuff is that you'll likely keep hitting the same problems kkushLUL

#

at least that's my experience so far

past bramble
median lagoon
#

is that

past bramble
#

oh, i see

past bramble
#

aw man, this is so much fun! lol

median lagoon
#

yeah every time i have to do something for macos i consider quitting my job kkushKEKW

#

(same for linux so that probably doesn't mean much)

past bramble
#

well, there some great documentation and its not like i cant find any prior stackoverflow questions of the binding library im using for appkit πŸ˜‚

#

i feel lucky to even have some old objc code to refer to

median lagoon
#

would it be easier to just have a maximized transparent window where you draw the selection stuff with idk a canvas?

past bramble
#

i made my maximized transparent window, actually

#

i dont know if you can see it

#

comparison ->

image.png
#

now i just need to add the drag selection thing

median lagoon
#

and what do you need the objc bindings for here?

What i'm thinking of is something like:

  1. maximized transparent window
  2. in your frontend, the user selects something for ex something like this https://jsfiddle.net/xkmqz9ho/
  3. get the dimensions/coords from the canvas
  4. use some rust crate like https://crates.io/crates/screenshots to get the image data of that area
  5. profit
past bramble
#

not specific to inside the app

#

oh wait you can just do that

median lagoon
#

which is why you have the maximized window on top of the whole screen

past bramble
#

oh, i see what you mean

#

im just writing this with objc bindings, i guess

#

didn't think of that

#

wait, can you configure the window to be transparent with some opacity in tauri?

median lagoon
#

good question

#

maybe the standard css stuff works

past bramble
#

i'll give it a shot

#

the whole reason why i went with this aproach is i already had code for making that kind of window

past bramble
median lagoon
#

you can make the window transparent via configs

#

but tauri can only make the window fully transparent

#

so the opacity stuff would be via css

#

btw, transparent windows prevents your app from being accepted in the macos store harold

past bramble
#

i dont have the money to put it there anyway

#

lol

median lagoon
#

fair enough

past bramble
#

oh i forgot to enable macOSPrivateApi

#

i see

#

this is a good idea

#

thank you

past bramble
# median lagoon you can make the window transparent via configs

Okay. On a completely different note, how should I dynamically create another window with the capture page?
I have a command called area_capture that should spawn this window. Currently, it looks like this:```rs
#[tauri::command]
fn area_capture(handle: tauri::AppHandle) -> (i32, i32, i32, i32) {
let docs_window = tauri::WindowBuilder::new(
&handle,
"Capture Screen",
tauri::WindowUrl::App("index.html".into()),
)
.build()
.unwrap();
docs_window.show().unwrap();
(0, 0, 0, 0)
}

However, I'm not sure where I should put the `index.html` file nor do I know where to configure the window size and transparency. How should I do those things?
Thank you
#

might as well make another thread for that window thing tbh

#

its a diferent question

median lagoon
#

we can keep it here too i don't care

past bramble
#

aight Β―_(ツ)_/Β―

#

anyway, where do i put my index.html?

median lagoon
#

depends a bit on your frontend setup

#

the generalized answer is: in your devPath and distDir

#

and btw if you didn't figure it out yet, for a transparent window just add .transparent(true) before .build()

past bramble
past bramble
median lagoon
past bramble
#

cra

median lagoon
#

unless ejected i think the easiest solution would be to use react-router actually πŸ€”

past bramble
#

πŸ‘

#

wait then would would be my path

median lagoon
#

if you use the hashbrowser it'd be something like index.html/#/area-route or maybe just /#/area-route idk

past bramble
#

yay! thank you!

image.png
past bramble
#

can i make the window the same size as the screen with some option, or would I have to fetch the screen size

#

and, i dont think you can hide the tile bar, correct?

median lagoon
#

doesn't maximize work for you?

#

or actually fullscreen?

past bramble
#

wait maximize oh right

median lagoon
#

the latter should also hide the titlebar right?

past bramble
#

let me give fullscreen a try - i believe it just goes to a new desktop though

#

yeah fullscreen goes to a new desktop

median lagoon
#

unfortunate

past bramble
#

and gives me some nice trace πŸ˜„

image.png
#

semi-transparent πŸ˜‚

image.png
past bramble
#

At least if I make this it will be cross platform! Thanks a ton.

past bramble
rare meadow
#

Make the window undecorated to get rid of the menubar

If you have the display size then translating the coordinates in the window to coordinates on the desktop should be a piece of cake

median lagoon
#

yep ☝️ basically just add the window coords on top