#The Built-in AI APIs are not supported across all browsers, plaforms and devices...

1 messages · Page 1 of 1 (latest)

pearl anchor
#

This is a tough one, and I think it really depends upon the use case of the application. For instance, some are going to inherently be more privacy focused and thus it may be "On device or nothing", but others that offer an augmentation or a way of simply offloading the computation the fallback would be nice (a progressive fallback maybe, though Im not sure how likely it would be that normal people would already have a local llm to fall back to, so apis make the most sense here)

#

I would say another worry on my side would be prompt adherence. A prompt that works well in one model is not guaranteed to work well in another, so if you did have a fallback option there might also need to be fallback prompts as well.

#

In a related note, when I was making my submission I hit an issue that actually got me started on thinking about local compute vs when it is best to offload it.

Essentially, due to the way my submission works, if it became popular it is extremely likely if not guaranteed that many of the same website elements will be processed by many different people. This is... to say the lease, not efficient.

So I had built a way to tokenize elements on webpages with a unique checksum that would be added to when new elements on websites were detected, and new elements would be processed by a (hopefully) single client. (The rest of the remaining elements could be pulled from the api).

Client loads website, elements detected and given tokens/hashs/checksums for their contents, then those are sent over to my api for processing to determine which is new and to return the already processed items, then resolve and compute.

However, I realized a massive problem with this that applied directly to my submission. Mine has to do with mental healthcare, and while I did anonymize the data and more or less make it so I wouldnt have the original data... It was still, sensitive.

#

I imagine most wont have this issue inherently, but it's difficult to get around.

prisma creek
pearl anchor
prisma creek
#

Give your mental healthcare use-case, would falling back to a Cloud AI create any concerns around privacy?

pearl anchor
#

Yeah, I imagine it would. At the least users should have the ability to upfront choose if they want to participate in that or not.

Technically speaking it can be "secured", but concern wise, I would argue that privacy focused people already have an inherent concern about running anything AI that isn't local.

#

(How big of a segment is that in terms of the whole population? Probably pretty tiny. But, as someone who is privacy minded... and whose wife is a mental health counselor, I tend to think about these things alot. Haha)

prisma creek
#

Agreed it can be "secured". I wonder where the concern stems from - if that's more around folks not understanding how AI works , more around concerns on prompts being used as training data, or maybe a mix.

pearl anchor
#

There's also another interesting dilemma in there as well. If there is an extension that would greatly benefit a users mental health, but their computer wasn't strong enough to run it, then would barring someone from the option of offloading the processing be worse? I'd argue so.

Personally I would imagine its around training data, and their data being leaked and identified back to them. But that's a guess at best, would be interesting to have a focus group or survey on that.

prisma creek
#

Interesting dilemma, indeed.

pearl anchor
#

(With that my guy I have a meeting to catch, fun convo though. I'll check by sometime later.)

prisma creek
#

See ya! I'll prepare for sleeping here :D. 9 PM in London.