Yes with limits as with all of our AI

[sturdy zinc]

how do you sanitize it? what information is sensitive for you? can you tell more pls?

[safe wraith]

Uh I just delete it, any urls or any references to business logic that reveals what we actually do, any personally identifiable information, etc.

There is no magic script if that's what you're asking and it would be insane to rely on one

[sturdy zinc]

Makes sense. But why you it's insane to rely on one? I have a few apps that let's you remove PII/PHI/custom sensitive fields. Do you think they are not reliable?

[safe wraith]

absolutely not, I would always go through my code by hand without exception to make sure I'm not leaking information in variables, hyperlinks, comments, server database references, etc. Also the presence of fields that record sensitive data in code are just empty fields, they don't have actual sensitive data

I mean first off, it doesn't take that long, so it doesn't seem like a very big deal

I don't think an app will guess that

x=69 //TODO: John Smith - revisit

is PII. Also that question just doesn't make sense, if you don't know at a glance which information you shouldn't be leaking you really shouldn't be using GPT, much less code interpreter. It's usually as simple as something like replacing a real url like https://ouractualdomain.biz/tacos/index.html with https://junkdomain.gov/tacos/index.html,
or an email address [e-mail] with [e-mail]

[errant echo]

It becomes a much easier task if you can inject all your sensitive information in a service or appsettings etc.. os you can guarantee your "code base" minus the settings file is "sanitized"

[safe wraith]

Yep, most of the sensitive data in the webapp I was working with is in a few json and external .js files as well as some databases. Not so much with VBA automation scripts though, those are pretty self-contained

That's good practice anyway

[errant echo]

Ohh VBA, haven't had to play with that in a while

ChatGPT must be a god send for that language

[safe wraith]

Oh hell yeah it is, I am not about to go learn all the interop libs for VBA just to create some automation scripts. GPT 3.5 is only alright with VBA, 4 is pretty solid. Makes up functions sometimes still but 🤷‍♂️

[safe wraith]

I mean it sounds like you know what you're doing, you should be fine if you're not totally phoning it in. There's a lot of magical thinking surrounding GPT and tech in general so I'm extremely leery when I hear about simple software solutions to replace human review outright.

I'd just treat it the same as something you were sending to an anonymous outside contractor without an established business relationship (think Fiver or something) - you probably don't mind sending them business logic, even database structure is usually fine unless there are descriptive data relationships you don't want to share, but you don't really want them to know anything else about your company. Like you said, if your code is well-factored there won't be too much to address.

[errant echo]

Ohh no I totally get it, I understand how easy it is to accidently leak PII, connection strings, credentials, keys etc..

I have learned through mine and others mistakes 🙂

Sometimes even leaking the "business case" is not okay

So you have to mock even classes etc.. that are close but not the same

[safe wraith]

people leak keys like crazy, just straight up in plain text FE code all the time

[errant echo]

Ya there are multiple services that just crawl repos constantly for leaked keys.

[safe wraith]

Yeah, that's what I mean about certain data relationships or chunks of proprietary / highly specific logic. Usually it's a find-replace away though

I mean I'm sure MS has some closed source class named SecretCortanaSpyingFactoryBlob

[errant echo]

Haha na, someone would have reversed that by now 🙂