tmp-echo-feedback-questions

[quaint nebula]

Right, thanks for the update. I have some follow up questions:

1. Why doesn't the Privacy Policy explicitly mention this type of data collection and potential AI usage?
2. Do you (THM), plan to update the Privacy Policy to reflect these practices? It seems it's now updated, but I have yet to finish reading it.
3. How does the current opt-out, where the only option is; account deletion, align with GDPRs transparency and consent requirements?

Thanks again!

[civic furnace]

What data do you specifically collect and how is it used? Thank you for the initial feedback

[glacial violet]

Any response to those who were banned/stripped of roles when they spoke out?

[quaint nebula]

Since the Privacy Policy was updated without notifying users (At least I never got any info), how does THM, justify this decision, and what is your stance on keeping users informed?

[glacial violet]

Also if you say your not collecting data now, how are you gonna make sure that you don't collect children's data to comply with American and Australian legislation or are you just gonna block those countries?

[meager dew]

Hi Zync!

I'm not a solicitor, so I'm not sure on questions 1 and 3. However, I've been assured (more than twice now on different occasions) that what we have has been thoroughly reviewed and is legally compliant.

[meager dew]

For GDPR, there is requirement on transparency obligations, and so comms for the update go live on Monday. Even if the GDPR requirement didn't exist, we would let users know anyway. It will be through a notification popup on TryHackMe.

[rugged mica]

why is opt-out and not opt-in

[meager dew]

I've responded to this elsewhere multiple times - I've said that I'm not going to respond any further to queries about bans / roles. I've covered it in general.

[meager dew]

Answering questions on TryHackMe will rely on event info, since the platform uses your actions to give feedback and help you progress. It’s also a fundamental part of how TryHackMe works - without it, things like answering questions, spotting cheaters, helping you get unstuck, and giving feedback would be very difficult. I covered this in my main post, but because it’s so critical to the platform, it’s set as opt-out. In practice, you’re opting in when you sign up.

[rugged mica]

this data being collected was not up front on the privacy policy when shadow signed up if they recall correctly
meaning you more or less changed the contract after shadow signed up and are now opting shadow in to use their data

[split dome]

Question, are you currently collecting data? I get it that you are not training anything based off of it, but the collection, is that taking place? And are minors being excluded from this collection?

[meager dew]

Changes to the privacy policy was made to be better, more transparent, and clearer for users - not to make us compliant. We were previously compliant - this change makes our privacy policy even clearer.

[meager dew]

I had this question checked extensively - given the way we anonymise and use the data, means we're compliant. This was really important to us as we work with a lot of schools, and without capturing the event on a machine to understand if you've completed a task on TryHackMe, you wouldn't be able to use the platform. So we made sure our implementation was compliant across the board to not exclude schools. There is compliant, and morals (as someone else brought up before), and we believe its important to do this so we're able to build a much better experience (such as give feedback on approach to solving a challenge, being able to instantly help you, and ensuring you're not wasting time going down a rabbit hole)

The data is no different to an AV looking at what you're doing, or a bug diagnosing tool that are included on most sites globally.

(Just a heads up that I need to log off and jump to another task at 9:50pm - and will be back online to respond sometime soon - not tomorrow, but most likely next week. I'll collect questions, and respond in bulk if necessary.)

[quaint nebula]

Thanks for the reply. Do you know if legal will be responding to any follow up questions in this chat? If not, I appreciate the time and clarifications you’ve provided so far 👋

[devout sentinel]

Gave +1 Rep to @meager dew (current: #136 - 68)

[meager dew]

Thanks Zync! I'm not sure if legal will respond here. Maybe if there are enough questions, I could ask them to group and respond.

[devout sentinel]

Gave +1 Rep to @quaint nebula (current: #3157 - 1)

[civic furnace]

Will you answer this?

[quaint nebula]

As far as I understand your question, you can find the answer in the newly updated Privacy Policy https://tryhackme.com/legal/privacy-policy If that doesnt fully address your concerns, I’m sure Skidy, can provide further information in his reply next week

[rugged mica]

the speed of getting responses feels like tryhackme is stalling for time just for the sake of stalling

to make less people remove their accounts

[proud current]

What if I want to simply turn off echo, will there be any options for me to disable it

[quaint nebula]

If you are in Europe, GDPR require companies to respond "without undue delay" or within one month of your request, I suggest you e-mail support directly, that would probably be your best bet

[rugged mica]

yeah did a GDPR data request friday so they have one month to comply and give shadow said data they have on shadow

[quaint nebula]

As much as I appreciate the information and follow up on this, I will also be following up with my local data authority about this, mainly because I don’t have the specific expertise when things get this detailed

[wintry fjord]

@meager dew Even if the site necessitates some amount of data collection to remain functional as you said in your response citing task completion as an example, Echo is still a relatively new feature and THM's website has worked for long without it. So, clearly using our data to make Echo better as you say isn't functionally indispensable for the website to work. Then why does opt out have to mean account deletion? I'm sure you'd get a lot of people who wouldn't opt out and you'd have enough data to train Echo. On the other hand, if an opt-out feature without account deletion is given and the majority of users end up opting out, leading to you not having enough data to train Echo, I guess that says something about if the community actually wants or needs Echo in the first place.

[hollow fog]

I agree. There should be an option for it.

[meager dew]

There will be a way to turn Echo off (more than sleeping it for a specific room). This won't however not show feedback on your actions, but will stop all notifications from Echo in the room.

[meager dew]

What data are you referring to? If its Echo, its events when you use a machine. I can't list every event because its generic - a few examples would be: opened folder, performed nmap scan, attempted login; its worth noting a lot of this is already logged on the system and we just pull that (such as looking at apache log files to see when the web server made a request).

[meager dew]

You will soon start to see us remove flags from challenges, and only have questions answered based on what you do on machines or exercises (this will also hopefully mitigate cheating which has been a huge complaint in the community). This along with other functionality I mentioned in my post, will be introduced very soon, making the platform unusable without it.

[lofty bolt]

well that wouldn't help much. then teams could send each other scripts / commands to run

i'd propose slight differences in the challenges per VM or per team, dpeending on what room type it is

but that may be too hard to accomplish

AI hacking is also a very real threat against echo

[meager dew]

There isn’t a perfect solution. Looking at actions, and timings, is the best way. It will dramatically reduce cheating, as putting scripts together will take cheaters much much longer to do.

[timber wing]

Why not just integrate dynamic flags like HTB? There is still the matter that HTB also disallows writeups for 'Active' Machines and THM actively encouraging them for every challenge, but at least it'd make some sort of dent to cheaters straight up copying and pasting flags? At least they'd have to spend time, read, and copy paste 'commands', if a user want answers.

^ This is specifically only for Challenge rooms; not walkthrough rooms.

[rugged mica]

agree with bleu blue here

[meager dew]

Do you think rotating flags is better than looking at the actual behaviour and actions of a user? I don't.
Cheating on rotating flags is the same as cheating by writing a script to simulate the behaviour, just worse bc they can't actually see if the actions are legitimate; we have a better understanding to what normal vs abnormal behaviour looks like.

[timber wing]

I never said cheating was good at all. I said minimize where possible. I don't know what sort of telemetry you guys monitor but if it's actions within attackboxes, interactions with spun-up challenge machines or timings, then these will be monitored regardless of whether a flag is dynamic or not, is that not correct? + Dynamic flags just makes it more fun in my opinion, knowing that there isn't the same old easter egg somebody else already has gives another competition boost for me at least.

And I don't know what's this about not "seeing if the actions are legitimate" because anybody smart (that is solely focused on cheating), will time their answers and not skip their steps, and still cheat to meet an average "answering/flagging" baseline, regardless of your captured telemetry, even now.

[proper fractal]

I thought part of the fun with CTF's were that the goal is to get a flag hidden somewhere on the box regardless of how you do it. I would expect walkthroughs to lead you down a path, but for a challenge I would think it doesn't matter how you get to the end even if there is only one intended path.

[warped yarrow]

Sounds like a fear of becoming stale.

[proper fractal]

I've only been doing these a few months though so I don't know their true purpose.

[warped yarrow]

@meager dew Who do we contact regarding the legal issues mentioned here?

[opaque plank]

Probably with that.....

[zinc badge]

what if you don't rotate the flags but create random flags of a certain length? might be a difficult process to integrate into the room, etc., but could be a nice idea 🙂

I think this way people can still get a "reward" for getting into the machine but they can't just copy and paste from walkthroughs (but just a suggestion)

[maiden lotus]

Easier than you'd think. If the VMs are integrated properly with AWS then you wouldn't even need to update the boxes to get that working. Could be done entirely between AWS and the THM site.

If not then you'd need a ≈10 line bash / powershell script running at boot

[nova quiver]

Behavor monitoring is great, but it only catches cheating after it’s already happened. Dynamic flags actually make cheating harder up front, they raise the cost before someone even tries to cut corners.

With static flags, a user can bypass the challenge entirely by copypasting a value from a writeup. With dynamic flags.... they must either execute the exploitation steps themselves or invest effort into automation that do the process. That’s a huge barrier already

Surely dynamic flags won't remove cheating but it could reduce it drastically.

That could like allow you to then focus on the smaller pool of determined cheaters

[maiden lotus]

I did make a PoC for that a while back:
https://flag.muir.land
Built that as a response to widespread cheating in a THM competition. Trialed it out in a walkthrough room and it worked pretty well, but I never put another challenge room public on THM after that event, so didn't really get a chance to use it

Principle definitely works though, and obviously integrating the concept into the platform would be a whole lot smoother (e.g., the way HTB have done it)

[nova quiver]

If we can't remove cheating which is mostly the case, ATLEAST make it hard to cheat.

[timber wing]

Exactly what I'm saying.

[rose isle]

Hello,
How soon?
And how does this affect people using their own attack box?

[floral wind]

@meager dew respectfully, if u care about the community, why do you censor public opinion. THERE ISN'T A CUT BTW people than can vouch: (sorry for the ping) @timber wing @glacial violet @forest coral @unkempt silo

[floral wind]

@wraith lark too

[proud current]

There was also an announcement for echo last week from Skidy that got downvoted to the ground, don't know where that went

[cinder lance]

I would have to see how things are implemented, but I think there’s an element of enjoyment to retrieving an actual flag that might be lost with this solution. (Of course, it’s not like this sort of automatic detection of actions and flags are mutually exclusive, but it sounds like the plan is to replace flags rather than augmenting them.)

[junior obsidian]

Even ask how many of Jabba's fake accounts were used to increase 🔥 the count

[crisp ridge]

Highly doubt that would be needed when everyone is pinged - and that's a cheap shot coming from a fake account itself. Get your next one ready because it seems like you're not too far away from a ban yourself looking at your horrible behaviour since you joined.

[junior obsidian]

My bad sis

Oh u watching , I'll remember that

[crisp ridge]

https://tenor.com/view/i-see-you-got-my-eyes-on-you-gif-11000728512614562106

[junior obsidian]

Isn't DKob a brother?

I don't know

I wrote bro

He replied sis

So she's dkob

You replied "my bad sis"

[junior obsidian]

Yaa

[junior obsidian]

Here.

So dkobs a sis

No, XD

[little cradle]

So criticizing is "horrible behaviour" now?

[junior obsidian]

Yes, XD

[vital ore]

can you answer why that happened though? why have a platform that claims to care about community input turn around and hide the evidence of the community disagreeing with them?

if the community disagrees with you, why try and sweep it under a rug and pretend its not there? let them disagree with you, listen to their concerns, and answer their questions

[crisp ridge]

You're asking the question to the wrong person.

[vital ore]

dont pretend like theres nothing going on, and shove it all into the back corner where it can be more easily ignored

[vital ore]

then i suggest the right person reads it

[crisp ridge]

It went far away from "criticizing" - I suggest you read the chats from this afternoon until now and see how far it went. I don't care how you think or how you feel, there's no excuse for personally attacking someone.

[junior obsidian]

What

[crisp ridge]

If you honestly don't understand what's wrong, I'd suggest reviewing the English and the words you're using so you're aware of what you're typing.

[junior obsidian]

Hmm making fun of someone's english

Very bad behaviour from a mod🥺

[vital ore]

to be fair, its common knowledge that jabba uses fake accounts. asking if any were used to react positively to the message is no crime.

[crisp ridge]

If that's all you can see and understand, I'm very sad for you.

[vital ore]

this is the only personal attack ive seen in this chat recently

[junior obsidian]

Hmm i see, this is how they treat us in THM 🥺

[crisp ridge]

I do not see where the question is? Sounds more like an affirmation, or an indirect way of implying it.

[timber wing]

100% agree. These are crucial steps that @ebon hemlock should be taking as a company at this point to make this entire situation better for it's users.

First off we find out that our data is being collected against our will(?), then we get told that it's not being collected(?), then we get told that it's minimally getting collected(?); a very obscure line in the ToS that obviously reveals that "TryHackMe can do whatever it wants with user data as long as it serves a legitimate purpose" is very eyebrow-raising. Resulting in lost trust, "What are you guys gonna do to make it better?" is my only question.

This is a learning platform for all sorts of beginners and professionals alike, while it has been assured to us that our data has not been used for Echo 'yet', which was not very reassuring in any sense to be fair, some of us just don't wanna be lab rats or test subjects for y'all. and some of us really don't wanna use Echo either. TryHackMe is not a true competitive hacking platform as it is. And y'all wanna keep digging that notion deeper, fine; but some of us would like to independently learn and grow by ourselves, without AI assisting us. An Opt-out would be the best thing we could receive right now.

[vital ore]

raised eyebrows: question to be asked

[junior obsidian]

Whats wrong with a qn, it's a freedom to qn, is it not

[proud current]

Respectfully could you guys take the drama out of this channel so that the actual concerns don't get drowned out

[junior obsidian]

🫡

@crisp ridge plz stop it, ur not making any sense

[crisp ridge]

Typical - See whatever you wanna see. The replies have been horrible and there's no excuse for how people were spam pinging certain staff members and being passive aggressive.

There was a recent outrage in HTB too for the 60% price increase - even more people were involved and I am very confident when I say that they were definitely angrier than the people here. Unsurprisingly, it was nowhere near the childish and full-of-emotions behaviour that appeared in here.

Knowing that most people are kids in here, that's not surprising.

Not sure I understand is that a reply to my message or?

[junior obsidian]

Hmm calling people's opinion childish, I see

[crisp ridge]

You are definitely childish.

Look at your replies - nothing constructive.

[junior obsidian]

I see

[junior obsidian]

Doesn't have to be for u

[crisp ridge]

I see no point in not calling out what is actually happening. it's either this, or not saying anything and leaving people ramble alone and being toxic.

[finite radish]

Hey all,

I understand that everyone is upset-

If there are genuine questions that you want answers to, please post them here.

I would like to ask everyone kindly to avoid answering questions or otherwise stirring up discussions

The only people who have the answers you are looking for are employees; if you agree with someone’s question upvote it or otherwise write your own question.

As soon as someone is available to give you an answer you will receive one 🙂

[rugged mica]

why is there no link to old privacy policies on the privacy policy page??? why do we need archive.org to read old privacy policies to be able to check for changes??

also
when did the telemetry and analytics data collection start??

[storm kernel]

How to turn off echo, the logo of it constantly in the corner, moving and stuff is very distracting and I just want the functionality not to be there or enabled at all.

[rugged mica]

currently there is no way to do that

[storm kernel]

Well that’s embarrassing…. It’s distracting me from even just doing rooms, it’s not adhd friendly at all

[rugged mica]

it can also be problematic for visionally impaired people as it can trigger having to check that corner of your vision over and over again

[tranquil field]

There are likely ways to visually hide Echo if it impairs your ability to learn in your own style.

[finite radish]

More information on this soon 🙂

[tranquil field]

Despite Echo's obvious current shortcomings in it's advice noticed both by me and others, I have no doubt there are people that would value some sort of automated feedback on their actions. Having the ability to purposefully turn it on and off, at least from a visual and direct interaction point of view, would go a huge way along with the usability of the platform. Just my 2c.

[dense cloak]

I know THM has historically been very data driven
Perhaps presenting the users with a definite opt-in/out and analyzing the data?

[tranquil field]

Feel the same way. The constant nagging "hey you completed another task" is extremely annoying and totally not necessary when I'd like to focus. Honestly it feels a bit like those 90's popups in your face all the time.

[gleaming flare]

I have some questions:

1. What agent/model does Echo Use? (Claude or something else?)
2. What is the data set used to train Echo? (Is it room data? or some other pool?)
3. How long was the system trained for? Did it take a matter of days/weeks?

[tranquil field]

To sum up my thoughts on the matter: I get it's THM's shiny new toy and likely a lot of people have worked very hard to make it happen and they are very proud of it and would like to raise awareness to as many people as possible as fast as possible that it's there and ready to be used. But toning it down a bit and giving us the option to visually hide it completely as well as maybe some checkboxes enabling or disabling types of feedback from echo would go a long way appeasing most of the community.

[nova ledge]

ok

[quaint nebula]

I understand your frustration, but as a moderator I don’t think it’s appropriate to say things like "most people here are kids" or that the community showed "childish and full-of-emotions behaviour." Even when discussions get heated, framing members in that way isn't constructive. Looking back at the conversation, it also seemed like you were fueling the fire rather than being the bigger person and helping to de escalate. Your role should be to put a stop to situations when they get out of hand, not to add to the tension, leading to this reply from me. Something as simple as reminding members of the rules or redirecting the discussion would have been a much more effective way to calm things down and set the right tone, or even just not replied in the first place

[quaint nebula]

Where is my notification popup on TryHackMe? I got showed a Youtube video in my face. That's all...?

[finite radish]

I've addressed this outside of this channel with DKob-
I'd like to ask that if there are any further comments for this discussion to stay out of this channel; it will make it harder for our team to find genuine questions that deserve a response.

Preferably, any formal complaints regarding moderation or misconduct should go to my DMs if possible, or if you would prefer I can give you an email to submit a complaint to 🙂

[warped yarrow]

@meager dew still waiting on a response to who to contact about the legal issues.

[terse quest]

I mostly just lurk in this server only to see an AI be added to this site... uh-
Could we just not add AI to everything?... thank you... if you need i'll take my leave
-# sorry for burdening you all

[devout sentinel]

Gave +1 Rep to @finite radish (current: #6 - 1781)

[meager dew]

I'm sure we can think up some delightful moment still. Will pass along to the team as its a good point.

[meager dew]

Thanks DrHeat. We're introducing (this week) a way to disable Echo in all rooms.
Appreciate the teams recognition, we had about 7 different teams involved in the development of Echo; probably 20+ people total. They worked very hard on it - creative design, product design, software engineers, content engineers, product managers, AI/ML team, QA, and more.

[devout sentinel]

Gave +1 Rep to @tranquil field (current: #1085 - 5)

[meager dew]

I assure you it is coming.

[meager dew]

It depends on your legal issue. Without the details, I can't give any advice. Find a solicitor in your region, contact the ICO etc..

Appreciate all the comments, thoughts, suggestions and concerns. I don't believe there are any questions I've not responded to. Some I've already answered, some are statements.
I'm going to close this out. Thanks all.