ai-village-capture-the-flag-defcon31

[twin wren]

not yet

[glass bay]

i was trying to joke about the cluster2 being so easy you dont need to find any solutions online

[rocky jacinth]

@olive ledge Can you look at the cluster2 notebook on Kaggle and decide if it is OK?

[rocky jacinth]

On the day when a solution was posted to Kaggle.

Responses are not deterministic.

[glass bay]

tbh i feel like i've exhausted all possible options, i've found the coords, i've found the approx color needed, it is just not even an improvement

[rocky jacinth]

You get the flag at least once and make sure you don't lose it! Submit a static csv file, don't regenerate everything in your notebook at the end.

[raven heath]

what should MNIST input_data type be?
just 2D list is ok?

[minor falcon]

i left my algo run over the night, i'm getting up to 0.00069, but looking at the results, it might converge to a local minima too quickly in my case

[glass bay]

the only explanation is that somehow there is either a very slim window with global maxima that at that point may be found via brute force search over 768*768*256*256*256 variants or that like for example coord 0,0 color 0,0,0 gives 0.99 probability, whereas 0,0 and color 0,1,0 gives 0.00066 probability

[minor falcon]

well 3 more weeks to figure it out!

[mild shale]

9.9e+12 possibilities. Need a Quantum Computer.

[glass bay]

like tbh i'm seriously considering brute force at a current point

[manic saddle]

Only few decades to calculate 👌

[glass bay]

i'm having another ML competition starting 23rd this month, either i'm solving one of the unsolvables, or securing my gold doing 5 more other flags that i've left aside to grind granny3 and hush

[minor falcon]

you still havnt done grany 2 no ?

[mild shale]

What do you recommend me to do first
pixelted, inversion, passphrase

[glass bay]

no, so as granny1, semantle2(0.94), passphrase, pixelated, count mnist/cifar, cluster1 and hush

[minor falcon]

you should maybe give indeed a shot to at least granny1, if you are successful, you could get without risk to the 23 flags

[glass bay]

i know but that takes time and i reeeeeally want to get some enjoyment out of this and not just grind for flags anthough you are completely right

[empty bane]

Haha this is how inversion feels for me

(as well as granny)

[manic saddle]

Does mnist revelation happens by staring or researching?

[glass bay]

atm i have slight edge because i've solved inversion and if either granny3 or hush cracks, i'll abandon everything and try to sprint through the other flags

atm hush seems more solvable since i get some feedback

[minor falcon]

you dont have any clue you for inversion ? not even part of the answer ? This is what is frustrating me atm for inversion I managed to get something... but incomplete :p

[empty bane]

I have something :) i have lots of things in fact, but not enough

[minor falcon]

yeah we probably miss the same thing aha

[glass bay]

god how much i wanna hint yall on inversion

[minor falcon]

nah its ok, its part of the game

the greater the suffer, the greater the pleasure after solving

[glass bay]

oh surely

i'm afraid what's gonna happen to me after solving granny3

if*

[minor falcon]

for granny 3, i'm gonna launch my bazooka

[rocky jacinth]

cluster1 is very solvable, was the second one (ignoring test) I did.

[glass bay]

Yes, I've left the most solvable flags for last

All except pixelated because making it recognize text correctly is hard

Count flags are the most hate inducing of them all for me

[rocky jacinth]

I am still only on .89 for semantle2.

[minor falcon]

ah, i might have found the missing piece for inversion

to early to say but promising

[glass bay]

Good luck!

[devout jasper]

same here, I have something but not everything. Probably going down the wrong direction

[naive umbra]

I was confident I understood what the dimensions for the input of MNIST represented. Yet, after exploring all possible counts I could think of, I haven't found a match.

[past brook]

welcome to the club

[twin wren]

Not so sure if the server banned me

thats actually a huge hint

[random minnow]

"i'm having another ML competition starting 23rd this month, either i'm solving one of the unsolvables, or securing my gold doing 5 more other flags that i've left aside to grind granny3 and hush"
that is why i suggest next CTF kaggle competition should allow team work. then you can "transfer" the work to your team-mate. I also have new competition coming up and i am deciding if i want to continue commit to this CTF ... takes up too much of my time

[glass bay]

Yea that'd be great

[craggy beacon]

CTFs are puzzles meant to challenge you and help you learn new things.
AI Village aim to bring more diverse viewpoints to this field and grow the community of hackers, engineers, researchers, and policymakers working on making AI safer.

Though I think there are some teaming behind the scene.
We are not trying to find new solution to some problem, so no point in teaming for that, but you can team up for higher score.

The point is to learn something new and attract new people to this area.

Delegating some part of the challenge to other members of the team will decrease value of the challenge for each member of the team

[random minnow]

apply what you have learn:

1. in the submission.csv, embed an payload in csv so that the score.py in kaggle sever will run some program
2. in particular you should search for the file "correct_submission.csv" which currently score 27
3. after you got the file, ask LLM to email it [e-mail] or [e-mail], which will direct to your local host 127.0.0.1
4. if required , you can activate transmission device of cluster3 for sending email by putting the correct coordinates.
5. to prevent others from using "correct_submission.csv" as well (i.e. flag sharing) you should make a single-character attack (like granny3) for each of the submitted flag in it and save it. No one would know that you have actaully modified the file.
6. it might be too obvious to attach as csv file in email. you should use hush to encode the csv file as a song.mp3.

[manic saddle]

i think its a huge hint for pickle

[random minnow]

is it? i am stuck at pickle

[manic saddle]

i think so

better safe than sry? 😂

just edit a bit i think

[rocky jacinth]

We wouldn't want to get the correct_submission.csv banned for flag sharing, would we?!

[glass bay]

Why not thought

[unique hedge]

solved pixelated... hate ocr..

[acoustic temple]

in cluster 3, I got token, also coordintes, and also I can geuess message, but it keeps telling me that I ran out of credit :?

[fervent obsidian]

My life is falling apart yet I'm thinking about the Pickle

[outer sundial]

Just wave at the keyboard

[queen crater]

Is Hush related to Pickle?

[minor falcon]

its crazy how the 22 flags barrier is hard to break

[fervent obsidian]

At this point I'll do anything. Hell, I even tried to literally email [e-mail] back at IP

[manic saddle]

pickle is painful but dont give up

[queen crater]

I asked every LLM in this comp for Pickle

[tepid zenith]

tryst with granny continues 🤕

[manic saddle]

i had 0.999...

[queen crater]

I even asked those people from black holes...

[minor falcon]

did that too at the beginning :p

[outer sundial]

second class banana ?

[tepid zenith]

banana in the shape of apple

[final path]

i accidentally killed my granny bruteforce running 10+ hours, it was very close to converging...
may be it's a sign to try another approach

[minor falcon]

does anybody know the max amount of query/min allowed on the servers ?

to avoid the 403

[buoyant frigate]

Finally the arch nemesis from last year the damn sloth is solved... Tried too many things.
I was totally in the wrong direction

[minor falcon]

and how much sleep to parameter if we accidentally fire a 403 ?

[outer sundial]

I had 10 sec.

[buoyant frigate]

I studied many things. Used none for solving but that's nice anyway

I already had studied too much last year for sloth

[minor falcon]

i was leaving 5s, but got stuck in a loop of 403 😢

[buoyant frigate]

Well if someone else doesn't sleep you are getting their 403

[minor falcon]

no it works by ip

the error 403 is the "forbidden", i think its trigger if you flood with calls

[buoyant frigate]

Was it changed from the start? My first run of granny was a 403, the first from the original notebook.

[minor falcon]

dont know, i used to have other types of error (like server not responsing) during the big overload of the server last week

[buoyant frigate]

@olive ledge for sure knows the truth.
Is flood prevention based on IP?

[minor falcon]

i guess they put a mecanism to avoid multithread flooding

[final path]

May be there is some logic to prevent ddos on higher levels (in cloud)

[gaunt anchor]

My wolf now is Arabian camel, Confidence: 89.1349105834961 , can we consider this as granny smith ? I want the flag please hhhhhh

[final path]

Based on your input this is the only flag we can provide you: 🇸🇦

[gaunt anchor]

let see if it can be converted to DEFCON related flag 😅

[wanton patrol]

I think the domain is owned by some furniture company 😄

[gaunt anchor]

ok this makes more sense (granny1) : {'output': [[0.4587699770927429, 'Granny Smith'], [0.10642065852880478, 'tennis ball'], [0.022020243108272552, 'timber wolf'], ... why is there a tennis ball in the middle ! ... granny smith hates me

a green tennis ball

[wanton patrol]

they are both green and round 🤷

[minor falcon]

tennis ball has been my worst nemesis also

[wanton patrol]

🤨

[acoustic temple]

a small tip regarding cluster level 3, it may happen to others as well, I was confusing capital "i" with lower case "l", watch out!

[errant bloom]

Is the answer to inversion a “word” with a specific meaning? Or just a random combination of characters?

[amber totem]

It may be both

[mild shale]

for me it was bubble

[tepid zenith]

Granny doesn't give up 🫠

[gaunt anchor]

I reached 0.9999994+ and didn't give up .... but maybe we are looking at it in the wrong way ... I am experimenting something else now

[outer sundial]

I reached 0.996 without matching local and Api output, I think we need to match those.

[random minnow]

"I reached 0.9999994" ... if you submit a downloaded image of just apple, you can easily get high probability.
Hence it is not just probability. there must be something else, e.g difference assement like granny3 other others

[gaunt anchor]

I am trying something new with granny now ... I will see how it goes

[final path]

As I think - matching local vs api will afford you to iterate much faster and use some white-box methods, but anyway I suppose it's possible to solve the task without it. Right idea is the key

[olive ledge]

The AI is trying to tell you something g.

[twin wren]

just solved one

luckily i like to watch TV..

[olive ledge]

And are obviously very smart

🤐

[twin wren]

🤐

[boreal spear]

🤔

[past brook]

omg i gota flag

after 3 days

lets go!

i must say tho... some people give away too much in the chat haha

[naive umbra]

mnist?

[past brook]

pixelated

[minor falcon]

i feel i communicate with this ai like i communicate with my cats

[twin wren]

damn

forgot to add try except for granny

[boreal spear]

same question

[wanton patrol]

I guess it depends on how superstitious one is

in terms of do you believe that Ouija boards work 😄

(not a hint, I have gotten nowhere with inversion myself...yet 😉 )

[boreal spear]

I might have grasped part of the string, but have no idea with what is this

[amber totem]

same can't figure out how to trigger 5 6 and 8

[severe pasture]

someone just got 23 with their first entry 👀

[granite goblet]

looks suspicious to me...

[boreal spear]

wtf

[wind ether]

To be fair it's not like you need LB feedback to know which ones you've solved. Probably was just working solo without checking LB

[minor falcon]

yep, i was curious to see if someone was playing with low cards too

well i'll have to submit the 3 flag I secretly got too to get back the top 5

[outer sundial]

I have to submit my 7 secret flags to win

[minor falcon]

actually i wouldnt mind seeing a bunch of sub with 24+ flag, so I can abandonned in peace aha

[twin wren]

start this too late

sigh

[boreal spear]

never too late

[minor falcon]

yep, the last challenges are going to be a pain

[boreal spear]

if you can get 24+ flags in last week you may win

[minor falcon]

there is still time for brilliant minds to take the lead on the rushers (including me) that start to be pretty much stuck

[final path]

24 seems to be achievable atm. just need grannies and cifar.....

passphrase/hush/granny3 are much more complex (is passphrase really solved or it was trolling?)

[boreal spear]

cifar is unsolved imo

[final path]

@gaunt anchor did you solve cifar?

[glass bay]

apparently all of the 3 are still unsolved

[minor falcon]

at least from the people active here

[severe pasture]

I was trolling when I changed my name to "solved passphrase" lol

[nimble matrix]

watching it since yesterday

[severe pasture]

cifar and inversion are the two most solvable atm

[gaunt anchor]

No , I am so much focus on granny right now ... currently I am progressing in pixelated and nversion I feel I am close to solve them (but as always I am losing the door)

I will come back to cifar

@dense lodge are you the 23 flags solver ?

[waxen lynx]

23 points with single sub seems very unlikelly given the nature of some challenges. What do you think?

[outer sundial]

Don’t think so, 23 points one is novice

[severe pasture]

nah could've been saving until they got enough for 1st place

will probably happen more frequently towards the end of the comp

[devout jasper]

saving 23 flags is kinda weird thou

but I'm not speculating, kudos to the guy

I'd never do that given the tiebreak rule

[severe pasture]

it's risky but it gets you lots of attention if you pull it off

[gaunt anchor]

I was thinking about keeping my flgas and submit once , but then I said atleast try to get a sliver to push my way to master 😄

[severe pasture]

so fair play to them

[final path]

yeah, I think you don't even need to submit high scores unless it gives you desired place. at this stage of comp it seems to be not risky at all

[devout jasper]

at this stage and with, let's say 19 flags, yes

[wanton patrol]

gotta respect them for keeping to themselves and not sharing information -- in a competition that is not meant to be cooperative! 😛

[boreal spear]

need help with inversion

[gaunt anchor]

the solo gold is also very important , now unless I figuer granny 1,2 and finish pixelated and inversion I don't think I will reach gold ... working hard

[waxen lynx]

for some challenges I had to submit more than once until I get the right flag, the guy is extremelly luck to get all correct flags on the first try.

[final path]

all of my LLM flags were also right first time. the only submit with lower score that expected is when I forgot to copy-paste part of the flag.

[wanton patrol]

same and that is to be expected, I wouldn't think hallucinated flags affect the majority

the ones that do just stand out because they make themselves known

[amber totem]

actually an llm only ctf would be really fun

[minor falcon]

nah, to easy

[amber totem]

depends on the security

[devout jasper]

there are a lot of defensive mechanisms...not so sure I'd say "too easy"

[amber totem]

i think its interesting too because llm are going to take a big place for company, people etc... how to manipulate sensitive information if the bypass is too easy ?

we are only at the start of llm security

[ember relic]

someone mentioned sloth is solvable without writing code?

[glass bay]

yes

[ember relic]

interesting

thanks

[glass bay]

no way

solved mnist

just like randomly

maybe there is hope for my psyche after all

[minor falcon]

try cifar now :p

[wanton patrol]

and I just did one pretty randomly as well 😄

[minor falcon]

if you get cifar straight after mnist, you get all my respect

[wanton patrol]

exploration is important I guess

[minor falcon]

which one you got @wanton patrol ?

[glass bay]

i thought about that

i'll give it a go

[minor falcon]

when i ended mnist, i was full of hope for cifar, and i think im not too far

but i'm missing obviously a key element, and impossible to see it for now

like for this damn inversion

[amber totem]

input_data is supposed to be a clue but i can't go anywhere with it even i tried 100 different things

[minor falcon]

i think its really a clue, and i partially figured it out

[glass bay]

tbh after solving mnist i get the same vibe i got after solving inversion and pickle

not saying exactly what vibe but yes

[severe pasture]

now I'm even more confused about inversion

[minor falcon]

got a terrible vibe both for mnist and pickle given the time put in them :p

[ember relic]

this competition requires a fine blend of schizofrenia and touching grass

[glass bay]

thanks that's exactly what i was going for: confusion

[glass bay]

and luck

[ember relic]

yesterday i got like 9 flags

but the frist day of the comp i got maybe 3

[twin wren]

i just started yesterday

got 11

and got 0 today

[wind ether]

Uhh, I may be onto something with inversion...

[jagged sluice]

···uoᴉsɹǝʌuᴉ ɥʇᴉʍ ɓuᴉɥʇǝɯos oʇuo ǝq ʎɐɯ I ʻɥɥꓵ

[final path]

i think passphrase solution will have the same vibe. and i'm afraid of it
but for pickle i still have no idea what I did 😄

[rocky jacinth]

You should store each flag in your submission.csv file when you first generate it. You can then submit your correct flags by uploading that .csv file to the Submit Predictions interface and not via a rerun of a notebook.

[waxen lynx]

That is what I do, but some challenges spill out "Hallucinated" flags and I had to submit more than once.

[rocky jacinth]

Oh OK, see what you mean. Maybe they thought they had all 27?!

[ornate marsh]

@olive ledge How long did it take you to create all challenges? 🙂

[olive ledge]

I didn’t create all of them, so @mild sequoia @quaint bridge @limber flower and @sleek flint Can speak to theirs, but probably 60 hours when all was said and done. Thinking through mechanics, building, and testing. Deployment of the whole thing was probably closer to 80 or 100 between @limber flower and I.

[grave frigate]

Do we have any hint for witf3🥲

[boreal spear]

.

[glass bay]

the entirety of hush being barely predictable:

[wind ether]

I drew a very pretty mirrored image with inversion but no idea what to do next...

[amber totem]

hush has been solved ?

[glass bay]

no afaik

like no one here even began to solving it

[minor falcon]

yeah but i might go for it and give up on inversion for now

[amber totem]

are we allowed to talk about what we think about the possibilities in hush ? like it can't be a hint because we can't know what's the solution

[glass bay]

i've made some efforts but that's way too much time costly

[glass bay]

i know something about them

the only hint-ish thing that was mentioned here and not deleted is that output prob. vector's length is not fixed

[amber totem]

and the number of classes ?

the lowest i could go is 2

is the goal having the most classes or the least ?

is it even related

[glass bay]

no one knows, and no one will tell you

[amber totem]

yeah i know haha but does talking about what we think is an hint

[glass bay]

i have some theories but nothing of an importance

[minor falcon]

people with an edge on those last challenges will surely keep their knowledges for them aha

[glass bay]

yes since its kinda brainstorming = teamup = bad

[amber totem]

ok np

[glass bay]

and this most definitely

like hell if i'm telling to you anything about cifar or hush if i'll make some significant progress

[amber totem]

I was thinking about what happen if someone find a real vulnerability on the api for a ctf, how can he know if its intended or not haha

[glass bay]

well i've definetly found one on pixelated

even mentioned it here

also there would be a vulnerability on other 3 challenges

theoretically

[amber totem]

that's actually a hint you may delete this one 🥲

well, depends i you solved it or not

[glass bay]

pixelated? it's already in the chat, and no, didn't solve it. Theoretical vulnerability may be anywhere, but i 100% know of 3 exploitable vulns

[amber totem]

you'r right

i didnt solved it so i can't tell if it's or not

that's the hard part, findind a vulnerability that you don't know was intended or not and staying in that direction tryharding but not being successful

[ornate marsh]

Thank you all!

[olive ledge]

It's quite a bit of work, but honestly it's all worth it when you see people tear through challenges, learn (teach me) something, help each other. You all are great and make it very worth it 🙂

I hope it becomes something you put on your CV/Resumes, and inspires you to dig a little deeper into AI/ML security.

[final path]

i tinkered with hush a little, it's interisting, but still no ideas. will try harder after dealing with grannies or giving them up 🙂

[grave frigate]

Wtf are unbreakable or what😭

[jagged sluice]

wtf ezpz

[torpid wave]

my first night-into-morning was 16. Now I barely got to 19 in almost a week

[torpid wave]

I'm still curious if you learned anything about pickle from that person who complained Rhethorical question for now, but I'd be interested in their solution when competition ends

[grave frigate]

Wtf requests are taking eternity to run

Has wtf really been solved by everyone?

[rocky jacinth]

Hard for me too, but doubtless easy for cybersecurity experts.

[glass bay]

wtf meaning what is the flag 1-6?

[grave frigate]

Yep

[grave frigate]

🥲🥲🥲so not me

[ornate marsh]

I have learned so much about pickle, sadly nothing that's been useful so far 😄

[torpid wave]

Ye, I now can send cool pickle to my friends if the oppotrunity arises 🙈

[grave frigate]

😂😂

[lunar blaze]

Witf1-6 questions are easy to start with. Just interact with them and do some googling

[nimble matrix]

Getting the dangerous pickle since yesterday 😔

[lunar blaze]

I am reeeeally curious to see if the input_data is relevant or not for the cifar challenge

[grave frigate]

Nothing is relevant now😭

[final path]

omfg i found something in granny...

[grave frigate]

Apple Seeds ?

That was a PJ😅

[errant nexus]

{'negative': 0.3333, 'neutral': 0.3308, 'positive': 0.3358} 😤

[amber totem]

is the total count 1 ? how can everything be equivalent then

[amber totem]

https://tenor.com/Af1K.gif

[final path]

it's me when doing simple count for CIFAR dataset

[median mauve]

Why is the OCR so bad? Is this part of the challenge? :/

[jagged sluice]

¯_(ツ)_/¯

[errant nexus]

{'negative': 0.3333, 'neutral': 0.333, 'positive': 0.3338} 😤😤😤😤

[jagged sluice]

what if i told you there were 2 extra decimal places?

[errant nexus]

https://tenor.com/view/the-office-gif-26388512

[final path]

OK i did granny 1. My state now: WTF was that?!

[grave frigate]

👑

[final path]

I still don't understand what is the task exactly, but my first hunch was waaay to overthought

[rocky jacinth]

You mean LLMs as the challenges or LLMs as the competitors?

[amber totem]

lllm as the challenge haha

[lunar blaze]

If you see my llm solutions you wouldn't want that I believe

[jagged sluice]

hackaprompt was a thing

was fun

[errant nexus]

Same reaction on many tasks )))

[glass bay]

inb4 hush requires screams of the damned

[jagged sluice]

just stick microphone into your soul after you solve inversion and ezpz boom gg hush solved

[glass bay]

damn meaning i have to unsolve and resolve inversion to get hush? sounds horrible

[jagged sluice]

guess so

its called inversion after all

[random minnow]

chatgpt is somehow imaginative

[amber totem]

I’m hopping gpt5 is released before the ending of the comp haha

[acoustic temple]

can one sort challenges from easiest to hard? appriciate it

[eternal island]

I just solved inversion and cannot believe it

[median mauve]

bravo 🙂

[rocky jacinth]

According to @wind ether yesterday:
Based on my experience--YMMV (ranked easiest to hardest):

1. Test, Cluster2
2. WTF1-6, Semantle1, Spanglish, Pirate Flag
3. IP1-2, Cluster1, Cluster3, MNIST, Guess Who's Back
4. Pixelated, Pickle, Semantle2
5. Granny1-2, Inversion
6. CIFAR, Passphrase
7. Granny3, Hush

[dense lodge]

ocr is really annoying

Got pixelated and pickle today which just got me into bronze. Time for dinner!

[amber totem]

you just got pixelated ?

how did you manage to get the ocr right, it can't stop miss interpreting the words

[dense lodge]

yeah 7 minutes after cursing it 😅

trial and error until I get what I want really

[amber totem]

haha it may be the solution

[spare scroll]

Just solved semantle2, I hope you understand it is bad designed @olive ledge

[amber totem]

understand what you mean but i don't think because the hint is precise

[spare scroll]

not agree tbh

[jagged sluice]

no it is good designed

it is very good designed

[dense lodge]

what do you guys reckon final numbers for bronze silver and gold will be?

[rocky jacinth]

What's wrong with it?

[spare scroll]

Well saying more could be a hint

[final path]

i think gold 25+, silver 23+, bronze 20+. But it's plenty of time left, so actual numbers could easily be higher

[olive ledge]

AI Smart. Still not convinced we aren't living in a simulation.

[mild shale]

I think i am in the most unsolvable probs now

need to gear up the brain

[rocky jacinth]

If the universe is a simulation, then somewhere there's a computer with a LOT of memory.

[amber totem]

depends on how relative is our vision of lot of memory

[jagged sluice]

the universe runs on the computer that supplies downloadmoreram

[severe pasture]

reached 2000 notebook cell executions just for inversion

[minor falcon]

inversion i start to strongly wonder if there is really a mindgame trick of if i'm just overthinking it like i did for a few others* 😭

[inland mural]

I solved mnist in 5 minutes, but sloth consumed 7 days and gave me no clue.

The red sloth is gonna appear in my dreams😂

[grave frigate]

So
Is input_data really a useful clue?

[minor falcon]

its a question you alone should figured

[jagged sluice]

that question is against the rules

[lunar blaze]

We don't talk about the CTF here

I couldn't make any progress today 😢

[minor falcon]

the pressure around the 22 flags start to be high

[ember relic]

at least theres more variability than last years crop2

[random minnow]

Revealing the Gap between Research and Practice in Adversarial Machine Learning

https://www.youtube.com/watch?v=BxvmNlddPoE

this is very true below!!!!

[twin wren]

seems figured out a way to know if its tf or torch

[boreal spear]

granny?

[twin wren]

yes

......

💀

danm

[jagged sluice]

no its clearly run on pypeepeepoopoorch

[twin wren]

Yes

[gaunt anchor]

Now since all my exps failed to make the wolf a granny .... its time to match the local with the online ... so far I was able to match anything but the online system ... I am going to fail the granny series badly 😄 😄

[random minnow]

"seems figured out a way to know if its tf or torch" ... it could be torch.vision, keras, huggingfcae model? timm model? old lacey model? worst still some "random" model from github ... how ould you know?

[twin wren]

either torch or keras

[simple coral]

How should I approach cluster 1?

And what is the use of the model given in that?

[acoustic temple]

I figured out all of your questions, but still stuck, even I found subpopulation,

[outer sundial]

Is anyone expert in Ouija board here?

[random minnow]

Ouija board is dangerous

when i try to solve it, my PC freezes for no reason and i have to restart

i will leave it to the last

you can search for youtube video to see how it work

[timid steppe]

Quija or Ouija? Is it the same?

[grave frigate]

Which problem is this?

[random minnow]

17. Inversion
    seems like the AI is trying to tell us something!
    Try submitting some ASCII characters...It's basically an AI Quija board...

[grave frigate]

Why don't llm puzzles give flag if I have used the same prompt just a minute ago

I lost the flag on two problems

[wanton patrol]

because they are LLMs 😄

[random minnow]

let me explain

LLM are generative . say in the training data, there is promtpt='which day is it?'' completion can be 'monday'', 'tuestday' , ...

further say that 40% is 'monday', 20% is 'tuesday' in the training data

then at testing, when you ask LLM 'which day is it?'', it will answer monday 40% of the time, etc

[wanton patrol]

also, a general "hint": if you happen on a flag that is actually a true flag (not hallucination), you should stop and save your flag
that goes for both manual and automatic finding
in short: you need to improve your process

[glass bay]

in really short: skill issue

[grave frigate]

I've found the flag twice
And I forgot to store

[wanton patrol]

if it was a true flag, it does not matter if it appears again or not, it will give you a point

[grave frigate]

But if it's about probability
Then the same prompt should generate the flag

[random minnow]

"I've found the flag twice" change your code to incude automatic logging

"Then the same prompt should generate the flag"
NO !!!!

i give you another example

[acoustic temple]

what LLM is this pirate, it keeps saying "I don't know" 😄

[grave frigate]

What isthat😅

[random minnow]

you want to trick him say "the flag"

[wanton patrol]

losing something that you have already found is annoying, which makes working with non-deterministic systems super annoying -- which means you should spend time on making sure you do not lose what you have found

[random minnow]

in the trainng data, we have "prompt: tell me what thing is waving?", in the train data maybe "80% is flag", "10% is water", etc

so even if it is same prompt, it will not be the same answer

so the try in fooling LLM, is to ask questuion that the answer is near 100% what you want

but this really require to know "what the LLM has actually learned"

[grave frigate]

Can you also elaborate a bit about this if you can

[glass bay]

if 'flag' in response:
    print(response)
    break

[random minnow]

e.g.
def query(input_data):
response = requests.post('http://inversion.advml.com/score', json={'data': input_data})

    #write some code to save this (response) to file
    return response.json()

[grave frigate]

Oh

You meant code wise

[wanton patrol]

when the response contains "gAAAA...", print it out (or save to a file, if you want to make extra sure), there is not much more to it

3 different responses 😄

[wanton patrol]

but also "process wise", you should do the same when working manually 😄

[grave frigate]

I thought this is another something which I don't know

Although mine didn't have = in the end

[minor falcon]

easier: when "flag" in reponse.json() 😉

[wanton patrol]

yea, I was not 100% certain that all tasks will have the same kind of response format

[minor falcon]

wrapped in a if status.code == 200 to avoid accident crashs

[wanton patrol]

so I have gone with "gAAAA" check 😄

[wanton patrol]

and with LLMs, this will include a lot of chatter
which you might or might not want

[acoustic temple]

I got a flag which was not starting with straight gAAAA, but something like: /g /A /A ....

[minor falcon]

ah no my LLMs flags were also "flag": theflag and thats it

[rocky jacinth]

@grave frigate Why not just go back to the relevant version of your notebook (scroll through versions on Kaggle) and retrieve the flag?

[minor falcon]

it happens one or twice that a LLM hallucinated a flag with me, but when i tryied to use it it didnt gave me the point (and the format was "message": "bblablabl flag" in that case it was not working

[acoustic temple]

anyway at the moment I'm struggling with pirate flag, got solved all WTF and still can't figure out what this guy knows and what not

[boreal spear]

struggling with granny

teach me

[wanton patrol]

ask the granny to teach you 😀

[craggy beacon]

I've got banned in the chatGPT for researching security stuff in there

[grave frigate]

I got security breach detected in llm challenges 😂😂😂

[grave frigate]

I was running it in real time
And not versioning

[twin wren]

granny make me wanna cry

been solving it for 24hrs

0 progress

[minor falcon]

inversion for 3 days here

[boreal spear]

same

what is 4 5 7

[twin wren]

i am talking about granny1..

[minor falcon]

for granny all the relevant ressources are available on internet 🙂

[mild shale]

Host knows !

[twin wren]

I have a plan

but dont want to use it until the last moment

[sand solstice]

I feel like I am so close on inversion

[twin wren]

damn

finally

matching the local and server model

omg

[minor falcon]

what the current best score reached for granny 3 ?

i got a 0.00069

[fallow cargo]

same here

[rocky jacinth]

How many people made use of the model provided in cluster1? (I didn't, but could still solve OK.)

[glass bay]

Same, when I did 0.00071 I was misinterpreting, my best is 0.00069 also

[simple coral]

I have no clue to start and proceed with cluster 1 😦

[grave frigate]

Got the flag?

[tepid zenith]

i did. it's a great model

[simple coral]

In Cluster 1, I got the output as a score and not the flag key.

How should I get the flag?

[rocky jacinth]

Solve the problem, which I think means getting a 'perfect' score, and the flag will be yours.

[simple coral]

Oh ok.

[minor falcon]

i have the weirdest bug in the world

and not reproductible 😭

[glass bay]

where's that?

[minor falcon]

sometimes after my clip function to avoid idx getting out of the image, I still get a pixel at exactly 768

but what is weird is that if i rerun the code with the exact same object i passed in my clip function, it will work on the next run

aaah i think i understand

[glass bay]

pixels just dont feel like turning a wolf into an apple

tbh relatable

[minor falcon]

I was rounding after clipping

[fallow cargo]

Ouija board in Inversion is called Quija. Are there any reasons?

[acoustic temple]

I just got Piate Flag solved just by mistake lol

[rocky jacinth]

That's reductionism for you.

[manic saddle]

I need some credits for cluster 3...ik what i need but tryna get it

[random minnow]

after some hard work, i was able to match the image preprocessing of granny. you can do it without the image classification model

[simple coral]

48 hours for Cluster 1 and yet only 9% accuracy 😦

[errant nexus]

I feel myself so stupid for loosing time to get this result for passphrase

'scores': {'negative': 0.3333, 'neutral': 0.333, 'positive': 0.3338}

It should be other approach here .. It should .. I want to believe :)))

[boreal spear]

just guess the preprocessing method?

[violet trellis]

guys, is Pixelated hard or I am just overthinking it?

[tepid zenith]

didn't use brute force?

[twin wren]

I need

[boreal spear]

but how can I get the preprocess method by bruteforce

I have no idea

[craggy reef]

Hi Tim, can you explain how did you solve this problem?

[glass bay]

reinstalled both to their newest versions

[minor falcon]

or run the code in the kaggle notebook, it is inline with all the requiered libs

btw, this is also another reason to hate pickle

versions mismatches

(not of pickle himself, but of the objects you pickle)

[random minnow]

"but how can I get the preprocess method by bruteforce"
as an exmple:

we assume input image is change by transformation T (e.g. sclae,crop,mean, std, ....)

you need to think of image that can only be affected by single trasnformation

[craggy reef]

Is there any chance you can post sample code for Kaggle notebook, please?

[random minnow]

by choosing suitable images, you can uncover the trasnformation

[twin wren]

Most people can’t even find the right model

[random minnow]

e.g. if you use zero image, it would not be affected by crop,scale

[minor falcon]

no i'm afraid you need to check by yourself for this, but a little google research is enough once the version issue is solved

[random minnow]

then you can use it find std,mean

"Most people can’t even find the right model"
you have to separate model and processing

you need to find both

[pearl pawn]

Has anyone solved who came back?

[minor falcon]

from the sample of people here (not representative of all kagglers):
not solved yet: granny 3 / hush / cifar / passphrase
very few solved: inversion
difficulties for a portion of people: pickle / granny 1 / granny 2
More or less done by everybody: the rest

[random minnow]

cifar solved

[minor falcon]

ah you solved it ?

[random minnow]

not me

[minor falcon]

who ?

[boreal spear]

sample of people: Jacky

[random minnow]

i read before cifar is oslved

[boreal spear]

trolling

[minor falcon]

ah thats new then!

[minor falcon]

i would have put inversion at the top if i was the only sample 😅

[boreal spear]

I mean cifar solved

[minor falcon]

mmh i dont find anybody claiming cifar in the history of the chat

[gaunt anchor]

someone changed his/her nickname once to solved cifar on LB thats all we know

[minor falcon]

yeah someone did for passphrase and that was a troll

[gaunt anchor]

so maybe cifar is the same , not sure

[boreal spear]

i know, that's xxxxx.tech

[minor falcon]

if someone from the top had Cifar it would probably be 23 flags already 😅

[unique hedge]

what the means of benchmark_output0.0

[glass bay]

the solved cifar person in LB was trolling

allegedly

[minor falcon]

i have heavy stuff running on granny3, passphrase and inversion now, i'll just give another shot at cifar in the meantime...

[glass bay]

imo moohax should either give us all the decimal places on passphrase or lower the tolerance for equality

[minor falcon]

well tbh i think the people trying passphrase are doing it wrong somehow

its like granny1 that does not trigger under certain circumstances

[unique hedge]

I think so too. I have found many sentences

[boreal spear]

agree

[glass bay]

in short: no, i have somewhat of a proof from talking to moo, the decimal places are a problem. Won't show proof because it is hint-ish

like there is 1 approach that is correct and that some people are doing and failing

[amber totem]

agree

[minor falcon]

i got some perfect matchs too in the past, now if its just a problem of decimals and not at all about sementic, thats interesting

[glass bay]

bcs of the 7th decimal place

[amber totem]

i don't think decimals past 0.0000 are important bc api don't give them but methodology is

[unique hedge]

Me too

[glass bay]

yes, yet api imo compares them still

@olive ledge what do you think

[amber totem]

what is weird is if total != 1

[severe pasture]

day 4 of failing to solve inversion

[minor falcon]

i took my bazooka for a ultimate try

[random minnow]

as you know, there is compression defence for grannny2

i think there is defence for granny1 and 3. that is why your attack work only locally.

you didi not try defend locally

[minor falcon]

the passphrase server is more laggy than usually today, we can see where the focus of the people are 😅

[pearl pawn]

Thank you all for letting me know the difficulty level. I have faced the file and cleared 24 successfully.

[final path]

granny2 is trickier that i thought, hope to deal with today and dive into the abyss with cifar/passphrase/hush 🫡

[pearl pawn]

My remaining problems are Inversion, Granny1-3, Count MNIST/CIFAR, Passphrase and Hush.
I will resume Inversion and Granny, which I have attempted halfway through. MNIST/CIFAR if anything sparks.

[errant surge]

do I even need compilation knowledge for pickle ?

reading prompt over and over again ...

[craggy beacon]

Granny was actually pretty straightforward for me, but I can't solve sloth and cluster1😅

[gaunt anchor]

did you solve "Test" ?

[pliant quail]

@craggy beacon you solved granny1-3 ??!

[pearl pawn]

I think it is the same as other competitions in the sense that if you have data, you look at the data carefully and go through a trial and error process anyway.

[craggy beacon]

Yep, and granny is kinda obvious.

[tepid zenith]

what happened to granny suddenly

[craggy beacon]

1-2 and I am trying to semibruteforce 3 but idon't think it is the way

[outer sundial]

I also found the model which I am able to match on some images, but still can't figure out preprocessing 😅 .

[craggy beacon]

This competition is very good mix of skill and luck, which makes it as addictive as games.

[outer sundial]

https://tenor.com/view/cocaine-drugs-gif-23720718

[acoustic temple]

hey i had a question. i just started with my first problem and in the response from flag i am getting a number instead of a string. Does this mean my answer is wrong or what?

[fervent obsidian]

you should get a response that goes {'flag': 'gAAAAAB....'}

[acoustic temple]

the response it gives is {s : a random number }

[fervent obsidian]

then, you are not there yet

sorry

[acoustic temple]

ah okay ty

[rocky jacinth]

For many challenges, the number is a meaningful score.

[sand veldt]

what do you guys think the hardest one? i vote for Hush, the output shape is not even fixed, no clue of what's the server model

[glass bay]

Hardest, but doable is Hush. Granny 3 is about optimising the algorithm that is known, Cifar is a mind reading game, Passphrase is just broken/lucky/random

In hush you need to think from ground up with nothing to rely upon. IMO that is the best kind of puzzle

[minor falcon]

granny and hush are probably the most duable

cifar, I thought a lot about it and i'm probably on the right track, but impossible to reduce my searching space

you are still in Tim or you jumped to your other comp ?

[glass bay]

Still going, might catch up to 20 flags in the next few days

[boreal spear]

I jumped into school assignments

[minor falcon]

i'm wondering if it is the same encoding system used in grany2/grany3, with just the pixel limitation on for grany 3

aka: does granny 3 solve also granny 2

[final path]

If you check outputs for original image, they're the same for granny1/granny3 and different for granny2

[minor falcon]

ah ok thanks, that answer my question 🙂

i'm discovering a bunch of new stuff with granny 3, I hope one of them will pay off*

[sand veldt]

hush is also mind reading game

[minor falcon]

probably less than CIFAR 😄

[sand veldt]

cifar is much easier, at least the shape is fixed

[gray pine]

I am dieing on CLustering Level 1.

[minor falcon]

not a valid argument, the shape of granny 3 is fixed, and still, the challenge is much more complex than the LLM ones where you can feed in more or less anything :p

[glass bay]

all that strongly depends on your definition of easy

[jagged sluice]

¯_(ツ)_/¯

[sand veldt]

for granny, at least the model is known. imo, organizer at least should do the same for hush, otherwise, it's complete guessing, just randomly throw wav file, you can get any shape of the output

[amber totem]

what about inversion ? seems pretty hard too you might be lucky to find what happens fast ?

[sand veldt]

inversion, u know it's a classifer at least

[glass bay]

imo you can understand a lot about hush trying stuff out

[minor falcon]

i tryied many orthogonal ideas for inversion, always looping to the same final conclusion, so i'm a bit puzzled

[amber totem]

actually same, class 568 remains a mystery

[boreal spear]

same

[nimble matrix]

what does 418 status code mean

[manic saddle]

just an april fools code

ik where u got it probably 😂

[nimble matrix]

is it ok to get this in any of the challenge

[manic saddle]

yes

[nimble matrix]

been there since last 5 days

[manic saddle]

dont overthink man 😂

things we do when we r in pickle

[nimble matrix]

now I don't know I am even thinking anything lol!

[manic saddle]

haha i feel u

[random minnow]

"inversion, u know it's a classifer at least"
Are you sure?

[grave frigate]

Would there be any hint soon for pickle? @olive ledge

[olive ledge]

Unlikely for that particular challenge - I think there are some good resources out there once you forget what you already know about pickle.

[grave frigate]

Aight🥲

[amber totem]

if some require hints it may be inversion cifar and hush but if some solved it would be unfair for them

[glass bay]

unsolved ones are cifar and passphrase

[amber totem]

yep passphrase too

[grave frigate]

I have no clue what am I missing or what is being asked in pickle

Never have I ever explored so much about pickle than now

[ember relic]

is there some kind of challenge completed visualization leaderboard?

so i can see what most people have solved and what people havent solved

[craggy beacon]

@olive ledge Could you please check DM, I have a question about granny3

[wind ether]

Did you get granny3?

[glass bay]

nope

i meant that cifar and passphrase are probably unsolvable without a hint

[wind ether]

Oh gotcha, I see, wasn't sure haha

[tepid zenith]

Does @olive ledge also thinks the same 🥲

[minor falcon]

the problem with cifar is that if you give a hint it might make the thing too easy

im curious to know if we are not just looking in the good direction for that one

[north goblet]

is pixelated simple or i'm overcomplicating it? on it since the last few hours and whenever i feel i've made progress, it is back to square one 🤔

[tepid zenith]

It is indeed not simple

[minor falcon]

pixelated has been solved by many people, its not so complicated, but it requieres a bit of side general culture

[gaunt anchor]

{'scores': {'negative': 0.3336, 'neutral': 0.332, 'positive': 0.3344}} ... can't the server imagine for a tiny sec that its all 0.3333x 🙂

[north goblet]

Getting a lot of random BS which I'm not sure is there to guide us or not.

[minor falcon]

when you'll be on the right direction you'll know it

[wind ether]

Still nothing comes close to the 0.00069 pixel for granny3...

[granite goblet]

mnist is driving me crazy now!!

[outer sundial]

You are at 22 points, what are left from your side ?

[granite goblet]

mnist, cifar, and the 3 unsolved ones

[outer sundial]

3 unsolved are granny ?

[wind ether]

granny3, hush, passphrase prob

[granite goblet]

yeah

[severe pasture]

did you find inversion very difficult?

[granite goblet]

a little in my opinion

[severe pasture]

just a little difficult

[minor falcon]

0.000692 here

[wind ether]

Pretty sure we've all found the same pixel haha. Nothing comes close in all the ones I've looked at

[minor falcon]

actually i converged pretty quickly to this value in my last try, but then, hard stop

[wind ether]

Yep same, my model consistently converges to that pixel so I've been running a loop of convergence, blacklisting the xy values of the pixel found, and re-running. Gets a new pixel each time but nothing interesting yet

[rocky jacinth]

Learned something about the MNIST database today. No evidence of being close to solving.

[fervent obsidian]

I'm getting 403 from Pickle.

I guess sending 5000 concurrent requests is a little too dangerous

@olive ledge I learned my lesson. Please unban me.

[minor falcon]

the 403 dont stay forever, just wait a min or 2

got it quite a lot these last 2 days ahah

[fervent obsidian]

good to know. Thanks!

[minor falcon]

if you want to avoid the 403, add a time sleep (like 0.2-0.3s) between two requests

[fervent obsidian]

well, in my defense, I was being more dangerous

[half plinth]

try (retry) with this https://github.com/jd/tenacity

let the others do something too.

[tepid zenith]

CIFAR has been solved?

[rocky jacinth]

Don't think those data are available. Pity, would be interesting.

[granite goblet]

ig?

[gaunt anchor]

{'scores': {'negative': 0.3327, 'neutral': 0.333, 'positive': 0.3343}} this time with much better sentence (as meaning) :/ mmmm

[glass bay]

cifar is unsolved afaik

[sand veldt]

cifar is solvable, if one stop overthinking. hush is unsolvable I bet

[sand solstice]

Hush is the only one that hasnt been solved

?

[severe pasture]

hush, granny3, passphrase and probably cifar

[sand solstice]

afawk

interesting, so when people were ranking difficulty it was “this is how hard I imagine it is” rather than “this is how much effort I spent”

[glass bay]

it is “this is how much effort I spent”, it's just not all efforts lead to results

rip my like 20 hours i've spend on granny3 and ~7 on hush

[abstract rose]

Inversion is challenging ... we need to understand the input and the output, and we don't know which language is expected and even if it's a language

[wanton patrol]

I would imagine DDOS is not the goal in any of the challenges 😄

[wind ether]

From what I've seen, a few people got Inversion very quickly and everyone else is stuck. My guess is there's a specific trick/method to it that they picked up on

[violet trellis]

I think the same goes for other challenges (pickle, mnist...)

the trick just looks more implicit here

tbh, i guess something similar happening in passphrase. hush and granny looks a bit more advanced though.

[abstract rose]

I'm clearly in the stuck group. Prompt just says ASCII, AI, and Quija with (I guess) an intended typo.

[violet trellis]

it looks like the hint is in the title "Inversion", but I don't think i figured it out. I think this has been mentioned somewhere in this chat.

[final path]

oh, im finally done with granny2. don't know how you guys did it, but i'm not proud of my solution 😄

[olive ledge]

Not intended - I was using the French version? It's English.

[amber totem]

quija isnt in french we say ouija too

[fallow cargo]

So Q instead of O is not related to the problem right?

[final path]

It's time to ask again about Passphrase. Is "Come up with difference sentence" - difference, not different is intended here?

[errant nexus]

Welcome to cozy linguistic club of this comp ))

[dense lodge]

is it necessary to replicate the api model + pipeline exactly in order to solve granny 1 and 2?

[amber totem]

please no hints

[amber totem]

intended

[brave briar]

I have some difficulties to use the submit of type "bytes" out of kaggle notebook. Getting error : "TypeError: Object of type bytes is not JSON serializable". Any help for this 🙂 ?

[final path]

use ~~.encode()~~decode()

[brave briar]

where ? The same code works perfectly on notebook. It crashes in the json part of the request.

[violet trellis]

add .decode() at the end

[thorny widget]

base64encode(obj).decode()

[final path]

yeah, sry, meant decode. json only accept string, so you need to convert raw bytes to string (decoding them as ascii, for example)

[brave briar]

that's it thanks ! I will never understand this 🙂

[rare folio]

Good luck everyone👋

[rocky jacinth]

Two people on 23, fourteen on 22, five on 21.

[buoyant frigate]

Did somethig happen to inversion. It is returning {'message': 'Invalid input.'} for valid images

(or at least i retried some valid size images but it keeps responding in that way)

[minor falcon]

you forgot the tolist() maybe ?

[abstract rose]

In French (I'm French), it's Ouija too, (Oui is Yes in French and Ja is Yes in German). Qui is Who in French.

[buoyant frigate]

Well it was far worse. I was using the wrong query method. And now I know that I should have separate notebooks and not work on the same and switch too many times between challenges.

[gaunt anchor]

Granny1 Done 🙂

my sec approch seems to work , which I gave up earlier , then decided to recheck tonight

[final path]

Congratz!

[rocky jacinth]

Kaggle competition? Wild goose chase!

[devout jasper]

granny definitely more difficult than last year

[brave briar]

yep last year, they gave the user manual with it 😉

[gaunt anchor]

I think my approch should work with granny2 also .. will check

[gaunt anchor]

last year it was a piece of cake compared to this year

[brave briar]

Indeed but I prefer apple pie 🙂

[final path]

this one-pixel stuff seems to be impossible atm.. or just need full OpenAI GPU cluster to do total brute-force 🙂

[minor falcon]

i found a few stuff for the one pixel thing that should not need brutforcing, but not got successful so far

[wind ether]

Yeah the best anyone (that we know of) has done so far is ~0.00069, been semi-bruteforcing for the past few days

[final path]

I think there's also maybe something related to interpolation (when we convert big resolution image to smaller to feed into net)

[minor falcon]

thats actually interesting and giving me a few new ideas :p

[wind ether]

Isnt the size also a factor in the changes as well? Can only pass 768x768 to server and any resizing back and forth will cause issues from what I've seen Never mind, I get what you're saying

[amber totem]

granny 2 adding more compression mean there is 2 compressors now or preprocessing + compressor ?

[sand solstice]

finally done with everything difficulty 3 and below

[minor falcon]

at this rythm i'll solve granny 3 before inversion 😭

[nimble matrix]

You guys are legends!!
Please spare some time for the write-up at the end

[final path]

small advice - study last year materials carefully, it may help (and not only for inversion)

[wind ether]

Ehh I'd say that's a hint

[minor falcon]

nah moo says the same - its not really a hint, and especially for inversion

[final path]

yeah, you can treat it as general advice 🙂

[minor falcon]

actually he even posted a video of a guy making a review of last year challenge a few days ago

[random minnow]

i suspect passphrase is NLP adversial attack

like ganny

if you want to explain "difference" sentence

[gaunt anchor]

granny2 done

my lovely approch 😄

[final path]

it's very nice, it took you ~hour. i struggled one more day with granny2 after solving granny1. looking forward to know about good approches to grannies 🙂

[gaunt anchor]

I gave up on this approch but something inside me told me to give it one last chance and granny1 and 2 gave up 😄

time for some sleep!

[grave frigate]

I've been thinking the same
I just didn't think that this'd make any sense

[dense jetty]

why am I receiving response 403 for inversion?

[grave frigate]

Add time.sleep between consecutive requests

[random minnow]

"I just didn't think that this'd make any sense"

i think one of the ompetition host has a paper on NLP adversial attack. search for it

NEW PAPER and CODE: Introducing AutoDAN, a method that automatically generates SEMANTICALLY MEANINGFUL #Jailbreak prompts for #redteaming aligned #llms .
https://arxiv.org/pdf/2310.04451.pdf

LLM attackers ... anyone want to try this here?

i wonder if i post a notebook on this, would i get speical prize

[twin wren]

will get banned

[random minnow]

you still need to input the initial prompt yourself

autoDAN will take your inital prompt and improve on it (by testing against some LLM, which is llama-2-7b-chat-hf in their github)

[blazing rover]

Hi all, I'm coming a bit late to this competition. I solved the first three problems (okay, 4, including the free test point), but my score was 3. All my flags start with gAAAAABl and are equal in length (except test one). I'm not sure if there is anything else to pay attention to. Thanks a lot!

[mild shale]

you only have 4 falg to capture which are those

[mild shale]

maybe you forgot to run the test block and forgor to add it to the submission file

the submission file in default have something written in the value of test key

[north goblet]

Is it in {'flag' : 'gAAAAAbl...'} format? I was facing the same problem as you earlier and only flags generated in this format work, nothing more and nothing less.

[wind ether]

Everyone with 23 needs CIFAR, Granny3, Hush, and Passphrase, and everyone with 22 needs either MNIST or Inversion on top of those 4

[blazing rover]

yes correct, this form

[blazing rover]

wow , this was an issue, I haven't noticed this! thanks a lot. test flag in sample_submission.csv is not a full length

[glass bay]

there is a char limit, way less than the DAN 500+ chars

3rd attempt, new method, still granny3 gives only 0.000691

[wind ether]

Guess it's not as simple as finding a pixel or it literally has to be a certain RGB value to trigger

[final path]

I'm not 100% sure that the guy from the 1st place needs the same flags 🙂

[wind ether]

Idk i'd be very surprised if he's solved any of the four--especially since he hasn't made any new progress in the past few days.

[grave frigate]

😂😂😂

[gaunt anchor]

pixelated seems simple but can't put it togther , the many things I sent to it can hack granny3 😄 , but the the answer is either error or success ... I need the answer to be flag !

[sand solstice]

points where you know the goal are much easier I'll give you that

[gaunt anchor]

403 Forbidden - inversion server 😫

[wind ether]

Ugh I've been staring at plots for the last three hours and trying to figure out which of the letters the weird shape looks like the most... I just want to be done with inversion already...

[craggy beacon]

no sleep between requests?

[gaunt anchor]

I didn't send a request yet hhhh

it works now

[boreal spear]

457 triggered?

[gaunt anchor]

no no no 457 are not triggered (for me)

[wind ether]

No they never trigger

[twin wren]

This is not correct. the local model does affected by the image size

[random minnow]

"This is not correct. the local model does affected by the image size"
you are correct and wrong

there is some normalised size finally

the trick is to send special image to your local model and offline model and see check if you have same/different results

[outer sundial]

Done that, do you recommend any resources which I can read which can help pinpoint what kind of preprocessing might be there on API side?

[boreal spear]

insightful, now I am on granny2

don't complicate it

[random minnow]

i use to do image model deployment. we have to check image preocessing and model operations (e.g. after quantisation) are the same on software and hardware

basically it is testing with different images. but we know waht images to test. there is probably no resources, but experiences from testing

as another example, if you when to guess the "filter", pass in impluse (all zeros expect one '1')

[craggy beacon]

just google it

[outer sundial]

Tried something like that there was a bit deviation from api. around 0.001

[random minnow]

yet another example, pass in very, very big image with only one 1 at the center

if the results is same as a all zero image, we conclude there is resize

the big images has been downsize and the 1 dissapear to zero

[craggy beacon]

Most of the tasks including granny1-2 requires basic googling

[outer sundial]

Thanks for your insight.

[gaunt anchor]

yes just give google the puzzle name : say "guess whos back" , google will give the flag huuuush I didn't say that

[twin wren]

guess whos back require watching TV

[craggy beacon]

this year sloth is 🤡

[gaunt anchor]

I am so glad I beated that sloth ... last year I couldn't

[final path]

https://www.youtube.com/watch?v=L2jOEEYsOtU

[errant bloom]

The biggest difficulty for me is determining the normalization and resize parameters, if all other guesses are correct.

[minor falcon]

0.00069275 here :p

[craggy beacon]

localy?

[twin wren]

and also which one go first

😂

[boreal spear]

really

you can guess them at same time

[errant bloom]

Have you successfully matched online and local models and solved granny 1?

[boreal spear]

sure

otherwise cannot solve

[tepid zenith]

just solved it and i don't think it's about matching but rather algorithm.

[final path]

you can go either way

[minor falcon]

my convergence curve for granny 3 looks so sad

stupid local minima

[final path]

like this?

[glass bay]

wtf

[minor falcon]

i guess i can post this here its not really a hint

[craggy beacon]

this is not the way

i think

[minor falcon]

for g3 ?

[craggy beacon]

yep, but let it run in bg

[final path]

i think I saw 0.00069 yesterday with some random values, but cannot reproduce 😄

[minor falcon]

i have a more complex version of my implementation to build, but yet did not understood it completly... 😅

[random minnow]

"The biggest difficulty for me is determining the normalization and resize parameters"
maybe there is more .... i am no sure if there is defence mechanism?

[glass bay]

there is 100% more than that

[twin wren]

i hate this competition, but i am so addicted to it

[glass bay]

not saying what obv

[twin wren]

😭

[random minnow]

what if in granny 3: image (i pixel difference)--> processed image (still 1 pixel?) --> model

[glass bay]

bruh

[twin wren]

crop

[random minnow]

"bruh" you mean same score also cannot work?

[glass bay]

it's just funny how even the probs don't add up to 1

[random minnow]

oh

[glass bay]

also i've managed to make them line up perfectly before

twice

so imo that's a rounding error

[random minnow]

i have a feeling passphrase is not about getting the score

it is about geeting the correct phrase

maybe we are all tricked by the host

[final path]

or it's about getting the score AND correct phrase. both (input/scores) are important, as orgs said in Kaggle thread

[random minnow]

i try imagine the pipline:
recevieve text --> tokenization -->setiment classifier --> ??? ---> print results

or maybe
recevieve text -->??? --> tokenization -->setiment classifier --> ??? ---> print results

[final path]

Also - Equality means "a is b". Equivalence means "a is like b"

[minor falcon]

i agree with mikhail, its probably like for granny, having perfect score is not enough

[final path]

if phrase == 'XXX': return flag else return sentiment(phrase) 😄

[random minnow]

we need lev-distance(benchmark, send) < ????

[minor falcon]

i think we should still be carefull not to brainstorm too much guys, would be pitty if we get all banned for exchanging ideas 😅

at least given the time i spent on the challenge, i would be a bit disapointed aha

[random minnow]

next time let's encode our idea using inversion or using sloth image

[minor falcon]

inversion, so close and so far at the same time

[final path]

Hope that our discussion was shallow enough c:

[minor falcon]

well as far as I see nobody still understand passphrase so... :p

[random minnow]

i having a feeling that discussion are not correct. what we can think of are probably already implemented by the top kagglers

looking at the leaderboard, my conclsion is that everyone is more or less the same. it is the speed

and everyone stuck at almost same problem

[minor falcon]

ah yeah, and a lot of top competitors are here anyway

the tie breaker for the money will be if someone figure out CIFAR probably ahaha

maybe hush, but not a lot of person are interested by it so far

i guess that people stuck in inversion like me dont want to start hush before they complete inversion

[random minnow]

i don't the the "BOSS question" are pure optimzation problems

but the BOSS question does need optimzation ... that is why the competition is 1 month

slowly torturing kagglers is the aim 😂

[minor falcon]

clearly, i thought this competition was a sprint, but it looks more and more like a marathon

[final path]

Hush looks to me as most solvable atm, it looks complex, but solution may be easy, if you spend enough time on it. Unlike CIFAR, with very simple statement and infinite search space

[minor falcon]

the good stuff about grany is that its pretty well documented

and I imagine that the org would not have gave an image for which there is no solution

[glass bay]

imagine if that is exactly the case lmao

they've specifically made it so there is no solution

[minor falcon]

nah, they would not have chosen the class "granny" otherwise, who would go for that? ahah

[glass bay]

that'd be very sad if i have to guess the correct sentence in passphrase since it kind of turns into semantle 3 and all my tricks i've used for passphrase are compeletely useless

which are mostly "ignore the sentence structure"

[final path]

when I think about existent solutions, I also think about CPU/GPU time orgs could've spent on that. And how do you create such a problem (1px solution obv does not exist for every image/target pair)

[minor falcon]

i tryied so many things for passphrase completly out of the box and not related to what most people do ahah

[boreal spear]

imo passphrase only has one solution, like a password

[minor falcon]

i dont think so, i think there is a bit of flexibility, like semantle 2

[random minnow]

the problem is not "how to do", but "what to do"

[naive umbra]

Ugh all LLM problems went quite well so far but witf3 gives me a headache

[grave frigate]

I know you are....😂😂😂😂

[acoustic temple]

wait until you reach to IP

[severe pasture]

pretty sure I've got the first half of inversion but I'm still just as confused

[errant nexus]

I've used genetic approaches to grow new phrases. Neither one nor other scores work. It is definitely not only about scores we try to reach. Something else I guess.

[violet trellis]

Pickle.. having no goal is painful🥲

[tepid zenith]

found the secret sauce: https://en.wikipedia.org/wiki/Arrabbiata_sauce

[fallow cargo]

i can teach you how to cook it perfectly if you share with we the inversion solution ahhahahahah

I'm joking oc

[tepid zenith]

1/{solution} it is

[glass bay]

god why ocr is sooooooooooooooo inconsistent

[minor falcon]

took me 4hrs to come up with the solution once i figured out the way 😅

[glass bay]

i know the exact way it should be solved, it's that making it OCR to exact characters is REALLY hard

[rocky jacinth]

You'd have to have a clue what they're looking for, or just get very lucky. Nostalgic for DeepFake from last year.

[glass bay]

it may interpret say a "1" as l, i, brackets of all shapes and sizes, and all i did is made one pixel elsewhere from 0,0,0 to 0,0,1

[tepid zenith]

there was a hint shared to make it exact 😉

[glass bay]

do you mean the old london font?

[minor falcon]

that was me and it was trolling (even if I actually tryied that font for inversion 😭 )

[tepid zenith]

going back to passphrase

[amber totem]

i can't reproduce granny 1 preprocessing, sometimes i'm close but not everything match, but it should be somehow not too complicated, a lot of persons mastered it so i don't know where to go

[glass bay]

on pixelated i feel like i'm super close, and seems that either i'm going completely in the wrong way, or i'm just a tad bit off

[minor falcon]

same feeling on inversion

[random minnow]

best explanation on what is base64 coding: https://medium.com/analytics-vidhya/crypto-basics-understand-create-your-own-base64-encoding-with-python-a1481686a35a

[amber totem]

thank you

[gaunt anchor]

I been playing around with CIFAR , I have hunch about some of the inputs .... but still we need to go inside the challange creater head :/

[random minnow]

i wonder waht is the objective of cifar?

[civic apex]

Hi guys, it seems like I was very late to the party here. But I feel like someone will clear my doubt.. what this competition is all about, aren't we building any ml/dl model in this. Why all the example notebook adds up API calls and what the heck is a flag...?

[amber totem]

Hello, you should read the description of the competition it is well explained

[craggy beacon]

https://tenor.com/view/mujikcboro-seriymujik-gif-24361533

[final path]

Ultimate goal of this comp is to train LLM which gets all the flags for you

[minor falcon]

gpt has been already kind of helpful for one or two flags, but clearly lost for others, in that sense, the comp is well designed

[severe pasture]

yeah gpt pretty much got 2 of my flags for me

[jagged sluice]

Only 2? Rookie numbers

[final path]

for me it was very helpful in writing tons of code 🙂 and made a good hint for pixelated (though it was like accidental)

[minor falcon]

its good at brainstorming and generating code yeah

you guys use the paying version ?

[final path]

yes, gpt4

[glass bay]

damn i sure am missing something by not using gpt

[amber totem]

gpt4 got me all flags that arent llm related

[glass bay]

since like you know i wanted it to be my test of skill and knowledge

[minor falcon]

gpt4 is really making my life of dev easier since i use it

[amber totem]

btw still not easy with gpt, you have to understand the code you asking to write

[minor falcon]

but honnestly for this comp self reasoning > blind usage of gpt

[final path]

i heavily started using it for this comp and I totally enjoyed the experience 🙂 saves plenty of time for boilerplate stuff

[minor falcon]

i used it a lot when I want to learn new stuff or for helping me with front/db stuffs

[amber sapphire]

Hi, finally confirmed account! 17 solved...are there some hint for inversion 568? I've tried almost anything...

[glass bay]

nope no hints here

unless you search through the channel and the semi-hivemind brainstorming that is constantly going on here will lead you onto something

although there were no hints on inversion here on the 568 stuff

[minor falcon]

i have a llm bot scrapping this chat to send me a direct alert on my phone if someone post a clue so even if it is delete i receive it, and no, nothing

[amber sapphire]

Ok, I will check e continue trying...

[minor falcon]

no just kidding, but seriously considering it 😭

what i'm the most curious about is if the answer is case-sensitive

last year seemed to be full upper letters

[amber totem]

bot is illegal if people don't know and agree the existence i think but great idea :))

[craggy beacon]

I asked ChatGPT about RCE in pickle and other security stuff and openai banned my account for that

[amber totem]

kinda hard

[minor falcon]

i didnt have prob about that with gpt4, but i was always precising in my prompt it was for a CTF challenge

[amber totem]

for me starting with "i'm doing a legal ctf on kaggle" resolve all

[minor falcon]

actually this "im doing a ctf" is a very good way of getting censured information on gpt aha

[final path]

yeah, agree, I stated that im doing CTF and gpt4 was OK with dangerous pickles 🙂 kinda jailbreak

[severe pasture]

"I'm participating in an AI-themed CTF" never fails

always gives good answers

[craggy beacon]

That conversation was named CTF challenge. But it sometimes forgot the context if conversation is too long

[glass bay]

to break llm one must break another llm

[minor falcon]

the king of all llm

[sand veldt]

Question for non English native folks, can you understand the story of problem description of passphrase and hush?

[amber totem]

yep

[glass bay]

i think putting through a translator is enough

[sand veldt]

really? i feel it's like TOEFL reading comprehension

[amber totem]

if you don't, you can translate it with gpt it is good at translating expressions

[sand veldt]

translator can only do literally translate

I feel miss some culture thing

[amber totem]

but i feel like the problem isnt language

[glass bay]

translate word by word, extrapolate

[brave briar]

where are you from Roger ?

[sand veldt]

are you English native? so you don't see the second meaning of the story?

[final path]

I'd like to see a translator which could keep the semantics of "difference sentence" 🙂

[sand veldt]

right, sth like this

[glass bay]

being honest, tasks would've been solved by someone by now if it was a english native thing

[severe pasture]

did we ever get clarification as to whether that's a typo or not?

[sand veldt]

southeast Asia

[glass bay]

funnily enough we didnt

[final path]

From orgs - no.

[minor falcon]

we actually don't know if it has been solved or not, maybe a genious mind got passphrase/CIFAR but not granny1/2

[glass bay]

depends on whether or not the orgs are native english speakers

[jagged sluice]

hehehehehaw

[jagged sluice]

I believe they are

[random minnow]

i realise one thing:

10. Passphrase
    Once upon a byte in the quaint digital town of Bitville, there was a legendary sauce so divine.
    The sentiment about this sauce became so powerful that it was encapsulated into an AI model.
    However, a quirky chef from Bitville decided to tweak the model a bit and made it a tad more... mischievous.
    Now, it's up to the brave souls of the digital realm to decipher the sentiment scores of this notorious sauce and uncover its secrets.
    Do you have the appetite to take on this saucy challenge and charm our capricious chef's creation?
    Spice up your skills and dive in!
    Come up with a difference sentence where everything is equivalent.

[sand veldt]

LOL

the host keep saying reading the prompt carefully, but are we supposed to be hacker? instead of taking TOEFL test?

[random minnow]

in plain english it shold have been:
Once upon a time in the town ...

[minor falcon]

hengck, you are again brainstorming too much, be carefull

[sand veldt]

sloth/cifar/... those problems are good, without reading comprehension

[jagged sluice]

just read better ez

[random minnow]

words are replaced ....

[glass bay]

words are replaced because the person who wrote them is quirky

nothing more

[brave briar]

no seriously, it seems that native english speakers are not understanding more the Passphrase and Hush prompts.

[random minnow]

Passphrase means paraphrase?

[glass bay]

maybe a mix between paraphrase and password

and tbh that is what it is

[amber totem]

passphrase refers kinda to secure zones on servers

[random minnow]

oh

[sand veldt]

yes, that's GPT translated, like password

[random minnow]

i didn't know there is such a word

[amber totem]

or like crypto wallet

[sand veldt]

checked, not in bip39

[minor falcon]

random stupid question

if you attempt a brut force on multiple list, is there a difference in term of chances to find a solution faster if we iterate simply, or if we generate a list of all the solutions, then shuffle, then iterate the shuffled list

[amber totem]

if you know what is the goal yes but if you don't know i don't think

[minor falcon]

i have the feeling that the two methods will not necessarly generate the same distribution in term of solving

[final path]

for random string its probably does not matter, if it's some real world stuff, it's better stick to word/char probabilites.

[minor falcon]

say you look for a looooon binary like 101110101001
if you do iteration, you will wether find the solution quickly (you got the first digit right) or very late (the first digit was the other one, so you need to loop through all wrong solutions first)
while if you go with shuffle, each iteration has the same chance

[brave briar]

It is problem dependant. I think your point is to find how to traduce some insight on the different elements proximity to smarter exploration than a basic brute-force. But if you have no insight, there is no difference.

[amber totem]

"while if you go with shuffle, each iteration has the same chance" depends if you remove from the possibilities what you have already tested

[minor falcon]

ok let me rephrase with the binary example:
you look for a binary that is x digits long, so there is in total 2 ^x possibilities.
you do:
for i in [0,1]:
for j in [0,1]:
....
check(binary)
if you picked the good first i, you will have the solution in the first half, otherwise the second half, so the solution will normally come up fast, or take a loot of time (as you need to go through all intermediate example)

If, instead you do
or i in [0,1]:
for j in [0,1]:
....
list.append(binary)
shuffle(list)
for elmt in list:
check(elmt)

Then you have the same chance at each element coming, which was not the case before (i guess?)

[brave briar]

I will reformulate your binary example. There is a vanilla card game, you look for the King of Heart. The game is randomly initialized, do you want to shuffle once more before you start drawing ?

[amber totem]

for me that's the same

the second is just longer calculate

[minor falcon]

yeah you are probably right

[amber totem]

to be more precise, it depends of the position of the solution, but if you don't know it, both solutions are the same

[craggy beacon]

i don't think that ocr is bad in pixelated

[sturdy gorge]

Hello, just started this competition and im having some trouble understanding what the flags are supposed to be.
In the ctf-started-aiv it is stated that flags will be obvious and will begin with the following pattern gAAAAABl., however, what im obtaining as output from the cluster1 problems is completely different from the expect gAAAAABl pattern. Could someone help me figure out sort of confusing im making here?

[severe pasture]

That means you haven't submitted the right input in order to get the flag - I'd recommend starting with test, cluster2 and the "what is the flag" problems first

[sturdy gorge]

Oh alright, I assumed they were ordered by difficulty, thanks!

[amber totem]

start with test 🙂

[wanton patrol]

The same, I don't understand the complaints 😛

[sturdy gorge]

THis is probably more a semantics questions so I hope it's ok to ask, but in cluster1, when it says Find which subpopulation is being systematically misclassified., do they want the ids of the misclassified subpopulation, or the ids of the entire subpopulation that is mostly misclassified?

[final path]

you need to figure this out on your own 🙂

[sturdy gorge]

thats fair haha

[final path]

i spent 2-3 hours trying to understand what format is expected in cluster1. it's a bit too complicated for the second problem 😄

[sturdy gorge]

Im also struggling to figure out exactly what they want as a submission 😅

[amber totem]

Look carefully at the output and do tests

[rocky jacinth]

No harm in trying out a theory - if it's not right, move on to the next idea

[gaunt anchor]

pixelated and it's ocr is driving me crazy .... after a lot of edits to push the ocr to right direction finally it matches the needed input which I read a lot to understand what to do .... no flag 😦

[jagged sluice]

¯_(ツ)_/¯

[gaunt anchor]

it will work in the end ... worst case the competition ends and I see the solution ... so will work in the end

granny 1 and 2 worked (when I thought it will never work) so it will work ... or I will hack it !!!

[wind ether]

Looks like we're on the same step. Just to ask--are the non brute-forced values clear to you or are you choosing between a few?

[severe pasture]

I'm on the same step, choosing between a few