learning-agency-lab-automated-essay-scoring-2

[crisp bloom]

Hello, i just joined this competition.

[hollow steeple]

Hey all Sarvesh this side a PhD AI fellow from IIT Bombay, I am looking for a teammate. Someone who's well versed with coding (should be able to write a code given the algorithm) is welcomed. Would prefer to have someone having at least a Master's degree in AI/ML or is in the industry for a long time and has atleast done a proper submission for the competition (by proper I mean not just a public nb submission)

PS: Currently I am ranked 13 on lb, obv a lot of potential to improve the rank, and a lot of exploratory attempts yet to be made

[turbid knot]

sure, bhai would love to help out

[turbid knot]

turns out new entrants are not allowed @hollow steeple

[hollow steeple]

I am looking for someone who is already participating and has a decent rank without just copy pasting the publicly available nb

[turbid knot]

cool dude, I hadn't participated in this competition

so I can't team up

what's your team name, can help people look you up

I can't see you on the 13th LB

[hollow steeple]

Probably you are looking at the leaderboard of some different competition.
My position is 9th right now may change as time passes

[turbid knot]

correction : you're 7th buddy , congrats

yeah there are 2 learning agency lab comps

one is for PII another is for automated essay scoring

[hollow steeple]

No longer 7th lol, someone posted a publicly available nb with same score and people are just submitting that, but NVM as the methods are different, always can improve over it

[tough coral]

Can sb explain what is wrong with the Efficiency Score equation? It's already more than three weeks and it's hard to tell what is the score really about 😫 I'm surprised there is hardly any discussion around it, when it's about half the prize 😮

[jovial rover]

Hi everyone!

I am new to kaggle depsite research experience in ML at university + bigtech (ICML/ICLR), and I am looking for advice from more experience people.
I wanted to ask you how you approach hyperparameters tuning. For now, I am using optuna to tune the main hyperparameters of a lightGBM model with a ~sensible set of values. I am curious if you are using a different approach or have advice

Thanks in advance for your help, I hope this channel is suited to ask such a question 🙏

[halcyon finch]

Hi, guys! I'm participating in my first Kaggle Code Competition, so I'd be greatful if you could help me, a beginner, understand the requirements for an elligible submission.

From what I've read in the Kaggle documentation, it is necessary for the submission notebook to be ran "top to buttom" in less than 9 hours of CPU / GPU runtime. That means that in my submission I should train the model on the training dataset and also predict on the test dataset in less than 9 hours, right? So, if I'm using an ensemble solution, I should manage to train all my component models in the time limit. Or I'm getting it all wrong?!

I'm asking this question because I've noticed in this and other competitions' code tabs that there are public inferrence-only notebooks that import model(s) trained elsewhere (uploaded as Kaggle datasets) and use them directly to predict on the test dataset. This shortens the total runtime of those notebooks. Is this kind of notebook allowed to be a final submission? Or is this just a way to avoid exhausting the GPU weekly quota while also allowing one to see how well their predictions perform on the public leaderboard and also making certain notebooks public for the community without revealing too much of the training process used.

If this isn't allowed, then how are my submissions supposed to compete with these sped-up notebooks, with high public scores, especially in the efficiency section of the contest?

Thank you in advance!

[drowsy imp]

@halcyon finch Your submission does not have to train a model, it only has to inference on the test set that’s provided at runtime and is kept private, as long as your notebook references the test.csv file in the linked dataset and generates a submission.csv in the right format you should be good to go! 🙂

[hollow steeple]

Just wondering what parameters are you tuning, coz there are a large number of parameters.

PS: Would like to know more about your ICML/ICLR works as well (gonna submit soon at NeurIPS, and planning at next ICLR)

[hollow steeple]

I don't find any mistakes as such. What I can comment is that the score in the table is the score on public lb, and the ranks are based on the modified evaluation metric which considers time as a parameter as well

[jovial mason]

Hi all, I think there is an error in many LGBM baseline NB , in word tfidfvectorizer : char n-gram are used instead of word n-grams, made a NB to show this, would really appreciate feedback ? https://www.kaggle.com/code/guillaums/error-in-tfidf-vectorizer-in-baseline-nbs 🙏 thanks !

[shy stone]

Hi everyone, kinda of weird question but, how do I trace the error when submitting? My code version saved just fine with logs and everything, but when submitting it threw an exception for some reasons. Thanks!

[digital rock]

The efficiency leaderboard hasn’t been updated for 5 days. Is there a problem?

[hollow steeple]

Any updates on this?

[pseudo badger]

Hi, why does the test file have only 3 observations when the comment says there should 8k observations

[hot vessel]

I'm new to submitting, I have a pretrained word embedding dataset fed into an MLP, which doesnt take too long to train on my end, yet its been over 30 minutes in the submission, is this normal?

Oh it looks like the accelerator is set to None which I guess means the model is training on the cpu

[frozen mortar]

I joined competition 4 days ago and my first time doing data science is 0.771 score good and how do i get more?

it seems all my models seem to reach a point where they wont improve anymore , I converted the text into 36 numerical parameters(word_count, spelling_mistake_count etc) but i cant get any higher score

also visualising graphs didnt help either everything looks mixed up

[hollow steeple]

I am looking for folks to collaborate on learning agency lab.

The basic requirement would be, being in the top 35 at least on the public LB, with enough Kaggle GPU available. The major reason to look for a teammate is the 30 hr limit on Kaggle, and the data being private for publicly available nb makes it impossible to work in local

[fierce pine]

Hi, can anyone confirm if number of spelling mistakes correlated with score?

And is the language US English?

[earnest garnet]

anyone here haha? I had a question, maybe in general too... so if I let's say had embeddings for my train set, would I have to run that over every time when I submit or can I save the embeddigns on my kaggle directory and just look up the numpy array when I submit?

yes english

[earnest garnet]

I'm getting great CV without LB stuff, but might want partner. do you have one?

[dense pike]

you could save embeddings for train and just load them, but you still would have to run your models for hidden test obviously

[earnest garnet]

got it, thanks!

[hollow steeple]

Let's talk about more in DM, right now I am unable to beat the public lb, but ya it seems to overfitting

[hollow steeple]

I see you have sent a friend req, but didn't receive any dm yet. Probably you can let me know the cv after which we can talk about method

[earnest garnet]

I messaged you

[limpid wraith]

I found in a public notebook(0.824) that this line of code: predictor = Predictor([light, xgb_regressor], n=0.709), I was very confused about n=0.709, does anyone know how this is calculated? Thank you very much

[dense pike]

fitting to public lb most likely, he changes thresholds a bit and posts higher score from what I observed

[limpid wraith]

Is there any connection between the threshold and this weight？

[limpid wraith]

Hello, I want to ask you, after my notebook is set to Utility Script, and I run it successfully, there is no.py file in my output folder, what should I do, thank you very much

just like the image

[sleek nacelle]

3 days left in the competition and the best public forkable score is in the gold zone. Haven't seen anything like this before