❓┊ask-a-question

[lost crag]

thanks

[wraith sparrow]

https://hyperskill.org/courses/33-math-for-machine-learning

[last fern]

Is it ads?

[wraith sparrow]

only the yellow portion is ads (well its not adds its offer they saying )

but the maths course is free

as it ve no projects

only projects are premium everything else is free

[celest plover]

I have two questions for you nice people. If I upload an LLM model, will it be visible to everyone? I am trying to find the model of a competition winner, and I cant. Also is not open source (models) stronger (>) than open weight? why will someone call a model like llama 3.1 open-weight but something that was a fine tuned version of that, will be called opensource??

[silver sky]

for a person who did not get to study maths properly at college level, is the following plan for learning ML/DL to get a job workable?

Courses:

1. Math for Machine Learning from Hyperskill
2. Practical DL for Coders from FastAI
3. Maths for AI videos by Weights and Biases

Books:

1. Hands-on Machine Learning by Geron
2. Calculus Made Easy by Silvanus Thompson

Practice:

1. Kaggle

I have worked with Python, Numpy, and Pandas.

[pseudo storm]

Is it allowed to train ML model on a huge super powerful GPU cluster and then use it in a competition? Or all model training should only be done inside kaggle notebook and on kaggle (with limited resources, 9 CPU/GPU hours)? It depends on competition rules?

[wraith sparrow]

i guess its mentioned in competition rules

[silver sky]

is there a way to refresh the data in my kaggle account? or do i need to delete my account and register once again?

[wraith sparrow]

cannot import AutoModelForCasualLM in kaggle notebook any fix ?

[gilded fulcrum]

Hello I'm beginner with machine learning also kaggle. I'm not able to ask my questions on discussion and send my outputs to dataset part. would you please answer me what to do ?

[wraith sparrow]

ask

[clear sonnet]

hello kagglers, i have a quick question regarding the titanic competition. i see many poeple on the leaderbord with a score of 1.0 with only a single submission. when i compare the results of my code to train.csv, i get a score of 0.94, but only a score of .76 when i submit this. is train.csv accurate in those who died and survived?

[rain oar]

you might be overfitting your model and getting a good score using the public data, but when they run your model on private data it does not do as well

[rain oar]

are you having an issue submitting to a competition?

[wraith sparrow]

@rain oar

[rain oar]

@wraith sparrow Are you spelling "Causal" correctly 😄

What is the error and what did you try to do to debug

[wraith sparrow]

thanks got it 😅

[rain oar]

happens to the best of us

[cedar quail]

Hi my name is Mike and i am new to Kaggle. I just started my first trial with gradient boosted trees. I have a question about the tree structure, does it change during iterations? I looked at several guides and asked in the gpt chat, but I did not get a clear answer. Below I have attached a screenshot of the slide from the video. Can the primary structure of the features that the arrows point to change, or only the leaves/branches of the residuals change?

[wraith sparrow]

getting the following error which specific version of hf's bitsandbytes library to install or any other fix ?

[honest crypt]

please help me solve this issue

[late vale]

hey i have a lot of data that i need to collect from an api for further analysis. i am wondering what is the most efficient way to store it.
the data collection consists of a lot of request and therefore takes a very long time to run so in case i shut down my pc i dont want to have to run it all over again. currently i am saving the responses as a dictionary. how can i store the data so i wont have to run the data collection script every time?

[rain oar]

the trees are different each time. So instead of the top being "Gender=F" it can be "Color not blue" for instance

[rain oar]

save the data as a csv file and download it to your local machine. Then when you re-open a notebook, upload the downloaded file and open it as a dictionary or whatever

[late vale]

thank you

[wraith sparrow]

@rain oar

[mystic shuttle]

hello, is there a way I can monitor the VRAM from the console? I cannot use nvidia-smi

[glass crystal]

Task manager ?

If Linux you could try htop

[mystic shuttle]

inside kaggle

[rain oar]

have you tried reading the documentation provided for that error?

[glass crystal]

Maybe you can look for torch cuda library that has function to call for allocated memory

[tiny geyser]

Hi, yahoo finance has an api that collects stock market data

although ML/DL problems aren't easy for stock market data since it such a volatile problem and the feature space is so large and constantly changing it's almost always difficult to capture patterns that will represent the stock market

https://github.com/ranaroussi/yfinance

[tranquil gull]

thanks man, i’ve already familiar with the yfinance dataset so i was wondering if there were any different dataset available on the field

[tiny geyser]

I don't think so tbh, If I come across something I'll make sure I pass it along

[tranquil gull]

okay, thanks for the help!

[primal tinsel]

Hi, im trying to start an execise about Pandas and the first cell never load.
Can someone help me?

[rain oar]

sure, what exactly is the problem? Have you tried reloading the entire window?

[primal tinsel]

yes i have tried it.
Can i send a picture?

[rain oar]

Sure

[rain oar]

does it keep spinning and never complete?

[primal tinsel]

yes!
Also, i cant see the print message "setup complete."
So, i think it had never completed

[rain oar]

A good way to figure out these kinds of issues is to only run one line at a time. Click on where you see "+ Code"

it should open a new code block above this one you're trying to run

then put in just one line import pandas as pd and try running it

see if that completes

Also, use a fresh window

[rain oar]

that works too 😄

[primal tinsel]

ill try in a new cell, as you asked for

same 😦

[rain oar]

You might also need to check to see if your Draft Session is running

[primal tinsel]

it keeps always like that

[rain oar]

That's probably the issue. Try to refresh the page. Then go to the Run dropdown menu, select "Start Session"

try to start the session that way, instead of by trying to start it by running a cell block

[wraith sparrow]

patience would be the key (it could be due to slow network too)

[primal tinsel]

didnt work

[primal tinsel]

i spend more than 1 hour waiting haha

[rain oar]

might be an issue with your browser setting not allowing the session to start?

[primal tinsel]

no idea 😦
But ive downloaded the dataset, im runnin it a local jupyter-lab.

[rain oar]

Lots of potential solutions if you search "Kaggle session won't start" or "Session won't run" in discussions

sorry I don't have the exact answer!

[primal tinsel]

no worries

im glad you

[rain oar]

If you're still stuck, there's always the #1130785765274685500 channel

[umbral junco]

Hi there, I’m training food detection models and looking for a dataset with detailed nutritional information. Specifically, I need a dataset with fields like:

Barcode or unique identifier
Macronutrients (e.g., calories, protein, fat, carbohydrates)
Micronutrients (e.g., vitamins and minerals)
Dietary tags or allergen information (e.g., gluten-free, vegan)

I tried using the OpenFoodFacts dataset on Kaggle (https://www.kaggle.com/datasets/openfoodfacts/world-food-facts/data), but it seems to be either pulling from an API or lacking some of the specific structure I’m looking for.

Is there a dataset anyone would suggest with more comprehensive information?

[digital pecan]

Anybody doing the Jane Street Challange? I'm new to Kaggle, i had a question about the public score. They say it's ranked using weighted mean R^2, and the highest score rn is 0.0053, does that mean all models are doing really bad so far

[wraith sparrow]

Lets say i ve saved a notebook and i wanna improve say new version if i edit n save it will there be both versions or just recent one ?

[wide drift]

Hello, I'm curious does the kaggle coding environment only support python? Is there a way to use R code in it?

Nevermind found it!

[rain oar]

if you click the number next to "Save Version" you will see the history and you can access the previous saves

[earnest cliff]

Hello, I am learning data analysis, I currently work for a casino as an assistant, but I would like to ask for recommendations to improve my reports, I don't know if I can share here something of what I have done to be able to get some recommendations, I currently have a good level of Excel, some Tableau and some Power BI, but I want recommendations on how to improve.

[olive tinsel]

Heyy i have been working on a assignemnt and I am not being able to run adversarial-robustness-toolbox in kaggle, Can anyone help me out? https://www.kaggle.com/code/ashuradhipathi/assignment

And my gpu is not being used for some reason

[graceful axle]

So I guess your GPU needs to be sorted out before running the Tootbox?

[olive tinsel]

I think my ip got blocked bcz i was trying to run continuously since yesterday😅

So if I just access with another wifi it might help

[olive tinsel]

The pblm is with scipy version, art needs an old version of scipy, I tried downgrading scipy version

[graceful axle]

Oh, I see. Did it work?

[olive tinsel]

Nope

[olive tinsel]

It didn't

[graceful axle]

I checked your notebook, and it looks like TensorFlow ain't installed. Try installin' that, Fam

[olive tinsel]

I installed it in the middle cell

And it's the second version of the notes I guess

I am not being able to use gpus in colab and kaggle irrespective of what acnt I use

[graceful axle]

I see "ModuleNotFoundError: No module named 'tensorflow'" after cell 7. Go check the notebook and make sure to insert the model right.

[olive tinsel]

My issue isn't with that part, it's some random version
In my first version I used pip to install art and trained the models but I faced issue with predict function of randomizedsmooth classifier

Which had wrong scipy.stats version as the import from it was having a different name

[graceful axle]

Where yo second version notebook at? Share it here, and I’ll see if I can help you out.

[neat cobalt]

Hello, I'm currently working on a project that requires multi-modal vision. I'm using pre-trained models but I'm unsure about how to fine-tune Vision Language Models (VLMs). Can you help me with the fine-tuning process and the structure of datasets needed for this?

[olive tinsel]

Yeah thank you https://www.kaggle.com/code/ashuradhipathi/notebookd7ce958bf3

[finite pond]

hi kaggle community,
i am wondering why kaggle don't provide as with classic ML courses like lineare regresion and classification

[inner token]

Hi community! I've copied a notebook and I'm having some problems with the interact widgets. The code is the same, but it's not appearing in the output of the corresponding cell.

I copied and edited several notebooks that use the functionality of widgets to interact with the plots.

I didn't change any code at all, but the widgets still don't appear in the output of the cells where they are defined.

As the output of the cells, you can see a description line, here is an example: IntSlider(value=53, description='Int Slider 1', min=1).

Since no error is specified in the output or in the console log, I don't see what the solution could be.

[inner token]

Here's more info: when running on localhost, the widgets work.

[olive tinsel]

Actually I forked the library repo and made changes to it, it's working thank you

[compact snow]

hmm.. understood.. thanks..!

[robust anvil]

What kind of IDE and it's package at large do you normally use to develop the projects. I probably thought 💭 of changing for better .

[sick trout]

Hi there
I need help in kaggle jane street as there are so many partition_id which one i should chose to predict responder 6.Thanks

[wraith sparrow]

Pycharm professional would be best option

[slender phoenix]

I'm having issues installing optuna, is this not command to use?

[willow relic]

Hey anyone,

I have a set of question-answer pairs, and the idea is to pass our questions and documents to a library that uses an internal LLM to generate responses. Then, I’d like it to compare these generated responses with our existing answers using a default or customizable metric.

Is there an existing tool, library or approach that would facilitate this type of comparison?

[hollow nebula]

Hey, did anyone have an interview in Python? Can u give me some tips about it 😅

[real shore]

prepare for coding questions

[hollow nebula]

What kind of questions should I expect at live coding interview? I don't have anything infront to do, so I will find out what I'm having there

[real shore]

will dm you more about that

concentrate on DSA

[rain oar]

You could either keep working on Python with Kaggle competitions or go specifically to a data science coding prep site like stratascratch

[bleak grove]

I am facing this issue too. I added a bug in #1130785765274685500

[obtuse kayak]

Hi

[obtuse kayak]

im a beginner i only know about the fundamentals of the data science part so can someone help me to learn more

can some one include me to their team or tell what all i shouls be knowing

[rain oar]

You might have better luck finding a team on the #👥┊looking-for-a-team channel, but it's a hard sell for most beginners who are looking for mentorship. In the meantime, I suggest going through the "30 Days of ML" provided by Kaggle, as it's a good way to get up to speed with Python, ML, and Kaggle competitions. To follow it, I created a post here: https://www.kaggle.com/discussions/getting-started/541178

[pastel plank]

Thanks Tom

[brave lodge]

I signed up for the Gen AI Intensive course scheduled for next week. I haven't received a "Getting Started" email that was supposed to be sent a week before the course starts. Has anyone else received a "Getting started" email - is there any other page where I should be looking besides the course page? https://rsvp.withgoogle.com/events/google-generative-ai-intensive

[lone pewter]

Hi guys
I am building a pix2pix model that can generate maps from satellite images. The problem I am facing is for every step certain amount of ram is used and accumulated, which is causing it to run out of memory too soon. usually in other deep learning models the model use a constant amount of resources during the entire training but while in gan the ram is continuously increasing as a countdown and then runs out of memory I tried to clear session but didn't help I have seen other ppls code one among them worked fine which was completely different implementation can someone help

[untold lynx]

Same here, I haven't received any starter email.

[verbal crest]

Nothing to worry about, we ended up a bit delayed but the first email for the course will be going out tomorrow with tips to get started, then the first day of the course will go out on Sunday/Monday (depending on your timezone). Please make sure to put questions about the course in #5dgai-q-and-a so course moderators can see them.

[noble jacinth]

im not able to download my output files, it gives me 404 error when i click the link for it

[inner token]

thanks!

[safe yoke]

Why did I jump out of this screen when I clicked ”Submit Prediction“? how to solve it

[lofty plank]

Yes what will be starter email contains

I am still waiting for start

is identity verification compulsory for kaggle ?

[spring dome]

You will be assimilated. Resistance is futile.

[mortal slate]

hi everyone, if you are here for the 5day genai course with google, please heads over to the respective channels
#5dgai-general-chat: find official course announcements and livestream recordings, and discuss with other participants
#5dgai-introductions: introduce yourself and meet other participants from around the world.
#5dgai-q-and-a: ask questions and kick off discussions about the assignments.

[noble jacinth]

my browser keeps on running out of memory how to fix

[karmic jolt]

How many GB do you have, and which browser ? Though I would make a call - chromium - and advise to switch to Brave integrating "freeze" unused tab. Careful, disable "run on background" and GPU accel

[blissful hamlet]

Hello I cannot submit any notebook. I already finish my verification

[ruby fiber]

Hello

[nimble cairn]

Hi where do i find the link for live stream

[ebon wyvern]

yeah! where do we find??

[placid thunder]

Same question??

[rain oar]

@placid thunder @ebon wyvern @nimble cairn It's not gonna be in the Kaggle general chats, head over to #5dgai-general-chat , and keep an eye on your emails as I'm sure they will provide the links before the session starts.

[placid thunder]

@rain oar okay thanks! Will be on the lookout for it🙂

[strange surge]

is livestream statrs?

how can i join to live stream session?

[austere olive]

https://www.youtube.com/kaggle

here's the link. They will have the stream here

[nimble crane]

Will we be charged if we generate an API key?

[unkempt horizon]

how to access the today session please ?

[karmic jolt]

I have myself a free-tier, it should not be the case I think

In any doubt, no billing informations, no billing right

[digital cypress]

I just know python I am new in AI is this 5 day course helpfull for me please guide

[modern trellis]

did anybody got a bug like this?

it's just white background

but still I have no idea what causes this

[fathom peak]

This section "Server Guide" is not appearing for me now. how to find it please ? It had to all the suggestions how to get started as new KAggle member here

[summer trout]

Hey I couldn't link my kaggle account with my phone number , I wait for the code but no sms arrives , any idea what to do ?

[oblique hazel]

Generative AI is advancing so quickly that it feels like trying to drink from a fire hose. How can I best learn the core foundational concepts of generative AI in a way that will allow me to adapt to the rapid changes and keep pace with new developments?

[silk grail]

Does anyone know how to add Kaggle url to Discord?

[tawny igloo]

@silk grail its explained here: #❗┊link-kaggle-account message

[fluid hatch]

hi, everytime i verify my phone number it says "This phone number can't be verified." i sent an email about it 2 months ago but no one has gotten back to me.

[simple prism]

same to me

[glass vigil]

Hi, I get an error when running the cell with the high temperature setup. I guess it is about running out of calculation resources. Did anybody has this issue before?

high_temp_model = genai.GenerativeModel(
'gemini-1.5-flash',
generation_config=genai.GenerationConfig(temperature=2.0))

for _ in range(5):
response = high_temp_model.generate_content('Pick a random colour... (answer in a single word)')
if response.parts:
print(response.text, '-' * 25)

Error: ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).

[verbal crest]

If you are getting an error when trying to verify your phone number on Kaggle, ensure your phone is not signed up for the Do-Not-Call registry and double check your country code and area code are entered correctly. Ensure your phone number is not from a region sanctioned by the US government. If you are still having issues contact Kaggle support by filling out the form here: https://www.kaggle.com/contact#/account/activate/phone

[glass vigil]

No. It is free

[halcyon basin]

Hi, i got too many requests when trying to verify my phone no.

hi, what is the project to generate the secret api key?

[grizzled grail]

There are some mistakes on the captions/transcripts of the NoteBookLM generated videos, like "F shot" should be "Few shot" at 3:59 mins on the "Prompt Engineering" video. It could be caused with using auto-generated, or the new technical terms/band names, like the LLM model names from varies companies. I wish Google would generate officially edited transcripts for these kinds of educational videos. Thank you!

[haughty sedge]

Hi. Has this been resolved?
If not, I'd suggest deleting your current Kaggle account and creating a new one with your old credentials

PS: Try this if it's a new Kaggle account

[bronze blade]

Yeah i got the same error , it's because you're sending many request in a short time try to increase waiting time : time.sleep(60)

[opal flare]

hello i am new here!! please can someone get me up to speed

what is the project about

[elfin ruin]

Thanks! I got no error after inserting time.sleep(5) 🙂

[jovial nebula]

Hi everyone. How can I register my assignments or only copy and edit the notebook is enough?

[loud lantern]

Hey I copied the notebook and im getting an error when im running this cell:
flash = genai.GenerativeModel('gemini-1.5-flash')
response = flash.generate_content("Explain AI to me like I'm a kid.")
print(response.text)

---

InvalidArgument Traceback (most recent call last)
Cell In[5], line 2
1 flash = genai.GenerativeModel('gemini-1.5-flash')
----> 2 response = flash.generate_content("Explain AI to me like I'm a kid.")
3 print(response.text)

File /opt/conda/lib/python3.10/site-packages/google/generativeai/generative_models.py:331, in GenerativeModel.generate_content(self, contents, generation_config, safety_settings, stream, tools, tool_config, request_options)

and this is the error

[obtuse fulcrum]

Hi! I have a few questions.
It seems that you can't change usernames on Kaggle. I recently created an account using a Gmail login. If I delete the account, will the username be available for reuse and will I be able to use the same Gmail account for creating a new account with a different username?
Is this possible? Can this be done immediately? Or are there restrictions in place for email and/or username reuse? Is there a waiting period for reusing email accounts and usernames?

[midnight prairie]

how will strat here

this ai

it says that the assigment will come to our mail but it isnt happend

[final merlin]

When I run the first code cell it shows "Note: you may need to restart the kernel to use updated packages.". Is this normal or what should I do next

[hushed hound]

Hi everyone, How do I complete the assignment? Is it enough to copy and edit the notebook?

[solid arrow]

Hi all, I am also not clear what 'complete' this code lab means. I copy and edit the notebook, but is there anything else I need to do? how is it registering that I have actually run all the code?

[somber pawn]

I'm running into this issue now as well. Did it recover for you eventually?

[glass vigil]

Yeah i got the same error , it's because you're sending many request in a short time try to increase waiting time : time.sleep(60)
Thank you @bronze blade

[forest rapids]

%pip install -U -q "google-generativeai>=0.8.3" getting error. every time it asks to restart the kernel

unable to proceed

[glass vigil]

Is the live stream happening today?

Just started

[unkempt forge]

I had the same error. Doing a factory reset worked, under the Run menu

[calm wharf]

I have 11 pending submission range from 1hour ago to now and all of them are stucked
Does anyone also meet this problem?

oh they are processing now UwU

[forest rapids]

thanks

[dense atlas]

On the temparure handling 400 error, Increasing timeout time.sleep(60) did not work for me

[brazen tapir]

Why NotebookLM is not working in Ukraine?

[devout imp]

why temperature is so much Resource consuming?

[clever pasture]

Are the recordings of these sessions available for later? The discussions, that is?

[limpid vessel]

still doesn't work 😦

[neon vector]

Can anybody help me out with the white paper link and podcast link

[little thistle]

can anybody help me out with step by step, following the first day assignment, what to do?

[buoyant swan]

In the E-mail you used to register for the event, you would have received a mail today.

Email Title : "[Day 1 Assignments] 5-Day Gen AI Intensive"

All links are there.

[neon vector]

Seems i did not recieve the email

[buoyant swan]

Checked Spam Folders?

Mail was from '[e-mail]'

[stark eagle]

start with https://www.kaggle.com/code/markishere/day-1-prompting

[neon vector]

no its not there as well, do you mind forwarding it to [e-mail]

[stark eagle]

https://www.youtube.com/watch?v=F_hJ2Ey4BNc
https://www.kaggle.com/whitepaper-prompt-engineering

[limpid vessel]

does anyone know how to submit the assignment/ how to Kaggle confirm whether you finish the first assignment ?

[buoyant swan]

Links from the screenshot.

Whitepaper Companion Podcast - Foundational LLMs & Text Generation

• https://www.youtube.com/watch?v=mQDlCZZsOyo

Whitepaper Companion Podcast - Prompt Engineering

• https://www.youtube.com/watch?v=F_hJ2Ey4BNc

Day 1 - Prompting

• https://www.kaggle.com/code/markishere/day-1-prompting

You can see the whitepapers in the video description.

[exotic holly]

I am having the same problem with . Changed the sleep to 20000. Get the following error
ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).

....
for _ in range(3):
response = high_temp_model.generate_content('Pick a random colour... (answer in a single word)')
if response.parts:
print(response.text, '-' * 25)

Slow down a bit so we don't get Resource Exhausted errors.

time.sleep(20000)

[inland willow]

How to submit the assignment? Anyone ?

[verbal crest]

There is no need to submit anything

hi everyone, if you are here for the 5day genai course with google, please heads over to the respective channels
#5dgai-general-chat: find official course announcements and livestream recordings, and discuss with other participants
#5dgai-introductions: introduce yourself and meet other participants from around the world.
#5dgai-q-and-a: ask questions and kick off discussions about the assignments.

[valid trail]

Thaks Myles for clarifying

[somber pawn]

i ended up skipping the section on Temperature. You can observe the results here: https://www.youtube.com/watch?v=dnGuDNhD3Ag at the 34:20 mark.

[slender nexus]

And don't forget to sleep because tomorrow we will have another assignments.
Увидимся завтра 🫡

[lone badger]

Is there a livestream for day 1 happening?

[lusty river]

It happened already...

[somber pawn]

yes - see my link above.

[raven jasper]

i dont get the point of #ai-mathematical-olympiad-progress-prize-2, like how do i proceed cause its asking me to resolve all sorts of math problems and i am kinda new to this, so i dont really get the point

its vague

[mortal slate]

hi everyone, if you are here for the 5day genai intensive course with google, please head over to the respective correct channels/chatrooms for the course ( as this is the incorrect one)
#5dgai-general-chat: find official course announcements and livestream recordings.
#5dgai-introductions: introduce yourself and meet other participants from around the world.
#5dgai-q-and-a: ask questions and kick off discussions about the assignments.
#5dgai-announcements , a dedicated announcement channel with assignment details

[rain glen]

@mortal slate HI sir. I am participating in the program. But I failed to register, Will I be eligible for the Kaggle medal or any other benefits?

[pastel plank]

Kaggle's YouTube channel, top 3 latest videos are of the event

[verbal crest]

Yes you will be eligible if you open the codelabs on Kaggle

[rain glen]

🙏🏻 thanks i did that

[rain glen]

You could try https://discord.com/channels/1101210829807956100/1305239683156017202 for all resources and movement in the event.

[zinc temple]

Kindly some one help me resolve this issue

can anyone help me resolve this errot

[heavy mural]

Were you able to figure auto completion in kaggle?

[torpid dock]

Nope I am using cursor now, you can try codeium extension on kaggle it works as I remembered

[royal dust]

I need to write a chatbot to keep track of this Q&A. 😉 My work is mostly in genetics data science and generally at a decent scale (say thousands of samples). I am very interesting in embeddings of genetic and phenotype variation. Can you say something to that? Most data can't be put on a non-secure (from our perspective) cloud environment, and I expect to go broke trying to cajole gemini into doing what I "want", which I am not sure what it is since it is a complex manifold with massively redundant information and a lot of human hallucinations. What do you suggest?

[spiral elk]

day 1 related question - in the notebook we have used gemini-1.5-flash LLM and try to feed prompt ,once we feed prompt it given response . here my question is from where this LLM get the information to give us information based on prompt feed

[severe swallow]

Can ReACT be used to search web as illustrated in the example of searching in Wikipedia?

[gray lagoon]

Can gemini-1.5-flash LLM be used for Scientific writing? I could not use it for the same

[gray lagoon]

D

[agile pond]

it is going well till now. TYhank you!

[paper pike]

i got a weird task from work, im not much of a data scientist but i do know the fundamentals

[paper pike]

im supposed to train a model where it predicts profit based on a single category temperature i did all the postprocessing and stuff but the problem is that the dataset is quite small

like 72 rows

so my model accuracy is like 65%

any suggestions would be greatly appreciated

[nimble cairn]

Why Language models can Suggest Functions and can also call the Functions, But can't run the Functions?

[zinc temple]

I have ran the Assignment code provided for Day 1 to Day 3 in Kaggle. Is there a step for submission as part of earning the Kaggle Badge. Kindly let me know.

[pulsar blaze]

It's a golden opportunity to get back to the kaggel community

[fresh spindle]

maybe a longshot, but is there anyone here that would be interested in helping create a ML model in R for predicting a few different baseball stats? I already have the data pulled together from different sources and cleaned, i just would like some help / pointers on what algo to use, how to optimize hyperparameters best, etc. I'm currently using xgboost and mlr for optimization but am having some issues with over / underfitting and am pretty new to this. feel free to send a dm - thanks!

[dusty ermine]

Any evaluation ninja here? I need to go beyond accuracy for Day 2 exercises. If I pass metrics=['accuracy', 'precision'] or ['accuracy', Precision()] the model won't fit.

[forest rapids]

please let me know where can i learn step by step process to create agents and make them operate. well feeling like i have someone who is going to work for me

[modern pumice]

Hi everyone. I'm a little bit late I'm beginning the day 1. Some one can help me? I have a issue trying to install the sdk. It says that I need to restart the kernel

[lime flower]

Hey guys, can you please help me find out the whitepaper like reference docs for CV, especially focussing on OCR?

[zinc temple]

@zinc temple Hi Greetings!

[plush zodiac]

If I am way behind, how long will we have access to the videos and notebooks? (without the 'in 5 days' badge)

[craggy comet]

Hi everyone. I'm a little bit late I'm beginning the day 1. Can someone help me?

[clear blade]

Hi I had question regarding about how LLM's are capable to debug the code for any programming language which I provided as input if the error is supposed to be a syntax of programming language which is easier to find and rectify it because LLM's are already pretrained various programming languages but how it debug the logic error in code? also I wonder how LLM's understand the code what I written let say I wrote function for some task similar function written by other is way different from mine so everyone write the same function using n different ways and n different programming languages how LLM interpret and understand the function? because we trained the dataset to LLM's are limited I believe it not trained the same context which had different n possible way in our case is function

[final pebble]

hi folks, i have nvidia tx2 board connected to my local network(ssh) . is it possible to run kaaggle notebook on the local nvidia gpu that i have?

[frigid phoenix]

Hey can i connect a IDE to Kaggle notebook?

[cobalt path]

Hello, what does this mean, "your notebook has finished." and what should I do about it?

[forest rapids]

We have gone through so much in this 4 day learning. I think I need an 🧐 AI genie which helps me recall these from my brain when I need to utilize these.

[forest rapids]

ResourceExhausted: 429 Resource has been exhausted (e.g. check quota). . This workz good at google AI Studio. how to make it run in kaggle notebook

[thick oxide]

Note: you may need to restart the kernel to use updated packages.

[tardy ridge]

I need your help guys, where I can copy and editing on all assigned codelabs to earn my badge?

[sleek vine]

Can anyone tell me how to download the white paper locally to open and read thru without internet?

[verbal crest]

#1305695728554741830 download from here

[winter furnace]

can anyone point me to a gh repo/kaggle notebook or any example of an image classification (ideally more than 20 classes) that does preprocessing/feature extraction (like gabor/glcm?) then uses classical ML methods (like maybe SVM or xgboost)
it's pretty hard to find examples of these

[lost shore]

Good day! I didn't realize about this 5-Day Gen AI Intensive course. Any plans to do it again in the future?

[wintry birch]

HEllo guys,
How much probability, linear algebra and Math do you really need in general for some MLE job?

[brisk jetty]

No one is replying, How sad 😦

[toxic hollow]

Hi, Please help me.
I'm going to make a search engine based on customer behaviors.
Inputs: query embedding, history embedding (Metadata is stored with vector formats)
We use cosine similarity and train embedding models.
I have two questions about this.
First, how to get the gradient of embedding when use consime similarity?(Can that be estimated in torch?)
Second, for the search, we use two steps, updating the weights about historical embedding and query embedding at the same time, I think that can be noisy.
But I can't make sure. I attahced diagram. And if any questions, feel free to ask.
https://drive.google.com/file/d/1_vWxdasnHjCL6_momviQzcDYHAAgc-1M/view?usp=sharing

[torpid dock]

I can't share to public my notebook is there any limit?

[vital anchor]

haiii can anyone here help with stata data sets 😖 they have been destroying me with them as I am rather new to Stata.

[coral pebble]

Quick question

I am aware that the accomplishments forum do not reward medals anymore and any medals on existing posts there have been removed

But it may be the case that some users have older accomplishments lying around (posted before the accomplishments forum was created in August 2023). Back then, there wasn't a choice but to create these posts in General. And I noticed that these posts still have medals on them (both comments and topics)

I wonder if these medals are here to stay or will be removed in the future as well 🤔

One example: https://www.kaggle.com/discussions/general/438421

(just noticed I have a few of these topics/comments lying around in my profile - I will delete them myself if there's no plans to remove medals from these posts, don't really think they should count towards a user's medal tally and progression)

[sinful root]

📝 I've just crossed over the three month mark working in the AI/ML space. In that time, I've had to learn a LOT of vocabulary to understand what's being asked. I'd love to hear from you: what are some terms that you really wrestled with when you first heard them? What are some that people struggle to define?

[lone roost]

I want to learn RAG for building a chabot application for an e-commerce website, anyone knows some good resourses??

[distant pagoda]

I uploaded a 68GB dataset. How can I use it with a notebook? I open the notebook and it starts a sort of copy of the datasets, but the notebook only gives me 20 GB. So it won't be able to take all 68 GB.
How to do it? THANKS.

[hexed zephyr]

can anyone help me please i cannot verify my phone number on Kaggle it says too many attempts but i haven't tried too many time

[cerulean basalt]

same problem here... i dunno why

[wide hemlock]

Hello,
Quick, probably dumb question: how can we, using keras_nlp, fine tune a Gemma model for sequence to sequence problem?

[toxic hollow]

Hi, @everyone
Please help me. That's urgent.
I'm using sentencetransformers and try to train the model.
That's my error.

[devout storm]

I am new to ML, but saw this error. "pip install transformers[torch] accelerate>=0.26.0" worked for me.

[devout storm]

I am new to this domain, wondering if it is possible to train distill-gpt2, using a training set with below entries (assuming diversification in activity types, statistics, and how the formation of output ). Have around 1800 such entries for training :

{"input": "Date: 2014-12-20 08:16:45+0:0, Timezone: (GMT+05:30) Asia/Kolkata, Athlete: Sammy, Gender: Male, Sport: Run, Activity: 10.13 km, Elapsed: 1.10 hrs, Moving: 1.09 hrs, Elevation Gain: 0.0 m, Kudos: 0, Avg Pace: 6.45 min/km, Max Pace: 4.27 min/km, Photos: 0", "output": "2014-12-20 08:16:45+0:0 saw Sammy went for a run with a distance of 10.13 km with 0.0 m meters of elevation gain. It took 1.10 hrs, including 1.09 hrs of moving time. They garnered 0 kudos. Maintaining an average pace of 6.45 min/km , their fastest pace was 4.27 min/km. No photos were taken during this activity"}

I am expecting that if I provide data such as the one in input, I see a response similar to "output".
However, I am seeing echo of the input as response that if I provide input in the above format.

[fossil yew]

hello guys

anyone here

[devout storm]

Looking at few recent posts - looks like folks do not bother to answer here!

[pallid tinsel]

can anyone add confusion matrix to this code

[next dune]

Hello all ! How do I submit something to the Digit Recognizer competition and what should it contain ? The model itself or what ? Like I press Submit Prediction but it says something about a CSV file

[terse glen]

"TransportError: Failed to retrieve http://metadata.google.internal/computeMetadata/v1/universe/universe_domain from the Google Compute Engine metadata service. Compute Engine Metadata server unavailable." Please, how do I go around this? Thanks in advance.

[zinc flare]

Hello peeps, I am searching for datasets/past projects on object detection for astronomical domain, for example near earth object detection etc. Does anyone have any links or places I could check out?

[exotic spear]

How to merge team if i cannot found enough person to participate?

[hearty drift]

When importing an .ipynb file, downloaded from Google Colab (something is messed up so I can't import directly), why is it imported as a JSON-like structure in a single cell instead of correctly.

Is anyone else encountering this issue?

[plush cipher]

Hello all, I m not able to** add cell **to empty kaggle notebook, I press plus sign "+", nothing happens, OPtions is Python language.

[steel merlin]

I am learning about feature engineering and am experimenting on the house-price-prediction competition.
But my results are getting worse. I want to learn more about feature engineering and how to design good models with high accuracy and are not overfitted.
My goal is to get into the industry and make meaningful contributions, but I am not getting the hang of why the model is performing worse.
Can anyone who is currently in the industry point to resources from where I can learn and understand more about data, feature engineering and models.

[wanton tulip]

Hello guys! I'm in a competition n and I have one import question. For submision: is the model or the entire file tested?

[full fossil]

I’m working on a domain-specific problem where no pre-existing datasets are available. I’ve manually collected some examples and initially used them in Large Language Model (llama3.2) prompts (few-shot learning). However, the results are not very good, and i am doing fine-tuning for better outcomes.

I’ve created my own dataset for fine-tuning a Large Language Model (LLM). The goal is for the model to take a simple text input and output two Python lists: one with what the user is looking for and the other with what the user is not looking for. For example:

Input:

"I am looking for a laptop with a good camera and long battery life, but I don’t want any laptops that use only USB-C."

Output:

wants : ["laptop", "camera", "long battery life"]
does not want: ["does not want laptop that uses USB-C"]

Currently, my dataset consists of two columns—input and output. However, I’m still unsure how to format the dataset going forward. Should I use multiple columns, or is there a better approach? I’ve noticed that some datasets have multiple outputs, and I’m curious about why they do that and whether I should adopt a similar structure.

here are some other questions i have and it would be great if some people could answer it:

1. What types of data are most beneficial as input for this type of
   task (e.g., raw text, structured data)?
2. Best practices for preparing and formatting data for fine-tuning?
3. How should I format my dataset to make it more effective for
   fine-tuning?
4. Any best practices for preparing and scaling up my dataset for better results?
5. Any tutorials or tools that can help streamline the fine-tuning
   process for a text-and-logic generation problem.

Any tips, insights, or references to useful resources would be greatly appreciated!

[dapper socket]

Hey im new to DL and I really need help with a time series problem if anyone out there can lend a hand?

[distant pagoda]

Hi.
I am creating from scratch a transformers model to do speech to text, coupled with the connectionist temporal classifications (CTC). But I can't implement the CTC. Who to help me?
Thanks in advance.

[wheat coral]

Hello everyone , I was working on the kaggle notebook and wanted to know from where i can download the models which i saved during running the code ?? I really need the saved models for my research paper

[daring fulcrum]

Hi everyone. Can anybody tell me can I use standardscaler and onehotencoder and then use train_test_split or should I use train_test_split and after that use standardscaler and onehotencoder only for x_train and y_train?

[wheat coral]

@everyone can anyone please help it is very important for my research

[wheat coral]

ig that may varry according to the dataset and by default we may do encoding on dataset and then do the spliting

[quiet lake]

Hi everyone,

I hope you're doing well. I recently completed a data science course on Udemy and am eager to enhance my skills further. As I am new to the field, could anyone kindly suggest ways to practice and work on projects that would help me improve?

I would truly appreciate any guidance, tips, or resources you could share to help me grow in this journey.

[forest stream]

Hi guys, I’m working on a science fair project on synthetic data, and I’m hoping you could help me with my DCGAN model. After 150 epochs I get very mid results. I’m hoping you could help me adjust the model code.
https://www.kaggle.com/code/thorballer/cifar-dcgan

[forest stream]

You can export the results of the kaggle notebook to create a model under the outputs tab

Before you do that though you need to save it

Probably use model.save()

[forest stream]

Try to use this. https://youtube.com/playlist?list=PLhhyoLH6IjfxVOdVC1P1L5z5azs0XjMsb&si=VYbv3_OqCgRe99K8

Also learn about how all of it works

You can use resources like 1blue3brown on YouTube for that

[raven pulsar]

OSError: It looks like the config file at '/kaggle/input/gemma/pytorch/2b-it/2/config.json' is not a valid JSON file.

can someone help me how to fix this in notebook?

[forest stream]

Check the directory because I was having an issue with the data yesterday

Try os.listdir() maybe

[icy beacon]

Can anyone please suggest me which DL model I can use for background noise reduction from audio/speech?

[cedar grove]

Can anyone help me with submitting my prediction? my notebook is linked but submit button seems to be disabled for some reason, even added a description

[mortal nexus]

how to Check out ⁠5dgai-general-chat to find official course announcements and livestream recordings.

[celest hawk]

please, how do i join a team?, I am a data analyst seeking to build my data analyst portfolio

[inner matrix]

Guys I am stuck with "Regression with an Insurance Dataset" competition ,it has 1,200,000 row so when I visualize the data I cannot understand anything so I cannot remove or handle the outliers and this is my first time to deal with such a big data so any suggestions ?

[clever flax]

Please who can help me with a bug I have been getting on my code while using langgraph

[fast saffron]

Use box plot to see the outliers and use line plot to show the trends

[celest hawk]

please, I am having issues verifying my phone number on kaggle, any ideas?

[toxic hollow]

Hi, Everyone. Now, I'm using features of HMM for prediction of time series.
But, that doesn't work well and i consider some reasons.
So, i want to discuss about HMM with somebody who knows hmm well.

[calm oar]

Why do I get errors when using the API about a random .mp3.json file not existing in my temp directory?

api.dataset_create_version_cli("kaggle/dataset", version_notes="Automated via script")

[Errno 2] No such file or directory: 'C:\\Users\\Owner\\AppData\\Local\\Temp\\.kaggle/uploads\\kaggle/dataset_16121247.mp3.json'

Using kaggle d version -m "Automated via script" after CDing into the dataset folder works though

If I use os.chdir and run it with "", it works fine

Oh, it has to be kaggle\\dataset. Huh lol

Now I get tons of

2024-12-09 18:12:12,410 WARNING Retrying (Retry(total=5, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An 
existing connection was forcibly closed by the remote host', None, 10054, None))': /upload/storage/v1/b/kaggle-data-sets/o?uploadType=resumable&upload_id=...

If I run it via the CLI, I also get the same error, but not if I'm CDd into the directory with the dataset

So now I can't get kaggle kernels output me/kernel to do anything

It doesn't print anything, it doesn't make anything

It just quits

[tardy scroll]

Hello all, as a relatively new user/developer on the Kaggle platform, I was wondering if anyone knows how to prepare or study up on concepts for competitions and such that the platform hosts? Currently, all I’ve done is the “Intro to Machine Learning” course on the platform. Your responses are greatly appreciated.

[spare yoke]

question: What's the policy for use of llm generated code in competitions (getting started, playground, as well as regular ones)? I'm not that good at implementation yet in some parts and it would be a huge help to fill those in- not sure if it would be considered cheating here

[teal shuttle]

Hi everybody!

I have a study going on about AI and Social Sciences. I have questionnaires both for AI experts and for non-expert AI users.

I would be happy if you could join and/or share them with the people in your network, as I need more participants to join!

the link to survey for AI experts: https://form.jotform.com/242173206603043
Description: "This study aims to gather insights from professionals with extensive experience in artificial intelligence to better understand the current landscape, challenges, and future directions of AI in social science research.

Your expertise and experiences are invaluable to this research. The survey takes approximately 10-15 minutes to complete, depending on the depth of your responses to open-ended questions. It can be completed at your convenience."

the link to survey for non-expert AI users: https://qualtricsxmlqkc9pzfj.qualtrics.com/jfe/form/SV_e9GiBXTXJ1PBIp0
Description: "It aims to understand how non-expert users perceive and interact with AI tools like ChatGPT and DALL-E.

Your participation is crucial in helping us gain insights into the non-expert perspective on these emerging technologies. The survey should take 15 minutes (or so) to complete and can be done at your convenience."

Many thanks, let me know if you have any questions.

All the best,
Abdullah

[river crane]

How to crack interview on data scientist fresher

[ripe blaze]

I'm lost. someone that can help me?

https://www.kaggle.com/code/sergeynesteruk/retail-instance-segmentation-yolo/notebook
I'm trying to set up a yolo segmentation model. but no yaml file is in the download... So i created one my self, but 95% became corrupted. now that link ^ is someone else that created the yolo model allready. and he refers to a config.yml file. Any way I can steal that yaml file? since this notebook managed to train it iguess, but for me It ends up only seeing 3 classes from the 34.

https://www.kaggle.com/datasets/sergeynesteruk/packed-fruits-and-vegetables-recognition-benchmark?resource=download

this the dataset. How does one generate or create a yaml file based on this download? It has 2 csv files, but not sure what to do with them. I tried to generate my own yaml based on the csv's. But yeah. Most of the classes are corrupt, and unsure why.

[crimson oasis]

Is there any thread where we can discuss the errors which are coming while running code in notebook?

[slim idol]

How can I install files in specific Kaggle folders? I'm trying to use Comfy UI and trying to install Flux templates, but I need to install the necessary files in specific locations.

[chrome isle]

This is my multi gpu fine tuning notebook.
It gives of memory error after 5 steps of training , I've tried many things still can figure out.
Pls help
Link :- https://www.kaggle.com/code/shaswatsingh69420/ddp-sft-trainer

[loud nymph]

Is there anyone who I can ask questions over DMs about Kaggle?

[worthy ether]

Hey kagglers ,

Do you know any competitions for RAG ? Other than Financial RAG and currently active or recently completed ?

[obsidian harness]

While reading Gemini-API fine tuning (https://ai.google.dev/gemini-api/docs/model-tuning), there is this part that mention:

Adding a prompt or preamble to each example in your dataset can also help improve the performance of the tuned model. Note, if a prompt or preamble is included in your dataset, it should also be included in the prompt to the tuned model at inference time.

But the training data format is only a json array of objects consisting of text_input and output, e.g.:

training_data = [
{"text_input": "1", "output": "2"},
{"text_input": "3", "output": "4"},
{"text_input": "-3", "output": "-2"},

In this case how do I enter the pre-amble? Any example is appreciated.

[stray lily]

How can i use kaggle to learn eda and ml

[graceful axle]

How can I access the 5 day gen AI intensive course

[lavish parrot]

Guys, what is best way to start Data Scientist career? Can I start it with Data Entry?

[obsidian harness]

@graceful axle it is in the announcements group. The URL is: https://www.kaggle.com/learn-guide/5-day-genai

[graceful axle]

Thank you

[tepid shuttle]

Hi everyone! 👋,
Does anyone know how I can train the model for a certain amount of time, then stop it, and continue training from where it left off on another day?
I guess i need to save all the hyperparameters like optimizer, learning rate, model's learnt params(till training at that time).
I want to learn it for using torch, transformer.

[crimson oasis]

Hi everyone, is there anyone I can ask about errors on Kaggle?

[wispy talon]

not that I can help with the errors, but just curious on what you are running into. I recently have been trying to submit entries to a competition but they keep failing, even though my notebook runs successfully

[cyan mantle]

Hello, I would like to reset my course as I have been away and forgotten a lot. How can I do this please?

[crimson oasis]

I am trying 'Is it a bird' case and trying to connect with duckduckgo with below code:

import os
iskaggle = os.environ.get('KAGGLE_KERNEL_RUN_TYPE', '')

if iskaggle:
!pip install -Uqq fastai 'duckduckgo_search>=6.2'

But it's throwing me error like:

error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> [3006 lines of output]

If I go to that line, below is the error message it shows:
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for numpy
Running setup.py clean for numpy
error: subprocess-exited-with-error
× python setup.py clean did not run successfully.
│ exit code: 1
╰─> [10 lines of output]
Running from numpy source directory.
setup.py clean is not supported, use one of the following instead:

          - `git clean -xdf` (cleans all files)
          - `git clean -Xdf` (cleans all versioned files, doesn't touch
                              files that aren't checked into the git repo)
       
        Add `--force` to your command to use it anyway if you must (unsupported).
       
        [end of output]
  
    note: This error originates from a subprocess, and is likely not a problem with pip.
    ERROR: Failed cleaning build dir for numpy
  Failed to build numpy
  ERROR: Could not build wheels for numpy, which is required to install pyproject.toml-based projects
  
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Thanks for extending help!

[gentle perch]

hey guys I am getting started with open ai in python but I keep having this problem yes I read the github repo but it seems complicated and as a windows user I don't want to install wsl so if you have a better idea and can explain simply what I need to do please help me

[tepid shuttle]

How does kaggle really evaluate notebooks, when they evaluate them on their own test dataset, after the competition ends.

[tidal verge]

Is it possible to purchase gpu hours?

[obsidian harness]

@tidal verge search for rent GPU hours in the internet.

[tidal verge]

I mean gpu hours for kaggle notebook

[obsidian harness]

u can upgrade rite?

@tidal verge https://www.kaggle.com/docs/notebooks
"""Google Cloud AI Notebooks

If you run into compute constraints while using notebooks on Kaggle, you can consider upgrading to Google Cloud AI Notebooks. These notebooks run under your project in Google Cloud, and can be configured to use your choice of virtual machine, accelerators and run without limits

To export your notebook to Google Cloud, you can go to the File menu and select "Upgrade to Google Cloud AI Notebooks" from within the Notebooks Editor. You can also upgrade a notebook from the Viewer by clicking on the three-dot menu on the top right."""

[tidal verge]

thanks

[worldly osprey]

can anyone help me on 422 Unprocessable Entity error

[stray lily]

Answer for this plzz...........

[strong fjord]

Yo anyone here?

[raw cedar]

.....................................................................................................................

[ancient flume]

How can I share a Kaggle badge on LinkedIn?

[slow plank]

Hello everyone

[verbal crest]

You can screenshot it and post it. There is no direct integration.

[marble bane]

I'm getting "Too many requests" when I try to post on discussion forums. I haven't posted much in few days , I would like to know what caused it ? How long do I need to wait to post ? Thank you !

[marble bane]

I figured out - It's this url I was trying to post - https://rashmibanthia--eedi-misconception-analyzer.modal.run My Kaggle username rashmibanthia is part of the url and its not allowed. But it's allowed with https://github.com/rashmibanthia/

[flat pawn]

There is no way to recover when you cancel it or kaggle cancel it . Your full instance was closed when canceled .

[flat pawn]

https://www.kaggle.com/rashmibanthia
What happened, here is your kaggle username with url 🙂 it's allowed .

[flat pawn]

Actually too many requests means exceeded rate limit .

[flat pawn]

Use youtube. Then go to courses and complete one by one 🙂

[marble bane]

I know , adding my own link caused "Internal error" - and then next error was "Too many requests" and it still is. I can only add comments no posts

[flat pawn]

You can pass strings there , one by one .

Like

Let say there is a list of json objects.

Js= pd.read_json("json path")
js = js["data"] access a keys value .
for key in list :
Strin = key["input"]

[flat pawn]

Have you tried clear cache ? Browser data ?

[marble bane]

Yes did all that, tried different browser as well

[flat pawn]

Kaggle rank people on basis of submission file only . No notebooks role directly. Notebooks only produce submission.csv

[flat pawn]

What is bad in installing old version?

[flat pawn]

You cannot reset courses actually. But you can go and practice lessons again 🙂

[verbal crest]

You just need to wait a day or two so without trying and the rate limit will fall off. This issue can happen if you trigger errors which cause the rate limit to trigger even if you haven't successfull posted. We are working to make this a bit less stringent (it's a balance since we also need to stop spam bot attacks).

[marble bane]

Got it - I think following triggered Internal error and then too many requests — It's this url I was trying to post - https://rashmibanthia--eedi-misconception-analyzer.modal.run My Kaggle username rashmibanthia is part of the url and its not allowed. But it's allowed with https://github.com/rashmibanthia/

[broken jay]

Hi guys
I want to ask questions about the dataset I downloaded from the government of Singapore. I want to change the datatype from object to float. The data itself looks like numbers [98.7, 95.6, etc] but I try everything and nothing is working. If anyone wanna share some experience I will be very thankful.
the code I already tried:
df["name_col"].apply(pd.to_numeric, errors='coerce')
df["name_col"] = df["name_col"].astype(float)
df['name_col'] = pd.to_numeric(df['name_col'],errors = 'coerce')

[hazy root]

Hello, @broken jay
When you say it doesn't work, do you mean an error occurs? Or does it remain an object? Or something else?

[rich kettle]

I usually use the second one and it works. I am not sure if you tried each individually or all altogether?

[broken jay]

No error occurs, They remain an object.

[broken jay]

I try it one by one🥲

[hazy root]

Try df[“name_col”].unique(). You may find elements that contain whitespace or tabs.

Alternatively, it may be faster to use apply to identify rows where is_numeric is false.

@broken jay

[spare garden]

Hello, is anyone experiencing issues with cell commenting or auto-scrolling when running the cells?

[glossy crown]

Commenting is fixed, auto-scrolling is not fixed yet

open a #1130785765274685500

[ornate musk]

Does anyone know how to use the docker file in Kaggle Notebooks?
Is it already in the environment?

[quick pulsar]

Hey guys, I am new in kaggle, and I don't really have a clue how should I submit my code, I mean I just did basic logistic regression just to check how platform works, but I have not really used submission.csv and I don't know how. And when I try to submit, it just doesn't allow me to do that, what should I do?

I suppose I should write an evaluation code for a submission csv but I don't get it, how is it done, thanks in advance for a help )

The last code I just made a confusion matrix for a test set

[sharp plume]

Hey guys I am planning to start learning and exploring blockchain development. Can anyone here please tell me some good resource to learn blockchain development. Open to all ur views on this. Please tag me with ur replies

@everyone

[balmy tundra]

What math do I need to learn in order to do well in these competitions?

[violet flare]

this is kaggle competition. and submission.csv is basically your prediction. which you predict with your model(your model: Logistic Regression). And you will have to make predictions from your test data. And you will have found this test file with your train data.

[wraith sparrow]

is there any kaggle api for http requests ?

[daring fulcrum]

Hi guys,

I need your help. I was building the linear regression model, and I was observing multicollinearity in this model. I found 4 features, that had a variance_inflation_factor score more than 6, consequently I deleted them. Nevertheless, I had had the R2 score of 87%, but after removal of these features I have 63%. Can I neglect multicollinear factor in order to improve model or It is vital?

[fast saffron]

Statistics and probability, linear algebra and calculus these are essential for AI/ML

[stray lily]

Why so

Guys, can you suggest some competitions to get strong in eda ?

[toxic hollow]

Hello, kagglers.
I have one question.
I need to make a news sentiment ananlysis tools.
Now, I have 2 methods, first method is to use open source models such as deberta or roberta and second method is to use HMM(I heard IBM used this method).
Which method should I select?

[modern trellis]

Is it possible to use a model from kaggle (for example this one https://www.kaggle.com/models/shelterw/qwen2.5/Transformers/qwq-32b-preview-awq/1) on google colab?

I know I can just download it and upload it

but is it possible to do this with CLI?

[fast saffron]

Go through kaggle competition for practice

[spring smelt]

Hi Kagglers, after finishing my first upload to the titanic competition (which is my first interaction with ml ) the website suggested a 3 hour course to understand better machine learning. I tried to search it today but I can't find it. Do any of you have a link to it?

[violet flare]

Hii, I have a imbalanced dataset. Target column has a two category. One is 1.5% and second is 98.5%. And dataset shape is (100000,32). There is 1lkh rows. If i use smote then data is increase in high numbers of rows. So, according all of you what should i do ?

[elder mist]

Hello everyone, can someone tell me all the short forms used frequently in ds/ml . like llm eda etc

[wraith sparrow]

Everytime u listen one Google it

[broken jay]

Hey guys, has anyone had a problem importing fbphropet here? I tried to use Phopet for time series forecasting, but I could not import it. The error message said: ModuleNotFoundError: No module named 'fbprophet'

[fast saffron]

hi guys, i am want to learn deep learning but i am stuck which one framework should i pick , tensorflow or pytorch which one will you suggest and why it will be better choice according to your suggestion

[vivid oyster]

tensorflow much easier to use , has a ton of pretrained models

[wraith sparrow]

Pytorch much easier to learn

[obsidian harness]

do some R&D and see which is suitable for you. I use pytorch coz the samples I see are using pytorch so ...

[runic fulcrum]

has anyone taken the 5 day GenAI course from Google on Kaggle?

[frail compass]

Hello, I need to install bitsandbytes and peft for fine-tuning LLM in the notebook, but it needs internet access, which is not allowed for submission. Does anyone have a workaround? (I also had similar problems in other competitions)

[toxic hollow]

Hi, everybody.
Is there anyone who knows HMM(hidden markov model) tokenization?

[near basin]

I am just beginning to learn Python. At what point did you enter into the titanic competition?

[snow spoke]

Hi

[wraith sparrow]

do anyone uses uv package manager ?

[thorny jungle]

Hello, can anyone help me with TPU initialization?

https://www.kaggle.com/code/mautlej/notebook5b598bc4fc

command: %cd /kaggle/working/aidiff !python train_run.py --experiment e2e-tgt-tree --app "--init_emb diffusion_models/diff_roc_pad_rand128_transformer_lr0.0001_0.0_2000_sqrt_Lsimple_h128_s2_d0.1_sd101_xstart_e2e --n_embd 16 --learned_emb yes" --pretrained_model bert-base-uncased --epoch 6 --bsz 10 error: in the message.txt

github: https://github.com/mkfdj/aidiff

[cyan mantle]

Hello, I have done some of the practice things some time ago and want to go through them again. how can i reset my progress and clear all the code cells please?

[rose quiver]

Reproducing some text from Day 1 of Google #GenAI 5d course: can some one explain what is meant by "However, the cost of self-attention in the original

transformers is quadratic in the context length which limits the size of the context, while
RNNs have a theoretically infinite context length." - Appreciate an explanation of context length?

[radiant storm]

Essentially think of it as no.of words (technically, it will be subwords), so for attention, we need to calculate the attention between every pair causing the quadratic complexity on the context length, but since RNNs technically just perform the addition of new word representations to a single context vector the complexity is independent of context length.

[toxic hollow]

Hello, everybody.
I saw the llm models supported by vllm.
But, can we use quantized variants of those models using vllm?

[thorny jungle]

Could anyone help me, please?

[toxic hollow]

Hi, everyone. Please tell me what is the difference between using API and open source models.
Well, when we use open source models on serverless gpu, how much money can we save?
Thanks.

[thorny jungle]

DeepSeek v3 is cheap as dirt

[thorny jungle]

till Trump goes into office, it's owned by the Chinase

[thorny jungle]

Can you help me with the TPU initilization Jason?

[toxic hollow]

Thanks, mautlej.

And What do you mean?

[thorny jungle]

look I have a weird error with using the TPU, im trying to port DiffusionLM (smarter ai) from cuda to TPU i've managed to create the main learning step, but the syntactic parser part seems to keep fail idk im buying a Orion nx so I can get rid of Kaggle (But would rather fix that error than buy some hardware) The issues is because im wrongly implementing the Kaggle TPU initilazation and if anyone could help I will strongly appreciate that!

look at my message

[thorny jungle]

...

[thorny jungle]

Any help is appreciated!

[thorny jungle]

If you want to test it out

The deepseek v3 API for free

https://app.hyperbolic.xyz/models/deepseek-v3/api

[toxic hollow]

Yes, I will do that, thanks.

[thorny jungle]

10 USD free

[warm moat]

Hi, I am new to ML and I want to join a team to learn more about it. I am interested in working on the dataset: CIBMTR - Equity in Post-HCT Survival Predictions.
anyone can help me how i can do

[silver rover]

I'm reading about foundation models, I don't get why they are considered self-supervised? Can anyone explain why they might be self-supervised and not supervised?

[nimble tusk]

It’s about the labeling of the data. With supervised ML you need data to be explicitly labeled (i.e. have features and target columns). However, with self-supervision, such explicit labeling is not needed. With auto regressive transformer based models for example, previous tokens form the features and future tokens form the target and there’s no need to explicitly define them. Key difference in both is whether manual labeling is required or not.

[bleak sapphire]

Hi everyone. I am new to Kaggle and preparing for data science positions. Is adding the Kaggle notebooks to my portfolio/ website as a personal project okay? Even if I don't do well in the competition or I don't get a good score.

[toxic hollow]

Hi, everybody.
Who knows deepseek well?
Which version of deepseek is for deepseek chatbot?

[quiet nymph]

https://www.kaggle.com/datasets/steve1215rogg/student-lifestyle-dataset my faculty has asked me to prove the validity of this dataset. how can i go about doing that? please help 🥹

[rigid trench]

Hi guys, is this a good place I can ask queestions on jane street competition?

[rigid trench]

anyone faces this issue 'training data did not have the following fields:" in the jane street competition?

[livid ember]

Hey guys

I am looking for a metric which check similarity between sentences with semantic understanding.

ground_truth = [
    "There is no pleural effusion or pneumothorax.",
    "The heart size is within normal limits.",
    "Mild interstitial prominence is noted.",
    "No acute abnormality detected."
]
 
generated1 = [
    "No Pleural effusion or pneumothorax is observed.",
    "Heart size appears normal.",
    "There is mild interstitial prominence.",
    "No significant acute abnormality seen."
]
 
generated2 = [
    "Pleural effusion or pneumothorax is observed.",
    "Heart size appears abnormal.",
    "There is massive interstitial prominence.",
    "Very significant acute abnormality seen."
]

Here generated1 is similar to ground truth and generated2 contradicts ground truth.

So
Basically I need a metric which give good score for ground truth and generated1, low score for ground truth and generated2

[kindred ember]

Hey guys I need a help. Does anyone know how to combine multiple machine learning models into one for increasing accuracy. I have seen some notebooks doing it, is it really possible? If yes then is there any sources to learn that?
Thanks ; )

[hybrid tiger]

hello! Yes, you can do so. It's called ensemble learning. It is available on youtube, You can even break a model in 2 if it is large and then make a pipeline to combine the 2 broken models together (random stuff, but felt interesting to tell, hehe) hope it helpss

[kindred ember]

Thanks man!! Sounds very much interesting!

[toxic hollow]

Hi, everybody.
I'm looking for ar related to project where that can detect palne and argument the objects on that.
If there is anybody who knows, please tell me.
Thanks.

[thorny jungle]

Hello, how can i fix this: error happends when using this basic command: !python -c "import torch_xla; print(torch_xla._XLAC._xla_get_default_device())" ERROR: WARNING: Logging before InitGoogle() is written to STDERR E0000 00:00:1736283219.556070 9287 common_lib.cc:818] Could not set metric server port: INVALID_ARGUMENT: Could not find SliceBuilder port 8471 in any of the 0 ports provided in `tpu_process_addresses`="local" === Source Location Trace: === learning/45eac/tfrc/runtime/common_lib.cc:483 Traceback (most recent call last): File "<string>", line 1, in <module> RuntimeError: Bad StatusOr access: UNKNOWN: TPU initialization failed: open(/dev/accel0): Operation not permitted: Operation not permitted; Couldn't open device: /dev/accel0; Unable to create Node RegisterInterface for node 0, config: go/debugproto device_path: "/dev/accel0" mode: KERNEL debug_data_directory: "" dump_anomalies_only: true crash_in_debug_dump: false allow_core_dump: true; could not create driver instance

link: https://www.kaggle.com/code/mautlej/aidiff

Idk im new

just please help

[thorny jungle]

Could anyone help???

[toxic hollow]

Hi, everybody.
We're building news analysis models and need to collect news data of 20 years.
Is there anybody who knows news data service well?
Please tell me.

[inland niche]

Hi Everyone. I am doing a proof of concept on Angular to React framework conversion. I am looking for a fully automated approach (if possible) using RAG or some other approaches. Can some experts help me with this by sharing some useful podcasts or docs or anything which could really help in doing it ?

[thorny jungle]

Can anyone help?

[weary gull]

why did TPU init fail: start there. Get soething that doesn't fail this step. Modify for whatever you intend

[thorny jungle]

bro

It's problem with initilization of the TPU

you can't just recode the error when you don't know what it is

[thorny jungle]

I need help not some motivational speech

could not create driver instance

Hmmm

I think it's a bug with the packages

but the main learning proccess works 🤷‍♂️

[weary gull]

Alright: is this your own personally crafted piece of code, or soething you copy pasted and modified. If the latter, maybe you don't understand it that well.

btw, i didn't give you a motivational speech by a margin

[thorny jungle]

alright

its made by hugging face

transforms

also using the DiffusionLM

I think those thinking models are just wack

so why don't implement the logical thinking into the core of the ai model

thats why im trying to experiment with DiffusionLM

[thorny jungle]

it sounded like one

[thorny jungle]

Im asking in the run_clm.py how to fix the error, because interesting to point out !python -c "import torch_xla; print(torch_xla._XLAC._xla_get_default_device())" This also give the same exact error which means that the error isn't caused by the project itself

but by the torch_xla

Here is the error: WARNING: Logging before InitGoogle() is written to STDERR E0000 00:00:1736283219.556070 9287 common_lib.cc:818] Could not set metric server port: INVALID_ARGUMENT: Could not find SliceBuilder port 8471 in any of the 0 ports provided in `tpu_process_addresses`="local" === Source Location Trace: === learning/45eac/tfrc/runtime/common_lib.cc:483 Traceback (most recent call last): File "<string>", line 1, in <module> RuntimeError: Bad StatusOr access: UNKNOWN: TPU initialization failed: open(/dev/accel0): Operation not permitted: Operation not permitted; Couldn't open device: /dev/accel0; Unable to create Node RegisterInterface for node 0, config: go/debugproto device_path: "/dev/accel0" mode: KERNEL debug_data_directory: "" dump_anomalies_only: true crash_in_debug_dump: false allow_core_dump: true; could not create driver instance

so if I could get help with this part

torch_xla

it would be nice

i have the github

https://github.com/mkfdj/aidiff

[weary gull]

I'll be honest. Beyond me. However, whatever you are working with, if you can find and get a related working example, and successfully run it, that would be a start. Then mod code for your use case.

[thorny jungle]

I fixed that error, but got this instead: RuntimeError: Bad StatusOr access: UNKNOWN: TPU initialization failed: ioctl failed; [0000:00:07.0 PE0 C3 MC-1 TN0] Failed to set number of simple DMA addresses

[weary gull]

I have 0 XP with TPU's. Maybe use chatGPT to problem solve it, by asking it questions on subject. Hey, if we've got AI, why not use it. Good luck.

for insatnce i'd input the error message above in quotes and ask gpt why it might have occured

[thorny jungle]

cus it statisticaly predicts the answer

it can't solve it

[weary gull]

I dont know what is going on, but it seems like a failure to interact with your TPU. As in "wires ain't connecting right", so to speak. Again, beyond my XP.

[thorny jungle]

ok

but I created a bug

on the fourms on kaggle and got zero responds

[weary gull]

Reponses from chatGPT: The error you’re encountering, RuntimeError: Bad StatusOr access: UNKNOWN: TPU initialization failed, typically occurs when there are issues with TPU (Tensor Processing Unit) initialization or configuration. The specific details about the failure to set “number of simple DMA addresses” suggest a low-level issue with the TPU hardware or its driver configuration.

Possible Causes and Fixes:

1. Incorrect TPU Configuration
   • Cause: The TPU might not be properly configured in the environment you’re working in (e.g., Google Cloud, Colab, or on-prem TPU).
   • Fix:
   • Verify that the TPU is properly enabled in your environment. For example, if using Google Cloud, ensure the TPU is attached to your instance.
   • For Colab, ensure TPU runtime is selected under Runtime > Change runtime type > Hardware accelerator > TPU.

2. Incompatible TPU Driver or Software
   • Cause: The TPU driver or software stack may be incompatible with your environment or TensorFlow/PyTorch version.
   • Fix:
   • Check the compatibility matrix for the TPU version and your deep learning framework version (TensorFlow or PyTorch).
   • Update your TPU runtime version or the framework version accordingly.

3. TPU Not Ready or Overloaded
   • Cause: The TPU may not have initialized properly, or it is overloaded due to resource constraints.
   • Fix:
   • Restart your TPU node or runtime.
   • If in Colab or a similar service, restart the kernel or reconnect to the TPU.

4. Hardware or Network Issues
   • Cause: The error could stem from physical or network-related problems with the TPU hardware.
   • Fix:
   • Ensure there are no network issues between your machine and the TPU.
   • If you’re using an on-prem TPU, ensure it’s properly connected and powered.
   (other answers ommitted ; check for yourself)

if you figure it out, i suggest you brag about it on your resume 😁 (that was motivational btw)

[thorny jungle]

But the problem is that all documentation for the torch_xla is for the version 2.1

or the fourms are 5 years old

so I have no information

I will pay someone like 10 bucks to fix it

[sick trout]

I am not able to verify my ID plz hel

not able to sumbit my ID

[tall snow]

did anyone try creating a text translation model, like from english to some unique language!
Let's say we want to create a model that translate English to LangX.
Any ideas?

[thorny jungle]

You can just input something via g4f and prompt it to translate smth from LangX to LangY

[torpid meteor]

Can anyone please tell me if I can join a kaggle competition if am a 15 and i do have my parents consent to join? and do i need anything written? thanks!

[wraith sparrow]

high schooler ?

[torpid meteor]

Yes I am a high schooler.

[silk rock]

Hi, I am new to Machine Learning anyone can explain why we need to Normalize and standardize the data?

[lapis dirge]

To not make the gradients explode or vanish during the model learning or loss calculation

[haughty thicket]

they help improve the performance and stability of many algorithms, you basically can handle different scales in the data

[sick trout]

can some1 help in selfie verifcation, I am contacted support but haven't got any respond to that

I am not to verify my kaggle ID plz help

https://cdn.discordapp.com/attachments/1101210830688751628/1327661137050996776/image.png?ex=67848909&is=67833789&hm=22589165bb451e98bf7fddfcd9066a3426f6f191565ca83bd75eda6ecb36ae98&

[sick trout]

@true flint Please help me here. I am getting rejected and contact support has been useless since

[torpid cipher]

Hey people
Does anyone know why I can save my notebook during a competition and the submission.csv file is created, but I always get my scoring Failed?

[rich mica]

Hello.
Which channel do we need to watch to share the kaggle profile link?

[pulsar valley]

is anyone familiar with this error?

ValueError: It looks like you are using a PerReplica object while not inside a replica context, which is not supported. Try running your op or function inside a replica context by using strategy.run

[thorny jungle]

Hello, could anyone help me regarding this issue

[delicate root]

Hello everyone, have any one has good expiernce with Kaggle courses ? Thank you

[warm shoal]

Yes. I think they are very useful in the beginner level
but they do have to be interested and be committed
I feel as they are just text
Readers can get bored easily,

[delicate root]

Yes text only and blaring agree, but someone might say they are very Straight to the point. Kaggle is a platform is learn by doing, I was saying they must kept this in mind.
I was hoping to hear from someone who went through them.
Thank you @warm shoal

[lusty solar]

hey guys can you help me build a model for my eeg analysis you can find the notebook here - https://www.kaggle.com/code/pramitroy/data-processing

[thorny jungle]

Can anyone help me

???

[brave yoke]

hey guy, I got into a problem when I try to use !conda install,the error shows that the kernel isn't conda env so how can I change into conda env

https://tenor.com/view/cute-fingers-fixed-cat-cut-fingers-shy-gif-24961649

[dusky lake]

hi

[final phoenix]

You can activate your conda environment by running:
conda activate <your_env_name>
Then restart your kernel so it uses the activated environment.

[burnt anchor]

Hello I am new to kaggle and ML and I need some help with sumbmiting my code,can anyone help

I will stream

please just help with submission part

[final phoenix]

Hey, happy to help! Let me know when you are available

[burnt anchor]

now are you free

[plain verge]

Hi all, I have a hopefully simple question. Say I created a submission.csv file on Google Colab. the competition wants me to submit from kaggle. So how do I upload my submission file to kaggle and submit it?

[burnt anchor]

try downloading it from collab and then upload on kaggle

I got 77 % accuracy in Titanic project are there any beginer tips to improve accuracy

[plain verge]

You mean upload directly into the submission file section of kaggle? Because file upload is there but I cna't press it

[plain verge]

Try to see how your target variable depends on each individual independent variable and find what kind of interactions between independent variables improve your models. I played with that dataset very briefly so I can only share ideas 🙂

[burnt anchor]

you mean correlation right

[burnt anchor]

try refreshing

[plain verge]

No no. Interaction means that if y depends on x1, x2, x3, etc then maybe we can build a new variable x4 = x1*x2 or x5=x1/x2 etc and then see how y depends on the new variable x4, x5, etc

Technically this is called feature engineeing

[burnt anchor]

I was thinking of combining parentchild and siblings spouse column

[plain verge]

Nop, I need to run a notebook

[plain verge]

Yes, something like that

[burnt anchor]

okay but I am new so I dont know exactly how to do that do you have any idea where I can learn

[plain verge]

Maybe here: https://www.kaggle.com/learn/feature-engineering

[burnt anchor]

thank you

[pliant lodge]

Dear Administrators, where can I find the real link of the original tfhub model link: https://tfhub.dev/tensorflow/deeplabv3/1 on Kaggle?

Dear Administrators, the models I found on this page "Find Pre-trained Models | Kaggle" are various and with different sizes, could you please tell me which one is the original one?

[earnest pawn]

Yo does anyone know how to implement lag features into the model itself? How would you implement it to the test set if the stuff needed to generate lag features for the test set isn't actully there?

[slender ingot]

I can't see the whole output of Kaggle's kernel! Yesterday, I put my notebook on run, generating some images and model files that I now want to download.

I tried the following things which didn't work:

1. Load more images in the output section. (But there is no option for it as seen in the image)
2. The option to download output from 3 dots->Download Output right side of the output label. (It failed 3 times, I guess because of the large output size)
3. Tried Kaggle's API with a token that only downloaded output/all-dogs and output/log file. (Kaggle kernels output aayushjariwala/gan-introduction -p "C:\Aayush Jariwala\Downloads\GAN Introduction Output")

[drowsy berry]

How to get started with kaggle competitions if you are a beginner

Should a beginner participate in these competitions?

[knotty goblet]

Hi everyone, I have a question about the blue box that appears every time I press Shift+Enter.

Previously, when I pressed Shift+Enter, the cell would execute, and the next cell would automatically follow without me needing to scroll down to reach it.

However, now a blue box appears, and it doesn't follow the cells. I have to scroll down every time.

Does anyone know how to make this blue box disappear?

[proper osprey]

Hi, I have the most simple notebook, just trying to submit successfully.

https://www.kaggle.com/code/brianedwards/equity-post-hct-survival-predictions-submission-1

I have Internet off. I have followed the troubleshooting recommended here. My csv looks like…

$ cat submission.csv
ID,prediction
28800,0.35852779938980817
28801,0.5879658618398832
28802,0.47118444853486113

I made a debugging notebook where I tried the suggestion of creating a submission based on the larger train.csv. It looks good.

https://www.kaggle.com/code/brianedwards/equity-post-hct-survival-predictions-debug-1

There is no useful info in my notebooks logs. It runs OK, and I can save a version OK, but I only get the vague message "Notebook Threw Exception" when I submit.

Any guidance is welcomed 🙂

[next badger]

Hi, I am very new to machine learning and EDA part. Currently I am trying to work on a problem set where i have to deal with computer hardware and software data so there are like 76 attributes of the dataset in which there are many versions can you please help me with how to handle that kind of data because those are number but i numerical data even if it's numerical how can i draw perform any mathematical operation over that because it's something like this '11.345.4532.33' .

[dire beacon]

Hello!! Im looking into tools for building AI tools with open models without having to worry about Infrastructrure... Any recommendations?

[orchid marsh]

<@&1303433601177751593> @verbal crest @delicate burrow another spammer

[verbal crest]

Fixed it. Never ending war with spammers 🥱

[drowsy berry]

Would be great to have an answer for this question

[verbal crest]

Start with the titanic tutorial competition, follow the guides and read all the instructions, it will teach you how to enter competitions. Once you have mastered titanic move on to the playground competition series.

[drowsy berry]

Alright, thanks for the help !

[twin agate]

am I allowed to share code I wrote for competition which has long since ended?

[tired venture]

hi ? what show i do with that

this is that file, my code

my code here,anything the matter?

thanks

https://anaconda.cloud/share/notebooks/aa9c7c7b-c809-43c4-8a2f-9a3aa02f73b2/overview

[copper anvil]

I am trying to build a ML model for my upcoming hackathon... Which can predict future insights on real time market data and give prsonalised recommendation... How should I do it..!?

[native spruce]

Hi, I am new to computer vision and currently I am trying a multiclass image segmentation using PyTorch, from scratch. From past couple of days, I am stuck with it. Anyone could help me, please?

[winged aurora]

Can anyone direct me to a right path. Some people on youtube recommend reading hands on machine learning then deal with the math. What should i do first. Should i cover the math first or the programs. Which books or courses should i use

[orchid marsh]

so I've been trying to adapt Ryan Holbrook's notebook for the Ames house prices dataset (https://www.kaggle.com/code/ryanholbrook/feature-engineering-for-house-prices) but ran into an odd problem. I don't know if anyone can help with this.

in Ryan Holbrook's notebook, if you print out the MI scores, it shows that OverallQual and Neighborhood are the two most important features.

I changed the imputer to impute the median for the numerical features instead of 0 and the MI score for several features, including OverallQual and Neighborhood, dropped to 0.

Does anyone know why this is? Have I done something wrong?

[pulsar valley]

anyone??

[neon karma]

libtmux is not allowed in Kaggle?

Bug

[lost crag]

what are your thoughts on DataCamp?

[worthy ether]

Hello Community, Just needed your opinion, I am hunting for a MLE/ AI Engineer job in US. I know basics of tensorflow (modelling, debugging) but not serving,etc. But, I see most of the jobs are in Pytorch and I read Google is switching to Jax/Pytorch. So, is it best for me to switch to pytorch now(heard it a bit to learn) or stay with Tensorflow?

[trim ledge]

Hello guys, I want to finetune Janus pro 7b and I'm getting this multi modality type error from the transformers package

is there any one who did solve it ?

"ValueError: The checkpoint you are trying to load has model type multi_modality but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.
" this is what I get !

[thorny jungle]

Does anyone know how to fine tune llama 3.2 on TPU?

[hollow frigate]

@worthy ether I work as normal dev not ai, so not expert , but if it was programming language I would say learning a second teaches you alot more about first language and u see the different design decisions and pros and cons, so I am guessing it would be same with tensorflow and pytorch

[worthy ether]

Thanks for your reply… and yes you so on point with the learning about the pros and cons when compared to the other

[nova radish]

I get this when I try to import a Colab notebook to Kaggle

The try again button doesn't do anything

and I'm not prompted at all

Is there a solution for this?

[pseudo storm]

Can I close the browser and poweroff my laptop while a model is training in a kaggle notebook?

Oh< I can commit the notebook and it will "in background", nice

BTW maybe someone knows, do google colab have the same feature?

[pearl ore]

I can't verify my phone number. I filed tickets since december but there is no any response. Can you please fix it.

[verbal crest]

File a ticket with support here to get your account verified, that is the only way: https://www.kaggle.com/contact#/account/activate/phone

Check your spam box, we respond to all messages within 48 hours.

[mighty harbor]

Why can i open WeChart

[wraith sparrow]

Google genai's imagen-3 model do anyone knows how to use it ? Following docs giving error

[sleek comet]

Hello all! I have a few questions relating to statistical learning topics that I was wondering if anyone could help me clear up. It is specifically related to content from the Intro to Statistical Learning Textbook with R by James.

1. What is the point of adopting the idea of having "true but unknown" parameters (and then minimizing the squared error function to obtain your best estimates of those parameters) when using linear regression when it could be possible that the relationship between the predictor and response variables is not linear at all?

Specifically in the book, it is stated that the population regression line describing the true assumed relationship is

y = b0 + b1(x) + epsilon

where epsilon is a random error term that follows a normal distribution with mean 0 and variance sigma^2.
And the least squares line that best estimates this population regression line is

yhat = b0hat + b1hat(x)

1a. Furthermore, does this mean that all parametric machine learning models (where the functional form of the relationship is assumed and specified beforehand) establish their models in terms of true but unknown parameters (and then the goal is to use either a closed-form solution or a computational algorithm to find the best estimates of those parameters)?

2. For non-parametric methods, is the idea of having "true but unknown" parameters also used? (I assume not because there is no functional form that is specified beforehand and therefore no population parameters to estimate)?

[atomic marsh]

The key here is understanding that linear regression is a parametric method, so it's making assumptions about the form. Even if the true relationship isn't linear, we're approximating it with the best linear model. The "true but unknown" parameters are part of this assumption, and least squares gives the best estimates under that assumption. But if the true model is non-linear, then our estimates are biased. So it's a trade-off between model simplicity and potential bias.

1. For 1a, yes, parametric models do assume a functional form with parameters to estimate. Whether it's linear regression, logistic regression, or others, they all have a predefined structure. The parameters are considered "true" in the context of the assumed model, even if the model itself is incorrect.
2. Then, non-parametric methods don't assume a specific functional form, so they don't have those "true" parameters in the same sense. Instead, they adapt to the data's structure. However, they might have other assumptions, like smoothness or the number of neighbors in KNN.
   parametric models are simpler but can be wrong, while non-parametric are more flexible but require more data.

[sleek comet]

thanks! that helps clear up the confusion

[sterile latch]

Hello
I am Wamaitha. I am currently finishing up on the python course on Kaggle and I am not able to check my last two subtopics for review and completion.
What do I do?

[orchid marsh]

is this function - copied from ryan holbrook's advanced house prices regression notebook - quite processor intensive if you try to run it on a large dataset? trying to run it on the february playground data on my laptop and it just takes forever and never finishes. i could run it on the house prices dataset. i think the playground dataset is a fair bit bigger.def make_mi_scores(X, y): X = X.copy() for colname in X.select_dtypes(["object", "category"]): X[colname], _ = X[colname].factorize() # All discrete features should now have integer dtypes discrete_features = [pd.api.types.is_integer_dtype(t) for t in X.dtypes] mi_scores = mutual_info_regression(X, y, discrete_features=discrete_features, random_state=0) mi_scores = pd.Series(mi_scores, name="MI Scores", index=X.columns) mi_scores = mi_scores.sort_values(ascending=False) return mi_scores

yeah the house prices training set is <2k rows, the february playground data is 200k rows

[rare rampart]

Remove highly correlated features and sample the dataset, you don't need all 200k rows to make an accurate prediction

[orchid marsh]

thanks. what's a good number to use? 20k? 2k?

oh my laptop managed to do it on the full dataset

took, i dunno, half an hour

maybe next time i'd try doing it on 20k

[rare rampart]

It's a bit more than just picking a number i suggest you study sampling techniques and statistics

[orchid marsh]

ok thank you

[brisk garden]

Guys aside from Correlation matrix, what other method one can conduct to see influential feature for a given target? Is there a simple solution that I can implement to Excel Spreadsheet?

[wraith sparrow]

do anyone know solution to this issue ?

[pearl ore]

Thank you very much. I filed a ticket and i got an email stating that the issue has been fixed.

[river stratus]

mutual information is good but is not directly built into excel

also ANOVA table

[brisk garden]

Thanks for the insight, I'll try it out

[river stratus]

Let me know how it goes!

[heavy fractal]

iraiwat has been warned

Reason: Bad word usage

[barren patrol]

i wanna know how to solve kaggle competetion

house one

[sleek garnet]

This is a noob question, but in the intro to machine learning course, we train the model with the training data and make predictions with it. But then in the end of a step 5, we just make a new model and fit it with full data. I know that the only difference in it is the max_leaf size, which I guess makes it more accurate?

Do we do the training before just to find out what we want the leaf size to be? I mean, the final model is not the same model that we trained with the training data?

[violet flare]

I installed tensorflow in vscode(in virtual environment). After this i typed from tensorflow.keras import models (in this code im getting yellow line means it cant find keras) and import import tensorflow as tf (in this line it works perfectly ). I use python 3.10 version. I tried all ways.... If someone has a solution so tell me..

[haughty mulch]

Can anyone suggest good real life neural networks for performing verification tasks?

[ruby stump]

hey... I have a little knowledge in Gen Ai Models, however i may need to train a few, like image gen, video gen, resume gen, i want take pre trained models and re train them myself, will it be that hard, where can i find the datasets and stuff, anyone knows!

i am a software Engineer that is starting out in AI so my knowledge is way too little ngl

[tawdry lichen]

hi guys, I was looking into the house prices competition and noticed some entries having scored error of flat 0, how is that possible, is it a mistake in the scoring system?

[cyan mantle]

Hello, I have a years sub to datacamp and feel it is more miss than hit. I think there SHOULD be a lot of opportunity to practice but there is not. Thing progress quickly regardless of whether you understand them or not. I would recommend freecodecamp as it is free and has some good python stuff on there. The SQL stuff is also good, along with bash. Best of luck either way!!

I would like to set my courses back to zero for everything, if I delete my account can I join back?

[exotic spear]

how should we know our kaggle I'd?

How do we check kaggle I'd?

[spring cobalt]

According to you what will be the h0 and h1 should be?

[digital fulcrum]

Hi, everyone! I am new here. I want to participate in March Machine Learning Mania 2025 competition. Can I know what to do in this. I am first time participating in the competition

[pseudo storm]

What is the point of running own kaggle competition? Why companies would want to invest in that?

[wraith sparrow]

To seek for newer ideas n innovation that's how every major shift in ai took place till today

[mighty ferry]

I need help installing tensorflow GPU for windows I installed wsl, cuda toolkit and Nvidia drivers, what else should i do, I need detailed explanation please help

[final oxide]

Hey, has anyone dealt with classification using an Academic Success dataset? I have some questions. Thanks!

[vivid oyster]

greetings , i am looking forward to write some research papers if u have any tips or u can help me in anyway pls dm . i am willing to become co - author in research papers aswell

[lunar wind]

Hi, I'm having trouble downloading the OpenWebText2 dataset using "wget https://mystic.the-eye.eu/public/AI/pile_preliminary_components/openwebtext2.jsonl.zst.tar". The link "https://the-eye.eu/public/AI/pile_preliminary_components/openwebtext2.jsonl.zst.tar" doesn't work either. Does anyone know any other way to download the OpenWebText2? Thanks.

[sick peak]

for a submission, can you change test.csv? That is, if you want to create responses for essay topics other than the 3 given, can you submit a different test.csv with your submit.csv (and get scored on the correspondence between the test and submit?) or can you only submit submit.csvs which correspond to the 3 essay topics in the existing test.csv? Thanks for help, I"m probably misunderstanding something!

[hardy basalt]

Hello, I have a Logistic Regression model that predicts win probabilities. I currently have N=5600 games in my dataset. However, when I shuffle the dataframe and then train my model on literally N=2 games, and then test on the remaining N=5598 games, my accuracy is around 70% most of time. Sometimes my accuracy will drop lower, dropping down to around 50% every 1 in like 5 runs. What could be causing this, do I just have a faulty implementation or are my features like too correlated to the results? I'm really confused so any help would be greatly appreciated

[violet flare]

(tnv) PS C:\Users\DELL\Desktop\ht_pd> python app.py
Traceback (most recent call last):
File "C:\Users\DELL\Desktop\ht_pd\tnv\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 70, in <module>
from tensorflow.python._pywrap_tensorflow_internal import *
ImportError: DLL load failed while importing _pywrap_tensorflow_internal: A dynamic link library (DLL) initialization routine faile
d.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\DELL\Desktop\ht_pd\app.py", line 5, in <module>
import tensorflow as tf
File "C:\Users\DELL\Desktop\ht_pd\tnv\lib\site-packages\tensorflow_init_.py", line 40, in <module>
from tensorflow.python import pywrap_tensorflow as _pywrap_tensorflow # pylint: disable=unused-import
File "C:\Users\DELL\Desktop\ht_pd\tnv\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 85, in <module>
raise ImportError(
ImportError: Traceback (most recent call last):
File "C:\Users\DELL\Desktop\ht_pd\tnv\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 70, in <module>
from tensorflow.python._pywrap_tensorflow_internal import *
ImportError: DLL load failed while importing _pywrap_tensorflow_internal: A dynamic link library (DLL) initialization routine faile
d.

Failed to load the native TensorFlow runtime.
See https://www.tensorflow.org/install/errors for some common causes and solutions.
If you need help, create an issue at https://github.com/tensorflow/tensorflow/issues and include the entire stack trace above this
error message.

what this erorr massage try to say? i already installd Microsoft Visuals c++ 2015-2022 Redistributable (x64)

this error occurs when i try to import only tensorflow.

[hardy basalt]

How can I tell the difference between a goodfit & underfit learning curve? I'm confused here because I heard that a good fit and underfit learning curve both show convergence so how do I tell the difference?

[sudden notch]

It's quite a big problem. Your dataset is supposed to contain different types of situation to be able to predict things with a broader perspective. But your dataset is too correlated. Your model is not learning. It's memorizing.

[hardy basalt]

Gotcha how can I identify when my dataset is too correlated from my learning curve?

Is the telltale sign the early convergence?

Cause I’m kinda confused how u were able to tell my model was memorizing instead of learning just from looking at my learning curve

[potent cloud]

@hardy basalt I have a question for you: Why would you train on only 2 examples in the first place?

Try it with a 70-30% or a 80-20% split, see what the end result is.

Also try adding polynomial features.

See if the model captures the relationship better.

[stuck owl]

Hello everyone, I am a beginner in the field of machine learning, how to perform data preprocessing and feature extraction on pulse waveform data, are there any related recommended books or articles with more details? Thanks

[potent cloud]

There's this book that I used to read from called "Data Mining: The Textbook" by Charu C. Aggarwal. Chapter 2 includes feature extraction. Check it out.

[hardy basalt]

not sure I just wanted to see the full learning curve

[hardy basalt]

isn't the learning curve doing that alraedy though if I understand how it works

[potent cloud]

Your diagram isnt exactly correct is it?

why do you have training set size on the x axis

isnt that supposed to be the epoch #

[potent cloud]

what do you mean the curve

[spring cobalt]

*I’m working on a data project analyzing NYC taxi rides to see how payment methods affect revenue. One surprise? I found 8-passenger rides making up about 7% of my data!

Since I’m not sure what type of vehicles are in service, I decided to combine 5+ passengers into a single category. This keeps the data (instead of dropping 7%!) while simplifying analysis.

My question to the community: Is this the right approach? Or would you handle these outliers differently? Let me know your thoughts and any best practices you follow in such scenarios!

#DataScience #Analytics #EDA #DataCleaning #NYCTaxiData #MachineLearning*

[stuck owl]

Thanks. I'm glad to here that. Thanks for the suggestion.

[marsh orchid]

my persona verification is failing

how can i resolve it

[supple prairie]

hello guys pls i got a problem related a bulding ocr model if someone is interesting in helping me fix it i will be very happy , i have been facing this probleme for 4 days , pls dm me for more information if you are intersted .

@everyone

[supple prairie]

like just how is the ctc loss decreasing while the prediction are still offfff

[brave sage]

Hello Kaggle Team, Kaggle Community, and Competition hosts,

Our team participated in the Jane Street Real-Time Market Data Forecasting competition, and we encountered a critical issue during the forecasting period where none of our submissions were properly scored on the Private Leaderboard.

During the Public Leaderboard phase, our submissions were successfully evaluated, and we had no issues. However, once the competition transitioned into the forecasting period, every submission we made—whether it was our own developed solution, a publicly available notebook solution, or a combination of both—failed to be scored correctly. This happened regardless of whether we selected those solutions as our final submissions.

The submission logs indicate that all our submissions were marked as "Succeeded," yet they were not evaluated on the Private LB. The attached image provides evidence of this issue.

We want to clarify that our team did not engage in any rule violations or unethical practices. Given that multiple solutions were affected, we believe this could be a technical issue rather than a problem specific to our team.

Could the Kaggle team and Community please investigate this matter and provide clarification on why our submissions were not evaluated on the Private LB? We would appreciate any insights or possible resolutions.

[dusky axle]

i have tried to verify my identity on persona tooooo many times now. is there anything that can be done for that?

each time i try to contact customer service, i get the same reply about having glare in the pictures 😢

[sudden notch]

Make sure you have enough brightness around you. Also, keep blinking even if it doesn't ask to do it. Rotate your head properly. Clean your front camera to remove noise in the scanning.

[undone iris]

I am trying to sign up for the 5-Day Generative AI course on Kaggle, and whenever I submit registration, it always sends me back to the registration form (as in it is having trouble processing the registration). Any help on that? Thanks.

[fair light]

anyone not able to sign up for the course?

[swift pier]

same here!

[fair light]

we too low rank lol

[swift pier]

ohh is that why

i thought beginner friendly

[bright sable]

same error

[pine mortar]

Same error! can the moderators help please, thanks! @eager thicket @fickle mulch

[hot rune]

Hello everyone! Registering works now!

[potent cloud]

Hi, what would your recommendations be in terms of hardware (whether local or cloud) for someone on a tight budget and just starting in the field of NLP and wanting to learn more about transformers, LLMs, etc. ? Also send any resources that might help me in this learning journey. Thanks!

[fair light]

im joking they fixed it

[inland dove]

Hello everyone, I have a question on removing multicollinearity among features using VIF. I have seen models where they don't add constant column to compute vif_score. Is it necessary to add constant column to compute vif_score ? Thanks!

[topaz apex]

How do I train on kaggle

[ornate cove]

Seeking Trending Research Ideas in AI/ML/DL for a Q1 Journal Submission

Hello, AI/ML/DL researchers and practitioners!

I’m currently exploring cutting-edge research topics in artificial intelligence, machine learning, and deep learning for a Q1 journal submission. Given the rapid advancements in these fields, I’d love to hear from experts and enthusiasts about emerging trends, impactful challenges, and unexplored research gaps that are gaining traction in 2025.

Some areas I’m considering:
✅ Foundation models & multimodal learning
✅ AI for economics & behavioral science
✅ Efficient AI (Green AI, model compression, etc.)
✅ Causal inference in ML
✅ Neurosymbolic AI & reasoning
✅ Self-supervised & few-shot learning

If you’re working on or have come across promising research problems, novel datasets, or groundbreaking methodologies, I’d greatly appreciate your insights. Let’s collaborate and contribute to the future of AI!

Drop your thoughts in the comments or feel free to connect!

#AI #MachineLearning #DeepLearning #Research #Q1Journal #ArtificialIntelligence #DataScience

[scenic canopy]

Hello. As I've been learning ML (linear/stochastic regression, grad descent, etc.), I've been interested in finding effective ways to work out the derivations and matrix transformations myself using a very small case. For example, I have a data set with two fields and then I have the formula for linear regression and gradient descent. Does anyone have any resources that I could look at? Thanks!

p.s. while ChatGPT has some ideas, I was looking for something more technical and/or textbook-ish

[terse dawn]

We see it all the time, people looking for AI and ML developers and professionals, to bang out the idea in their head. To get their app / business off the ground.

But what about the other side? What if we have the technical side down, about ready to push, but don’t know the best way to incorporate, market, and sell it to the correct handlers in that industry?

Is there any sort of “hub” or job board type thing that could aid in finding the business-minded folks of this industry? Any suggestions?

[potent cloud]

I guess you should check this out:

https://weworkremotely.com

[stable dust]

Is it official standford University organising ?
https://www.kaggle.com/competitions/stanford-rna-3d-folding

[verbal crest]

Yes it is

[charred matrix]

I need a little help in my project anyone who is having good experience in image enhancing and preprocessing

[tepid shuttle]

In lux-ai S3, We can only have two submission notebooks running at any time for competing with other agents, how do i decide which two those notebooks should be, right now it disabled the old ones, and run only last 2.

these bottom two got disabled, any way to choose what two notebooks to run, rather than kaggle, selecting two most recent uploaded notebooks for run

[tepid shuttle]

also, can anyone tell me where i can reach out to kaggle team, to discuss about this to them

[desert nexus]

Hello, I'm was comparing this flan-t5(https://huggingface.co/achimoraites/flan-t5-base-samsum) model for fine-tuning with bart(https://huggingface.co/facebook/bart-large-xsum) model . Bart model outperformed flan-t5 , so, I tried to first fine-tune flan-t5 on a larger dataset(https://huggingface.co/datasets/EdinburghNLP/xsum) and then compare it after fine-tuning on samsam dataset with bart. But still I'm getting low performance on flan-t5.
Any Idea on how i can improve flan-t5 for better results.

[digital fulcrum]

Hey guys, can anyone tell me ---
How to configure path for json file in jupyter notebook on winfows.

[verbal crest]

Posting in the forums is the correct way to reach the Kaggle team about a competition.

[tepid shuttle]

is there any issue in how i am handing competition test data, that it;s throwing "Notebook threw exception" error while scoring on my submission.
Logs are showing no error, notebook ran successfully.

[tropic swallow]

guys i need help setting up the kaggle gpu, for some reason it's not working even when it's set on the session

[empty wharf]

Random; Anyone got a popsicle stick bridge blueprint?

Preferrably can hold up to 150kgs

[digital fulcrum]

Is my code on text generation and prompt engineering correct

Pls reply if anyone knows about prompt engineering

[violet flare]

hello friends... my name is Tark(Data Science) i created my portfolio websiteCan you tell me if there is any mistake or not? I glad to hear that
https://tarkptel.github.io/

[stark jewel]

hello everyone... can anyone please help !!??

[green basalt]

is it necessary to ask question while setting up discord for kaggle ?

[wraith sparrow]

No it's not

[sharp hawk]

I asked a question in the discussion board on kaggle, but I'll repeat it here... How do I enable auto suggest? While doing the Titanic competition or the machine learning course, when I'm in the notebook, I can't get auto suggest to work. Even when I hit tab, nothing shows up. Tab brings up an empty suggestion box. I've been spoiled by colab and can't code without a suggestion box as I don't remember most method names..

by empty suggestion box, i mean something like this, where hitting tab doesnt bring up any pandas methods

[sharp hawk]

got it to work by adding adding hinterland dependency

[sharp hawk]

now dependency isn't working after i install it in one notebook...

[fast saffron]

hello all how can i leave my team any idea?

[wraith sparrow]

U can't actually

[wraith sparrow]

having difficulty loading mistral's mathstral 7b model using transformers in kaggle

[sacred basin]

How to Bypass Ngrok Browser Warning on kaggle ?

[verbal crest]

Most Kaggle staff are not on Discord and we offer no support through discord. The only way to reach someone directly is through a forum post.

[ripe blaze]

I'm using python for prototyping AI transformer 7-70b models. Now i'm curious if python still the best choice for inferencespeed / performance once I know what model to use, and want to deploy it as a service. Is it still smart to use python? or would c# or something else be better to put them model in one of those languages for faster inference speed/ performance.

[tepid shuttle]

is this same as last one or change in topics??🧐

[rocky pewter]

is anyone familiar with WGAN? I've got some questions about it if anyone can help ✌️

[pale osprey]

Pls leave the ur questions

[rocky pewter]

its a matter of showing my code, but pretty much the WGAN i have has really high losses for both disc and gen

around 200-300-400

and from what i understand it should be around 0

[thick jetty]

In jupyter, how do I run all cells without moving to the last cell but instead staying on the current cell? This is the behavior for "restart and run all", but not for "run all". Why?

[bitter bluff]

Is anyone here doing the 5-day AI intensive in April?

[soft trench]

I have a twin brother and it's not allowing me to verify 😭

anyone knows what should I do now?

[verbal crest]

Reply to support with proof of your twin brother (eg. photos together). This is a common excuse by people caught with two accounts, so you need to share evidence.

[soft trench]

oh okay, thanks

[ionic thicket]

hey, if i really want to become ML engineer or data scientist
if i do really good in kaggle compititions will it alone help me to get to my goal?

[hard pebble]

Yes. The experience is invaluable. It’s also humbling

[ionic thicket]

wow, thankyou

[neat spoke]

Hey, I am new to kaggle
My aim is to learn python as fast as possible and practice ML projects
I am aiming to work on a research paper in AI/ML domain so i need to refine my skills such that i am eligible enough to work on a good project.
But while i am kaggle i cannot understand it, I already know java and c++ so learning a new language is not hard for me.
There are lots of things available in the website and i feel like a lost soul, can anyone guide me how to get my way through and work on projects/competitions present on kaggle and learn python parallely

[wraith sparrow]

not for free

[neat spoke]

wdym? how to do it >?

[ember void]

Just stop thinking and do. Get stuck at things, load a dataset, check how it looks, what patterns do you see. Load model, see how your RAM and GPU are getting jammed up. First learn your dataset then teach it to your model. Run some algorithms to process your data. Make predictions, see where your models sucks at. Correct them.

But yeah, **JUST DO IT, NOBODY WILL GUIDE YOU UNLESS YOU KNOW WHICH DIRECTION TO TAKE **

[ionic thicket]

hey I am new to building hands on ML but i am strong in indepth math behind ML
my scenario:-
1.)i am doing data analysis, feature selection, encoding in kaggle note book each step with multiple code blocks and these steps are before training a model and i am performing all this on entirely training data
2.) So now i can train the model with no issue with this preprocessed data
my question is:-
1.)To find model accuracy i need to pass test data so i need to do this encoding and preprocessing all above steps on test data as well

so for this what should i prefer?
option that i had is:-
just keep all the preprocessing steps in a function and pass this test data to that function
or is there any more professional way of doing it like using any framework or something or any good practices?
(i want to become an ML engineer so i want professional way of doing it please help)

[thorn portal]

Hey everyone, I am 14 years old and new to Kaggle. I’ve been learning HTML, CSS, JavaScript, Python, SQL, and Data Analysis (Pandas, NumPy, Matplotlib). I’ve also covered Git and version control. Recently, I completed an (EDA) project using Jupyter Notebook. Right now, I’m working on data wrangling and plan to dive into ML soon. What topics should I focus on before getting deeper into ML? Any advice or recommended resources, I prefer learning by doing so, hands-on projects or practical exercises ?
"PLEASE HELP, I STUDY BY MYSELF AND DON'T HAVE A GUIDE"

[wraith sparrow]

Saw u asking the same thing again n again

[ember void]

hi guys, I am running inference notebook. When I ran it on gpu, it got submitted successfully, but when I am running it on cpu, my submissions are failing
what can be the reason?

It is about BirdClef + 2025 submission

[molten anchor]

hi all, I was running a lightgbm model with randomsearchCV on my local environment ( python 3.13) , and it went smoothly. However, when I ran it on Kaggle Notebook, it raised a problem that showed : "UserWarning: One or more of the test scores are non-finite: [nan nan nan nan , ............]". Additionally, the lightgbm training process( with the parameters I got on the local run) was alright and yielded the same outcome. im wondering what could be the reason. Thx

[crimson charm]

anyone know what this is, kind of new to kaggle system? i folllowed all the rules of the competition and worked within my data set. very strange, i put so much work into my models 😭 For reference im doing this competition: March Machine Learning Mania 2025

oh nvm i guess the org mods messed up the sumbission.

[fickle umbra]

is 499 the limit to how many files I can have in my dataset? The notebook contains probably around 50000 files but I cant use them to create a dataset

[simple flower]

hello i am trying to implement language translation using pytorch transformer (torch.nn.transformer). i have used hugging face for tokenization. now the problem that arises that the trainingi error is huge and the model is learning nothing (which is proved when i run inference and it outputs random combination of words). i am attaching the src code and dataset. any help is appreciated.

[misty elk]

i am having troubles using the kaggle api. my goal is to download the ames dataset into vscode (i have just finished the machine learning course). i have made a folder named .kaggle in my user directory and placed the kaggle.json file inside of it

i have asked chatgpt for help and he is telling me to accept the terms of conditions of the API but i cant find that checkbox

[steady yoke]

Do someone know if it is mandotory to be registered ?
https://rsvp.withgoogle.com/events/google-generative-ai-intensive_2025q1

[cold crest]

Hey guys, I need help. We have a project when we are trying to do classification as 'good' and 'bad' of an item. When the item has scratches, tear, twisted structure, it will considered as bad. Majority of the time these anomalies are very small(5-10%). I did a search and it comes under fine grained features classification task. Can anyone please help me with this? Thank you.

[wraith sparrow]

@glossy crown

[gentle bridge]

Hi guys, I'm new to the kaggle discord and new to the kaggle platform in general. I have recently taken part in the titanic challenge I was wondering where I could ask for feedback in kaggle or discord.

[wraith sparrow]

At kaggle

Kagglers more active on kaggle site than discord to answer

[gentle bridge]

I asked in titanic discussions forum they gave me a warning

[wraith sparrow]

Why

[gentle bridge]

I'm not sure either tbh they removed my post also

I got an email the next day that I had been issued a warning

[wraith sparrow]

Don't violate their TOCs simple

[gentle bridge]

I think I understood why they gave me that warning I think they recognized it as a self promotion since I attached a link to my notebook

[wraith sparrow]

U can attach links in comments though ig

[gentle bridge]

Sorry if I'm asking again how can we ask on our notebooks feedback without linking our notebooks. Do I keep it as markdown in the notebook itself ?

[wraith sparrow]

Oh ok ig u should ask here about feedback then

[gentle bridge]

Thanks dude. I'm really wanting to improve as much as I can. Since I'm learning on my own I'm uncertain about a few things may work in kaggle.

[sick trout]

my kaggle got struck after i leave it for some time and cells run forever
what should i do @everyone

[glossy crown]

interrupt execution

This one

[sick trout]

not working still

[glossy crown]

show the cell which is running

the first cell

if nothing works then just restart the kernel

[tepid shuttle]

Is the syllabus same as the first time when it happened. Or there have been significant changes to it??
can anyone confirm.

[verbal crest]

Some changes and updates, but this version is very similar to the first.

[fast tapir]

Hello! I've tried to share some info on my dissertation research to invite members for a survey and interview, but the bot flagged it as "bad". Is there more information somewhere on how to phrase the "call for participants"? Thank you in advance!

[verbal crest]

It's fine to post a call for participants, the bot likely flagged you if you tried to tag everyone or included a link to a discord server.

[viral grove]

Hello,

I am reaching out to the community in pursuit for an experienced individual in the field of Bayesian Networks. I am working on my thesis for MSc Data Science, and I have chosen a topic that explores the applications for integrating Bayesian Networks and Graph Models.

As a student with no real applications experience, I was able to identify how these topic can fit in my thesis from the limited classroom taught experience. I am requesting guidance through one 1:1 meeting with an experienced individual who I can present idea to, to help in giving me any guidance in steering me in the right path to ensure my success in this pursuit.

Their assistance will be very instrumental for my learning journey and success.

Thanks,
Sam

[sick trout]

i have to start everytime is there any alternative way?

[sick trout]

i suffer everytime my notebook will get struck and won't work until i start refresh

[glossy crown]

if it is not a competition notebook, you can make it public(if you want to ofc)

someone might try to reproduce

[wraith sparrow]

ofc none other than u

No one's fond of reproduction except Qafig, the ai-er here

[glossy crown]

??

[wraith sparrow]

Bro traded his sense of humour for more gpu quota

[wraith sparrow]

R u good at cv btw ?

U should focus on one of the three very strongly ml, NLP or cv

[glossy crown]

pruge the last few messages

he learnt the library

[wraith sparrow]

He rote learnt entire library u mean 💀

[glossy crown]

no he learnt hf transformers

I usually use vllm for inference

it is distributed

[wraith sparrow]

Why not lmstudio ?

[glossy crown]

wth 🙏

[wraith sparrow]

Btw is it an app or command line utility ?

[wraith sparrow]

Why u don't like lmstudio it's good too

It's the most latest ig

[glossy crown]

🥰

it is a python library

[wraith sparrow]

Oh u only meant inferencing ?

Isn't hf transformers great ?

[glossy crown]

https://github.com/vllm-project/vllm

[wraith sparrow]

Indeed it's experts' choice

[glossy crown]

it is more flexible but that comes at a cost of performance

[wraith sparrow]

Fr ???

[glossy crown]

when I need to track grads and stuff ofc I use transformers

[glossy crown]

huh

ofc

[wraith sparrow]

U meant vllms more rigid

[glossy crown]

it is

[wraith sparrow]

How many GitHub stars transformers got compared to this btw 💀

That would be a good metric to compare

[glossy crown]

no

tps is much higher in vllm

especially with large batch sizes

I used to get 2k with 7b awq with a batch size of 16 or 32

@wraith sparrow can you purge some of the prev messages regarding stage 2?

[weary gull]

Is one of the Mods a Zombie? 🤣

[heavy fractal]

nocturnemirage has been warned

Reason: Bad word usage

[iron igloo]

^im not sure why it said I used a bad word 😳

But, hello, I was just asking about has anyone received the “Getting Started” email for the 5 Day GenAI Intensive Course?

[wraith sparrow]

Zombie mods kaggle is unique 💀😅

[golden willow]

Is anyone on right now?

I've got a pressing question

I woke up today and was super worried about the whole climate of AI in data science and was thinking about switching to cybersecurity. For the people who are more knowledgable than myself in the field of data science. Do you think data science/engineering + ML engineering with be mostly automated within the next 10 years leaving us without jobs?

[wraith sparrow]

Idts even the most advanced research studios out there is planning to aid the engineers n researchers not to replace them

[golden willow]

@wraith sparrow Thank you, I feel like I finally created a roadmap on how to become a good data engineer and then I just got all flustered on AI taking everything over. I think I will continue with data science and try to shift to machine learning in the future

[wraith sparrow]

Yeah an optimistic mindset now

(former was like js because the world gonna end one day there's no point of doing anything at all)

[golden willow]

Fr man. I'm scared but I at least want to learn how this stuff works and if it becomes obsolete in the next 10 years then I'm sure the skills I've gained will be useful somewhere else

[wraith sparrow]

My friend too thinks almost same way yet he might be exploring it this year

[golden willow]

Btw what role are you trying to get? Or do you already work in the DS field?

[wraith sparrow]

I'm a student learning

[golden willow]

What year?

[wraith sparrow]

First

[golden willow]

In data science or data analytics?

[wraith sparrow]

Ofc data science

[golden willow]

Goated goated

I'll friend request you

[wraith sparrow]

U should friend request this guy @glossy crown he's the real ai goat at 18

[honest hedge]

this is the truest thing I have ever heard

[glossy crown]

hello I got pinged

[glossy crown]

hello kartik

you are here too 😂

[honest hedge]

Yes sir

[glossy crown]

what 💀

no I am not a goat

[honest hedge]

you are not goat, you are G.O.A.T

[honest hedge]

hello

[glossy crown]

I am not a goat in any sense of the word 🙏

[honest hedge]

TU ES!

[glossy crown]

non

[honest hedge]

OUI !!!!!

[wraith sparrow]

Espanol 💀

[honest hedge]

Non, francais

[honest hedge]

Tu es la GOAT

[wraith sparrow]

Oh right when u try to learn so many u confuse the langs 😅

[glossy crown]

francais

I learnt a few french words in school

[wraith sparrow]

Let the goat touch n eat grass now

[honest hedge]

Qafig est le GOAT
tout le monde est perdant

[wraith sparrow]

I was privileged only to learn few sanskrit words

[wraith sparrow]

Je ne suis pas d'accord, les experts existent, il vient de commencer maintenant, votre chèvre a commencé il y a environ 6 ans, mais il va bientôt le dépasser.

[fallow heath]

nice french lol

[honest hedge]

It's prolly google translate 😂

jk

[honest hedge]

non non non non non non

[obtuse storm]

I can ask any questions?

[obtuse storm]

Which one is better? Tensor or pytorch?

[glossy crown]

pytorch is easier for beginner I believe

I started with tf and eventually shifted to pytorch

[obtuse storm]

I too use tensor but have to try pytorch

[glossy crown]

what kind of work do you do?

[obtuse storm]

Oh iam a electrical Engineer but into machine learning (beginner)

[glossy crown]

nice 👍

[obtuse storm]

Still a student so iam doing a project on prediction of trends on Covid 19 like that

[glossy crown]

ok great

[obtuse storm]

How do I start writing a paper more like a research paper

[wraith sparrow]

Use Chatgpt

[obtuse storm]

Ahhh.... Will that work?

I mean most of my project code ideas are from chatgpt tho

[wraith sparrow]

#stanford-rna-3d-folding message
#stanford-rna-3d-folding message

U need to know how to use it precisely

[glossy crown]

no lol 😂

[fallow heath]

for god's sake no

[glossy crown]

hit up some professor or smth

[fallow heath]

just spend time to learn and make search on google and doc

[obtuse storm]

Yeah

[wraith sparrow]

Btw it worked proof is above

[obtuse storm]

No way writing an article is that easy making it write a paper with your data is different from completely using ai

If that would be that easy why study and research I guess

[fallow heath]

the point to make it with ai at 100% is that you'll not learn anything and just paste things you don't really understand

[glossy crown]

I can't verify the quality of the ideas put forward by chatgpt in this case, but in my experience whatever chatgpt suggests is really dumb/or have been already implemented

[obtuse storm]

I have already read 55 research articles and still have a long way to go my prof says read and ref for atleast 120 papers 😭😭

[glossy crown]

research articles on what?

you said you are a beginner

[obtuse storm]

Yeah ai uses existing data so most they exist if too bad thay are already licensed

Oh my uni is kind of crash course type thing so yeah for credits iam doing this

[glossy crown]

where are you from?

[obtuse storm]

They said write a paper and we will give you 100 in ML lab

[glossy crown]

wow

which uni?

[obtuse storm]

Me just a tier 3 uni in India you wouldn't even know it it's more like a college than a unique i guess

Uni**

[glossy crown]

oh, I think tier of college does not matter if you are passionate enough

[wraith sparrow]

Tell this being in tier 3 urself btw (btw yeah it doesn't apply to u i forgot 💀😅)

[glossy crown]

uhh?

[obtuse storm]

Passionate enough to sit 5 hours to learn ml

[glossy crown]

🙏

[obtuse storm]

Let me tell you being in tier 3 college being passionate is not enough 🤣

[wraith sparrow]

I can't sit even 1 hr that's really good

Meanwhile qafig spending 23+ hrs to learn each day

He even analyse new ideas in ml while sleeping

[obtuse storm]

I sometimes pull allnighters for project reviews i mean iam a complete rookie and need to do it fast I guess but let me tell you iam barely passing in my main electrical subjects

[wraith sparrow]

Fr I can understand bro (being myself in tier 4)

[obtuse storm]

But ML seems easier when compared to the other option I have thay were EMBEDDED AND VLSI 😭

For the whole ML lab there is only one pc which has a 4090 gpu for training the LLM and you have wait in a line

[glossy crown]

a single 4090?

you can try out kaggle

you get 30hrs of GPU per week

[obtuse storm]

Yeah I just found out today lol
I was using Google colab and it's bad really bad

[glossy crown]

it runs me 1 or 2 days 🙏

[glossy crown]

ikr

[obtuse storm]

But let me tell you out of 100 pc they have all of them have i7-13 gen processor i mean good I guess

[glossy crown]

yeah good

but you can't really play around with llms unless you have a good gpu

or at least enough ram

even then it is kinda painful

gpu makes things a lot faster for bigger models

[obtuse storm]

The only fun thing to do In the lab is to swap the keys of keyboard to annoy others lol

[glossy crown]

😂

I do that sometimes 😂

[obtuse storm]

Oh yeah it's like 24 gb or more like that

[wraith sparrow]

Ofc i meant the subscription one

[glossy crown]

o3 mini high is not that different from o3 mini low

I have been trying out 2.5 pro for some time now

even it is not great at suggesting new things

[wraith sparrow]

Meanwhile i found grok biased towards rocketry stuff while image gen 💀

When u give input very lil related to it

[glossy crown]

I never tried grok

maybe they did not align their model very well

[wraith sparrow]

They like misalignment

[nova quartz]

Hello, what roles are available in this discord channel and how can I get them?

[noble notch]

Hi everyone,

I have a JSON log file with lane detection coordinates for each frame of a video. I need to determine in which lane my vehicle is located despite the detections being very noisy. What are some robust approaches to assign a lane based on such noisy log data?

If someone have relevant experience solving this kind of problem, it would be so nice to gain an experience. Thank you!

[sudden notch]

How do I calculate the MSE?

[quick dagger]

Hello. I am new to this server. Can you please tell if there is a separate channel for the mlcourse.ai competition.

Thank you.

[sinful root]

Yes, there are separate channels #5dgai-announcements is the best place to get started!

[mellow spade]

I got the email cofirmation of 5-Day Gen AI Intensive. and I have completed all the steps that mentioned in mail and should I wait for my first assignment?
can anyone tell whenI will recieve my first assignment?

[obtuse storm]

When the course starts

[quick dagger]

Thanks a lot.

[graceful axle]

I would like to ask everyone for advice—
I just bought a MacBook Pro M4, but I don't know how to set up the environment for Java, C, and frontend development.
I used to work on Windows before.

[wraith sparrow]

Install satyrn

For python/ai/ml/ds

[graceful axle]

Compared to UV and Ruff, the one you mentioned—do you think it's better?

[wraith sparrow]

Bro uv n ruff are rust tools 😅

They r already in it though

Satyrn is like ide for ai/ml

Better alt to Google Collab n other jupyter notebooks

[sly dawn]

hi guys, i have an issue with the tensorflow library

when i try i import it, it shows this error. Then when i try to uninstall and reinstall my notebook hangs...

any solutions from u guys?

[graceful axle]

Thank u bro

[glossy crown]

it is very good, but it is not better than collab or vscode

it is also new so there is not a lot of community behind it

[wraith sparrow]

The order is lightening studio > satyrn >= marimo > vscode > Colab

[wraith sparrow]

Lightening studio is orz in that case but less free GPU quota

[glossy crown]

what?

[brittle shoal]

Excuse me guys ... i was not able to see the chat for a long time... so can anyone answer mine small doubt?

[wraith sparrow]

Yes

Js simply ask this channel is to js ask

[brittle shoal]

do we have a recorded section for this course ?

[wraith sparrow]

Ofc

We'd last year too

In youtube

[brittle shoal]

ohhh ... thanku for ur help

it will only start at 11pm for us

[rustic hawk]

what do i need to do after creating api in aistudio