#optiver-trading-at-the-close

1 messages · Page 1 of 1 (latest)

wary falcon
tardy tree
#

yeah i found it really difficult understanding what was going on in the start. it was funny once i figured it out

pale mesa
#

I cannot wait for excellent discussions from you guys! cheers~

topaz iris
#

can we somehow pull technical indicators for this project or can we only base it off of the data that we are given, bid, ask, price imbalance, etc

frozen siren
fervent spindle
#

Hi

#

I'm looking for a teammate

#

@frozen siren do you want to be a teammate

topaz iris
# frozen siren I'm wondering how to get this done as well? Also is it continuous timestamp bet...

yeah i have a strong background in finance so i am trying to incorporate some stochastic calculus and technicals/fundamentals intot hit sbut am unsure how this would work since it is being testing on live data and whatnot, please lmk if you find any solutions. "Also is it continuous timestamp between train and test?" I am unsure about this, only slightly looked through the data last not, didnt look too deep.

candid pawn
dull harness
#

Hi! I just jumped into the competition. Do we happen to have other materials like a video other than the documentation regarding explaining the data items?

dull harness
#

this one is great

cloud wyvern
#

Anyone getting the following error when trying to import optiver2023?

#

the competition file it's referring to is a shared object (.so) binary

runic pivot
cloud wyvern
#

thank you

inner shell
#

I am sure that my notebook is plagiarized by 2 users. How do I report to kaggle??

#

I have written in the forum and have reported the notebooks also. Is this enough??

wary falcon
dull harness
#

I think someone copied the notebook and edit it is fine(because the website will give the credit to the original author). But it's not fine to create a new notebook, copy every lines of code into it, and don't clarify where those codes come from.

#

I reuse some lines of codes from other people's notebook occasionally but I always try to make sure to annotate which line of code is not original and come from where(even if it's from stack overflow).

inner shell
#

The 2 notebooks simply copy pasted my work and made it public

#

Even the title is not changed

#

I have no issue in this but I read somewhere that it should be reported

#

I think this is happening for the first time with me so just wanted to know

inner shell
#

I read somewhere that we should report such activities

waxen roost
inner shell
#

This does not do that also
Just copy and share

outer jackal
#

Hi all ! Interesting competition !! Likely my comeback to kaggle after almost 3 years of non competition ^^

quasi oriole
#

Hi everyone! I have a few questions about Closing Auction..

https://www.nasdaqtrader.com/content/productsservices/Trading/ClosingCrossfaq.pdf

What all inferences can I get from this example?

At 3.55pm I see Near < Far. Since Far is more , I am concluding that there is more buying sentiment at the close compared to the whole trading day. Is this right?

Also At 3.55pm Ref Price < Near, Far
What can I conclude from that?

At 3.56pm 3.59pm
Near and Far are $10
But Near and Far were different at 3.55pm
So what happened in the time between that they became equal?
Also what can we say about current Ref price

I see that from 3.55pm QBBO is the same. Is it because MOC is discontinued? Wont LOC change QBBO or it can change and in this example it just so happens that they are same..

Please share any resources that might be helpful to understand this better. Thanks! 😄

Screenshot_2023-10-01_at_6.54.44_PM.png
worthy axle
#

Hey I am not able to understand Seconds In The Data
Can, Anyone Help me Out?

dull harness
#

You mean the time I'd for each day's closing period?

#

*Time ID

dull harness
#

*I mean for more details since I also think that I have a poor understanding of the example.

quasi oriole
round dust
#

Hey can someone explain why some far_price / near_price is missing? And what do they actually mean?

quasi oriole
round dust
quasi oriole
#

I think here they give example with Limit Orders only. Idk how this extends to market orders also.

Screenshot_2023-10-03_at_7.13.13_AM.png Screenshot_2023-10-03_at_7.13.32_AM.png
ember mountain
quasi oriole
#

Financial Markets and Trading: An Introduction to Market Microstructure and Trading Strategies Anatoly B. Schmidt
Its from this book

round dust
#

Did they have some sort of induction zoom call or something?

frozen siren
#

There are so many questions raised in the discussion thread but organisers are not responsive at all. What's going on?

round dust
brazen narwhal
#

Hi, can someone help me understand this dataset and how to go about working on this dataset. I have to work on this kaggle competition as a part of my coursework.
Thanks!!

worthy axle
#

Should we predict the entire time series while the Object Is In The bucket?
Am I right? and the Y Is target right?

frozen siren
#

Actually why the complete silence from the organisers?
Just wondering... Like for starter do respond to questions in Q&A board

pseudo oracle
#

how long the scoring typically take ?

#

i have been waiting for more than 30min 😦

tacit zealot
#

What other data is allowed to use in this kaggle challenge?

smoky pebble
#

Hi all, I’ve started to work on this project and I don’t have much information about the financial terms. Feature engineering requires sector specific knowledge, and I was wondering if anyone could help me understand how to learn more about this! Any resources, connections are welcome! 🙂 Thanks a lot

naive stone
#

I noticed the Optiver Trading Competition has been running every year. I am wondering how this year is different from the previous years.

nova tapir
#

Hi all, have anybody met the same submission problem as this? "Cannot submit
Your Notebook cannot use internet access in this competition. Please disable internet in the Notebook editor and save a new version."

image.png
pulsar eagle
#

I remember seeing a notebook/discussion that would explain all the terms like explaining it to a child a while ago. But I am not able to find that notebook again.. Does someone have link to that notebook/discussion?

pulsar eagle
inner shell
#

Just switch off the internet and it will work

nova tapir
inner shell
#

Just look at the right side panel
You will see a slider with internet on/ off option

#

Just slide to off the internet

nova tapir
inner shell
#

Most welcome

#

All the best

lament hemlock
lament hemlock
shy sentinel
#

Any one got luck trying to do this challenge with LSTM?

limber fog
#

Hi, do I have to make a public notebook or can I keep the notebook private?

granite ingot
#

Do we have only limited time for submissions through the whole process?

twin vine
#

Hi! My current model requires a few "past values" to predict the next value, this unfortunately causes my model.predict() on the "test.csv" to miss the first 200 rows. Is it possible to get a LSTM model to predict all entries in the "test.cvs"?

dull harness
#

I would try dummy values or using target mean values in the same period as predictions?

main turret
#

Hello I just joined the competition

#

Anyone has any submission issues? The code all runs fine and generates a submission.csv file but I can’t submit

main turret
#

Also scoring takes over 5 hours

#

Why

neon token
#

Has anyone gotten good scores using DL?

gusty raptor
#

just check other people's scores on the leaderboard

slim sphinx
slim sphinx
#

Oh, also, whats up with the 9 hour limit, thats just for one run from blank kernel to prediction right? Not total cumulative time running with the API?

still osprey
#

hey, is there any documentation for the API used for this challenge?

gaunt mantle
#

Has anyone tried using RNN to solve this?

hollow marsh
#

When i try to save submission.csv i got error , opration is not permitted as in working directory there i s already submission.csv file with zero bytes , how to solve this problem

polar fable
#

Hi, is it natural that reference price being greater that best ask or lower than best bid?

twin vine
#

Hi! My model generates good predictions outside of kaggle for train and test data around 2.6 MAE. But when I run the same model on kaggle it will populate the submission file with the exact same prediction over and over again. Does anyone know how to resolve this issue?

twin vine
#

With outside of kaggle I'm referring to when I run my code/jo-notebook with anaconda.
Im splitting the train.csv into train_data and val_data and using the test.csv as test_data.

The model generates unique predictions for all time steps outside of kaggle and for the validation_data in kaggle. While using the "iter test" loop the submission.csv gets populated with one prediction for all rows, similar to if you'd did sample_prediction['target' ] = 0.
So I guess the issue comes from within the loop itself or from the optiver environment.

I haven't published it yet but might do it later on

round dust
#

Hey guys, how does one actually submit? is it a notebook or a csv??

buoyant leaf
#

the notebook gets ran

there's an example loop about how to submit that just predicts all 0's

round dust
#

Ohhhhhhhh

#

I seeeee

buoyant leaf
#

env.predict handles submitting the predictions

#

it's abstracted away

round dust
#

I thought were meant to submit a csv

#

Ahhh gotcha thanks mate

polar fable
#

This paragraph states that the reference price should be near price bounded by best bid and best ask in continuous trading. But why it is not like that in the training data?

image.png
pulsar bear
rose quartz
#

Hi, i just joined this competition. Also a noob in AI/ML space. hope to learn some insights here

iron nacelle
#

Hello! I'm new to Kaggle. Is the data for the leaderboard necessarily composed from the same stocks with the same id's?

spice warren
#

Hello can anyone explain this competition in simple term ? the data and what should be the input and what should be the output ?

iron nacelle
spice warren
#

thanks

elfin sparrow
#

Getting a weird error where rerunning models that worked a few days ago now produces a mismatch of 1 in the number of features expected and found. Has anything around the competition data changed, and has anyone else had this happen?

fringe fjord
#

What is the significance of those stocks who have null targets upto a certain time (usually 300 seconds) and then start having no null values till EOD

e.g stock 131 on day 35 shows such behaviour

#

More specifically, what is the meaning of those stocks where even far and near price is null

still osprey
#

Hey, when there is an error in the submission is there anyway to check what went wrong? If not, how do people usually debug it? I am not sure why my submission is generating an error

full ember
outer cargo
#

Hi all, I'm another one facing an error with the submission and mine seems to be quite weird one. I made a notebook to reproduce it and feel like I'm out of ideas: https://www.kaggle.com/code/shuthdar/reproducing-a-submission-scoring-error

If a commented out block of code in the last cell is uncommented, the scoring fails despite the execution being completed without errors, if it's commented out - scoring works fine, even though the rolling_test dataframe isn't used for the prediction.

Reproducing a submission scoring error

Explore and run machine learning code with Kaggle Notebooks | Using data from Optiver - Trading at the Close

outer cargo
slim sphinx
timber latch
#

When implementing the baseline prediction vs. the simple mapping using imbalance flag (the first-pass strategy mentioned in the competition intro https://www.kaggle.com/code/tomforbes/optiver-trading-at-the-close-introduction), I get a better score for the baseline prediction than for the simple/improved mapping strategy. The training data seemed to perform better on the improved simple mapping, but somehow the test data doesn when I tried it out. Anyone else noticed this, or am I overlooking something here?

still osprey
#

Yeah I think if you try it with the train data it performs better like you said but if you upload it on the test data it has a worse score

#

I think it’s not very different from baseline anyway and it just happens to perform worse on the test set

#

Wouldn’t overthink it - I got the same result

timber latch
#

thanks! appreciated!

fossil timber
#

Im a bit confused on what we are submitting

#

is it the target?

timber latch
fossil timber
#

would you mind helping me out?

gaunt mantle
timber latch
# fossil timber i have trained a model but im a bit confused on how to get it to make prediction...

if you already done training you're model, you're already a step ahead of me😃. personally, this notebook https://www.kaggle.com/code/sohier/optiver-2023-basic-submission-demo/notebook has answered most of my questions so far when trying to tweak my code to make the actual predictions.

Optiver 2023 Basic Submission Demo

Explore and run machine learning code with Kaggle Notebooks | Using data from Optiver - Trading at the Close

fossil timber
#

Thanks

mellow pulsar
#

I ran a dummy submission as


    for idx, (test, revealed_targets, sample_prediction) in enumerate(iter_test):
    
        print(f'Prediction {idx}')
    
        if not test.shape[0] == 200: 1/0

        sample_prediction['target'] = 0
        env.predict(sample_prediction)

and it fails after 90 seconds after submission. So the must be a data frame which does not have 200 stocks?

Any ideas why the test data frame is not always of size (200,16)? We should always have 200 stocks and 16 features, no?

and if it does not have all 200 how do I know which stocks it has?

twilit sierra
#

Not sure about the features, but I would suspect the model needs to be generalized to support more than the 200 sample stocks, as there are thousands that trade in the US markets each day. Feel free to correct if I’m wrong, I didn’t see it specified in the rules.

rustic ravine
#

Hey, i want to use this competition for a course on time series.
Did somebody used CNN and had success with it?
I'm interested in what you think should be the baseline score for a CNN model without feature engineering.

sterile quiver
#

can anyone tell how long does it takes to run the notebook during the submission... been waiting for 2 hours ... my actual notebook ran in around 45 mins only!

slim sphinx
#

max is 5 hours i believe

fringe dove
#

Hello guys

now I`m stuck in submission problem. Dose anyone here to check up my code?

hollow marsh
#

Hey im not able to submit a CSV file using even with api help me

atomic quail
#

Scoring has been taking some time today, been waiting for roughly over 2 hours now since my notebook ran and generated the predictions, and it's still at "scoring."
Are there recommended times of the day when the servers are usually more responsive? Thx!

atomic quail
winter cedar
#

guys, a quick question

#

how did you deal with missing data in far and near price?

twilit sierra
#

probably have to form an opinion about when/how/if you think the continuous trading is affected by auction imbalances/potential cross prices - i.e. when they are present and when they aren't.

winter cedar
#

Hello guys, would someone tell me that deadline is 11:59 PM on Dec/19 or 11:59 PM on Dec/20? Thank you!

unique oyster
#

2 days to go. I guess you are now training at the close.🥁

grave topaz
#

this competition is already close right ?

#

it will be another change about this subject? I will love to participate with a team and get to learn deeper.

light temple
#

When will the data be accessible? (asking as someone who has not joined the competition before the deadline)

magic pike
#

hi

#

im new to ai feild but i have completed some introduction courses. please can i know how to get the practical experience is there better suggestions?