Ok
#tensorflow proj
fossil root
blissful beaconBOT
Ok
- Wait patiently for a helper to come along.
- Once someone helps you, say thank you and close the thread with:
+close
- Feel free to nominate the person for helper of the week in #helper-nominations
- Do not ping the mods, unless someone is breaking the rules.
- If you're happy with the help you got here, and the server overall, you can contribute financially as well:
fossil root
If u can provide those 3 questions so I give u the best answers from last night @hallow garden
hallow garden
Sure
I am just asking questions so that we can contextualize, as accurately as possible, your problem
And identify your needs
First of all: What is your dataset, i.e. what are the input features (each column), and what is the target?
Assuming that you have a system that takes this input and outputs something, you want the input to be accurately known
fossil root
- The dataset is the results of an ordinary differential equation solved in Lyupanov time where the time span is 1000. Is that a good enough answer
(Given whatever params)
hallow garden
Well I want to know exactly what is X and what is y, in machine learning terms
At a certain time step t, you have one row of inputs
fossil root
X is the input correct? And y is the ground truth in this case
fossil root
Yep so that ground truth is our prediction target but it’s given to us already
hallow garden
Well, yes, but I want to know what it physically corresponds to
You said earlier that it's your displacement x(t)
in physics terms
fossil root
Yeah correct
hallow garden
But what are the input features of the system?
What do you need to predict x(t) from
fossil root
You would need alpha beta gamma delta omega
hallow garden
Like, idk, temperature? overall system height?
fossil root
The model should make it more clear lemme send it
hallow garden
Also, let's change notations
fossil root
Like here, those different variables represent thing such as damping, amplitude etc
hallow garden
Let's name the displacement y(t)
because y also corresponds to target notation in ML
So input data would be denoted x(t)
So your dataset should be sequences (x, y) such that you want to predict y from x, with both being sequences
fossil root
Ok so we can do this even in the case we’re predicting the ground truth given we already have it
hallow garden
Yes, but just to be clear, just because you have a ground truth doesn't mean it's input data, yeah?
to your system
fossil root
Yea that’s true
hallow garden
It's used to train your model
so that your model learns to map x to y
but that doesn't mean x = y
otherwise you already have a trivial predictor
which does nothing to x
And my question remains: what is x comprised of?
I haven't got an answer to this yet
fossil root
X is the input
hallow garden
yes but what is in that input
example: x(t) = [temperature at t, acceleration at t, speed at t, ...]
hallow garden
Such as?
fossil root
Alpha beta gamma delta
hallow garden
Try to write it in the same format as me above
x(t) should be a finite 1D array with certain components
which is what I am asking you about rn
what are each of those components
fossil root
Okay:
X and y aree simply the split of the solution in lyupanov time.
X is the first third and y is last 2/3rds.
Oh ok hold up I just read that
hallow garden
Ok, so the sequence x would be: [displacement at t0, ..., displacement at t0 + n]
and y would be: [displacement at t0+n+1, ..., displacement at t0 + n + k]
fossil root
Yeah
hallow garden
Ok, very good
Exactly what I wanted to know
so here x(t) is just a 1D array with 1 number
which is: displacement @ t
Good start
Now the second question: your task is to predict y from x, but what loss function do you use?
fossil root
Like MSE?
hallow garden
Sounds pretty ok to me
So you have a system that outputs y_hat from x
and you compute ||y_hat - y||² / k
and that is your loss per sample
yes?
fossil root
Ok to follow along lets back track for sec
fossil root
Up to this point is where we got (x train and y train) which is fine..right?
hallow garden
what is this "solution" array?
fossil root
That’s the solution from the equation
hallow garden
what is its shape?
fossil root
The solution?
hallow garden
The "solution" array
fossil root
It’s 2D
hallow garden
what are its dimensions?
fossil root
2 dimensional
hallow garden
... so it's in the shape (n, p) with certain n and p, right?
fossil root
Yeah
hallow garden
Sorry you're not answering my question rn
fossil root
Like what they represent?
hallow garden
[[0, 1],
[[0, 1],
[[0, 1]] is of shape (3, 2)
and is two-dimensional
I assume your array looks like that but with a different number of rows and columns
so how many is my question
and what do they represent
hallow garden
What is the second column?
Or rather what are the two columns for
position coordinates in the plane?
fossil root
Like for example [[0, 1]] are u asking what 0 , 1 represent?
hallow garden
I assume the 1000 stands for the time period where you observe
and the 2 represents what?
fossil root
Yes that’s correct
hallow garden
coordinates ij?
fossil root
The dimensions because it’s 2D array
hallow garden
Well yes I know it's a 2D array, I just want to know why you have two columns and not an arbitrary amount of columns
fossil root
It’s just an array of each solved equation at each time point
hallow garden
is it because the displacement is represented in two dimensions in the plane?
fossil root
Yes
hallow garden
as in, x(t) = [i(t), j(t)]
Ok, that was my question
I have a different suggestion for you
Something that may be more helpful or less, who knows
Assume the sequence of displacements is s
so s(t) is the state of your system at t
So you have all s already
fossil root
Meaning the time span?
hallow garden
Basically, i(t) and j(t) for all t
You simulate the system with your program
and obtain s(t)
for all t
which is the solution array that you have
of shape (1000, 2)
We will make dataset from this
a X and Y
fossil root
ok
hallow garden
x:
[s(t0),
s(t0+1),
...
s(t0+p]
and instead of predicting the next k
you will only predict y = s(t0+p+1)
so you have a chunk of time frame p
fossil root
So like individual time steps?
hallow garden
and you predict the displacement at the next instant
yes
why is it going to be equivalent?
because now, if you want to predict the whole sequence
you predict s(t0+p+1)
fossil root
I tried something like that but idk if it’s conceptually correct if I can show u
hallow garden
then you plug it into the sequence and make a new x
x_new = [s(t0+1), ..., s(t0+p), y]
and then you predict the next again
and you keep doing this
I'm just thinking that 1 target prediction may be better for your neural network
rather than an entire time window
fossil root
hallow garden
I mean that's pretty sound if you ask me
So that would be your training data
you can simulate some more for testing
but anyway
fossil root
I’m gonna test it out
hallow garden
Back to your architecture
I fear that a LSTM may not be the most appropriate
Try a bi-lstm
fossil root
Sadly it’s required for this
Well
BiLSTM is generally more efficient?
Technically still lstm so
hallow garden
There are plenty of other architectures that work on sequential data, but if you're bent on using LSTMs
you may as well use bidirectional lstms
the idea is that you read your input sequence both from left to right and from right to left
Also, more regarding your architecture
After your LSTM, if one dense layer does not suffice, add an entire MLP
i.e. consecutive dense layers with activations
fossil root
@hallow garden give me like 10 minutes I’m gonna run some tests and get back
hallow garden
Sure
fossil root
Yeah
hallow garden
If you're using regularization in your network, it'd be a good idea to rescale your data, yes
fossil root
I am
hallow garden
Even if physically it may not make sense
Then yes rescale your data
And apply the inverse scaling manually as postprocessing to examine your predictions
It makes a lot of sense because if for example your last dense layer is regularized, it can only reach so far
in terms of prediction norm
fossil root
Fixed it but i got diff error now with the graphing
@hallow garden yea the shape is not the issue
Mb
Now it’s just the graphing I need to fix to visualize it
hallow garden
I will try to replicate the experiment on my side, out of curiosity
Can you give me the code for making the dataset?
@fossil root
hallow garden
Yes
fossil root
@hallow garden got it?