tensorflow

[weary lynx]

Project

[hasty stirrup]

1. Wait patiently for a helper to come along.
2. Once someone helps you, say thank you and close the thread with:

+close

3. Feel free to nominate the person for helper of the week in #helper-nominations
4. Do not ping the mods, unless someone is breaking the rules.
5. If you're happy with the help you got here, and the server overall, you can contribute financially as well:

[weary lynx]

@heady osprey

[heady osprey]

Ok

So first of all, give me more context about your problem

[weary lynx]

Alright ima give a brief background but tbh it doesn’t matter, I’m certain the architecture I made for the neural network is good

[heady osprey]

Your task, your dataset, why you use a neural network

[weary lynx]

The dataset is based off an ordinary differential equation called duffing oscillator

And the ground truth is the last 2/3 of the dataset given whatever parameter

Lemme send u

So that’s the model we used but don’t worry u don’t need to know this advanced stuff

What matters is this:

[heady osprey]

That's computing d²x/dt² with x and dx/dt and a bunch of params yeah?

[weary lynx]

Yes exactly

[heady osprey]

I follow for now

[weary lynx]

Do u know about lstm

[heady osprey]

Yeah, though that's pretty old in itself

[weary lynx]

K basically I made a neural network with 3 lstm cells and each has a drop out but it’s the right way to do it (was told to do it that way)

Lemme just show u the issue first

[heady osprey]

I don't use it anymore if that's what you mean

[weary lynx]

no that’s fine I don’t use it either it’s just for this project

K here’s the problem

[heady osprey]

What is your target variable though ?

As in your prediction target

[weary lynx]

This is the prediction target:

Y train

[heady osprey]

No I mean

What is the thing you are trying to predict?

Acceleration?

[weary lynx]

That ground truth (displacement)

[heady osprey]

So x?

[weary lynx]

Yep

[heady osprey]

What are your features?

What I mean is, what are the columns of X_train

[weary lynx]

Oh

It’s a reshaped input solution I think?

[heady osprey]

Wdym by that?

[weary lynx]

Idek tbh lmao fudge

[heady osprey]

Well that's important

Your target is x(t) alright but what are you trying to predict it from?

[weary lynx]

I’m trying to predict from the ground truth

[heady osprey]

Yeah but isn't the ground truth x(t)

Predicting x(t) from x(t) should be trivial

[weary lynx]

yea but it’s given to us through the input so I think it’s meant to be trivial

[heady osprey]

I don't understand the objective here

You have a network that predicts the input from the input?

What do you even need the network for

[weary lynx]

Aye I’m not the one who made this project 😭 I guess so

[heady osprey]

I'm just generally very confused

[weary lynx]

Maybe I’m not explaining it well tbh

[heady osprey]

There is no point in using an lstm to predict x from x

The predictor is just: do nothing

It will overfit with 0 error

I mean I don't doubt that there is more to it, I just don't think I can help before I understand what it is

[weary lynx]

U saw the graph right?

[heady osprey]

Yeah

I mean try adding a simple linear layer at the end to correct biases

[weary lynx]

Basically I’m trying to produce a prediction that goes over the ground truth that’s all

[heady osprey]

If that's what you're looking for

Yes I get that

[weary lynx]

After the dense layer?

[heady osprey]

But the prediction based on what?

Lstm, lstm, lstm, dense

[weary lynx]

Ye that’s what I have

[heady osprey]

And your dense layer doesn't correct biases?

That's pretty weird

I mean, gradient should be oriented so that the bias of the dense layer goes down

Based on your figure

[weary lynx]

Dense(1) is below it

[heady osprey]

Can you view the current bias?

Of that dense layer

[weary lynx]

How would I do so?

[heady osprey]

Good question, try dense_layer.bias

With dense_layer being the dense layer object

Otherwise it's also possible that your learning rate is too high

Not sure the dropouts are useful either

Just set the dropout rate to 0

I mean in theory if you predict x from x you should be overfitting

If you have the code on github it would help me

[weary lynx]

I don’t but I might get back onto it tmr 😭 I really wanna solve this but it’s 4 am idk

[heady osprey]

I mean I wish I could help more but I hardly understand the problem in itself based on your explanation

[weary lynx]

Yeah I do appreciate the help though

What time are u on usually? In EST

[heady osprey]

It's 11am rn

[weary lynx]

OOF

well I will probably sleep 6-7 hours since I don’t have to wake up early

[heady osprey]

As much as I would like to help, I think it would be of great help if you do think about formulating the problem more clearly

More specifically, answer the following questions

1. What is your dataset? What are the input features, and what is the target?

2. What is the task you are trying to solve? Regression? Classification? Or something else? What loss function and evaluation metric are you using?

3. Do you need to use a neural network? If yes, why is your architecture adapted to this problem?

If you can answer these three questions exhaustively, I should be able to help you

[weary lynx]

Alright I will definitely try to get back to u, thanks so much have a great rest of ur day