#🔒 Why does my neural network virtually only predict positive values and very small values?

9 messages · Page 1 of 1 (latest)

vital nova
#

My LSTM neural network is not learning to predict large values even with a fairly comprehensive dataset for a school data science project to predict market changes. Here are some notes:

  • The target value is predicting 24 hour percentile change
  • The dataset is head -> tail, as tail is the most recent data.
  • shuffle = False when splitting the data for training and validation.
snow berryBOT
#

@vital nova

Python help channel opened

Remember to:

  • Ask your Python question, not if you can ask or if there's an expert who can help.
  • Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
  • Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.
vital nova
# 
def train_percent_model(df):
    # 1. Set X as all columns except for 'future_pct_change' and Y as 'future_pct_change'
    X = df.drop(columns=['future_pct_change']).values
    Y = df['future_pct_change'].values

    # 2. Train-test split X and Y with shuffle=False
    X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.2, shuffle=False)

    # 3. Scale X and Y
    scaler_X = MinMaxScaler()
    scaler_Y = MinMaxScaler()
    X_train_scaled = scaler_X.fit_transform(X_train)
    X_test_scaled = scaler_X.transform(X_test)
    Y_train_scaled = scaler_Y.fit_transform(Y_train.reshape(-1, 1))
    Y_test_scaled = scaler_Y.transform(Y_test.reshape(-1, 1))

    # 4. Reshape Y for LSTM input
    Y_train_scaled = Y_train_scaled.reshape(-1, 1)
    Y_test_scaled = Y_test_scaled.reshape(-1, 1)

    # 5. Set LSTM architecture with dropout
    model = Sequential()
    model.add(LSTM(units=32, input_shape=(X_train_scaled.shape[1], 1)))
    model.add(Dropout(0.2))
    model.add(Dense(units=1))

    # 6. Compile the model
    model.compile(optimizer='adam', loss='mean_squared_error', metrics=['mean_squared_error'])

    # 7. Add early stopping
    early_stopping = EarlyStopping(monitor='val_loss', patience=10, restore_best_weights=True)

    # 8. Train the model
    history = model.fit(X_train_scaled, Y_train_scaled, epochs=100, batch_size=32, validation_data=(X_test_scaled, Y_test_scaled), callbacks=[early_stopping])
#

After, that, I did sample predictions:

    # 9. Inverse transform Y for predictions
    Y_pred_scaled = model.predict(X_test_scaled)
    Y_pred = scaler_Y.inverse_transform(Y_pred_scaled)

    # 10. Print out sample predictions versus reality
    print("Sample Predictions versus Reality:")
    sample_indices = np.random.choice(len(Y_test), size=100, replace=False)
    for idx in sample_indices:
        print(f"Predicted: {Y_pred[idx][0]}, Actual: {Y_test[idx]}")

    # 11. Calculate RMSE on validation dataset
    Y_val_pred_scaled = model.predict(X_test_scaled)
    Y_val_pred = scaler_Y.inverse_transform(Y_val_pred_scaled)
    val_rmse = np.sqrt(mean_squared_error(Y_test, Y_val_pred))
    print(f"RMSE on Validation Dataset: {val_rmse}")
#

It literally virtually never predicts negative, and always stays around the same number:

Predicted: 0.8162628412246704, Actual: 8.324773475172648
Predicted: 0.9226087927818298, Actual: -4.965803527802767
Predicted: 0.8761753439903259, Actual: 0.2076627290621414
Predicted: 0.8702692985534668, Actual: 6.168307114771315
Predicted: 0.8807251453399658, Actual: 4.841132850420182
Predicted: 0.862083911895752, Actual: 0.39664041490384977
Predicted: 0.8798286318778992, Actual: -4.684464680181358
Predicted: 0.8997178077697754, Actual: 17.820865067148123
Predicted: 0.8688065409660339, Actual: 4.126377647065396
Predicted: 0.8233547210693359, Actual: -3.03902736123739
Predicted: 0.8556399941444397, Actual: -0.5620095838936776
Predicted: 0.6641810536384583, Actual: 17.318374497246346
Predicted: 0.868669867515564, Actual: 4.786246135727129
Predicted: 0.7901090979576111, Actual: 2.538566184417079
Predicted: 0.847415566444397, Actual: 4.012792624391186
Predicted: 0.768677294254303, Actual: 7.50696261888396
Predicted: 0.8814547061920166, Actual: 14.446133233511746
Predicted: 0.8206351399421692, Actual: 13.728181617555268
Predicted: 0.764897882938385, Actual: 0.15758652859572717
Predicted: 0.8237789869308472, Actual: -5.76218047134008
Predicted: 0.8869897723197937, Actual: -0.17241448106975427
#

Any ideas? It trains for 46 epochs before early stopping.

#

Thanks in advance

snow berryBOT
#

@vital nova

Python help channel closed

This help channel has been closed and it's no longer possible to send messages here. If your question wasn't answered, feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.