🔒 [Discussion] "Help! Machine Learning Model Struggling with High-Dimensional Reflectance Data

[valid rock]

I'm assisting with a research project at my university, and I've run into a bit of a roadblock. I've incorporated reflectance data variables into my machine learning model to predict a Y. There are 1500 different wavelengths, significantly increasing the dimensionality of my data. After combining the datasets, the model's performance declined. I tried reducing dimensionality, but the model continued to worsen with this data.
The research lead (a Ph.D. student) suggested I use a genetic algorithm because they've seen it used with reflectance data before. I found the implementation to be pretty complex, and I don't think this clustering will improve the model (I'm studying the implementation and believe I'll be able to test it soon).
What do you guys suggest? I think there are two approaches: either I start removing wavelengths that are worsening the model through exhaustive search, or I do this reduction using a genetic algorithm.
Has anyone encountered a similar problem? I haven't gotten satisfactory answers from AI because I understand they're not that advanced in this area yet.

[rustic kindle]

@valid rock

Python help channel opened

Remember to:

• Ask your Python question, not if you can ask or if there's an expert who can help.
• Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
• Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.

[rustic kindle]

@valid rock

Python help channel closed

This help channel has been closed and it's no longer possible to send messages here. If your question wasn't answered, feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.