#🔒 Missing column information

5 messages · Page 1 of 1 (latest)

little berry
#

I have a float column in my dataset and it has missing values in some rows. If I fill them with mean or something similar, won't i loose the info about its being missed? Isn't it important info that the column in that row is missing?

If yes, then how do I save this information and avoid ml models to give errors because of the missing column at the same time?

zinc marshBOT
#

@little berry

Python help channel opened

Remember to:

  • Ask your Python question, not if you can ask or if there's an expert who can help.
  • Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
  • Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.
viral condor
# little berry I have a float column in my dataset and it has missing values in some rows. If I...

There is no universal way to handle missing data, and before doing anything with it, you must understand why it is missing.

  • the best case scenario would be fixing some issue that caused it to be lost, in which case you would now have the full data
  • another good scenario would be that the data being missing in that column represents something, in which case you may want to encode it as a separate value not present in the actual data (e.g. 1, 2, 3, 4, 5, missing -> 1, 2, 3, 4, 5, -1)
  • if it is just "do not know / did not collect when creating the dataset in first place", you could do the above or fill in with the mean or median, but you are in fact losing some information by doing this
  • in some cases, if it's caused by some rare bug, human error or some other weird issue, it might be justifiable to simply drop the rows with missing data
zinc marshBOT
#
Python help channel closed

This help channel has been closed and it's no longer possible to send messages here. If your question wasn't answered, feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.