🔒 Pandas filtering boolean mask

[spark violet]

Hello guys, sorry to disturb you all; I'm learning some basics pandas filtering. I just learnt that we can use boolean masks (from what I've understood, a boolean mask is just a set of columns and rows where each cell is mask with a True/False value used to filter data - please do correct me if I'm wrong) Can someone explain why the following statement doesn't work while the other work please:

rewards_df[rewards_df.isna().sum() > 0] // didn't work

rewards_df.isna().sum()[rewards_df.isna().sum() > 0] // this one work

Why the first one is incorrect

[radiant bay]

@spark violet

Python help channel opened

Remember to:

• Ask your Python question, not if you can ask or if there's an expert who can help.
• Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
• Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.

[rose panther]

.sum() on a dataframe sums over the index by default, so it returns a value per each column, which I don't think is something you can index a dataframe with

I suspect you're looking for something like rewards_df.loc[:, rewards_df.isna().sum() > 0], perhaps

[spark violet]

yeah chatGPT gave me something like that but I didn't understand, what are we doing here when we use the .loc method

ah I see, the : means every rows, we are filtering by the columns where we have only null values

but then hmm the .loc returns what I forgot 😭

[rose panther]

yeah, df[thing] is more like df.loc[thing, :]

[spark violet]

ah I see, so it will return a dataframe then, just 2sec I try it

ah I see the difference

in fact yes we get a dataframe, the dataframe of only null values

the difference with this one:

rewards_df.isna().sum()[rewards_df.isna().sum() > 0]

is that this return a series, where the rows are indexed with the column names; we won't have every column name though, only those where we have null values, then for each column we have their respective sum of null values

[rose panther]

yeah

[clever dragon]

I noticed that you used “😭” in your comment. Just wanted to say, don’t give up anything in your life. I don’t know what you’re going through but I’m always here to help.

[radiant bay]

Python help channel closed

This help channel has been closed and it's no longer possible to send messages here. If your question wasn't answered, feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.