🔒 Categorical data type vs discrete data type in a dataset in pandas

[patent glen]

Hello guys, sorry to disturb you all; consider the following values for some columns in a dataset:

SEX: Gender (1=male, 2=female)
EDUCATION: (1=graduate school, 2=university, 3=high school, 4=others, 5=unknown, 6=unknown)
MARRIAGE: Marital status (1=married, 2=single, 3=others)

My question is, when we say something is categorical, does that means it only takes binary values? In that case, education and marriage should be categorical? Or should they be discrete please.

Here is the link for the data set with the above information:
https://www.kaggle.com/datasets/uciml/default-of-credit-card-clients-dataset/data

[midnight sail]

@patent glen

Python help channel opened

Remember to:

• Ask your Python question, not if you can ask or if there's an expert who can help.
• Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
• Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.

[patent glen]

Anyone to help please

[eager venture]

My question is, when we say something is categorical, does that means it only takes binary values?
no - categorical does not mean it only takes binary values.
In that case, education and marriage should be categorical? Or should they be discrete please.
could you explain what you meant by discrete please?

[patent glen]

like here, they take values of 1,2,3,4 etc ... this goes on until a max value, say 6 for eg, so 1-6

We can't have a value of 1.5 for education so in this case, we say it is discrete?

[eager venture]

and in your point of view how is that different to categorical?

just trying to understand what you meant by discrete before answering properly just in case there is confusion

[patent glen]

hmmmm basically from what I know, an attribute, can either be continuous, discrete or categorical. For eg, a continous variable can be height of something, a discrete variable can be the number of cylinders in a car, something like that. But I never really understood what differs categorical and discrete variable because consider the number of cylinders in a car, here can we say it is categorical? or it depends on the context? Like for education, here we have been explicitly been told that 1= something, 2= another thing for eg

The difference will be how I will need to handle them if they contain missing values :c

[eager venture]

perfect. so continuous and discrete like in math/stats.

[patent glen]

yeah

[eager venture]

in one or two sentence, what are you most confused about?

[patent glen]

how to identify whether an attribute is categorical or discrete in case the attribute itself isn't binary, like it can take values from 1 to 6 for eg

[eager venture]

understood!

it's sometime really just a personal preference and there are no clear answers.

it's important to note that even "binary" (as in things that ONLY take on value of 0 or 1) itself can also be called categorical (2 categories)

sometimes it's really obvious though, like sure you can model age of a person as categorical but there is a progression that makes sense, and it makes much more sense to treat it as discrete (or even continuous)

for categorical, sometimes it's also very easy to tell, like countries of the world for example, sure you can model it as the lexicographical order of the ISO 3166 country code like (US,GB,HK) into (3,1,2)... but it doesn't make much sense does it? it's much easier to just use US,GB,HK as is. (also what happens if i add more countries in my dataset? the order might change?)

[patent glen]

yep, it is easier to make countries behave as categorical data type

[eager venture]

it's also important to note that pandas treats categorical data a bit differently.

e.g. if you groupby categorical and do aggregation, i believe it sometimes will return any missing categories in its result as well.

also it's a lower memory footprint to use categorical, sometimes.

[patent glen]

yep I see, if we have to handle missing values in categorical data, is there a best practice ?

[eager venture]

as in your dataset has missing values for a categorical type data?

[patent glen]

nope not in this one but just in general

[eager venture]

it's highly dependent on your dataset and what your goal is

missing value in itself could be some sort of signal as well.

sometimes blindly trying to fill them in is counterproductive.

and some models are capable of handling them out of the box, e.g. decision trees/random forests iirc

[patent glen]

yep I see

Thanks !

[midnight sail]

Python help channel closed

This help channel has been closed and it's no longer possible to send messages here. If your question wasn't answered, feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.