🔒 Problem with merging data

[spring reef]

@brittle hollow

Python help channel opened

Remember to:

• Ask your Python question, not if you can ask or if there's an expert who can help.
• Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
• Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.

[ripe gate]

Hello, please give the code and error message as text

!code

[spring reef]

Formatting code on Discord

Here's how to format Python code on Discord:

```py
print('Hello world!')
```

These are backticks, not quotes. Check this out if you can't find the backtick key.

For long code samples, you can use our pastebin.

[gaunt pelican]

what columns are in mf_data

[ripe gate]

This is a screenshot, which is not text.

But just give the code as text for now.

[brittle hollow]

from pathlib import Path
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from scipy import stats
import seaborn as sns
from linearmodels import PanelOLS, IV2SLS

crsp = pd.read_csv(Path('data/CRSP_msf_2.csv.gz'), compression='gzip')
crsp['MthCalDt'] = pd.to_datetime(crsp['MthCalDt'])
mf_data = pd.read_parquet(Path('data/MutualFund_data_Comp.parquet'))
mf_data['Date'] = pd.to_datetime(mf_data['Date'])

crsp_q = crsp.set_index('MthCalDt').groupby('PERMNO').resample('QE')['MthPrc'].last().reset_index().assign(Quarter=lambda x: x['MthCalDt'].dt.to_period('Q'))

crsp_q['log_price'] = np.log(crsp_q['MthPrc'])
crsp_q['log_price_change'] = crsp_q.groupby('PERMNO')['log_price'].diff()
crsp_q['lag_log_price_change'] = crsp_q.groupby('PERMNO')['log_price_change'].shift()

def merge_datasets(crsp_q, mf_data):

merged = pd.merge(
    crsp_q,
    mf_data.assign(Date=lambda x: x['Date'].dt.to_period('Q')),
    on=['PERMNO', 'MthCalDt'],
    how='inner'
)
return merged

merged_data = merge_datasets(crsp_q, mf_data)

[spring reef]

Hey @brittle hollow!

Please edit your message to use a code block

```py
print('Hello, world!')
```

This will result in the following:

print('Hello, world!')```

[ripe gate]

@brittle hollow please do what the bot said.

[ripe gate]

@brittle hollow please copy and paste the text output of print(crsp.to_dict('list'), mf_data.to_dict('list'))

[brittle hollow]

there you go!

[gaunt pelican]

mf_data has a 'permno' column, not 'PERMNO'

[brittle hollow]

I did tried to change it to permno but I gave the same result

[gaunt pelican]

try to make both dataframes have the same column name

[brittle hollow]

mf_data = mf_data.rename(columns={
    'permno': 'PERMNO',  # Correction du nom de colonne
})

I tried to rename the column by rename it

but now it give this error

[gaunt pelican]

same error, MthCalDt doesn't exist in one of the dataframes

[brittle hollow]

I was able to not have an error

but now, it idk why but my dataframe is empty

[spring reef]

Python help channel closed using Discord native close action

This help channel has been closed. Feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.