🔒 Problem with django and celery

[snow junco]

I have this function, generate_and_save_recommendations(), if i run it alone, it works fine, but when i run it with celery as generate_and_save_recommendations.delay(), I get an error where it can't access a newly created Item.. any ideas of how i can fix this?
I'm calling the function like this:

def update_recommendations_on_item_save(sender, **kwargs):
    the_instance = kwargs['instance']
    generate_and_save_recommendations.delay()
post_save.connect(update_recommendations_on_item_save, sender=Items)

error:

[2024-04-01 22:42:10,686: ERROR/MainProcess] Task Recommender.Contentbased.generate_and_save_recommendations[f5297247-aaf2-4dc2-bd0c-6fef93be23dd] raised unexpected: KeyError('example_item69')
Traceback (most recent call last):
  File "C:\Users\sorou\miniconda3\envs\.venv\lib\site-packages\pandas\core\indexes\base.py", line 3805, in get_loc
    return self._engine.get_loc(casted_key)
  File "index.pyx", line 167, in pandas._libs.index.IndexEngine.get_loc
  File "index.pyx", line 196, in pandas._libs.index.IndexEngine.get_loc
  File "pandas\\_libs\\hashtable_class_helper.pxi", line 7081, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas\\_libs\\hashtable_class_helper.pxi", line 7089, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'example_item69'


"""it's longer than this but i hit the message limit if i send that..."""

[buoyant spear]

@snow junco

Python help channel opened

Remember to:

• Ask your Python question, not if you can ask or if there's an expert who can help.
• Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
• Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.

[lament jetty]

I assume you have code that looks like this ```py
item.save()
do_something_with(item.recommendation)

if I'm right, your problem is that the second line there runs before the celery task completes.

[snow junco]

the function that I'm calling, is this:

@app.task(name='Recommender.Contentbased.generate_and_save_recommendations')
def generate_and_save_recommendations():
    start = perf_counter()
    print("************************************")
    print('generating recommendations...')
    total_items = len(Items.objects.all())
    processed_items = 0
    for item in Items.objects.all():
        recommendations = get_itembasedrecmerge(item.item_id, item.site_id, get_itembasedrec)
        ib_rec, created = ItemBasedRec.objects.update_or_create(
            site_id=item.site_id,
            item_id=item.item_id,
            defaults={'item_ids': recommendations}
        )
        processed_items += 1
        progress = (processed_items / total_items) * 100

        sys.stdout.write("\033[K")
        print(f'Progress: {progress:.2f}%', end='\r')
        

    print("Item recommendation database updated")
    stop = perf_counter()
    print("Completed in: ", stop - start)
    print("************************************")

it's calling another funciton get_itembasedrecmerge
and that one is calling another one , get_itembasedrec

the error originates form get_itembasedrec

I think it happens at idx = indices[item_id]

indices = pd.Series(filtered_df.index, index=filtered_df['item_id']).drop_duplicates()

def get_itembasedrec(item_id, site_id, cosine_sim=cosine_sim, indices = indices):
    
    idx = indices[item_id]
    etc...

[lament jetty]

I don't want to carefully study all your code; that'd be too much like work 🙂

You should consider my idea, though

[snow junco]

just look at my last message

[lament jetty]

I did; nothing leapt to mind

[snow junco]

I feel like its because it's not getting the updated version of this?
indices = pd.Series(filtered_df.index, index=filtered_df['item_id']).drop_duplicates()

I see
thanks anyway

[buoyant spear]

Python help channel closed

This help channel has been closed and it's no longer possible to send messages here. If your question wasn't answered, feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.