🔒 Reducing API request time

[near flare]

I have 2 API endpoints that I use in a Ruby on Rails Web Application.

Those two enspoints deal with providing me with various recommendations for a user (products and forum posts).

Regarding the forum posts API endpoint. I have notices that the average request time is about 2.2 seconds and sometimes going over 5 seconds which is a lot for a non-sql database (mongo) containing under 100 forum posts.

The preprocessing that is done on the forum post uses a pre-trained Bert model to analyze the contents of the forum posts and uses matrix factorization to take inti account the liked and disliked posts from that user.

I will provide my source code in the next message

[proper roost]

@near flare

Python help channel opened

Remember to:

• Ask your Python question, not if you can ask or if there's an expert who can help.
• Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
• Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.

[near flare]

tokenizer = AutoTokenizer.from_pretrained('Twitter/twhin-bert-base')
model = AutoModel.from_pretrained('Twitter/twhin-bert-base')

def process_post_data(posts):
    post_texts = [post.title + " " + post.content for post in posts]
    inputs = tokenizer(post_texts, return_tensors="pt", padding=True, truncation=True)
    return inputs

def get_recommendations(posts, liked_posts, disliked_posts, following_ids):
    all_post_inputs = process_post_data(posts) if posts else None
    liked_post_inputs = process_post_data(liked_posts) if liked_posts else None
    disliked_post_inputs = process_post_data(disliked_posts) if disliked_posts else None


    with torch.no_grad():
        all_outputs = model(**all_post_inputs)
        all_embeddings = all_outputs.last_hidden_state[:, 0, :]

        if liked_post_inputs:
            liked_outputs = model(**liked_post_inputs)
            liked_embeddings = liked_outputs.last_hidden_state[:, 0, :]
        else:
            liked_embeddings = torch.zeros(1, model.config.hidden_size)

        if disliked_post_inputs:
            disliked_outputs = model(**disliked_post_inputs)
            disliked_embeddings = disliked_outputs.last_hidden_state[:, 0, :]
        else:
            disliked_embeddings = torch.zeros(1, model.config.hidden_size)

    similarities_liked = torch.matmul(all_embeddings, liked_embeddings.T)
    similarities_disliked = torch.matmul(all_embeddings, disliked_embeddings.T)

    recommendation_scores = torch.zeros(len(posts))

    for i, post in enumerate(posts):
        if post.author_id in following_ids:
            recommendation_scores[i] += 50

    recommendation_scores += similarities_liked.mean(dim=1) - similarities_disliked.mean(dim=1)

    sorted_indices = torch.argsort(recommendation_scores, descending=True)

    recommended_posts = [posts[i] for i in sorted_indices]
    return recommended_posts

Shoukd I consider parallelization?

[proper roost]

@near flare

Python help channel closed

This help channel has been closed and it's no longer possible to send messages here. If your question wasn't answered, feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.