#🔒 How can I scrape a subreddit

13 messages · Page 1 of 1 (latest)

glass creek
#

I wanted to scrape my University subreddit, but the reddit api has limits of 1k posts per tag....I have managed to scrape abt 2.4k posts but it is just over 1yr discussion....I am making a rag model based on this data so I needed more (like at least the last 3yrs). any other way????

harsh nexusBOT
#

@glass creek

Python help channel opened

Remember to:

  • Ask your Python question, not if you can ask or if there's an expert who can help.
  • Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
  • Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.
south turret
#

!rule 5

harsh nexusBOT
#
Rules

5. Do not provide or request help on projects that may violate terms of service, or that may be deemed inappropriate, malicious, or illegal.

glass creek
#

Can I get help with PRAW

south turret
#

if you're using the reddit API appropriately

glass creek
somber iron
glass creek
#

Can I know how can I get specific timeline based posts

I tried this with PRAW

But It repeats the newer posts for some reason

And never reaches the end

wary zenith
#

I also tired it once and had success with selenium, as it is able to load older posts, but you will have to most likly filter the results as sometimes ads are included

harsh nexusBOT
#
Python help channel closed for inactivity

This help channel has been closed. Feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.