#How to listen to changes in an external website?

4 messages · Page 1 of 1 (latest)

regal rock
#

I am trying to do the following:

  • Scrape rows (announcements) from a third party website
  • Store the scraped announcements into my PostgreSQL database
  • Have a GET route to fetch all announcements.

I want the scraping part to be automated; maybe with something like a goroutine?
Is there a way to constantly listen to any changes?
If my only option is a goroutine, how would I go about implementing it?

sudden creek
#

Unless the website has webhook support or something your only option is to scrape it every x minutes and compare with the last time you scraped

#

You can use time.Ticker for that

regal rock
#

Oh