Question about data management | Node.js | Page 1

torpid siren Dec 26, 2022, 3:36 PM

#

So, I retrieve JSON data every x-time and would like to store that data to be able to get a feed of the data and update it with the new retrieved data frequently, the new data that might contain already-stored data which I don't want to duplicate.
What I want to do with is being able to "alert" for every single item retrieved that isn't already in the stored data.

modern moon Dec 26, 2022, 4:27 PM

#

Ah, yes. This is very doable

#

I'll get back.

torpid siren Dec 26, 2022, 5:07 PM

#

Alright, thanks man!

torpid siren Dec 26, 2022, 10:32 PM

#

modern moon Ah, yes. This is very doable

I just got to know about RSS Flux, would that be a great option?

#

https://www.rssboard.org/rss-specification

RSS 2.0 Specification (Current)

The current version of the RSS 2.0 specification (2.0.11), published on March 30, 2009.

torpid siren Jan 8, 2023, 4:41 PM

#

Hey! @modern moon

modern moon Jan 8, 2023, 4:43 PM

#

Lol, sorry. Completely forgot this

#

RSS is a bit overkill imo

#

Is it a very very big set of data?

mild locust Jan 9, 2023, 9:37 PM

#

If the retrieval is an action, you can do a reduce on the two datasets, but depending on the size of the data that might not be the best action.

Since this is polling, is there not a way for you to only poll data that came in after your last request? Eg. If it's your own data, you should be able to make this happen.

#Question about data management