#Avoiding rerenders using `setState + reconcile`

21 messages · Page 1 of 1 (latest)

warm plume
#

Hi,

Currently I have a webworker producing data slices which I want to render.

It looks a bit like this:

// limit frontend updates to max 100 per second or min 1 per second
const timer;
const lastUpdate = Date.now();
worker.onmessage = ({data}) => {
  clearTimeout(timer);
  const now = Date.now();
  const update = () => {
    lastUpdate = now;
    setState('events', reconcile(data.evts));
  };
  if (now > lastUpdate + 1000) {
    update();
  } else {
    setTimeout(update, 10);
  }
};

The problem is, when I have a <For each={state.events}> somewhere, the whole loop keeps rerendering, even if only a few elements have been prepended (they always get prepended).

I thought of trying to specify the key in reconcile (the data itself doesn't change, just position in the array) but it is 2 levels deep. The events array is of type:

type EventDataFoo = {timestamp: Date, some_other_data: any};
type EventDataBar = {timestamp: Date, some_data: any};
type Events = ({kind: 'foo', data: EventDataFoo} | {kind: 'bar', data: EventDataBar})[]

the key would be data.timestamp which is guaranteed to be unique, even over different data kinds.
reconcile expects the key to be a string tho which I can only provide for 1 level.

Am I misunderstanding something completely or is this really hard to achieve?

slow hawk
#

mm, strange that the elements re-mount with reconcile. u could try <Index/> instead? But mb also not ideal since you are prepending. It won't re-render the components but it will update all the dom-nodes.

mb you could only send the new events from the worker and do something like setState('events', produce(events => events.unshift(new_data)))?

or use <Index/> and not prepend but flip the order with flex-direction?

#

reconcile expects the key to be a string tho which I can only provide for 1 level.
wdym with this?

warm plume
warm plume
#

that's the problem I am currently trying to fix

#

right now, after like 3000 events it gets unusable slow (we had all the processing done in the mainthread aswell tho)

warm plume
#

append + flex-direction might work. need to check how good it works with cutting off the beginning tho, since we only show a slice of the data

#

(data can be up to a few million events)

#

the webworker has a list of all the events, I postMessage the number of events in (signal that gets updated with an intersection observer) and it produces back the last N events. (so I can get more data by scrolling). by default it's only 35 events displayed tho, so when new events come in, the list gets shifted

#

js needs SharedArrayBuffers for more complex data types 🙂

#

it would all be so nice if I could just have all the objects in a buffer and just get start and end index

slow hawk
#

(data can be up to a few million events)
o waw.

warm plume
#

biggest we've seen so far was like 4 million events.

#

the data processing pipeline also cuts off at around 9 million (Buffer max size reached) so there is a hard limit, it is just very high lol

slow hawk
#

by default it's only 35 events displayed tho, so when new events come in, the list gets shifted
Ok then I think <Index/> would be a good candidate actually. So if I understand correctly, every 10ms there is new data coming in and being presented, with the 35 newest being displayed on the screen? If you would scroll down you can see earlier events?

Because if that's the case, it actually would make sense if all the components re-render, since <For/> is reference-based. So if it's constantly new references being shown, it will constantly mount new components.

While <Index/> is index-based, so if you wouldn't scroll, it would just mount those 35 components and then update the props. If you would scroll down a bit you would mount more components, but there is probably only so much list-elements you can show anyway so maybe virtualisation can help there.

Also the reconciliation-process has overhead too, to match the immutable data to its references, and with millions of elements this can become significantly expensive. With <Index/> you wouldn't need to worry about references and you can skip reconcile all together.

slow hawk
slow hawk