#Syncing updates when multiple users are changing the same thing

54 messages · Page 1 of 1 (latest)

rocky wigeon
#

So imagine you have a note taking app like notion. If you have a get query and an update mutation on the content of that note - what's a good way to keep it in sync but not overreactive on each user's client?

If I'm typing something and the update mutation fires, then the get query sets the old value of what i typed in the note and it appears to be very buggy. I feel like I'm using convex wrong here and not sure if i should be doing this at all?

The same goes for if another user types/deletes something while im typing in that note - it goes crazy on both our clients since itll sync a value a bit delayed from the current one u typed ( since the api call itself takes lets say 200ms )

rocky wigeon
#

I'm trying this in Nextjs with something called tiptap btw - A solution that would work is if i use a separate websocket server for each note that uses somehting like hocuspocus to do the merging of updates from various clients and I only use convex to load data initially in that server ( when a client connects ) - rest of the updates is synced by the separate websocket server

subtle sun
#

I wonder how well it would work to use y.js direct in convex functions to merge updates against a stored note

rose drift
#

@rocky wigeon as it so happens, @hexed coyote is about to release a notion clone built on convex very soon

rocky wigeon
rocky wigeon
rose drift
#

at the end of the day, the strategy most editors using for multiple simultaneous authors when faced with network delay or network disconnectivity is Operational Transformation (https://en.wikipedia.org/wiki/Operational_transformation )

Operational transformation

Operational transformation (OT) is a technology for supporting a range of collaboration functionalities in advanced collaborative software systems. OT was originally invented for consistency maintenance and concurrency control in collaborative editing of plain text documents. Its capabilities have been extended and its applications expanded to i...

#

Scroll shows off how to do that with convex

rocky wigeon
#

yjs uses crdt right?

#

is the notion clone also gonna be using OT?

rocky wigeon
rose drift
rocky wigeon
#

where did you read that? i cant find it in the tweet

#

but that clone is very cool, would be massively useful as an example with convex

rocky wigeon
rocky wigeon
#

dont understand how to use typecell in your own projects tho catThink theres no 'npm install' or anythin

rose drift
#

me neither. part of why I'm looking forward to antonio's video

fresh zephyr
rose drift
#

same!

subtle sun
#

Interestingly, Typecell's main page says it's powered by yjs crdt

#

(the collaborative functionality, that is)

shell iron
#

I'd love to see Yjs in Convex! It should be a great platform for it.

rocky wigeon
#

Let me try all of these and see if i can come up with some simple example that would help future ppl coming here

young maple
#

@rocky wigeon, I actually did use Tiptap in that project, but I used a ProseMirror plugin (https://github.com/rjdellecese/scroll/blob/b2f0caf4bf588141db8eafc7b54f63db59c7745d/src/elm-ts/note.tsx#L562-L564) rather than the Tiptap collaboration extension (https://tiptap.dev/api/extensions/collaboration) because doing the latter would have required building a Yjs provider for Convex, and at least at the time, there was no documentation on how to build a custom Yjs provider. From some cursory internet searching, it looks like that's still the case (e.g. https://docs.yjs.dev/tutorials/creating-a-custom-provider).

#

But generally speaking, I do think that CRDTs and Yjs in particular are where all the momentum is at right now.

#

Although OT is still possibly simpler to integrate and definitely works well for the vast majority of regular collaborative editing use cases!

rocky wigeon
#

I WAS struggling with yjs cus yea there isn’t much docs 😭

#

I’m currently tryna figure out ur open source notes app :3

young maple
#

Yes, I struggled with that too! I knows it's a lot to take in, let me know if I can answer any questions about how it works 🙂

rocky wigeon
rose drift
# young maple Although OT is still possibly simpler to integrate and definitely works well for...

Yeah, I think in general the biggest difference is if you have a central server, OT is usually much easier to manage. If you want truly universally reconcilable divergences (e.g. peer to peer communication), they you'll need to use CRDTs b/c you can't rely on an entity providing one serialized order of the steps. so when you're using convex [which acts as a centralized mediator], OT is probably simpler

shell iron
rocky wigeon
young maple
rocky wigeon
#

o oops didnt read it fully

#

maybe i cud just simply delete old steps if everyone is disconnected?

#

ah but what if they were offline i cant do that hmph

#

okay ur right after all, need some version number

#

im curious what was your motivation to make scoll.ink? i think its pretty cool

young maple
#

My gut says just let the table grow forever and see if you run into issues, and by the point at which that happens (if ever), you'll have a better sense of the needs/habits of your app/users and therefore a better sense of how to fix it

#

Thanks 😀 I spent ~2 yrs (👴) trying to build a note-taking/knowledge management app to sell to businesses/small teams. After I gave up on that, I took some time off and discovered Convex, and used Convex (plus my experience building the note-taking/knowledge management app) to prototype some ideas about what I'd want in a personal note-taking app.

#

Specifically the idea behind Scroll was to treat note-taking as a continuous process without explicit boundaries between text. Like a never-ending scroll, conceptually—so the next thing you write is always after the last thing you wrote, unlike most note-taking apps, which asks you to create a new note at the top every time.

rocky wigeon
#

ohh i see

#

im building one specifically for unorganized notes because despite there being a lot of note taking apps my inability to organize/find things doesnt allow me to use any of them effectively

young maple
rocky wigeon
#

i definitely will! im almost done with like a barebones version of the concept but got stuck on syncing the notes xD

#

but thanks to ur repo i kinda get how to make it work with "OTs" ( that prosemirror plugin mentions its not exactly how the paper implemented it but whatever it works xd)

rocky wigeon
#

i found this recently '