Sync has stopped working again | Atuin | Page 1

stray raft Oct 14, 2023, 6:29 PM

#

I spoke to soon, all of my boxes fell out of sync again. 😦
Robert Sink — Today at 2:25 PM
Box A:
[Local]
Sync frequency: 0
Last sync: 2023-10-14 18:23:18.081970 UTC
History count: 44003

[Remote]
Address: https://atuin.my.domain
Username: robert
History count: 43180

Box B:
[Local]
Sync frequency: 0
Last sync: 2023-10-14 18:23:19.890372 UTC
History count: 43409

[Remote]
Address: https://atuin.my.domain
Username: robert
History count: 43180
What can I do to diagnose this? It looks like something either remote history 43180 count is causing the issue.

fallow drum Oct 14, 2023, 6:30 PM

#

Do you have any logs from your server?

#

RUST_LOG=debug

stray raft Oct 14, 2023, 6:31 PM

#

It's not on, but I guess I should just run with it on perpetually. Let me turn it on.

#

ok, here''s the log from a sync from a client that will not push new records:

📎 message.txt

fallow drum Oct 14, 2023, 6:35 PM

#

I’m mostly interested in the server

stray raft Oct 14, 2023, 6:35 PM

#

That is the server log.

#

That's what was generated from the server when I tried atuin sync

fallow drum Oct 14, 2023, 6:36 PM

#

ahh I see

stray raft Oct 14, 2023, 6:38 PM

#

I'm at a loss as to why it goes out to lunch. Doing atuin history list, capturing to a file and looking at the lines circa the last remote count (43183 presently), I don't see a heinous history entry that would explain why things would go off the rails.

#

The SQLite part seems to work beautifully, but when we involve the sync to PostgreSQL, things just go awfully bad.

#

I'm kinda at a point of juts giving up. This has persisted across many versions and there doesn't seem to be a fix. I thought hosing out my history was the fix, but since after v12 (I think), I've just had no luck and no one really knows why. I'm not doing anything crazy over here.

stray raft Oct 14, 2023, 7:17 PM

#

Last ditch effort, I tried switching back to api.atuin.sh, and my counts still differ. Oh well, I'll just use unison to diff in important history changes and call it a day. Hopefully one day you'll figure out what's going on. I can't be the only outlier here.

stray raft Oct 14, 2023, 10:19 PM

#

Is this my problem? The HTTP 413? [entity too large]

2023-10-14T22:18:58.181402Z DEBUG request{method=POST uri=/history version=HTTP/1.1}: tower_http::trace::on_response: finished processing request latency=0 ms status=413

fallow drum Oct 14, 2023, 10:20 PM

#

Yeah that will be an issue. How large is your history? 😅 if it’s the one I host we don’t accept post bodies beyond 100mb

stray raft Oct 14, 2023, 10:21 PM

#

No, this is my localhost again. My history count is 89861.

fallow drum Oct 14, 2023, 10:22 PM

#

Ahhh. It’s nothing to do with your history account, it’s paged up in post bodies of a set size. One of those pages is too large

#

History count*

#

You’ll need to set max history size to 0, and make sure your reverse proxy can handle it (if you’re using one)

stray raft Oct 14, 2023, 10:23 PM

#

I'm talking directly to localhost:8888 on the same box. And max_history_size is already set to 0.

#

Hold on, is it max_history_size or max_history_length?

fallow drum Oct 14, 2023, 10:24 PM

#

It’s in our docs and code, I’ll try and check but I’m on my phone

stray raft Oct 14, 2023, 10:24 PM

#

A can grep it.

#

Yep, max_history_length is set to 0 and I'm still getting the 413.

ornate patrol Oct 15, 2023, 12:08 AM

#

feels like there needs to be a client-side setting for page-size in what it will generate, rather than needing the server to go ever larger

fallow drum Oct 15, 2023, 12:24 AM

#

@stray raft try setting a smaller page size on the server to work around that

#

The issue with having it set on the client is that it’s still up to a maximum of whatever the server supports. For 99% of people each line of history is tiny, so a huge page size is absolutely fine. To be honest, I don’t know what’s in Robert’s history and I’m not sure I’d wsnt to support it on the server if it’s that large

#

So it’s almost the case here that it’s working as expected, because huge history isn’t good for the process or system as a whole

ornate patrol Oct 15, 2023, 12:57 AM

#

oh, if the server advises the client of the page size, fine.

fallow drum Oct 15, 2023, 1:01 AM

#

Yep! It’s included in the index. Though it’s worth checking where the 413 is coming from. If there’s no reverse proxy in the way we probably have a middleware that could do with configuring. So long as the operator is ok with giant post bodies 🤷‍♀️

stray raft Oct 15, 2023, 12:55 PM

#

So, yesterday I went the other way and had page_size ridiculously high (65535) during my localhost testing and it was failing miserably (i.e. not syncing at all). After seeing this thread, I dropped to page_size = 10 and was actually able to get all 89k history records into the DB.

This is a very good 1st step, but given previous false starts I'm afraid to call success just yet.

#Sync has stopped working again