dreamy hull
I am a current Nimbus Notes user and would like to migrate my workspaces from there to Notesnook. There is no direct importer, or there doesn't seem to be one. Nimbus is kind of an outlier in the notes space, so that's not entirely unbelievable.
I would like to not lose my tags. Anyone tried this and is there a clean way to do it?
cerulean escarp
I am a current Nimbus Notes user and would like to migrate my workspaces from th...
Writing an importer for nimbus isn't impossible. What export formats do they support?
dreamy hull
PDF and HTML
cerulean escarp
That's it? They don't have their own backup format or something like that?
dreamy hull
cerulean escarp
Do the HTML files contain tags someplace?
dreamy hull
I'm looking. Some of my notes have been through multiple services. The one I was just looking at the HTML export for, I thought I'd identified the HTML for it, but it turned out to be from Bear. 😦
cerulean escarp
We support textbundle imports (supported by bear & a few other notes apps).
dreamy hull
I stopped using Bear because it wasn't cross-platform.
a note
I don't think it includes the tags
cerulean escarp
Yep. No tags.
dreamy hull
And they're not in the PDF export, all you get when you do that is as if you printed the page.
They do backups, but you can't download the backup
Just restore from it
The world won't end if I can't export the tags.
It's just a PITA
I need to experiment with their bulk export too, see how it structures the notes. Notesnook says it can import HTML, but I'm sure there has to be a particular structure to it.
cerulean escarp
@dreamy hull do you have some coding experience?
dreamy hull
some.
Some, but I'm no programmer.
Now I wonder if the export works from the Mac, from their Windows client, their instructions on the web don't match the menu structure of the note.
cerulean escarp
I think the directory structure represents tags
dreamy hull
They tell you that you have to use the desktop client(s) to do a bulk export, but neither one of them has the described options in the menu
On the Mac their client is Electron.
On Github I found a python script to transform their export into SimpleNote, but if you can't bulk export, it isn't worth much
dreamy hull
This'll teach me to not save previous export files 😦
Shoulda saved the file when I left Evernote, at least then I'd have something.
I opened a support ticket with Nimbus, will report on what they say.
cerulean escarp
They might be able to give you a backup.
dreamy hull
Watch them provide a SQL dump or something.
cerulean escarp
Haha well if it doesn't work out, tell me and I can probably write up a quick n' dirty script to automate exporting for you.
dreamy hull
scrape their web client?
cerulean escarp
Yep
dreamy hull
I'll let you know what they say.
cerulean escarp
Alright.
dreamy hull
Thanks!
dreamy hull
@cerulean escarp Well, I was directed to this support post, where they make available a version of their Windows client that supports the notes export.
What you get is a folder for each folder you have in the client, inside of which you have a zip file for each actual note. If the note has attachments, they are included in the zip file.
Still no tags, but hey, you can get the actual data out.
hello, helps if I provide the URL... https://support.nimbusweb.co/portal/en/community/topic/export-all-notes-is-missed-in-desktop-app
cerulean escarp
@dreamy hull Locking out users is not great and there should be a way to get all the data + metadata out.
In any case, I took a stab at their API and was able to wire up an exporter that can export everything including tags. It's not finished though.
Let me know which you'd prefer.
dreamy hull
I'm certainly game to test, how do I go about it?
cerulean escarp
I haven't finished it yet but once its done, I can ping you here.
Oh and if I take too long feel free to ping me as well.
dreamy hull
Sounds good.
dreamy hull
I took a shot at running an export from Nimbus though the script from here: https://github.com/SpaceShaman/Nimbus-Note-to-Simplenote
I got something the importer could digest (in that it produced no errors as a result of the import), but I wouldn't call the result usable.
Which is totally a result of the input data, not the import tool.
dreamy hull
And it didn't bring the attachments, which... Simplenote, so only makes sense. (sigh)
cerulean escarp
@dreamy hull yes that won't work.
dreamy hull
@cerulean escarp well yeah, but I wanted to try.
dreamy hull
@cerulean escarp Did you get anywhere with a Nimbus importer?
cerulean escarp
<@268906913855438848> Did you get anywhere with a Nimbus importer?
Yes, it's 90% done. Just have to see if we can integrate it with the Importer directly. But exporting is done and it exports everything.
dreamy hull
Ooo, then I can kick Nimbus to the curb.
I saw there were a couple of issues opened on Github from people wanting to import Nimbus, so I’m not the only one.
cerulean escarp
For sure. Although the amount of those users isn't that big but we'll support what we can.
dreamy hull
Hello @cerulean escarp Any update/news on the Nimbus Notes importer? Thanks!
cerulean escarp
Hello <@268906913855438848> Any update/news on the Nimbus Notes importer? Thank...
I'll try and get something up this week. Sorry for the long wait.
dreamy hull
@cerulean escarp W00t!
cerulean escarp
@dreamy hull alright so I published the exporter. You can use it and see if it can export all your notes etc. More details here: https://github.com/thecodrr/nimbus-note-exporter
NOTE: this is the exporter. Not the importer. I still have to add support for this in Notesnook Importer but that should be trivial.
Let me know if you don't understand something.
dreamy hull
I have to install npm locally? will try tonight!
cerulean escarp
yes
dreamy hull
actually fsck that
one moment
hm, "found 0 notes across 0 workspaces"
"waiting for download urls" and it's got a spinning cursor
cerulean escarp
Yeah, because there's no notes so it is stuck.
The main problem is that it isn't finding the workspaces for some reason.
dreamy hull
there are three
cerulean escarp
Can you send a screenshot of the output it shows?
https://posguy99.nimbusweb.me/gwapi2/ft%3Atasks/workspace-infos
Can you open this in your browser where you are logged into Nimbus note?
See what it returns
You also forgot to mask your email in the screenshot. Feel free to delete your message.
cerulean escarp
Ah so that's why.
It should have a list of all your workspaces. It was working for me but I might have to find a different way to get the workspaces.
I'll take another look at this in a couple of hours and then ping you for another try.
dreamy hull
ok
cerulean escarp
@dreamy hull update and run again. Should work now.
dreamy hull
Ok I'm on transit on my way home, if you're around in a hour or so I will post a result then.
cerulean escarp
Awesome
dreamy hull
@cerulean escarp Well, I did 'sudo npm update', it didn’t claim to have updated anything. I am not a node/npm user, just a dabbler, so wouldn't necessarily know if it had updated anything
cerulean escarp
No do the installation steps again just for the exporter.
dreamy hull
'k, moment
cerulean escarp
You don't need to update npm or node.
dreamy hull
oh, THAT'S what that syntax means
Well, that's the right number of workspaces, but not nearly the right number of notes
And then it stalled again at the waiting line
There are over 2500 notes between the three workspaces
it does a bunch of "exporting garbagethatisprobablyanoteid as html" before it stalls at"waiting for download urls" again
cerulean escarp
And then it stalled again at the waiting line
It will stall. I'll add some progress to it so you have a better idea what's going on. Also I think the notes are paged and its only getting the first page
cerulean escarp
@dreamy hull update and try again. I added pagination that should take care of you getting all the notes.
dreamy hull
@cerulean escarp It stalls after the password prompt now with just a spinner and "logging you in..." I'm sorry to be a pain.
I think they're having a problem, the web app is slow slow slow
but it finally connected
I came back to the exporter terminal and it had moved on
I'll let you know what it says when it finishes. It says it's getting tags right now
dreamy hull
cerulean escarp
I think they're having a problem, the web app is slow slow slow
Definitely. Their service went down with 503 errors and even now stuff isn't working properly. Individual note exporting is broken (nothing happens when you try it).
dreamy hull
@cerulean escarp Ok, starting again. Let's see how it works. ^_^
Folders now!
I left it running this morning and it got wedged here: "Saving download urls (2583/2791)" ... but I'm sure that was THEM
dreamy hull
@cerulean escarp It got stuck again. Any way to tell what it's doing? It hasn't created any files in the current directory.
dreamy hull
Still stuck @2785 notes
Which is odd because Nimbus itself thinks there are only 2776 notes
cerulean escarp
Which is odd because Nimbus itself thinks there are only 2776 notes
Those must be the trashed notes.
cerulean escarp
<@268906913855438848> It got stuck again. Any way to tell what it's doing? It ...
It moves on to downloading the files after all the URLs have been saved.
dreamy hull
there are things in the trash, I can clear them out if that's the problem. It's not like I expected them to be exported
whatcked
whacked, even
cerulean escarp
there are things in the trash, I can clear them out if that's the problem. It's...
It makes no distinction but you can try it.
dreamy hull
I thought you meant that it stalled (it's been sitting on the same number for the last couple hours) because of the trash
I hate Nimbus
cerulean escarp
I thought you meant that it stalled (it's been sitting on the same number for th...
I think it stalls because Nimbus isn't sending out the export URLs for some notes.
dreamy hull
Now the tool thinks there are 2769 notes
cerulean escarp
It might not be a bad idea to introduce a cache so you can continue where you left off.
dreamy hull
and Nimbus' management tool thinks there are 2754.
My head hurts
This is one of the reasons I want out, their tools are so... messed up
cerulean escarp
Not sure where those extra notes are coming from.
dreamy hull
Yea, it doesn't say how many are in each workspace.
I mean Nimbus does, but who knows if it can be trusted
I assume the API is rate-limited, it takes a long time to fetch URL's
cerulean escarp
I assume the API is rate-limited, it takes a long time to fetch URL's
Not sure. It's asynchronous. You request the URLs for the notes and then Nimbus responds if they want independent of that. It's like telling a random person you want a Pizza and then its up to them whether you get one or not.
They don't make it easy or simple.
dreamy hull
I like the analogy.
That's another reason to leave them they're deliberately taking steps to make it hard to leave
cerulean escarp
That they are. This should be part of the main app's functions.
dreamy hull
Makes me wish for Evernote again (shudder). And that''s bad.
There aren't a lot of truly cross-platform notes systems
Probably would benefit from even a timeout, if it no longer gets responses offer to save what it has. OTOT, if I understood you correctly, it;'s just building the list, and it might get just as stuck when it actually tries to download the data.
cerulean escarp
Probably would benefit from even a timeout, if it no longer gets responses offer...
Most probably. I think a small caching layer would make sense. I'll see if I can add one. It'll also help take the load off of Nimbus. They keep crashing probably due to this.
dreamy hull
That's kinda appropriate (although not useful) if doing something like this is truly crashing their endpoint. Poetic justice.
dreamy hull
@cerulean escarp I've been cleaning up, I cleaned out two workspaces, deleted a ton of notes, the current version of the exporter still thinks there are two more notes than there are, so it gets to the end, and it stalls I assume waiting for the download URL of those last two notes.
It shows the total number of notes twice, the first time it has it right, the second time there are two more.
The "getting notes metadata" line has the right number of notes.
Then when it starts this, it's added two.
✔ Found 2137 notes across 2 workspaces
⠼ Saving download urls (534/2137)..
The actual number is 2135.
As I was cleaning out stuff, the difference between them would change, but the "notes metadata" line was always right. Or I think it was always right.
cerulean escarp
@dreamy hull what would be the best way to find those 2 notes?
dreamy hull
@cerulean escarp What is it doing that the two numbers don't match? If the one message line says there are N notes, then when it actually goes to try getting download URL's, why is it N+somenumber?
cerulean escarp
<@268906913855438848> What is it doing that the two numbers don't match? If the...
The first number is present in the workspace metadata. The actual number of notes differ from the number in there. Whatever the case, it should still download the notes.
The fix would be to detect which 2 notes are extra and then see why they are being included and how to ignore them.
dreamy hull
@cerulean escarp I wonder if you can have it write out, whether to a file, or to stdout/stderr, the titles it thinks it's trying to export. I've continued to trim out old crud from Nimbus, and I'm down to a count of one difference between the two limits, so it still hangs. 😦