New install: Postgres doesn't appear to be running | Immich | Page 1

left obsidian Oct 17, 2023, 12:39 AM

#

Re-installing on a debian setup on pi.

On running web client:
500 connect ECONNREFUSED 172.19.0.8:3001

Seems to me that there should be ports assigned to postrges:14-alpine ?

#

@dry geode are you able to assist on this?

dry geode Oct 17, 2023, 12:43 AM

#

Immich-server logs?

left obsidian Oct 17, 2023, 12:46 AM

#

Error: getaddrinfo ENOTFOUND immich_postgres
immich_server | at GetAddrInfoReqWrap.onlookupall [as oncomplete] (node:dns:118:26)
immich_server | [Nest] 6 - 10/17/2023, 12:46:05 AM ERROR [TypeOrmModule] Unable to connect to the database. Retrying (2)...

#

dry geode Oct 17, 2023, 12:47 AM

#

And your env?

#

Is the database just not starting?

left obsidian Oct 17, 2023, 12:48 AM

#

dry geode Oct 17, 2023, 12:48 AM

#

What are in the database logs?

left obsidian Oct 17, 2023, 12:48 AM

#

whats the command to just view the db logs

dry geode Oct 17, 2023, 12:49 AM

#

docker compose logs database

left obsidian Oct 17, 2023, 12:49 AM

#

#

there was a chown error at the end of the install script

dry geode Oct 17, 2023, 12:50 AM

#

It needs to own the postgree directory

left obsidian Oct 17, 2023, 12:50 AM

#

error was "/media/michael/IMMICHMSTR/immich/library: operation not permitted"

dry geode Oct 17, 2023, 12:51 AM

#

What are the permissions on your database volume mount?

left obsidian Oct 17, 2023, 12:52 AM

#

#

i actually have no idea where the database is b'cause that database folder is empty

#

this is the relevant section from the .yml

#

I thought the intention was to have the database reside on the volume that is the ssd drive, i.e /media/michael/IMMICHMSTR/immich/database

#

(from the conversation with @mellow spindle yesterday)

dry geode Oct 17, 2023, 12:56 AM

#

Where are you running the docker compose command from? What directory?

left obsidian Oct 17, 2023, 12:56 AM

#

/media/michael/IMMICHMSTR/immich

dry geode Oct 17, 2023, 12:57 AM

#

Oh just saw your other message

#

Ok

left obsidian Oct 17, 2023, 12:58 AM

#

even though I ran the compose with sudo - I got this on the initial build... Error response from daemon: error while creating mount source path '/media/michael/IMMICHMSTR/immich/library': chown /media/michael/IMMICHMSTR/immich/library: operation not permitted

dry geode Oct 17, 2023, 12:58 AM

#

The folder is empty because it can't be initialized yet

#

For the database folder

left obsidian Oct 17, 2023, 12:58 AM

#

so what i did then was https://docs.docker.com/engine/install/linux-postinstall/#:~:text=If you don't want,members of the docker group.

#

now i can run docker commands without sudo

#

but i didnt know how to re-run the build

#

as when i do compose up - d now - it all looks fine

#

but yes i think a step in creating the db was skipped becuase of the " error while creating mount source path"

#

should i just purge and rebuild?

dry geode Oct 17, 2023, 1:03 AM

#

No

#

Can you show me the fstab entry for the mount?

left obsidian Oct 17, 2023, 1:07 AM

#

sorry linux newb - how do I do thaat?

dry geode Oct 17, 2023, 1:09 AM

#

Idk lol

#

cat /etc/fstab ?

left obsidian Oct 17, 2023, 1:10 AM

#

dry geode Oct 17, 2023, 1:12 AM

#

Two things, the volume mount might just be wrong or you actually have a permission issue. A bad mount would mean it is perpetually not writable.

left obsidian Oct 17, 2023, 1:14 AM

#

i just made a folder as a test

#

worked fine

#

so no issues writing to that database folder

#

there was a permissions issue when i ran the initial sudo docker compose - d

#

as i mentioned - not sure why - becuase it was ran as superuser

#

so whatever happens in the script from that point where it failed never happened

dry geode Oct 17, 2023, 1:18 AM

#

Weird.

left obsidian Oct 17, 2023, 1:18 AM

#

hence my asking maybe its best for me to purge and re-run now that i have given permissions to all users

#

here is another thought - I might not have a root user

#

is that possible?

dry geode Oct 17, 2023, 1:18 AM

#

I don't think so

left obsidian Oct 17, 2023, 1:18 AM

#

when i made the pi image i specified my name as the root user - so maybe there is no "root" user

#

how do i chack that

#

check

#

hmm

#

seems to be

dry geode Oct 17, 2023, 1:22 AM

#

Have you restarted since installing docker?

left obsidian Oct 17, 2023, 1:22 AM

#

nope

dry geode Oct 17, 2023, 1:22 AM

#

Worth a shot. Sometimes docker is a bit weird with permissions after the initial install

left obsidian Oct 17, 2023, 1:23 AM

#

ok rebooting

#

ok no difference

#

dry geode Oct 17, 2023, 1:28 AM

#

Still same permission error?

left obsidian Oct 17, 2023, 1:28 AM

#

see how postgres just says "starting"

#

not running

dry geode Oct 17, 2023, 1:28 AM

#

Right

left obsidian Oct 17, 2023, 1:30 AM

#

#

yeah still same

dry geode Oct 17, 2023, 1:34 AM

#

I'm not sure, I can take another look tomorrow.

left obsidian Oct 17, 2023, 1:35 AM

#

ok no worries thanks for your time!

#

i'll keep palying

marsh dew Oct 17, 2023, 2:22 AM

#

maybe try to recreate container, since you created it as a root user?
docker compose up -d --force-recreate --remove-orphans

After that, you can check if the directory mounted with proper permissions inside the container
docker exec immich_postgres "ls -lan /var/lib/postgresql/data"

#

sometimes a "hard reset" like that works
docker compose down -v --remove-orphans
docker compose up -d

left obsidian Oct 17, 2023, 2:31 AM

#

I just did a clean install of the o/s and then after installing docker too the steps it reccomded to Manage Docker as a non-root user

#

then i ran the immich build and it went through with no errors!

#

#

oh - still the same issue

#

#

man this is so frustrating

#

i am following all the strps

#

steps

#

it was easier to install on the mac docker desktop app

#

why is the chown command failing for the db - that is the question

left obsidian Oct 17, 2023, 2:39 AM

#

marsh dew maybe try to recreate container, since you created it as a root user? `docker co...

what is ' docker exec immich_postgres "ls -lan /var/lib/postgresql/data ' meant to do as it seems to do nothing

marsh dew Oct 17, 2023, 2:40 AM

#

it should show you permissions for the data directory, and its contents

left obsidian Oct 17, 2023, 2:40 AM

#

it just returns ">"

marsh dew Oct 17, 2023, 2:41 AM

#

I forgot the "

#

now, try it

#

if you are sstuck on ">" just Ctrl + C

left obsidian Oct 17, 2023, 2:42 AM

#

marsh dew Oct 17, 2023, 2:43 AM

#

do again ls -lan in your directory

left obsidian Oct 17, 2023, 2:44 AM

#

marsh dew Oct 17, 2023, 2:47 AM

#

can you mount a volume to your database instead?

left obsidian Oct 17, 2023, 2:47 AM

#

surely many have trodden in this path before me - I just dont get how on a clean install using reccomended scripts I am hitting this wall? 😦

left obsidian Oct 17, 2023, 2:48 AM

#

marsh dew can you mount a volume to your database instead?

I dont know how

#

as in happy to but I need guidnace on that

marsh dew Oct 17, 2023, 2:48 AM

#

np

#

instead - ./database: ... do a
- database: ...

#

without the ./

#

now, on the bottom, find volumes: and add one more, called database

#

after all is done, docker compose up -d --force-recreate

left obsidian Oct 17, 2023, 2:53 AM

#

ok that was a reccomendation from daniel..

#

but clearly that ain't working so I will take the steps you just mention

dry geode Oct 17, 2023, 2:54 AM

#

Have you tried running sudo chown on the directory yourself?

left obsidian Oct 17, 2023, 2:54 AM

#

nope

dry geode Oct 17, 2023, 2:55 AM

#

I think it needs to be owned by root instead of your user for it to work.

left obsidian Oct 17, 2023, 2:55 AM

#

should I try that first

#

before the change to the yml

marsh dew Oct 17, 2023, 2:56 AM

#

yeah, wouldnt hurt

left obsidian Oct 17, 2023, 2:56 AM

#

ok sorry what would the specifc command be

marsh dew Oct 17, 2023, 2:56 AM

#

sudo chown root:root ./database

left obsidian Oct 17, 2023, 2:57 AM

#

marsh dew Oct 17, 2023, 2:58 AM

#

you are not in the sudoers

left obsidian Oct 17, 2023, 2:58 AM

#

all my other sudo s have worked

#

i can try it as root

marsh dew Oct 17, 2023, 3:00 AM

#

hmm, maybe you are

#

stop the container database first docker compose down

left obsidian Oct 17, 2023, 3:01 AM

#

left obsidian Oct 17, 2023, 3:02 AM

#

marsh dew stop the container database first `docker compose down`

ahh ok yep

marsh dew Oct 17, 2023, 3:02 AM

#

if you switch to root user, there is no need to use sudo

left obsidian Oct 17, 2023, 3:02 AM

#

yep got that sorry i cut and paste it

#

🙂

marsh dew Oct 17, 2023, 3:02 AM

#

as sudo stand for Super User DO

#

np

left obsidian Oct 17, 2023, 3:02 AM

#

yup

dry geode Oct 17, 2023, 3:02 AM

#

You know what

#

I think your file system is read only still. Can you create a file in that folder?

#

The permissions are only r-x for user and group

left obsidian Oct 17, 2023, 3:03 AM

#

marsh dew Oct 17, 2023, 3:03 AM

#

rwx for user

dry geode Oct 17, 2023, 3:04 AM

#

For group and everyone

marsh dew Oct 17, 2023, 3:04 AM

#

well he downloaded compose and stuff

left obsidian Oct 17, 2023, 3:05 AM

#

#

marsh dew Oct 17, 2023, 3:06 AM

#

alright, try the volume thingy now

marsh dew Oct 17, 2023, 3:06 AM

#

left obsidian

that's some dark magic there

#

wait a sec, I see it now

#

OMG

left obsidian Oct 17, 2023, 3:07 AM

#

marsh dew instead `- ./database: ...` do a ` - database: ...`

try this now?

marsh dew Oct 17, 2023, 3:07 AM

#

you don't have root permissions

#

no no no

#

your system is badly set up

#

you are root user, creating folders, but they are owned my "michael"

#

so you aren't really a root user

left obsidian Oct 17, 2023, 3:08 AM

#

ok yeah thats an rpi default image thing

dry geode Oct 17, 2023, 3:08 AM

#

Can you stop and rrmove the stack, delete the database directory and try to start it again?

left obsidian Oct 17, 2023, 3:09 AM

#

or do i just create a root user and login as that

dry geode Oct 17, 2023, 3:09 AM

#

It should create it and it should just work. I don't understand why it is running into this error or why the directory is owned not by root

marsh dew Oct 17, 2023, 3:09 AM

#

left obsidian when i made the pi image i specified my name as the root user - so maybe there i...

this is where you did no-no

#

if it asks you for a name, just give it your name

#

a root user should be left alone

left obsidian Oct 17, 2023, 3:10 AM

#

well actually no that was an error - the root user still exists - I thought that was the case then but have read up on it

#

#

i could do this

dry geode Oct 17, 2023, 3:11 AM

#

Did you manually create the database folder? I'd delete it and restart the stack first.

marsh dew Oct 17, 2023, 3:11 AM

#

left obsidian

where is this from?

left obsidian Oct 17, 2023, 3:12 AM

#

https://thepihut.com/blogs/raspberry-pi-tutorials/how-to-change-the-default-account-username-and-password

The Pi Hut

How to Change the Default Account Username and Password

By default your Raspberry Pi comes with an account 'pi' with the password 'raspberry'. For security reasons it's probably a good idea to change the password, but you may also wish to change the username as well. There are a couple of different ways to change the default username but I found the following method the eas

#

left obsidian Oct 17, 2023, 3:13 AM

#

dry geode Did you manually create the database folder? I'd delete it and restart the stack...

i actually cant recall I think i might have whne the disk was plugged into my PC

#

so that could be an issue

#

let me delete the database folder

#

and create it again

marsh dew Oct 17, 2023, 3:13 AM

#

rm -r ./database

left obsidian Oct 17, 2023, 3:15 AM

#

ok done that

#

marsh dew Oct 17, 2023, 3:16 AM

#

sudo cat /etc/sudoers

left obsidian Oct 17, 2023, 3:17 AM

#

marsh dew Oct 17, 2023, 3:18 AM

#

looks alright

dry geode Oct 17, 2023, 3:19 AM

#

left obsidian ok done that

Can you start docker compose without creating the folder?

marsh dew Oct 17, 2023, 3:19 AM

#

okay, lets do a database directory elsewhere?

left obsidian Oct 17, 2023, 3:20 AM

#

dry geode Can you start docker compose without creating the folder?

let me try that

#

so delete the database folder

#

then do a compuse

#

ok that was interesting

dry geode Oct 17, 2023, 3:21 AM

#

This looks more and more like a filesystem problem to me

left obsidian Oct 17, 2023, 3:21 AM

#

#

ok but this is a 2nd new install of the o/s so i am not sure how or why there should be a filesystem issue

dry geode Oct 17, 2023, 3:23 AM

#

Output of du and lsblk?

marsh dew Oct 17, 2023, 3:23 AM

#

your SD card could be corrupted

left obsidian Oct 17, 2023, 3:24 AM

#

differnt sd card

#

same thing last time

marsh dew Oct 17, 2023, 3:24 AM

#

did you try to run it with a volume tho

dry geode Oct 17, 2023, 3:24 AM

#

You could always comment out the volume entirely.

left obsidian Oct 17, 2023, 3:24 AM

#

oh, btw that last command DID acgtually create the database folder

dry geode Oct 17, 2023, 3:24 AM

#

No persistence to see if it works at all

marsh dew Oct 17, 2023, 3:24 AM

#

marsh dew instead `- ./database: ...` do a ` - database: ...`

.

dry geode Oct 17, 2023, 3:24 AM

#

left obsidian oh, btw that last command DID acgtually create the database folder

With what permissions?

left obsidian Oct 17, 2023, 3:25 AM

#

so this Error response from daemon: error while creating mount source path '/media/michael/IMMICHMSTR/immich/database': chown /media/michael/IMMICHMSTR/immich/database: operation not permitted

#

it did create the database folder before that

#

as part of the compose -up

marsh dew Oct 17, 2023, 3:25 AM

#

yeah, the problem is not with creating the folder but changing its owner

left obsidian Oct 17, 2023, 3:25 AM

#

left obsidian Oct 17, 2023, 3:26 AM

#

marsh dew did you try to run it with a volume tho

no havent yet

marsh dew Oct 17, 2023, 3:26 AM

#

just try to run it without persistence first

#

without volumes I mean

left obsidian Oct 17, 2023, 3:27 AM

#

so so steps to do that would be to edit the yml as you mentioned before?

marsh dew Oct 17, 2023, 3:27 AM

#

just a database without any volumes for now

left obsidian Oct 17, 2023, 3:27 AM

#

sorry need more specifics

marsh dew Oct 17, 2023, 3:27 AM

#

paste your compose, the database part

left obsidian Oct 17, 2023, 3:28 AM

#

marsh dew Oct 17, 2023, 3:30 AM

#

It's harder to edit but alright,
comment out with #

  #- ./database:/var...

left obsidian Oct 17, 2023, 3:31 AM

#

marsh dew Oct 17, 2023, 3:31 AM

#

yea, lets go

left obsidian Oct 17, 2023, 3:32 AM

#

daniel asked me to remove the pgdata in the "volumes" at the very bottom of the file too if that is relevant

marsh dew Oct 17, 2023, 3:32 AM

#

not relevant

left obsidian Oct 17, 2023, 3:33 AM

#

ok

#

so whats next - compose down

marsh dew Oct 17, 2023, 3:33 AM

#

so you did have the same problem using the pgdata volume?

left obsidian Oct 17, 2023, 3:33 AM

#

and then compose up?

#

no i never tried an install with the standard yml in this build

#

daniel suggested i mod the file so that the setup will be portable as I have ordered a new box to move this install onto

#

so i wanted an easy to move instance of immich

marsh dew Oct 17, 2023, 3:34 AM

#

so much for portable then

#

it's still possible to easily move the database using volumes, I'm doing it that way

left obsidian Oct 17, 2023, 3:35 AM

#

ok

marsh dew Oct 17, 2023, 3:35 AM

#

left obsidian and then compose up?

yea, do the composeup

left obsidian Oct 17, 2023, 3:35 AM

#

so compose down and force a rebuild?

#

or just a usual compose up -d

marsh dew Oct 17, 2023, 3:37 AM

#

rebuild? we're not building anything
docker compose up -d --force-recreate if you want to be sure

left obsidian Oct 17, 2023, 3:37 AM

#

yeah i meant that

#

recreate not rebuild

#

i'ldd just do basic up -d first

#

ok worked it seems

#

FFS

#

#

so why was that

#

such an issue

#

#

thank you murek

dry geode Oct 17, 2023, 3:40 AM

#

Some strange permission issues. Maybe just use a database volume mount and call it a day for now lol

marsh dew Oct 17, 2023, 3:40 AM

#

we're not done yet, after restart you won't have your account and config

left obsidian Oct 17, 2023, 3:40 AM

#

does it mean the DB is on the pi SD card though?

marsh dew Oct 17, 2023, 3:41 AM

#

you have to attach the pgdata volume, as it was at beginning. It will be then on the SD card. Don't worry, it's not that big

#

the pg_dumpall will help you move data, when your hardware arrives, the procedure is there https://documentation.immich.app/docs/administration/backup-and-restore

Backup and Restore | Immich

Database

left obsidian Oct 17, 2023, 3:43 AM

#

marsh dew we're not done yet, after restart you won't have your account and config

I dont have one... I lost it all b'cuase Mac docker sucks

#

i am starting from zero again

marsh dew Oct 17, 2023, 3:44 AM

#

well yes, but I mean you want to persist your data from now on?

left obsidian Oct 17, 2023, 3:44 AM

#

mac docker operates in a VM and the VM image has all my data, but we couldn't mount the image to get the db files

dry geode Oct 17, 2023, 3:45 AM

#

How it is set up right now if you recreate the container you lose your database so you should add back the volume mount how you had it on your Mac with pgdata:/...

left obsidian Oct 17, 2023, 3:45 AM

#

@murek do you want to have a crack at that? (trying to mount the docker VM disk image?)

marsh dew Oct 17, 2023, 3:45 AM

#

EZ

#

just kidding, never done that

left obsidian Oct 17, 2023, 3:46 AM

#

oh lol you had me exicted for a min

marsh dew Oct 17, 2023, 3:47 AM

#

don't delete it yet, there is this volumes_from command that could come in handy

#

well I could have crack at that, but later. Imma head out it's 6AM here

left obsidian Oct 17, 2023, 3:48 AM

#

dry geode How it is set up right now if you recreate the container you lose your database ...

so you mean UNDO these mods...

marsh dew Oct 17, 2023, 3:49 AM

#

yea, you're switching from ./database to pgdata

dry geode Oct 17, 2023, 3:49 AM

#

Correct since we can't get past your permission issues

left obsidian Oct 17, 2023, 3:49 AM

#

ok so restore the yml to what it was with pgdata and then down and up

#

ok

dry geode Oct 17, 2023, 3:49 AM

#

Yup

left obsidian Oct 17, 2023, 3:51 AM

#

#

correct?

marsh dew Oct 17, 2023, 3:52 AM

#

yes

left obsidian Oct 17, 2023, 3:53 AM

#

boom

#

ok so i can work with this now and start importing all my assets again?

dry geode Oct 17, 2023, 3:53 AM

#

Yup

left obsidian Oct 17, 2023, 3:53 AM

#

and when i get new machine we'll be able to migrate the DB even though its on the pi SD card?

dry geode Oct 17, 2023, 3:54 AM

#

Maybe wait until the import is done before you start sorting

dry geode Oct 17, 2023, 3:54 AM

#

left obsidian and when i get new machine we'll be able to migrate the DB even though its on th...

Correct

left obsidian Oct 17, 2023, 3:54 AM

#

yeah i will turn off all ML stuff

marsh dew Oct 17, 2023, 3:55 AM

#

left obsidian and when i get new machine we'll be able to migrate the DB even though its on th...

yes, you can try exporting now, if you want to be sure

left obsidian Oct 17, 2023, 3:55 AM

#

wont do any "work" such as faces or albums

left obsidian Oct 17, 2023, 3:55 AM

#

marsh dew yes, you can try exporting now, if you want to be sure

well there aint anyting to export yet 🙂

marsh dew Oct 17, 2023, 3:55 AM

#

there are some tables for sure

left obsidian Oct 17, 2023, 3:56 AM

#

true

#

i trust you guys 🙂

#

i'll get my CLI upload running

#

and leave it for a day

#

whats the linu

#

linux command for disk space DF?

#

#

can either of you let me know what is the space left on the pi os sd?

#

40% of 32gb i guess

#

12.8 gb

#

11.5 gb

#

hmm

#

my last database was 6gb i think

#

ok thanks again guys

#

man what a polava

marsh dew Oct 17, 2023, 4:00 AM

#

du -hs .

marsh dew Oct 17, 2023, 4:01 AM

#

left obsidian my last database was 6gb i think

that's a lot of photos

left obsidian Oct 17, 2023, 4:01 AM

#

i had 160,000 in there

#

2tb assets

marsh dew Oct 17, 2023, 4:06 AM

#

DAMN

#

alright, manually create a directory "backups" and let's bind mount it. Let's try to do a backup

left obsidian Oct 17, 2023, 4:07 AM

#

yeah lost the DB - it was about 5 days of work

#

here is the thread if you are interested https://discord.com/channels/979116623879368755/1163189245138186340

left obsidian Oct 17, 2023, 4:09 AM

#

marsh dew alright, manually create a directory "backups" and let's bind mount it. Let's tr...

ok sure - where should i make the backup dir on the external ssd?

marsh dew Oct 17, 2023, 4:10 AM

#

next to your compose file

#

it's just for testing

left obsidian Oct 17, 2023, 4:10 AM

#

done

marsh dew Oct 17, 2023, 4:10 AM

#

an then add it to your db

dry geode Oct 17, 2023, 4:11 AM

#

It didn't need to be mounted

left obsidian Oct 17, 2023, 4:12 AM

#

#

ahh need colon

#

colon added

#

marsh dew Oct 17, 2023, 4:12 AM

#

dry geode It didn't need to be mounted

well, we're trying to see if we can access it from the host

left obsidian Oct 17, 2023, 4:12 AM

#

done

marsh dew Oct 17, 2023, 4:12 AM

#

yea, and just do docker compose up -d

dry geode Oct 17, 2023, 4:13 AM

#

But why did it need to be mounted?

left obsidian Oct 17, 2023, 4:13 AM

#

down first or just up

dry geode Oct 17, 2023, 4:13 AM

#

You can pipe pgdumpall to stdout

marsh dew Oct 17, 2023, 4:13 AM

#

we're trying do do a database dump, for backup purposes

#

oh yea

#

forgot about that

dry geode Oct 17, 2023, 4:13 AM

#

Yes. You don't need a coming mount for this

#

Or, just use the backup container

vapid dockBOT Oct 17, 2023, 4:14 AM

#

https://docs.immich.app/docs/administration/backup-and-restore

marsh dew Oct 17, 2023, 4:14 AM

#

then docker exec -t immich_postgres pg_dumpall -c -U postgres | gzip > "./backup.sql"

dry geode Oct 17, 2023, 4:15 AM

#

.SQL.gz really since it is gzipped

marsh dew Oct 17, 2023, 4:15 AM

#

right, maybe without gz for now, because we just want to see if it works

left obsidian Oct 17, 2023, 4:16 AM

#

marsh dew Oct 17, 2023, 4:16 AM

#

docker exec -t immich_postgres pg_dumpall -c -U postgres > "./backup.sql"

left obsidian Oct 17, 2023, 4:16 AM

#

backup.sql is there but not in the backusp folder

marsh dew Oct 17, 2023, 4:16 AM

#

forget the folder, it's not necessary

dry geode Oct 17, 2023, 4:16 AM

#

Cd the backup folder and run the command there instead

left obsidian Oct 17, 2023, 4:18 AM

#

worked

dry geode Oct 17, 2023, 4:19 AM

#

Actually heading to bed now lol

left obsidian Oct 17, 2023, 4:19 AM

#

dry geode Actually heading to bed now lol

many thanks for all your help 🙂

#

@marsh dew are you interested in seeing if we can get the old database back from the mac docker VM disk image?

#

or not

marsh dew Oct 17, 2023, 4:21 AM

#

yea, but later. I'm going to uni soon

left obsidian Oct 17, 2023, 4:22 AM

#

ok cool, be intersting to see

#

if we can...

#

thanks again ttyl hit me up when you want to look

left obsidian Oct 17, 2023, 10:41 AM

#

After all that I had to abandon the pi

#

the SSD kept giving i/o errors and dropping out

#

i tried three different usb cases for it

#

and also tried going via a powered USB 3.0 hub (thinking maybe pi was not giving it enought power)

#

But nope, couldn't keep it from giving errors. Drive is perfect plugged back into my PC. So..... I am back to MAC

#

@marsh dew @dry geode @mellow spindle I am getting great performance!!!!!!

#

FML

#

it should be blisteringly fast SSD to SSD

mellow spindle Oct 17, 2023, 11:02 AM

#

Oh no... I was seeing the notification

I am getting great performance!!!!!!
without the image and I was like "Oh yay, a ping from you and it isn't bad news!"

#

Well, turns out it is bad news

#

:(

#

May I ask what SSD you've got?

left obsidian Oct 17, 2023, 11:05 AM

#

SAMSUNG QVO 870 4TB

#

oh man @mellow spindle that reccomendation you made to put - ./database: in the yml to try to get it to set up the db on the drive was a nightmare

mellow spindle Oct 17, 2023, 11:06 AM

#

I've read about this, yes

left obsidian Oct 17, 2023, 11:07 AM

#

so that took half the day then the pi giving issues took another few hours

mellow spindle Oct 17, 2023, 11:07 AM

#

Although the issue is very likely with file stuff and not the actual change

left obsidian Oct 17, 2023, 11:07 AM

#

and now i have gone back to mac

#

and along with slow speeds, i get these

mellow spindle Oct 17, 2023, 11:08 AM

#

left obsidian SAMSUNG QVO 870 4TB

I mean that ssd is pretty shitty and definitely not "blisteringly fast", but it should perform better than 7MB lol

left obsidian Oct 17, 2023, 11:08 AM

#

i was getting these all the time which is why it took 7 days or so to get all my data in there

mellow spindle Oct 17, 2023, 11:08 AM

#

left obsidian and along with slow speeds, i get these

Huh? Could you scroll down a little?

left obsidian Oct 17, 2023, 11:08 AM

#

uploads just suddenly fall over and print 100s of lines. I showed it to @solar marsh when I first saw that behaviour he was aware of it

mellow spindle Oct 17, 2023, 11:09 AM

#

Keep in mind that just pinging everyone all the time can be pretty annoying...

#

Especially since Alex doesn't need to look at this

left obsidian Oct 17, 2023, 11:12 AM

#

i just mentioned him b'cause he's aware of it and might want to add something

#

but yes noted thanks

mellow spindle Oct 17, 2023, 11:13 AM

#

I think if he mentioned that's fine, we simply don't need to care about it :)

left obsidian Oct 17, 2023, 11:13 AM

#

so thats an issue with the upload script not failing gracefully

mellow spindle Oct 17, 2023, 11:16 AM

#

So it didn't complete, correct?

#

Only ~80% got transfered and then it died

left obsidian Oct 17, 2023, 11:16 AM

#

nope, so what i do is run the exact same import again, and it will get further and die again at some point

#

this is why it took me soooo long to get my 190,000 assets in

#

and why i was heartbroken to lose the DB

mellow spindle Oct 17, 2023, 11:17 AM

#

I think the cli is about to be refactored pretty soon...

left obsidian Oct 17, 2023, 11:17 AM

#

i think i should wait until the upload script is fixed maybe, is that being worked on?

mellow spindle Oct 17, 2023, 11:17 AM

#

Is it possible that it isn't handling duplicates very well?

#

https://github.com/immich-app/immich/pull/4434

left obsidian Oct 17, 2023, 11:18 AM

#

well that folder i was importing from was an OUTPUT asset folder from my old installation - so it should be perfefct with no dupes or weirdness

mellow spindle Oct 17, 2023, 11:19 AM

#

Yeah ok, that's fine

left obsidian Oct 17, 2023, 11:19 AM

#

i just took the year 2020 as a test

mellow spindle Oct 17, 2023, 11:19 AM

#

Any server logs while uploading?

left obsidian Oct 17, 2023, 11:20 AM

#

whats the log command again? the docker gui is dead

#

docker logs immich_server ?

mellow spindle Oct 17, 2023, 11:20 AM

#

Yep

left obsidian Oct 17, 2023, 11:22 AM

#

no output from that command

mellow spindle Oct 17, 2023, 11:22 AM

#

docker ps

left obsidian Oct 17, 2023, 11:23 AM

#

just a cursor

mellow spindle Oct 17, 2023, 11:23 AM

#

Oh, I think that's the reason why the upload is failing...

#

Can you reach Immich?

left obsidian Oct 17, 2023, 11:24 AM

#

nope she's dead

#

ok makes sense

#

docker crashes and then the upload routine has noone to talk to

mellow spindle Oct 17, 2023, 11:25 AM

#

Exactly

left obsidian Oct 17, 2023, 11:25 AM

#

sigh

#

ok giving up on mac docker a 2nd time

#

what a p.o.s

mellow spindle Oct 17, 2023, 11:25 AM

#

So that the only "issue" with the cli is that it isn't handling a connection loss very gracefully

#

Yeah... Sorry that you have such a bad experience with docker and therefore with Immich as well... :(

left obsidian Oct 17, 2023, 11:26 AM

#

mellow spindle So that the only "issue" with the cli is that it isn't handling a connection los...

yes

#

be nice for it to gracefully say - lost connection to docker.. aborting

mellow spindle Oct 17, 2023, 11:27 AM

#

Yeah...

#

That's something the new cli will probably do

left obsidian Oct 17, 2023, 11:27 AM

#

ok just for shits and giggles I am going to try docker on windows

#

becuase when you have lost 5 days of your life on soemthing now is not the time to give up!

#

at least i can now get containers running in no time at all 🙂

mellow spindle Oct 17, 2023, 11:30 AM

#

I honestly don't think you want to do this...

#

a) does docker on windows also just spin up a VM, b) is windows really not meant to be running docker

#

Actually, your best bet would probably be to install docker on wsl lmao

left obsidian Oct 17, 2023, 11:32 AM

#

actually wsl is just installing now as part of the docker install

mellow spindle Oct 17, 2023, 11:32 AM

#

Oh ok

left obsidian Oct 17, 2023, 11:32 AM

#

so it must use it

mellow spindle Oct 17, 2023, 11:32 AM

#

Apparently, yes

#

Tbh I still don't get why you had that many issues with your drives on debian

left obsidian Oct 17, 2023, 11:33 AM

#

i am not worried about it being in a VM now that I exectued a practive backup with the help of murek

left obsidian Oct 17, 2023, 11:33 AM

#

mellow spindle Tbh I still don't get why you had that many issues with your drives on debian

was very strange

left obsidian Oct 17, 2023, 11:54 AM

#

ok immich up and running already on windows docker!

#

let's see how this goes

left obsidian Oct 17, 2023, 12:26 PM

#

happy to report it is going sooo much faster on windows docker, and the machine is only an surface 4 i5 7th gen with only 8gb ram

#

wow

mellow spindle Oct 17, 2023, 12:31 PM

#

Funny ok

#

Are you uploading to localhost now?

#

So same machine running immich and running the cli

left obsidian Oct 17, 2023, 12:32 PM

#

yes was also uploading to local host on the mac too

#

same scenario

#

just mac docker issues I am sure - the mac m1 is a way faster machine

mellow spindle Oct 17, 2023, 12:32 PM

#

Ah ok, so you had two drives connected to the resepctive computer. The ssd for immich and another drive

left obsidian Oct 17, 2023, 12:33 PM

#

ya

mellow spindle Oct 17, 2023, 12:33 PM

#

And the other drive is also a ssd I'm assuming?

left obsidian Oct 17, 2023, 12:34 PM

#

its the 10TB usb 3.0 5400 rpm

#

but that is capable of 160MB/S

#

the ssd 500ish write

mellow spindle Oct 17, 2023, 12:35 PM

#

Got it ok

left obsidian Oct 17, 2023, 12:35 PM

#

on mac it was doing 7 and on windows just now doing 60 sustained

mellow spindle Oct 17, 2023, 12:35 PM

#

I mean 60MB/s is still very bad, but that is probably the hard drive

#

I don't think you can get 160MB/s out of that...

left obsidian Oct 17, 2023, 12:36 PM

#

yeah not on a no internal fan ancient surface tablet

#

lol

mellow spindle Oct 17, 2023, 12:37 PM

#

Oh wtf

left obsidian Oct 17, 2023, 12:37 PM

#

i see what you mean about powershell its great at pretenting to be linux on the whole, until you try run a nodejs command

#

then it just says nup

#

but hysterically, old windows cmd runs the node stuff fine

mellow spindle Oct 17, 2023, 12:38 PM

#

I mean that was probably just some " missing or something

left obsidian Oct 17, 2023, 12:38 PM

#

mellow spindle Oh wtf

yes its very hot right now so will be throttling

mellow spindle Oct 17, 2023, 12:38 PM

#

It's just that I'm not used to powershell pretty much

mellow spindle Oct 17, 2023, 12:38 PM

#

left obsidian yes its very hot right now so will be throttling

It'll die lmao

left obsidian Oct 17, 2023, 12:41 PM

#

powershell doesn't know how to deal with the command "immich"

#

so it doenst know where the script is maybe\

#

whereas in cmd.exe i can just type immich upload yada yada and it goes

#

so not sure on that

mellow spindle Oct 17, 2023, 12:42 PM

#

Hm this sounds weird

left obsidian Oct 17, 2023, 12:42 PM

#

will need to work out where the immich script is? and maybe call it with a path?

mellow spindle Oct 17, 2023, 12:42 PM

#

I mean the PATH env should be the same, so idk

#

But as long as it works with cmd whatever, right?

left obsidian Oct 17, 2023, 12:51 PM

#

ok I solved it

#

it was windows execution policies banning scripts

#

there is a command to run in powershell to ovveride it and allow scripts to be exectued by the current user 🙂

#

working now

left obsidian Oct 17, 2023, 1:07 PM

#

edge case obviously on windows powershell, but can probably be fixed, how to raise that as a ticket?

mellow spindle Oct 17, 2023, 1:12 PM

#

a) I don't understand why you're so worried that it will "fill the buffer up" and b) I would wait for the new CLI before reporting any bugs

#

Although it seems like this could even be just powershell being stupid, since it works in all other shells. So might even be we can't fix it at all

solar marsh Oct 17, 2023, 1:16 PM

#

left obsidian edge case obviously on windows powershell, but can probably be fixed, how to rai...

issue with powershell I believe, you can ignore it

left obsidian Oct 17, 2023, 1:17 PM

#

mellow spindle Although it seems like this could even be just powershell being stupid, since it...

yes likely

#

Dan comments re buffer are becuase there must be a line maximium before it runs out of ram or somewho cuases other failure

#

and i am doing 10 or 20 thousand pics in one upload so thats why i mention it

#

noted Alex its windows - edge case... but so far its performaing way better than mac docker so that is interesting. the machine is a solid state i5 tablet from 2017 too

mellow spindle Oct 17, 2023, 1:19 PM

#

left obsidian Dan comments re buffer are becuase there must be a line maximium before it runs ...

It doens't need to store all lines, no?

#

It can just drop old ones

left obsidian Oct 17, 2023, 1:19 PM

#

well it is cuasae the scroll bar shrinks more and more so i dont think it disposes of any

mellow spindle Oct 17, 2023, 1:20 PM

#

At some point it certainly will

#

If the powershell can run out of memory by having too many lines that would be ridiculous

left obsidian Oct 17, 2023, 1:29 PM

#

man no wonder the pi struggled. this thing is sitting flatlining at 100% cpu and thats with the ML engine turned off!

#

how insane google is doing this for likley 20% of the entire world!

dry geode Oct 17, 2023, 1:30 PM

#

You should probably decrease the job concurrencies

left obsidian Oct 17, 2023, 1:31 PM

#

yeah ok will put them all on 1

#

but now I know the windows setup was so easy and my son is away for a week - I might commandeer his 12th gen i9 with 32gb ram 😄

#

mwahahaha

mellow spindle Oct 17, 2023, 1:32 PM

#

lol

left obsidian Oct 17, 2023, 1:32 PM

#

that should last until my mini pc arrives (which i checked out performs the i5 i am using now)

left obsidian Oct 17, 2023, 1:33 PM

#

mellow spindle lol

is that a yes!

#

omg i think it has to be done

#

will be the beefiest immich server ever

mellow spindle Oct 17, 2023, 1:33 PM

#

left obsidian is that a yes!

I don't think I can say yes or no to this :P

#

I would just ask him tbh

left obsidian Oct 17, 2023, 1:34 PM

#

shit i'll whip out his HDD and install linux on it!!

mellow spindle Oct 17, 2023, 1:34 PM

#

left obsidian will be the beefiest immich server ever

Actually, I wouldn't say this with that much confidence. People have crazy homelabs!

left obsidian Oct 17, 2023, 1:34 PM

#

he'll never know

mellow spindle Oct 17, 2023, 1:34 PM

#

left obsidian shit i'll whip out his HDD and install linux on it!!

lmao

left obsidian Oct 17, 2023, 1:34 PM

#

i'll be able to get all my injestion done in a day!

mellow spindle Oct 17, 2023, 1:34 PM

#

Not with that poor hdd lol

#

But it would be much faster, yes

left obsidian Oct 17, 2023, 1:35 PM

#

alot of it is on a nas with raid 10

#

Solved my thermal throttling!!

#

lol

mellow spindle Oct 17, 2023, 1:37 PM

#

OMG

left obsidian Oct 17, 2023, 1:37 PM

#

that air purifier is pumping large volume of air haha its cool to the touch now whereas before it felt it was about to melt

#

CPU and memory both maxed

#

haha the poor little thing its been in a drawer for the past 5 years

#

back to using my sons beast - surely i can just set up a linux o/s on a usb drive and boot from that, therefore not needing to work out which of the 3 x m2 drives on his mainboard is the o/s drive.

#

prefer to not pull them out

dry geode Oct 17, 2023, 1:44 PM

#

Just install and run the docker machine learning container on it

left obsidian Oct 17, 2023, 1:46 PM

#

ooo thats an idea too

#

i'll look into that

#New install: Postgres doesn't appear to be running