[SOLVED] NVME migration

[ocean zephyr]

Info about old system:
Pure arch
GRUB as bootloader
root on /dev/nvme0n1p5
boot on /dev/nvme0n1p6
root filesystem of type ext4
boot of type fat32

What i wish to achieve:
migrate installation to /dev/nvme1n1 while converting the partitions to btrfs with a seperate home

please note i have never touched btrfs

[wet breach]

Create the normal partitions on the 1n1 but instead of ext4 format the rootfs as btrfs

Then create subvols according to the archinstall layout because it's good for snapshotting since it separates the pkg cache

https://github.com/archlinux/archinstall/blob/1faac77c0d476165a9dee1616274c1a2749c49db/docs/cli_parameters/config/disk_config.rst#btrfs

[ocean zephyr]

hm.. how does btrfs work? is it a single partition setup? like, do you just format the root (hence its called "subvolumes"?)

[wet breach]

Have all of them mounted right and them you can rsync the old install to the new

https://wiki.archlinux.org/title/Rsync#Full_system_backup
rsync -aAXHv

[wet breach]

Yeah

[ocean zephyr]

i see. thanks for some useful stuff as this is my first time touching btrfs

[open venture]

hm7 received a thank you cookie!

[wet breach]

https://btrfs.readthedocs.io/en/latest/Subvolumes.html

With the mount command you use -o subvol=subvolname
Name could be for example @ or @home

If you mount a btrfs without specifying the subvol you mount the top-level subvol which depending on your layout isn't what you want (when using @ subvol as the rootfs)

[ocean zephyr]

i see. think its easier to just manually regenerate /etc/fstab, or should i just use the genfstab in the archiso once done to automatically deal with the fstab?

[wet breach]

You can mount the subvols and then generate the fstab, it's not hard to mount them right

[ocean zephyr]

i mean, i wouldnt assume it is once ive read more about subvols via the arch wiki

[wet breach]

"btrfs": [
       {
           "mountpoint": "/",
           "name": "@",
       },
       {
           "mountpoint": "/home",
           "name": "@home",
       },
       {
           "mountpoint": "/var/log",
           "name": "@log",
       },
       {
           "mountpoint": "/var/cache/pacman/pkg",
           "name": "@pkg",
       }
   ],

The format is like that because it's from the archinstall config, btrfs itself doesn't use any config like that

[ocean zephyr]

i assume this still requires a dedicated boot partition, right?

or should it also be a btrfs subvol?

[ocean zephyr]

according to the wiki, one uses btrfs subvolume create /path/to/subvolume to make a subvolume. where does the "name" part of the json you posted come in? and is path/to/subvolume essentially the "mountpoint"?

[wet breach]

Grub can boot btrfs

[wet breach]

Mount the top-level subvol and then create the subvols in it

Then unmount and mount the rootfs subvol and create the dirs for the other mountpoints

[ocean zephyr]

could you guide me, as this felt a bit confusing? what ive done so far:

used cfdisk to make a partition on the disk.
ran mkfs.btrfs -f -n 32k -L root /dev/nvme1n1p1
ive also mounted that partition to /mnt.

what i didnt quite get is the fact that i first mount it, make subvolumes and then unmount it before mounting the rootfs. does that mean i should make a subvol for root using btrfs subvolume create /mnt (as thats where its mounted)?

[wet breach]

Create /mnt/@, /mnt/@home and so on

[ocean zephyr]

how would i make the @log one seeing it is nested inside /var?

[wet breach]

The -n isn't needed btw

[wet breach]

/mnt/@log

It's not the same as the mountpoint

This layout is called the flat layout btw

[ocean zephyr]

i see. Arch wiki mentioned the flag and said a higher one may yield less fragmentation with the downside on slight memory increase on operations

seems fine?

if youre wondering where ID 259 is, i made the log, thought it was incorrect, deleted it, asked you and then recreated it again

[wet breach]

Good

[ocean zephyr]

is this where i unmount it and mount the rootfs?

[wet breach]

Don't create a separate @boot because it makes it more complicated to use snapshotting because you won't be able to just rollback the rootfs and have the kernel rolled back too

[wet breach]

Yeah

[ocean zephyr]

and then use mount -o subvol=/@ /dev/nvme1n1p1 /mnt, and repeat that for each subvolume similar to mounting normal partitions? just that i change subvol and mount location (as opposed to the device and mount location)?

[wet breach]

subvol=@ i don't recall if /@ works

Yeah

[ocean zephyr]

oh. wiki says mount -o subvol=/subvol_root device /mnt/ (though they used subvol_root as label)

thats why i assumed the trailing slash

[wet breach]

*Leading slash

You can try

No harm

[ocean zephyr]

it seems to have accepted it

maybe..?

shows up the same in lsblk both with and without

i went without the leading / for safety

is it just rsync + fixing /etc/fstab and fixing grub left?

[wet breach]

What esp are you going to use

You need a esp but not a separate boot partition

[ocean zephyr]

huh? what do you mean by an ESP? just.. /boot as the relative location or..?

[wet breach]

Efi system partition

The fat32 partition where the efi file is installed by grub-install or similar

Grub can read btrfs but the uefi can't directly

Nor can uefi read ext4 or fs other than fat

Usually it's mounted at /boot/efi or /efi

[ocean zephyr]

well that info likely makes it so i have to redo this so far. now, in a hypothetical scenario, say that nvme1n1p1 was the efi partition formatted as fat32, while nvme1n1p2 was the btrfs one with all subvols. would i just mount it into /boot of the dystem as normal?

[wet breach]

Some mount it at /boot but that's usually only required when using systemd-boot

[ocean zephyr]

wait really? not /boot?

[wet breach]

Some mount it there for various reasons

I think for btrfs snapshot use it would be better to keep that on the rootfs so it's rolled back when a snapshot of the rootfs is booted

[ocean zephyr]

kind of makes sense. so i should have an efi partition mounted to either /boot/efi or /efi as opposed to /boot for rollback reasons. just need to debate which. is there a difference otherwise to setting up the rest of the system?

[wet breach]

No diff

[ocean zephyr]

thanks! ill redo most of the steps with an EFI partition this time around, run Rsync and fix fstab. though rsync will likely shove the old grub files into into /boot. i assume ill just remove whatever files grub generates in /boot/efi from /boot then

[open venture]

hm7 received a thank you cookie!

[ocean zephyr]

i very much appreciate the help

[wet breach]

The grub files will still go to /boot, only the efi location changes

[ocean zephyr]

right. but when using grub-mkconfig i should aim at /boot/EFI?

[wet breach]

Don't wipe the other disk right away before booting and confirming everything you want transferred gas transferred

[wet breach]

No

[ocean zephyr]

understandable as its a functioning system still

[wet breach]

In grub-install you only change the --efi-directory, nothing else changes

The --boot-directory has the default of /boot which remains unchanged

[ocean zephyr]

ah. my bad. wrong command. i did mean the one with the --efi-directory

also. i remade partitions + btrfs subvolumes. i noticed that Arch wiki mentioned that the rsync command uses that current root, so i then unmounted the subvols, mounted the original system and then chrootrd before mounting the subvolumes in the chroot instead. ill just have to remember to exclude it from the clone

[wet breach]

You can just change the command to specify the source

[ocean zephyr]

well, i was silly and didnt know. this is close to the equivilant. besides. i had to remember to install btrfs-progs, so i did that as well while i was at it

welp. time to prepare the long rsync command

[wet breach]

rsync -aAXHv --exclude='/lost+found/' /oldroot /mnt

Actually I dunno if exclude is relative to the source

[ocean zephyr]

no clue. id say chroot is easier

arch wiki also says to exlcude a bunch of /proc + /dev and other things, so im mostly using that. makes sense as devices and such are discovered dynamically rather than being static

[wet breach]

It's a pattern so '*/lost+found' would match it

[wet breach]

Yeah that needs to be done when in the system

[ocean zephyr]

i didnt ignore /mnt but thats because i didnt mount the new filesystem there. i have a symlink there i wish to retain without having to remake it. i did remember to exclude the new root though

[ocean zephyr]

right.. so i used grub-install. so i now have an EFI folder inside my /boot/EFI. do i just delete the old GRUB and BOOT folder inside /boot/EFI from previous grub?

note that i chrooted into the new system

[ocean zephyr]

wait. was i stupid and misunderstood? should i have mounted the efi partition as /boot during the clone?

hmmm..

[wet breach]

No

[ocean zephyr]

so ive done it right?

[wet breach]

Yeah

[ocean zephyr]

what about this? ^

[wet breach]

You can delete any grub files and they'll be created in the right places by grub-install and grub-mkconfig cmd from the wiki

Really you could remove all files from /boot and recover with -S linux X-ucode and the grub commands

[ocean zephyr]

i just removed the GRUB dir + anything in /boot/EFI and reran the grub commands. i think thats it. also remade the /etc/fstab with genfstab. anything else, or do you think im ready to reboot? because i certainly feel ready

[wet breach]

Yup

[ocean zephyr]

hmmm.. something is up with my boot entries

color is wrong, but thats fine as i previously used a theme. likely destroyed some theme files from grub which ill deal with later. but i have duplicate entries

that is however unexpected

interesting..

[wet breach]

Os prober added the previous install

[ocean zephyr]

first entry (despite UEFI saying it points to the 2TB SSD) booted my previous install (the old partitions were mounted as / and /boot also get this weird error:

get the same error on the second entry, but also get this (unlike the 1st one where it boots)

[ocean zephyr]

thats weird.. i removed os-prober and commented the line again in the install before rerunning grub-mkconfig

[wet breach]

/boot as btrfs doesn't allow savedefault in grub

You need to edit the grub config to disable that feature

[ocean zephyr]

/etc/default/grub?

[wet breach]

Yeah

https://wiki.archlinux.org/title/GRUB/Tips_and_tricks#Recall_previous_entry

[ocean zephyr]

uhh.. im not sure if GRUB_CMDLINE_LINUX="rootfstype=ext4" is doing me much of a favour either

[wet breach]

That isn't even needed on ext4 lol

[ocean zephyr]

ill just comment it

second entry now works. but the old one still exists after a reboot. thoughts on that or..?

seems grub-customizer wishes to remove the entry. so.. i guess thats that

well done on stupidity Jorgen

it removed the wrong Arch entry

1 moment while i once again fix GRUB

screw it. i think ill just do what you mentioned and delete the entirety of /boot and reinstall linux amd-ucode and fix grub after

nice joke there. see the grub config? exactly why did it decide to keep the fallback of the new install but use the original install for the normal entry

right. so removed everything in /boot, reinstalled linux amd-ucode and remade the grub config. het it still only shows the old entry. im guessing this is thanks to grub-customizer

its likely /etc/grub.d thoughts on what to do?

fixed it. removed /etc/grub.d, reinstalled grub via pacman and remade the config

thanks @wet breach . everything seems resolved now. ill run this new install for a bit until i feel confident everything works. after that i may then repurpose the old NVME. i owe you big time

[open venture]

hm7 received a thank you cookie!

[ocean zephyr]

[SOLVED] NVME migration