Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Image | Raspberry Pi 5: Testing and firmware migration script #6676

Open
TDuffinNTU opened this issue Oct 9, 2023 · 375 comments · Fixed by #6795 or #6892
Open

Image | Raspberry Pi 5: Testing and firmware migration script #6676

TDuffinNTU opened this issue Oct 9, 2023 · 375 comments · Fixed by #6795 or #6892

Comments

@TDuffinNTU
Copy link
Contributor

TDuffinNTU commented Oct 9, 2023

ADMIN EDIT

First Raspberry Pi 5 testing images are available now on our download page: https://dietpi.com/#download
You can also find images with the new kernel/firmware package set for other RPi models here: https://dietpi.com/downloads/images/testing/
The ones with the new firmware have "RPi1", "RPi2" and "RPi234" (64-bit) in their names. To migrate an existing system, use the migration script:

bash <(curl -sSf 'https://raw.githubusercontent.com/MichaIng/DietPi/dev/.meta/dietpi-rpi-firmware-migration')

WARNING: This is currently a one-way ticket. dietpi-backup cannot be used to restore the old system, since the boot partition mount point has changed. If you want to be able to revert, create an image of the whole SD card/drive. dietpi-imager can be used from another DietPi (or Debian/Ubuntu) system, to minimise partition and filesystem automatically, to have a small compressed backup image.


Is the SBC officially supported by the Debian installer?

  • Yes(?) Looks like the board is inter-compatible with the same distro images as the 4B. That's pure speculation, mind you, but I did see a video of a reviewer transferring a Raspbian installation from a Pi4B to a Pi5 with no configuration necessary.

Notes

  • Not sure if this needs any specific developer legwork as the previous Pis seem to be inter-compatible with OS images for the most part (though I'm sure testing and drivers will be necessary!). Thought I'd make this issue now since it might be good to mark the Pi5 as provisionally supported on the website if it at least works out of the box.
@MichaIng MichaIng changed the title Raspberry Pi 5 Image | Raspberry Pi 5 Oct 10, 2023
@MichaIng
Copy link
Owner

MichaIng commented Oct 19, 2023

Okay this requires more work, sadly. With Bookworm and hence for RPi 5, the packages as well as the filesystem layout have changed dramatically:

  • For RPi 4 and earlier, the package is called linux-image-rpi-v8, which is a meta package and current pulls linux-image-6.1.0-rpi4-rpi-v8.
  • For RPi 5, the package is called linux-image-rpi-2712, which is a meta package and current pulls linux-image-6.1.0-rpi4-rpi-2712.
  • These packages are more streamlined with mainline Linux packages, installing kernel image, config, map, device trees and overlays like common Debian Linux packages:
    /boot/System.map-6.1.0-rpi4-rpi-2712
    /boot/config-6.1.0-rpi4-rpi-2712
    /boot/vmlinuz-6.1.0-rpi4-rpi-2712
    ...
    /usr/lib/linux-image-6.1.0-rpi4-rpi-2712/broadcom/bcm2712-rpi-5-b.dtb
    ...
    /usr/lib/linux-image-6.1.0-rpi4-rpi-2712/overlays/README
    
  • For the bootloader, there is now a raspi-firmware package, which installs the files as well more streamlined into /usr/lib/raspi-firmware/.
  • libraspberrypi0, libraspberrypi-bin and raspberrypi-sys-mods remain the same.
  • The RPi ROM bootloader stage still requires bootloader, config and kernel on a partition 1 with FAT filesystem, which was mounted to /boot. This cause the issue that, since dpkg does not support FAT, this rpikernelhack back and forth copying of all files in /boot was required, to bypass the need to dpkg to replace a file (which would fail). This procedure has now been replaced with another, IMO not less ugly workaround: The FAT filesystem of partition 1 is now mounted to /boot/firmware. The raspi-firmware package strictly requires this directory to be a mount point (now allowing a dir within a mount point) and then copies all kernel, dtb and bootloader files from /boot and /usr/lib/linux-image- over to /boot/firmware, applying the known naming schemes. So a copy back and forth of all those files has been replaced with a single copy, but files remaining permanently on both partitions. And of course, for the 64-bit kernel, it is now actually two kernel images and all related modules, hence disk space requirements have increased.
  • Furthermore, the raspi-firmware package does now strictly require an initramfs. I have to check whether this means that the kernel has essential modules no builtin anymore, or whether it is actually not required, but just installed and handled for in case. As far as I can see, there is a config flag to disable this copy to /boot/firmware stuff, which then would need to be done manually. So we could write our own script to do this, skipping the initramfs. But at least at the beginning, I am not keen to maintain such, as it will likely change a lot within the next months. Probably a tiny-initramfs is possible, like the moreless dummy one we do now use for Sparky SBC, one which does not include any kernel module, hence servers nearly no purpose (aside of supporting UUIDs on top of PARTUUIDs) aside of satisfying bootloader and/or package script expectations.
  • Same for the 32-bit image, with the same kernel packages names and suffixes: v6, v7, v7l and v8. But one quirk is that the linux-image-rpi-v8 packages are not contained within the armhf package list, only in the arm64 ones, which is not enabled by default on 32-bit userland.

@MichaIng MichaIng added this to the v8.24 milestone Oct 19, 2023
@Joulinar
Copy link
Collaborator

uhh does not sounds like a quick win. 🙄

@onurcoskun14
Copy link

Any updates?

@axxis-creator
Copy link

Same Question. :-) I buy a RPI-5 and want to install DietPi for LoxBerry.
When can we expect a version?

@gaurishhs
Copy link

any updates on this?

@MichaIng
Copy link
Owner

We will post any updates here when we have some. I was looking further into it. The migration of the partition and installation of the new kernel packages is not too hard. But there is another big problem: Our scripts, and probably RPi's own scripts/programs expect config.txt and cmdline.txt in /boot, while for booting, both must be located in /boot/firmware (with the new package set). This is solved in RPi OS via symlinks. The problem is now that sed -i naturally replaces symlinks with actual files, and we use it a lot to edit their contents. Hence we must replace every occurrence across our whole code with sed -i --follow-symlinks to avoid this. Not understandable why this is not the default since ages, since I cannot imagine any circumstance where one would want to have a symlink replaced with its target file, when using sed on a symlink. However, I can do this quickly with a single command for the whole repo with the use of ... sed 😃. But this needs to be carefully reviewed, since there are variances like sed -Ei and probably some cases or code comments where the replacement must not be applied.

Another problem or better question is what we do with /boot/dietpi and /boot/dietpi.txt. The first, I think is fine to stay on the ext4 partition in the first place, resp. being moved there as part of the migration. The dietpi.txt is however nice to be on the FAT partition, at least before first boot, so one can apply automation and pre-configure the image from Windows and macOS easily. We just added a trailing FAT partition to all other images to make it as easy there as on RPi, so it would be horrible if we now removed this possibility for RPi. Theoretically we could do it on RPi the same way we do on other SBCs: As part of the first boot rootfs expansion script, mount the FAT partition and copy over dietpi.txt, dietpi-wifi.txt and all other optional pre-config files to the rootfs, then unmount it again. This must of course only happen on those RPi images which have the FAT partition mounted to /boot/firmware already.

Btw, does someone of you guys have an RPi 5 already and can tell me the revision code? That way we can add support to our hardware detection script already and provide RPi 5 compatible images between releases, if I am not able to finish this until this Saturday (v8.24 release):

mawk '/^Revision/{print $3;exit}' /proc/cpuinfo

The 3rd and 2nd last characters define the model. "14" was CM4, so it is probably "15" or "16".

@jboots07
Copy link

C04170

@adrianog91
Copy link

adrianog91 commented Nov 16, 2023 via email

@MichaIng
Copy link
Owner

Thanks. Found it now here as well: https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#new-style-revision-codes

So "15" is the CM4S, which is a special industry customer variant, not publicly sold, and "16" for "internal use". Let's see whether "16" is a surprise, not announced yet 🙂.

@MichaIng
Copy link
Owner

First step: 6e848a1

@jboots07
Copy link

Thanks for your hard work and update! Let me know if I can help in any way, I have a pi 5.

@sorriso93
Copy link

I just received my PI5 and can't wait to install on it Dietpi to replace my PI4 :-)

@BugDave
Copy link

BugDave commented Nov 19, 2023

I just received mine too and am excited to put dietpi on it :-)

@MichaIng MichaIng modified the milestones: v8.24, v8.25 Nov 19, 2023
@m0jek
Copy link

m0jek commented Nov 21, 2023

@MichaIng when you are ready for testing in the wild on RaspberryPi 5, I am happy to test.

@FabioEight
Copy link

Available for testing on Pi 5 as well! Thanks for your efforts!

@elisenlebkuch
Copy link

Where can I get it?

@gaurishhs
Copy link

gaurishhs commented Nov 22, 2023

I don't think so there has been any image supporting Pi 5, You'll have to wait

@elisenlebkuch
Copy link

elisenlebkuch commented Nov 22, 2023

@gaurishhs
Copy link

I think that one is for Risc-V #6212

@LittleFreak
Copy link

LittleFreak commented Nov 26, 2023

Is it possible to add the dietpi later on a running raspbian?
I'll plan to docker all my programs anyway, so the host will stay pretty much raspbian lite until then.

If so, I'd just give my raspbian a diet later, when its compatible with raspi 5.

@MichaIng
Copy link
Owner

Is it possible to add the dietpi later on a running raspbian?

You mean on a Raspberry Pi OS? Currently does not work as dietpi-installer installs the old kernel/bootloader/firmware packages and our scripts use sed in various places a way that the /boot/config.txt => /boot/firmware/config.txt (same with cmdline.txt) symlinks are replaced with the actual files, so that the intended changes will not actually apply. See my longer comment above about this issue. Once we added compatibility, dietpi-installer can run on a new RPi OS image, but it will remove all installed software, including Docker in your case. What you could do is backup your /var/lib/docker. Then once dietpi-installer ran through and you boot DietPi, do not install Docker at first, but restore /var/lib/docker to /mnt/dietpi_userdata/docker-data, then install Docker via dietpi-software.

@cgmcfall
Copy link

Have a Pi5, happy to test :)

@alfredoanton82
Copy link

Also have a RPI5, happy to help testing.

@sergio-ingrao
Copy link

Here, I have another one ready to test.

@MichaIng
Copy link
Owner

MichaIng commented Sep 3, 2024

Done: RPi-Distro/repo#376

@MichaIng
Copy link
Owner

MichaIng commented Sep 12, 2024

The migration script should now work again.
EDIT: Yes, it works fine now 🙂.

@user2745
Copy link

Hey y'all, I hope this isn't an off topic question:

Is the img (available on the download page) ready for use in prod?

Thanks

@abusr
Copy link

abusr commented Sep 24, 2024

Is it not possible to have a single image for both raspi4 and raspi5, which will be containing both sbcs' dtbs (or whatever else detail) and their kernels together? So that at boot the user can select/set_default which path to boot, the 4 path or the 5 path. I might be asking something impossible because I don't know their boot chain at all but if it is possible, then ynot?

Because I am considering mixed environments with 4s or 5s and because this 'migration script' appears a one way solution, correct? Once one 'migrates' to 5, (s)he boot this installation in 4.

Since both 4 and 5 are well alive, flexibility could be appreciated here.

@Joulinar
Copy link
Collaborator

We have introduced an option especially for the RPi5 that allows you to select different kernel modules to ensure compatibility of the running system between RPi5 and other RPi SBCs. This should make it possible to use the SD card from an RPi5 in a RPi4 as well.

@MichaIng
Copy link
Owner

MichaIng commented Oct 7, 2024

Additionally, the "RPi234" image from here boots on RPi 5 as well. It contains the needed device tree, the kernel is just 4k page size instead of 16k, which is maybe a little performance penalty, but on the other hand is able to run 32-bit binaries (which the 16k page size kernel cannot).

@mdr92
Copy link

mdr92 commented Jan 10, 2025

I've just got a Pi 5, the image works fine, I setup a new card although somewhere I have my old Pi 4 I ran the migration script on months ago before I sold it. Anyway, it all seems to work fine but the power button doesn't shutdown, is it disabled to stop accidental shutdowns or am I missing something, holding down for 5 seconds does a force powerdown but I'd like it to do a clean shutdown too if possible. Thanks

@MichaIng
Copy link
Owner

@mdr92
I guess is uses ACPI, hence try this:

systemctl unmask systemd-logind
apt install dbus
systemctl start systemd-logind

Among others, systemd-logind provides ACPI functionality. An alternative would be apt install acpid.

@frank-f
Copy link

frank-f commented Jan 11, 2025

I see the tag "testers required" so I thought I'd just leave a quick feedback. I got here looking to find out why I have a much newer kernel on my PiOS systems than on the DietPi ones (the DietPi kernel wasn't recent enough to use NUMA).

I ran the script, which worked absolutely flawlessly, rebooted the system and it came right back with the newer kernel. The process couldn't be more smooth, if you ask me. 👌

I'm running DietPi 9.9 on a CM4 with eMMC, but I'm using an NVMe disk as boot drive.

@mdr92
Copy link

mdr92 commented Jan 24, 2025

@mdr92 I guess is uses ACPI, hence try this:

systemctl unmask systemd-logind
apt install dbus
systemctl start systemd-logind

Among others, systemd-logind provides ACPI functionality. An alternative would be apt install acpid.

That worked great thanks, shutsdown gracefully on button press.

@MiguVT
Copy link

MiguVT commented Jan 25, 2025

USB boot is not working for me when using testing version from web, when using Trixie version its working, idk why, where is the changelog or commits made?

@MichaIng
Copy link
Owner

MichaIng commented Jan 25, 2025

@MiguVT
Can you specify which testing and Trixie images you tried? Both with the new Linux 6.6 kernel and firmware stack? Between Bookworm and Trixie there is no difference in this regards, kernel and bootloader are the same. But between the old Linux 6.1 and the new Linux 6.6 stack, the difference is huge (at the kernel side), not really a changelog to show, as changes are split across upstream Linux code, RPi Ltd. adjustments, switching the build system, changes at the firmware, dropping old/legacy APIs etc etc. And the changes RPi Ltd show in blog posts, mix kernel/firmware changes together with changes of their image composition, desktop environment etc, which does not apply to DietPi. The kernel repository is here: https://github.com/raspberrypi/linux That orga contains also other firmware and tools repositories, depending on what you are looking for in particular.

One thing we recognised on RPi 5, likely also older models with the new bootloader (upgrading rpi-eeprom package), is that it prevents booting from USB if the PSU does not provide sufficient power. It is a little inconsistent: I can sometimes boot without problems with a weak PSU, but sometimes it hangs at the bootloader, telling that a stronger PSU is needed. On RPi 5, one can then still proceed by clicking the power/reset button once. Not sure whether/how this works on older RPi models. In any case, the official RPi 5 PSU should never have this issue, and if it has, RPi support had some work to do.

@Joulinar
Copy link
Collaborator

Joulinar commented Jan 25, 2025

To disable this check set usb_max_current_enable=1 in config.txt as workaround.

@MichaIng
Copy link
Owner

But keep in mind that the check has a reason: the PSU might be insufficient in peak situations, possible leading to crashes and data loss. So better to solve it with a proper PSU, unless it is for testing only.

@MiguVT
Copy link

MiguVT commented Jan 26, 2025

After checking it more deep it was mb, srry, it was because my sandisk extreme ssd dont works with linux or idk, i tried with an usb stick and works, weird, first time i see an issue like that (magically it booted with Trixie but i tried again and i cant repeat it)

@lukaszsobala
Copy link
Contributor

because my sandisk extreme ssd dont works with linux or idk, i tried with an usb stick and works

Portable SSDs definitely require more power than memory sticks so I don't think it's weird at all. You can try to connect the SSD to the USB2 port and probably achieve the same effect (but of course you'll lose any SSD speed advantages, USB2 is also only half duplex.

@MiguVT
Copy link

MiguVT commented Jan 26, 2025

because my sandisk extreme ssd dont works with linux or idk, i tried with an usb stick and works

Portable SSDs definitely require more power than memory sticks so I don't think it's weird at all. You can try to connect the SSD to the USB2 port and probably achieve the same effect (but of course you'll lose any SSD speed advantages, USB2 is also only half duplex.

its weird because i have a good power supply, and it starts with other os but not with dietpi, usually power supply alert should appear if im not wrong

@MichaIng
Copy link
Owner

Is that SSD separately powered, or only via USB? Do never run a 2.5" drive without dedicated PSU. It may work until it does not work anymore, and then can result in crashes and data loss.

And if it really is this warning/halt of the bootloader, it will be the very same with RPi OS. But as said, it is somehow random, my RPi 5 sometimes boots with a weak PSU without the bootloader interrupting, sometimes not. This may falsify your "starts with other os" test.

But to know for sure, you need to attach a screen or better a USB-UART adapter, to check bootloader output.

@lukaszsobala
Copy link
Contributor

@MichaIng Sandisk Extreme SSDs are portable external SSDs (not any standardised form factor). I think that power consumption is probably similar though. Even x86 machines sometimes struggle to power SSDs, I would definitely not trust development boards with them

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment