RHEL9 systems

RHEL9 Experiences

Experiences with RHEL9 based systems and changes that are good to know.

XFS incompatibility

XFS filesystems made under 9 aren't backwards compatible with 7/8. At Edinburgh we couldn't mount/edit our VM template from a 7/8 host unless it was made using EXT4.

I don't know how/if this will impact proxmox.

JournalCtl

Logs in RAM

Journalctl by default put all of the system journal in /run which is in tmpfs. After some period of time the system locked up due to memory exhaustion which required a reboot and re-configure of journalctl. (Most of the noise in the logs was from external scanning services/tools probing http(s) endpoints, but on one host a significant few GB or so was due to the box being hammered with ssh requests.)

This was the case for Alma 9.0/9.1, it's unknown if Rocky configures this differently.

Performance Bottleneck

In high verbosity environments (multiple-podman containers, or dCache) incorrectly tuning journalctl can lead to problems with performance and useful debugging messages being lost. Edinburgh is investigating a good set of configuration parameters to recommend for high-verbosity environments running on hdd moving forward.

Security

Certificate Key Length Policy

To fix this for SSH see: https://access.redhat.com/solutions/6973518

This is mainly to allow connections back to legacy systems and I don't think changing the system policy to allow usage on RHEL9+ is encouraged

Certificate Encryption Type

SHA-1 at the time of writing is used by

update-crypto-policies --set DEFAULT:SHA1