Hi,

 

I am trying to determine whether this is expected behavior or a bug/ordering issue in Armbian's armbian-ramlog.

 

My /etc/systemd/journald.conf contains:

Storage=persistent

SystemMaxUse=20M

 

Persistent journal storage is available:

/var/log/journal -> /var/log.hdd/journal

and /var/log.hdd is mounted read-write.

 

However, after every reboot, systemd-journald initially writes to /run/log/journal.

 

I verified this directly:

sudo ls -l /proc/$(pidof systemd-journald)/fd | grep journal

Immediately after boot I get:

33 -> /run/log/journal/d1d6e2a49fd24499b90cb6d5198cb70e/system.journal

 

So journald is actually holding the runtime journal open.

 

The system has armbian-ramlog enabled. Its unit contains:

Before=rsyslog.service sysinit.target syslog.target RequiresMountsFor=/var/log /var/log.hdd

 

Interestingly, running this manually:

sudo journalctl --flush

immediately switches journald to persistent storage.

 

Afterwards:

sudo ls -l /proc/$(pidof systemd-journald)/fd | grep journal

shows:

33 -> /var/log.hdd/journal/d1d6e2a49fd24499b90cb6d5198cb70e/system.journal 34 -> /var/log.hdd/journal/d1d6e2a49fd24499b90cb6d5198cb70e/user-1000.journal

 

So the persistent filesystem is definitely accessible and writable.

 

This makes me wonder:

Is armbian-ramlog causing systemd-journald to remain on /run/log/journal after boot, or is this simply an ordering issue involving systemd-journal-flush.service?

In other words, should armbian-ramlog allow/trigger the journal flush automatically once /var/log.hdd is mounted?

Any insight from the Armbian side would be appreciated.

Your debugging is already spot on — journald only stays on /run/log/journal until something runs journalctl --flush, and the fact that a manual flush works proves the storage side is fine. So this is purely an ordering problem: systemd-journal-flush.service runs early in boot, and on Armbian images /var/log.hdd is typically mounted by the armbian-ramlog machinery, which replaces /var/log with its tmpfs/overlay quite late compared to when the flush service expects a real /var/log/journal to exist. If the flush fires while ramlog hasn't finished restoring the /var/log/journal symlink into /var/log.hdd, it silently gives up and journald keeps the runtime journal open — exactly what you're seeing in /proc.

You can confirm by comparing timestamps: systemctl status systemd-journal-flush.service versus armbian-ramlog (or mount time of /var/log.hdd). The clean fix is a drop-in for the flush unit, something like After=armbian-ramlog.service plus RequiresMountsFor=/var/log.hdd, so flush waits until the persistent path is actually there. Alternatively, an override on the ramlog unit adding Before=systemd-journal-flush.service achieves the same ordering from the other side. Either way the runtime journal gets flushed on the next boot instead of being stranded in /run until a manual flush.