Subject: devuan-project: After an upgrade (apt update; apt full-upgrade) the PC fails to reboot correctly.

From: David Haworth <[email-masked]>
Date: Fri, 05 Jan 2024 12:58:15 +0100

Package: devuan-project
Severity: grave
Justification: renders package unusable

Dear Maintainer,

Yesterday I updated my OS in order to install AusweisApp2.
On the first attempt I got some errors from the update command, but they
seemed to be related to source packages. The second attempt appeared to
succeed, and AusweisApp2 worked as expected.

Today, after shutdown/reboot, there were lots of error messages. The switch to
high-resolution text display failed, no display manager started and I was dumped
into an 80x25 text screen, from which I could log in.

From the error messages that flashed past, I was able to determine a few problems:

* One of the init scripts fails because it cannot find /sbin/modprobe.
  ** modprobe appears to be in /usr/sbin now.
  ** As a workaround, I created a symlink:  ln -s /usr/bin/kmod /sbin/modprobe
* Similar failures for depmod and modinfo, similar symlinks created.

* After the above more error messages from /etc/init.d/mountall.sh, /etc/init.d/mountdevsubfs.sh and others
  about not being able to find grep and sed. On examination, these files I noticed that the PATH is forced
  to /sbin:/bin at the top. grep and sed are in /usr/bin now.
  ** As a workaround, I edited the files and added :/usr/sbin:/usr/bin to the path.

* After the above I got error messages about eudev. On examination of /etc/init.d/uedev I noticed a similar
  problem with PATH, which I worked around in a similar way.

Now my PC boots to a display manager and I can log in as expected. I didn't notice any further error messages.

As you can see, the problems seem to be associated with several packages, so it's hard to pinpoint one package.

My guess is that the programs that I mentioned - modprobe, depmod, modinfo, grep, sed - and many others are
being installed in the wrong place. My reasoning is that, in a traditional Unix system, it should be possible
to place /usr in a separate filesystem that won't be available at the time these scripts are called.
However, on modern hardware, it's hardly ever necessary to have a separate filesystem for /usr, so it
could be simply a path problem in several init scripts of at least two packages (initscripts and eudev)

Best wishes,
Dave

-- System Information:
Distributor ID:	Devuan
Description:	Devuan GNU/Linux 6 (excalibur/ceres)
Release:	6
Codename:	excalibur ceres
Architecture: x86_64

Kernel: Linux 6.1.0-1-amd64 (SMP w/8 CPU threads; PREEMPT)
Kernel taint flags: TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=en_US:en
Shell: /bin/sh linked to /bin/dash
Init: sysvinit (via /sbin/init)
LSM: AppArmor: enabled

---

From: Lorenzo <[email-masked]>
Date: Fri, 5 Jan 2024 13:44:25 +0100

On Fri, 05 Jan 2024 12:58:15 +0100
David Haworth <[email-masked]> wrote:

> On the first attempt I got some errors from the update command, but
> they seemed to be related to source packages. The second attempt
> appeared to succeed, and AusweisApp2 worked as expected.
> 
> Today, after shutdown/reboot, there were lots of error messages. The
> switch to high-resolution text display failed, no display manager
> started and I was dumped into an 80x25 text screen, from which I
> could log in.
> 
> * After the above more error messages from /etc/init.d/mountall.sh,
> /etc/init.d/mountdevsubfs.sh and others about not being able to find
> grep and sed. On examination, these files I noticed that the PATH is
> forced to /sbin:/bin at the top. grep and sed are in /usr/bin now. **
> As a workaround, I edited the files and added :/usr/sbin:/usr/bin to
> the path.
> 
> * After the above I got error messages about eudev. On examination of
> /etc/init.d/uedev I noticed a similar problem with PATH, which I
> worked around in a similar way.
> 
> Now my PC boots to a display manager and I can log in as expected. I
> didn't notice any further error messages.
> 
> As you can see, the problems seem to be associated with several
> packages, so it's hard to pinpoint one package.
> 
> My guess is that the programs that I mentioned - modprobe, depmod,
> modinfo, grep, sed - and many others are being installed in the wrong
> place. My reasoning is that, in a traditional Unix system, it should
> be possible to place /usr in a separate filesystem that won't be
> available at the time these scripts are called. However, on modern
> hardware, it's hardly ever necessary to have a separate filesystem
> for /usr, so it could be simply a path problem in several init
> scripts of at least two packages (initscripts and eudev)

I agree but split /usr without initrd is not supported (upstream) since
like a decade, it more or less worked until now but Debian is finally
breaking it for good with the usrmerge fallout. See my reply in #827
https://bugs.devuan.org/cgi/bugreport.cgi?bug=827

> Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8),
> LANGUAGE=en_US:en Shell: /bin/sh linked to /bin/dash
> _______________________________________________
> devuan-dev internal mailing list
> [email-masked]
> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/devuan-dev

---

From: David Haworth <[email-masked]>
Date: Fri, 5 Jan 2024 14:28:26 +0100

Hi Lorenzo,

> https://bugs.devuan.org/cgi/bugreport.cgi?bug=827

Don't get me wrong - I don't have a split system like that.
I was just wondering what a "correct" solution would look like.

Best wishes,
Dave

---

From: David Haworth <[email-masked]>
Date: Fri, 5 Jan 2024 15:58:42 +0100

Some more information:

/dev/loop0 /dev/loop1 etc. are missing. It might be related to the
eudev problem.

Dave

---

From: David Haworth <[email-masked]>
Date: Fri, 5 Jan 2024 17:37:36 +0100

Hi,

the cause of the missing loop devices was that the loop kernel
module wasn't loaded.

Workaround:
  modprobe /lib/modules/6.1.0-1-amd64/kernel/drivers/block/loop.ko

---

From: Lorenzo <[email-masked]>
Date: Fri, 5 Jan 2024 18:06:20 +0100

On Fri, 5 Jan 2024 14:28:26 +0100
David Haworth <[email-masked]> wrote:

> > I agree but split /usr without initrd is not supported (upstream)
> > since like a decade, it more or less worked until now but Debian is
> > finally breaking it for good with the usrmerge fallout. See my
> > reply in #827 https://bugs.devuan.org/cgi/bugreport.cgi?bug=827
> 

I think the safest thing is to merge your system, by installing
the usrmerge package; it will move stuff for you and turn /bin
/sbin /lib into symlinks to their /usr/* conterparts.
(example /bin ---> /usr/bin)
This way you go to the layout that is officially supported
by Debian and all this trouble will disappear.

If for some reason, you don't want to do that, expect more
breakage in the incoming weeks. You can try to counter that
with symlinks or by extending the PATH var in scripts,
removing hardcoded paths and so but I'm not sure it will be
sustainable.
Note that while Debian decided to move binaries inside packages,
they decided to not break the amd64 ABI, so binaries can still
use /lib*/ld-linux-x86-64.so.2 as intepreter, and scripts will
keep using /bin/sh. I'm really not sure that will be possible
to create symlinks with 'ln' when /lib/ is empty.. prepare a
rescue system to repair the links, just in case.

Lorenzo