Devuan bug report logs - #168
apt-get update fails in ascii if a local mirror is used using a file:/// URL due to missing .xz files

Full log

🔗 View this message in rfc822 format

X-Loop: owner@bugs.devuan.org
Subject: bug#168: apt-get update fails in ascii if a local mirror is used using a file:/// URL due to missing .xz files
Reply-To: Daniel Abrecht <me@danielabrecht.ch>, 168@bugs.devuan.org
Resent-From: Daniel Abrecht <me@danielabrecht.ch>
Resent-To: devuan-bugs@lists.dyne.org
Resent-CC: Daniel Reurich <daniel@centurion.net.nz>
Resent-Date: Thu, 14 Dec 2017 16:33:02 UTC
Resent-Message-ID: <handler.168.B.15132690077031@bugs.devuan.org>
Resent-Sender: owner@bugs.devuan.org
X-Devuan-PR-Message: report 168
X-Devuan-PR-Package: amprolla
X-Devuan-PR-Keywords: ascii
Received: via spool by submit@bugs.devuan.org id=B.15132690077031
          (code B ref -1); Thu, 14 Dec 2017 16:33:02 UTC
Received: (at submit) by bugs.devuan.org; 14 Dec 2017 16:30:07 +0000
Delivered-To: devuanbugs@dyne.org
Received: from tupac3.dyne.org [195.169.149.119]
	by fulcanelli with IMAP (fetchmail-6.3.26)
	for <debbugs@localhost> (single-drop); Thu, 14 Dec 2017 17:30:07 +0100 (CET)
Received: from danielabrecht.ch (75-128-16-94.static.cable.fcom.ch [94.16.128.75])
	by vm6.ganeti.dyne.org (Postfix) with ESMTP id 83CEFF60870
	for <submit@bugs.devuan.org>; Thu, 14 Dec 2017 17:23:17 +0100 (CET)
Authentication-Results: vm6.ganeti.dyne.org;
	dkim=pass (1024-bit key; unprotected) header.d=danielabrecht.ch header.i=@danielabrecht.ch header.b="j81jR1pO";
	dkim-atps=neutral
Received: from [10.60.10.140] (unknown [10.60.10.140])
	(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
	(No client certificate requested)
	by danielabrecht.ch (Postfix) with ESMTPSA id DECCC2E02DD0
	for <submit@bugs.devuan.org>; Thu, 14 Dec 2017 16:23:00 +0000 (UTC)
DKIM-Filter: OpenDKIM Filter v2.9.2 danielabrecht.ch DECCC2E02DD0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=danielabrecht.ch;
	s=mail; t=1513268580;
	bh=eLCX8slc2wdewo/gWNCndeqSQYO+R/nGqEUvNcBN49A=; h=To:From:From;
	b=j81jR1pOGILTxErJaaCv2DTe+p6y9dwZzEYRE7aU/u6SvAyNyEdCKmjY0MyWbsFnh
	 7+x1gEt3Wba6p/Z8mJ7WiLjZ0xBljI1P5lsanjVJs+CILynOqO2CE8/pz/PaNAOMLw
	 ViU/YeSMNGSgi9kmb60hRfdSJqJ53+RSm0bE/pn8=
To: submit@bugs.devuan.org
From: Daniel Abrecht <me@danielabrecht.ch>
Message-ID: <8502a42b-9535-d799-2ebc-3013600d6037@danielabrecht.ch>
Date: Thu, 14 Dec 2017 16:23:54 +0000
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.5.0
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Language: de-CH
Content-Transfer-Encoding: 7bit
X-Spam-Status: No, score=-0.1 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,
	DKIM_VALID_AU,SPF_HELO_PASS,SPF_PASS autolearn=disabled version=3.4.1
X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tupac3.dyne.org

Package: amprolla
Version: 3
Tags: ascii

In devuan ascii, apt tries to use .xz archives before .gz archives by
default, but the .xz archives are missing. This doesn't cause any
problems when using http or https transport to access the repos, since
apt will just use the .gz archive when it can't download the .xz
archive. But it fails if a local mirror is used using a file:/// url.
I'm using apt-mirror for my local mirror, but that shouldn't matter.

My current workaround for this is to create a config file at
/etc/apt/apt.conf.d/20archivetypes to specify which archive type to use.
It contains the following line:
Acquire::CompressionTypes::Order:: "gz";

This Problem doesn't affect devuan jessie.

There are multiple possible solutions:
 1) Generate the .xz files
 2) Don't list the .xz files in the release files etc.
 3) Change the behaviour of apt to fallback to other compression types
when accessing a repo over a file:/// url
 4) Creating a config file to change the order using a package


Here is the output of a failed apt-get update. It will only fail if the
files it tries to download have changed since the last successful update:
> root@duck:~# apt-get update
> Get:1 file:/var/local/apt-repo ascii InRelease [63.5 kB]
> Get:1 file:/var/local/apt-repo ascii InRelease [63.5 kB]
> Get:2 file:/var/local/apt-repo ascii-updates InRelease [63.5 kB]
> Get:3 file:/var/local/apt-repo ascii-security InRelease [62.9 kB]
> Get:2 file:/var/local/apt-repo ascii-updates InRelease [63.5 kB]
> Get:3 file:/var/local/apt-repo ascii-security InRelease [62.9 kB]
> Get:4 file:/var/local/apt/mirror/devuanfwojg73k6r.onion/devuan
ascii-proposed InRelease [122 kB]
> Get:4 file:/var/local/apt/mirror/devuanfwojg73k6r.onion/devuan
ascii-proposed InRelease [122 kB]
> Get:5 file:/var/local/apt-repo ascii/main Sources [8,675 kB]
> Err:5 file:/var/local/apt-repo ascii/main Sources
>   Hash Sum mismatch
>   Hashes of expected file:
>    - Filesize:8675180 [weak]
>    -
SHA256:536af199dd5d825a4866a8de4960d98f8cb20921d6de52df8c20ae0376be8bb2
>   Hashes of received file:
>   Release file created at: Wed, 13 Dec 2017 23:04:02 +0000
> Get:6 file:/var/local/apt-repo ascii/main amd64 Packages [9,667 kB]
> Err:6 file:/var/local/apt-repo ascii/main amd64 Packages
>
> Get:7 file:/var/local/apt-repo ascii-updates/main Sources [3,740 B]
> Err:7 file:/var/local/apt-repo ascii-updates/main Sources
>   Hash Sum mismatch
>   Hashes of expected file:
>    - Filesize:3740 [weak]
>    -
SHA256:669f1a2b277cd1e7d2fd4349adc2ee82d53c676afc54376fb0e1ce2405ce4b83
>   Hashes of received file:
>   Release file created at: Thu, 14 Dec 2017 03:18:02 +0000
> Get:8 file:/var/local/apt-repo ascii-updates/main amd64 Packages [6,540 B]
> Err:8 file:/var/local/apt-repo ascii-updates/main amd64 Packages
>
> Get:9 file:/var/local/apt-repo ascii-security/main Sources [115 kB]
> Err:9 file:/var/local/apt-repo ascii-security/main Sources
>   Hash Sum mismatch
>   Hashes of expected file:
>    - Filesize:115364 [weak]
>    -
SHA256:906afc78836447f892c7f063b5991641deb05e5688dc5cb6d6634c9c8543b0c5
>   Hashes of received file:
>   Release file created at: Wed, 13 Dec 2017 23:04:01 +0000
> Get:10 file:/var/local/apt-repo ascii-security/main amd64 Packages
[280 kB]
> Err:10 file:/var/local/apt-repo ascii-security/main amd64 Packages
>
> Get:11 file:/var/local/apt/mirror/devuanfwojg73k6r.onion/devuan
ascii-proposed/main amd64 Packages [19.7 kB]
> Reading package lists... Done
> E: Failed to fetch
file:/var/local/apt-repo/dists/ascii/main/source/Sources.xz  Hash Sum
mismatch
>    Hashes of expected file:
>     - Filesize:8675180 [weak]
>     -
SHA256:536af199dd5d825a4866a8de4960d98f8cb20921d6de52df8c20ae0376be8bb2
>    Hashes of received file:
>    Release file created at: Wed, 13 Dec 2017 23:04:02 +0000
> E: Failed to fetch
file:/var/local/apt-repo/dists/ascii/main/binary-amd64/Packages.xz
> E: Failed to fetch
file:/var/local/apt-repo/dists/ascii-updates/main/source/Sources.xz
Hash Sum mismatch
>    Hashes of expected file:
>     - Filesize:3740 [weak]
>     -
SHA256:669f1a2b277cd1e7d2fd4349adc2ee82d53c676afc54376fb0e1ce2405ce4b83
>    Hashes of received file:
>    Release file created at: Thu, 14 Dec 2017 03:18:02 +0000
> E: Failed to fetch
file:/var/local/apt-repo/dists/ascii-updates/main/binary-amd64/Packages.xz
> E: Failed to fetch
file:/var/local/apt-repo/dists/ascii-security/main/source/Sources.xz
Hash Sum mismatch
>    Hashes of expected file:
>     - Filesize:115364 [weak]
>     -
SHA256:906afc78836447f892c7f063b5991641deb05e5688dc5cb6d6634c9c8543b0c5
>    Hashes of received file:
>    Release file created at: Wed, 13 Dec 2017 23:04:01 +0000
> E: Failed to fetch
file:/var/local/apt-repo/dists/ascii-security/main/binary-amd64/Packages.xz

> E: Some index files failed to download. They have been ignored, or old
ones used instead.


Here is the output of apt-get update after the workaround:
> root@duck:~# apt-get update
> Get:1 file:/var/local/apt-repo ascii InRelease [63.5 kB]
> Get:1 file:/var/local/apt-repo ascii InRelease [63.5 kB]
> Get:2 file:/var/local/apt-repo ascii-updates InRelease [63.5 kB]
> Get:3 file:/var/local/apt-repo ascii-security InRelease [62.9 kB]
> Get:2 file:/var/local/apt-repo ascii-updates InRelease [63.5 kB]
> Get:3 file:/var/local/apt-repo ascii-security InRelease [62.9 kB]
> Get:4 file:/var/local/apt/mirror/devuanfwojg73k6r.onion/devuan
ascii-proposed InRelease [122 kB]
> Get:4 file:/var/local/apt/mirror/devuanfwojg73k6r.onion/devuan
ascii-proposed InRelease [122 kB]
> Get:5 file:/var/local/apt-repo ascii/main Sources [8,675 kB]
> Get:6 file:/var/local/apt-repo ascii/main amd64 Packages [9,666 kB]
> Get:7 file:/var/local/apt-repo ascii-updates/main Sources [3,678 B]

> Get:8 file:/var/local/apt-repo ascii-updates/main amd64 Packages
[6,477 B]
> Get:9 file:/var/local/apt-repo ascii-security/main Sources [115 kB]

> Get:10 file:/var/local/apt-repo ascii-security/main amd64 Packages
[280 kB]
> Reading package lists... Done

Send a report that this bug log contains spam.