Devuan bug report logs - #168
apt-get update fails in ascii if a local mirror is used using a file:/// URL due to missing .xz files

Full log

🔗 View this message in rfc822 format

X-Loop: owner@bugs.devuan.org
From: owner@bugs.devuan.org (Devuan bug Tracking System)
To: KatolaZ <katolaz@freaknet.org>
Cc: Daniel Reurich <daniel@centurion.net.nz>
Subject: bug#168: marked as done (apt-get update fails in ascii if a local mirror is used using a file:/// URL due to missing .xz files)
Message-ID: <handler.168.D168.1513450206915.ackdone@bugs.devuan.org>
In-Reply-To: <20171216184816.GK26595@katolaz.homeunix.net>
References: <20171216184816.GK26595@katolaz.homeunix.net> <8502a42b-9535-d799-2ebc-3013600d6037@danielabrecht.ch>
Precedence: bulk
X-Devuan-PR-Message: closed 168
X-Devuan-PR-Package: amprolla
X-Devuan-PR-Keywords: ascii

Your message dated Sat, 16 Dec 2017 18:48:16 +0000
with message-id <20171216184816.GK26595@katolaz.homeunix.net>
and subject line solved
has caused the attached bug report to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what I am
talking about this indicates a serious mail system misconfiguration
somewhere.  Please contact me immediately.)

Devuan Bugs Owner
(administrator, Devuan bugs database)

--------------------------------------
Received: (at submit) by bugs.devuan.org; 14 Dec 2017 16:30:07 +0000
Return-Path: <me@danielabrecht.ch>
Delivered-To: devuanbugs@dyne.org
Received: from tupac3.dyne.org [195.169.149.119]
	by fulcanelli with IMAP (fetchmail-6.3.26)
	for <debbugs@localhost> (single-drop); Thu, 14 Dec 2017 17:30:07 +0100 (CET)
Received: from danielabrecht.ch (75-128-16-94.static.cable.fcom.ch [94.16.128.75])
	by vm6.ganeti.dyne.org (Postfix) with ESMTP id 83CEFF60870
	for <submit@bugs.devuan.org>; Thu, 14 Dec 2017 17:23:17 +0100 (CET)
Authentication-Results: vm6.ganeti.dyne.org;
	dkim=pass (1024-bit key; unprotected) header.d=danielabrecht.ch header.i=@danielabrecht.ch header.b="j81jR1pO";
	dkim-atps=neutral
Received: from [10.60.10.140] (unknown [10.60.10.140])
	(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
	(No client certificate requested)
	by danielabrecht.ch (Postfix) with ESMTPSA id DECCC2E02DD0
	for <submit@bugs.devuan.org>; Thu, 14 Dec 2017 16:23:00 +0000 (UTC)
DKIM-Filter: OpenDKIM Filter v2.9.2 danielabrecht.ch DECCC2E02DD0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=danielabrecht.ch;
	s=mail; t=1513268580;
	bh=eLCX8slc2wdewo/gWNCndeqSQYO+R/nGqEUvNcBN49A=; h=To:From:From;
	b=j81jR1pOGILTxErJaaCv2DTe+p6y9dwZzEYRE7aU/u6SvAyNyEdCKmjY0MyWbsFnh
	 7+x1gEt3Wba6p/Z8mJ7WiLjZ0xBljI1P5lsanjVJs+CILynOqO2CE8/pz/PaNAOMLw
	 ViU/YeSMNGSgi9kmb60hRfdSJqJ53+RSm0bE/pn8=
To: submit@bugs.devuan.org
From: Daniel Abrecht <me@danielabrecht.ch>
Subject: apt-get update fails in ascii if a local mirror is used using a
 file:/// URL due to missing .xz files
Message-ID: <8502a42b-9535-d799-2ebc-3013600d6037@danielabrecht.ch>
Date: Thu, 14 Dec 2017 16:23:54 +0000
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.5.0
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Language: de-CH
Content-Transfer-Encoding: 7bit
X-Spam-Status: No, score=-0.1 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,
	DKIM_VALID_AU,SPF_HELO_PASS,SPF_PASS autolearn=disabled version=3.4.1
X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tupac3.dyne.org

Package: amprolla
Version: 3
Tags: ascii

In devuan ascii, apt tries to use .xz archives before .gz archives by
default, but the .xz archives are missing. This doesn't cause any
problems when using http or https transport to access the repos, since
apt will just use the .gz archive when it can't download the .xz
archive. But it fails if a local mirror is used using a file:/// url.
I'm using apt-mirror for my local mirror, but that shouldn't matter.

My current workaround for this is to create a config file at
/etc/apt/apt.conf.d/20archivetypes to specify which archive type to use.
It contains the following line:
Acquire::CompressionTypes::Order:: "gz";

This Problem doesn't affect devuan jessie.

There are multiple possible solutions:
 1) Generate the .xz files
 2) Don't list the .xz files in the release files etc.
 3) Change the behaviour of apt to fallback to other compression types
when accessing a repo over a file:/// url
 4) Creating a config file to change the order using a package


Here is the output of a failed apt-get update. It will only fail if the
files it tries to download have changed since the last successful update:
> root@duck:~# apt-get update
> Get:1 file:/var/local/apt-repo ascii InRelease [63.5 kB]
> Get:1 file:/var/local/apt-repo ascii InRelease [63.5 kB]
> Get:2 file:/var/local/apt-repo ascii-updates InRelease [63.5 kB]
> Get:3 file:/var/local/apt-repo ascii-security InRelease [62.9 kB]
> Get:2 file:/var/local/apt-repo ascii-updates InRelease [63.5 kB]
> Get:3 file:/var/local/apt-repo ascii-security InRelease [62.9 kB]
> Get:4 file:/var/local/apt/mirror/devuanfwojg73k6r.onion/devuan
ascii-proposed InRelease [122 kB]
> Get:4 file:/var/local/apt/mirror/devuanfwojg73k6r.onion/devuan
ascii-proposed InRelease [122 kB]
> Get:5 file:/var/local/apt-repo ascii/main Sources [8,675 kB]
> Err:5 file:/var/local/apt-repo ascii/main Sources
>   Hash Sum mismatch
>   Hashes of expected file:
>    - Filesize:8675180 [weak]
>    -
SHA256:536af199dd5d825a4866a8de4960d98f8cb20921d6de52df8c20ae0376be8bb2
>   Hashes of received file:
>   Release file created at: Wed, 13 Dec 2017 23:04:02 +0000
> Get:6 file:/var/local/apt-repo ascii/main amd64 Packages [9,667 kB]
> Err:6 file:/var/local/apt-repo ascii/main amd64 Packages
>
> Get:7 file:/var/local/apt-repo ascii-updates/main Sources [3,740 B]
> Err:7 file:/var/local/apt-repo ascii-updates/main Sources
>   Hash Sum mismatch
>   Hashes of expected file:
>    - Filesize:3740 [weak]
>    -
SHA256:669f1a2b277cd1e7d2fd4349adc2ee82d53c676afc54376fb0e1ce2405ce4b83
>   Hashes of received file:
>   Release file created at: Thu, 14 Dec 2017 03:18:02 +0000
> Get:8 file:/var/local/apt-repo ascii-updates/main amd64 Packages [6,540 B]
> Err:8 file:/var/local/apt-repo ascii-updates/main amd64 Packages
>
> Get:9 file:/var/local/apt-repo ascii-security/main Sources [115 kB]
> Err:9 file:/var/local/apt-repo ascii-security/main Sources
>   Hash Sum mismatch
>   Hashes of expected file:
>    - Filesize:115364 [weak]
>    -
SHA256:906afc78836447f892c7f063b5991641deb05e5688dc5cb6d6634c9c8543b0c5
>   Hashes of received file:
>   Release file created at: Wed, 13 Dec 2017 23:04:01 +0000
> Get:10 file:/var/local/apt-repo ascii-security/main amd64 Packages
[280 kB]
> Err:10 file:/var/local/apt-repo ascii-security/main amd64 Packages
>
> Get:11 file:/var/local/apt/mirror/devuanfwojg73k6r.onion/devuan
ascii-proposed/main amd64 Packages [19.7 kB]
> Reading package lists... Done
> E: Failed to fetch
file:/var/local/apt-repo/dists/ascii/main/source/Sources.xz  Hash Sum
mismatch
>    Hashes of expected file:
>     - Filesize:8675180 [weak]
>     -
SHA256:536af199dd5d825a4866a8de4960d98f8cb20921d6de52df8c20ae0376be8bb2
>    Hashes of received file:
>    Release file created at: Wed, 13 Dec 2017 23:04:02 +0000
> E: Failed to fetch
file:/var/local/apt-repo/dists/ascii/main/binary-amd64/Packages.xz
> E: Failed to fetch
file:/var/local/apt-repo/dists/ascii-updates/main/source/Sources.xz
Hash Sum mismatch
>    Hashes of expected file:
>     - Filesize:3740 [weak]
>     -
SHA256:669f1a2b277cd1e7d2fd4349adc2ee82d53c676afc54376fb0e1ce2405ce4b83
>    Hashes of received file:
>    Release file created at: Thu, 14 Dec 2017 03:18:02 +0000
> E: Failed to fetch
file:/var/local/apt-repo/dists/ascii-updates/main/binary-amd64/Packages.xz
> E: Failed to fetch
file:/var/local/apt-repo/dists/ascii-security/main/source/Sources.xz
Hash Sum mismatch
>    Hashes of expected file:
>     - Filesize:115364 [weak]
>     -
SHA256:906afc78836447f892c7f063b5991641deb05e5688dc5cb6d6634c9c8543b0c5
>    Hashes of received file:
>    Release file created at: Wed, 13 Dec 2017 23:04:01 +0000
> E: Failed to fetch
file:/var/local/apt-repo/dists/ascii-security/main/binary-amd64/Packages.xz

> E: Some index files failed to download. They have been ignored, or old
ones used instead.


Here is the output of apt-get update after the workaround:
> root@duck:~# apt-get update
> Get:1 file:/var/local/apt-repo ascii InRelease [63.5 kB]
> Get:1 file:/var/local/apt-repo ascii InRelease [63.5 kB]
> Get:2 file:/var/local/apt-repo ascii-updates InRelease [63.5 kB]
> Get:3 file:/var/local/apt-repo ascii-security InRelease [62.9 kB]
> Get:2 file:/var/local/apt-repo ascii-updates InRelease [63.5 kB]
> Get:3 file:/var/local/apt-repo ascii-security InRelease [62.9 kB]
> Get:4 file:/var/local/apt/mirror/devuanfwojg73k6r.onion/devuan
ascii-proposed InRelease [122 kB]
> Get:4 file:/var/local/apt/mirror/devuanfwojg73k6r.onion/devuan
ascii-proposed InRelease [122 kB]
> Get:5 file:/var/local/apt-repo ascii/main Sources [8,675 kB]
> Get:6 file:/var/local/apt-repo ascii/main amd64 Packages [9,666 kB]
> Get:7 file:/var/local/apt-repo ascii-updates/main Sources [3,678 B]

> Get:8 file:/var/local/apt-repo ascii-updates/main amd64 Packages
[6,477 B]
> Get:9 file:/var/local/apt-repo ascii-security/main Sources [115 kB]

> Get:10 file:/var/local/apt-repo ascii-security/main amd64 Packages
[280 kB]
> Reading package lists... Done
---------------------------------------
Received: (at 168-close) by bugs.devuan.org; 16 Dec 2017 18:50:06 +0000
Return-Path: <katolaz@freaknet.org>
Delivered-To: devuanbugs@dyne.org
Received: from tupac3.dyne.org [195.169.149.119]
	by fulcanelli with IMAP (fetchmail-6.3.26)
	for <debbugs@localhost> (single-drop); Sat, 16 Dec 2017 19:50:06 +0100 (CET)
Received: from [127.0.0.1] (localhost [127.0.0.1])
	(Authenticated sender: katolaz@freaknet.org)
	with ESMTPSA id 36296F60946
Date: Sat, 16 Dec 2017 18:48:16 +0000
From: KatolaZ <katolaz@freaknet.org>
To: 168-close@bugs.devuan.org
Subject: solved
Message-ID: <20171216184816.GK26595@katolaz.homeunix.net>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha1;
	protocol="application/pgp-signature"; boundary="xeVkHwl2xWn7ZMXD"
Content-Disposition: inline
User-Agent: Mutt/1.5.23 (2014-03-12)
X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED
	autolearn=disabled version=3.4.1
X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tupac3.dyne.org


--xeVkHwl2xWn7ZMXD
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

Solved by the following patch:

  https://git.devuan.org/devuan-infrastructure/amprolla3/commit/590e458822ade96226382d34bea568d1d0c7d5ad

Closing

--xeVkHwl2xWn7ZMXD
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iEYEARECAAYFAlo1anAACgkQXyCzrgtfBi9lkwCeNOgfiGGhAGfj3rLJj3GbTJtD
KdkAn1A61eI8ndHzghdXYtcVB68zXnqD
=rP/0
-----END PGP SIGNATURE-----

--xeVkHwl2xWn7ZMXD--

Send a report that this bug log contains spam.