Devuan bug report logs - #406
apt-2.0.0: Runs very, very slow after update to version 2.0

Package: daptup; Maintainer for daptup is (unknown);

Reported by: Klaus Ethgen <Klaus@Ethgen.ch>

Date: Mon, 9 Mar 2020 18:33:01 UTC

Severity: normal

Tags: debian

Reply or subscribe to this bug.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to devuan-bugs@lists.dyne.org, devuan-dev@lists.dyne.org:
bug#406; Package apt. (Mon, 09 Mar 2020 18:33:01 GMT) (full text, mbox, link).


Acknowledgement sent to Klaus Ethgen <Klaus@Ethgen.ch>:
New bug report received and forwarded. Copy sent to devuan-dev@lists.dyne.org. (Mon, 09 Mar 2020 18:33:04 GMT) (full text, mbox, link).


Message #5 received at submit@bugs.devuan.org (full text, mbox, reply):

From: Klaus Ethgen <Klaus@Ethgen.ch>
To: submit@bugs.devuan.org
Subject: apt-2.0.0: Runs very, very slow after update to version 2.0
Date: Mon, 9 Mar 2020 19:16:48 +0100
[Message part 1 (text/plain, inline)]
Package: apt
Version: 2.0.0

After updating apt to version 2.0.0, there are two times that takes ~10
minutes:
~> apt-get update; apt-get --auto-remove dist-upgrade && apt-get clean && sync
Erstelle alte Liste von Paketen...
Erstelle alte Liste von verfügbaren Aktualisierungen...
Erstelle alte Liste von beobachteten Paketen...
-> 10 Minutes
...
Es wurden 212 MB in 33 s geholt (6.340 kB/s).
Erstelle neue Liste von Paketen...
-> again 10 minutes.

It seems that there is all a apt-cache running that long. It seems that
daptub is running that.

Wishes
   Klaus

Ps. That system is a old sid system witch was converted to ceres. It is
    my fastest system but with many source lists.
-- 
Klaus Ethgen                                       http://www.ethgen.ch/
pub  4096R/4E20AF1C 2011-05-16            Klaus Ethgen <Klaus@Ethgen.ch>
Fingerprint: 85D4 CA42 952C 949B 1753  62B3 79D0 B06F 4E20 AF1C
[signature.asc (application/pgp-signature, inline)]

Information forwarded to devuan-bugs@lists.dyne.org, devuan-dev@lists.dyne.org:
bug#406; Package apt. (Tue, 10 Mar 2020 10:33:02 GMT) (full text, mbox, link).


Message #8 received at 406@bugs.devuan.org (full text, mbox, reply):

From: Mark Hindley <mark@hindley.org.uk>
To: 406@bugs.devuan.org
Cc: Klaus Ethgen <Klaus@Ethgen.ch>
Subject: Re: apt-2.0.0: Runs very, very slow after update to version 2.0
Date: Tue, 10 Mar 2020 10:18:48 +0000
Control: tags -1 debian

On Mon, 9 Mar 2020 19:16:48 +0100 Klaus Ethgen <Klaus@Ethgen.ch> wrote:
> Package: apt
> Version: 2.0.0
> 
> After updating apt to version 2.0.0, there are two times that takes ~10
> minutes:

Klaus,

Thanks for this.

However, since apt is not a forked package (devuan uses Debian's package
directly) you would be much better reporting this issue directly to Debian's
BTS.

Thanks.

Mark

Added tag(s) debian. Request was from Mark Hindley <mark@hindley.org.uk> to 406-submit@bugs.devuan.org. (Tue, 10 Mar 2020 10:33:03 GMT) (full text, mbox, link).


Information forwarded to devuan-bugs@lists.dyne.org:
bug#406; Package apt. (Tue, 07 Apr 2020 18:03:01 GMT) (full text, mbox, link).


Acknowledgement sent to Andreas Messer <andi@bastelmap.de>:
Extra info received and forwarded to list. (Tue, 07 Apr 2020 18:03:04 GMT) (full text, mbox, link).


Message #15 received at 406@bugs.devuan.org (full text, mbox, reply):

From: Andreas Messer <andi@bastelmap.de>
To: 406@bugs.devuan.org
Cc: Klaus Ethgen <Klaus@Ethgen.ch>
Subject: Re: apt-2.0.0: Runs very, very slow after update to version 2.0
Date: Tue, 7 Apr 2020 19:48:43 +0200
[Message part 1 (text/plain, inline)]
Hello Klaus,

A devuanized version of the apt package has just been uploaded to
unstable/ceres. (2.0.1+devuan1) Should be available within the 
next hours, can you confirm the problem still exists with it?

Would you mind providing your sources.list(s) to me, such that
I can try to reproduce the problem? Thanks!

cheers,
Andreas
-- 
gnuPG keyid: 8C2BAF51
fingerprint: 28EE 8438 E688 D992 3661 C753 90B3 BAAA 8C2B AF51
[signature.asc (application/pgp-signature, inline)]

Information forwarded to devuan-bugs@lists.dyne.org, Andreas Messer <andi@bastelmap.de>:
bug#406; Package apt. (Thu, 16 Apr 2020 20:33:01 GMT) (full text, mbox, link).


Acknowledgement sent to Klaus Ethgen <Klaus@Ethgen.ch>:
Extra info received and forwarded to list. Copy sent to Andreas Messer <andi@bastelmap.de>. (Thu, 16 Apr 2020 20:33:04 GMT) (full text, mbox, link).


Message #20 received at 406@bugs.devuan.org (full text, mbox, reply):

From: Klaus Ethgen <Klaus@Ethgen.ch>
To: Andreas Messer <andi@bastelmap.de>
Cc: 406@bugs.devuan.org
Subject: Re: apt-2.0.0: Runs very, very slow after update to version 2.0
Date: Thu, 16 Apr 2020 21:12:06 +0100
[Message part 1 (text/plain, inline)]
Hello Andreas,

Am Di den  7. Apr 2020 um 18:48 schrieb Andreas Messer:
> A devuanized version of the apt package has just been uploaded to
> unstable/ceres. (2.0.1+devuan1) Should be available within the 
> next hours, can you confirm the problem still exists with it?
> 
> Would you mind providing your sources.list(s) to me, such that
> I can try to reproduce the problem? Thanks!

Sure.
```
/etc/apt (git-[etc:masterU])> grep -vh '^#' sources.list sources.list.d/* G -v '^$'
deb file:///home/ftp/pub/debian ceres unofficial
deb-src file:///home/ftp/pub/debian ceres unofficial
deb file:///home/ftp/pub/debian-security ceres unofficial-secured
deb http://auto.mirror.devuan.org/merged ceres main contrib non-free
deb-src http://ftp.ch.debian.org/debian sid main contrib non-free
deb http://ftp.ch.debian.org/debian experimental main contrib non-free
deb-src http://ftp.ch.debian.org/debian experimental main contrib non-free
deb [trusted=yes] file:///home/local_repo sid main
deb http://www.deb-multimedia.org/ sid main non-free
deb [check-valid-until=no] http://snapshot.debian.org/archive/debian/20150521T214404Z sid main
deb [check-valid-until=no] http://snapshot.debian.org/archive/debian/20161010T164513Z sid main
deb https://www.mirbsd.org/~tg/Debs/ sid wtf
```

install daptub and run `/usr/bin/daptup --hook --post`, that will take
10 Minutes.

apt is currently in the following version:
```
~> apt-cache policy apt
apt:
  Installiert:           2.0.1
  Installationskandidat: 2.0.1
  Versionstabelle:
 *** 2.0.1 500
        500 http://auto.mirror.devuan.org/merged ceres/main amd64 Packages
        100 /var/lib/dpkg/status
     1.3.1 400
        400 http://snapshot.debian.org/archive/debian/20161010T164513Z sid/main amd64 Packages
     1.0.9.9 400
        400 http://snapshot.debian.org/archive/debian/20150521T214404Z sid/main amd64 Packages
```

Regards
   Klaus
-- 
Klaus Ethgen                                       http://www.ethgen.ch/
pub  4096R/4E20AF1C 2011-05-16            Klaus Ethgen <Klaus@Ethgen.ch>
Fingerprint: 85D4 CA42 952C 949B 1753  62B3 79D0 B06F 4E20 AF1C
[signature.asc (application/pgp-signature, inline)]

Information forwarded to devuan-bugs@lists.dyne.org:
bug#406; Package apt. (Sun, 03 May 2020 17:33:01 GMT) (full text, mbox, link).


Acknowledgement sent to Andreas Messer <andi@bastelmap.de>:
Extra info received and forwarded to list. (Sun, 03 May 2020 17:33:04 GMT) (full text, mbox, link).


Message #25 received at 406@bugs.devuan.org (full text, mbox, reply):

From: Andreas Messer <andi@bastelmap.de>
To: Klaus Ethgen <Klaus@Ethgen.ch>
Cc: 406@bugs.devuan.org
Subject: Re: bug#406: apt-2.0.0: Runs very, very slow after update to version 2.0
Date: Sun, 3 May 2020 19:20:30 +0200
[Message part 1 (text/plain, inline)]
Hi Klaus,

On Thu, Apr 16, 2020 at 09:12:06PM +0100, Klaus Ethgen wrote:
> > Would you mind providing your sources.list(s) to me, such that
> > I can try to reproduce the problem? Thanks!
> 
> Sure.
> [ Removed apt sources.list ]
> 
> install daptub and run `/usr/bin/daptup --hook --post`, that will take
> 10 Minutes.

I had to remove some of your local repos for testing, so my test result
is probably not of much use. Running the command took about 6 seconds
on my machine.

As far as I understand daptup is another tool not part of apt package. 
So maybe it is at the moment not apt to blame. We should figure out were 
the delay occurs. Did you check that you local file repos are available 
(e.g. mounted properly before running the tool?)

Could you please install libdevel-dprof-perl and then run daptup as
follows within /tmp:

    $ sudo perl -d:DProf /usr/bin/daptup --hook --post

Then please invoke:

    $ dprofpp

And mail the output.

Best regards
Andreas

-- 
gnuPG keyid: 8C2BAF51
fingerprint: 28EE 8438 E688 D992 3661 C753 90B3 BAAA 8C2B AF51
[signature.asc (application/pgp-signature, inline)]

bug reassigned from package 'apt' to 'daptup'. Request was from Andreas Messer <andi@bastelmap.de> to control@bugs.devuan.org. (Sun, 03 May 2020 17:33:07 GMT) (full text, mbox, link).


No longer marked as found in versions 2.0.0. Request was from Andreas Messer <andi@bastelmap.de> to control@bugs.devuan.org. (Sun, 03 May 2020 17:33:07 GMT) (full text, mbox, link).


Information forwarded to devuan-bugs@lists.dyne.org, devuan-dev@lists.dyne.org:
bug#406; Package daptup. (Sun, 03 May 2020 20:03:02 GMT) (full text, mbox, link).


Acknowledgement sent to Klaus Ethgen <Klaus@Ethgen.ch>:
Extra info received and forwarded to list. Copy sent to devuan-dev@lists.dyne.org. (Sun, 03 May 2020 20:03:04 GMT) (full text, mbox, link).


Message #34 received at 406@bugs.devuan.org (full text, mbox, reply):

From: Klaus Ethgen <Klaus@Ethgen.ch>
To: Andreas Messer <andi@bastelmap.de>
Cc: 406@bugs.devuan.org
Subject: Re: bug#406: apt-2.0.0: Runs very, very slow after update to version 2.0
Date: Sun, 3 May 2020 20:37:19 +0100
[Message part 1 (text/plain, inline)]
Hi Andreas,

Am So den  3. Mai 2020 um 18:20 schrieb Andreas Messer:
> I had to remove some of your local repos for testing, so my test result
> is probably not of much use. Running the command took about 6 seconds
> on my machine.

Strange..

> As far as I understand daptup is another tool not part of apt package. 
> So maybe it is at the moment not apt to blame. We should figure out were 
> the delay occurs. Did you check that you local file repos are available 
> (e.g. mounted properly before running the tool?)

Yes, they are. Sure they are. And the speed change came exactly with the
new apt release.

> Could you please install libdevel-dprof-perl and then run daptup as
> follows within /tmp:
> 
>     $ sudo perl -d:DProf /usr/bin/daptup --hook --post
> 
> Then please invoke:
> 
>     $ dprofpp
> 
> And mail the output.

No sudo here and I did it in root $HOME:
```
~> dprofpp
Total Elapsed Time = 669.6876 Seconds
  User+System Time = 1.747609 Seconds
Exclusive Times
%Time ExclSec CumulS #Calls sec/call Csec/c  Name
 44.6   0.780  0.780      1   0.7800 0.7800  Daptup::Backend::AptOrCupt::get_updates
 35.4   0.620  1.689      1   0.6196 1.6892  main::do_post
 15.4   0.270  0.270  80346   0.0000 0.0000  Term::ANSIColor::color
 1.72   0.030  0.069     11   0.0027 0.0062  main::BEGIN
 0.57   0.010  0.010      3   0.0033 0.0033  main::diff_cmd
 0.57   0.010  0.010      1   0.0100 0.0100  main::find_old_packages
 0.57   0.010  0.010      4   0.0025 0.0025  Daptup::Backend::Aptitude::BEGIN
 0.57   0.010  0.010      7   0.0014 0.0014  IO::Handle::BEGIN
 0.57   0.010  0.010      8   0.0012 0.0012  File::Spec::Unix::BEGIN
 0.57   0.010  0.010      7   0.0014 0.0014  Log::Agent::BEGIN
 0.57   0.010  0.029     21   0.0005 0.0014  File::Temp::BEGIN
 0.00   0.000  0.000      1   0.0000 0.0000  Exporter::Heavy::heavy_export_ok_tags
 0.00   0.000  0.000      1   0.0000 0.0000  Getopt::Long::GetOptionsFromArray
 0.00        - -0.000      1         -       -  main::get_watch_regex
 0.00        - -0.000      1         -       -  IO::Handle::__ANON__
```

Gruß
   Klaus
-- 
Klaus Ethgen                                       http://www.ethgen.ch/
pub  4096R/4E20AF1C 2011-05-16            Klaus Ethgen <Klaus@Ethgen.ch>
Fingerprint: 85D4 CA42 952C 949B 1753  62B3 79D0 B06F 4E20 AF1C
[signature.asc (application/pgp-signature, inline)]

Information forwarded to devuan-bugs@lists.dyne.org, devuan-dev@lists.dyne.org:
bug#406; Package daptup. (Sun, 03 May 2020 20:33:01 GMT) (full text, mbox, link).


Acknowledgement sent to Klaus Ethgen <Klaus@Ethgen.ch>:
Extra info received and forwarded to list. Copy sent to devuan-dev@lists.dyne.org. (Sun, 03 May 2020 20:33:03 GMT) (full text, mbox, link).


Message #39 received at 406@bugs.devuan.org (full text, mbox, reply):

From: Klaus Ethgen <Klaus@Ethgen.ch>
To: Andreas Messer <andi@bastelmap.de>
Cc: 406@bugs.devuan.org
Subject: Re: bug#406: apt-2.0.0: Runs very, very slow after update to version 2.0
Date: Sun, 3 May 2020 21:18:28 +0100
[Message part 1 (text/plain, inline)]
Maybe the daptup.conf would be of some help.

Gruß
   Klaus
-- 
Klaus Ethgen                                       http://www.ethgen.ch/
pub  4096R/4E20AF1C 2011-05-16            Klaus Ethgen <Klaus@Ethgen.ch>
Fingerprint: 85D4 CA42 952C 949B 1753  62B3 79D0 B06F 4E20 AF1C
[daptup.conf (text/plain, attachment)]
[signature.asc (application/pgp-signature, inline)]

Information forwarded to devuan-bugs@lists.dyne.org, devuan-dev@lists.dyne.org:
bug#406; Package daptup. (Sun, 03 May 2020 21:18:02 GMT) (full text, mbox, link).


Acknowledgement sent to Klaus Ethgen <Klaus@Ethgen.ch>:
Extra info received and forwarded to list. Copy sent to devuan-dev@lists.dyne.org. (Sun, 03 May 2020 21:18:04 GMT) (full text, mbox, link).


Message #44 received at 406@bugs.devuan.org (full text, mbox, reply):

From: Klaus Ethgen <Klaus@Ethgen.ch>
To: Andreas Messer <andi@bastelmap.de>
Cc: 406@bugs.devuan.org
Subject: Re: bug#406: apt-2.0.0: Runs very, very slow after update to version 2.0
Date: Sun, 3 May 2020 21:58:38 +0100
[Message part 1 (text/plain, inline)]
Another think I seen..

`apt search '.*'` takes hours (as does all apt search or list calls).

But when I do the usual way `apt-cache search '.*'` it gives me the list
instant. (But it runs ages.)

The man page of apt says that this two should be identical and it
definitively was in the old version but now they are complete different.

By the way, even apt-cache needs some time to finish. I let it count the
lines (packages) and I get a count of 80318 lines which takes nearly 11
minutes time. My /var/lib/dpkg/available file only counts 90655
packages.

Gruß
   Klaus
-- 
Klaus Ethgen                                       http://www.ethgen.ch/
pub  4096R/4E20AF1C 2011-05-16            Klaus Ethgen <Klaus@Ethgen.ch>
Fingerprint: 85D4 CA42 952C 949B 1753  62B3 79D0 B06F 4E20 AF1C
[signature.asc (application/pgp-signature, inline)]

Send a report that this bug log contains spam.


Devuan BTS -- Powered by Debian bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson,
2005-2017 Don Armstrong, and many other contributors.

Devuan Bugs Owner <owner@bugs.devuan.org>.
Last modified: Mon Jun 1 19:56:14 2020;