Subject: bug#748: nfs-ganesha: ganesha has a memory leak

From: Bodo Eggert <[email-masked]>
Date: Wed, 08 Mar 2023 01:10:40 +0100

Package: nfs-ganesha
Version: 3.4-1
Severity: grave
Justification: renders package unusable

Dear Maintainer,

the shipped version of ganesha, version 3.4, does have a memory leak that
causes it to consume all available RAM and more.  I have 4 GB installed, but
yesterday ganesha did use about 6 GB and each stat() required swapping.

I did research about this some time ago, the recommended fix was to upgrade
to 4.x.  Today I tried to find that information again so I could link it but
I couldn't.  Still I think that it needs an upgrade as the proper fix, there
are a lot of memory leak threads on the maintainers' github.

In case the upgrade is not possible: (Otherwise ignore this please)

Currently I'm mitigating this with cgrpups and daily restarting the service. 
This (below) is the configuration that I'm trying since yesterday; I added
memory.soft_limit_in_bytes.  memory.limit_in_bytes alone did not help. 
Today the process did behave as expected.

$ cat /etc/boot.d/00cgconfig
#!/bin/sh
/usr/sbin/cgconfigparser -l /etc/cgconfig.conf
/usr/sbin/cgrulesengd
exit 0

/etc/cgconfig.conf:group daemon/nfsd {
/etc/cgconfig.conf:  cpu {
/etc/cgconfig.conf:    cpu.shares = 500;
/etc/cgconfig.conf:  }
/etc/cgconfig.conf:  memory {
/etc/cgconfig.conf:    memory.limit_in_bytes = 1000000000;
/etc/cgconfig.conf:    memory.soft_limit_in_bytes = 1000000000;
/etc/cgconfig.conf:  }
/etc/cgconfig.conf:}

/etc/cgrules.conf:*:/usr/bin/ganesha.nfsd       memory  daemon/nfsd

-- System Information:
Distributor ID:	Devuan
Description:	Devuan GNU/Linux 4 (chimaera)
Release:	4
Codename:	chimaera
Architecture: x86_64

Kernel: Linux 5.10.0-18-amd64 (SMP w/2 CPU threads)
Locale: LANG=C, LC_CTYPE=C.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /bin/dash
Init: sysvinit (via /sbin/init)
LSM: AppArmor: enabled

Versions of packages nfs-ganesha depends on:
ii  dbus              1.12.24-0+deb11u1devuan1
ii  libacl1           2.2.53-10
ii  libblkid1         2.36.1-8+devuan2
ii  libc6             2.31-13+deb11u5
ii  libcap2           1:2.44-1
ii  libcom-err2       1.46.2-2
ii  libdbus-1-3       1.12.24-0+deb11u1devuan1
ii  libgssapi-krb5-2  1.18.3-6+deb11u3
ii  libkrb5-3         1.18.3-6+deb11u3
ii  libnfsidmap2      0.25-6
ii  libntirpc3.4      3.4-2
ii  librados2         14.2.21-1
ii  liburcu6          0.12.2-1
ii  libuuid1          2.36.1-8+devuan2
ii  libwbclient0      2:4.13.13+dfsg-1~deb11u5
ii  nfs-common        1:1.3.4-6
ii  rpcbind           1.2.5-9

nfs-ganesha recommends no packages.

nfs-ganesha suggests no packages.

-- Configuration Files:
/etc/ganesha/ganesha.conf changed:
NFS_CORE_PARAM {
	## Allow NFSv3 to mount paths with the Pseudo path, the same as NFSv4,
	## instead of using the physical paths.
	#mount_path_pseudo = true;
	## Configure the protocols that Ganesha will listen for.  This is a hard
	## limit, as this list determines which sockets are opened.  This list
	## can be restricted per export, but cannot be expanded.
	Protocols = 3,4,9P;
}
EXPORT_DEFAULTS {
	## Access type for clients.  Default is None, so some access must be
	## given either here or in the export itself.
	#Access_Type = RW;
	Protocols = 3,4,9P;
	#FSAL {
	#	Name = VFS;
	#}
}
MDCACHE {
	## The point at which object cache entries will start being reused.
	#Entries_HWMark = 100000;
	Entries_HWMark = 50000;
}
EXPORT
{
	## Export Id (mandatory, each EXPORT must have a unique Export_Id)
	Export_Id = 1;
	## Exported path (mandatory)
	Path = /export;
	## Pseudo Path (required for NFSv4 or if mount_path_pseudo = true)
	Pseudo = /;
	## Restrict the protocols that may use this export.  This cannot allow
	## access that is denied in NFS_CORE_PARAM.
	Protocols = 3,4,9P;
	## Access type for clients.  Default is None, so some access must be
	## given. It can be here, in the EXPORT_DEFAULTS, or in a CLIENT block
	## Whether to squash various users.
	## Allowed security types for this export
	#Sectype = sys,krb5,krb5i,krb5p;
        FSAL {
                Name = VFS;
        }
	CLIENT
	{
		Clients = 192.168.redacted.0/24;
		Squash = None;
		Access_Type = RW;
		Sectype = sys;
	}
}
	## Default log level for all components
	#Default_Log_Level = WARN;
	## Configure per-component log levels.
	#Components {
		#FSAL = INFO;
		#NFS4 = EVENT;
	#}
	## Where to log
	#Facility {
		#name = FILE;
		#destination = "/var/log/ganesha.log";
		#enable = active;
	#}

-- no debconf information

---

From: Mark Hindley <[email-masked]>
Date: Wed, 8 Mar 2023 08:57:49 +0000

Control: tags -1 debian
Control: fixed -1 4.0.12-3~bpo11+1

Bodo,

> yesterday ganesha did use about 6 GB and each stat() required swapping.

This is not a forked package in Devuan and we use Debian's packages directly
without recompilation. Please report this to Debian's BTS to be addressed.

If you think 4.x series will fix it, you might be interested in the stable-bpo
version 4.0.12-3~bpo11+1.

Mark