RPM Community Forums

Mailing List Message of <rpm-devel>

Using posix_fallocate to eliminate rpmdb disk extents?

From: Jeff Johnson <n3npq@mac.com>
Date: Sun 21 Sep 2008 - 17:29:14 CEST
Message-id: <D5D8FDB2-CCE4-464B-8D7B-69AD1C3D6AAB@mac.com>
I was asked by a XFS developer to look at the feasibility
of using posix_fallocate(3) for /var/lib/rpm Berkeley DB files.

The motivation is apparently that /var/lib/rpm files have
many extents even after a fresh install using rpm.

E.g. here's the number of disk extents I see:

     $ sudo filefrag /var/lib/rpm/Packages
     /var/lib/rpm/Packages: 1474 extents found, perfection would be 1  

What remains an open question is whether having multiple extents
actually matters. Linus thinks (I'm told) that having disk extents is  
a perfomance
issue. Note "thinks", I'm unable to find any credible measurement
of Berkeley DB performance with (or without) fragmentation.

But its likely feasible to use posix_fallocate(3) with --rebuilddb.  
Note that
using posix_fallocate(3) doesn't necessarily guarantee fewer extents
(although that is likely the case for many file system types,  
extent based file systems), but only reserves disk space.

Berkeley DB (as used by rpm) also uses the mpool cache. Performance
statistics on the mpool cache statistics are available by doing
	cd /var/lib/rpm
	sudo rpmdb_stat -m

When I run this command, I'm seeing cache hit rates of 66% -> 96%,
average ~80%, hinting that extents (and file system and disk layout) of
the backing file system data store perhaps don't matter much at all.

However, that's a guess, I'd rather see an objective measurement.

If someone can demonstrate an improvement in rpmdb performance
with fewer disk extents, I'll attempt to add posix_fallocate(3) to -- 
and run objective benchmarks.

Otherwise, the change hardly seems worth the effort afaict from reading
about Linux fallocate(2) and ext[234] files system fragmentation  
imho. YMMV.

Any other opinions?

73 de Jeff
Received on Sun Sep 21 17:29:47 2008
Driven by Jeff Johnson and the RPM project team.
Hosted by OpenPKG and Ralf S. Engelschall.
Powered by FreeBSD and OpenPKG.