RPM Community Forums

Mailing List Message of <rpm-devel>

Mandatory/enforcing checks for "reproducible builds"

From: Jeff Johnson <n3npq@mac.com>
Date: Sun 28 Sep 2008 - 21:11:51 CEST
Message-id: <CF002901-38C4-425B-BEF5-0FFDE3572316@mac.com>
There are two big lies with rpm packaging methodolgy.

(aside) The other lie is that rpm install transactions are
atomic iff there are no packaging flaws or install host failures.

But this lie is "reproducible builds", which is true wrto rpmbuild  
iff the build
host is set up (including configuration) equivalently/identically.

E.g. if you change gcc on the build host, and one can easily
attempt a rebuild of a package, or achieve a rebuilt package,
that is non-functional. And for hysterical reasons, BuildRequires:
don't (and perhaps cannot) provide a sufficiently strong assertion
check to detect "reproducible builds".

While there is no way to generally detect all possible build failures,
a better job of capturing, say, configuration state can be attempted
that improves "reproducible builds".

Here's a overly simple example:

Assume you have a build that requires
     CFLAGS=-O0
as one of many otherwise Yet To Be Discovered configuration checks.

Running a digest on the set of variables that need to be configured,  
e.g.
     $ echo 'CFLAGS=-O0' | md5sum
     ac95c320c8ff4a1a65fb6c57e4f52c5f  -
provides a strong invariant on build system configuration.

Note that other FOO=bar assignments can be added to extend
the build configuration checking explicitly. One would likely
arrange for additional conventions, like sorting on FOO, or always
adding CDATA white space deterministically, in order to ensure that
permutations or "doesn't matter" cosmetic editting does not invalidate
the digest on the configuration environment. One might also wish to
add versioning, or other attributes, about the representation of the
build environment.

The main reason for reducing the build environment to a digest
is so that it can be added to binary packages, much like
RPMTAG_SOURCEPKGID is currently used to associate
the build elements (but not the build configuration) with
binary packages.

The alternative means, if one doesn't care about the bloatery involved,
is to add the build configuration directly by value to all binary  
packages.

I believe OpenPKG is already adding the set of packages installed on
the build host  to binary packages with lua and arbitrary tags. The only
additional suggestion here is to extend to "build configuration"  
generally,
and add by digest reference, not by value, to binary packages.

I'll work up a proof-of-concept example over the next couple of  
months ...

Opinions?

73 de Jeff
Received on Sun Sep 28 21:11:57 2008
Driven by Jeff Johnson and the RPM project team.
Hosted by OpenPKG and Ralf S. Engelschall.
Powered by FreeBSD and OpenPKG.