RPM Community Forums

Mailing List Message of <rpm-users>

Re: About the consistency of /var/lib/rpm/__db.00?

From: Marc MERLIN <marc_rpm@merlins.org>
Date: Sat 22 May 2010 - 04:52:23 CEST
Message-ID: <20100522025223.GH6064@merlins.org>
On Fri, May 21, 2010 at 08:38:23PM -0400, Jeff Johnson wrote:
> > So running most rpm versions from this century :) creates
> > /var/lib/rpm/__db.00? when accessing the rpm dbs.
> > 
> > My understanding is that they are a temporary index created by the DB
> > libraries and I know they get recreated as needed.
> > 
> 
> Nope. There are (at least) 3 RPM user visible uses of __db* files, none of them
> "temporary" or "index":
> 	1) a version stamp
> 	2) names of locks (and with thread_count, "stale lock" registration
> 	for clean-up after exceptional conditions like reboot and segfault
> 	and programmer error). Note "names", not the locks themselves.
> 	3) data caching
 
Ok.

> > 1) can I configure rpm and/or the DB lib not to create those files?
> >   (they cause issues when we look at a 32bit image on a 64bit system
> >   for instance since rpm64 will try to read them and fail with a weird
> >   error message, also they are bloat on the image syncing process that
> >   I'd rather do without)
> > 
> 
> Remove the files when moving from 32 <-> 64 bit. This is/was done automagically
> several years now for the DB_INIT_CDB model. While the error message
> may be "weird", it is in fact the error message that indicates that
> you need to remove the __db* files.
 
Right, I figured that out :)

> > 2) if I can't stop them from being created, would I be able to look for
> >   some exit code path in the rpm source and just delete them there?
> 
> You don't want to delete the files because that _IS_ where the locks
> are found. Removing the files introduces races (but is perfectly okay
> if you are _SURE_ that nothing else is using an rpmdb at that moment,
> which is often a pretty safe assumption).

I don't have locking issues, but noted.
 
> > 3) what if a system has an rpmdb with those files and I revert the rpm
> >   db files (excluding those specific __db.00? files from the sync) so
> >   that the rpm db is now different form what it was when they were
> >   created. 
> >   Does that create an inconsistency that could severly confuse RPM?
> 
> What happens if you do
> 	cp /dev/zero /dev/mem
> ;-)
 
Not so good :)

> What is "the sync"? Are you using rsync to copy between machines?

something like that.

> Don't copy __db* files, they are useless on any other machine,
> can only be used on the local machine. (Well one _CAN_ have
> concurrent access through NFS+fcntl from multiple sufficiently
> similar machines, but let's not go there ...)

I know, I don't copy them, but I copy the rest of the rpmdb and all the
files that rpm knows about.
In other words, rpm gets run on master server to make an image and the
image, including the rpmdb gets copied to machines.

> There are failure modes if you screw around, typically DB_PAGE_NOTFOUND
> when a page has gone AWOL in the cache. But rpm needs very little
> from an rpmdb and deliberately has a KISS schema so that --rebuilddb
> regenerates all the secondary indices, any time you want, and
> the primary store of headers is both digest/signature checked, so
> "inconsistency" is largely limited to dropping a registered package
> that fails signature/digest/sanity checks during --rebuilddb.
 
Right. So it sounds like
1) __db* files can't be turned off since for most users they are useful (not
   for us though). 
2) I could hack rpm to find all the exit paths or wrap it and auto delete
   the db files on exit
3) if I have an rpmdb synced over a machine that has non matching __db*
   files, nothing really bad will happen, but rpm may act weird and it's
   just better not to get in that situation, potentially via #2 or other
   means.

> Again, rpm-5.3 has full transactional protection and ACID properties,
> so none of the above comments apply to rpm-5.3.

Right. We need to weigh the new size and library bloat vs features 
we'd actually benefit from :)

On Fri, May 21, 2010 at 08:42:36PM -0400, Jeff Johnson wrote:
> Database backups are a bit trickier than just blasting with rsync because
> all the files need to be consistent.
 
They are.

> Likely the best method is to do
> 	rsync /var/lib/rpm/package
> and then do
> 	rpm --rebuilddb
> 
> One can rebuild into a different directory and check if paranoid;
> but that's largely what --rebuilddb is already doing.

Right. In our case though because we push a fully consistent image, we don't
need to use --rebuilddb, but it's there should something really bad happen :)

Thanks for the answers,
Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems & security ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  
Received on Sat May 22 04:52:48 2010
Driven by Jeff Johnson and the RPM project team.
Hosted by OpenPKG and Ralf S. Engelschall.
Powered by FreeBSD and OpenPKG.