On Sun, Jul 29, 2007, Ralf S. Engelschall wrote:

> On Sun, Jul 29, 2007, Ralf S. Engelschall wrote:
>
> > [...]
> > Thanks for all your efforts, Jeff.
> >
> > But unfortunately, although this procedure correctly upgraded db/
> > subdir content, you missed to append the corresponding entries to
> > CVSROOT/history! This way the CVSTrac frontend and the CVS backend are
> > out of sync!! And because you already made two new commits we cannot fix
> > it easily by _NOW_ appending the CVSROOT/history entries (as the two new
> > entries were already taken over into the CVSTrac SQLite RDBMS)...
> >
> > As I said multiple times in the past, whatever you do, you always
> > _HAVE_ to go through the "cvs" command line and _NEVER EVER_ touch
> > the repository manually. Replacing content on the server side gets
> > the backend and frontend totally out of sync and it cannot be fixed
> > afterwards easily.
> >
> > Puhh.... may I suggest that we reverse this again by moving
> > /v/rpm/cvs/db-OLD into place again and _I_ do your above procedure again
> > by using the regular remote CVS access?
>
> Ok, Thomas and I've looked into the problem in more detail. We cannot
> bring the frontend and backends into sync without lots of hacking
> (again, as done in May 2007). So, sorry, we decided to revert the db to
> the db-ORIG dir again to be in sync again. We'll now look what the best
> approach is to repeat your import operation so the stuff does not again
> go out of sync.

Ok, we've reinvestigated. A few points to notice:

- It is clear that your "trac" script didn't worked out-of-the-box
  locally on rpm5.org. The script requires "cvs-cmp", requires Bash
  instead of /bin/sh, has a few nasty things like /usr/tmp hard-coded,
  etc. Even after fixing all of them, the script cannot be used as it
  used things like "cvs co -d /tmp/xxx", i.e., absolute (local) paths
  on checkouts, which CVS 1.12 fails on (even on remote access).

- As already mentioned a few weeks ago, the "db" CVS module
  is a pure mess. It uses a totally bogus vendor branch name named
  "db-3_2_9-vendor" (it contains a particular release information), it
  contains a mixture of imported and regular committed data, etc. There
  is even data which has no vendor branch at all.

- Your approach of manually manipulating the CVS repository by importing
  DB 4.5.20 into a new local CVS and then do the upgrade to DB 4.6.18
  and then copy over the *,v files has destroyed the old history in the
  "db" module and especially brough the CVS backend (CVSROOT/history)
  out-of-sync with the CVStrac frontend (rpm.db SQLite RDBMS).

So, thanks to the db-ORIG we were able to revert to the old status to
again bring CVS and CVSTrac into sync again. Then we uses a copy of the
rpm5.org CVS to come up with the following particular fix and upgrade
procedure:

-----------------------------------------------------------------------
#   operate on a particular repository only
export CVSROOT="rpm-cvs@rpm5.org:/v/rpm/cvs"

#   checkout the old CVS "db" module
rm -rf db-cvs
cvs co -d db-cvs db

#   once-time fixups
(cd db-cvs && cvs admin -n vendor:db-3_2_9-vendor)
(cd db-cvs && find * -type f -print | grep -v CVS | xargs cvs admin -ko)

#   unpack the new DB version and remove dangling "tags" symlinks
rm -rf db-4.6.18
gunzip <db-4.6.18.tar.gz | tar xf -
(cd db-4.6.18 && find . -type l -print | xargs rm -f)

#   import new DB version
(cd db-cvs && cvs tag -F pre-db-4_6_18) || exit $?
(cd db-4.6.18 && cvs import -ko -I '!' -m "import Berkeley-DB 4.6.18 onto vendor branch" db vendor db-4_6_18) || true
(cvs co -jpre-db-4_6_18 -jdb-4_6_18 -d db-cvs-new db) || exit $?
(cd db-cvs-new && cvs ci -m "merge conflicts on HEAD after import of Berkeley-DB 4.6.18") || exit $?

#   determine the remaining changes
(cd db-cvs && cvs -q up -P -d)
diff -ru3 db-4.6.18 db-cvs | egrep -v "^Only in db-cvs.*: CVS" >db.diff

#   remove newly imported docs
( cd db-cvs
  for dir in docs docs_src; do
     find $dir -type f -print | grep -v CVS >x
     xargs <x rm -f; xargs <x cvs rm
  done
  cvs ci -m "remove newly imported documentation files, too" docs docs_src
  cvs up -P -d docs docs_src
) || exit $?
-----------------------------------------------------------------------

Finally, we run this procedure against the original rpm5.org CVS. For
the next DB import, the script to use is mostly similar to the above
one, just a few changes:

-----------------------------------------------------------------------
#   operate on a particular repository only
export CVSROOT="rpm-cvs2@rpm5.org:/v/rpm/cvs2"

#   checkout the old CVS "db" module
rm -rf db-cvs
cvs co -d db-cvs db

#   unpack the new DB version and remove dangling "tags" symlinks
rm -rf db-4.7.XX
gunzip <db-4.7.XX.tar.gz | tar xf -
(cd db-4.7.XX && find . -type l -print | xargs rm -f)

#   import new DB version
(cvs rtag -r vendor pre-db-4_7_XX db) || exit $?
(cd db-4.7.XX && cvs import -ko -I '!' -m "import Berkeley-DB 4.7.XX onto vendor branch" db vendor db-4_7_XX) || true
(cvs co -jpre-db-4_7_XX -jdb-4_7_XX -d db-cvs-new db) || exit $?
(cd db-cvs-new && cvs ci -m "merge conflicts on HEAD after import of Berkeley-DB 4.7.XX") || exit $?

#   determine the remaining changes
(cd db-cvs && cvs -q up -P -d)
diff -ru3 db-4.7.XX db-cvs | egrep -v "^Only in db-cvs.*: CVS" >db.diff

#   remove newly imported docs
( cd db-cvs
  for dir in docs docs_src; do
     find $dir -type f -print | grep -v CVS >x
     xargs <x rm -f; xargs <x cvs rm
  done
  cvs ci -m "remove newly imported documentation files, too" docs docs_src
  cvs up -P -d docs docs_src
) || exit $?
-----------------------------------------------------------------------

You can now hacking wild again...

                                       Ralf S. Engelschall
                                       rse@engelschall.com
                                       www.engelschall.com