Larry McVoy [interview] recently announced a real time CVS mirror of the BitKeeper linux kernel archives [story], offering both the 2.5 development tree and the 2.4 stable tree. There was some confusion over the conversion process, as the end result is around 10% less the number of changesets, this leading to another flamewar. Additionally, now that the kernel trees are being actively exported to a GPL'd format, BitKeeper's file format will be changed to offer significant speed enhancments and other improvements to their customers.

Regarding the difference in changesets, Larry explained that some of BK's complex branching support could not be translated to CVS, and in those cases the data was combined, not dropped. He says, "So we actually captured 100% of the checkin information, both in data files and in the pseudo ChangeSet file, not one byte of that is lost. All we did is collapse all the branches into the longest possible straight line, which is actually for many purposes nicer than the rats nets that you get with BK." To those complaining about the upcoming change in BitKeeper's file format, Theodore Ts'o sarcastically offered, "So to the people are complaining that they won't be able to get out their data if a future version of BK uses a more powerful representation than SCCS files ---- would you like some more whine with your cheese?"

In a side discussion of source control alternatives, Larry commented, "You should really look at Arch, it may be a better fit. And these days, if you could find a better fit, none of us at BitMover would shed a tear if you moved off BK. This has *not* been a pleasant experience for us." Like BitKeeper, Arch [story] has a distributed design, a requirement of the current kernel development process.

Eventually people started actually looking at the exported CVS tree, and a more technical discussion followed. What bugs have been found so far, such as patches being attributed to the wrong contributers, were quickly fixed.

From: Larry McVoy
Subject: [ANNOUNCE] BK->CVS (real time mirror)
Date: Tue, 11 Mar 2003 19:43:30 -0800

We've been working on a gateway between BitKeeper and CVS to provide
the revision history in a form which makes the !BK people happy (or
happier).

We have the first pass of this completed and have a linux 2.5 tree on
kernel.bkbits.net and you can check out the tree as follows (please don't
do this unless you are a programmer and will be using this.  Penguin
Computing provided the hardware and the bandwidth for that machine and
if you all melt down the network they could get annoyed.  By all means
go for it if you actually write code, though, that's why it is there.)

    mkdir ws
    cd ws
    cvs -d:pserver:anonymous@kernel.bkbits.net:/home/cvs co linux-2.5

Each of the releases are tagged, they are of the form v2_5_64 etc.

Linus had said in the past that someone other than us should do this but
as it turns out, to do a reasonable job you need BK source.  So we did it.
What do we mean by a reasonable job?  BitKeeper has an automatic branch
feature which captures all parallel development.  It's cool but a bit
pedantic and it makes exporting to a different system almost impossible
if you try and match what BK does exactly.  So we didn't.  What we
(actually Wayne Scott) did was to write a graph traversal alg which
finds the longest path through the revision history which includes
all tags.  For the 2.5 tree, that is currently 8298 distinct points.
Each of those points has been captured in CVS as a commit.  If we did
our job correctly, each of these commits has the same timestamp across
all files.  So you should be able to get any changeset out of the CVS
tree with the appropriate CVS command based on dates.

We also created a ChangeSet file in the CVS tree.  It has no contents, it
serves as a place to capture the BK changeset comments.  Each file which
is part of a changeset has an extra comment which is of the form

	(Logical change 1.%d)

where the "1.%d" matches the changeset rev.  So you can look for all files
that have (Logical change 1.300) in their comments to reconstruct the 
changeset.  NOTE!  That information is actually redundant, the timestamps
are supposed to do the same thing, let us know if that is not working, we'll
redo it.  I expect we'll find bugs, please be patient, it takes 4 hours of
CPU time on a 2.1Ghz Athlon to do the conversion, that's a big part of 
why this has taken so long.  That's after a week's worth of optimizations.

Each ChangeSet delta has a BK rev associated with it in the comments.
We'll be giving you a small shell script which you can use to send Linus
patches that include the rev and we'll modify BK so that it can take
those patches with no patch rejects if you used that script.

We have a first pass of a real time gateway between BK and this CVS tree 
done.  Right now it is done by hand (by me) but as soon as it is debugged
you will see this tree being updated about 1-3 minutes after Linus pushes
to bkbits.  

Once you guys look this over and decide you like it, we'll do the same
thing for the 2.4 tree.

We're also talking to an unnamed (in case it doesn't work out) Linux
company who may host bkbits.net for us.  If they do that, we'll turn
the GNU patch exporter feature in BKD.  That means that you'll be able
to wget any changeset as a GNU patch, complete with checkin comments.
I'm working with Alan on the format, I think we're close though I have
to run the latest version past him.

If all of this sounds nice, it is.  It was a lot of work for us to do
this and you might be wondering why we bothered.  Well, for a couple of
reasons.  First of all, it was only recently that I realized that because
BK is not free software some people won't run BK to get data out of BK.
It may be dense on my part, but I simply did not anticipate that people
would be that extreme, it never occurred to me.  We did a ton of work to
make sure anyone could get their data out of BK but you do have to run
BK to get the data.  I never thought of people not being willing to run
BK to get at the data.  Second, we have maintained SCCS compatible file
formats so that there would be another way to get the data out of BK.
This has held us back in terms of functionality and performance.  I had
thought there was some value in the SCCS format but recent discussions
on this list have convinced me that without the changeset information
the file format doesn't have much value.

Our goal is to provide the data in a way that you can get at it without
being dependent on us or BK in any way.  As soon as we have this
debugged, I'd like to move the CVS repositories to kernel.org (if I can
get HPA to agree) and then you'll have the revision history and can live
without the fear of the "don't piss Larry off license".  Quite frankly,
we don't like the current situation any better than many of you, so if
this addresses your concerns that will take some pressure off of us.

Another goal is to have the freedom to evolve our file formats to be
better, better performance and more features.  SCCS is holding us back.
So you should look hard at what we are providing and figure out if it
is enough.  If you come back with "well, it's not BitKeeper so it's
not enough" we'll just ignore that.  CVS isn't BitKeeper.  On the
other hand, we believe we have gone as far as is possible to provide
all of the information, checkin comments, data, timestamps, user names,
everything.  The graph traversal alg captures information at an extremely
fine granularity, absolutely as fine is possible.  We have 8298 distinct
points over the 2.5.0 .. 2.5.64 set of changes, so it is 130 times finer
than the official releases.  If you think something is missing, tell us,
we'll try and fix it.

The payoff for you is that you have the data in a format that is not
locked into some tool which could be taken away.  The payoff for us is
that we can evolve our tool as we see fit.  We have that right today,
we can do whatever we want, but it would be anywhere from annoying
to unethical to do so if that meant that you couldn't get at the data
except through BitKeeper.  So the "deal" here is that you get the data
in CVS (and/or patches + comments) and we get to hack the heck out of
the file format.  Our changes are going to move far faster than CSSC or
anyone else could keep up without a lot of effort.  On the other hand,
our changes are going to make cold cache performance be much closer to
hot cache performance, use a lot less disk space, a lot less memory,
and a lot less CPU.

So take a look and tell me what you think.
-- 
---
Larry McVoy            	 lm at bitmover.com           http://www.bitmover.com/lm 



From: Ben Collins
Subject: Re: [ANNOUNCE] BK->CVS (real time mirror)
Date: Tue, 11 Mar 2003 23:16:21 -0500

> The payoff for you is that you have the data in a format that is not
> locked into some tool which could be taken away.  The payoff for us is
> that we can evolve our tool as we see fit.  We have that right today,
> we can do whatever we want, but it would be anywhere from annoying
> to unethical to do so if that meant that you couldn't get at the data
> except through BitKeeper.  So the "deal" here is that you get the data
> in CVS (and/or patches + comments) and we get to hack the heck out of
> the file format.  Our changes are going to move far faster than CSSC or
> anyone else could keep up without a lot of effort.  On the other hand,
> our changes are going to make cold cache performance be much closer to
> hot cache performance, use a lot less disk space, a lot less memory,
> and a lot less CPU.

Larry, I don't mean to start yet another anti-bitmover, anti-bitkeeper or
anti-larry flame-fest, but I have to be honest that I am a little bit
worried.

You are giving us approximately 90% of our data in exchange for the one
thing that made using bitkeeper not a total sellout; the fact that the
revision history of the repo was still accessible without proprietary
software.

I honestly appreciate the work that you and BitMover do for the kernel,
but not giving us access to 100% of _our_ data is unacceptable to me.
Quite honestly, I think your move is to restrict the possible
alternatives to the BK client (the CSSC based ones like I and others had
done), which were able to extract 100% of the data, even if they
couldn't make use of it in the same way as bitkeeper. Atleast it was
there.

You've made quite a marketing move. It's obvious to me, maybe not to
others. By providing this CVS gateway, you make it almost pointless to
work on an alternative client. Also by providing it, you make it easier
to get away with locking the revision history into a proprietary format.

-- 
Debian     - http://www.debian.org/
Linux 1394 - http://www.linux1394.org/
Subversion - http://subversion.tigris.org/
Deqo       - http://www.deqo.com/



From: Jens Axboe
Subject: Re: [ANNOUNCE] BK->CVS (real time mirror)
Date: Wed, 12 Mar 2003 09:55:17 +0100

On Tue, Mar 11 2003, Ben Collins wrote:
> You've made quite a marketing move. It's obvious to me, maybe not to
> others. By providing this CVS gateway, you make it almost pointless to
> work on an alternative client. Also by providing it, you make it easier
> to get away with locking the revision history into a proprietary format.

This is a really good point, deserves high lighting imho...

The BK candy is getting increasingly bitter to swallow here, I may just
have to drop it soon. A shame.

-- 
Jens Axboe



From: Andreas Dilger
Subject: Re: [ANNOUNCE] BK->CVS (real time mirror)
Date: Wed, 12 Mar 2003 03:26:14 -0700

On Mar 12, 2003  09:55 +0100, Jens Axboe wrote:
> On Tue, Mar 11 2003, Ben Collins wrote:
> > You've made quite a marketing move. It's obvious to me, maybe not to
> > others. By providing this CVS gateway, you make it almost pointless to
> > work on an alternative client. Also by providing it, you make it easier
> > to get away with locking the revision history into a proprietary format.
> 
> This is a really good point, deserves high lighting imho...
> 
> The BK candy is getting increasingly bitter to swallow here, I may just
> have to drop it soon. A shame.

Sadly, some people see the dark side of everything.  I don't see how making
a CVS repository available with comments and an as-good-as-you-can-do-with-CVS
equivalent of a BK changeset equals "locking the revision history into a
proprietary format".  Yes, Larry said that this would allow him to change the
BK file format to break compatibility with CSSC, but it is no more "locked
away" now than before for those people who refuse to use BK.

Ironically, SCCS was a former "evil proprietary format" that was reverse
engineered to get CSSC, AFAIK.  People are still free to update CSSC to
track BK if they so choose.

Some people will just never be happy no matter what you give them.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/



From: Jens Axboe
Subject: Re: [ANNOUNCE] BK->CVS (real time mirror)
Date: Wed, 12 Mar 2003 11:31:55 +0100

On Wed, Mar 12 2003, Andreas Dilger wrote:
> Some people will just never be happy no matter what you give them.

I've been very happy with BK, been using it shortly after Linus started
doing so. Mostly out of curiosity at first, later because it was
actually quite useful. I even see myself as a fairly pragmatic
individual, but even so I do find it increasingly difficult to defend my
BK usage.

So please stop thinking you can judge that easily by pushing me into
your nice little 'some people will never be happy bla bla' category.

-- 
Jens Axboe



From: Andreas Dilger
Subject: Re: [ANNOUNCE] BK->CVS (real time mirror)
Date: Wed, 12 Mar 2003 03:56:15 -0700

On Mar 12, 2003  11:31 +0100, Jens Axboe wrote:
> I've been very happy with BK, been using it shortly after Linus started
> doing so. Mostly out of curiosity at first, later because it was
> actually quite useful. I even see myself as a fairly pragmatic
> individual, but even so I do find it increasingly difficult to defend my
> BK usage.

Interesting.  I _had_ lumped you into the "unhappy with BK" camp that has
become so vocal on l-k these days.  My apologies.  I do find it sort of sad
that you (or anyone) actually have to defend your BK usage to others.

I'm personally a "do what you want and let others do what they want as
long as it doesn't interfere with me" kind of person, but it seems that
lots of people here have the opinion that they know what is better for
everyone else, and have no problem telling the list over an over about it.
Probably time to fork a linux-code-repository mailing list and have everyone
spend their time over there instead of rehashing BK flamewars and/or BK
replacement here every week.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

From: H. Peter Anvin
Subject: Re: [ANNOUNCE] BK->CVS (real time mirror)
Date: 12 Mar 2003 08:13:18 -0800

Followup to message
By author:    Andreas Dilger
In newsgroup: linux.dev.kernel
> 
> Sadly, some people see the dark side of everything.  I don't see how making
> a CVS repository available with comments and an as-good-as-you-can-do-with-CVS
> equivalent of a BK changeset equals "locking the revision history into a
> proprietary format".  Yes, Larry said that this would allow him to change the
> BK file format to break compatibility with CSSC, but it is no more "locked
> away" now than before for those people who refuse to use BK.
> 
> Ironically, SCCS was a former "evil proprietary format" that was reverse
> engineered to get CSSC, AFAIK.  People are still free to update CSSC to
> track BK if they so choose.
> 
> Some people will just never be happy no matter what you give them.
> 

From what I can gather, the question is very simple:

"Can we get our data out of BK into some kind of open format?"

It's an important question.  If the answer is "yes, but only the stuff
that can be mapped onto CVS" then that's a significant data loss, and
if BitMover changes the data format without documentation, then there
is no longer a way to get all the data out.

Presumably the CVS exporter could get augmented with some kind of
metadata export... perhaps an XML schema that describes how the
various points are to be linked or whatnot... it won't turn CVS into
BK overnight (so Larry can still sleep at night), but it would give
BitMover the freedom to change their data format.

	-hpa
-- 
"Unix gives you enough rope to shoot yourself in the foot."
Architectures needed: ia64 m68k mips64 ppc ppc64 s390 s390x sh v850 x86-64



From: Dana Lacoste
Subject: Re: [ANNOUNCE] BK->CVS (real time mirror)
Date: 12 Mar 2003 11:30:59 -0500

On Wed, 2003-03-12 at 11:13, H. Peter Anvin wrote:
> "Can we get our data out of BK into some kind of open format?"

> It's an important question.  If the answer is "yes, but only the stuff
> that can be mapped onto CVS" then that's a significant data loss, and
> if BitMover changes the data format without documentation, then there
> is no longer a way to get all the data out.

This sounds like the old GPL argument.

The GPL'd redistributor has to supply the source, they don't have to
supply it in the format that's best for you, being an 80mm tape drive
cuz you're stuck in the punch card age.

Seriously, if CVS loses all that data, is that BK's fault?  BK's so
powerful because it has more information than anyone else, but it's
not their fault (and it's not proprietary data) that no-one else can
deal with the data when it's exported, now is it????

It's not a significant data loss when you try to view a 24bpp image
on an 8bpp display, so it's not a significant data loss that CVS can't
handle the BK.  If it could, Linus would've switched to CVS instead....

I'm not saying Larry's a God or anything, I'm just hoping you guys can
give it up already.  Linus uses BK, nobody else needs to, so move on!

Dana Lacoste
Ottawa, Canada



From: Roman Zippel
Subject: Re: [ANNOUNCE] BK->CVS (real time mirror)
Date: Wed, 12 Mar 2003 18:08:51 +0100 (CET)

Hi,

On 12 Mar 2003, Dana Lacoste wrote:

> Seriously, if CVS loses all that data, is that BK's fault?  BK's so
> powerful because it has more information than anyone else, but it's
> not their fault (and it's not proprietary data) that no-one else can
> deal with the data when it's exported, now is it????

That's not the point. Larry does not own the data in the Linux repository, 
this was one of the conditions for the bk usage, so Larry cannot say, that 
you only get all the data if you use bk. If cvs can't represent all the 
information, we have to find another solution.

bye, Roman



From: Alan Cox
Subject: Re: [ANNOUNCE] BK->CVS (real time mirror)
Date: 12 Mar 2003 21:50:41 +0000

On Wed, 2003-03-12 at 17:08, Roman Zippel wrote:
> this was one of the conditions for the bk usage, so Larry cannot say, that 
> you only get all the data if you use bk. If cvs can't represent all the 
> information, we have to find another solution.

CVS can't represent it all because CVS isnt up to the job. If the rest
exists as comments then its your problem to write a VCS that can extract
the comment data and represent it in full

From: John Bradford
Subject: Re: [ANNOUNCE] BK->CVS (real time mirror)
Date: Wed, 12 Mar 2003 17:57:40 +0000 (GMT)

> > This sounds like the old GPL argument.
> > 
> > The GPL'd redistributor has to supply the source, they don't have to
> > supply it in the format that's best for you, being an 80mm tape drive
> > cuz you're stuck in the punch card age.
> > 
> > Seriously, if CVS loses all that data, is that BK's fault?  BK's so
> > powerful because it has more information than anyone else, but it's
> > not their fault (and it's not proprietary data) that no-one else can
> > deal with the data when it's exported, now is it????
> > 
> > It's not a significant data loss when you try to view a 24bpp image
> > on an 8bpp display, so it's not a significant data loss that CVS can't
> > handle the BK.  If it could, Linus would've switched to CVS instead....
> > 
> 
> You're missing the point completely.
> 
> Of course it's not BK's fault that CVS can't represent the data. 
> However, one of the (valid!) selling points of BK was "we won't hold 
> your data hostage."  That requires that you can export both the data and 
> the metadata into some kind of open format.  Since CVS clearly can't be 
> that open format (CVS being insufficiently powerful), the additional 
> metadata needs to be available in some kind of auxilliary form.  It's 
> then, of course, not BK's fault that CVS can't possibly make use of that 
> auxilliary metadata.

I thought that BK has been able to export everything to a text file
since the first version.

(Ah, but of course, unless that text file is available in EBCDIC, we
still have a problem...)

John.




From: Larry McVoy
Subject: Re: [ANNOUNCE] BK->CVS (real time mirror)
Date: Wed, 12 Mar 2003 10:03:04 -0800

> I thought that BK has been able to export everything to a text file
> since the first version.

bk export -tpatch -r1.900 > patch.1.900
bk changes -v -r1.900 > comments.1.900

Been there forever.  So has ways to get all the metadata from the command
line without having to reverse engineer the file format.  See

    http://www.bitkeeper.com/manpages/bk-prs-1.html

it's all there.  Always has been.

Wayne wanted me to point that it is easy to write the BK to CVS exporter
completely from the command line, we prototyped it that way, the only
reason we rewrote part of it in C was for performance.  The point being
that you guys could have done this yourself without help from us because
all the metadata is right there.  Ditto for anyone else worried about 
getting their data out of BK now or in the future.  The whole point of
prs is to be able to have a will-always-work way to get at the data or
the metadata, it makes the file format a non-issue.  
-- 
---
Larry McVoy              lm at bitmover.com          http://www.bitmover.com/lm
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/



From: Theodore Ts'o
Subject: Re: [ANNOUNCE] BK->CVS (real time mirror)
Date: Thu, 13 Mar 2003 02:59:15 -0500

On Wed, Mar 12, 2003 at 10:03:04AM -0800, Larry McVoy wrote:
> > I thought that BK has been able to export everything to a text file
> > since the first version.
> 
> bk export -tpatch -r1.900 > patch.1.900
> bk changes -v -r1.900 > comments.1.900
> 
> Been there forever.

More importantly, even if someone isn't allowed to use the BK command
line tool because once upon time, a long time ago, they submitted a
patch to arch or subversion, they can still find someone is allowed to
set up a bk daemon under the terms of the FUL, connect to the BK
daemon using a http client, and extract the full diff of any changeset
that way.  This doesn't have to be the bkd on bkbits.net; anyone who
is authorized to use BK under the terms of the FUL can set up a bk
daemon to be listening on a port of any machine for which they have
shell access (it doesn't even require root privs).  And every last
changeset can always be made available using this path.

So to the people are complaining that they won't be able to get out
their data if a future version of BK uses a more powerful
representation than SCCS files ---- would you like some more whine
with your cheese?

						- Ted

From: John Bradford
Subject: Re: [T-Shirt] BK->CVS (real time mirror) (fwd)
Date: Wed, 12 Mar 2003 14:40:48 +0000 (GMT)

Forwarded as suggested :-)

Date: Wed, 12 Mar 2003 06:24:51 -0800
From: Larry McVoy
To: John Bradford
Subject: Re: [T-Shirt] BK->CVS (real time mirror)

You should post this, it's a hoot!  Thanks.

On Wed, Mar 12, 2003 at 11:48:37AM +0000, John Bradford wrote:
> Hi Larry,
> 
> The latest discussions on LKML about the BK->CVS gateway just inspired
> a mental image of a T-Shirt printed thus:
> 
>                           |          |
>                           |          |
>        __________________/            ___________________
>       /                                                   
>      /                                                     
>     /                                                       
>    /      /|                                         |      
>   /       ||                                         ||       
>   |       ||     > > > Hi Larry, BK is evil,         ||       |
>   |       ||     please make a BK->CVS gateway.      ||       |
>   |       ||                                         ||       |
>   |       ||     > > Well...  OK, I guess I can      ||       |
>   |       ||     do that without hurting my          ||       |
>   |       ||     business.                           ||       |
>   |       ||                                         ||       |
>   |       ||     > Larry, this BK->CVS gateway       ||       |
>   |       ||     is pure evil!!!                     ||       |
>   |       ||                                         ||       |
>   |_______||                                         ||_______|
>            |                                         |         
>            |                                         |         
>            |                                         |         
>            |                                         |         
>            |                                         |         
>            |                                         |         
>            |                                         |         
>            |                                         |         
>            |                                         |         
> 
> John.

From: Larry McVoy
Subject: Re: [ANNOUNCE] BK->CVS (real time mirror)
Date: Wed, 12 Mar 2003 09:42:44 -0800

[BK is locking up our data]
[BitMover has to give us our data in an open format]
[The BK pill is oh-so-bitter]
[My tummy hurts and it's Larry's fault]

Boo hoo, cry me a river.

Those of you complaining ought to at least look before you complain.
You just assumed that we were screwing you and you couldn't be bothered
to verify it before you complained.  We didn't screw you at all, all
the data is there.  And BK itself has always had the ability to export
any data in any format, if you read the man pages you might notice that,
but that would be too much work, it's easier to complain.

If you had actually gone and looked at the CVS repository you would have
seen that there is nothing of value missing, in almost 100% of the files,
the full revision history is preserved:

	CVS: 110,076 deltas over all files
	BK:  121,891 deltas over all files

You guys don't have that much parallelism in your files and the exporter
is capturing all that it can which is virtually everything.  It's worth
noting that many deltas in BK are just event recorders, they are just
empty merge delta noise and in fact many people have asked us to get rid
of them.  Once again, it's easier to complain than think.  I'm detecting
a trend.

The graph traversal managed to capture an amazing amount of information,
it's bloody awesome, which you might have noticed if you had looked.
But, nooooo, let's just piss and moan.  What a bunch of friggin' whiners.

The next time you open your mouth, the words that come out of it should be
"thank you".  Nothing else, just that.  If you can't say something nice,
now is a good time to say nothing at all because we are sick and tired of
dealing with people who complain far more than they code.  I'm serious,
we've done way more than anyone could reasonably expect and you react
with no basis in fact, assume bad things that aren't true, don't bother
to look to see if there is a real problem, and don't bother to say thanks.
Aren't you the slightest bit ashamed of your behaviour?  
-- 
---
Larry McVoy            	 lm at bitmover.com           http://www.bitmover.com/lm 



From: Ben Collins
Subject: Re: [ANNOUNCE] BK->CVS (real time mirror)
Date: Wed, 12 Mar 2003 13:34:13 -0500

> 	CVS: 110,076 deltas over all files
> 	BK:  121,891 deltas over all files

(I can recalculate this if you tell me how many of the BK ones are empty
 merge pointers)

90.31%

I wasn't far off by saying 90%. And don't tell me I can get all the
data, when in fact, I can't. Unless of course you give me an explicit
variance from your license, I pay for a license, or I get someone else
with BK to get me the data.

Larry, I am not trying to knock your efforts for the kernel. I am going
on record as saying "thank you, Larry". Linus has been much more
productive since using BK. The kernel patch quality and productivity of
the core kernel developers has increased. A new paradigm in source
control has come about.

But being a person who also has certain beliefs, I am not going to stand
on the side lines and watch the fight. Please don't drop me into the
pool of people who believe all source should be free. I work for
companies that retain some of their IP for good reason. I have signed
NDA's to get at source and work on things that I cannot give out for
free. I'm ok with that choice for a company. What I am not ok with, is
seeing something that I work with everyday slowly becoming engulfed in
gray area.

The kernel's revision history is always available. I get the cset
emails. I can extract all the info I want manually.

The problem I have is that you are going to make it so that the original
files that hold this data cannot be extracted in any meaningful way
without your tools. So if bitkeeper suddenly could not be used by Linus
or any others, for whatever reason, we are locked out of that original
dataset.

-- 
Debian     - http://www.debian.org/
Linux 1394 - http://www.linux1394.org/
Subversion - http://subversion.tigris.org/
Deqo       - http://www.deqo.com/

From: Nicolas Pitre
Subject: Re: [ANNOUNCE] BK->CVS (real time mirror)
Date: Wed, 12 Mar 2003 14:21:53 -0500 (EST)

On Wed, 12 Mar 2003, Larry McVoy wrote:

> [BK is locking up our data]
> [BitMover has to give us our data in an open format]
> [The BK pill is oh-so-bitter]
> [My tummy hurts and it's Larry's fault]
> 
> Boo hoo, cry me a river.
> 
> Those of you complaining ought to at least look before you complain.
> You just assumed that we were screwing you and you couldn't be bothered
> to verify it before you complained.  We didn't screw you at all, all
> the data is there.

Larry, please don't fall into that trap.

There will _always_ be people to whine at you.  For _ever_, and even longer 
if your business remains healthy.

There was a conflict issue before since one was forced into using BK in
order to have real time access to the very latest changeset in the reference
development kernel repository.  Now that this issue has been resolved there
is rationally nothing else to complain about.  Those who want to/can use BK
just do it, those who don't want to/can't use BK aren't penalized anymore
with regard to access to the very data they care about.

Oh then why are they still whining?  Because they are humans with their own
ego, pride and beliefs.  Maybe they are upset because you just removed the
best argument they had against BK up to now, maybe they have a hard time
convincing themselves that BK is superior because you managed to create a
favorable climate for its rapid development, maybe they are religious
extremists (read Free Software extremists) always about to go on a crusade
for the only True Way of living, etc.  There's nothing you can do to help
those irrational issues besides just ignoring them.  Linus himself just
mentioned recently that he's getting more and more effective at ignoring BSD
extremists for his Linux licensing choice.  It's about time you do the same.

So please Larry don't let your feelings be trapped into critics that will 
_never_ end.  There will still be people to disagree with your licensing 
decisions, to hate your business model, and to simply hate you.  But if 
_you_ know that you've made everything to make this part of the world a 
better place then it should be enough to make you feel good.  Seeking for 
public rewards is mostly impossible as long as you're alive.

> The next time you open your mouth, the words that come out of it should be
> "thank you".  Nothing else, just that.  

Larry, I say "Thank you".

> If you can't say something nice, now is a good time to say nothing at all
> because we are sick and tired of dealing with people who complain far more
> than they code.

Then why do you let them turn you down?  Why are those people so credible to 
you so you feel you must listen to them?

> I'm serious, we've done way more than anyone could reasonably expect and
> you react with no basis in fact, assume bad things that aren't true, don't
> bother to look to see if there is a real problem, and don't bother to say
> thanks.

If you expect that aspect of human beings to change frankly you are 
dreaming.  I'm sorry to make it sound bad but that's reality.  There will 
_always_ be people to disagree and whine at something, and statistically 
that something will be BK from time to time.  Those who are happy with the 
system usually just say nothing unfortunately and they go back to coding 
right away.

Nicolas



From: Larry McVoy
Subject: Re: [ANNOUNCE] BK->CVS (real time mirror)
Date: Wed, 12 Mar 2003 11:51:20 -0800

> > Boo hoo, cry me a river.
> 
> Larry, please don't fall into that trap.

I'm not, I was just blowing off steam.  Ya gotta admit it is pretty 
unreasonable for people to complain without even checking.  I'm tailing
the history log in the CVS repository, there have been 3 or 4 downloads
since last night, that's it.  All that whining and no looking at all.

> > If you can't say something nice, now is a good time to say nothing at all
> > because we are sick and tired of dealing with people who complain far more
> > than they code.
> 
> Then why do you let them turn you down?  Why are those people so credible to 
> you so you feel you must listen to them?

Well, I agree that they should be able to get at the information in a 
neutral way.  I guess it was unrealistic, but I was expecting that people
would go download the CVS tree and poke around and see if it is what they
wanted.  We could have had a nice technical discussion about what was 
missing, if anything.  If the discussion had happened, they would have 
found out that even for the missing deltas we captured the information.

Here's an example.  Suppose the graph is like

	1.1 (torvalds) -> 1.2 (alan) -> 1.3 (sct) -> 1.4 (torvalds) 
	                                          /
		      -> 1.1.1.1 (davej) --------/

and we picked the straight 1.1 to 1.4 path.  When we created the CVS 1.4
delta, we knew that it was a merge delta and we needed to capture the 
data off on the branch.  We already capture the contents, the missing part
is what davej may have typed in as comments.  We capture that as well, it
looks like this:

    revision 1.342
    date: 2003/03/07 15:39:16;  author: torvalds;  state: Exp;  lines: +7 -1
    [PATCH] kbuild: Smart notation for non-verbose output

    2003/03/05 19:50:27-06:00 kai
    kbuild: Make build stop on vmlinux link error

    (Logical change 1.8166)

That particular example is from the top level Makefile, Linus merged
in Kai's work and we added the "kbuild: Make build stop on vmlinux link
error" comments from the merged in delta.  If there were more than one
delta, they get merged as well, so the rlog output is completely accurate.

So we actually captured 100% of the checkin information, both in data
files and in the pseudo ChangeSet file, not one byte of that is lost.
All we did is collapse all the branches into the longest possible straight
line, which is actually for many purposes nicer than the rats nets that
you get with BK.

Anyway, to get back to your question, what gets me down is that we did
what we believe to be the absolute perfect job.  All the data is captured,
all the checkin comments are captured, we made all the dates go forward
properly so that diffs would work, there is nothing wrong with the CVS
tree, it's perfect.  It would have been nice if people had actually
looked at it.  You can, go look at

    http://linux.bkbits.net:8080/linux-2.5/hist/Makefile

and compare it to this:

    cvs -d:pserver:anonymous@kernel.bkbits.net:/home/cvs rlog linux-2.5/Makefile

Poke around, play with your favorite files, you'll see your checkin
comments, we didn't lose anything at all.  Apparently, that's too much
to ask, and that's what gets me down.  I don't expect people to say
"rah rah, you guys are great" but I did expect the people who have been
bitching non-stop that they can't get what they want would at least go
see if they could get it now.  Silence would be a more than adequate
reward as far as I'm concerned, I don't need the strokes but I am sick
of the baseless whining.  Fear not, I'll get over it.

What I expected from Ben was a polite request for a tarball of the CVS
tree so he could go convert it to SVN and see if all his checkin comments
are there.  No such request, polite or otherwise, has happened, from him
or anyone else.  So it's becoming apparent that the whole data/metadata
whatever is a red herring, they just want to flame.  Whatever, somebody
will get some good use out of the CVS trees, if I were in your shoes
I'd want them as a safety net so it's cool they exist.  
-- 
---
Larry McVoy              lm at bitmover.com          http://www.bitmover.com/lm



From: Ben Collins
Subject: Re: [ANNOUNCE] BK->CVS (real time mirror)
Date: Wed, 12 Mar 2003 15:08:49 -0500

> So we actually captured 100% of the checkin information, both in data
> files and in the pseudo ChangeSet file, not one byte of that is lost.
> All we did is collapse all the branches into the longest possible straight
> line, which is actually for many purposes nicer than the rats nets that
> you get with BK.

Now that wasn't apparent from your original post. You made it sound more
like meta-data was missing (we all know that via the merges, as long as
you picked a line, none of the diffs were missing).

That's good to know, and makes my whole rant kind of pointless now.

I still don't like your move to change the SCCS format. Regardless of
your intentions, it makes my gut hurt. That doesn't mean I don't like
you or what you do. I never said I didn't in this discussion, although
some people decided to get rude on my behalf, when they shouldn't which
triggered others to target people in the discussion with rudeness, which
shouldn't have happened.

I can dislike the change, just the same as you can license your product
and develop it any way you want. The day someone takes that right away
from either of us is when we have a real problem.

-- 
Debian     - http://www.debian.org/
Linux 1394 - http://www.linux1394.org/
Subversion - http://subversion.tigris.org/
Deqo       - http://www.deqo.com/

From: Arador
Subject: Re: [ANNOUNCE] BK->CVS (real time mirror)
Date: Wed, 12 Mar 2003 19:38:06 +0100

On Tue, 11 Mar 2003 23:16:21 -0500
Ben Collins wrote:

> You've made quite a marketing move. It's obvious to me, maybe not to
> others. By providing this CVS gateway, you make it almost pointless to
> work on an alternative client. Also by providing it, you make it easier

I don't think so. This also bits Larry. If he does well enought, there'll be
some people here that won't use bitkeeper just because they can use the cvs
gateway and they don't need/miss the features they could get with bk.

And i don't think it avoids creating a free bk clone. I guess that there's
a lot of people out there interested in such tool, and not only for kernel
development; this won't stop them.

As far as i can see; Larry is just wasting time (money) to help the kernel
development and people who doesn't use BK just because it isn't free. And
he's not charging me, so i find this a good movement for everybody. I only
can say thanks.

Diego Calleja



From: Ben Collins
Subject: Re: [ANNOUNCE] BK->CVS (real time mirror)
Date: Wed, 12 Mar 2003 13:47:10 -0500

On Wed, Mar 12, 2003 at 07:38:06PM +0100, Arador wrote:
> On Tue, 11 Mar 2003 23:16:21 -0500
> Ben Collins wrote:
> 
> > You've made quite a marketing move. It's obvious to me, maybe not to
> > others. By providing this CVS gateway, you make it almost pointless to
> > work on an alternative client. Also by providing it, you make it easier
> 
> I don't think so. This also bits Larry. If he does well enought, there'll be
> some people here that won't use bitkeeper just because they can use the cvs
> gateway and they don't need/miss the features they could get with bk.
> 
> And i don't think it avoids creating a free bk clone. I guess that there's
> a lot of people out there interested in such tool, and not only for kernel
> development; this won't stop them.
> 
> As far as i can see; Larry is just wasting time (money) to help the kernel
> development and people who doesn't use BK just because it isn't free. And
> he's not charging me, so i find this a good movement for everybody. I only
> can say thanks.

You're missing the point. I am not against the CVS->BK gateway. I'm all
for it. But it's kind of sour given that he now wants to change the disk
format of the repo to make it harder to get the data from it.

If all he announced was "you now have a CVS->BK repo", I wouldn't be
complaining, I'd be patting him on the back.

-- 
Debian     - http://www.debian.org/
Linux 1394 - http://www.linux1394.org/
Subversion - http://subversion.tigris.org/
Deqo       - http://www.deqo.com/



From: Andreas Dilger
Subject: Re: [ANNOUNCE] BK->CVS (real time mirror)
Date: Wed, 12 Mar 2003 12:12:09 -0700

On Mar 12, 2003  13:47 -0500, Ben Collins wrote:
> You're missing the point. I am not against the CVS->BK gateway. I'm all
> for it. But it's kind of sour given that he now wants to change the disk
> format of the repo to make it harder to get the data from it.

Actually, that is purely YOU reading something into what he wrote.  He
didn't say "now I'm going to make the repo harder to get data from it".
What he said was "now I'm free to change the format from SCCS to something
that is more efficient for BK to use".  Who knows, maybe the new format
will be _easier_ to reverse engineer/parse using 3rd party tools?

Also, it's not like he can change things overnight, because there are lots
of customers/users who have repos in the old SCCS format, and he doesn't
want to completely throw away his current code just to piss off some whiny
l-k users.  At worst, if it bothers you so much, you can take up the now
seemingly forgotten Linux trait of "taking things into your own hands and
fixing it to your own needs" and write bk_evil_format_2_CVS conversion tool
instead of bitching on l-k about it.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/



From: Martin J. Bligh
Subject: Re: [ANNOUNCE] BK->CVS (real time mirror)
Date: Wed, 12 Mar 2003 16:29:26 -0800

> You're missing the point. I am not against the CVS->BK gateway. I'm all
> for it. But it's kind of sour given that he now wants to change the disk
> format of the repo to make it harder to get the data from it.
> 
> If all he announced was "you now have a CVS->BK repo", I wouldn't be
> complaining, I'd be patting him on the back.

As long as we continue to get all the data in an open format, I'm
not sure this really matters, personally. If there's some data loss,
let's focus on that issue ... but it seems there isn't at the moment.

I'd rather we *didn't* go trying to clone BK and make it file-format 
compatible underneath ... that seems more incendiary than useful. 
Cloning other products is always a loosing game, the best you can do 
is catch them. Personally, I'd prefer we spent the effort making a 
usable simple SCM that 95% of us can use that does merges and stuff, 
and not bother trying to follow someone else in file format.

Of course, I'm in no position to dictate to others what they should
implement, do what you like ... just my personal opinion. But there's
always the possiblity we can make something that fits kernel development
*better*, rather than playing catchup to BK all the time ;-)

M.



From: Larry McVoy
Subject: Re: [ANNOUNCE] BK->CVS (real time mirror)
Date: Wed, 12 Mar 2003 16:56:27 -0800

> Of course, I'm in no position to dictate to others what they should
> implement, do what you like ... just my personal opinion. But there's
> always the possiblity we can make something that fits kernel development
> *better*, rather than playing catchup to BK all the time ;-)

I like it when I agree with people, especially you since we've bumped
heads.  It's much more fun to agree...

My personal opinion is that BK maps only so so well onto the kernel
development effort.  It's not horrible, it's closer than any other SCM,
but it could be better.  The kernel guys tend to be "more loose" than
commercial guys, i.e., stuff is tried, it sits in Alan's tree for a
while or DaveJ's tree and then is rejected if it is found to be bad.
You really need a sort of "lossy" SCM system, one which is willing to
throw data away.  BK is absolutely not about losing information, we view
everything as valuable, even bad ideas.  That matches the commercial
world better than the Linux world.

I _think_ that Arch is closer.  You will definitely give up some stuff
if you move to Arch but you will also gain some stuff.  Arch is willing
to pick and choose, we aren't, we're sort of an all or nothing answer.
Pavel is all hot and bothered about PRCS but PRCS is sort of BK without
the distribution, gui tools, and scripting.  It's a step backwards as
far as I can tell (don't get me wrong, we've acknowledged the coolness
of PRCS on our website for years and I tried to team up with Josh, I'm
a fan).  You should really look at Arch, it may be a better fit.  And 
these days, if you could find a better fit, none of us at BitMover 
would shed a tear if you moved off BK.  This has *not* been a pleasant
experience for us.
-- 
---
Larry McVoy              lm at bitmover.com          http://www.bitmover.com/lm

• Online Archive Of Above Thread (There's much more to it...)