David Teigland posted some updated patches for implementing a distributed lock manager. In an earlier thread David explained, "many distributed/cluster applications use a dlm for inter-process synchronization where processes may live on different machines. (GFS and CLVM are two examples which our group also develop; GFS using the dlm from the kernel and CLVM from userspace.)"

Andrew Morton [interview] looked at the submission and replied, "not only do I not know whether this stuff should be merged: I don't even know how to find that out. Unless I'm prepared to become a full-on cluster/dlm person, which isn't looking likely." He agreed to merge the patches into his -mm tree [story] for testing, but added cautiously, "that wouldn't signify anything further." In response to some of Andrew's questions, David provided an overview of existing distributed locking implementations and uses.

From: David Teigland [email blocked]
To:  linux-kernel
Subject: [PATCH 0/8] dlm: overview
Date:	Mon, 16 May 2005 15:19:49 +0800

These are the distributed lock manager (dlm) patches against 2.6.12-rc4
that we'd like to see added to the kernel.  We've made changes based on
the suggestions that were sent, and are happy to get more, of course.

The original overview
http://marc.theaimsgroup.com/?l=linux-kernel&m=111444188703106&w=2

For those interested in performance, there is some information here
http://sources.redhat.com/cluster/dlm/

Dave



From: Andrew Morton [email blocked]
Subject: Re: [PATCH 0/8] dlm: overview
Date:	Tue, 17 May 2005 00:11:33 -0700

David Teigland [email blocked] wrote:
>
>  These are the distributed lock manager (dlm) patches against 2.6.12-rc4
>  that we'd like to see added to the kernel.

Squawk.

Not only do I not know whether this stuff should be merged: I don't even
know how to find that out.  Unless I'm prepared to become a full-on
cluster/dlm person, which isn't looking likely.

The usual fallback is to identify all the stakeholders and get them to say
"yes Andrew, this code is cool and we can use it", but I don't think the
clustering teams have sufficent act-togetherness to be able to do that.

Am I correct in believing that this DLM is designed to be used by multiple
clustering products?  If so, which ones, and how far along are they?

Looking at Ted's latest Kernel Summit agenda I see

 Clustering

 	We need to make progress on the kernel integration of things
 	like message passing, membership, DLM etc.

 	We seem to have at least two comparable kernel-side offerings
 	(OpenSSI and RHAT's), as well as a need to hash out how user-space
 	plays into this.

 	(There is now a plan for a Clustering Summit prior to KS - need
 	to validate if this will be useful, still)

So right now I'm inclined to duck any decision making and see what happens
in July.  Does that sounds sane?  Is that Clustering Summit going to happen?

In the meanwhile I can pop this code into -mm so it gets a bit more
compile testing, review and exposure, but that wouldn't signify anything
further.



From: David Teigland [email blocked]
Subject: Re: [PATCH 0/8] dlm: overview
Date:	Tue, 17 May 2005 17:57:04 +0800

On Tue, May 17, 2005 at 12:11:33AM -0700, Andrew Morton wrote:

> The usual fallback is to identify all the stakeholders and get them to say
> "yes Andrew, this code is cool and we can use it", but I don't think the
> clustering teams have sufficent act-togetherness to be able to do that.
> 
> Am I correct in believing that this DLM is designed to be used by multiple
> clustering products?  If so, which ones, and how far along are they?

Correct.  Red Hat has multiple clustering products that do and will use
this.  GFS and CLVM are two notable ones that do now.  GFS kernel patches
that use this will be sent in the near future; CLVM uses the dlm from user
space.

Here are my impressions of where other clustering groups are at:

OpenSSI: this project is interested in integrating an existing dlm into
their system.  They don't have any effort under way to develop a dlm
themselves.  They are also interested in using gfs which is indirectly an
interest in the dlm.

Linux-HA: seem to be in a similar situation as OpenSSI

Lustre: this project includes a locking system called "dlm".  The api is
similar, but the implementation is not distributed in the style of a
traditional dlm; what they have is largely Lustre-specific [1].

OCFS2: this project includes a dlm intended for OCFS2, but there have been
some steps to make it more generic.  I believe their goal is still to
develop their dlm primarily for ocfs2's needs, not to develop an entirely
general purpose dlm such as this one.  So again, what's there is limited
and largely OCFS2-specific [1].

It would be very good to hear from any others who are interested in using
a dlm, too.

[1] Application-specific lock managers such as Lustre's and OCFS2's can be
good ideas, and I'm not criticising them.  It allows you to make
specializations and simplifications to suit your particular needs.  So,
while I'm sure it's possible for them to use this general-purpose dlm,
some may still want to do their own specialized thing.  We'll try to add
features and options to the general-purpose dlm to meet specialized needs,
but there's a limit to that.


> Looking at Ted's latest Kernel Summit agenda I see
> 
>  Clustering
> 
>  	We need to make progress on the kernel integration of things
>  	like message passing, membership, DLM etc.
> 
>  	We seem to have at least two comparable kernel-side offerings
>  	(OpenSSI and RHAT's), as well as a need to hash out how user-space
>  	plays into this.
> 
>  	(There is now a plan for a Clustering Summit prior to KS - need
>  	to validate if this will be useful, still)
> 
> So right now I'm inclined to duck any decision making and see what happens
> in July.  Does that sounds sane?  Is that Clustering Summit going to happen?

To some extent I'm sure the different clustering meetings will happen,
although I won't be at any of them.  I'm forging ahead trying to work
things out rather than waiting.  (Frankly, I don't think waiting for a
cluster summit will matter much.)

It's worth noting that most of the clustering discussions are now about
user space stuff.  Since the dlm is agnostic about what's in user space,
the dlm discussion and other clustering topics are largely independent.


> In the meanwhile I can pop this code into -mm so it gets a bit more
> compile testing, review and exposure, but that wouldn't signify anything
> further.

Sure, more feedback is what we're after.

Dave



From: Andi Kleen [email blocked]
Cc: linux-kernel, linux-cluster
Subject: Re: [PATCH 0/8] dlm: overview
Date:	Tue, 17 May 2005 15:10:46 +0200

Andrew Morton [email blocked] writes:
>
> Squawk.
>
> Not only do I not know whether this stuff should be merged: I don't even
> know how to find that out.  Unless I'm prepared to become a full-on
> cluster/dlm person, which isn't looking likely.
>
> The usual fallback is to identify all the stakeholders and get them to say
> "yes Andrew, this code is cool and we can use it", but I don't think the
> clustering teams have sufficent act-togetherness to be able to do that.

My impression is that it is unlikely everybody will agree on a single
cluster setup anyways, so it might be best to use a similar strategy
as with file systems ("multiple implementations - standard API to the
outside world")

This would mean the DLM could be merged if the other cluster people
agree that the interface it presents to the outside world is good
for them too. Is it? Perhaps these interfaces should be discussed
first.

-Andi

• Archive of above thread
• KernelTrap interview with Andrew Morton