Hey guys,

After some discussions with some other developers, I think it might be
a good idea to explore how the OACS community can improve its approach
to collaborative development.

Right now, I think the OACS community is head and shoulders above any
other free software project that I'm aware of. I'm on the mailing list
of quite a few projects simply to keep abreast of other systems, and I
have yet to come across one with a community that is as knowledgeable
and helpful as OACS. We've got some top of the line hackers here and
we've got a lot fewer arrogant curmudgeons than most communities have
(that's a compliment, btw :)).

That being said, some have pointed out that a lot of development goes
on in private. That means that a lot of the important decisions
regarding infrastructure and architecture are not done terribly
transparently. I am as guilty as anyone else in this regard and have
chosen this approach for many projects, for instance some email
dialogues I've had about the admin UI and others regarding ETP.
However, I think that I made a mistake in these times.

Since we're about to release beta, and many of these discussions
probably took place in private in order to expedite the porting
process, we should seriosly consider making a community wide effort to
make development conversations and decisions more transparent for the
next release or generation of OACS.

I don't think that I'm entirely the right person to propose ways to
facilitate this transparency, but I would like to pose this commitment
to the community. But I'll also make it a policy at Musea to try and
discuss general development approaches more public, especially with
regards to ETP.

Any thoughts or comments would be appreciated.

talli

I think Talli makes an excellent point. Transparency is difficult to
accomplish but is fairly important in the long run.

In the past few weeks, I've received a few murmurs of
disapproval concerning OpenForce's delay in releasing an initial
version of dotLRN. I suspect more people have this feeling and
are maybe reluctant to mention it. We've been struggling with the
right model for dotLRN because we need to do extremely rapid
development, but we also want community input and
participation (it is an open-source effort). It seemed to us that we
needed a solid base before we could fully open up the process. I
think we're reaching this solid base.

In the next couple of weeks, OpenForce is going to work hard to
reach this higher level of transparency. We're asking for
everyone's patience as we progressively add source code
availability, bug submission mechanisms, and eventually fully-
privileged external contributors. But we will get there, and we
have, of course, complete confidence that with a solid base of
code, complete openness is the way to go.

As much as I respect ESR, the successful open-source projects
are the ones that start with a solid base. Linux was not a group
project at first. OpenACS received community interest only once
it was at least somewhat functional. Etc...

But yes, we will definitely release often. We wanted to make sure
we had something workable and usable before we put out any
code. We're almost there. And we're looking forward to
everyone's comments, contributions, and real participation.

Thanks for bring up this subject Talli. I am guilty of this myself.
I think sometimes people believe their little problem is not worth posting to the bboard because it is not of general interest.

The bboard and openacs.org should be the central store of OpenACS  knowledge so that valuable information is available to the community. It is not as easy to post to the bboard as it is to send an email, but the benefits to everyone involved are worth it.

I appreciate all input from the community. When I am working on something I do like to check if someone is working on a similar problem, or has a better approach than one I have thought of.

http://developer.gnome.org/news/summary/#10

This is the Gnome hacker summary page I mentioned above. Something like this where the module names link to a summary page with an activity log and module owners would open up a nice window into OACS development, especially as plans for fully-privileged external contributors unfold. Not sure if that's the kind of thing you had in mind, Talli, but I think it would be helpful nonetheless.

Well ... a lot of this kind of discussion comes up informally, embedded in other e-mail.

I don't find it practical to say "let's run to the bboard before inserting any comments about adding features to X while discussing such-and-such topic in private".

Also, how about ideas spawned by a single person?  Do I have to post every idea I have before solidifying it into a solid proposal?  Yuk.  Count me out of that process!

And I think in many cases we're better served by having folks put up a target for comment that's organized logically and reasonably well-written.

What I want to avoid is design lock-in and certainly implementation without public feedback.  If we meet that goal (and we haven't always in the past) I really don't see any room for complaint.  If the design  of the portal package written by OF had been presented before implementation I don't think people would be complaining.

Before this degenerates into talking about how dotLRN is
"unavailable," let me attempt to clarify things. The issue is not
that community input would slow down the development of
dotLRN. The issue is that someone needs to set a direction for
dotLRN before it can take off. That's what we've been doing.
Finding that direction. Defining the initial high-level goals of
dotLRN and taking a first pass at implementation of this. We are
almost there. These decisions cannot be made by committee. Is
dotLRN a course management system? A learning
management system? An extension to OpenACS that everyone
will want to use? *Those* discussions would have slowed
things down.

And don't downplay the role of CVS and bug tracking. One of the
biggest killers of open-source projects is the inability to provide
feedback. Putting source out there isn't enough. Successful
open-source projects have strong leadership, a well-defined
direction, and a solid process for feedback.

I don't think that dotLRN is really at issue here. Sloan and OF are responsible for how dotLRN is going to be released, used and built and no one is questioning that. For better or for worse, that is OF and Sloan's decision. I agree with Don that had some module's specifications, like portals, that are applicable to the entire OACS framework been at least released earlier on, it would have been nice. But what's done is done.

Also, I will be the last one to suggest that a client project necessarily be made totally open just because it's been promised to be released. That screws with deadlines, contractual responsibilities, client expectations, etc. Neither am I suggesting that all development be made totally available, nor should it go on in the bboards only.

But we do need to realize that way too much is going on in the back rooms. There are some consistent and core people, but there are a great many lurkers out there as well. Transparency is not only nice for the core developers to be able to watch as things develop but also provides an in for lurkers either to participate or to test new pieces out.

Allow me to provide an example for how this kind of transparent development can take place without compromising development efficiency from my experience. Dave Bauer and Jun Yamog have taken a great deal of interest in using ETP for their own projects. They contacted Luke Pond with a lot of their fixes, suggestions and questions. As the three of them have poured over the code, decided on improvements and culled a feature set, they've decided to build ETP2.

Before beginning development, Luke wrote up a preliminary design document and posted it on the boards to solicit feedback. If anyone is interested in adding to the list or in helping to develop the package they are free to ask or post suggestions.

In the meantime, Luke, Dave and Jun are working out of a private CVS on Musea's server that was set up for expediency's stake. Once there is an alpha or something it will be merged back into the main CVS. If it were possible to set up a private CVS for small development teams on the OACS box, it would have been there.

IMHO, I think this might satisfy everyone's desire for greater transparency while not limiting anyone's personal work preferences.

As far as dotLRN, Ben, no one is questioning your leadership or your right, along with Sloan, to choose the direction of dotLRN. You've got a lot more invested in its success than anyone else. I want to make it very clear that I and the community recognizes that and does not question it. Particularly since you must deliver a rather focused scope to your client.

Some are only asking that in the future when an opportunity arises like the portal package that you consider strongly to solicit community input before pushing ahead on its development. I think that this will not only benefit your development process and schedule with increased design feedback but will strengthen your position as benevolent leader.

And this should apply to EVERYONE that is building new packages for OACS, just as a courtesy to the rest of the community and in the interest of building a better OACS.

I have no, and want no control over anyone elses project. I have been arguing the opposite, and would even take issue with Tali's seemingly benign statement that perhaps Ben could have solicited input on the portals package at an early stage. Ben has the absolute right to full control of all the projects he is working on, and anything that finds it's way to the public is a gift.

What I am saying is that design discussions and code are extremely valuable to more than those immediately involved at that point in time. I also say that developers have to communicate anyway, and that the incremental cost of placing discussion on a public bboard rather than in an email is nill. If you agree with me on those points, then I hope you'll also agree that every discussion which is syphoned away from this bboard and into private email devalues the OpenACS community.

I hesitate to mention .LRN again, as it's status as a for-client project means to me that this community has precisely zero say in it's future, but perhaps Ben is wondering what I am asking for if not for control.

Consider the state the project would now be in if at it's inception a bboard was created, either here or at OF's site, and possibly made read only to the public, and the OF developers had design and development discussions there rather than send each other email or use an internal bboard. Would .LRN now have less strong leadership or any less direction? Would it be any less of an open source project? No (in fact at this point in time .LRN is not an open source project because no code, no documentation has been released. Hence "unavailable").

A slightly cleaner way to do it would've been to solicit interest from others first, inviting people with a serious interest in contributing actively to the upgrade to join the effort. They might've gotten a couple of other folks involved at an early who would've been willing to roll up their sleeves and get down to it.

So what you're suggesting Don is that before work begins the community (made up of professional developers with other time commitments, hobyists etc., let's not foget) should be solicited for active, willing to roll up their sleeves and get down to it contributors, i.e. people who will commit to dedicating some significant ammount of time. These developers will then depart and between themselves via email or some other private means collaborate on the project for a week, month, year... Everything that is generated during the discussions -- the bugs encountered, the solutions assesed and rejected, the knowledge gained about some corner of the toolkit -- are all discarded, lost forever. During this period of private collaboration, anyone unable to meet the original commitment deadline is denied the oppertunity for involvement. When the project is complete the active developers, sleeves rolled, will present to the Community, and it will provide the thumbs up, thumbs down?

Sarcasm acknowledged, you seem to be saying that collaboration of the form I'm talking about i.e. letting people openly see what is going on, has some overhead which increases with the number of people involved, and that our Open Community System is not up to the task of facilitating that collaboration. Should we conclude from the number of participants in this thread that they are the only ones who are interested? Should we be having this discussion via email because the burden of over 100 people looking over our shoulders is holding us back?

My one and only suggestion is that if you feel as I do that design and development discussion is the valuable information which binds this community together, and that posting it to the bboard rather than keeping it private, where you are able to, is not too much of a burden, then you do so. To suggest otherwise leaves me genuinely quite puzzled.