Forum OpenACS Development: Need better sign-posting in openacs.org

1: Need better sign-posting in openacs.org

Posted by Edmund Lian on 04/01/01 07:05 PM

Hi all,

Just got back from an extended business/pleasure trip. I've been out
of touch for 3 weeks, and have been trying to catch up with what's
happening regarding the ACS 4 port.

One thing that I've suddenly noticed in trying to get started on
porting stuff is that it's actually very hard to figure out what the
state of anything is if one has been in the wilderness for an
extended period of time. E.g., is the query dispatcher up? Is the APM
up? What decisions have been made regarding issue A, B, C, etc.

There has got to be an easier way to catch up with the development
effort than ploughing through all the forum messages. Is it just me,
or do we need some kind of summary page for developers that clearly
records resolutions, decisions, file locations, etc.?

2: Response to Need better sign-posting in openacs.org (response to 1)

Posted by Ken Kennedy on 04/01/01 09:02 PM

I think that'd be a great idea. aD had something similar when they were working towards 4.X, and I thought it worked really well. It is really hard to get a bird's-eye view of how the port is going at this point...If there's not anything we could use right now, I could hack something up, maybe based on glassroom or something. Of course, it would mean that we'd need to keep it up to date. *grin* Any interest?

3: Response to Need better sign-posting in openacs.org (response to 1)

Posted by Kapil Thangavelu on 04/02/01 02:00 AM

i totally agree with you. i've already made suggestions for a weekly
status report both on this bboard and privately that have fallen on
deaf ears. i'd be happy to volunteer for the work as well.

as for development via bboards, i never understood why aD and
openacs seem so hung over on using them other than toot their own
horn so to speak. every other open source development project in
existence uses a mailing list. bboards seem to reach a critical mass
when they're not useful for tracking a product's development. the
big problem with bboards is that its too easy to lose track of a new
message in an older thread, they have little sense of chronological
view, they scale poorly in terms of finding relevant information in
a busy forum, and its hard to realize that an older thread might
still be maintaing an active discussion. a good mailing list
(list.org), with a good search and archive interface goes a long way
to solving these issues (btw, i think the archive and search
interface for pg kinda bites). bboard style development tends to
favor those who are subscribed via email and who check it daily (in
fact email subscriptions are the only thing that even makes bboard
development at all viable or useful, see the point). its a
dentriment to a new developer who is trying to catch up on
developments of interest. hopefully a move to mailing list will also
cut down on all the person to person email that goes on and subverts
the open source development process.

4: Response to Need better sign-posting in openacs.org (response to 1)

Posted by Roberto Mello on 04/02/01 02:34 AM

I've been thinking about this for a while. IMHO, if we had to post reports and such, it would only be a waste of time for them, time that they could use coding.

I agree that having one place that everyone else can see what's going on and what needs done is a good thing. This is exactly what Momjian does for the Postgres team.

Therefore, since I am already taking the role of "gatekeeper" for the 3.x tree, I'd like to volunteer to take the extra step and keep a TODO. I already follow the bboards and adding items and links to relevant bboard threads is trivial, but important. Ben? Don?

On the mailing list issue, I also agree. The bboards, in the current state are cumbersome to work with. I think they are great, but if I could post to them from my e-mail client, it'd make my life easier. Following messages in a threaded interface such as the one in mutt would also be very nice. And last, but not least, splitting different bboards into separate mail folders is a biggie for me.

5: Response to Need better sign-posting in openacs.org (response to 1)

Posted by Ken Kennedy on 04/02/01 02:54 AM

Roberto...you're a great guy, and I think it's great to step up and volunteer to coordinate everything, but come on. "Posting reports would only be a waste of time..time that they could use coding"? *grin* Sometimes, a coder needs to take a break!

I'm talking something that would easily updatable over the web, an extension of something like the glassroom or ticket module, allowing an indication of where things are in the process. Sure, it would take a couple of mintues to update...but it would give all the OTHER folks that are spending half an hour at a time trying to track down what's going on a single place to go and see a high-level project status.

It's great that you're willing to take on the responsibility of organizing it all, but remember, that makes YOU the bottleneck. Not that you probably wouldn't do a great job, but why not let the computer do the work? Especially when we're talking about organizing workflow and status reports for software that...organizes workflow and status reports?? *grin*

If I can whip something up in the next few days, we can take a look at it (on my site). No pressure or anything, but maybe we can "scratch the itch" here...

Ken

6: Response to Need better sign-posting in openacs.org (response to 1)

Posted by Don Baccus on 04/02/01 03:05 AM

I do plan to post regular reports, and in fact posted my second semi-regular report yesterday. I know the importance of making public progress reports. I know this without being told, so am not sure why it is felt that a request for regular reports has fallen on deaf ears.

It would be nice if we had a little reporting tool so folks could post their reports on a regular basis which I could summarize. It would be nice if such a tool would e-mail reminders to folks that they owe a weekly report 24 hours in advance. At the moment I'm just pinging Ben and Dan and hope for answers. I need to get something systematic going as our list of volunteers is now large.

Where should we post reports? Should I keep posting them here? That's been my plan but a separate status report "forum" would make it easier for folks to track. Mozilla has a web-based system that shows weekly reports and tells you who hasn't reported, as well.

As far as switching to e-mail lists, I really don't know what to say. Some folks like these forums, others don't. Some folks like e-mail lists, others (including me) don't. We're not going to please everyone. Yes, it would be nice if folks could post here via e-mail. This would let folks interact with the forum via browser or e-mail client depending on their personal preference. I run a mailing list (the pacific northwest eco-forum) and belong to several others, and they threaten to run my life. Because of this, I've slowly developed a strong dislike of them.

Roberto, if you want to keep a "todo" list personally I think it would be great. We'd want that posted in a prominent place on the site.

7: Response to Need better sign-posting in openacs.org (response to 1)

Posted by Ken Kennedy on 04/02/01 03:17 AM

Don, that (the Mozilla tool) sounds pretty much like the sort of tool I'm talking about. I'll go take a look at their site...see what we can whip up. What version of OpenACS is openacs.org running on?

And Roberto, I hope you haven't taken anything I said the wrong way. A personally managed "todo" list (especially by someone as organized as you), would be great. If it was me, OTOH, it would be bad...very bad. *grin* (I use the todo module to remind me to pay my bills...)

Off to mozilla.org!

8: Response to Need better sign-posting in openacs.org (response to 1)

Posted by Roberto Mello on 04/02/01 03:48 AM

Ken, I didn't mean that the developers should not let us know what is happenning. What I was trying to say is that adding layers and layers of things that a developer must post to and report can become anti-productive.

I want things to be better, and I want people to be able to get quickly up-to-speed with what's going on, but not at the expense of transforming OpenACS in a huge bureaucratic ball. I like the KISS principle (Keep it Simple Stupid).

In the PostgreSQL project, Bruce Momjian follows the mailing list discussions, where the developers post their reports. If we can have the computer do that, great. But then it's two places the developer has to post to. Or two places to sign up for alerts.

On the TODO list issue, what happens on the PG project is that when people hit bugs or issues and ask about them in the lists, the developer familiar with that portion of the code responds to that question with something like "oh, yes. this is an issue becaus of xyz that needs to be resolved."

Momjian then adds a line or two to his TODO with a link to the relative thread on the mailing lists. This is easy on the developer (because he doesn't have to post it _again_) and easy on those that are following development, because Momjian goes and marks the issues as resolved, since the developers usually post to the list saying "I just fixed isseu xyz".

Now, applying this to the OpenACS project, what I could do (on the TODO issue) is to follow the discussions (as I do) and post the issues to the SDM with the appropriate assignment.

On the reporting issue, we could have a module to the SDM (IMHO, the appropriate place for this to go) that will keep track of reports. The developer would be able to either post directly there, or we could imply post a title and a link to a bboard message.

Thoughts?

9: Response to Need better sign-posting in openacs.org (response to 1)

Posted by Ken Kennedy on 04/02/01 06:43 AM

Oh, excellent, Roberto! We are SO in general agreement here. I don't want to make things bureaucratic either. I'd just really like there to be a "go-to" place to see how the port is going. I agree, it probably ought to to associated with the SDM. As some examples of what I'm thinking about...aD's ACS 4.0 Prj Central has some good features (though it's also WAY overkill for what I have in mind). It's got the "Subsytem Owner" list that shows sub-packages, their owner, and in some instances, status info (I really like the bboard news link, but I'm not expecting anything like that).

Don's suggestion of the Mozilla weekly status updates (see the main list, and a specific example) I also like. If the reports came out like that (by module: highlights, lowlights, todos, stuff done, etc.), it would be easy to review current status.

My main question about using the SDM is regarding subsystems/modules/packages (whatever they're called). At this point in the game, I think that seeing things explicitly broken out (ie, Query Dispatcher, APM, Installer, BBoard, Bookmarks, etc.) with their statuses, todos, etc. would be a great thing. Can we do that in the SDM now?

Thoughts?

10: Response to Need better sign-posting in openacs.org (response to 1)

Posted by Don Baccus on 04/02/01 06:20 PM

Yes, I was thinking of enhancements to the SDM as well, in fact this project we're embarking upon will be a great source of ideas for improving the SDM to the point where it becomes a really great tool for helping manage distributed software development projects. (Intranet has things that would help, too).

I have a hesitancy to suggest we invest too much effort in the short-term, though, because openacs.org is 3.2 based and of course any general enhancements we do would need to be dragged over to 4x. If we made use of the back-port of the 4x db API and tools like the ad page validation stuff the amount of rehacking necessary to bring it all over to 4x for general release would be minimized.

While I'm writing, here's a status report for you that I wish were an April Fools' Day joke:

I worked on the APM for an hour or so on the plane home yesterday. Got up today, did an Oracle startup via svrmgrl, no complaints. Fired up my oracle acs_4 installation and it died with horrible "block 1 corrupted" error messages. svrmgrl/shutdown gave horrible whining internal-error messages and refused to shutdown the db. Upon rebooting svrmgrl/startup Oracle barfs in a similar fashion.

I can't even build a new database it is hosed so bad. So here I sit about to reinstall oracle on my laptop.

And to think that for the past two years when people ask me about PG vs. Oracle I've been saying "PG's getting a lot better but if you really, really need a bullet-proof guarantee against database corruption Oracle's still your best bet".

Yeah. Harumph.

I still hope to finish up my APM changes by today but the day promises to be a longer one than I imagined when I awoke this morning.

11: Response to Need better sign-posting in openacs.org (response to 1)

Posted by Kapil Thangavelu on 04/03/01 07:52 PM

i want to apologize, i think my first post on this thread was a bit
extreme. my requests for status updates haven't been met by deaf
ears but by busy hands. don, thank you for taking the time to post
updates to the forums. my frustration emerged from the fact that
before the last three days updates have been sporadic and far apart
and that all of those updates have referenced non public knowledge
from private exhanges on critical topics.

as for what a status update should comprise i don't want to
inconvience developers with redtape. what i envisioned
was something more along the lines of general project status, news
from any of the specific module ports, core status/news , brief
summaries and links to discussions of interest on the bboard. all of
this could be done by one person with developers of modules emailing
that person with news when relevant. accurate timelines are a bit
hard to nail down, so i think other than the core that news would
suffice (as long as the module porter/author is clearly identified
so interested parties can converse with him/her/them directly).

i think these summaries should be linked directly off the openacs4
project page something like http://www.openacs.org/4/status (or what
have you). putting summaries on the forums kinda makes me think of
chicken and egg scenarios. i think the point of this type of
information is to have prominent display of the projects current
status without having to wade into the forums.

while putting this kinda of info in the sdm is useful and i'm all
for it, i want to mimimize the amount of link clinking needed to get
a sense of where the projects is at. if a module summary page could
be integrated into the sdm all the better.

as for mailing lists v bboard, it is a matter of opinion (although i
could still go on at length about it:). but i honestly doubt that if
any mailing list is overwhelming its going to benefit from moving to
a bboard format. mail aware bboards would be a huge leap forward.
hmmm... except for the fact that clients are pretty arbitrary in
their behavior (hence the mail thread assoc would be difficult) this
type of behavior should be straightforward implementable. perhaps in
the port of bboard4.

12: Response to Need better sign-posting in openacs.org (response to 1)

Posted by Don Baccus on 04/03/01 08:19 PM

Apology accepted, thanks. You hit the nail on the head by stating your request hasn't fallen on deaf ears but rather busy hands. Also, I spent a week in Florida and it took a couple days after my return to get back in the swing of things.

We've been working hard to make the decision-making process more open. That will grow as we get experience working together as a team.

Ben, Dan Wickstrom and I did much of the ACS 3.x port so are used to working together already. That's really the main reason we began work quietly amongst ourselves. We were already a team.

Roberto did most of the doc work but that was more of an independent task not requiring coordination. I am truly eager to work with our ever-growing group of volunteers and hope we all feel like we're part of a well-functioning team by the time we're done with this project.

As far as the SDM and status reporting goes, the kind of enhancement I'd envision would be in essence to provide a link to status reports associated with modules and one to a page that builds a summary status report for the project at large. Don't know if I or anyone else will have time to play with such a notion in the near future but some kind of integration seems like a useful idea.

I'll try to come up with a short-term proposal for module porting status that doesn't require any such work, though. Probably I'll just ask people to e-mail me and I'll put together a summary and post it here for the time being, until we have time to put together something more interesting and useful.

Yes, e-mail accessible bboards are a priority for many of this, there's no argument or disagreement in this regard. It has been done for ACES. I'll definitely be doing it for my own most active site in the next three months or so and would be thrilled if someone beat me to it so I could poach rather than write code...

13: Response to Need better sign-posting in openacs.org (response to 1)

Posted by Ken Kennedy on 04/03/01 09:23 PM

Don wrote:

> As far as the SDM and status reporting goes, the kind of enhancement
> I'd envision would be in essence to provide a link to status reports
> associated with modules and one to a page that builds a summary 
> status report for the project at large. Don't know if I or anyone 
> else will have time to play with such a notion in the near future 
> but some kind of integration seems like a useful idea.

Don, that sounds very much like what I was thinking of. Since I've now discovered that I have SDM installed *DOH! grin*, I can play with this at home. Cool!

14: Response to Need better sign-posting in openacs.org (response to 1)

Posted by Kapil Thangavelu on 04/03/01 10:39 PM

don wrote
<blockquote>>
</blockquote>
I am truly eager to work with our ever-growing group of volunteers
and hope we all feel like we're part of a well-functioning team by
the time we're done with this project.
<<

i think the key to doing this is by creating enough documentation
artifacts that volunteers can quickly can get up to speed and become
productive developers (not to mention it assists in the process of
converting new users to developers). in this the openacs has been
lucky because in the past they could rely on the documentation
generated by aD. but as the openacs makes changes to the core
architecture, the onus of documentation falls upon us. this is why
private email discussions irk me so much, because it subverts the
process of documenting discussion on key issues, leaving the source
as the only documentation artifact. its also why i favor an email
list approach, because they offer the best means of examining the
development of a project in a consistent chronological fashion. i
think after the core is ported, all architecture changes should be
documented in some form of spec that discusses the problem, the
solution, the scope of both, and risk factors. this doesn't have to
be lengthy, a one page summary will do wonders. if we don't do this
than i have little doubt we will pay the price later. i've already
experienced this at my current job where i'm telecommuting around
the world, and now i'm paying the price.

15: Response to Need better sign-posting in openacs.org (response to 1)

Posted by Don Baccus on 04/04/01 04:09 PM

Documentation is key. A couple of our volunteers have offered to help out and Roberto's already done some work on internal documentation (his PL/SQL vs. PL/pgSQL document for instance).

The problem won't be our understanding of the importance, but rather execution. This is *always* the problem with documentation in my experience, because good documentation requires input from developers (even external documentation) and most developers have too much on their plate and have to prioritize their work.

With a volunteer-based effort like OpenACS we need to work hard to discipline ourselves to make sure we do get all our changes documented.

We can start with the code itself, I pepper the code with little "DRB" comments (my initials) documenting my changes, usually with at least a little bit of background explanation. For minor changes this is sufficient, more major and systematic changes need to be separately documented as Kapil suggests.