LAMP exists, what would changes to OACS add to LAMP? If merely a me-too bullet point, it seems pointless, LAMP exists! Use it if it satisfies your customer's needs.

Refactor to xoTcl? For one I can say that I can never figure out what is going on in an xoTcl script. If the original code works, why would it need refactoring to xoTcl, why not refactor to Java or LAMP?

One thing OACS requires is a database. It is impossible to refactor OACS into modules which would work without a database. Also, remember that a lot of code is defined with ad_proc, etc., so moving code out of OACS is nearly impossible.

It is a short job to allow editing of templates via a web browser. The problem is more with supporting this as a core function, and then writing packages which use the feature.

One interesting point is that LAMP in most generic form simply means using the best tools at each layer, and keep code in the most applicable layer. Our layers are Tcl, AOLserver, AOLserver Modules, OACS-Core, OACS Packages.

So LAMP not so much a layered system as a code grouping and coordination system. The OACS system can push code groupings into different layers using general or specific API. This is more flexible that LAMP, but usually what you get with LAMP is an application which has to handle the whole process as a single bag of code. The OACS system allows for lots of reuse.

The fact that more specific tools are not used in OACS is really a matter of developer taste, which I have no interest in changing. I can say that I write packages which handle specific tasks so I can get lots of reuse and lots of separation of function. I've also written a package which allows lots of configuration/customization by design. Each content type can have a separate master, type, page and item template (or reuse an existing or default template). Each item can also have a different template. Templates are also content types, so they can have parameters and customizations: added and updated in the same way as other content.

All templates are editable via a web interface. My 14 year old edited her blog template by herself. The same installation has other applications, like an inventory/product list with ecommerce and photo galleries for each product. So it can be done, without any changes to OACS or interference from dotLRN. Just add data, configuration and templates, no Tcl coding, no database coding.

If you really know Tcl and OACS, you must know that writing an app will be faster with these tools than with PHP, but maybe not fast for a marketable product. These things don't write themselves. Remodeling your kitchen doesn't improve your cooking skills.

Well, a couple of other things ...

I very much dislike the notion that all templates should go in a global directory and that css and resources should all be global.

Follow that model to its logical conclusion and EVERYTHING should follow a flat model, no modularization at all. It's so 1945.

The above works nicely when a function is documented with ad_instproc, sadly it miserably fails for anything else.

Furthermore, when I type in e.g. "get_object_type", I see the generic class procedure (which is documented, so things are fine). Sadly, unless I know that a FormPage is a CrItem, I would not understand what xowiki::FormPage get_object_type does. So, I would love to see, if I type in "xowiki::FormPage get_object_type" that the API Browser finds me the Generic::CrItem get_object_type procedure and displays this (probably along with a hint that this is the version being used by xowiki::FormPage).

Does this make my problems a little bit clearer with the currently provided documentation? I do have the slight feeling, regardless what can be done with the API Browser and the XoTCL browser we still need make a lot more procedures using ad_instproc and Classes using the -ad_doc parameter for us to really "digg" XoTCL in OpenACS.

And I agree with Gustaf that on the XoTCL Browser having "Show all methods including inherited ones" will probably lead to pages of procedures / methods being displayed, but at least it helps us reading the code.

Not having developed in Java or other OO languages, how do the IDEs for them solve the problem of inherited Methods and the documentation for them. Any Pointers?

Tom, is your package available somewhere to look at and would it be a worthwhile addition to OpenACS as a whole. It seems to me that it is highly useful for custom build content types, yet I do not understand how this would work on the existing types like users. Yet I see a huge benefit if this work could make it into a generic dynamic attribute solution, as it would fill the missing gap. Do you store the templates in the database or do you write them to disk as well?

You can browse the package at http://rmadilo.com/files/cams/

This is not the same thing as building custom types, but look at the data model. Each content type begins with a title and content. You can add attributes to the type. When you add content, you first enter the title/content and then you can add the attributes. For instance, your content type could have an image attribute, or call it whatever you want MyImage, whatever. When you add an attribute, you define the content type and the form widget. If it is a file widget, you can upload a file, or more than one file for the attribute. The package name CAMS stands for Content Application Management System. The idea was to allow developers to create certain types of applications focused on content: how to get it into the system and how to display it.

Templates are used for the master, content type and content item. In addition, the content of each item can be treated as a template. Special tags can be included in the template to format/insert the attributes associated with a content item. So if you upload an image, you can use the same data to insert a link to the image, or display it inline, anywhere in the content.

New templates are stored in the database, they are content items. But they are also 'compiled' and stored as files for use. The 'compiling' just applies any attributes, so that the same base template could be reused.

Some applications are here: http://whogeenie.com/cams/

Wasn't there a project trying to refactor ACS in Java, which failed badly?

For instance, you didn't write HTML-based template files ala ACS 4. You wrote XML which got fed through XSLT to create HTML which then got streamed to the user, and whatever the abstract attractiveness of this approach might be, the particular implementation was obtruse and difficult to use by experienced programmers. Designers? Forget it.

That's just one example.

A straightforward rewrite might've been interested, in the sense at least of giving some empirical evidence that a Java-based framework's always going to be a PITA to work with.

One might say it had a similar relationship to ACS 4 as ACS 4 did to ACS 3, with a language change thrown in for fun.

My comment about xotcl simply reflects a belief that AOLServer+tcl+xotcl will be no more popular than AOLserver+tcl.

Well, there's still a pro-Mysql bias in the web world, too. Mysql's gotten better, at least, though the idea of giving up PG would make me sick :)

- Certanly, How OACS manages CSS is a problem. I'm not sure what it's the best solution, but our approach now, at least, is modular and, for recurrent users (because of the cache) works quite well. But yes, first time, I had to load a lot of files.
- It's impossible to install OACS in 20 hours (complexity and well trained people), so my students were always happy with LAMP, but ..., it's not fare compare a typical LAMP app with OACS. And for me, Aolserver and Postgres is easy. TCL, OR AOLServer API it's not cool (neither Emacs), but works very well (It solve my problems quickly)


But, what is the future? I miss some things, some bugs that remain a lot of time, and a lot of distributed effort. But I think it's a common problem of a complex and mature project in open source world (like OACS).

But. sure. we have to learn of other approaches

Rewriting in XoTCL does not make any sense to me either. Why change a running system to make it run slightly better yet radically different with even less documentation for the developer to go around than there is for OpenACS per se.

This said, I love XoTCL and what it did for my application development. And more of it will popup in OpenACS CVS in the medium time for new functions. But I would not go and rewrite e.g. Forums in XoTCL unless there is a compelling reason to do so (read: customer requirement).

What would in my opinion be killer apps for OpenACS are things which haven't been discussed here:



WEBSERVICES

TWiST works great and I am sure Stefans code works beautifully as well. Yet we need to bring them into OpenACS properly and devise methods where an administrator can quickly say: "I would like this information available as a webservice". Ideas on that:

-- Make a form a webservice. If we use ad_form, provide a switch that, if querying a URL with a certain parameter gives you a webservice for the form.
-- Enhance listbuilder to return WSDL formatted XML in addition to CSV

I am aware that I will get lot's of bashing about this for not understanding the concepts of Webservices, but what it boils down to is:

"I need the information on this table available in my financial data system"

"We need to interface our user creation with yours to keep it in sync"

"We have a Yahoo Widget which wants to add a new task"

If OpenACS as a toolkit provides interfaces OUT OF THE BOX, without the developer having to do anything about it, then we talk killer app. Oh, and yes, we would need to have security on the webservices and lots and lots of other things to take into account, above is just a suggestion.

What can be done? Have the most experienced people with regards to webservices (Stefan and Tom) along with interested OCT members come up with a generic strategy how to support webservices in OpenACS. Get "customers" like Quest and ]po[ and E4 involved to state what they need and see if the model suits them. Then collaborate together to come up with the funding for a generic solution to be put into OpenACS. But as there a customers with needs I have the slight suspicion there will be money as well (or at least development help).

EXTEND FIELDS

I think the reason that we have 5 (or more) ways of dealing with dynamic extension of data fields shows me that there is huge demand, great intellect and capacity for building it and ZERO coordination to make this a killer feature for OpenACS. But this is what customers want.

They want to be able in their application to extend the fields a user has to fill out when signing up, WITHOUT hiring a developer to make that happen.

They want to store their custom datafields for projects and show them in a project overview.

Why is it at the moment impossible to do this with OpenACS?

If the most common needs for a user of OpenACS can only be achieved by hiring a developer who is knowledgeable in OpenACS, chances for him using the toolkit are slim, because he can use a LAMP application and hire a smart PHP kid from Ukraine for $20 per hour to make the changes for him.

What can be done about it? Collaborate on http://openacs.org/xowiki/dyntypefields by writing up what we want to have and pointing to bits and pieces of code that already provides this. Then ask who can work on achieving this and then have those who can work on it vote on which existing solution to base the new one (or to write a different one from scratch).


DOCUMENTATION

Our documentation sucks and the one for the packages is even worse (XoWIKI becoming an exception to the norm). But without documentation on how to achieve certain things in OpenACS we will not be able to attract new users. And if our goal "only" is to aquire new developers who love the toolkit, than at least the developer documentation needs to be up to date. We (Anett and I) have worked over the last months to clean up http://openacs.org/test-doc and enhance it bit by bit with new information (mainly get an update for TCL for Dummies there), yet this showed us how MUCH we are out of date and how difficult this job to update the information is going to be. And to make matters worse, it probably needs the best people we have (in terms of development experience) to write this documentation to attract new developers, which is a catch 22 as they are usually in high demand (due to the fact that the documentation sucks so badly that noone else can quickly learn this).

What can be done about it? Among the companies in need for new developers come up with a "fund" for the outcome to have a developer tutorial along with updated Problem Sets and training / teaching instructions. They need this anyway if they want to grow, so they might just as well shoulder the burden and not do the same effort each.

Then try to get some EU or other funding to partly cover the writing of documentation for packages used in the funded project and have them written. There are excellent writers for documentation out there and I am fairly sure you can point them to our application and as long as you are open for questions, they can write up (and screencast) what they have been doing on the application, without having used it for a couple of years. And yes, this obviously depends on the level of complexity of the application.


ATTRACT NEW DEVELOPERS

We have a lot of universities using .LRN. They have in house staff that are knowledgeable with the solution. They have students who might need to learn about developing these kind of solutions, so why not teach them? As Gustaf mentioned, a lot of volunteer effort's benefits come from the acknowledgment of your work. So why not invite OCT members to be guest lecturing a course at your institution? This would already get into the direction to providing the tutorial as discussed above and if the lecture is taped and edited it can then be used as a Web based Training. I am not an expert in this, but with all the effort going into LORSM and IMS LD, I assume we have enough knowledge in the community to achieve this.

Why not providing/ exposing functionality and data over different protocol channels (SOAP and XML-RPC), without the need of multiplying the coding effort (e.g.,rewriting the serving bits of code, interface descriptions)? In fact, multi-protocol support is something rather new in framework design, most established frameworks are rather short-handed in this respect. Note, by multi-protocol i do not simply mean providing an xml-rpc and soap package and having the individual developer deal with exposing their code to them. There should be a certain protocol transparency from the developer's perspective. I don't want to give examples from literature, but just a hands-on example: Ruby on Rails comes with a somewhat multi-protocol support (at a rather low level, for my taste) in its Action Web Service module. By switching around some parameters, you either go for an XML-RPC or SOAP call/provision, through a common interface.

Everything should be automatic! This is like ASP.NET, just add a few decorations to you code and it becomes a web service...or not. Actually if you want to have complex types or validate the input you are totally screwed, now you need to write your own code for doing this. Apache AXIS is worse. Although it _is_ true that once the bits get to the protocol level the developer doesn't need to know anything, but by that time who cares? There are no switches to throw anywhere in any system, and you still lack the most important ingredient: a published description of the service and how to access it. Maybe somebody will recognize the importance of following a standard so that external clients can actually use your interface.

Now, what are the issues with OpenACS? The main issue is that most OpenACS applications depend upon page views. By that I mean that pages serve as a procedure. This makes it nearly impossible to reuse this code by a different protocol. For instance, my query-writer package requires all form submissions to go through a special page, very similar to rpc, minus the xml. Until I write a procedure, or set of procedures which covers this functionality, no amount of developer support will help me.

Another issue specific to Tcl is important to recognize: Tcl does not maintain type information, ad_proc actually complicates the situation, and we know nothing about certain things like return types, use of uplevel/upvar, etc. Many programming languages know all this information and so it is 'easy' to serialize a data structure to XML. This is why there are so many rpc packages in every language. But rpc has many problems which were never solved because they are much harder. First problem is that server and client really need to use the same language for full interoperability. Otherwise, either one or the other needs to learn to handle conversions. Second is that it is unlikely that anyone anymore really wants to expose an internal procedure to the network. The best method for handling this is to write a wrapper procedure to restrict use or perform any additional checks that usually take place prior to execution. One of these 'checks' is validation. This is where the wheels fall off of rpc and it is why nobody, not .NET or AXIS performs validation out of the box! And by validation, in this situation I only mean simple validation like is 4 an int? Derived types and complex type validation...this isn't even supported if you wished to do it. Just make up you own method of figuring out if the input is valid, test, debug and hope.

So if the big boys are not offering this as a feature of their products, with billions spent on development, maybe it isn't so easy. Configuring a procedure as a remote application, or whatever it is called, will always require a certain amount of information. How to collect, store and expose this information so it can be reused by new protocols, maybe at the same time...that would be helping the developer.

Hmmm maybe look at tWSDL. tWSDL impliments a single protocol: SOAP with document/literal. What is probably not appreciated is that the protocol is just a few pages of code, and is chosen based upon the client request. If someone were to write their own 'binding' and protocol handler they could reuse the entire tWSDL code base. The service would be fully described by the WSDL, input messages would be validated prior to use. If XML was not used as the representation, then the protocol handler would use some other method besides tDOM to deserialize and some other method to serialize the result. Writing the code is the least time consuming part of this. Deciding what your protocol is and how it works will take the most time.

One thing has become clear over the last few years: developers are starting to recognize that interoperability is the key. Interoperability is achieved by restricting the number of options and requiring public descriptions of services which don't require out of channel communications.

A note about code generation from WSDL. TWiST does this, not just stubs creation like AJAX. But it is only for the server. I have begun work on a TWiST Client. The client can run along with the server, but it will very likely not require AOLserver, that is it will be pure Tcl. But I'm developing it against Google's AdSense service. Everything is https, so I'm using ns_https to pull in the WSDL to read. The service developed on AXIS and uses a feature not in tWSDL/TWiST: SOAP headers. Anyway, the goal is a client which is created from reading the WSDL with API creation for each operation. Creating a document and sending it should work just like in the TWiST server. The return document will be available, but there will likely be a series of generic procedures for turning the document (as a Tcl representation) into Tcl structures, lists, arrays, etc. Of course there will be optional or automatic validation on the return document.

Working with a procedure defined via ad_proc actually presents very similar issues as an application page. ad_proc serves many purposes, but the way it works makes it harder to expose as a web service. This is why TWiST works the way it does: configuration is done via <ws>proc, which provides a wrapper for the internal API. Inside this wrapper, you can specify any amount of code. Maybe you could source a tcl page and do something with the resultant state. Whatever goes on inside the wrapper, you just need to return a list which represents the output message.

One point about typing with TWiST: more important is the structural typing, such as the names and order of the complexType elements, how many of each of the elements, can it be missing, is there a default, if missing, does this indicate it is 'nil', etc. Otherwise, your return document will just be a smashed together mess, but internally TWiST does not care about types, just structure. Essentially the type information is a description of the template: what is the document going to look like. Nobody would ever suggest we just run a tcl script and return a list of all the variables, the results need to be formatted, and this is what the type specification does: defines the non-data parts of the document. The wrapper code doesn't need to know anything about XML or any of this configuration, it just needs to return a list which will be formatted into a document.

When the TWiST client is done it will only handle document/literal, whatever is available now is in the wsclient package of tWSDL. I think everything is in a web page, and it requires the ssl module to be installed, at least to use ns_https.