Forum OpenACS Development: Developmnet tools, debuggers, IDEs, etc

How do you work on OACS? what tools do you use? how do you debug it?

I'm am little annoyed with my OACS development technique which I find very inefficient, coding with vi, writing ns_log Notice lines, no debugger, no IDE, restarting server, checking nsd-error.log for clues why this does not work as it should.

I'm sure there are people who have developed better techniques then this one, and I hope you share your ideas.

Collapse
Posted by Andrei Popov on

You code with (X)Emacs, where you can employ nsd.el to parse ns_log notice/warning/error messages, and also use OACS mode to develop your ADP/Tcl/XQL bits of OpenACS site.

Collapse
Posted by Bart Teeuwisse on
Jay,

over time the Emacs OACS Minor Mode will incorporate a successor to nsd.el. The current beta however addresses several of your concerns. Do give it a try. Emacs in vi compatability mode (VIPER) should ease the transition.

/Bart

Collapse
Posted by Jade Rubick on
Just a little testimonial here: the oacs.el package that Bart has put together rocks! I can't imagine life without it. It makes it very easy to have consistent definitions for your procs, and it makes documentation much easier as well.

And installing only takes a few minutes.

If you've held off installing it, take a few minutes and try it out.

Collapse
Posted by Mark Aufflick on
A little rant from me:

I used vi/vim/gvim for over 10 years of my life, and nothing will ever get me back the time i wasted.

emacs makes you more productive period.

The mode-less editor was invented at Xerox-Parc in the 1960s. Don't clutter your mind with mode switching!

Plus things like the sql-postgres/oracle mode are helpful, as is the CVS integration.

I really want to modify the tcl mode/imenu to recognise ad_procs. I modified my old vim ctags/hilite modes to do this, but my lisp isn't really up to scratch yet.

Collapse
Posted by Jay Dubanik on
Thanks for your suggestions,
I will definitely check emacs OACS mode,

From what I understand OACS will help me extract and build proper XQL files, and execute SQL queries, but from what I have read on Barts' website about OACS mode I don't see however how OACS mode helps in writing ad_procs and/or tcl/adp pages.

The emphasis there is on building xql files and executing sql queries, but I don't consider this to be of my concern.

For everything regarding database I use Tora (Open toolkit for Oracle) http://www.globecom.se/tora/ which covers all my database needs(linux version is free), I test my sql code there and then knowing it works I copy it into xql or tcl file.

The problem that I constantly encounter is building ad_procs and tcl/adp pairs from scratch.

Ideally I would like to see something like an editor with a tcl debugger that would recognize oacs and ns calls.
I was thinking maybe having oacs procs written as a tcl package and using a tcl debugger with adding 'package require <oacs-package-name>' in tcl pages, but I have never actually tried.

Collapse
Posted by Jade Rubick on
You might also want to install the developer-support package. It lets you directly interact with the Tcl processor of Aolserver.
Collapse
Posted by Bart Teeuwisse on
Jay,

take a look at the news column (http://www.thecodemill.biz/services/oacs/) or the changelog (http://www.thecodemill.biz/services/oacs/changelog/index?revision_id=10054) of Emacs OACS. You'll notice that Emacs OACS does a lot more than what is mentioned in the articles. As this is still a beta release, documenation is behind the development curve.

If you are an Emacs user (and even if you are not) I recommend giving Emacs OACS a shot. Use the changelog to learn what Emacs OACS has to offer.

/Bart

Collapse
Posted by Andrei Popov on
You may also want to do a quick search in this forum, there were a few other threads on the same.  In particular, the use of etags was discussed a few times, which allows you to jump to procedure definition, etc.

Unfortunately, if you are expecting context-senstive prompts, etc. then you are out of luck (at least at this time), although you may (I have no idea, as I have not tried, this is just a guess) be able to do some of it using AcitveState's Komodo/ActiveTcl -- it may be possible to teach it Aolserver-specific procedures, etc.  This won't be free, however.

There's also TclPro (again, ActiveState) that allows you to debug Tcl procedures.  Again, I have no experience with this, but am sure that a few people on this forum may tell you whether it worked for them.

Collapse
Posted by Bart Teeuwisse on
I tried TclPro recently w/ AOLserver and it proved to be useless as AOLserver get's stuck after it closes the connection to the browser.

I've made a custom script available that generates tags for an OpenACS site. See http://www.thecodemill.biz/publications/blog/one-entry?entry%5fid=10033 for details.

Andrei, what kind of context sensitive prompts are you refering too?

/Bart

Collapse
Posted by Andrei Popov on
Bart -- things that pop up when you type?  Like, say typing up ad_fo... and getting suggested completion to ad_form?  Or like opening a bracket after ad_page_contract { and getting prompted for possible blocks (documentaion, parameter declaration, etc.)?

I said it's not available -- and am not sure whether this really is needed/worth th etime and effort.

I sort of expected that TclPro would not help much, btw.

Collapse
Posted by Bart Teeuwisse on
Andrei,

you should definately give my Emacs OACS Minor Mode a try. (http://www.thecodemill.biz/services/oacs/). Emacs OACS automatically inserts headers for OpenACS files such as ad_page_contract, ad_library, .XQL and .ADP files.

In addition, EMACS OACS offers several abbreviations that expand to procedure definitions, debug messages and the like. Each of these (headers and abbreviations) will ask for input.

See the Emacs OACS changelog for documentation (http://www.thecodemill.biz/services/oacs/changelog/index?revision_id=10054). Emacs OACS is in beta and stable enough to be used. A hand full of developers are using it daily, but documentation is behind the development curve.

/Bart

Collapse
Posted by Andrei Popov on
Bart -- Sure, I've tried it out some time ago, but not in the past few weeks.  Will take it for a further spin -- I've seen the changelog et al, but I'd like to really try it out :)
Collapse
15: OACS-mode (response to 14)
Posted by Andrei Popov on
Bart, this really does kick ass (pardon my French).
Collapse
Posted by Claudio Pasolini on
Hi Bart,

thank you for sharing your oacs-mode! It looks very useful for oacs developers.

I followed the installation instructions without problems until I tried adding sql mode to XQL files. After copying your code snippets into my .emacs file I get this error during emacs startup:
Symbol's value as variable is void: xml-mode
I don't know lisp and so I'm not able to debug it: perhaps you can help posting your .emacs file?

TIA, Claudio

Collapse
Posted by Andrei Popov on
You probably are missing psgml or have not set it up correctly.  In the least you should have something along the lines of:

(add-to-list 'load-path "/full/path/to/psgml/dir")

(setq auto-mode-alist
    (append
      (list
    '("\\.xql$" . xml-mode))

in your .emacs file

Collapse
Posted by Bart Teeuwisse on
Claudio,

as http://www.thecodemill.biz/services/oacs/ describes, Emacs OACS depends on the PSGML and MMM modules. Be sure to install them. Installation instructions are included each module. You'll find links to these modules from http://www.thecodemill.biz/services/oacs/.

/Bart

Collapse
Posted by Steve Manning on
Yes komodo can do this. You simply add the commands into the tcl-core.tip file and they appear as auto-complete options and  argument prompts.

    - Steve
Collapse
Posted by Jarkko Laine on

Just an additional note: It's not enough to just install mmm-mode and psgml according to the INSTALL files that come with them (they are generic). At least I didn't get the needed .el and .elc files copied to /usr/share/emacs/site-lisp, so I had to manually make subdirs for both of them in that dir and then copy the files over.

.emacs file also has to be configured so that they get loaded:

(autoload 'sgml-mode "psgml" "Major mode to edit SGML files." t)
(autoload 'xml-mode "psgml" "Major mode to edit XML files." t)

(require 'mmm-auto)

(add-to-list 'load-path "/usr/share/emacs/site-lisp/mmm")
(add-to-list 'load-path "/usr/share/emacs/site-lisp/psgml")
(add-to-list 'load-path "/usr/share/emacs/site-lisp/oacs")
(require 'oacs)
Collapse
Posted by Jay Dubanik on
finally I have installed all the components and attemted to use it.

What I have learnt so far about oacs-mode for emacs:
====================================================

- there are modes and submodes for different file types.
  + for .adp - mode: XML OACS
  + for .xql - mode: XML OACS and XML[SQL] OACS within sql query
  + for .tcl - mode: TCL OACS and TCL[SQL] OACS within sql query

Question that I have at this stage:
====================================

1. Is there syntax highlighting for TCL in TCL OACS mode?

2. there is ADP sub-mode mentioned in change log, How do I get to it? my .adp file is always in XML OACS mode

Collapse
Posted by Bart Teeuwisse on
Jay,

to answer your questions.

1) Yes, just turn syntax highlighting on. Emacs OACS doesn't change the syntax highlighting.

2) Like all files for which OACS is activated, OACS identifies it self as OACS. OACS is a so called minor mode which combines with the major mode of the file being edited. You will that the OACS menu option in ADP files is specific to the ADP sub mode. In short you automatically enter the ADP sub mode when you open an ADP file.

/Bart

Collapse
Posted by Mark Aufflick on
Stuff I just figured out:

you need a bunch of dtd's:

http://www.thecodemill.biz/repository/emacs-dtds.tgz

you need a bunch of SGML stuff in your .emacs:

http://www.thecodemill.biz/repository/.emacs-psgml-snippet.txt

except the dtd line in the .emacs snipped should read catalog instead of dtd.cat.

This got rid of my "external entity not found" errors

Now to get the highlighting more adp-friendly... any ideas?

Collapse
Posted by Bart Teeuwisse on
Mark,

you've gotten rid of the "external entity not found" warning because of the last line in the catalog:

    DOCTYPE master "xhtml1-transitional.dtd"

This tells PSGML to treat all files opened in PSGML mode starting with <master> to be treated as XHTML.

While this gets rid of the aforementioned error, it does have a side effect which might be worse. I found that indenting a region (M-x indent-region or C-M-\) doesn't work well anymore. In particular when the region contains <if> ADP tags.

Should find indenting to be more important than the present of an warning message when you open an ADP file then I recommend removing the last line in the catalog.

On the other hand, the added benefit of the catalog line is that you can now use C-c C-e (and other PSGML key bindings) to insert valid XHTML tags. Of course this doesn't help with ADP tags as the XHTML dtd doesn't know about those.

Please note that you would still get the "extrenal entity not found" warning for ADP files NOT starting with <master>.

In short, there is no perfect solution, which is not suprising as ADP files are not valid XML. But depending on your preferences you can configure a workable environment.

/Bart

Collapse
Posted by Talli Somekh on
How hard would it be to get the AOLserver community to begin considering making ADP valid XML? is it an impossibility? what would be the advantages?

talli

Collapse
Posted by Bart Teeuwisse on
Talli,

ADPs can't always be valid XML. See https://openacs.org/forums/message-view?message_id=158094.

/Bart

Collapse
Posted by Tom Jackson on

Talli, also check this thread on XML like syntax for a templating system. IMHO, actually making ADPs and ATS conform to XML would be unhelpful. XSL achieves it by interpreting its XML tags and rewriting the surrounding XML/text. But it isn't very readable, and this type of process is probably impossible for ADP and ATS processing.

In the above thread I discuss the possibility of a separate templating system with improved properties in certain areas. One area is validation of the template.

Without going to the extreme of XSL, I doubt you can construct an valid XML document that is also a template. The main problem is attribute values, whose value validation probably wouldn't match up in many cases with the variable format. But a template doesn't have to be valid XML to be a valid template, obviously. What would be nice is if the templating system could validate the template while transforming it into code. This is the approach I have taken.

The ATS compiler could be rewritten to do this as well. Essentially you would throw out the adp tag processor and replace it with a top down scanner/parser. The recursive nature of the ATS compiler makes it difficult to understand and extend. Since my new toy is top down, it is very easy to extend. For instance, this weekend I added the following 'tag':

[resource type name args]

# which is converted by the scanner/parser to:
if {[::resource::exists $type $name]} {
   ::resource::eval $type $name $args
}

# ::resource::eval in turn, just runs:
::resource::$type $name $args

So to answer your question, Talli: it may not be impossible, but I don't see any advantages given what the result would look like. And it doesn't appear that the toolkit has room for another templating system, regardless of the properties.

Collapse
Posted by Andrei Popov on

Tom,

I can't see why this:

<?xml version="1.0" encoding="utf-8"?>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
 <title>@title@</title> <!-- keep @var@'s -->
</head>

<body>
  <p>Got a list @list@</p>
  <p>Looping over list:</p>
  <ul>
    <tcl:foreach name="l" select="list">
      <li><tcl:value-of select="l" /></li>
    </tcl:foreach>
  </ul>

  <p>Got an array @array@</p>

  <p>Looping over Array:</p>
  <table>
    <tr>
      <th>Name</th>
      <th>Age</th>

      <th>City</th>
    </tr>
    <tcl:foreach name="a" select="array">
      <tr>
        <td><tcl:value-of select="a(name)" /></td>
        <td><tcl:value-of select="a(age)" /></td>
        <td><tcl:value-of select="a(city)" /></td>
      </tr>
    </tcl:foreach>
  </table>

</body>
</html>

would look much worse than this:

<html>
<head>
 <title>$title</title>
</head>
<body>

<p>Got a list $list

<p>Looping over list:
<ul>
[foreach l $list]
 <li>$l</li>
[/foreach]
</ul>

<p>Got an array $array
<p>Looping over Array:

<table>
<tr>
 <th>Name</th>
 <th>Age</th>
 <th>City</th>

</tr>
[foreach a $array]
<tr>
 <td>$a(name)</td>
 <td>$a(age)</td>
 <td>$a(city)</td>

</tr>
[/foreach]
</table>

</body>
</html>

Naturally, it *is* different, but as stated in the same thread mentioned by you --- there is no need/point in really implementing XSL. Borrowing *some* structures/approachs is all that can make ADPs valid, and hence much easier handled by a variety of tools. For one, you could actually apply a style sheet to them and display them in a browser and/or create printable output, etc.

The <tcl:value-of-select> could be replaced by the current @@ syntax for easier readability, if so required. Or shortened to <tcl:value-of>.

Collapse
Posted by Tom Jackson on

Andrei, I wasn't going for look. I started the other thead with a set of criteria for my templating grammar. I initially thought there might be some benefit to an XML-like look, but alas, it isn't XML, and my purpose was beyond producing XML-like documents.

I believe the main problem with the XSL approach is that in order to place values inside other XML tags, you have to construct the tag, piece by piece. To me this is too verbose and unreadable.

As far as development tools, debuggers, etc. I shouldn't have much problem with my grammer, since I know it is a valid grammar. Debugging is easy: compiling checks the grammar, then you can read the simple tcl script which results to see if there are problems there. Getting Dreamweaver to recoginze the tags will also be easy. I could have separate icons for each language element, so that in design view, you could still tell what language tags were being used. Fortunately the template compiler is essentially complete. The resource tag was the last main requirement, allowing the developer to tailor exactly what is available to the designer. I'm still considering whether to add while and switch/case tags, but the for tag is too dangerous since it executes the first arg.

Collapse
Posted by Rafael Calvo on
Jay
I think having more/better development tools is really important. Emacs does a good job but it is not as good as the tools that come with commercial application frameworks.

The work by Jun with Eclipse:
https://openacs.org/forums/message-view?message_id=156811

had mixed results (too slow), but it would be a great way to go.

I will also have a student looking into using argoUML to generate acs_types and generate OACS code with a modelling tool. It would be great ... but it is ages away.

cheers

Rafael

Collapse
Posted by Talli Somekh on
Rafael,

there was a little chatting in #openacs on irc about CEDET. Perhaps CEDET + oacs.el would be a more complete IDE solution?

talli

Collapse
Posted by Bart Teeuwisse on
Talli,

could you provide a link to the #openacs log discussing CEDET? OpenACS search is failing me. Anyhow, Emacs OACS already works with speedbar. Install speedbar and you can browse SQL functions or TCL procedures w/ speedbar.

Speedbar is not required. An index to function/procedure definitions is already available from the Index menu using stock Emacs OACS.

I personally lack the time to investigate integration with CEDET or any of the other EMACS programmer utilities (http://www.emacswiki.org/cgi-bin/wiki/CategoryProgrammerUtils) such as the Emacs Code Browser (http://ecb.sourceforge.net/). All Lisp hacker contributions are welcome. Just mirror the GNU Archive of Emacs OACS (see http://www.thecodemill.biz/services//oacs/) and hack at it!

/Bart

Collapse
Posted by Torben Brosten on
My vision consists of Bart's emacs OACS mode, an end-user programming environment (like Collaboraid's skinny-spec[1] or integrated spreadsheets), and an end-user-IDE optimizer/compiler (that converts the "dynamicness" to a new, more static and scalable package).

1. http://www.collaboraid.biz/developer/skinny-spec

Collapse
Posted by Alfred Werner on
Two things that look interesting to me from the database side: Xdobry and ERW. Xdobry seems to be a nifty reverse/forward engineering tool with a TK front end. ERW takes an XML formatted ER spec and generates schemas by way of a java processor