Forum OpenACS Q&A: Assertions in OpenACS

Posted by Kolja Lehmann on 11/30/03 08:07 AM

I have just recently begun using tclwebtest to set up testsuites for the project we run at the moment, and now I'm fascinated of the idea of getting some quality into my software

So I wanted to use assertions in the code to be sure, certain assumptions are right. I wrote up a little proc that evaluates an expression and if that is false, returns an error message to the user and sends some explanations to the log and (optionally) as a mail to the programmer.
Has anybody some more ideas what to include there or interest in the code? Maybe it could be included as a utility proc, I think it's useful.

2: Re: Assertions in OpenACS (response to 1)

Posted by Randy O'Meara on 11/30/03 07:22 PM

Kolja,

There is, in general, great interest in anything that will contribute to higher quality code in OACS. There has been interest expressed in incorporating a tclwebtest core UI test capability in 5.1 that will supplement the existing automated (tcl api) test capability.

Randy

3: Re: Assertions in OpenACS (response to 1)

Posted by Don Baccus on 11/30/03 11:13 PM

One thing we do that diminishes, at least, the need for assertions in the code is to enthusiastically embrace SQL constraints in the datamodel, and triggers to maintain various relationships between columns and tables.

Good datamodel design of this sort make impossible whole classes of errors.

A problem with assertions in Tcl is that they're always executed - there's no equivalent of "ifdef" which allows one to turn them on or off (you could have your proc not evaluate your assertion clause unless assertions are enabled, but there's no way to remove the proc call itself).

4: Re: Assertions in OpenACS (response to 1)

Posted by Kolja Lehmann on 12/01/03 04:46 AM

Well but that comes down to
- call the proc
- evaluate one expression (ok, it depends on the sort of test, how long this will take)
- return
in general. And if you are very much concerned about the performance, you can also redefine assert on the production server to return at once. Still normally I think it is better to have a short test which discovers an error you made in your design than saving a few statements and don't discover it.
The code that is executed after an assertion failed doesn't count, because it shouldn't ever be executed. and if it is, it should be worth the effort to find a bug.

5: Re: Assertions in OpenACS (response to 2)

Posted by Simon at TCB on 12/01/03 06:09 PM

Hi there!

If I remember correctly the automated-test package we developed some time ago (see dotLRN test server) contains support for assertions.....

I think....

6: Re: Assertions in OpenACS (response to 1)

Posted by Don Baccus on 12/01/03 10:01 PM

Other thing we do that reduces the need for assertions is to support rather smart datatype checking and custom data validation blocks for both ad_page_contract and ad_form.

Validation blocks are a form of assertion but return errors in a standard way.

Proper use of datamodel constraints, ad_page_contract, and ad_form datatype and validation features will cover the vast majority of checking that's necessary.

The popularity of explicit assertions is inversely proportional to the built-in assurances of the language being used, IMO. Which is why they're more popular in the C world than any other place I can think of ...

Certainly use assertions when you think they're useful but only after you've properly designed your datamodel and have used the safety features we provide with tools like ad_page_contract ...

7: Re: Assertions in OpenACS (response to 1)

Posted by Peter Marklund on 12/02/03 11:09 AM

Kolja,
I think it's great that you are getting involved in tclwebtest testing! Here are some tips from my experience so far:

1) Make sure you have the latest tclwebtest sources from CVS as a number of bug fixes and improvements have been made. We are continually working on improving tclwebtest and everybody in the OpenACS community is invited to participate in this.

2) Please see if your tests can fit in with the suite of tclwebtest tests that you can find in openacs-4/etc/install/*.test and use the support API that I have developed in opeancs-4/etc/install/tcl/*.tcl. The most extensive test suite so far is in openacs-4/etc/install/dotlrn-demo-data-setup.test.

3) The acs-automated-testing package has assertion methods such as aa_equals, aa_true, and aa_false. For a good example for how to write Tcl API tests, see the test cases in the file in the directory openacs-4/packages/acs-authentication/tcl/test. We now have convenient support for rollbacks in our tests and we've already seen a lot of benefit from the Tcl API tests that we have written. So far our tclwebtest tests have been running outside OpenACS. However, once we have AOLserver 4 the plan is to make tclwebtest execute from withing OpenACS and integrate it with the acs-automated-testing package. That way we can achieve the goal of having one canonical page to go to in OpenACS for the test status of the system.

4) I have developed a crawler that can visit all links under a certain URL. Take a look at openacs-4/etc/install/crawl-links.test.

Let me know if you need commit rights for any of the software mentioned above.

8: Re: Assertions in OpenACS (response to 1)

Posted by Peter Marklund on 12/02/03 12:24 PM

One thing I forgot to mention is that you need to obtain the install scripts from CVS HEAD rather than from the oacs-5-0 branch as they are not being maintainted on the branch. Here is how I check out the install scripts for a server called myserver

cd /web
cvs -d :pserver:anonymous@cvs.openacs.org:/cvsroot checkout -d install-myserver openacs-4/etc/install

After that you edit the install.tcl file and then run install.sh as root.

9: Re: Assertions in OpenACS (response to 1)

Posted by Michael Bryzek on 12/03/03 07:23 PM

We modified ad_proc to understand preconditions to method execution through a @pre comment (ala icontract - http://www.reliable-systems.com/tools/iContract/iContract.htm)

This lets us test assertions on entry into a given tcl procedure on our development servers while completely avoiding the assertion in production.

Example:

ad_proc foobar { zip radius } {
  @pre [empty_string_p [vs_validate_zip $zip]] 
  @pre $radius >= 0 
} {
  
}

On our development servers, we parse out the preconditions from the documentation for the procedure and add assertions to the body of the tcl proc itself.

10: Re: Assertions in OpenACS (response to 9)

Posted by Jeff Davis on 12/03/03 07:33 PM

can you submit a patch for this. It sounds extremely useful.

11: Re: Assertions in OpenACS (response to 1)

Posted by Don Baccus on 12/03/03 08:58 PM

Yes, I like this! Is there any reason you didn't implement post-conditions as well?

12: Re: Assertions in OpenACS (response to 1)

Posted by Michael Bryzek on 12/04/03 09:47 PM

Regarding a patch: We forked a long time ago (originally ACS 2.x). I'm happy to send you the code we use if you want to create a patch for OpenACS.

A couple other "additions" to ad_proc which may be of interest:

1) -private flag which acts like a protected method in Java using TCL namespaces, and which is only enforced in development servers:

ad_proc -private bryzek::test {} {

}

## This will throw an error since we are calling it 
## from the global namespace
namespace eval :: bryzek::test

2) a -cache flag with parameters request|precompute. This makes it easy to evaluate a procedure on server startup permanently caching its value, or to cache the value for the duration of a request.

If you are interested in the code, pls let me know - I have a feeling it will be easier to chat for 5 minutes and then reimplement this code in OpenACS rather than to try to reuse what we developed.

Regarding post conditions - We didn't implement them because we didn't have a real need at the time. I can see their use, and it should be relatively simple to add. That said, adding support for preconditions was simpler as we did not have to worry about exceptions in the body of the procedures.