Forum OpenACS Improvement Proposals (TIPs): TIP #143 (Approved): Make Tcl 8.5 a platform requirement for OpenACS 5.6+

OpenACS and, in particular, the acs-core development have been tied to
the Tcl 8.4 release series for more than a decade now, starting with
OpenACS 3.2.* in 2000/2001. This dependency is not adequate anymore,
given that the 8.4 series was declared discontinued from active
maintenance by the Tcl core team (TCT) two years ago and major OS
package distributions are about to default to Tcl 8.5 (e.g., Debian,
Ubuntu, macports, ActiveTcl). The purpose of this TIP is to change
the OpenACS dependency requirement to the current, stable 8.5 release
series of Tcl (i.e., 8.5.8 at the time of writing). This change takes
effect with the pending 5.6 release of OpenACS.

<h1>1. Rationale</h1>

Advertising compatibility with Tcl 8.4 *only*, as well as recommending
8.4 for non-core package development and strictly requiring 8.4 for
acs-core development appears inappropriate and obsolete due to the
following reasons:

<h4>Tcl 8.4 was discontinued from routine maintenance starting with
8.4.19 by the TCT in April 2008
</h4>

"Routine maintenance of Tcl/Tk 8.4 has now come to an end. We intend
Tcl/Tk 8.4.19 to be the final 8.4.* releases. The current stable releases of
Tcl/Tk are 8.5.2, and development has begun on Tcl/Tk 8.6. It is possible
that a bug fix of great enough importance or severity might prompt a
release of 8.4.20, but there is no plan for that at the moment. Even
that level of support will fade away quickly."

see http://sourceforge.net/project/shownotes.php?release_id=592164

The maintained series is 8.5, feature additions and enhancements go to
8.6. Therefore, bug fixes related to our platform stack end up
directly in the 8.5 branch and its releases while backports to 8.4 are
not guaranteed and will be most likely never released. Even if
backports from 8.5 and 8.4 are provided, lacking further 8.4 releases,
OpenACS deployers and platform operators will become increasingly
responsible for fetching the amended sources from Tcl's CVS and
provide source installations on their own. Tcl package distributors
(e.g., Debian Tcl/Tk package maintainers) do not monitor unreleased
CVS sources and are reluctant to apply selected fixes to upstream
sources (for good reasons).

<h4>8.5 as standard in bundled/packaged Tcl distributions </h4>
Debian and Ubuntu already ship Tcl 8.5 as optional Tcl installation
since its last stable release (Lenny) while they default to
8.4.16. The Tcl default, however, will change to 8.5 with the upcoming
stable Debian release (Squeeze).

see http://qa.debian.org/developer.php?login=pkg-tcltk-devel%40lists.alioth.debian.org

Besides, there is substantial pressure to switch the default to 8.5
even earlier for the staging distributions of Debian, i.e., in the
unstable distribution. This means that distributions derived from
Debian unstable, such as Ubuntu, will be affected earlier; and so will
our Debian and Ubuntu packaging efforts.

see http://wiki.debian.org/Teams/DebianTclTk/SqueezeReleaseGoals

macports already tracks the 8.5 release series:

see http://trac.macports.org/browser/trunk/dports/lang/tcl/Portfile

The Windows installer advertised on http://openacs.org wraps up Tcl 8.5.7.

More generally speaking, packaged and bundled Tcl distributions (such
as ActiveState) already deliver 8.5 as the recommended or default
version. This substantially decreases the visibility of Tcl 8.4,
contributing to the next issue.

<h4>Tcl 8.4 usage decreases, increasing the cost of maintenance</h4>

The 8.4 series becomes less and less used, especially for the
occasional and "speculative" OpenACS developer (just think of my
students in our Tcl courses). 8.4 disappeared from
http://www.tcl.tk/software/tcltk/, standard package distributions set
out to switch over to 8.5. As a consequence, many info sources (man
pages, the ActiveTcl User Guide) default to 8.5-specific
documentation. While forward compatibility (8.4 -> 8.5) has not been
an issue in the Tcl tradition, backward compatibility (8.5 -> 8.4)
certainly is. A developer working with 8.5 has to check for every
command its whereabouts (Is it available in 8.4? Does it have the same
options?). Practically, this means to have two different Tcl version
installed, leading to confusion for novel users and installation
chaos.

<h4>Using 8.5 features in the acs-core</h4>

8.5 comes with a suite of new features which might turn out
advantageous for our acs-core development:

a) the so-called expand {*} operator which helps avoid the unnecessary (and
apparently unsafe) [eval] to unbox list-protected dispatch statements
or variable argument vectors (e.g., args)

b) [dict] adds maps as Tcl values, avoiding the back-and-forth
transformation of Tcl array into and from lists. There is also an
phalanx of convenient (and optimized) manipulation commands for this "data type".

c) improved and generalized Tcl channel handling and introspection
through [chan]

d) scriptable functions to be used in [expr] statements

e) improved list handling, e.g., the "in" and "ni" [expr]-operators
for list searches, many byte-code compiled functions

see http://wiki.tcl.tk/10630 for a more or less complete 8.4->8.5
comparison

<h4>Testing both for 8.4/8.5 compatibility as well as testing under 8.4
and 8.5 is unrealistic</h4>

Given the above drivers towards 8.5 and given the lack of resources
for testing, it appears unachievable for me to test for 8.4 AND 8.5
compatibility, especially if 8.5 code contributions are
encouraged. Requiring Tcl 8.4 discourages contributions from
institutions working with Tcl 8.5. Therefore, the 8.4
compatibility requirement should effectively be dropped asap.

<h1>2. Proposed changes</h1>

* Explicitly announce the new Tcl 8.5 dependency for 5.6 (news
announcement, ChangeLog, compatability matrix, announcement to the
developer forum)

* Bump package and binary dependencies in our Debian and Ubuntu
packages for the 5.6 release

<h1>3. Implications</h1>

Different 8.5 versions (including the most recent 8.5.8) have been
successfully used to host runtime environments for OpenACS releases
prior to the upcoming 5.6 release, including https://openacs.org/ itself
(!). So far, no issues related to 8.5 as such have been brought to my
attention.

Regarding pre-release testing (by OCT members, test instances), my
uninformed guess is that the pending 5.6 release has been as
thoroughly tested on 8.5 as it was tested on 8.4 ;) openacs.org has
been running on Tcl 8.5.* for more than a year without any flaws. At
Vienna, we changed all our installations to Tcl 8.5 quite some time
ago.

Why should we consider scheduling this change already for 5.6?

Well, our release cycles become longer and longer (or, at least, they
are volatile targets). So given the hard drivers towards 8.5 outlined
above, we might lack windows of opportunity here. In my opinion, such
a change must be communicated with a major OpenACS release. In short,
requiring Tcl 8.5 will remove some of the dust. It will also make code
maintenance and the life of a distribution package manager (=Héctor,
me) easier.

Approve ... though I think the TIP is too short and that stefan could've made it 10,000 words long, at least :)
While Tcl 8.4.* crashes (kills the aolserver) when the C-stack size (parameter stacksize in the config file) is too small, Tcl 8.5 gives a Tcl runtime exception "out of stack space". Therefore, Tcl 8.5 adds stability to the server ... one more reason to use Tcl 8.5 instead of 8.4.