Forum OpenACS Development: RFC: automated installer

Janine,

That is a tough problem. One I have been working on for a while. I actually have a tcl based installation script, currently tagged 'app-install', which installs any software that can be installed via a command line.

Tcl is good for a number of reasons, not the least of which is that it _stops_ when it reaches an error. Shell scripts tend to keep chugging along. Currently my script runs through a loop for each application:

• Query the user to set variables, using a default if available, i.e. install locations, usernames, etc.
• Run a series of commands. If a command fails, prompt the user for advice: abort, skip, substitute command.
• Write/append/permission files.

I ran through the entire 5.0 install doc and used the defaults provided for a postgresql install. I found a number of errors in the docs by doing this.

I have to admit it is a bit tedious to setup, but it does work. All the commands which are actually executed are recorded in a log file along with any output from the commands.

I have command scripts for the following:

• Download/install tcl
• Download/install daemontools
• Download/install aolserver
• Download/install aolserver-modules (including tDOM)
• Download/install postgresql
• Download/install init.d script for postgresql
• Create OpenACS startup scripts -- nsd.tcl, db user, createdb.
• Download/install OpenACS software.
• Setup daemontools control directory for service.
• Initialize cvs repository.

Usually all that is left is to start up the service:

$ ./bin/service-run

Then install OpenACS by filling in the form.

Each application is independent, but they can be chained together and variables set previously default to the already entered values.

The main issue I need to address is to switch over to using sudo. Some commands were executed 'su'. In general it would be nice to use sudo on every command, and allow switching users. Then some configuration of the sudoers file would be required to run the script.

Example script files are at http://zmbh.com/app-install/. The .rhf files are the configuration files. The application is included with the rmadilo package: http://zmbh.com/rmadilo/.

Sounds very useful!

I'll second what Tom said about the importance of the script stopping on error.  We don't want the user to wait on the whole install process only to find out that the script bombed in step one.  Also makes it easier to find the source of the error if it is the last line. :)

When you get to the point of having something ready to test, I can volunteer a fleet of disposable RH7, RH9, and Debian vservers ready to be messed up and deleted.  Takes me about 10 minutes (two command lines and some waiting) to spawn a vserver, and it can be totally goofed up by shell scripts run amok without any problems. :)

1) tcl/tk would be a good combo to make a combined text or graphical installet - and since we all have to be tcl experts anyway it makes sense to use what we know

2) i like the way fink works (fink is a MacOS X package manager similar to bsd ports)

on OSX I, for instance, type "fink install postgres" - fink makes sure all the dependancies exist, downloads the source, downloads os X specific patches from it's own server, does the magic and compiles/installs the whole shebang.

3) you need to decide if you are going to use the native platform package manager or not. if so, the way to check dependancies is obvious, albeit different on every platform. you don't have to be locked in to the default packages since you could roll your own, or you could make, eg. an postgres-openfts package that relies on the standard package.

I personally think that it would be best to ignore the package manager since noone will be willing to maintain the up to date packages for every distro.

Instead I would use an autoconf style script to check for the required libraries/versions etc. as a bonus this would give you the paths to everything so you don't need to hardcode specific paths for specific OSs.

http://darwinports.opendarwin.org/docs/ch01s02.html

Darwinports is also another target for where openacs should be able to install from.

The reason Sloan wants to build and contribute an automated installer is to make it easier for a new user with no technical background to download and install dotLRN.  The long list of "stuff" that has to be downloaded from here, there and everywhere and installed is thought to be scaring away potential adopters, who don't have much time to spend on this and feel they lack the skills to get through the manual installation.

The question about using Tcl vs Bourne shell boils down to this:  Is tclsh going to be installed on all systems by default?  All of the different flavors of UNIX I have access to already have it, but that's no guarantee.  I don't know how to find out, without trying each one, whether all the major distros include tclsh in a default install and will continue to do so - any sugestions?

I completely agree that the script should stop on error.  I haven't done much shell scripting lately but it seems like it ought to be possible to trap errors somehow;  on Solaris in particular many installers still use Bourne shell, AFAIK. Is there some reason why this can't be done?

My gut feeling is that it would be better to have one script (or set of scripts) that takes the user all the way to installing OpenACS, as Tom describes above, rather than having one set to do the external package installs and another (the tclwebtest-based script) to do the setup.  However, this would mean there would be some duplication between my script and the exisiting one.  Can someone familiar with the existing script comment on how widely used it is, who is supporting it, and what the future plans for it are?  If no-one is really championing it then it's probably not such a big deal to make it potentially obsolete.

Otoh, you can definitely exit on errors with a shell script

set -e

I would like to see packages for .debs, .rpms, and ebuilds and any of the half dozen other package managers people use.

I suspect the best way to get there would be to have a place in CVS where we keep the source for building with the various package tools (for example there are a set of ebuilds to install all the aolserver stuff + openfts and writing an openacs ebuild would be pretty easy). Maybe we should create an openacs-4/etc/installers/DISTRO directory and under that have the source for generating the various packages/build scripts. If we get them working well then I imagine they will get pulled into more standard distributions.

Another alternative would be to try harder to become the maintainers of aolserver and openacs on various standard distributions. The problem seems to be that the people maintaining aolserver for the various distributions where it is present are not really very good at keeping up to date. I don't know how hard it would be to get commit on gentoo and debian for example, and I don't follow fedora at all.

Anyway, there are a lot of options; I think getting things into standard distributions is important but I don't know how it will happen until some people here get commit to maintain the necessary packages.

http://www.sussdorff.de/oacs5dot2v2.iso

This is the OpenACS Knoppix run out of the box, should not use 512MB of RAM (but might), .LRN 2.0 enabled, Jabber running demo live CD.

We have to work a little bit more on it, but are keen to collect feedback and if you want to get your custom made CD don't hesitate to contact Björn Kiesbye (the author), or myself.

It should be that hard to use a portable TCL solution for an installer I have done with customer projects, successfully, in serveral instances.

After all this discussion I'm still not sure yet what the best way to accomplish that would be.

Unfortunately OpenACS relies on many packages which the OpenACS community cannot demand be maintained, and available in rpm, etc. format. To require this is to say no to an auto installer.

Even OpenACS doesn't have a standard which adequately addresses running multiple independent installations on the same machine.

You only install sendmail once, same with qmail, daemontools, and a bunch of other standard packages. It makes sense to have rpms for these singleton packages. However AOLserver and OpenACS and PostgreSQL are often installed in multiple locations on the same machine. Plus you have to address the possibility that a user might not have root access to their machine and might want to install OpenACS with their own privileges.

Even if rpm packages existed, you still need to maintain an installation document to explain how to do the install. Users still have to find and deal with the same issues. What is needed is a tool to combine installations of multiple independent software products.

Also, note that it isn't rocket science to install any individual package. Most of the time is spent figuring out what options to use to ./configure, how to do the post install configuration, etc. The problem is in glueing the packages togeather (plus finding the up-to-date packages is a pita). OpenACS installation itself couldn't be easier, and with 5.0 it seemed to get even easier.

It is possible something as simple as 'configure' and 'gmake' could work. Some AOLserver modules have gmake targets to create release version of the module, so it is obvious it can be used to perform any command line task, handle dependencies, etc.

first create all the rpms for the supporting software. (tcl, tdom, nsd-*, aolserver) these packages would all have the appropiate dependancies with each other.

then create the openacs rpm package (again with the appropiate dependancies) This package will not create an installed instance of openacs but create what unix sys-admins refer to as a skeleton directory as well as supplying a bunch of installation-generation scripts.

these scripts would then be used to run the useradd command (or equiv) to create the openacs user, database, directory structure.

the admin would then modify the site specific .conf file and WHamm0! they have openacs installed.

to the admin this could be made extremely easy if we made all the packages avalible from a yum or apt4rpm repo.

this would be the overall view from the admin's perspective:

apt-get install openacs

this downloads all the dependancies, and installs them

acsinit testinstall
vi /srv/openacs/testinstall/etc/config.tcl
svc -u testinstall

where acsinit is the propigation script and it sets up the directory structure into /srv/openacs/<username>

any thoughts to this method?

I think it would be a mistake to work on something to just help the newbie. While the tool should support a hands off automated install, it should be customizable in an easy way for the expert. Otherwise the expert will never use it and keep it up-to-date.

What would be the benefit for the expert? A record of how a piece of software was installed so it can be repeated when necessary, both for disaster recovery and service duplication.

The installer should focus on software which traditionally has not been available in an rpm or other type of compiled format, or those which are routinely installed in a non-standard location, or when multiple installations per machine is expected. Of course, an installer which could handle this situation, would easily handle installation of compiled, standard packages.

Whatever the installer, it must have two separate parts: the actual installer, and the configuration file. New applications should only require producing a new configuration file to direct the installation. In the case of configure/gmake, you only have to create the configure script and the makefile, no programming required.

The problem with gmake/configure is, I would guess, most developers here don't spend too much time writing configure/gmake scripts. What they do know is the sequence of shell commands required, plus necessary edits of files.

We do.  It's part of OpenACS, in /etc/install, and instructions to get it and use it are located at https://openacs.org/doc/openacs-5-0-0/openacs.html#install-with-script and linked from the top middle part of the home page of openacs.org.

Does that cover everyone?

Janine,

and pre-determining the location of the openacs directory you would rather allow people to specify it like this:

    openacs-init /var/lib/aolserver/<testinstall>

or

    openacs-init /web/<testinstall>

The reason I suggest this is that the documentation used to suggest /web/ and now suggests /var/lib/aolserver/ and as you have said you use a different directory, I would guess that it is common for people to use whatever directory they see fit. I would also sugest that the command start with the name of the apt-get able package (i.e. openacs). Lastly, it would be nice if it defaulted to installing openacs, but if the script also allowed for other "prepackaged" systems to be installed such as dotlrn. Maybe a command line option for openacs-init.

Another idea would be when you run openacs-init or whatever its called it starts an interactive menu that asks with logical defaults then you don't even need to edit config.tcl. I.E.

what is the servername [default to hostname]?
what is the database name [default to directory name]?
what ip address will this server use [localhost]?
what port will we use for this server [80]?
should we run an ssl connection [Y]?
  - if YES then: what ssl port should we use [443]?
what build would you like (openacs,dotlrn,dotwrk,custom) [openacs]?
  - if custom ask for the install.xml file location
would you like to start openacs now [N]?

openACS gurus could just drop in the needed switches like this:

# openacs-init -d /some/random/dir -u username -g groupname -h localhost:8000 ...

that command would just do the install, populate the needed directories, create the database user, shell user, database etc.
and the new users/noobies could just run:

# openacs-init

And they would be walked through all the questions interactively with all the defaults shown. Then everything would be done.

I plan on creating all rpms for all of the above distros as well as testing the install script/installer I may include mandrake if I see a call to support that distro

I can add a vm for the following distros, however I'm not familiar with creating packages.
Debian
Gentoo
Slackware
{free,open,net}BSD

if anyone is interested in developing packages for these distros, please contact me mailto:derek@gurulabs.com and I'll spool a vm and give you access

derek