Forum OpenACS Development: Reference Platforms and Supported Platforms

1: Reference Platforms and Supported Platforms

Posted by Joel Aufrecht on 03/02/03 02:41 AM

A reference platform is a complete list of software, sources for that software, file locations, settings, and instructions to put it all together. For OpenACS it would include hardware parameters, operating system, database, supporting software, and OpenACS. I would like to get a complete list of platforms that people are using and figure out (and put into the docs) what their statuses are. I propose:

Reference Platform: An "official" platform that is fully documented, supported, and reproducable. This should include the platforms that most of the core contributors use.
Supported Platform: A platform that someone, somewhere, has working and is committed to maintaining documentation for.
Unsupported Platform: everything else?

I have documented a reference platform comprising Red Hat 8.0, PostGreSQL 7.2.3, and OpenACS 4.5 (and a bunch of other details like qmail, daemontools, etc), server root in /web/openacs-dev; log files in /var/log/aolserver; control files in /usr/local/aolserver, etc. For 4.6.x I'm planning Red Hat 8.0, PostGreSQL 7.2.3, OpenACS 4.6.x; and eventually I'll get an Oracle version together. I have pointers to docs for Mac OS X and VMWare (thanks!) which I plan to incorporate. The last version of the docs was Debian. What other platforms do people propose?

Here's what I think goes into a reference platform. what am I missing?

A list of all software, including
- Version number
- licensing information
- where to get it
- file paths for everything
- All of the config files
- all of the necessary accounts
- documentation on how to reproduce it

2: Re: Reference Platforms and Supported Platforms (response to 1)

Posted by Malte Sussdorff on 03/03/03 09:29 AM

I was wondering what should be our common installation locations. I think installation into /usr/local/aolserver ist fairly common, but what about the control files (we store them in /usr/local/aolserver/servers, but wouldn't it make more sense to store them in /etc/aolserver) ?

How about the logfiles. We store them in /web/log/ as they have a tendency to grow very large and the /web directory is normally on a partition of it's own, so I can prevent the AOLserver logs of filling up my /var partitions.

I asked an Oracle DBA at azri.biz to have a look at the Oracle installation process and look into the use of RMAN for backups (online, using archive mode) as well as LVM (logical volumes) using RAW devices. Though this will be a setup most of us are not going to use, it would give us some information how to scale the database backend. And we could make this as part of a reference plattform for highly visible websites, e.g. together with the multiple webserver setup that Greenpeace has for their Greenpeace planet site.

3: Re: Reference Platforms and Supported Platforms (response to 1)

Posted by Lars Pind on 03/03/03 01:34 PM

Joel,

I just want to applaud you for taking this on. I'm really happy to see you on the documentation team.

Only thing I'd add is PG 7.3 and perhaps Oracle 9i for the 4.6 version, though I'm not sure what the level of support currently is for PG 7.3 (I now it's non-existent for Oracle 9i, but Mohan's working on it.)

/Lars

4: Re: Reference Platforms and Supported Platforms (response to 2)

Posted by Joel Aufrecht on 03/03/03 07:28 PM

Control files in /etc/aolserver makes sense to me, but should it be /etc/openacs instead? Files that go there include: openacs-dev.tcl, the analog config file, and the daemontools directory; anything else?

What is the clearest name for the example instance in the documentation? Current docs use either openacs-4 or birdnotes. I used openacs-dev because it goes with openacs-pre and openacs-prod, but it doesn't have much meaning by itself. Can we get new proposal/consensus for the default instance name?

Your /web directory is on its own partition; my /var directory is on its own partition. I don't see either as obviously more right, except that /var/log is pretty standard in unix. Opinions?

The 4.6 docs say that "For security and compatibility reasons [using /web for the web root and running AOLserver as the nsadmin user] is no longer recommended." What are the security and compatibility reasons, and which user should you run aolserver as now? Something doesn't feel right about putting web roots in /home/joeuser/web, at least not for production sites.

5: Re: Reference Platforms and Supported Platforms (response to 1)

Posted by Don Baccus on 03/03/03 08:19 PM

PG 7.3 support won't be in for 4.6 or 4.6.1, however I'm almost done with it for 4.6.2 (including dotLRN 1.0)

If 9i can hit 4.6.2 that would be utterly fantastic, of course, other than the fact that I may feel compelled to upgrade my Oracle server from RH 6.2 and 8.1.6 and installing Oracle just gives me bad vibes ... :)

6: Re: Reference Platforms and Supported Platforms (response to 1)

Posted by Matthew Geddert on 03/03/03 09:04 PM

Files that go there include: openacs-dev.tcl, the analog config file, and the daemontools directory; anything else?

I would be inclined to add the ssl key certificates here, possibly in a subdir (i worked on this last week to it is fresh in my mind)... I know Scott Goodwin has great instructions for installing SSL, but i would think that it would also be good to tell people that aren't used to certficicates how to create the cert and key files for themselves (like this, or by directing them there: http://linuxlab.dk/fcl/technotes/ssl_on_aolserver... and possibly having nsopenssl parameters commented out in the default nsd.tcl file would also help. I would also recommend putting the log files in /var/log/openacs/* if we are trying to use standards...

I too am interested to know why it is a "security risk" to have websites hosted in /web by nsadmin. The only thing i could think of is that using an nsadmin account (i.e. the default), would give hackers a username on your system - but if you don't allow nsadmin to log in remotely what is the problem?

7: Re: Reference Platforms and Supported Platforms (response to 1)

Posted by Vinod Kurup on 03/03/03 09:45 PM

First off, I want to thank Joel for taking over the install docs. It's nice to see it being done in such a thoughtful, organized fashion.

I changed the AOLserver installation instructions based on advice from Pascal Scheffers in this thread.

I'm no security expert, but here's my understanding of the issue:

AOLserver is running as nsadmin
Someone finds an exploit in OpenACS or AOLserver, giving them access to the filesystem.
Since they have nsadmin privs, they can alter the nsd binaries.
They kill the server which usually causes an automatic restart
nsd initially starts off with root privs, so the new binary can do whatever it wants on your system

Now this can be fixed simply by making the nsd binary unwritable by nsadmin. But Pascal advised me that giving the webserver any more privs than it needs is a security risk, so I changed the docs to show AOLserver running as 'nobody', like Apache does. I mentioned these changes in this thread and, not hearing any dissenting opinions, went ahead.

I moved the serverroot from /web to /home/joeuser/web because I was advised (can't remember from whom) that it was more FHS compatible. This easily allows 2 users on my machine to have completely separate OpenACS installations that the other user can't access. I don't think there's any problem with putting a production site in /home/joeuser/web. I do it

8: Re: Reference Platforms and Supported Platforms (response to 1)

Posted by Jerry Asher on 03/03/03 11:24 PM

Echoing Vinod, given the limited flexibility of aolserver config files, I think it is a mistake to put them into /etc.

I place most aolserver things into ~/aol<version> and most openacs things into ~/openacs<version> and then I cons a new user for each aolserver/openacs installation.

9: Re: Reference Platforms and Supported Platforms (response to 8)

Posted by Joel Aufrecht on 03/04/03 02:59 AM

If I understand, the security problem is that the user under which aolserver runs should not have write access to the binary. I'm uncomfortable with the "joeuser" setup because I don't think that accounts used for routine login should also be running services. I also don't think we should have any instance stuff in /home. What about this:

a dedicated, nologin user for each instance.
Each instance has a directory in /web/instance-name which is only readable by the user
All instance-specific config files, including (instance-name.tcl, ssl subdir (ssl instructions, including generating self-signed certs, are being integrated into the install doc, by the way), analog config, daemontools subdir), go in /web/instance-name/etc

10: location for nsd.tcl (response to 1)

Posted by Andrew Piskorski on 03/04/03 07:58 AM

Malte, the nsd.tcl (or whatever you happen to name it) AOLserver config file for a particular site should be in your site's CVS! I usually put mine in either "/web/mysite/nsd.tcl" or "/web/mysite/nsd/config.tcl". This also makes sharing the same config file between Dev, Staging, and Production servers particularly effective.

Some AOLserver modules (at least some versions of nsopenssl, I think) seem to insist on having a "servers/mysite/modules/" directory beneath the AOLserver install directory, but I never keep anything in there at all.

Your AOLserver nsd.tcl config file is integral to the functioning of your OpenACS site, and isn't needed by anything else, so I never understood why people often put it in some other entirely different place (e.g., somewhere in the AOLserver install directory), often not even under version control at all.

11: Re: Reference Platforms and Supported Platforms (response to 1)

Posted by David Walker on 03/04/03 08:10 AM

You are probably right that nsd.tcl should be under source control. I have mine located outside of the CVS root because I put settings into it telling it that it is a dev or live server and such.

12: Re: Reference Platforms and Supported Platforms (response to 1)

Posted by Jun Yamog on 03/04/03 05:38 PM

Not sure if this is helpful or not. What I have done for infiniteinfo is something like this:

Each site has a single user for it. Say "Foo Inc." has a site, so the server has "foo" user.

We then have /servers/foo where all OpenACS files are. Each nsd is controlled by deamon tools. foo users can stop, start and restart the service. Also foo user has its own pg database or oracle db. So a full dump of the db will only yield a clean dump in PG. SSH keys are then given to the developers that work on foo project, so foo is passwordless/ no login. We also put the config file in /usr/local/aolserver/config or etc. We also separate the log files in each dir. So each site a its own dir of access.log and server.log. Also the config files are only edited on the top. Much of the config .tcl file are not touched anymore, except in special cases.

This is far from perfect but worked pretty ok on multiple internal developers on a server that has multiple sites. It used to have a shell script in 3.x days, but then I could not maintain it. The above infrastructure was further developed by Hamilton and others.

When I started to use other systems I think moving to /web/foo the log files, ssl, config, etc is a good idea. It s easier to transfer the sites, also easier for a new developer to come in and have all the important files there. What would be good is have each site run a separate user and not the login of the developer.

It would also be good if anyone has the time to make shell scripts to setup things. Its better sometimes to just have a shell script to do things. We normally only need the db user, type of db, user, ip address. I used to have one in 3.x days. CCM also comes with a shell script that make it relatively easier to setup a development environment. I think a shell script to install maybe good for OpenACS. If only someone has time and have good shell script capabilities. I have also seen shell script help the LAMP platform, I help a friend of mine setup one. Its been a while since I touched apache, but just using the shell scripts/make files. It seem to work.

13: Re: Reference Platforms and Supported Platforms (response to 1)

Posted by Joel Aufrecht on 03/06/03 12:26 AM

This is turning into a standardized test problem: given these constraints, how must the paths be organized? :) I think we need two lists, the contraints and the recommendation. Here are all of the constraints I've heard:

The user under which a process runs should not have write access to the nsd binary, for security
Different instances of OpenACS on one machine should each run under a different user, and should not be able to access each other's files, for security for multi-hosted setups
All of the config files (nsd config file, ssl certificates, analog config file, daemontools directory) should be together, for ease of backup and version control
The site content (the five directories in site root, plus a recent database backup file) should be backed up in sync with each other and with matching config files
The error and traffic logs should go in a seperate location so that they can be on their own partition (this begs the question: how does aolserver behave when there is no space on the log partition?)

It seems to me that we can satisfy all of this by doing this (basically Jun's setup):

/web/sitename
             /etc
             /log
             /tcl
             /packages
             /content-repository-content-files
             /www
             /bin
             /database-backup

All of this secured so that only a dedicated "sitename" user, under whose authority the instance ran, had access.

/etc is for config files, including any subdirs that aolserver demands (I too remember nsopenssl failing if it didn't have a particular subdirectory relative to the binary's location - that may require an ugly workaround).

Do we need to move the database data files, too? I don't think so; the backup files are more useful for recovery, so those should go into our tree, but (at least in PostGreSQL) the data directory has restrictive permissions. By the way - since we don't use passwords, how are we authenticating? How are we securing the postgresql install?

14: Re: Reference Platforms and Supported Platforms (response to 1)

Posted by Jun Yamog on 03/06/03 04:25 AM

Hi Joel,

That seems to be a good setup. As for nsopenssl maybe some one can ask ScottG to fix the hardcoded path. Tom Jackson maybe? Based from memory I believe you can put the certs on a different path, but you just have to create the modules... blah blah dir since nsopenssl checks it on startup. Sorry poor memory still have to get back to OpenACS this days.

Putting no password for a user essentially makes the user unable to login unless you are coming from root. So you have to be root in order for you to su to postgres. It is however possible to ssh to a user without a password using keys.

It is really great to have volunteers like Joel moving forward OpenACS in the docs and installation process. Once OpenACS is easy to install its one less reason not to use it. LAMP tends be easier to install even the fact that it has more parts.

15: Re: Reference Platforms and Supported Platforms (response to 1)

Posted by Talli Somekh on 03/06/03 07:51 AM

A reference platform is certainly critical, and IMO we should probably follow the standards that the Linux Standards Base Project is establishing.

Standards seem to only work when people start using them, and since some of the heavy Linux distros are buying in it would be great to be able to take advantage of common file arrangements.

talli

16: Re: Reference Platforms and Supported Platforms (response to 15)

Posted by Joel Aufrecht on 03/06/03 10:08 AM

LSB discussion continued here. Is there anything in LSB that sheds light on the /web stuff? The only thing I saw in a quick scan seemed to imply that we should put everything in /var/opt/package.

17: Re: Reference Platforms and Supported Platforms (response to 1)

Posted by Chris Johnson on 03/10/03 07:44 AM

New aolserver-4.0b2 gentoo package available.

Please see my discussion of the way my package implements a similar strategy re: LSB/FSH and security as does apache in gentoo linux.

My opinion on /web etc.: up to the admin. Aolserver probably belongs in /home/aolserver, but your web root can be in a) /home/aolserver/web/instance or in b) /usr/local/aolserver/instance or in c) /web/instance... I'll probably stick with a) or c). =) --Chris

18: Re: Reference Platforms and Supported Platforms (response to 1)

Posted by Joel Aufrecht on 01/06/04 06:22 PM

A list of all software. For 5.0.0: Aolserver 3.3oacs1, PostGreSQL 7.3.4, Oracle 8.1.7
file paths for everything.
All of the config files. Included in the standard tarball in /etc.
all of the necessary accounts All specified in the install doc. For 5.0.0: postgres, service0
documentation on how to reproduce it

19: Re: Reference Platforms and Supported Platforms (response to 1)

Posted by xx xx on 01/06/04 09:50 PM

RedHat and OpenACS - Wake Up Call: Support for RH 7.x to 8.x will terminate on Dec 31, 2003 and for v. 9.x on April 30, 2004

20: Re: Reference Platforms and Supported Platforms (response to 18)

Posted by Andrew Piskorski on 01/07/04 02:27 PM

Joel, lots of excellent information you have there.

However, I think you have one mistake in your "Version Compatibility Matrix". You list AOLserver 3.5.5 as "Verified" for OpenaCS 4.6.2 and 4.6.3, but AFAIK no stock AOLserver 3.5.x version include the ArsDigita-derived character set support required by OpenACS 4.x

I remember when the character support was discussed on the AOLserver list and on openacs.org, and AOLserver team's decision was to skip the work necessary to add it into any AOLserver 3.5.x point release, and instead to put it into 4.0 (which they did). So unless somebody changed their mind about that, I don't see how AOLserver 3.5.5 could be fully "verified" for use with OpenACS 4.6.x. There might be a patched version of 3.5.x around somewhere that has the necessary character set support, though.