Today packages generally have this directory structure, which is understood by the bootstrap code and the request processor:

/tcl (read by bootstrap code)
/www ("/package-key/foo" is mapped to "/package-key/www/foo" rp requires user have read priv)
/www/admin (request processor requires user have admin privs)

One issue is that currently packages put .gif, .jpg, .png and other image elements under various /www directories.  An issue for performance is the fact that references are then permission checked by the request processor, which typically isn't necessary.

Sloan created a global /graphics directory which is then not checked by the request processor, but this doesn't allow APMs packages by third parties to take advantage (as their files are all supposed to be in their package directory tree)

My proposal to fix this is simple: define "/www/resources" to be a directory that isn't permission checked by the request processor.  We already have the "/www/admin" precedent for treating a subdirectory of "/www" specially.  Package could then stash their little left and right arrow .gifs and the like there knowing that they'd be available whether or not the user's logged in and without any permission checking overhead.

The other issue concerns templates that are meant to be included, rather than referenced by URL.  For instance file-storage has a template named "folder-chunk" under its /www directory.  Now ... if permission checking is done by the caller of an includable template rather than the template itself, a security hole is introduced (potentially at least.)  Outsiders can call folder-chunk directly, in this example (though I think it does the proper permission checking, perhaps I should check!)  One way around this: put the template someplace where the request processor won't see it.

My suggestion is simple: /templates under the package key.  If an includable template is stuck there no one can reference it with an HTTP request, so relying on the caller doing security checking is no longer a security issue (if it is done properly, of course!)

So my suggestion is that we support the following structure:

/tcl
/www
/www/admin
/www/resources
/templates

What do others think?  I'd make the simple request processor change needed for /www/resources shortly.  Of course current packages would still work fine with files stored as they are, but I'd anticipate people would start taking advantage of the /www/resources directory to avoid permission checking quite soon, particularly in .LRN ...

Those are some pretty common problems, good to address them at this level. What will be the structure of the /www/resources and the /templates directories, I'm assuming you need to duplicate directories under /www?

I wonder why the choice of .adp was used for template files in the first place. It seems too late to do anything about that, otherwise the rp could just refuse to serve files ending in something like .tpl, or whatever.

Right master templates ought to go there too, hadn't thought of that.  If no one had brought it up I would agree with Jun's suggestion of "/includes" but now that's no good, sorry :)

I think it will be clear enough ...

".adp" was originally chosen by the AOLserver team, and the acronym is AOLserver Dynamic Pages I believe.

I'm just putting it into HEAD.  I'd already sold Sloan on this idea so we could undo the global /graphics directory kludge ...

There will be a 4.6.4 but probably no .LRN 1.0.2 because it's full steam ahead on 5.0/2.0 ...

If folks feel strongly about backporting we could do it but taking advantage involves a fair amount of CVS munging that's not all that much fun.

Actually I've just put this into HEAD and munged the dotlrn package (not all of dotlrn, just the main dotlrn package!) to take advantage.  One immediate benefit is that we can now use the real .LRN logo when displaying the login page to visitors who aren't logged in ...

Don, you forgot to mention the www/doc special directory.

In fact, how about making the /packages/package-keywww/resources directory available as /resources/package-key/, just like we do with doc, so that you can rely on a fixed URL for your graphic, regardless of whether and where any packages are mounted.

That would be really useful for master templates, portlets, and other renderings, that don't necessarily know where the package is mounted.

/Lars

Are the security checks not going to take place in /resources even if the connection is secure via https? (my vote is that it never check permissions even if it is a secure connection). Currently RP treats permissions on secure connections differently then insecure ones. i.e. the public could not see the graphics in my /graphics directory on the https://site.com/register page without a hack to the  ad_login_page proc (found in packages/acs-tcl/tcl/security-procs.tcl file), though they could see it when on straight http:// ... is there a reason why the request processor treats permissions differently if you are connected via https?

Timo - I agree about putting the default template in acs-subsite.

Lars - after looking into things after our discussion earlier today why can't you just use "resources/foo.gif" and ignore the url to the mounted package?  If for some reason you *really* need a URL to the package that is guaranteed to work, why not use [ad_conn package_url]?  I don't know why the dotlrn package was written to reference /dotlrn in hardwired fashion, but changing to "resources/foo.gif" works just fine...and packages shouldn't be cross-referencing small graphics buried in other packages (other than our current kludge of using acs-subsite/www/shared, something we can fix as we get time)

I'm also adding a suggestion made by Jeff Davis: disallow execution of scripts in the unchecked resources directory.  Actually I'm only allowing one to reference .png, .gif, .jpg and .css files - I suppose we could parameterize this but is it necessary?  Packages need to be able to depend on a set of extensions being guaranteed to work and parameterizing the list would allow the site admin to break everything.

We also discussed using ns_returnfile to quickly return such content without going through a bunch of request processor codef.  I've reviewed the code that's executed when the request processor is in performance mode, and the path through the code to the ns_returnfile used to return non-script files is quite efficient and I don't think any special treatment is necessary.

i echo matthew's question about authentication under https - Don?

If it is the case, that's great, because it's getting annoying having to code in the kludge every time i setup a secure site.

Two points:

1) Since the special feature about /resources is that it's not permission-checked, could the name reflect that?  What about /public-resources or /public?

2) Can we make this the first TIP?  (Or OIP?)  One quick-and-dirty way to support the OIP process technologically might be a new forum.  Anybody can post, first post in a thread must be a full, formal OIP proposal; then the thread has a week or two (or whatever is in the OIP plan - we should link that in the forum title) to accumulate yes and no votes from the Core Team.  I'm sure this process has many drawbacks compared to a new custom package or a special version of Bugs, but are any of those drawbacks fatal?  If not, I hope we do this or something similar to get momentum in a formal OIP process.

Good point about .xql files - it is easy to get the request processor to not serve them and you're right, we shouldn't.  I don't want to move them all, though, it's very convenient having them in the same directory when developing/bug fixing.  And there are a lot of them, moving them would be a PITA.

As far as the TIP process goes I guess informally we've kinda done this as Jeff, Lars and I talked this over in private and we're also talking about it in public.

We really do need to get moving on the governance stuff, everyone seemed to run out of steam ...

Realistically, is there any time a developer won't have ssh access to their server?  I mean ... not only do you *look* at your query files, you edit them...

...and if he doesn't, he should probably use developer support.

Which reminds me: Why is the db query analysis disabled by default in dev-support? Isn't it the single most useful feature in the package? Quoting Peter: "Why would anyone use the package if not for that?".

Never mind, I hardly use it myself, but at times it is faster to look at the queries in the browser than to use ssh. ...just add (-oracle/postgresql).xql to the page (file) your looking at to find out if there is a related xql in /www...

I really think we should do this as a TIP.  It's simple and not very controversial, so we can focus our argument energy on process instead of content. :)  We're all waiting for one of us to get around to implemented a more complete system.  Meanwhile, we could go ahead and do it in a dedicated forum.  It wouldn't be perfect, but it would let us practice the organizational issues.  It would also uncover hidden requirements for a more complete technological solution.  If someone on the core team agrees, would you please set up the new forum?  The last available draft of the OpenACS TIP thing should be linked from the description or charter, and a summary of this thread could be the first TIP.

Dave. I think that's a good approach but not necessarily Joel's point, which I think is that we should take this oppotunity to implement or flesh out the governance proposals.

I think everyone has trust in Lars, Don and Jeff, but the fact  much of the discussion took place in private is because we hadn't worked out a lightweight governance bureacracy for deciding these issues.

Don called for reigniting the governance discussion, so I'll do so in another thread.

talli

Revised proposal: /tcl /www /www/admin /www/public-resources /template /test

• resources renamed to public-resources
• automated tests move out of /tcl to reflect the fact that they may be broader and more general.

We want to incorporate tclwebtest into the automated test framework so we can do page testing as well as API testing.

One more quick question. Don, did you create the acs-resources package mentioned above?

Thanks.

Randy

Thank you, Lars.