Forum OpenACS Development: XML-RPC package ready

Collapse
Posted by Vinod Kurup on

I've been working on getting Blogger and MetaWeblog API support into lars-blogger and figured my first step should be to create a package that just takes care of the XML-RPC part. I've done this by taking ns_xmlrpc, updating it to use tDom (well actually, to use the XML abstraction procs in acs-tcl) and adding documentation and some functionality. You can preview the documentation to get an idea of what the package is about. Before I commit, I got a few questions.

  1. Should I wait until 5.0 is released before I do anything?
  2. If I commit now, should I commit to contrib? I'd like to commit it to the main distribution because I want lars-blogger to depend on it. The XML-RPC package needs to be installed in order for lars-blogger to accept Blogger & Metaweblog requests and secondly, the XML-RPC package will have a XML-RPC client in it as well, which lars-blogger can use send out any XML-RPC calls (i.e. the weblogs ping)
  3. Should the XML-RPC server default to be enabled or disabled? It's somewhat of a security risk, since the Blogger API and MetaWeblog API accept passwords in clear text. I would vote for disabling it by default and adding a link to lars-blogger's admin pages to turn it on. Not all users will want to use it.

I'm also just about done with the Blogger and MetaWeblog API support. This involves adding a bunch of TCL procs to lars-blogger. Is it OK to commit that, or should I wait til release is done?

Finally, I want to add RSD support to lars-blogger, which would allow blogging tools to automatically find out which APIs are supported and where they are located. This simply means adding a link tag, just like we do for the RSS file.

Any opinions?

Collapse
2: Re: XML-RPC package ready (response to 1)
Posted by Ola Hansson on
I think the documentation is so good I'll actually be able to understand this here XML-RPC, and use it.

This is a great addition to 5.0 (or 5.1), thank you Vinod!

Do you know if it will work to serve requests from Jerry's nsvhr/nsunix virtual hosting?

Collapse
3: Re: XML-RPC package ready (response to 2)
Posted by Vinod Kurup on
Thanks Ola!

I think this should work fine with nsvhr/nsunix as long as the XML-RPC client that you use sends an appropriate Host: header. This was a problem with the client inside ns_xmlrpc, since it used util_httppost to POST requests and util_httppost doesn't send Host: headers. So, I rolled my own HTTP POST method (well, stole it from lars-blogger). I should probably fix util_httppost too...

Collapse
4: Re: XML-RPC package ready (response to 1)
Posted by Andrew Piskorski on
Vinod, that sounds excellent! It should be very useful to have xmlrpc supported in OpenACS.

However, are you sure that ns_xmlrpc was is best foundation to build on? I had lots of problems and failures during some initial experimenation and testing with it back in July 2002. Besides just plain nasty bugs, I remember ns_xmlrpc didn't do any automatic converson between XML-RPC and Tcl data structures at all, even though that's obviously one of the most useful things and Tcl xmlrpc package like that could do. Pretty kludgy.

At the time, people said that TclSOAP was definitely the better (best) tool to use for XML-RPC from Tcl or AOLserver, which I think has used tDOM all alongfor its's XML stuff. I never tried it however, nor do I know which of ns_xmlrpc's old failings (besides the bugs, presumaby) it improves upon.

Maybe ns_xmlrpc was improved since then though? Is it all working to your satisfaction, or should we be encouraging somebody to still look into using TclSOAP instead?

Collapse
5: Re: XML-RPC package ready (response to 4)
Posted by Andrew Piskorski on
Oh, oops, TclSOAP appears to use TclDOM, not tDOM. They both have Tcl XML APIs of some sort. Apparently, TclDOM is implemented in pure Tcl, while tDOM is some high performance C code and much, much faster. And as of 5.0, OpenACS is already using tDOM for other XML stuff.
Collapse
8: Re: XML-RPC package ready (response to 4)
Posted by Vinod Kurup on
Hey Andrew,

I did look at the SOAP-Gateway package which seems real nice and should be useful for SOAP. I couldn't, in my brief inspection, figure out how to have it do plain XML-RPC, it only took care of SOAP requests.

I'm not sure if ns_xmlrpc was the best foundation to start with, but it was the quickest. I wanted to use tDOM since we were already using tDOM in OpenACS 5, therefore I didn't look at TclSOAP. I fixed a few bugs here and there and it seems to work well.

But, I didn't do the automatic conversion between Tcl and XML-RPC that you mention. I thought about doing it, but I couldn't figure out a clean way to do it. XML-RPC to Tcl conversion is automatic (which is easy since Tcl doesn't care about datatypes). But Tcl to XML-RPC conversion requires the programmer to specify the datatype that he's returning. i.e. return [list -int 23]. Now in some cases, it would be very easy to figure out the datatype from the value and take care of that little detail ourselves. For example, 23 is an int. But what if you're implementing a method that's supposed to return a string, whose value happens to be 23. Our XML-RPC would look at 23, call it an int and return an int. If the XML-RPC client on the other side is using a strongly-typed language, it would be expecting a string and might fail. A similar problem would be trying to differentiate between XML-RPC arrays and structs. In TCL, these are arrays and lists, respectively. But if your proc returns a list with 4 items, there's no way to tell if I should code that as an XML-RPC array of 4 items or an XML-RPC struct with 2 name-value pairs (or even as an XML-RPC string with four words). We'd have to have a way of procs specifying their signature (with datatypes) somewhere and that seemed harder to implement than having the programmer specify the datatype.

Collapse
6: Re: XML-RPC package ready (response to 1)
Posted by Andrew Piskorski on
Hm, is William Byrne's SOAP Gateway package from January 2003 relevant in any way?
Collapse
7: Re: XML-RPC package ready (response to 1)
Posted by Talli Somekh on
The AOLserver folks have promised me that they will be releasing some XMLRPC code they wrote on top of tDOM. Considering the quality of their work which (judging by their webdav code which we built on) is very good, I hope this can be a good solution to your worries, Andrew.

I've been waiting for a few weeks to see the code because the AOLserver folks are overwhelmed and want to give the code a lookover before they release it. Hopefully it will be soon, though.

talli

Collapse
14: Re: XML-RPC package ready (response to 7)
Posted by Andrew Piskorski on
Hey Talli, did the AOL folks ever release that tdom-based xml-rpc code you were talking about above, back in Oct. 2003?
Collapse
9: Re: XML-RPC package ready (response to 1)
Posted by Tilmann Singer on
Since we have branched now, you could commit it to HEAD. Or did you already put it up somewhere and I missed it?
Collapse
10: Re: XML-RPC package ready (response to 9)
Posted by Vinod Kurup on
Thanks for the reminder Til - I haven't looked at this stuff since my last post. I'll plan on committing it to HEAD this weekend unless I hear differently.
Collapse
Posted by Vinod Kurup on
I committed the XML-RPC package to the HEAD. I'll get back to the Blogger and MetaWeblog API in the next few days - just working on the automated tests.
Collapse
Posted by Tilmann Singer on
I'm having a problem with the xmlrpc package - for any xmlrpc request it hangs for appx. 30 seconds and then errors out, with this in the error log:
Error writing content: resource temporarily unavailable
    while executing
"ns_conncptofp $fp"
    (procedure "xmlrpc::get_content" line 13)
    invoked from within
"xmlrpc::get_content"
    invoked from within
"if {[string equal [ns_conn method] POST]} {
...
It is mounted on /RPC2 as per the default. Which gets redirected to /RPC2/ by the way, didn't you want to make it a registered proc once to avoid that? But that is not the problem I think - the client seems to follow the redirect correctly. I tried both xmlrpc::remote_cal from another openacs installation and the python client api.

This is happening on AOLserver 3.3.1+ad13 which is running on a non-standard port number, without any proxies involved. One empty temporary file like /tmp/fileddoVAw-562318384.xmlrpc2 is left after each attempt.

Any ideas?

Collapse
Posted by Tilmann Singer on
With aolserver 4.0 final it works fine.
Collapse
15: Re: XML-RPC package ready (response to 1)
Posted by Andrew Piskorski on
Bart Teeuwisse has released an XML-RPC server and client for AOLserver and OpenACS.
Collapse
Posted by Rocael Hernández Rizzardini on
excellent!
Bart, are going to put this into oacs cvs rep?
Collapse
Posted by Bart Teeuwisse on
Rocael,

I have no plans to put the XML-RPC server in the OpenACS CVS repository. I've switched over to GNU Arch (http://www.gnuarch.org) for all my development. Not only is it technically superior to CVS, I also belief that the distributed model fits a project like OpenACS better.

A public copy of my XML-RPC archive is accessible from the Code Mill, see http://www.thecodemill.biz/services/xml-rpc/ for details. In addition, the XML-RPC client will always be available in APM format too. The XML-RPC server is available as a tarball (as it is not an OpenACS package).

/Bart

Collapse
Posted by Bart Teeuwisse on
Versions 1.2 of the new XML-RPC client/server have been released. Version 1.2 simplifies the creation of XML-RPC messages. Upgrading is recommended but not required.

Messages created by a client of version 1.2 are compatible with all XML-RPC servers. Client and server don't have to be of the same version number.

See http://www.thecodemill.biz/services/xml-rpc/ for more details.

An reference implementation of the XML-RPC test suite is available at http://www.thecodemill.biz/RPC2. Point you XML-RPC client to this URL to test your client installation.

/Bart

Collapse
19: ns_xmlrpc (response to 1)
Posted by Andrew Piskorski on
Dossy just announced the client half of the new ns_xmlrpc module he's written (in Tcl) for AOLserver, and he's asking for review and feedback.
Collapse
20: Re: XML-RPC package ready (response to 1)
Posted by Dossy Shiobara on
Thanks, Andrew. The server half is /almost/ done. Just trying to decide if it should use NSV's to store the method-to-proc mapping or not. It probably should ...
Collapse
21: Re: XML-RPC package ready (response to 1)
Posted by Jade Rubick on
I'm curious why Dossy wrote another package when Bart's already existed. Dossy, did Barts not fit what you were looking for, or were you not aware of it?
Collapse
Posted by Andrew Piskorski on
Keep in mind that Dossy's "ns_xmlrpc" module mentioned above is entirely different from the "ns_xmlrpc" code in OpenACS CVS which I tried using a bit back in 2002, and is also distinct from the xmlrpc in AOLserver SourceForge CVS.

There is also now Malte and Timo's tsoap package, in the OpenACS CVS.