Forum OpenACS Development: Re: Win32-OpenACS Version 1.8

Collapse
Posted by Maurizio Martignano on
Dear Gustaf,
about point (b) the function seems to attempt the proper file (i.e. the file name is correct).
What I noticed is that there's a difference between the Postgresql and Oracle caes about how the file names are built.... may be this is the cause of the problem.....

please have a look at the following code:

d_proc -public db_load_sql_data {{
-dbn ""
-callback apm_ns_write_callback
} file } {

Loads a CSV formatted file into a table using PostgreSQL's COPY command or
Oracle's SQL*Loader utility. The file name format consists of a sequence
number used to control the order in which tables are loaded, and the table
name with "-" replacing "_". This is a bit of a kludge but greatly speeds
the loading of large amounts of data, such as is done when various "ref-*"
packages are installed.

@param dbn The database name to use. If empty_string, uses the default database.
@file Filename in the format dd-table-name.ctl where 'dd' is a sequence number
used to control the order in which data is loaded. This file is an
RDBMS-specific data loader control file.

} {

switch [db_driverkey $dbn] {

oracle {
global env

set user_pass [db_get_sql_user -dbn $dbn]
set tmpnam [ns_tmpnam]

set fd [open $file r]
set file_contents [read $fd]
close $fd

set file_contents [subst $file_contents]

set fd1 [open "${tmpnam}.ctl" w]
puts $fd1 $file_contents
close $fd1

cd [file dirname $file]

set fd [open "|[file join $env(ORACLE_HOME) bin sqlldr] userid=$user_pass control=$tmpnam" "r"]

while { [gets $fd line] >= 0 } {
# Don't bother writing out lines which are purely whitespace.
if { ![string is space $line] } {
apm_callback_and_log $callback "[ad_quotehtml $line]\n"
}
}
close $fd
}

postgresql {
global tcl_platform

set pguser [db_get_username]
if { ![string equal $pguser ""] } {
set pguser "-U $pguser"
}

set pgport [db_get_port]
if { ![string equal $pgport ""] } {
set pgport "-p $pgport"
}

set pgpass [db_get_password]
if { ![string equal $pgpass ""] } {
set pgpass "<<$pgpass"
}

if { [string equal [db_get_dbhost] "localhost"] || [string equal [db_get_dbhost] ""] } {
set pghost ""
} else {
set pghost "-h [db_get_dbhost]"
}

set fd [open $file r]
set copy_command [subst -nobackslashes [read $fd]]
close $fd
set copy_file [ns_mktemp /tmp/psql-copyfile-XXXXXX]
set fd [open $copy_file "CREAT EXCL WRONLY" 0600]
puts $fd $copy_command
close $fd

if { $tcl_platform(platform) == "windows" } {
set fp [open "|[file join [db_get_pgbin] psql] -f $copy_file $pghost $pgport $pguser [db_get_database]" "r"]
} else {
set fp [open "|[file join [db_get_pgbin] psql] -f $copy_file $pghost $pgport $pguser [db_get_database] $pgpass" "r"]
}

while { [gets $fp line] >= 0 } {
# Don't bother writing out lines which are purely whitespace.
if { ![string is space $line] } {
apm_callback_and_log $callback "[ad_quotehtml $line]\n"
}
}

# PSQL dumps errors and notice information on stderr, and has no option to turn
# this off. So we have to chug through the "error" lines looking for those that
# really signal an error.

set errno [ catch {
close $fp
} error]

# remove the copy file.
file delete -force $copy_file

if { $errno == 2 } {
return $error
}

# Just filter out the "NOTICE" lines, so we get the stack dump along with real
# ERRORs. This could be done with a couple of opaque-looking regexps...

set error_found 0
foreach line [split $error "\n"] {
if { [string first NOTICE $line] == -1 } {
append error_lines "$line\n"
set error_found [expr { $error_found || [string first ERROR $line] != -1 || \
[string first FATAL $line] != -1 } ]
}
}

if { $error_found } {
global errorCode
return -code error -errorinfo $error_lines -errorcode $errorCode $error_lines
}

}

nsodbc {
error "db_load_sql_data is not supported for this database."
}
default {
error "db_load_sql_data is not supported for this database."
}
}
}

About point (d) the version of Categories I'm using is 1.1.2d3 .

Hope it helps,
Maurizio

Collapse
Posted by Gustaf Neumann on
concerning db_load_sql_data:

Seems as if the following sequence leads to the problem

 set fd [open "|[file join $env(ORACLE_HOME) bin sqlldr] userid=$user_pass control=$tmpnam" "r"]

... close $fd

Do you have the sqlldr under [file join $env(ORACLE_HOME) bin sqlldr]?
Are the provded parameters ok (e.g. the $tmpnam)?

Concerning (d): You are using the head version of categories, where it seems, that the column "widget" was not added in the Oracle version. Although i hate add something to CVS, which i cannot test, i added the column in the Oracle creation procs. Please get the head version from CVS and test it with Oracle. If the error is gone, i will apply the same fix to the oacs-5-4 branch, which has apparently the same problem.

Collapse
Posted by Maurizio Martignano on
Dear Gustaf,
success!!!

1. The first problem was due to $tmpnam (which was wrong: in Vista you can't write files wherever you want).
To fix it I had to introduce the following change:

# set tmpnam [ns_tmpnam]
set tmpnam [ns_mktemp /tmp/ctl-XXXXXX]

2. Your correction to the categories Oracle creation procs worked.

Thanks a lot for your help!

Cheers,
Maurizio

Collapse
Posted by Gustaf Neumann on
Dear Maurizio,

i count current 77 occurrences of ns_tmpnam in OpenACS and its packages. These places will have the same problem, so the fix is not sufficient in the general case. Since ns_tmpnam is implemented in C, it is most probably the best solution to address the problem in the aolserver implementation. Again, as i do not have a WIN32 environment, please test the following replacement of NsTclTmpNamObjCmd() in aolserver/nsd/tclfile.c. With this fix, set tmpnam [ns_tmpnam] should be ok again.

best
-gustaf neumann

int
NsTclTmpNamObjCmd(ClientData arg, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[])
{
#ifdef WIN32
    /*
      The WIN32 implmentation of tmpnam() ignores the environment
      variable TMP and generates filenames for the root
      directory. Unfortunately, new WIN versions (Vista) don't allow
      this. The suggested replacement is _tempnam().

      The first argument of _tempnam() is the default directory, in case
      the environment variable TMP is not set or points to a directory
      that does not exist.
    */
    char *buf = _tempnam("/tmp", NULL);

    if (buf == NULL) {
	Tcl_SetResult(interp, "could not generate temporary filename.", TCL_STATIC);
        return TCL_ERROR;
    }
    /*
       The documentation says that _tempnam() allocates memory via
       malloc(); to be sure, that the "right" free() is used, we do
       not use TCL_DYNAMIC but the TCL_VOLATILE followed by the manual
       free().
    */
    Tcl_SetResult(interp, buf, TCL_VOLATILE);
    free(buf);
#else
    char buf[L_tmpnam];

    if (tmpnam(buf) == NULL) {
	Tcl_SetResult(interp, "could not generate temporary filename.", TCL_STATIC);
        return TCL_ERROR;
    }
    Tcl_SetResult(interp, buf, TCL_VOLATILE);
#endif
    return TCL_OK;
}