Alfred, Thanks for the comments. The changes for address and phone will only map to HR-XML and people can add packages as they see fit using these as a base. There will also be a slight change to the data model in 5.0 (easily backported to 4.6.x)

Telecom-number and postal-address are services. They are not packages. They are analogous to the reference-* packages. People are free to use the underlying data model as they see fit.

Everything you describe can be accomodated with the HR-XML framework. The reason other packages use seperate data is historical; The original address book was almost useless beyond a personal list and was most definitly hardly internationalized. Plus, many package designers just added their own tables without regard for what was already in the toolkit (states and countries come to mind).

My contacts-lite package is just that. It was (like much of the code here) written for a client who was kind enough to let me release it to the OACS community. It does however show how to use the underlying services (although certainly not everything has been completed, I have other obligations and they don't involve these packages anymore except for my personal use).

As anyone who has followed my posts on reference data and contacts in general knows, I am very concerned with internationalized contact information. Modelling these packages after HR-XML fulfills this need.

I also saw a thread recently from someone saying they are going to implement a contact system and ignore any other tables in the schema (sigh).

I'm that person - and no i did not say i would not use postal-address just that i have worked without it until now and will integrate it later - when postal address is "stable". Things like the ones you are mentioning are the types of issues that kept me from designing it with postal address built in - but i need to street it will be EASY to add it later... once things like this are sorted out.

Your concern about the necessity of persons related to postal-addresses is one of mine as well that i would have brought up later. I want a "contact" to also be assigned an address and they won't necessarily be a person in the system... so i agree with Alfred on this one. Having a seperate mapping table, where you map and address_id to a object_id would take care of this - because presumeably all services that would use postal-address would have objects that the address can be linked to. I also don't think it is necessary for a postal-address to even be an acs_object. We won't be needing to do permissions on individual addresses and there is no need (in my view) to clutter up those other systems. A simple unique address id created by a sequence should be enough.

Now, regarding alfreds points

1. Fantasitic idea - i modeled mine after "deleted on this date" but yours is better.
2. I agree with Jon that HR-XML takes care of htis problem - its a matter of correctly inputting data into the HR-XML standard - and this requires the appropriate fields to be filled out for US costomers - but it also allows people in other countries to satisfy their address requirements. I also based my contacts addresses on HR-XML before i knew postal-address existed as a package and independenly came up to the same conclusion as Jon on this one so i'll say i think its the right one.
3. I think it makes more sense to store longitude/latitude with zip codes and reference it that way. or, do you know of reference data that says house number "1234" on street "X Avenue" is at thisxthat degrees? This is primarily a matter of having longitude and latitude data for every house (and we can't expect users to enter it correctly themselves).
4. If postal-address is a service then this suggestion should be handled by some other package that references the postal-address service.
5. Same point as above
6. With the contacts system i am creating this will be possible - though i don't think it should be mandatory throughout openacs, but be something that can be activated via a parameter. Many online services don't care about a persons address.

i just remembered acs_rels - if we are trying to consolidate things this should be the place where addresses objects are mapped to person objects or anything else for that matter...

Having a ref-address package does nothing. Address data is not reference data. (reference data is seldom changed and usually from a 3rd party.).

Unless someone can come up with a very compelling reason not to have addresses and telephone numbers acs_objects they are going to stay that way. Comparisons to the way "AD" did it aren't reasons.

The current model is designed to make all addresses and phone numbers objects for many reasons, including site wide search and auditability.

As Alfred suggested, I am planning on tcl procs to do address verification. This is not trivial though and the system now works the way it is. Also, none of this has any effect on the data model or HR-XML.

Everything you guys are talking about should be built on top of the telecom-number and postal-address services. They are only repositories of data, what you do with them is another package.

Marketing data has to be stored seperatly, that is another package.

Longitude/Latitude is already incorporated into the ref-zipcodes service. Please note that getting this data is country specific with many if not most countries not having it. It really has no place embedded in the core data model.

There will be some changes to persons:

alter table persons add column formatted_name varchar(200);
alter table persons add column preferred_given_name varchar(100);
alter table persons add column middle_name varchar(100);
-- need to drop the not null constraint, I believe PG 7.3 allows this.

Also, person__new et al need to be replaced. I already did this for PG and Oracle will be just as simple.

I guess I don't fully appreciate the distinction between making them objects and not making them objects or calling it a service...

If you do 'verification' in tcl procs the best you are going to do is confirm that an address 'looks' well-formed.

I make a strong distinction between well-formed and valid data. Well formed passes some initial regexp rules that you can do either client side or server side with a round trip. It's not VALID unless its actually accurate. It's just like the email registration process - OACS makes sure there is an @ sign and thats about it. If I want to ensure that I have VALID emails, I have the system send the email and make the new user click through from the email to fully register.

I am trying to get the community to incorporate at least the fundamentals of correct address processing - the USPS and Bundespost will both give good discounts for properly formatted and address verified mail.

If I can save .03 per mailing per piece, I save a good amount of money over time. If the company sends even a moderate amount of mail, its worth doing properly.

The added bonus to us as developers is we can charge our customers to validate the data through 3rd party address quality providers.

Longitude/Latitude at the ZIP code level? That is based on the centroid and can be many miles away from the address. I can get it to the street address level - its used for proximity, routing, etc. It's all from TIGER anyway. If a particular user doesn't need it - they can leave it blank, but if I send out my list to get geocoded, I'd like to have a consistent place to put it.

And when you say marketing data - remember that once you wipe off the polish - that is also just TIGER data from the census bureau. Classifying an address as falling into a specific DMA is a US government distinction, not some arbitrary marketing decision. That doesn't work across all lands, but I'm willing to bet that a lot of other countries over time will do it similarly.

I would also argue that you might want to add timezone to the addresses - it's more of a warehouse mentality - put the facts in with the record itself. If you want to call someone who is registered in your system (customer support for example), there are only certain hours where you're allowed to do so.

Whee! Another can of worms.

Jon, i forgot about site wide search - yea, i agree they should be objects... I agree that address verification via tcl is important - do you also intend to make an address "widget" that can be inserted as an element in forms? i think this would be a great thing - i.e. similar to the way date has a number of fields as an element i think address should also have a numbmer of them - possibly selected via a parameter via acs-templating that lets you select the default countries setup of that input field.

But Jon, as a repository of data, postal-address should not rely on persons - it should rely on itself. I really don't like the constraint on persons and in fact it would make it impossible for me to use it with my contacts app without creating bogus "persons" in all i'm suggesting is changing this part of the postal-adderss table creation script

    party_id            integer
                        constraint postal_owner_id_fk
                            references parties (party_id),

I would like it to become

    object_id           integer
                        constraint postal_owner_id_fk
                            references acs_objects (object_id) 
                        constraint postal_owner_id_nn not null,

since this field can be null in its current state - which i hadn't realize it was - i can use it. in fact i am going to use it and implement it today so i don't start doing things that aren't standardize. However, if that field is not entered then there could be no way of "tracking down" addresses "belong" i.e. you could track the address object 1234 by the use of a map table with contact object 4321. But if lots of apps end up depending on postal-address (which they hopefully will), then i think it would be good to be able to search the postal-address table for all addresses belonging to the objects that particular application owns...

Alfred - john's point that ref-addresses isn't a good idea is a very valid one - that's not the way it would work with openacs standards. we could have a ref-addresses package that stores all this "tiger" data you are talking about but it wouldn't make sense to put unique addresses in there. Do you know of any gnu or gpled reference data that would let us calculate longitude and latitude? As Alfred suggested, I am planning on tcl procs to do address verification. This is not trivial though and the system now works the way it is. Also, none of this has any effect on the data model or HR-XML. Everything you guys are talking about should be built on top of the telecom-number and postal-address services. They are only repositories of data, what you do with them is another package. Marketing data has to be stored seperatly, that is another package. Longitude/Latitude is already incorporated into the ref-zipcodes service. Please note that getting this data is country specific with many if not most countries not having it. It really has no place embedded in the core data model. There will be some changes to persons: alter table persons add column formatted_name varchar(200); alter table persons add column preferred_given_name varchar(100); alter table persons add column middle_name varchar(100); -- need to drop the not null constraint, I believe PG 7.3 allows this. Also, person__new et al need to be replaced. I already did this for PG and Oracle will be just as simple.

The constraint is not on persons it is on party and that is a whole different thing having nothing to do with a person. It could be any data (i.e. organization, group etc)

And parties is going to change in 5.0 anyway to normalize the tables correctly. (i.e. a party doesn't need an email). Part of this is to work with other methods of registration etc. See other threads for this discussion.

As I stated above, this is all fine. No matter if you call it marketing or whatever. It just needs to be its own package.

It really isn't a part of the addresses in the world.

See if you can verify this legitimate address:

Calle 48 % 29b y 29f
Num. 29b03
Buena Vista Playa 
La Habana Cuba

This is a legitimate address and will probably never have data quality needs.

There are many ways of guaranteeing the data purity, being in the core address data model is not really a good solution (in fact I would argue it is wrong). So why don't you just make a package that depends on postal-address and use the data from there. That is easy and what a service is meant to do. You can even make tcl procs to verify whatever you need.

Just to follow up on Jon's point here: the constraint he is talking about is not with persons. The organizations you enter into the system are entered as *parties*, which is a completely different thing.

By entering them as parties, you're using the standard OpenACS way of entering organizations like this, and other packages can take advantage of the organizations you've entered.

I haven't looked thoroughly into the parties data model, but I believe this is the best way to do things.

I'm gonna claim Godwin's Law here - I'm out :) If I see some package (or is it a service?) called postal-address-verification-and-spatial-constructs I will contribute what I can... In the meantime, I'll just send my files to Melissa Data and get good addresses.

The fact that you think my Amnesty International response to the Cuba non sequitur was actually anything related to Greenpeace is an indication you haven't been reading what I said in much depth - which is your right of course.

I don't really care about the address verification, which is what you and Matthew both seem to think I'm primarily suggesting. I even tried to spell out that doing address verification does NOT ensure valid addresses - the best you can do in a TCL proc or javascript is make sure it's "well formed" - just like an email address or a credit card number.

My two primary issues were
#1 - decouple addresses from people.
#2 - store appropriate information with the addresses - lat/lon, DMA, timezone, etc.

I appreciate your adventures in Cuba - I am working with a school in Ecuador to put up a dotLrn site. I think all the stuff we're talking about here is largely irrelevant to either of those cases. The corporate guys are the ones who like the marketroid stuff - this is a great sales tool in that market and it provides an additional revenue stream for the developer/hoster if they periodically ship the addresses out to be cleaned up.

I clearly got lost in the distinction between services and packages (is a service something that uses a service contract? or just something with no ADP pages?) Parties - the only references I found were from AD people in the party::permission_p -

"Wrapper for ad_permission to allow us to bypass having to specify the read privilege " sounds like a way to skirt the permission system?

and you said they are changing in 5.0? I also am a bit vague on the _rels mentioned in the thread here ..

I guess there are enough internals here that I'm not familiar with and can't find documentation on that I'll just shut up and see what comes out. It's likely that I'll just add the fields I want on your table and keep a private version.

Thanks for the lively dialog :)

I actually did read it, but also had some greenpeace code in an ssh window. I don't have any need for Amnesty Intl. either. But that is irrelevant. And, I also don't hold animosity towards them, I just don't have a use for their politics.

A service is meant to be used by other packages and has nothing to do with service contracts. They in general don't have adp pages, but that is not the real distinction. The adp pages would be for admin purposes. Also, services should only be mounted once (usually)

I also think you are confusing my proposal for a validation package. Your package can use a tcl proc to consolidate/validate do whatever you want to the data. If you want to send it out to xxx-data service, no problem, ecommerce does a similar thing with payment gateways. There used to be one payment method. It has since been rewritten to use service contract to extend the ecommerce package without touching the core data model.

A similar approach should be used in the address-validation package. I am sure that every list cleaning/validation service has different requirements so you write a gateway and then add specifics for each company. By trying to put the US centric address information in the core of postal-address, you are locking everyone into your preferred company.If there was a standard for data cleaning/validation that would be a different matter, but as far as I know that doesn't exist.

The changes in 5.0 (which certainly are only being talked about) are changes to parties to eliminate persons information at the party level.

A party is an entity that relates to an object. That can be a person, group, organization or even another object. A person is a specific type of party and once a person becomes registered they become a user. But, all of the types inherit from parties.

Addresses don't have to be attached to a person, they are only attached to a party. You don't even need that information to send an address for validation. Also, you can create a history of that address by simply creating a package that tracks address_id and party_id with dates or whatever you want, but this isn't part of postal-addresses.

Ironically, I just found this:

http://openacs.org/bugtracker/openacs/bug?bug%5fnumber=241

in the bugtracker - requesting incorporating a GPL-ed copy of the TIGER street address to lat/lon data :)

http://ofb.net/~egnor/google.html

I will make it a ref-tiger package. Give me a few days.
But, I believe most of it is already in ref-zipcodes.