Forum OpenACS Development: Re: Compatibility Matrix

Collapse
17: Re: Compatibility Matrix (response to 1)
Posted by Torben Brosten on 02/02/12 11:22 PM
One hopes that sequential scans include an interpolation process on sorted keys to help reduce the scan range.
Collapse
18: Re: Compatibility Matrix (response to 17)
Posted by Don Baccus on 02/03/12 05:34 AM
As someone who understands db implmentations quite well, I gotta say this makes no sense.

What you describe can be used to filter ouput to the client of the scan, but no, table data's not sorted so seq scans do scan the whole table.

Absent partitioning, I guess, which is useful for data warehousing, not dynamic websites.

Collapse
19: Re: Compatibility Matrix (response to 18)
Posted by Torben Brosten on 02/03/12 06:18 AM
Hi Don,

Good to know. Thank you.

It seems to me that smart (whatever it's called) interpolation of an ordered set could be faster than sequencing once the count reaches some massive number.. By "smart" I'm referring to a modified conventional mathematical interpolation, where the next best guess uses the last two data points to suggest an intersecting line and intersection point to jump to --providing the next best guess.

Assuming 69 million rows, ordered, non-linear values, one could probably cut to search range with a very small count within just a few steps.. Would be fun to play with.

Anyway, I've successfully used a technique like this to reduce the interpolation loops of some business math functions to something reasonable for dynamic web pages.. About 12 iterations on projects with millions of values, non-linear. I'd be glad to share, if you know someone that want to pursue this.

cheers,

Collapse
21: Re: Compatibility Matrix (response to 19)
Posted by Don Baccus on 02/03/12 11:42 PM
"Assuming 69 million rows, ordered"

But there's no order imposed on the rows in the db. Unless you insert them in a certain order and never delete or update even one single row in the table.

I suppose there are tables like that but it's certainly not typical of how a database is used.

Collapse
22: Re: Compatibility Matrix (response to 21)
Posted by Torben Brosten on 02/03/12 11:49 PM
Hi Don,

Yeah, hair brain idea. I get it. I just passed the idea on to someone that could do something about it if there was anything to it; They didn't see it applicable to DB scans either.