Forum OpenACS Development: NetV, memcached

Collapse
10: NetV, memcached (response to 1)
Posted by Andrew Piskorski on
Tom, something that sounds very much like your NetV idea (but as a stand-alone application, nothing to do with AOLserver) is already done and in wide production use: memcached
Collapse
Posted by Andrew Piskorski on
Naturally, all cacheing schemes that keep the cache outside the RDBMS - NSV, memcached, whatever - are inherently non-transactional. This is fine in many cases, and is keeps the NSV/cache implementation (but not necessarily the application that uses it!) simple.

What if, instead of the memory cache layer existing as a completely independent application, you turned it into part of the RDBMS? In essence, let the RDBMS keep its own in-memory cache of query results, but let it keep it on many other machines across the network, not just in it's own RAM. Obviously that means the cache servers and the master RDBMS have to exchane lots of short messages frequently in order to keep the network memory cache coherent and transactional, and getting that right would be critical for performance and scalability.

At least without thinking about it in much more detail, it sounds feasible to me. There are also obvious potential synergies with the stuff that the HPC cluster/Beowulf work on, optimized MPI libraries, low-latency high-bandwith networks and network drivers, etc. (Thus see also the somewhat related thread discussing Clusgres, Postgres-R, and the like.)

But, this seems like such an obvious R&D/thesis opportunity that I wonder whether it's already been done. Anyone know of any such projects?

Collapse
15: pgmemcache (response to 10)
Posted by Andrew Piskorski on
I just noticed that PostgreSQL has a memcached interface, pgmemcache. Also, Harish Krishnan provided a very small Tcl API to memcached back in Aug. 2005.
Collapse
16: SQLCacheD (response to 10)
Posted by Andrew Piskorski on
Hm, and an Ivan Voras is working on SQLCacheD, which is like memcached, but uses in-memory SQLite for greater flexibility (using SQL) in how one can manipulate (expire, update, etc.) cached data.