Think of data storage as a series of layers in a delicious multi-layer dip.

You have to hand it to Zend. Their Framework lets developers build solid LAMP applications with all kinds of useful, out-of-the-box support for RESTful servers, plugging into Lucene, handling LDAP interactions, and even interacting with Windows Azure! Yes! Windows Azure! Something I didn’t even know existed until I browsed Zend’s API docs. I wonder how much Microsoft paid them to include that package…

Anyway, back to the topic at hand. Storing things. In places. With stuff.

Your average web application generally starts off with a basic database back-end like MySQL orPostgreSQL. That works, for a while—until the database starts sweating under load because you are so awesome and successful that your website pulls in a million hits a minute. Reads begin to far exceed writes at the database level. And for most of the time, you’re looking objects with their primary keys, like user or blog post IDs. This means you have an opportunity to relieve the load on your database database (all these dumb ID lookups), and instead let it do what it does best: store, manage and query complex relational data in an atomic and transactional fashion.

Relieving this load is as simple as putting a caching mechanism in front of the database that can, given an entity type and ID, quickly spit out corresponding objects. A good caching system will actually return data much faster than a database, since the data is all stored in memory and built to handle this (and only this) scenario.

At least it should be this simple, in theory.

(more…)