Envysion is having so much growth that we’ve recently had to expand our database capacity to store the ever higher volume of event data flowing in.  (We still store video and the video database at the remote site)

Adding database capacity isn’t such a big deal if one just goes out and buys some “big iron” servers, a big storage area network system and pays a huge license fee for the latest Oracle enterprise database software.  This is the traditional enterprise way to grow a “data warehouse”.  This is sometimes called “Scaling Up” and it is extremely expensive, often taking up the a huge slice of any IT organization’s budget whenever there is truly a large amount of data stored in databases.

Instead, Envysion is using cutting edge, cost savings methods to grow our hosted service.  By “Scaling Out” our data across multiple commodity market server hardware and using the MySQL database, we’ve been able to keep costs low, keep performance high and still grow big, like 100’s of millions or even billions of rows big.  This is the way of highly scaled and efficient web applications.

Why does this matter?  Simple!  The costs of a scale out is at least an order of magnitude lower than the big general purpose data warehouse solutions out there.

But doing a “scale out” isn’t so easy.  While a “scale-up” design utilizes a small number or even just one, single, ever larger growing (in both size and dollars!) database.  The scale-up design is somewhat simple from an application perspective.  Conversly, doing a “scale out” really requires that one know how your users want to see your data.  You also need to know your application and data well enough to distribute the data across many low cost servers.  In other words, there’s not yet just one simple way to “shard” out your data across many database servers.  (although many innovative companies such as codeFutures dbShards and Dataupia are working on this as we speak)