Until recently RainStor only supported SAN, NAS, DAS and CAS storage. Therefore, I thought it would be interesting to explain how our support for cloud storage evolved. While there was obviously a large amount of hard work and planning, I’m the first to admit there was also some good timing and serendipity.
The starting point was our core data management and storage technology developed over many years: a massively compressed and immutable data store supporting efficient SQL queries over JDBC/ODBC. This technology is typically integrated with application archiving and data retention solutions to provide the optimal store for historical data. As such, the software has been designed to run on inexpensive commodity hardware, as typified by the servers you can find running in the AWS EC2 cloud. With these unique capabilities in the bag, like many software vendors we spotted the opportunities and benefits of virtualization and cloud computing, and began to optimize our software for these environments.
In late 2008, SNIA finalised the XAM specification for Content Addressable Storage which allowed us to virtualize our entire file-system using XAM. The main benefit for us was the ability to efficiently store and query data on a huge range of hardware storage platforms through a single interface. We then realised that it was simple for us to develop XAM drivers for software storage solutions, such as WebDAV, Symantec Enterprise Vault, EMC Documentum etc. From there, developing an AWS S3 driver was straightforward, with thanks to Bryan Ischo for getting us off to a flying start! We were now able to easily support SAN, NAS, DAS, CAS and cloud storage platforms.
Earlier this year, we started evaluating Amazon AWS more closely and were very impressed with the maturity and sophistication of the platform. More importantly, the cost model was reasonable. However, a key problem remained: how to get huge volumes of data into the cloud in a secure and efficient manner?
Fortunately, we were already using VMs to enable our Linux-based software to run on a huge range of different OSs and chip sets. This gave us the key to solving the data upload problem: a client-side VM software appliance. The virtual appliance is a small download that when used with RainStor compresses and encrypts the data before uploading it to S3, making it secure, fast and cost effective. This wouldn’t have been possible a year ago, but VMs are now widely accepted, enabling us to complete the RainStor for the cloud.
The net result is that by leveraging proven technology, adopting XAM standards and optimizing our software for virtualized environments we have been able to very quickly deliver and support technology for sending, storing and searching structured data in the cloud.