The joy of operating a container-based infrastructure is all of the change they introduce. Yes, that was sarcasm. My impression of containers is that they were originally more about application development and less about transforming infrastructure and operations. And yet, here we are. Containers are all the rage, touting benefits of reduced compute footprint, security, and portability. Containers are being used in more and more pockets of production.
With that increase in production, the immaturity of containers shows through, as operations teams are faced with the challenge of bringing feature parity to their container environments from their current environments. Bring on the conflict, as the stateful assumptions about normal application deployments don’t hold with the stateless assumptions of most containers.
By stateless, we mean that containers are supposed to be able to scale up and down on the fly with no impact to the application or user experience. Thus, if 6 containers are needed to meet demand but only 3 are running, an additional 3 are added to the mix and traffic spread around. If 3 containers are needed but 6 are running, then 3 can be torn down. That’s pretty easy to accomplish if the application instance running in a container doesn’t have to worry about state.
This set up and tear down isn’t required of containers, but is touted as a benefit for applications that have been broken up into microservice components. For a stateless microservice, i.e. one that does not require data written to disk, this flexibility is a wonderful thing. For stateful applications, i.e. one that is in the business of long-term, persistent data storage, a different thought process must be applied — one more akin to our pet workloads we lovingly handcraft, monitor, and maintain.
Addressing the enterprise storage requirements of stateful containers.
And so it is we find that container storage is far from wonderful. As StorageOS tells the story, several attributes of container operations are leaving storage behind.
- Container operations that were meant for primarily stateless microservices.
- Traditional storage systems without the APIs necessary to integrate with automation tooling.
- Storage performance that fails to scale in lock-step with container auto-scaling.
- Storage that is not easily portable, while moving container workloads is easy.
- A lack of tooling to manage storage for containers.
- Containers mapping readily to opex (pay as you go), while storage maps better to capex (buy it all now and depreciate the asset).
If you’ve worked with containers, you know that storage volumes exist. It’s entirely possible using, let’s say, Docker to create a container and map a volume to that container to be used for persistent storage. Stateful containers are already a thing.
The issue is in managing the underlying storage. Storage for containers needs to have the fluidity of the rest of the infrastructure. StorageOS is taking this challenge head-on. Their goal is to re-think data center storage, and make it work seamlessly with the rest of an infrastructure stack built for containers. And not just containers, but stateful containers running persistent workloads by grown-up enterprises.
The StorageOS team came from financial services. In my time supporting payment card operations, we didn’t measure success in terms of limited down time. Down time was not a concept for us. The environment was always up, period. Therefore, when StorageOS says they want to bring a financial services mindset to container storage operations, this is what I believe they are getting at. Seamless, 100% uptime, and no excuses.
A tall order.
How StorageOS aims to deliver storage nirvana to container operations.
Running storage at the application layer. This means that StorageOS scales storage with the application. As a container comes up to run an application workload, the storage comes along with it as a single operation. As compute is consumed, storage is consumed.
This is a change that introduces flexibility into the application infrastructure, but means that processes are going to change. And with process change comes changes to operations, and very probably, individual responsibilities.
Tight integration with the dominant container-related ecosystems — Docker, Swarm and Kubernetes. StorageOS is a 40MB container offering a global namespace that provides volumes to applications. As StorageOS is integrated with Docker, Swarm and Kubernetes, stateful workloads migrating to other platforms will keep their persistent storage with them, but in an automated way. This also means that the storage volume is going to be placed on the same node as the container, to make sure performance isn’t hampered by latency.
Secure automation of storage provisioning. Securing persistent data within the data center is an easily managed concern. We all have processes to handle data at rest and in flight, limit access to data, and retain data. But what about data that moves with a workload into one or more public clouds? Here, StorageOS applies encryption in accordance with business policies both for data at rest or in flight.
Working with key container communities. StorageOS works with the Linux Foundation, the Cloud Native Computing Foundation, and Docker as an Ecosystem Technology Partner for Storage. I couldn’t find StorageOS listed as yet on the Docker Ecosystem Technology Partner page, but StorageOS mentioned it was a recently established partnership. I’m assuming the docker.com webmaster needs a heads up.
How can I take StorageOS for a spin?
StorageOS wants trying their product to be “frictionless” both for developers and infrastructure teams. Go to my.storageos.com and register to download.
As you do, realize that StorageOS is a container-based operating system for storage, and not a storage array. You’re supplying the disk that StorageOS will manage. According to the StorageOS FAQ, that storage could be “any type of disk (e.g. SSD, SAS, SATA as well as virtual drives such as EBS).”
The FAQ also mentions, “StorageOS is optimised for SSDs and is an ideal platform to build out all-flash systems using commodity SSD devices. In all configurations, a small amount of SSD capacity will significantly increase performance as it will be automatically used for caching and/or consistency. SSD is mandatory if deduplication is enabled.”
StorageOS pricing starts out as low as free at the Developer level, the aim to make it easy to work on stateful containers running apps being developed. I.e., the first hit is free, but the next one’s gonna cost you. The more feature-rich Professional licensing tier is $30 (okay, okay, $29.95) per month for 1TB of storage. Although not stated explicitly, I assume that to mean $30 per month per terabyte.