TheMagIT took a closer look at four distributed storage technologies, which are deployable in the form of containers.
The second of this series is Minio, the fruit of the work of Abu Periasamy, one of the fathers of GlusterFS.
Minio
Co-founded by Anand Babu Periasamy, a leading developer and former CTO of Gluster , the Californian start-up Minio has designed an oOpen Source object storage solution compatible with S3 , whose goal is to deliver a simple storage, fast and powerful to cloud application developers.
Established in November 2014, the company has fewer than 20 employees and raised a total of $ 3.35 million from AME Cloud Ventures, General Catalyst Partners, Index Ventures and Nexus Venture and from private investors including Mark Leslie (CEO). background of Veritas), Jeff Rottschild (co-founder of Veritas), Ben Golub (CEO of Docker ) or Marten Mickos (ex-MySQL). Minio already has a large community, with over 125 active contributors to the software code.
Simple and compact object storage, deployable as a Docker container
The object storage software of the firm, Minio Cloud Storage, has been developed in Go language and comes with several SDK for Go, Java, JavaScript and Python languages. The source code of the software is accessible on GitHub (under Apache license) and has already been downloaded more than 500,000 times.
Minio can be deployed as a traditional executable on Windows, Linux, MacOS or FreeBSD. But in 90% of the cases, it is implemented in the form of a container Docker. The software is not intended to compete with large, massive object storage systems such as Scality or IBM Cloud Storage. Rather, it aims to deliver storage closer to the applications. Ideally, says Anand Babu Periasamy, the goal is for developers to programmatically deploy their applications at the same time as Minio.
The software has two modes of operation. The first makes it possible to set up an S3-compatible storage over conventional volumes in block mode. Deployed on an x86 or ARM server, Minio can consume a storage volume and make it appear as an S3 storage. In this mode, Minio allows a developer to have object storage virtually anywhere, including their work machine. The integrity of the data is ensured by the same mechanisms of protection of the underlying volume. If it is a simple hard drive and it goes down, the data is lost. On the other hand, if it is a SAN volume, the protection level depends on the storage array.
The second, more sophisticated deployment mode is called Distributed Minio. In this mode, Minio is able to aggregate the capacity of multiple volumes residing on one or more physical or virtual servers into an object space protected by an erase code mechanism . Distributed Minio can aggregate up to 16 disks scattered across up to 16 different nodes (1 disk per server on 16 machines, 2 disks per server on 8 machines, etc.). The software ensures data integrity as long as at least half of the cluster's disks remain available. And it is possible to create new objects on the cluster as long as on a total of n disks, n / 2 + 1 disks are accessible. Minio also includes a mechanism to protect against silent data corruption (the so-called "bit-rot").
The object store is compatible with the S3 protocol and accepts the writing of any type of object as long as their unit size does not exceed 5 TB. Interestingly, Minio also supports AWS Lambda notification mechanisms - which allows at certain events to trigger calls to specific functions. This can range from the automatic conversion of an image or video after ingestion, to the implementation of scoring mechanism or data filtering.
Co-founded by Anand Babu Periasamy, a leading developer and former CTO of Gluster , the Californian start-up Minio has designed an oOpen Source object storage solution compatible with S3 , whose goal is to deliver a simple storage, fast and powerful to cloud application developers.
Established in November 2014, the company has fewer than 20 employees and raised a total of $ 3.35 million from AME Cloud Ventures, General Catalyst Partners, Index Ventures and Nexus Venture and from private investors including Mark Leslie (CEO). background of Veritas), Jeff Rottschild (co-founder of Veritas), Ben Golub (CEO of Docker ) or Marten Mickos (ex-MySQL). Minio already has a large community, with over 125 active contributors to the software code.
Simple and compact object storage, deployable as a Docker container
The object storage software of the firm, Minio Cloud Storage, has been developed in Go language and comes with several SDK for Go, Java, JavaScript and Python languages. The source code of the software is accessible on GitHub (under Apache license) and has already been downloaded more than 500,000 times.
Minio can be deployed as a traditional executable on Windows, Linux, MacOS or FreeBSD. But in 90% of the cases, it is implemented in the form of a container Docker. The software is not intended to compete with large, massive object storage systems such as Scality or IBM Cloud Storage. Rather, it aims to deliver storage closer to the applications. Ideally, says Anand Babu Periasamy, the goal is for developers to programmatically deploy their applications at the same time as Minio.
The software has two modes of operation. The first makes it possible to set up an S3-compatible storage over conventional volumes in block mode. Deployed on an x86 or ARM server, Minio can consume a storage volume and make it appear as an S3 storage. In this mode, Minio allows a developer to have object storage virtually anywhere, including their work machine. The integrity of the data is ensured by the same mechanisms of protection of the underlying volume. If it is a simple hard drive and it goes down, the data is lost. On the other hand, if it is a SAN volume, the protection level depends on the storage array.
The second, more sophisticated deployment mode is called Distributed Minio. In this mode, Minio is able to aggregate the capacity of multiple volumes residing on one or more physical or virtual servers into an object space protected by an erase code mechanism . Distributed Minio can aggregate up to 16 disks scattered across up to 16 different nodes (1 disk per server on 16 machines, 2 disks per server on 8 machines, etc.). The software ensures data integrity as long as at least half of the cluster's disks remain available. And it is possible to create new objects on the cluster as long as on a total of n disks, n / 2 + 1 disks are accessible. Minio also includes a mechanism to protect against silent data corruption (the so-called "bit-rot").
The object store is compatible with the S3 protocol and accepts the writing of any type of object as long as their unit size does not exceed 5 TB. Interestingly, Minio also supports AWS Lambda notification mechanisms - which allows at certain events to trigger calls to specific functions. This can range from the automatic conversion of an image or video after ingestion, to the implementation of scoring mechanism or data filtering.
Minio Client: A set of shell commands to manage, replicate, or synchronize data between S3 storage
Minio is also bundled with Minio Client or "mc", an independent client component that works with any type of S3 storage and delivers the equivalent of Unix shell commands for object storage. Minio Client includes commands such as ls, cp, diff, mkdir, mirror, rm or rsync, and allows for example to list the objects of a bucket, to transparently replicate the contents of a bucket of an S3 storage to another, or sync data from a Minio local storage service to another S3-compatible storage (and vice versa). It should be noted that Minio Client works with Minio, but also with any form of S3-compatible storage.
The various software of Minio, 100% open source, are downloadable and usable free of charge. The publisher, Minio generates revenue through support services to large customers. "When we become a critical component in some customers, they are ready to sign a contract with us to have a support offer," says the CEO of Minio. The firm, however, does not intend to disperse and wants to focus on large contracts. She is quite happy to see partners providing support for smaller customers. This allows him to focus most of his efforts on enriching the Minio code.
Minio is also bundled with Minio Client or "mc", an independent client component that works with any type of S3 storage and delivers the equivalent of Unix shell commands for object storage. Minio Client includes commands such as ls, cp, diff, mkdir, mirror, rm or rsync, and allows for example to list the objects of a bucket, to transparently replicate the contents of a bucket of an S3 storage to another, or sync data from a Minio local storage service to another S3-compatible storage (and vice versa). It should be noted that Minio Client works with Minio, but also with any form of S3-compatible storage.
The various software of Minio, 100% open source, are downloadable and usable free of charge. The publisher, Minio generates revenue through support services to large customers. "When we become a critical component in some customers, they are ready to sign a contract with us to have a support offer," says the CEO of Minio. The firm, however, does not intend to disperse and wants to focus on large contracts. She is quite happy to see partners providing support for smaller customers. This allows him to focus most of his efforts on enriching the Minio code.
0 Comments