Container Clustering & Management: Docker Swarm, Kubernetes & Apache Mesos and Marathon

Docker Swarm:

Swarm is the native Docker clustering solution, Swarm extends the existing Docker API to make a cluster of machines look like a single Docker API.

Key Components:

  • Managers: Distribute tasks across the cluster, with one manager orchestrating the worker nodes that make up the swarm.
  • Workers: Run Docker containers assigned to them by a manager.
  • Services: An interface to a particular set of Docker containers running across the swarm.
  • Tasks: Individual Docker containers running the image, plus commands, needed by a particular service.
  • Key-value store: etcd, Consul or Zookeeper storing the swarm’s state and providing service discoverability.

Calsoft Whitepaper: File Virtualization with the Direct NFS Protocol

This paper discusses a SANergy-like file system that works on UNIX clients against UNIX servers using NFS, and provides the performance of a SAN, and the manageability of NAS. This paper was published at the NASA/IEEE Symposium on Mass Storage Systems, Maryland.

When to use Docker Swarm:

Swarm has been viewed as suitable for experiments and smaller scale deployments; Docker Swarm gives you an easy way to move into container orchestration.


Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. Kubernetes was built by Google.

Key Components:

  • Master: A single master handles API calls, assigns workloads and maintains configuration state.
  • Minions: The servers that run workloads and anything else that’s not on the master.
  • Pods: Units of compute power, made-up of one or a handful of containers deployed on the same host, that together perform a task; have a single IP address and flat networking within the pods.
  • Services: Front end and load balancer for pods, providing a floating IP for access to the pods that power the service, meaning that changes can happen at the pod-level while maintaining a stable interface.
  • Replication controllers: Responsible for maintaining ‘X’ replicas of the required pods.
  • Labels: key-value tags that the system use to identify pods, replication controllers and services.

When to use Kubernetes:

Kubernetes is well suited to medium to large clusters running complex applications

Apache Mesos and Marathon:

Mesos is a Multi-Framework orchestration solution for containers.

Mesos is less focused on running just containers. Mesos focuses on scheduling, and plugging in multiple different schedulers,  and  the result, the Hadoop and Marathon can co-exist in the same scheduling environment.

Key Components:

  • Masters: Zookeeper manages a minimum of three master nodes and enables high availability by relying on a quorum amongst those nodes.
  • Slaves: these nodes run the tasks passed down by the framework.
  • Framework: Mesos itself knows nothing about the workloads, whereas specialist frameworks decide what to do with the resources offered to them by Mesos.

On top of Mesos, Marathon then offers a highly available framework delivering:

  • Service discovery: through a dedicated DNS service, as well as other options.
  • Load balancing: through HAProxy.
  • Constraint management: to control where in the cluster certain workloads run, maintain a set level of resources for those workloads, enable rack awareness and other constraints.
  • Applications: the long running services you want to run on the cluster; may be Docker containers but can also be other types of workload.
  • REST API: deploy, alter and destroy workloads.

When to use Mesos and Marathon:

Mesos and Marathon are, perhaps, a good choice if you need to run non-containerized workloads alongside containers and you want the reassurance of something that is proven at very large scale.


The three main options right now vary considerably in implementation. Docker’s Swarm gives you the easiest route into orchestrating a cluster of Docker hosts. Kubernetes is container-centric but focuses less on the containers themselves and more on deploying and managing services. Mesos with Marathon promises huge scale but introduces additional complexity.

To know more email:

Yogesh Patil | Calsoft Inc.

Container Ecosystem Services

Calsoft has deep expertise in containerization of Storage and Networking products. With our in-depth understanding of various containerization technologies like Docker, Kubernetes, Apache Mesos and Coreos, we have helped ISVs to design and develop solutions in and around these technologies.




Related Posts

7 Useful ServiceNow Integrations

ServiceNow has created its position as one of the best platforms and workflow integration tools. It enables enterprises to develop custom plug-ins as well to meet their unique business requirements.

GoLang Memory Management

GoLang Memory Management

One, created at Google and growing rapidly now having around 90,000+ repositories. Go can be used for cloud and systems programming and extended to the game server development and handling text-processing problems.

Unit Testing with MockitoPowerMockito

Unit Testing with Mockito/PowerMockito

Mockito is an open-source Mocking framework in Java. The features it provides for unit-testing is important.
It has simplified test case writing for developers. While Mockito can help with test case writing, there are certain things it cannot do viz:. mocking or testing private, final or static methods.

Understanding Desktop Virtualization VDI vs RDS

Virtual Desktop Infrastructure (VDI) Vs. Remote Desktop Services (RDS)

Both RDS and VDI are core components of desktop virtualization, and they satisfy specific computing requirements and scenarios with deployment readiness and flexibility. VDI and RDS have peculiarities that adapt to the different needs of a business, but making a choice between them could be difficult for some companies. Benefits of Virtual Desktop

Navigating Big Data Storage Challenges

Storage Considerations for Big Data Storage

The last decade or so has seen a big leap in technological advancements. One of the technologies to come up at this time and see a rapid…


A Deep Dive into 5G Service-Based Architecture (SBA)

5G technology roll out signifies an immense revenue opportunity for telecom industry.


This Post Has 2 Comments

  1. This blog explains the details of most popular technological details. This helps to learn about what are all the different method is there. And the working methods all of that are explained here. Informative blog.

Comments are closed.