Docker Compose and CockroachDB
Introduction
If you’re planning to run a multi-mode instance of a CockroachDB cluster, Docker may be the ideal tool for the job. Both Docker and Docker Swarm can be used to start up and orchestrate a CockroachDB cluster. In this article, we’ll show you how to get started with Docker and CockroachDB by running a cluster using Docker Compose.
Prerequisites
Before we move forward with this tutorial, let’s review some of the key prerequisites for the task. First, you’ll need to make sure that a stable and supported version of the Docker engine is installed on the machine that’s running CockroachDB. If you’re working in a Linux environment, you should use your distribution’s package repository to uninstall and reinstall the docker, docker-engine, and docker.io packages. This will ensure that you’re using the latest version of the Docker Engine.
Docker was originally designed with Linux in mind, and it was meant to run as a background daemon service that can run containers by executing commands in the terminal.
Install Docker Desktop on macOS and Windows
If you’re running Windows and macOS, you can sign into Docker and download the Docker Desktop UI engine. You can then install it by following the steps of the package’s interactive installer.
Create a project directory for CockroachDB
Be sure to create a project folder for the CockroachDB docker-compose.yml file on the machine that’s running your cluster. You can use the mkdir command in a terminal or command prompt window to create the folder and then use the cd command to change into it:
1 2 | mkdir cockroach-docker cd cockroach-docker |
You’ll also need to create a docker-compose file that uses the .yml or .yaml file extension. If you’re running Docker locally, you’ll want to edit the file with an IDE that has YAML indentation support, such as Sublime, Atom or Visual Studio Code.
The command shown below will use the Sublime IDE to create the Docker Compose YAML file:
1 | subl docker-compose.yml |
Run CockroachDB with Docker
Now that we’ve completed the necessary setup tasks, let’s look at a few examples of how we can create a multi-node Docker container of a CockroachDB server. We can use the docker run command to download the CockroachDB image and run instances of it as containerized processes; however, we will first need to create a bridge network for the CockroachDB nodes.
Docker bridge network for the CockroachDB nodes
Each CockroachDB node must have its own unique container, so we’ll use a Docker network that has the bridge property. This will allow them to can communicate with one another and work together as one cluster.
We’ll use the docker network create command to create the bridge network as seen in the following example:
1 | docker network create -d bridge cockroachdb_net |
This command should return the ID from the created cockroachdb_net network. To list all of your networks, simply use the docker network ls command.
If you don’t create your bridge network in advance, you’ll get an error response from the daemon stating that the network is not found when you attempt to start the CockroachDB cluster
Use ‘docker run’ to start CockroachDB
The command shown below is an example of docker run that creates a two-node cluster of CockroachDB. It also bind-mounts ports and a local /data directory for a volume on the local machine:
1 2 3 4 5 6 7 8 9 | docker run -d \ --hostname=node_1 \ --name=node_1 \ -p 26257:26257 -p 8080:8080 \ --net=cockroachdb_net \ -v "data:/cockroach/cockroach-data" \ cockroachdb/cockroach:latest start \ --join=node_1, node_2 \ --insecure |
NOTE: If you want Docker to pull a specific version of CockroachDB, you’ll need to specify that version number in place of latest. Keep in mind that the example shown above will start an insecure cluster, but this cluster is being used for development purposes only.
Use the docker ps command to view all of the running containers, and add the -a flag to list all containers instead of just running ones. You can stop the container by using the docker stop command followed by the container ID.
CockroachDB and Docker-Compose
The following example shows some sample YAML data for your docker-compose.yml file. This will pull the latest CockroachDB official image from Cockroach Labs in order to create a two-node CockroachDB cluster using Docker Compose:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 | version: "3.2" services: node_1: container_name: node_1 image: cockroachdb/cockroach:latest volumes: - ./data/node_1:/cockroach/cockroach-data command: start --insecure ports: - "26257:26257" - "8080:8080" networks: cockroachdb_net: aliases: - node_1 node_2: container_name: node_2 image: cockroachdb/cockroach:latest volumes: - ./data/node_2:/cockroach/cockroach-data command: start --insecure --join=node_1 networks: cockroachdb_net: aliases: - node_2 networks: cockroachdb_net: driver: bridge |
This YAML will bind-mount and create, if needed, a local directory called /data. It will also create respective sub-directories for each node’s data, and it will use the cockroachdb_net bridge network so that the nodes can communicate with one another.
Spin up CockroachDB containers with Docker Compose
Make sure to save your YAML code, then go to your terminal and execute the following shell command at the same directory level as the .yml file:
1 | docker-compose up --build |
This command will pull the image if needed and will automatically rebuild the containers as it spins them up:
1 2 3 | Creating network "cockroach-docker_cockroachdb_net" with driver "bridge" Pulling node_1 (cockroachdb/cockroach:latest)... latest: Pulling from cockroachdb/cockroach |
Docker Compose with the CockroachDB cluster
Since the containers are running in the foreground, you’ll need to open another tab or terminal window and use the following command to list the nodes’ containers:
1 | docker ps |
You’ll see both of the CockroachDB nodes running in separate Docker containers with unique identifiers.
Troubleshooting a CockroachDB cluster in Docker
If you encounter a port conflict, you can try to stop and remove the containers for the nodes using docker stop and docker rm. You can also use lsof to look for the processes using the 26257 Cockroach port as seen below:
1 | sudo lsof -t -i:26257 |
The sudo kill -9 command can be used to kill processes by specifying their respective process IDs.
Initialize and interact with the CockroachDB cluster
You can use the following command to initialize your cluster if needed:
1 | docker exec -it node_1 ./cockroach init --insecure |
NOTE: If you get an error stating: ERROR: rpc error: code = Unknown desc = cluster has already been initialized, you can assume that CockroachDB has already been initialized.
Access the CockroachDB SQL interactive shell
We’ll use the docker exec command to connect and interact with the node_1 container by taking advantage of the -it interactive options:
1 | docker exec -it node_1 /bin/bash |
Once you’re inside the container, use the ./cockroach sql command to enter the SQL interactive shell for the CockroachDB node:
1 | ./cockroach sql --insecure |
To confirm that CockroachDB is working, execute the following CREATE DATABASE SQL statement:
1 | CREATE DATABASE some_db; |
This SQL statement should return a response like the following:
1 2 3 | CREATE DATABASE TIME: 9.490273ms |
At this point, there should be some persistent CockroachDB data in the /data/node_1 folder in your project directory. Even if the containers and image are destroyed, the data will remain in that folder and will be accessible once a new container is spun up.
Exit the Docker container for the CockroachDB node
When you’re done with your work in the CockroachDB SQL interface, use the \q command to quit out of it. You can type exit to disconnect from the interactive interface with the Docker container.
Conclusion
If you’re interested in running multiple nodes in your CockroachDB cluster, it’s a good idea to harness the power of Docker Compose. In this article, we showed you how to use Docker Compose to run a multi-node instance of a CockroachDB cluster. With these detailed instructions to guide you, you’ll be ready to use Docker Compose and CockroachDB together in your own environment.
Pilot the ObjectRocket Platform Free!
Try Fully-Managed CockroachDB, Elasticsearch, MongoDB, PostgreSQL (Beta) or Redis.
Get Started