<- Back

Starting a local FoundationDB cluster using Docker

FoundationDB is an open-source distributed key-value (NoSQL) database capable of incredible performance and some impressive resiliency in the case of a node failure. For some reason, it's a bit unknown, even though it might be one of the only key-value databases capable of what it does. Some of this might be related to how Apple doesn't really market the database or how they don't provide a whole lot of public information about releases. It might also be related to the sometimes outdated or non-existent documentation.

I recently wanted to try out FoundationDB in Python as its interface is basically just a distributed, fault-tolerant dictionary (very useful). The install instructions for FoundationDB are mainly designed for bare-metal servers or virtual machines that are provisioned from scratch. Now a days, I almost never install databases to my local machine and instead rely on Docker for local testing and Kubernetes or Docker for production deployments.

FoundationDB has a Docker container built for every release but it's completely undocumented. Figuring out how it works requires taking apart the Dockerfile and its launch scripts. Rather than force you through that, I put together a small example of how to launch a local cluster using Docker compose.

Example

FoundationDB requires coordination servers to host the API and "coordinate" the cluster. Coordination servers also participate in the cluster, so they count to the total capacity. All other servers that are not coordinators are just "servers". A cluster needs at least one coordination server, and preferrably an odd number for failover reasons. Some cluster configurations are (1C, 0S), (1C, 1S), (3C, 3S), and so on. The configuration you choose will dictate what level of redundancy you can use.

The below docker compose file is a launch of six servers with three of them being coordination servers. Each server in the cluster needs the same cluster file, which contains the list of coordination servers. We map this into the container using the FDB_CLUSTER_FILE_CONTENTS env var. We also map each container to a different host port for diagnostic purposes. Each server also has a volume created to store data between restarts, with it mapped to /var/fdb/data in the container.

Note: The version of FoundationDB you use does not need to be 7.4.1 but it must match the version of the client you install locally. You can also use another Docker container to run your client code instead of installing the client to your machine.

services:
  fdb-coord-1:
    image: foundationdb/foundationdb:7.4.1
    environment:
      FDB_PORT: 4500
      FDB_CLUSTER_FILE_CONTENTS: |
        docker:docker@fdb-coord-1:4500,fdb-coord-2:4501,fdb-coord-3:4502
    ports:
      - 4500:4500
    volumes:
      - fdb-coord-1-data:/var/fdb/data
  fdb-coord-2:
    image: foundationdb/foundationdb:7.4.1
    environment:
      FDB_PORT: 4501
      FDB_CLUSTER_FILE_CONTENTS: |
        docker:docker@fdb-coord-1:4500,fdb-coord-2:4501,fdb-coord-3:4502
    ports:
      - 4501:4501
    volumes:
      - fdb-coord-2-data:/var/fdb/data
  fdb-coord-3:
    image: foundationdb/foundationdb:7.4.1
    environment:
      FDB_PORT: 4502
      FDB_CLUSTER_FILE_CONTENTS: |
        docker:docker@fdb-coord-1:4500,fdb-coord-2:4501,fdb-coord-3:4502
    ports:
      - 4502:4502
    volumes:
      - fdb-coord-3-data:/var/fdb/data
  fdb-server-1:
    image: foundationdb/foundationdb:7.4.1
    depends_on: [fdb-coord-1, fdb-coord-2, fdb-coord-3]
    environment:
      FDB_PORT: 4510
      FDB_CLUSTER_FILE_CONTENTS: |
        docker:docker@fdb-coord-1:4500,fdb-coord-2:4501,fdb-coord-3:4502
    ports:
      - 4510:4510
    volumes:
      - fdb-server-1-data:/var/fdb/data
  fdb-server-2:
    image: foundationdb/foundationdb:7.4.1
    depends_on: [fdb-coord-1, fdb-coord-2, fdb-coord-3]
    environment:
      FDB_PORT: 4511
      FDB_CLUSTER_FILE_CONTENTS: |
        docker:docker@fdb-coord-1:4500,fdb-coord-2:4501,fdb-coord-3:4502
    ports:
      - 4511:4511
    volumes:
      - fdb-server-2-data:/var/fdb/data
  fdb-server-3:
    image: foundationdb/foundationdb:7.4.1
    depends_on: [fdb-coord-1, fdb-coord-2, fdb-coord-3]
    environment:
      FDB_PORT: 4512
      FDB_CLUSTER_FILE_CONTENTS: |
        docker:docker@fdb-coord-1:4500,fdb-coord-2:4501,fdb-coord-3:4502
    ports:
      - 4512:4512
    volumes:
      - fdb-server-3-data:/var/fdb/data

volumes:
  fdb-coord-1-data:
  fdb-coord-2-data:
  fdb-coord-3-data:
  fdb-server-1-data:
  fdb-server-2-data:
  fdb-server-3-data:

To launch the cluster, copy the above yaml into docker-compose.yml, then run docker compose up -d. Afterwards, we need to setup the database:

echo "docker:[email protected]:4500,127.0.0.1:4501,127.0.0.1:4502" > fdb.cluster
fdbcli --exec "configure new double ssd"

The above creates a cluster file then uses fdbcli to configure the database with two replicas of all data and the ssd storage backend. You can then check the status of the database with fdbcli --exec "status". Once it reports healthy, it's ready to use.

As an example, here's a Python script that uses the above FoundationDB cluster:

import fdb

# Must run api_version() before anything else
# 740 is the newest version (does NOT match the version of the client installed)
fdb.api_version(740)

db = fdb.open('fdb.cluster')

db[b"hello"] = b"world"

print(db.get(b"hello"))
<- Back