Set up a Hyperledger Fabric State Database as a CouchDB Cluster Using Containerization

DLT Labs

Nov 2, 2020·6 min read

Overview

At DLT Labs we use several distributed ledger technologies, but the one that probably gets the most focus is Hyperledger Fabric.

We build enterprise-strength products. In my two years developing on that platform, I’ve encountered some unique challenges relating to the scale and additional requirements of large-scale systems.

We thought we would share one such situation in which we decided to introduce a clustered state database for our Fabric peers and to show interested developers how that was done.

The product was already using Apache CouchDB as its state database in Fabric so that a variety of other components could query the ledger directly and enjoy a richer querying experience. We wanted high availability of the database to those other components, and to ensure that the load generated by them did not impact the availability of the peer. We envisioned that adding additional CouchDB nodes, mediated by a load balancer, could accomplish that.

We also saw an additional benefit in that we could scale the state database by simply adding or removing nodes, without impacting the availability of the database or the peer that relied on it — something that wasn’t possible with a single-node state DB.

Let’s take a look at how we set up this CouchDB cluster and start using it in the Hyperledger Fabric network.

This cluster setup could also be used if we intend to provide multi-region support by keeping different nodes of a CouchDB cluster in different regions, which eventually would help in providing higher availability.

Since in Hyperledger Fabric we launch nodes as containers, we will be setting up the CouchDB cluster using containerization.

This process consists of 5 main parts:

1. Arranging the configuration files needed
2. Configuring the Couch DB cluster
3. Verifying the CouchDB Cluster Setup
4. Important Configuration Parameters
5. Using CouchDB Cluster in Hyperledger Fabric

Let us begin.

1. Arranging the Configuration Files Needed

This is the docker-compose file which we need to consume for setting up a CouchDB cluster:

version: ‘3’
networks:
  fabric:
          external:
                  name: example
services:
  couchdb1.example.com:
  container_name: couchdb1.example.com
  image: hyperledger/fabric-couchdb:amd64–0.4.15
  environment:
     - COUCHDB_SECRET=0123456789
     - NODENAME=couchdb1.example.com
     - ERL_FLAGS=-setcookie “brumbrum”
     - COUCHDB_QUERY_SERVER_JAVASCRIPT=”/usr/bin/couchjs -S 8589934592 /usr/share/server/main.js”
  volumes:
      - /data/couchdb1:/opt/couchdb/data
  ports:
    - “5001:5984”
  networks:
    - fabric  couchdb2.example.com:
    container_name: couchdb2.example.com
    image: hyperledger/fabric-couchdb:amd64–0.4.15
    environment:
       - COUCHDB_SECRET=0123456789
       - NODENAME=couchdb2.example.com
       - ERL_FLAGS=-setcookie “brumbrum” 
       - COUCHDB_QUERY_SERVER_JAVASCRIPT=”/usr/bin/couchjs -S 8589934592 /usr/share/server/main.js”
    volumes:
       - /data/couchdb2:/opt/couchdb/data
    ports:
       - “5002:5984”
    networks:
       - fabric  couchdb3.example.com:
    container_name: couchdb3.example.com
    image: hyperledger/fabric-couchdb:amd64–0.4.15
    environment:
       - COUCHDB_SECRET=0123456789
       - NODENAME=couchdb3.example.com
       - ERL_FLAGS=-setcookie “brumbrum”
       - COUCHDB_QUERY_SERVER_JAVASCRIPT=”/usr/bin/couchjs -S 8589934592 /usr/share/server/main.js”
               volumes:
       - /data/couchdb3:/opt/couchdb/data
    ports:
       - “5003:5984”
    networks:
       - fabric

To begin, we need to run the following command in the same location where the yaml file is present.

docker-compose up -d

We can check it by running docker ps. The output for this should look like:

2. Configuring the CouchDB Cluster

We need to do the following steps to complete setting up the couch cluster:

>> Step 1: To configure CouchDB in the cluster we need to open the Fauxton web interface. This can be accessed through this link.

>> Step 2: We click on the Configure a Cluster button.

After that, we will reach the page where we need to enter the username and password of the CouchDB cluster nodes.

Here we are using admin as both username and password (which you would not do in practice, of course).

>> Step 3: We specify the number of nodes that we are setting up for this cluster. Here, we have given 3.

In the remote host option, we enter the name of the other two nodes. For example, if we have opened the Fauxton interface of couchDB1 then we must provide the other two nodes’ names.

>> Step 4: Click on the button Configure Cluster and Fauxton will complete the cluster setup.

3. Verifying the CouchDB Cluster Setup

To verify if the cluster setup is complete and all nodes are in sync we need to execute the commands below:

>> Step 1:To check the membership of the cluster we need to hit the _membership endpoint:

curl http://username:password@localhost:5984/_membership

Here is an example of the expected output:

{"all_nodes": ["couchdb@couchdb1","couchdb@couchdb2.example.com","couchdb@couchdb3.example.com"],"cluster_nodes":["couchdb@couchdb1","couchdb@couchdb2.example.com","couchdb@couchdb3.example.com"]}

>> Step 2:To verify the state of cluster setup we need to hit the _cluster_setup endpoint.

curl http://admin:admin@localhost:5984/_cluster_setup

This is what the expected output should look like:

{"state":"cluster_finished"}

We can also verify the state of the setup from the Fauxton interface:

4. Important Configuration Parameters

In the yaml file, you probably noticed a number of parameters that we used for setting up the cluster.

These are some of the important ones to understand:

• COUCHDB_SECRET
  It sets the secret identifier expected to be used by all nodes of the CouchDB cluster. Path of this is `/opt/couchdb/etc/local.d/docker.ini
• NODENAME
  Inside the container, we need to set the name of the CouchDB node. It is read as couchdb@${NODENAME}`.
  The path of this is `/opt/couchdb/etc/vm.args`.
• ERL_FLAGS
  This key also is an identifier that will be common for all nodes in the CouchDB cluster. When any node tries to re-join the cluster, the value of this key confirms if it was initially part of the cluster or not.
• COUCHDB_QUERY_SERVER_JAVASCRIPT
  By default, the runtime allocation of storage for query processing, via the couchjs parameter, is set to 64 MB.

However, we can increase its value according to our use case by changing the value of this environment variable.

5. Using CouchDB Cluster in Hyperledger Fabric

We must update the value of the following env variables in the following peer container: CORE_LEDGER_STATE_COUCHDBCONFIG_COUCHDBADDRESS

Instead of pointing to the single CouchDB node, this will need to point to whatever is being used to load balance traffic to the CouchDB cluster, such as an HAProxy® load balancer or AWS Elastic Load Balancer URL.

This cluster setup could also be used if we intend to provide multi-region support. We can keep different nodes of a CouchDB cluster in different regions (for example, in a cloud-based deployment).

This eventually helps us in providing higher availability. The diagrammatic representation of the whole setup looks like this:

Conclusion

Through this article, we have examined how we can set up a Hyperledger Fabric state database as a CouchDB cluster using containerization to support direct querying requirements and increased request loads in an enterprise scenario, with high availability to all consumers.

We hope you found this topic and its explanation interesting and helpful.

Hyperledger Fabric is a project hosted by The Linux Foundation® and Hyperledger is its registered trademark. Apache CouchDB and CouchDB are trademarks of the Apache Software Foundation. AWS is a trademark of Amazon.com, Inc. or its affiliates in the United States and/or other countries. Docker is a trademark or registered trademark of Docker, Inc. in the United States and/or other countries.

DLT Labs is a trademark of DLT Global, Inc.

Author — Deepak Singh, DLT Labs™

About the Author: Deepak is currently working with the DL Tools team, and enjoys understanding code & algorithms at an elementary level.