Before Percona Server for MongoDB 4.4 (PSMDB), the best practice to create an index was doing it in a rolling manner. Many folks used to create directly on Primary, resulting in the first index being created successfully on Primary and then replicated to Secondary nodes.

Starting from PSMDB 4.4, there was a new parameter commitQuorum introduced in the createIndex command. If you are not passing this parameter explicitly with the createIndex command, it will use the default settings on a replica set or sharded cluster and start building the index simultaneously across all data-bearing voting replica set members.

Below is the command used to create an index using commitQuorum as the majority:

The above command will run the index create command on the majority of data-bearing replica set members. There are other options available too when using commitQuorum:

  1. “Voting Members” – This is the default behavior when an index will be created on all data-bearing voting replica set members (Default). A “voting” member is any replica set member where votes are greater than 0.
  2. “Majority” – A simple majority of data-bearing replica set members.
  3. “<int>” – A specific number of data-bearing replica set members. Specify an integer greater than 0.
  4. “Tag name” – A replica set tag name of a node is used.

Now we will see the scenarios of what happens when the index is created with the default and majority commitQuorum.

  1. When all data-bearing replica set members are available, and the index is created with default commitQuorum, below are the details from the Primary and the Secondary nodes. Create index:

    Primary logs:

    Secondary logs:

    Secondary logs:

    We can see the above index was created simultaneously on all the data-bearing voting replica set members.
  2. When one secondary is down, and the index is created with default commitQuorum, below are the details from the Primary and the Secondary nodes.

    Status of nodes:


    Index command:


    Replication status:


    Index status:


    CurrentOp:


    Logs from Primary node:


     Logs from up-and-running Secondary node:


    You can see above that when one node is down, and the index is created with default commitQuorum, the index command will keep running till that third data-bearing voting node comes up. Now we can check if the index is created on Primary or not:


    We can see the index is created, but you will not be able to use the above index as the index is not marked as completed.

    Below is the explain plan of a query, where we can see the query is doing COLLSCAN instead of IXSCAN:

    Now I will bring up the third node, and we will see that index op will complete.

    Index status:

    Now will run the same query, and we can see index (IXSCAN) is getting used as the index was created successfully above:

    Primary logs once the third node came up and the index was created successfully:

    Above, you can see how much time it took to complete the index build; the op was running till the third node was down.

  3. When one secondary is down, and the index is created with commitQuorum as the majority, below are the details from the Primary and the Secondary nodes. Status of nodes:

    Index command:

    Logs from Primary node:

    Above, we can see when one node is down, and we used commitQuorum as majority while creating the index, index op got completed as per expected behavior as two voting (majority) nodes were up and running.

So far, we have discussed how to use commitQuorum and when to use it. Now we will see a scenario when one node (voting) is down for any reason, and someone created an index with default commitQuorum. The op will keep running, and you want to kill the op.

I created the index with the default commitQuorum when one node is down.

Status of nodes:

CurrentOp:

Now you need to kill the above opid to release the above op:

Above, we can see when we killed the op, and the index creation op got killed.

Conclusion

We have seen how commitQuorum works while creating indexes from PSMDB 4.4. Still, the best practice is to create indexes in a rolling manner.

We recommend checking out our products for Percona Server for MongoDB, Percona Backup for MongoDB, and Percona Operator for MongoDB. We also recommend checking out our blog MongoDB: Why Pay for Enterprise When Open Source Has You Covered?

Subscribe
Notify of
guest

0 Comments
Inline Feedbacks
View all comments