As almost a tradition, a newer major release of MongoDB is born every year, and this year, it’s no different. Many changes and new features are brought to the system, and as part of keeping in tune with the changes and how they can impact us, we go through the changes to better understand them. From that, this article is born.

Our goal here is to review five new changes or implementations you should know in MongoDB 7.0. Not only that, the idea is to present you with changes flying under the radar, which, in my understanding, can influence your daily operation. Without further ado, let’s get started.

1. Dynamic WiredTiger tickets

As a brief recap, WiredTiger tickets in MongoDB serve as the concurrency control mechanism within the WiredTiger storage engine. These tickets are categorized into Read tickets and Write tickets. 

When multiple operations, such as reads and writes, attempt to access the database concurrently, WiredTiger uses tickets to ensure these operations do not conflict in a way that would compromise data integrity or performance. 

Each transaction obtains a ticket at its onset and releases it back into the ticket pool upon completion. Since the early days of the WiredTiger engine, the number of tickets are governed by: 

Those variables control the number of concurrent read/write transactions (tickets) allowed into the WiredTiger storage engine, which by default before MongoDB 7.0 was 128 tickets:

From MongoDB 7.0:

Tickets are now dynamically set:

A whole new algorithm to manage the tickets dynamically was implemented, which can be checked here. With that new change, it is worth mentioning to check the WiredTiger metrics in their entirety and not as a separate piece, as it can lead to wrong conclusions.

My personal take on that change: It’s an interesting and valid approach, mainly because it’s common to find a system with limited resources struggling with performance due to an overestimated threshold (very common on many parameters by default in MongoDB), adding unnecessary pressure on it. The dynamic allocation might have flaws, but it’s definitely a good addition.

2. Shard Key Analyzer helper – .analyzeShardKey()

The shard key is a critical component of your cluster, as it dictates the distribution of your data through the shards, and that’s where the problem lies.

A considerable part of shard cluster issues are related to bad shard key choices; we are not going through each component as it’s not the scope of this article, but for a good shard key, you must watch for:

It was immutable in the past, but now (+4.4), it has new features to improve or even change your shard key. Like refine or reshard it. 

But still, you had to manually query your collection to answer those questions, if your key has a good cardinality, frequency, and so on.

MongoDB 7.0 adds two new methods that return metrics that are very useful in evaluating either existing or potential new shard keys. The methods are:

The output provides three documents that hold their statistics:

  • keyCharacteristics: provides metrics about the cardinality, frequency, and monotonicity of the shard key.
  • readDistribution/writeDistribution: provides metrics about query routing patterns and the hotness of shard key ranges.

Here, we have an example of analyzeShardKey() on two candidates’ shard keys.

Since the collection has not been sharded yet, we set readWriteDistribution: false. You can use the new method to help you evaluate one shard key that has been in use or a future one as follows:

With that output, you can understand if your key has high cardinality by checking the total docs divided by the number of distinct values.

The mostCommonValues field adds the understanding of the frequency of a value; if that’s high, it might indicate possible jumbo in the future as the values often repeat, which increases the chunk, thus becoming indivisible.

Last but not least, analyzeShardKey() uses a query sampling configuration to calculate the metrics; as part of new features, you have the ability to configure query sampling via configureQueryAnalyzer().

As we can see, the analyzeShardKey() method is a handy implementation, especially because it supports a more assertive evaluation of the shard key, a critical component of the sharded cluster, and often a source of issues.

3. Cluster metadata checker – checkMetadataConsistency()

Alongside the shard key analyzer, the metadata checker is another step forward regarding additional instrumentation on the DBA’s life.

The sharded cluster is great but, at the same time, a complex feature, especially when we start to understand how It works behind the scenes to comply with things like consistency. 

For those who have to administrate a sharded cluster, you likely have faced problems with metadata inconsistency:

  • Collections with different UUIDS, routing tables with overlapping ranges, and many other unpleasant issues.

The problem here is that dealing with metadata is a challenging task, mainly because modifying it can lead to a series of issues and put the cluster in an undesired state, something that we run from.

Also, metadata inconsistency might be living silently in your cluster, and you will learn that only when a particular operation/scenario happens.

The checkMetadataConsistency() helper is a great addition to help you track that:

It performs a series of consistency checks (13 checks) on sharding metadata and indexes(not default), looking for inconsistencies.

That’s very welcome after upgrade/downgrade processes when metadata is touched after FCV change. It’s also helpful when performing regular checks, avoiding getting caught off guard by providing a thorough scan of your cluster on such delicate components.

4. Linux users can’t change taskExecutorPoolSize from 1

If you are a Mac or Windows user, this change doesn’t affect you, but for Linux users, you can not modify your connection pool from mongos:

The taskExecutorPoolSize has been using default:1 since MongoDB 4.0. After the rapid release 6.2 (available only for Atlas users), that can not be modified anymore; That change was then introduced to 7.0.

Although taskExecutorPoolSize has been using 1 for a long time, it was possible to “tweak,” which usually brings more bad than good because that parameter itself does not work alone.

The concept and problem behind that is simple:

  • To find the maximum number of outbound connections each TaskExecutor connection pool can open to any given mongod instance, we do the following calculation:

So, 1 Task pool can open up to 32767 connections to mongod.

Now, let’s assume you have a four-core CPU and you have set taskExecutorPoolSize=4.

  • Then 4 * 32767=131068

Not only that, but you also have more than one mongoS; let’s also assume four distinct mongoS, which is a good practice:

  • 4 * 131068 = 524272

At the end of the day, your cluster can open up to half a million connections between mongoS and mongod.

That’s a common source of issue because if your application also does not enforce any connection constraint and starts to request connections to fulfill an operation, you can easily end up with a storm of connections performing thousands of operations in a short span of time, which can definitely put pressure on the cluster and slow down the performance.

Not being able to modify taskExecutorPoolSize is a good action to protect against overestimated parametrization.

5. Stricter downgrade policy

The upstream documentation highlights the following:

Starting in MongoDB 7.0, binary downgrades are no longer supported for MongoDB Community Edition.

In previous releases, the downgrade process was relatively simple:

  1. Have a valid backup.
  2. Remove backward-incompatible features.
  3. Adjust the FCV to the previous version.
  4. Bounce the database, replacing the binaries with the older release.

Now, you might be thinking – “What if I perform a downgrade like we used to do?”

  • At the moment of writing this article, there was no blocker on the MongoDB side, such as a warning or an impossibility of moving forward with the process:

However, even though you have ensured that all backward incompatibilities are cleared, which may also require possible data removal due to incompatibilities, keep in mind this action is at your own risk.

Moving on, MongoDB 7.0 also brought another unsupported action.

Let’s assume you are catching up with the releases and want to be at the latest release available, and then you upgraded from 4.4.x -> 5.0.x -> 6.0.x -> 7.0.x.

After that, you notice a problem and want to Downgrade to 4.4.x or even 5.0x. For MongoDB 7.0, that’s not supported anymore, from the upstream documentation:

MongoDB only supports single-version downgrades. You cannot downgrade to a release that is multiple versions behind your current release.

Although not optimal, it’s not uncommon to see companies trying to catch up with the newest releases due to EOL dates and performing several upgrades in a very short period of time, which can now be a problem if rollback is needed.

You might have a similar question, and the answer is similar.

At the moment of writing this article, there was no blocker on the MongoDB side, such as a warning or an impossibility of moving forward with the process. 

However, even though you have ensured that all backward incompatibilities are cleared, and you should be good to go, keep in mind this action is at your own risk. 

As a final note, either upgrading or downgrading, It is always recommended to test the procedure in a lower/QA environment that mirrors the production environment, allowing you to get used to the process and clearing any doubts you may have.

Conclusion

MongoDB 7.0 brought several changes in this new release, and the above are the ones we believe might directly impact your daily use, in a good or bad way, such as the new policies about the downgrade. Speaking about that, if you are not comfortable performing upgrades/downgrades or are looking for specialized support for your MongoDB environment, contact us, and our team will assist you with the best service for you.

Contact us

 

If you have any questions or found any other interesting change on MongoDB 7.0, feel free to share in the comment session or also open a question in our Community Forum; why not a part two of this article?

Percona Distribution for MongoDB is a freely available MongoDB database alternative, giving you a single solution that combines the best and most important enterprise components from the open source community, designed and tested to work together.

 

Download Percona Distribution for MongoDB Today!

Subscribe
Notify of
guest

0 Comments
Inline Feedbacks
View all comments