Before MongoDB 4.4, we had a tool to use to analyze slow queries from the log file called mplotqueries from mtools (thanks, Thomas Rueckstiess, for the tool and the inspiration).
Mplotqueries is a great tool that helped analyze slow queries on MongoDB Community, Enterprise, and Percona for MongoDB versions. But when MongoDB 4.4 was released and the log file format changed (the new format is JSON), mplotqueries didn’t work anymore.
That’s why I wrote a simple Python script to do sort of the same thing as mplotqueries.
The script uses Pandas and Plotly to create the data frames and the dashboard using the log file as input. You can run it wherever you can run Python (Linux, Windows, macOS).
The usage is straightforward; you need to clone the repository from GitHub: https://github.com/zelmario/MongoDB-Log-Plotter
1 | git clone https://github.com/zelmario/MongoDB-Log-Plotter.git |
And launch the script with any log as an argument:
1 | python MongoDB_log_plotter.py /path/to/log/file.log |
You may need to install some dependencies like Pandas, Plotly, Dash, etc. Check the requirements on the GitHub repository.
At the time of this writing, the script is reporting:
- Graph with slow queries (duration/timestamp) grouped by namespace
- Graph with connections (total connections/timestamp)
- Graph with error/information messages (messages/timestamp)
After processing the log file, the script will launch a small web server on http://localhost:8050/. If you go to that address with your browser, you will see a dashboard with the log data:
At the top of the screen, you will have three plots: Slow queries, Connections, and Errors:
Slow queries
The slow queries from the log file are plotted by duration(ms)/timestamp. They are grouped by Namespace.
The plot is limited to 10,000 queries to improve performance, and the script takes a random sampling to plot the queries. You can filter by namespace, and if you click on a dot (a query), the query used will show at the bottom of the plots.
Connections
Total connections on the node over time:
Errors
All the messages about what’s happened in the node: 
The three plots share the X-axis so if you zoom in on one of them, the others will zoom in too.
Below the plots are three tables with some information:
Slow queries: grouped by namespace, the table shows the count and mean duration of the queries:
Additional Information: Some useful information like MongoDB version, Node name, Replica name, etc.
Errors and information messages: A simple table to count the messages and errors that appear in the log file:
It is a very simple script, but it helps a lot when we want to analyze slow queries or if we need to do an RCA of any issue. While I may not be a professional developer, contributions towards enhancing the script are warmly welcomed.
Percona Distribution for MongoDB is a source-available alternative for enterprise MongoDB. A bundling of Percona Server for MongoDB and Percona Backup for MongoDB, Percona Distribution for MongoDB combines the best and most critical enterprise components from the open source community into a single feature-rich and freely available solution.
Alternatively you can install. Elasticsearch and pipe all that formatted how you want ontobelsdtic with logstash , snd graph it with insane. Been doing that for k dr 20 years since v1 of mongodb. Solved sll the issues you had…plus can store all the wueries..think the most I had was around 2nm )yes billion) on elastic..so you get the full story. Not missing any queries means you don’t lose hhe first one get caused an issue, because it’s malformed.
gor h the ose with DTafog..you can use that as well…or instead of,