End-to-end request monitoring for popular Python frameworks with OneAgent SDK

As part of the Platform Extensions team, I’m one of those responsible for services that include the Dynatrace OneAgent SDKs, which are libraries that allow us to extend end-to-end visibility for technologies and frameworks for which there is no code module available yet. A common task we do is helping customers instrument Python applications with our OneAgent SDK.

To use the SDK, there are a few steps that you need to follow:

1. Import the SDK
2. Instrument key portions of your application
   • Web Requests entry points
   • Database calls
   • Messaging calls
   • Caching
   • Outgoing Web Requests
   • Custom code

The Dynatrace OneAgent SDK is very intuitive. It follows the same specification for all languages and has powerful features to trace different types of transactions. Sometimes, however, the codebase is just not a very good fit. Maybe there are external HTTP Calls in several different places, or dozens of different views with their own database calls. These things can make the instrumentation process difficult with a custom SDK.

Introducing Autodynatrace

After working on some projects, which depend on the OneAgent SDK for Python, you realize that most applications use a set of libraries. If we added support to automatically instrument these libraries without lengthy code change, we can help save developers many hours and make the monitoring experience easier. These common libraries include Flask, Django, Redis, MongoDB, requests, SQLAlchemy amongst others.

Autodynatrace is a python (2.7+, 3.4+) module that solves this problem. With a single import this module will instrument all the key libraries your application uses, utilizing the OneAgent SDK underneath and collecting transactional data (PurePaths) from your apps. The libraries supported include:

• DjangoFlasks
• SQLAlchemy Redis
• Celery
• RabbitMQ
• And many others

The library is open source, and we plan to add support a plethora of different modules, as we attempt to cover more and more Python use cases.

Walk the talk

Let’s go over instrumenting a real application with Autodynatrace, I chose an interesting Django app – https://github.com/n4nagappan/reddit-django – but any app would do!

The steps to get this app running are:

1. Clone the repo git clone https://github.com/n4nagappan/reddit-django
2. Create a virtual environment, and setup settings_secret.py (This is specific to this project, instructions in the README.md)

The steps to instrument this application with Autodynatrace, are:

1. pip install autodynatrace
2. Add wrapper.djangoto the Django INSTALLED_APPS.

We can now run the application with the standard python manage.py runserver, and Autodynatrace has already instrumented several different places of the app.

Demo

Now the application is running, let’s test it! Make a call to http://localhost:8000/subreddits/funny

Oops, I received an error!

We need to make sure that the reddit-django has a redis instance for the cache, and we can easily spin one up with docker docker run -d -p 6379:6379 redis, If we don’t start the redis instance, Autodynatrace will tell us exactly what the error with our app was:

I can see the entire flow, the web request, the view being called, the redis call, all the response times and even the exception:

Remember, this was done without changing any code. We simply added a single line to our application settings.

Let’s try the same thing again, but with Redis running. This is what I get on my console since I have DEBUG logging enabled:

And this is our service flow:

And the PurePath:

We can see that once the request comes in it:

1. Gets passed to the SubredditAnalysisDetailedView (This view attempts to get the information from Redis (the cache)
2. Doesn’t find it, so it makes HTTP calls to the Reddit API
3. Executes 100 selects, one for each of the top 100 posts
4. Saves the information to Redis

Dynatrace also created:

• A service for the Django requests and views
• The sqllite database
• Correlation of the Redis calls to our Redis container
• The outgoing remote calls to the Reddit API

What about proprietary code?

What if you don’t use these frameworks? Let’s make a simple app that calls our Reddit analyzer:

How do we monitor the call_reddit method? Autodynatrace has a very easy to use annotation, @Autodynatrace.trace, which you can see in the image below

And just like that the PurePaths are connected and we can analyze the entire call stack:

We hope this gets you excited about instrumenting your Python applications with Autodynatrace! You can find more information at https://github.com/dynatrace-oss/OneAgent-SDK-Python-AutoInstrumentation