article thumbnail

Building Netflix’s Distributed Tracing Infrastructure

The Netflix TechBlog

This was the most important question we considered when building our infrastructure because data sampling policy dictates the amount of traces that are recorded, transported, and stored. Our trace data collection agent transports traces to Mantis job cluster via the Mantis Publish library.

article thumbnail

Towards a Reliable Device Management Platform

The Netflix TechBlog

MQTT is an OASIS standard messaging protocol for the Internet of Things (IoT) and was designed as a highly lightweight yet reliable publish/subscribe messaging transport that is ideal for connecting remote devices with a small code footprint and minimal network bandwidth. This effectively ensures exactly-once processing on a per-event basis.

Latency 213
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

The road to observability with OpenTelemetry demo part 2: OpenTelemetry configuration and instrumenting applications

Dynatrace

/api/v2/otlp/v1/metrics'; $apiKey = '[API_KEY]'; // Set up trace environment $transport = (new OtlpHttpTransportFactory())->create($traceURL, 'application/x-protobuf', [ 'Authorization' => 'Api-Token '.

article thumbnail

Delta: A Data Synchronization and Enrichment Platform

The Netflix TechBlog

Data Pipeline using Delta In the following sections, we are going to describe the Delta-Connector that connects to a datastore and publishes CDC events to the Transport Layer, which is a real-time data transportation infrastructure routing CDC events to Kafka topics. In addition, we support Cassandra (multi-master).

article thumbnail

Evolution of Netflix Conductor:

The Netflix TechBlog

The Netflix Media Database (NMDB) is one such example. This addition also provides the option to use the Elasticsearch RestClient instead of the Transport Client which was enforced in the previous version. Many of the Netflix Content and Studio Engineering services rely on Conductor for efficient processing of their business flows.

Lambda 189
article thumbnail

Kubernetes vs Docker: What’s the difference?

Dynatrace

Just like shipping containers revolutionized the transportation industry, Docker containers disrupted software. This opens the door to auto-scalable applications, which effortlessly matches the demands of rapidly growing and varying user traffic. Containers can be replicated or deleted on the fly to meet varying end-user traffic.

article thumbnail

DBLog: A Generic Change-Data-Capture Framework

The Netflix TechBlog

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. In databases like MySQL and PostgreSQL, transaction logs are the source of CDC events. Designed with High Availability in mind.

Database 212