The Netflix TechBlog

SEPTEMBER 2, 2020

Edgar helps Netflix teams troubleshoot distributed systems efficiently with the help of a summarized presentation of request tracing, logs, analysis, and metadata. In an earlier blog post, we discussed Telltale , our health monitoring system. by Elizabeth Carretto Everyone loves Unsolved Mysteries. What is Edgar?

Latency 296

Latency Transportation Engineering Traffic 296

Percona

NOVEMBER 3, 2023

And now, execute the benchmark: -- execute the following on the coordinator node pgbench -c 20 -j 3 -T 60 -P 3 pgbench The results are not pretty. In order to speed up the benchmark indexes must be added. psql pgbench <<_eof1_ qecho adding node citus3. select citus_add_node('citus3', 5432); qecho rebalancing shards across TWO nodes.

Database 102

Database Benchmarking Latency C++ 102

Dynatrace

JANUARY 31, 2024

While off-the-shelf models assist many organizations in initiating their journeys with generative AI (GenAI), scaling AI for enterprise use presents formidable challenges. This blog post explores how AI observability enables organizations to predict and control costs, performance, and data reliability.

Cache 209

Cache Azure Infrastructure Monitoring 209

Dynatrace

FEBRUARY 4, 2021

As a discipline, SRE focuses on improving software system reliability across key categories including availability, performance, latency, efficiency, capacity, and incident response. The post Site reliability engineering: 5 things you need to know appeared first on Dynatrace blog. SRE drives a “shift left” mindset.

Engineering 257

Engineering DevOps Government Latency 257

Dynatrace

APRIL 25, 2023

In this blog post, we will discuss some of these challenges and how to overcome them. Higher latency and cold start issues due to the initialization time of the functions. Data visualization : how to present, explore and interpret observability data from serverless functions intuitively, clearly, and holistically?

Serverless 213

Serverless Lambda Azure AWS 213

The Netflix TechBlog

APRIL 27, 2023

The internals here are outside the scope of this blog post. The fundamental idea behind Telltale is to detect anomalies on SLI metrics (for example, latency, error rates, etc). Next, the Alerting service (via the atlas-eval library) maps the received data points for a data expression to the alert query that needs them.

Storage 288

Storage Cache Metrics Database 288

Brendan Gregg

JULY 4, 2021

This was a chance to talk about other things I've been working on, such as the present and future of hardware performance. The video is on [youtube]: The slides are on [slideshare] or as a [PDF]: I work on many areas of performance, but recently I've had a lot of demand to talk about BPF. Ford, et al., “TCP

Performance 140

Performance Latency Hardware Storage 140

The Netflix TechBlog

FEBRUARY 20, 2019

Today we are excited to announce latency heatmaps and improved container support for our on-host monitoring solution?—?Vector?—?to Remotely view real-time process scheduler latency and tcp throughput with Vector and eBPF What is Vector? to the broader community. Vector is open source and in use by multiple companies.

Performance 211

Performance Latency Open Source Metrics 211

The Netflix TechBlog

SEPTEMBER 3, 2021

Remote calls are never free; they impose extra latency, increase probability of an error, and consume network bandwidth. There are a number of utilities and conventions on how to use this message when it is present in an RPC request. FieldMask is a protobuf message. Bella Ciao Protobuf FieldMask is a simple, yet powerful concept.

Design 245

Design Java Efficiency Code 245

John McCalpin

DECEMBER 6, 2016

The Xeon Phi x200 (Knights Landing) has a lot of modes of operation (selected at boot time), and the latency and bandwidth characteristics are slightly different for each mode. This reduces the number of “hops” required for request/response/coherence messages on the mesh, and should reduce both latency and contention.

Latency 40

Latency Cache Testing Systems 40

The Netflix TechBlog

MARCH 4, 2024

In this blog post, we present our project on Auto Remediation, which integrates the currently used rule-based classifier with an ML service and aims to automatically remediate failed jobs without human intervention. Upon further profiling, we found that most of the latency came from the candidate generated step (i.e.,

Tuning 210

Tuning Efficiency Big Data Engineering 210

Dynatrace

APRIL 6, 2022

Observability can identify the baseline user experience and allow teams to improve it by optimizing page load times or reducing latency. Cloud environments present IT complexity challenges that don’t exist in on-premises data centers. appeared first on Dynatrace blog. Why full-stack observability matters.

DevOps 235

DevOps Innovation Infrastructure Analytics 235

Dynatrace

FEBRUARY 4, 2021

As a discipline, SRE focuses on improving software system reliability across key categories including availability, performance, latency, efficiency, capacity, and incident response. The post Site reliability engineering: 5 things to you need to know appeared first on Dynatrace blog. SRE drives a “shift left” mindset.

Engineering 181

Engineering DevOps Government Latency 181

ACM Sigarch

MAY 31, 2023

However, building and utilizing HCM presents challenges, including interconnecting various memory technologies (e.g., even lowered the latency by introducing a multi-headed device that collapses switches and memory controllers. Figure 2: Latency characteristics of memory technologies (source: Maruf et al.,

Latency 52

Latency Hardware Cache Architecture 52

Percona

MARCH 29, 2023

In this blog, we will discuss both data and network-level compression offered in MongoDB. When this data block is read, it decompresses it in memory and presents it to the incoming request. You can learn more about it in the blog MongoDB: Why Pay for Enterprise When Open Source Has You Covered?

Storage 105

Storage Network Open Source Latency 105

The Netflix TechBlog

AUGUST 30, 2021

In this blog post, we will focus on the latter feature set. The challenge, then, is to be able to ingest and process these events in a scalable manner, i.e., scaling with the number of devices, which will be the focus of this blog post. In particular, the Kafka integration is the most relevant for this blog post.

Latency 213

Latency Traffic Transportation Hardware 213

Adrian Cockcroft

JANUARY 29, 2023

The mean and percentile measurements hide this structure, but the rest of this post will show how the structure can be measured and analyzed so that you can figure out a useful model of your system, understand what is driving the long tail of latencies and come up with better SLAs and measures of capacity.

Lambda 98

Lambda Latency Cache C++ 98

Adrian Cockcroft

AUGUST 1, 2022

photo by Adrian I gave a talk at Monitorama in Portland Oregon in June, which set out the idea that carbon is just another metric to monitor, and that in a few years most of the monitoring and performance tuning tools are going to be reporting and optimizing for carbon alongside latency, throughput, availability and cost.

AWS 52

AWS Virtualization DevOps Latency 52

The Netflix TechBlog

FEBRUARY 9, 2021

We need to be able to easily determine what imagery is present for a given platform, region, and language. Server-generated assets, since client-side generation would require the retrieval of many individual images, which would increase latency and time-to-render. The imagery needs to be localized.

Engineering 165

Engineering Storage Latency Entertainment 165

ACM Sigarch

MARCH 18, 2019

With two blog posts, we argue for more use of simple models beyond Amdahl’s Law. This Part 1 discusses Bottleneck Analysis and Little’s Law, while Part 2 presents the M/M/1 Queue. To this end, we present three simple models that we find useful like Amdahl’s Law, split here into two blog posts. Answered in Part 2.).

Systems 60

Systems Latency Performance Analytics 60

Adrian Cockcroft

JANUARY 20, 2023

The next day Jack presented his Turing Award Lecture as the keynote for the event ( HPCwire has a good summary of the whole talk), and later in the the week he was on a panel discussion on “Reinventing HPC” where he repeated the point. I presented a keynote for Sun at Supercomputing 2003 in Phoenix Arizona and included the slide shown below.

Architecture 52

Architecture Latency Benchmarking AWS 52

Percona

JULY 20, 2023

Meanwhile, Hans-Jürgen Schönig’s presentation , which brought up the old discussion of Unix socket vs. TCP/IP connection, triggered me to write about other aspects of network impact on performance. Following wait events are captured from a really fast/low latency network. See the next session for details.

Network 61

Network Performance Latency Servers 61

ACM Sigarch

MARCH 25, 2019

With two blog posts, we argue for more use of simple models beyond Amdahl’s Law. Part 1 previously discussed Bottleneck Analysis and Little’s Law, while this post (Part 2) presents the M/M/1 Queue. How many buffers are needed to track pending requests as a function of needed bandwidth and expected latency? and 1/S as ?.

Systems 53

Systems Latency Performance C++ 53

The Netflix TechBlog

MARCH 13, 2020

As mentioned in our earlier blog post , Intel and Netflix have been collaborating on the SVT-AV1 encoder and decoder framework since August 2018. In this tech blog, we will report the current status of the SVT-AV1 project, as well as the characteristics and performance of the encoder and decoder.

Open Source 154

Open Source Efficiency C++ Speed 154

CSS Wizardry

MARCH 3, 2019

This means that, if the Cache-Control: max-age=60 header is still present, the cached file’s 60 seconds starts again. A great candidate for must-revalidate is a blog like mine: static pages that seldom change. We can completely cut out the overhead of a roundtrip of latency. 120 seconds overall cache time for one file.

Cache 264

Cache Latency Strategy Servers 264

Dynatrace

DECEMBER 22, 2019

Metrics for each service instance are presented in detailed charts—see the example for ECS below. The example below visualizes average latency by API name and stage for a specific AWS API Gateway. The post Get up to 300 new metrics out of the box with AWS supporting services (GA) appeared first on Dynatrace blog. Requirements.

AWS 137

AWS Metrics IoT Storage 137

Dynatrace

DECEMBER 22, 2019

Metrics for each service instance are presented in detailed charts—see the example for ECS below. The example below visualizes average latency by API name and stage for a specific AWS API Gateway. The post Get up to 300 new metrics out of the box with AWS supporting services (GA) appeared first on Dynatrace blog. Requirements.

AWS 133

AWS Metrics IoT Storage 133

Percona

MAY 2, 2023

If you want to know more about how to calculate a good redo log size, check out this blog post: MySQL 8.0 The answer does not consider the queue or latency of the sample, which could indicate a disk with issues. Also, the redo log capacity is not the same as the redo log buffer, so it mixes different technologies and concepts.

Social Media 119

Social Media Database Servers Cache 119

Brendan Gregg

SEPTEMBER 25, 2021

A Cassandra database cluster had switched to Ubuntu and noticed write latency increased by over 30%. The change was obvious in the production graphs, showing a drop in write latencies: Once tested more broadly, it showed the write latencies dropped by 43%, delivering slightly better performance than on CentOS. in total.

Speed 126

Speed Java AWS Virtualization 126

Dynatrace

DECEMBER 22, 2021

Metrics are measures of critical system values, such as CPU utilization or average write latency to persistent storage. A platform approach, on the other hand, presents a more effective option for understanding observability as a whole. The post Observability platform vs. observability tools appeared first on Dynatrace blog.

Artificial Intelligence 183

Artificial Intelligence Metrics Architecture DevOps 183

The Netflix TechBlog

DECEMBER 21, 2020

This article will list some of the use cases of AutoOptimize, discuss the design principles that help enhance efficiency, and present the high-level architecture. These principles reduce resource usage by being more efficient and effective while lowering the end-to-end latency in data processing. More processing resources.

Storage 203

Storage Latency Efficiency Data Engineering 203

Brendan Gregg

JULY 2, 2021

Here are the top ten tools you can run and present as a generic BPF observability dashboard, along with suggested visualizations: Tool Shows Visualization. biolatency Disk I/O latency histogram heat map. runqlat CPU scheduler latency heat map. biolatency Disk I/O latency histogram heat map. opensnoop Files opened table.

Latency 145

Latency Cache Energy Systems 145

The Netflix TechBlog

FEBRUARY 9, 2021

For more background on Growth Engineering and the signup funnel, please have a look at our previous blog post that covers the basics. When we merge these two concepts together and present them to the customer, we have the plan selection page (shown above). Conclusion This work was a large cross-functional effort.

Engineering 142

Engineering Scalability Architecture Innovation 142

Brendan Gregg

JULY 4, 2021

This was a chance to talk about other things I've been working on, such as the present and future of hardware performance. The video is on [youtube]: The slides are [here] or as a [PDF]: first prev next last / permalink/zoom I work on many areas of performance, but recently I've had a lot of demand to talk about BPF. Ford, et al., “TCP

Performance 52

Performance Latency Hardware Storage 52

Speed Curve

FEBRUARY 5, 2024

Latency – How much time does it take to deliver a packet from A to B. Here are a couple of great blog posts from last year's Web Performance Calendar. Today, it's possible to add these headers from your CDN with ease, if they aren't already set up out of the box. Also measured by round trip time (RTT). Definitely worth a read!

Servers 57

Servers Cache Retail Benchmarking 57

Dotcom-Montior

MAY 12, 2020

Websites are now more than just the storage and retrieval of information to present content to users. Network latency. Network Latency. With the evolution of cloud technologies, such as Single Page Applications (SPAs), Web APIs, and Model View Controller (MVC), network latency has become a crucial factor to be monitored.

Monitoring 63

Monitoring Entertainment Hardware Latency 63

All Things Distributed

JUNE 21, 2018

This doesn't mean relational databases do not provide utility in present-day development and are not available, scalable, or provide high performance. Use cases such as gaming, ad tech, and IoT lend themselves particularly well to the key-value data model where the access patterns require low-latency Gets/Puts for known key values.

Database 167

Database AWS Games Latency 167