Brendan Gregg

DECEMBER 30, 2017

My last talk for 2017 was at AWS re:Invent, on "How Netflix Tunes EC2 Instances for Performance," an updated version of my [2014] talk. A video of the talk is on youtube : The slides are on slideshare : I love this talk as I get to share more about what the Performance and Operating Systems team at Netflix does, rather than just my work.

Tuning 52

Tuning AWS Best Practices Network 52

IO River

NOVEMBER 2, 2023

Enter the concept of Content Delivery Networks (CDN) with Multiple Origins Load Balancing. ‍ Think of a CDN Load Balancer (or LB, if you like to keep things short and sweet) as the internet’s traffic police. ‍But how does it decide where to send this traffic? That’s where rules come in. What is “Origin”?

Traffic 52

Traffic Cache Servers Latency 52

IO River

NOVEMBER 15, 2023

The OSI Model is like a layer cake of how data moves through networks. This means it specifically looks at the content of the data (like the writing inside our letters from the analogy) rather than just the envelope.So, what is the difference between a Firewall at the application level and network level?Network-level

Traffic 52

Traffic Network Logistics Architecture 52

The Netflix TechBlog

OCTOBER 19, 2020

which is difficult when troubleshooting distributed systems. Reconstructing a streaming session was a tedious and time consuming process that involved tracing all interactions (requests) between the Netflix app, our Content Delivery Network (CDN), and backend microservices.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

The Netflix TechBlog

AUGUST 30, 2021

System Setup Architecture The following diagram summarizes the architecture description: Figure 1: Event-sourcing architecture of the Device Management Platform. When a new hardware device is connected, the Local Registry detects and collects a set of information about it, such as networking information and ESN.

Latency 213

Latency Traffic Transportation Hardware 213

Percona

DECEMBER 13, 2023

Error logging is a critical aspect of database administration, providing insights into issues, warnings, and errors that may affect the system’s stability and performance. MySQL 8 introduces Error Log Filtering as a mechanism to fine-tune the error log, allowing administrators to focus on the most critical issues.

Database 98

Database Open Source Tuning Traffic 98

Scalegrid

JANUARY 25, 2024

Understanding Redis Performance Indicators Redis is designed to handle high traffic and low latency with its in-memory data store and efficient data structures. These essential data points heavily influence both stability and efficiency within the system. A reduced throughput could generally indicate that the server gets fewer queries.

Metrics 130

Metrics Monitoring Latency Cache 130

IO River

NOVEMBER 2, 2023

Enter the concept of Content Delivery Networks (CDN) with Multiple Origins Load Balancing. Think of a CDN Load Balancer (or LB, if you like to keep things short and sweet) as the internet’s traffic police. It takes on the crucial role of directing web traffic across various data centers, known as “origins.” What is “Origin”?An

Traffic 40

Traffic Cache Network Servers 40

Dynatrace

MAY 1, 2020

Our mission is to provide automatic answers, including root cause analysis, for performance degradation across all these systems and environments, regardless of the underlying hardware architecture. Network measurements with per-interface and per-process resolution. Network metrics are also collected for detected processes.

Java 133

Java Hardware Tuning Metrics 133

IO River

NOVEMBER 2, 2023

The OSI Model is like a layer cake of how data moves through networks. This means it specifically looks at the content of the data (like the writing inside our letters from the analogy) rather than just the envelope.So, what is the difference between a Firewall at the application level and network level?Network-level

Traffic 40

Traffic Network Logistics Architecture 40

SQL Performance

APRIL 15, 2020

This capability is essential when performance tuning since query events include discrete CPU and IO metrics as well as runtime parameters, which are key for troubleshooting query performance problems such as parameter sniffing. Network hit from the ADS Profiler extension (4-minute range). For some reason this thing will not go away!

Azure 96

Azure Database Tuning Network 96

Particular Software

APRIL 9, 2018

Based on our experience running business systems in production, we know we need to monitor our theme park to make sure it's working properly. We can use this data to extrapolate when we need to upgrade the electrical system, add a new water pipe, add more bays to our carpark, or commission more trucks to haul away our trash.

Systems 40

Systems Infrastructure Monitoring Tuning 40

Percona

APRIL 17, 2023

In this blog post, we will discuss the best practices on the MongoDB ecosystem applied at the Operating System (OS) and MongoDB levels. The main objective of this post is to share my experience over the past years tuning MongoDB and centralize the diverse sources that I crossed in this journey in a unique place. 25.84 - Total 21.04

Best Practices 52

Best Practices Design Tuning Database 52

Percona

OCTOBER 18, 2023

Parallel run : Operating both the existing and new database system simultaneously during the transition, allowing for a seamless fallback in case of any issues. Look closely at your current infrastructure (hardware, storage, networks, etc.) The logic and syntax of stored procedures may differ between database systems.

Open Source 96

Open Source Database Hardware Strategy 96

The Netflix TechBlog

FEBRUARY 18, 2022

To this end, we developed a Rapid Event Notification System (RENO) to support use cases that require server initiated communication with devices in a scalable and extensible manner. In this blog post, we will give an overview of the Rapid Event Notification System at Netflix and share some of the learnings we gained along the way.

Systems 334

Systems Traffic Architecture Mobile 334

Dynatrace

DECEMBER 8, 2020

This result s in w eb – service integrations perforat ing firewalls, network content not being captured by intrusion detection and vulnerability scanners miss ing what’s running in production. . Stay tuned – this is only the start. It ’s beyond human ability to keep track of all services and their interdependencies.

Cloud 292

Cloud Open Source Internet Internet 292

The Netflix TechBlog

MAY 26, 2020

Without having network visibility, it’s not possible to improve our reliability, security and capacity posture. Network Availability: The expected continued growth of our ecosystem makes it difficult to understand our network bottlenecks and potential limits we may be reaching. 43416 5001 52.213.180.42 43416 5001 52.213.180.42

Network 150

Network Tuning AWS Big Data 150

The Morning Paper

OCTOBER 8, 2019

Ours is a story of the elements we found useful in applying neural networks to a real life product. “ This made the moment ripe for trying sweeping changes to the system.” From this foundation, they took their first steps towards neural networks. I hope you enjoy it as much as I did! ” You need to be this tall.

Network 82

Network Architecture Tuning Traffic 82

Alex Russell

MARCH 31, 2022

Like every other successful platform, the web operates on a finely tuned understanding of risk and trust , and deprecations are particularly explosive. [2] De-duplication can prevent multiple copies on disk, but we're out of luck on reducing network costs. Shared caches must make sense within this value system. Trust Falls #.

Cache 82

Cache Government Traffic Network 82

The Netflix TechBlog

JUNE 14, 2023

The control group’s traffic utilized the legacy Falcor stack, while the experiment population leveraged the new GraphQL client and was directed to the GraphQL Shim. The AB experiment results hinted that GraphQL’s correctness was not up to par with the legacy system. The Replay Tester tool samples raw traffic streams from Mantis.

Traffic 353

Traffic Latency Cache Metrics 353

Dynatrace

NOVEMBER 20, 2019

Typically, these shops run the z/OS operating system, but more recently, it’s not uncommon to see the Z hardware running special versions of Linux distributions. Our goal is to provide automatic answers including root-cause analysis of performance degradation across all these systems and environments.

Availability 31

Availability Hardware Java Tuning 31

Apica

MAY 10, 2019

The spread of where consumers want to consume content, at home, at the office, at the gym, on a plane, at any time where there is connectivity, puts significant pressure on the network and performance monitoring teams for streaming brands. Apica’s scale is enterprise-grade.

Testing 40

Testing Performance Social Media Wireless 40

Rigor

SEPTEMBER 3, 2019

It’s about ensuring that your front-end is also working perfectly, that your site can deliver a delightful experience to your users or customers, and that it is functional – even when it’s experiencing up to seven or more times the typical traffic load. Traffic patterns outside of normal [RUM or Analytics].

Website 74

Website Cache Ecommerce Traffic 74

Dynatrace

AUGUST 8, 2019

Typically, these shops run the z/OS operating system, but more recently, it’s not uncommon to see the Z hardware running special versions of Linux distributions. Our goal is to provide automatic answers including root-cause analysis of performance degradation across all these systems and environments.

Availability 25

Availability Hardware Java Tuning 25

KeyCDN

SEPTEMBER 20, 2018

Not all back-end errors affect the user experience, but keeping track of them can prove helpful when tuning your app. Of course, there are also times when the problem lies not with your application but with the user’s connection or network configuration.

Strategy 67

Strategy Monitoring Website Google 67

The Netflix TechBlog

NOVEMBER 2, 2020

How viewers are able to watch their favorite show on Netflix while the infrastructure self-recovers from a system failure By Manuel Correa , Arthur Gonigberg , and Daniel West Getting stuck in traffic is one of the most frustrating experiences for drivers around the world. CRITICAL : This traffic affects the ability to play.

Traffic 252

Traffic Metrics Infrastructure Architecture 252

Brendan Gregg

SEPTEMBER 5, 2017

What follows are topics that may be of interest to anyone looking to migrate their systems and skillset: scan these to find topics that interest you. ## ZFS ZFS is available for Linux via the [zfsonlinux] and [OpenZFS] projects, and more recently was included in Canonical's Ubuntu Linux distribution: Ubuntu Xenial 16.04 LTS (April 2016).

Virtualization 76

Virtualization AWS Engineering Hardware 76

The Netflix TechBlog

APRIL 29, 2019

Open Connect Open Connect is Netflix’s content delivery network (CDN). video streaming) takes place in the Open Connect network. Various software systems are needed to design, build, and operate this CDN infrastructure, and a significant number of them are written in Python. are you logged in?

Open Source 279

Open Source Network Infrastructure Big Data 279

The Netflix TechBlog

JANUARY 10, 2024

This architecture shift greatly reduced the processing latency and increased system resiliency. By integrating with studio content systems, we enabled the pipeline to leverage rich metadata from the creative side and create more engaging member experiences like interactive storytelling.

Processing 229

Processing Media Latency Innovation 229

O'Reilly

SEPTEMBER 19, 2023

And that brings our story to the present day: Stage 3: Neural networks High-end video games required high-end video cards. You can download these models to use out of the box, or employ minimal compute resources to fine-tune them for your particular task. This technique simulates interactions in a complex system.

Hardware 99

Hardware Storage Big Data Blockchain 99

Smashing Magazine

SEPTEMBER 6, 2021

Next, we’ll look at how to set up servers and clients (that’s the hard part unless you’re using a content delivery network (CDN)). Using just a few (but still more than one), however, could nicely balance congestion growth with better performance, especially on high-speed networks. What Does It All Mean?

Network 104

Network Servers Cache Traffic 104

All Things Distributed

OCTOBER 2, 2017

With these requirements in mind, and a willingness to question the status quo, a small group of distributed systems experts came together and designed a horizontally scalable distributed database that would scale out for both reads and writes to meet the long-term needs of our business. This was the genesis of the Amazon Dynamo database.

Internet 128

Internet Internet AWS Performance 128

Smashing Magazine

AUGUST 22, 2021

As we will see, QUIC and HTTP/3 indeed have great web performance potential, but mainly for users on slow networks. If your average visitor is on a fast cabled or cellular network, they probably won’t benefit from the new protocols all that much. An often used metaphor is that of a pipe used to transport water. Congestion Control.

Performance 105

Performance Network Latency Servers 105