The Netflix TechBlog

DECEMBER 10, 2019

By Adam Wang , Andy Swan , Raja Senapati , Shilpa Jois , Anjali Chablani , Deepa Krishnan , Vidya Sundaram , and Casey Wilms You can also check out highlights from our past events: May 2019 , November 2018 , March 2018 , August 2017 , January 2017 , May 2016 , November 2015 , March 2015 , February 2014 & August 2014.

Innovation 200

Innovation Mobile Technology Technology 200

Brendan Gregg

NOVEMBER 28, 2017

In this configuration, the AMI and boot is paravirt (PV), the kernel is making hypercalls instead of privileged instructions, and the system is using paravirt network and storage drivers. AWS EC2 uses a different image type and boot process for PV and HVM, as described on the [Linux AMI Virtualization Types] page.

Virtualization 40

Virtualization AWS Hardware Storage 40

Dynatrace

MAY 6, 2021

Operating systems support. Future Dynatrace Managed operating systems support changes. The following operating systems will no longer be supported starting 01 September 2021. Linux: Amazon Linux AMI 2017.x. The following operating systems will no longer be supported starting 01 October 2021. x86 (64bit-only).

Operating System 234

Operating System AWS Metrics Storage 234

The Netflix TechBlog

OCTOBER 28, 2021

During earlier years of my career, I primarily worked as a backend software engineer, designing and building the backend systems that enable big data analytics. I also built online serving systems and microservices powering Walmart’s e-commerce. At this conference, I also got to know my current team: Consolidated Logging (CL).

Data Engineering 173

Data Engineering Engineering Big Data Software Engineering 173

Brendan Gregg

OCTOBER 28, 2017

For my first trip to Paris I gave the closing keynote at [EuroBSDcon 2017] on performance methodologies, using FreeBSD 11.1 In the past I've shared similar methodologies applied to other operating systems, and finished porting them to BSD for this talk. EuroBSDcon 2017]: [link] [thread state analysis]: [link] [tstates.d]: [link]

Systems 52

Systems Performance C++ Operating System 52

Brendan Gregg

SEPTEMBER 5, 2017

What follows are topics that may be of interest to anyone looking to migrate their systems and skillset: scan these to find topics that interest you. ## ZFS ZFS is available for Linux via the [zfsonlinux] and [OpenZFS] projects, and more recently was included in Canonical's Ubuntu Linux distribution: Ubuntu Xenial 16.04 LTS (April 2016).

Virtualization 76

Virtualization AWS Engineering Hardware 76

Dynatrace

MARCH 2, 2023

Focusing on the fundamentals Grabner recalled a 2017 tweet from Hightower that outlined how his technology career gathered steam once he started to “focus on the fundamentals.” The whole goal is, we want to reduce the amount that a human needs to give to the system in order to make it do what you want it to do,” he said.

Best Practices 221

Best Practices Internet Internet Cloud 221

The Morning Paper

JUNE 6, 2019

Towards federated learning at scale: system design Bonawitz et al., This is a high level paper describing Google’s production system for federated learning. The FL system contains a number of privacy-enhancing building blocks, but the privacy guarantees of any end-to-end system will always depend on how they are used.

Systems 71

Systems Design Google Servers 71

The Netflix TechBlog

DECEMBER 14, 2020

The mystery begins Towards the end of 2017, I was on a conference call to discuss an issue with the Netflix application on a new set top box. They supplied a script to automate the process. The role of a Partner Engineer at Netflix is to help device manufacturers launch the Netflix application on their devices. aka “Lollipop”.

Engineering 352

Engineering Code Hardware Open Source 352

The Netflix TechBlog

DECEMBER 10, 2019

By Adam Wang , Andy Swan , Raja Senapati , Shilpa Jois , Anjali Chablani , Deepa Krishnan , Vidya Sundaram , and Casey Wilms You can also check out highlights from our past events: May 2019 , November 2018 , March 2018 , August 2017 , January 2017 , May 2016 , November 2015 , March 2015 , February 2014 & August 2014.

Innovation 130

Innovation Mobile Technology Technology 130

The Netflix TechBlog

DECEMBER 10, 2019

By Adam Wang , Andy Swan , Raja Senapati , Shilpa Jois , Anjali Chablani , Deepa Krishnan , Vidya Sundaram , and Casey Wilms You can also check out highlights from our past events: May 2019 , November 2018 , March 2018 , August 2017 , January 2017 , May 2016 , November 2015 , March 2015 , February 2014 & August 2014.

Innovation 130

Innovation Mobile Technology Technology 130

The Netflix TechBlog

OCTOBER 19, 2020

which is difficult when troubleshooting distributed systems. Reconstructing a streaming session was a tedious and time consuming process that involved tracing all interactions (requests) between the Netflix app, our Content Delivery Network (CDN), and backend microservices. Stream Processing: to sample or not to sample trace data?

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

Scalegrid

JULY 13, 2020

Oracle Database is a commercial, proprietary multi-model database management system produced by Oracle Corporation, and the largest relational database management system (RDBMS) in the world. Compare ease of use across compatibility, extensions, tuning, operating systems, languages and support providers. Comparison Overview.

Open Source 244

Open Source Tuning C++ Database 244

ScaleOut Software

MAY 4, 2018

Traditional stream-processing and complex event processing systems, such as Apache Storm and Software AG’s Apama , have focused on extracting interesting patterns from incoming data with stateless applications. Michael Grieves (U.

Processing 40

Processing IoT Programming Software Engineering 40

ScaleOut Software

MAY 4, 2018

Traditional stream-processing and complex event processing systems, such as Apache Storm and Software AG’s Apama , have focused on extracting interesting patterns from incoming data with stateless applications. Michael Grieves (U.

Processing 40

Processing IoT Programming Software Engineering 40

Brendan Gregg

FEBRUARY 28, 2023

This talk originated from my updates to Systems Performance 2nd Edition , and this was the first time I've given this talk in person! CXL in a way allows a custom memory controller to be added to a system, to increase memory capacity, bandwidth, and overall performance. Ford, et al., “TCP

Performance 52

Performance Latency Cache Virtualization 52

Dynatrace

JUNE 15, 2023

Cloud application security is a combination of policies, processes, and controls that aim to reduce the risk of exposing cloud-based applications to compromise or failure from external or internal threats. This process can delay deployments or cause developers to skip security testing completely to meet project deadlines.

Cloud 181

Cloud Open Source Architecture Monitoring 181

Dynatrace

DECEMBER 16, 2019

Commonly applied to development processes, technical debt accrues overtime when we choose an inefficient path of least resistance. Even small amounts of technical debt compound as new code branches from old, further embedding the shortcomings into the system. Technical Debt—Dilbert Comic Strip on 2017-01-03.

Artificial Intelligence 151

Artificial Intelligence Traffic Efficiency Innovation 151

ACM Sigarch

APRIL 26, 2018

APU: Accelerated Processing Unit is the AMD’s Fusion architecture that integrates both CPU and GPU on the same die. BPU: Brain Processing Unit is the design of the AI chips by Horizon Robotics. They unveiled their first two embedded AI chips fabricated with TSMC 40nm process in December 2017: “Journey 1.0

Hardware 99

Hardware Artificial Intelligence Architecture Google 99

All Things Distributed

MAY 19, 2017

We decided to use our inventive powers to design and build a new database engine that would give database systems such as MySQL and PostgreSQL reliability and performance at scale, at a level that could serve even the most demanding OLTP applications.

Database 120

Database Design Cloud Storage 120

IO River

NOVEMBER 2, 2023

However, consumers often prioritize availability in many systems. Furthermore, there are many recognized standards to measure the availability of a service or system, and the most common one is to measure it as a percentage."Five minutes of downtime per year, which means the system is almost always operational.

Availability 52

Availability Social Media Traffic Games 52

IO River

NOVEMBER 2, 2023

However, consumers often prioritize availability in many systems. Furthermore, there are many recognized standards to measure the availability of a service or system, and the most common one is to measure it as a percentage."Five minutes of downtime per year, which means the system is almost always operational.

Availability 52

Availability Social Media Traffic Games 52

Brendan Gregg

JULY 14, 2019

BPF (eBPF) tracing is a superpower that can analyze everything, and I'll show you how in my upcoming book BPF Performance Tools: Linux System and Application Observability , coming soon from Addison Wesley. A time where you can pose arbitrary questions of the system, and it can answer them. BPF is in the Linux kernel. ## Why Your Book?

Systems 40

Systems Performance Education Technology 40

Percona

AUGUST 17, 2023

StackOverflow statistics show that 26% of developers preferred it in 2017, 34% in 2019, and 40% in 2021. In another survey, using multiple metrics for overall popularity, PostgreSQL ranks as the fourth most popular database management system ( DB-Engines, March 2023 ).

Open Source 76

Open Source Healthcare Social Media Database 76

Alex Russell

MARCH 6, 2021

TL;DR: A lot has changed since 2017 when we last estimated a global baseline resource per-page resource budget of 130-170KiB. To update our global baseline from 2017, we want to update our priors on a few dimensions: The evolved device landscape. Advances in browser content processing. The Moto G4 , for example. Hard Reset.

Performance 124

Performance Network Cache Metrics 124

Dynatrace

FEBRUARY 25, 2021

since 2017, and many customers have used it with great success while we collected requirements for the next iteration of our Lambda extension. In theory, an existing code module or agent can be used to monitor a Lambda function if there’s a way to load it into the running Lambda process. Why metrics alone aren’t enough.

Lambda 224

Lambda AWS Monitoring Serverless 224

All Things Distributed

SEPTEMBER 29, 2016

Voice makes access to digital services far more inclusive than traditional screen-based interaction, for example, an aging population may be much more comfortable interacting with voice-based systems than through tablets or keyboards. However, it is still Day One, and a lot of innovation is underway in this world.

Artificial Intelligence 139

Artificial Intelligence Healthcare AWS Government 139

Brendan Gregg

SEPTEMBER 25, 2021

CLI tools The Cassandra systems were EC2 virtual machine (Xen) instances. What about short-lived processes, like a service restarting in a loop? Microbenchmark os::javaTimeMillis() on both systems. Running this on the two systems saw similar results. cat /sys/devices/system/clocksource/clocksource0/current_clocksource.

Speed 126

Speed Java AWS Virtualization 126

Brendan Gregg

APRIL 27, 2023

Keeping overhead low can require tradeoffs in other areas: tcpdump(8), for example, will drop packets if the system is overloaded, resulting in incomplete visibility. Long ago I encountered systems which met strict security auditing requirements with the following behavior: If the kernel could not log an event, it would immediately halt !

Open Source 52

Open Source Monitoring Engineering Design 52

Randon ASCII

DECEMBER 3, 2018

Subtitle: Making Windows Slower Part 3: Process Destruction. In the summer of 2017 I wrestled with a Windows performance problem. Process destruction was slow, serialized, and was blocking the system input queue, leading to repeated short mouse-movement hangs when building Chrome. The problem, circa 2017.

Programming 71

Programming Processing Testing Code 71

Adrian Cockcroft

JANUARY 29, 2023

There is no way to model how much more traffic you can send to that system before it exceeds it’s SLA. This is unfortunate, because we’d really like to be able to build systems that have an SLA that we can share with the consumers of our interfaces, and be able to measure how well we are doing.

Lambda 98

Lambda Latency Cache C++ 98

SQL Server According to Bob

DECEMBER 18, 2018

Durability: “In database systems , durability is the ACID property which guarantees transactions that have committed will survive permanently. For example, if a flight booking reports that a seat has successfully been booked, then the seat will remain booked even if the system crashes.” – [link]. Starting with the Linux 4.18

Servers 90

Servers Media Cache Storage 90

SQL Server According to Bob

MAY 19, 2019

The reduction effort started when we shipped SQL Server 2017 ( [link].) Partitioning allows SQL Server to scale to the largest systems with record-setting performance. When promoted to a super-latch on a 64 CPU system the memory requirement becomes 32 + (32 *64) = 2080 bytes. However, partitioning can require more memory.

IoT 61

IoT Servers Development Cache 61

Dynatrace

OCTOBER 17, 2019

And with today’s increasing financial, availability, performance and innovation requirements meaning applications need to be geographically dispersed to constantly changing dynamic powerhouses, it has become simply not possible to provision, update, monitor and decommissions them by only leveraging manual processes.

Code 110

Code Infrastructure DevOps Serverless 110

SQL Performance

JULY 8, 2020

Unnesting/substitution of table expressions is a process of taking a query that involves nesting of table expressions, and as if substituting it with a query where the nested logic is eliminated. What actually happens is that the query parsing process produces an initial tree of logical operators closely reflecting the original query.

Servers 110

Servers Code Processing Performance 110

SQL Server According to Bob

JULY 17, 2018

When starting SQL Server on Linux why are there two (2) sqlservr processes? The simple answer is the first entry (85829) is not what you are used to on a Windows system as sqlservr.exe and does not listen for TDS traffic or open database files. The parent process handles basic configuration activities and then forks the child process.

Servers 40

Servers Processing Software Engineering Traffic 40

Scalegrid

APRIL 2, 2019

PostgreSQL is an open source relational database system that has soared in popularity over the past 30 years from its active, loyal, and growing community. were in the process of migrating to PostgreSQL and the last 14.1% different database systems, including PostgreSQL itself. are using public clouds.

Database 185

Database Cloud Open Source Azure 185