The Netflix TechBlog

DECEMBER 14, 2020

The mystery begins Towards the end of 2017, I was on a conference call to discuss an issue with the Netflix application on a new set top box. Figure 2: Visualizing Audio Throughput and Thread Handler Timing The orange line is the rate that data moved from the streaming buffer into the Android audio system, in bytes/millisecond.

Engineering 352

Engineering Code Hardware Open Source 352

Brendan Gregg

JULY 4, 2021

I summarized these topics and more as a plenary conference talk, including my own predictions (as a senior performance engineer) for the future of computing performance, with a focus on back-end servers. This was a chance to talk about other things I've been working on, such as the present and future of hardware performance.

Performance 52

Performance Latency Hardware Storage 52

Brendan Gregg

FEBRUARY 28, 2023

This talk originated from my updates to Systems Performance 2nd Edition , and this was the first time I've given this talk in person! CXL in a way allows a custom memory controller to be added to a system, to increase memory capacity, bandwidth, and overall performance. Or even on a plane. It was a great privilege.

Performance 52

Performance Latency Cache Virtualization 52

O'Reilly

SEPTEMBER 15, 2021

While it’s sadly premature to say that the survey took place at the end of the COVID-19 pandemic (though we can all hope), it took place at a time when restrictions were loosening: we were starting to go out in public, have parties, and in some cases even attend in-person conferences. We’re not sure what that means.

Azure 145

Azure Social Media Programming AWS 145

Percona

DECEMBER 27, 2023

In our conversations with developers and engineers during the conference, we confirmed the CNCF GItOps Microsurvey data – 91% are already using GitOps. Creating repositories for memory-safe patterns in cloud systems to improve overall security.

Cloud 80

Cloud Open Source Strategy Best Practices 80

ACM Sigarch

MAY 31, 2023

Introduction Memory systems are evolving into heterogeneous and composable architectures. Heterogeneous and Composable Memory (HCM) offers a feasible solution for terabyte- or petabyte-scale systems, addressing the performance and efficiency demands of emerging big-data applications.

Latency 52

Latency Hardware Cache Architecture 52

The Morning Paper

DECEMBER 8, 2019

At the start of November I was privileged to attend HPTS (the High Performance Transaction Systems) conference in Asilomar. On the last morning of the conference Daniel Bittman presented some of the work being done in the context of the Twizzler OS project to explore new programming models for NVM. What about security?

Hardware 56

Hardware Programming Media Storage 56

The Morning Paper

MAY 27, 2019

Last week we saw the benefits of rethinking memory and pointer models at the hardware level when it came to object storage and compression ( Zippads ). The protections are hardware implemented and cannot be forged in software. And this all has to work for whole-system executions, not just the C-language portion of user processes.

C++ 61

C++ Hardware Virtualization Benchmarking 61

Alex Podelko

OCTOBER 20, 2016

I am looking forward to share my thoughts on ‘Reinventing Performance Testing’ at the imPACt performance and capacity conference by CMG held on November 7-10, 2016 in La Jolla, CA. Cloud seriously impacts system architectures that has a lot of performance-related consequences. First, we have a shift to centrally managed systems.

Performance Testing 100

Performance Testing Architecture Testing Performance 100

Sutter's Mill

FEBRUARY 13, 2017

” This contains updated and new material that reflects the latest C++ standards and compilers, with a focus to using modern C++11/14/17 effectively on modern hardware and memory architectures. On April 25-27, I’ll be in Stockholm (Kista) giving a three-day seminar on “High-Performance and Low-Latency C++.”

Latency 51

Latency C++ Hardware Performance 51

VoltDB

JUNE 1, 2017

Earlier this month I spent a week at the OpenStack conference. But once you realize that there are, in fact, several OpenStack conferences happening at the same time and place, it makes more sense. Hardware Optimizers” want to get the maximum utilization out of hardware.

Hardware 56

Hardware Latency Virtualization Cloud 56

Adrian Cockcroft

JUNE 2, 2023

Eric asked me to give one of the keynotes at the end of the first day, and invited local Denver people to join the conference for an AI focused meetup that included the final three keynotes, the drinks and snacks reception and some additional talks in the evening. There are already a few hundred AI authored books that have been published.

Social Media 98

Social Media Innovation Open Source Entertainment 98

Percona

NOVEMBER 1, 2023

MySQL backups play a pivotal role in safeguarding the integrity of your data, providing defense against various unforeseen calamities, hardware malfunctions, data loss, corruption, and inadvertent deletions. The key question related to RTO is, “How quickly must the data on this system be restored?” ” Great advice.

Best Practices 79

Best Practices Storage Strategy Database 79

Smashing Magazine

OCTOBER 11, 2022

In his recent Smashing Conference talk, green software expert Asim Hussain mentioned that the biggest shift he is seeing is as a result of regulation — or the threat of regulation. Another great bit of advice from Asim’s talk from Smashing Conference was on branding. On design systems, UX, web performance and CSS/JS.

Strategy 124

Strategy Development Energy Website 124

Brendan Gregg

SEPTEMBER 25, 2021

CLI tools The Cassandra systems were EC2 virtual machine (Xen) instances. As a Xen guest, this profile was gathered using perf(1) and the kernel's software cpu-clock soft interrupts, not the hardware NMI. Microbenchmark os::javaTimeMillis() on both systems. Running this on the two systems saw similar results.

Speed 126

Speed Java AWS Virtualization 126

VoltDB

JUNE 1, 2017

Earlier this month I spent a week at the OpenStack conference. But once you realize that there are, in fact, several OpenStack conferences happening at the same time and place, it makes more sense. Hardware Optimizers” want to get the maximum utilization out of hardware.

Hardware 40

Hardware Latency Virtualization Cloud 40

All Things Distributed

SEPTEMBER 20, 2013

Not everybody agreed that the "N-ary Storage Model" (NSM) was the best approach for all workloads but it stayed dominant until hardware constraints, especially on caches, forced the community to revisit some of the alternatives. Khoshafian, in the Proceedings of the 1985 SIGMOD International Conference on Management of Data.'

Storage 87

Storage Hardware Cache C++ 87

The Morning Paper

SEPTEMBER 19, 2019

There were so many interesting papers at the conference this year though that I haven’t been able to cover nearly as many as I would like. ” Crusher is a Google system for automatically discovering email templates (e.g. Could it be Analyzing efficient stream processing on modern hardware ? What’s their secret???

Blockchain 46

Blockchain Hardware Google Analytics 46

All Things Distributed

AUGUST 24, 2012

Werner Vogels weblog on building scalable and robust distributed systems. The 5 Minute Rule for Trading Memory for Disk Accesses and The 10 Byte Rule for Trading Memory for CPU Time , Jim Gray and Gianfranco Putzolu, Proceedings of the ACM SIGMOD Conference, pp. All Things Distributed. By Werner Vogels on 24 August 2012 04:00 PM.

Storage 108

Storage Hardware AWS Data Engineering 108

Adrian Cockcroft

JUNE 21, 2022

Photo by Adrian I spent six years at Cambridge Consultants, building some interesting systems, managing our Sun workstations and learning a lot, but by then Sun had opened a sales office across the street, and I wanted to find out what they were going to release next, before everyone else. as a result there were product and messaging changes.

Tuning 52

Tuning Benchmarking Engineering C++ 52

Brendan Gregg

AUGUST 18, 2019

BPF has been extended (aka eBPF) in recent years for providing a safe way to extend kernel functionality, and has become a hot topic in systems engineering, with at least 24 talks on BPF at the last Linux Plumber's conference. hardware Hardware counter-based instrumentation. system(".") was just released.

Latency 68

Latency C++ Cache Programming 68

ACM Sigarch

NOVEMBER 4, 2019

Krste Asanovic from UC Berkeley kicked off the main program sharing his experience on “ Rejuvenating Computer Architecture Research with Open-Source Hardware ”. He ended the keynote with a call to action for open hardware and tools to start the next wave of computing innovation. Conference.

Hardware 18

Hardware Architecture Programming Innovation 18

ACM Sigarch

NOVEMBER 1, 2019

This blog post gives a glimpse of the computer systems research papers presented at the USENIX Annual Technical Conference (ATC) 2019, with an emphasis on systems that use new hardware architectures. USENIX ATC is a top-tier venue with a broad range of systems research papers from both industry and academia.

Architecture 12

Architecture Hardware Cache Storage 12

O'Reilly

JANUARY 13, 2020

Binder, she notes, “was widely used last year for teaching workshops and tutorials at many Python conferences.” ” Java. .” ” As for Swift for TensorFlow, Paris Buttfield-Addison calls it “a radical new use for Swift.”

Programming 130

Programming Java Google C++ 130

Brendan Gregg

SEPTEMBER 25, 2021

CLI tools The Cassandra systems were EC2 virtual machine (Xen) instances. As a Xen guest, this profile was gathered using perf(1) and the kernel's software cpu-clock soft interrupts, not the hardware NMI. Microbenchmark os::javaTimeMillis() on both systems. Running this on the two systems saw similar results.

Speed 52

Speed Java AWS Virtualization 52

PDC

NOVEMBER 9, 2018

In such scenarios, scalability or scaling is widely used to indicate the ability of hardware and software to deliver greater computational power when the amount of resources is increased. AFIPS Conference Proceedings. (30): References. Amdahl, Gene M. 30): 483–485. doi: 10.1145/1465482.1465560. Gustafson, John L. 31 (5): 532–533.

Scalability 65

Scalability Programming Government Hardware 65

John McCalpin

OCTOBER 16, 2016

“Memory Bandwidth and System Balance in HPC Systems” If you are planning to attend the SuperComputing 2016 conference in Salt Lake City next month, be sure to reserve a spot on your calendar for my talk on Wednesday afternoon (4:15pm-5:00pm). McCalpin Presents “Memory Bandwidth and System Balance in HPC Systems”.

Architecture 40

Architecture Systems Technology Technology 40

Adrian Cockcroft

NOVEMBER 11, 2019

A resilient system continues to operate successfully in the presence of failures. The system needs to maintain a safety margin that is capable of absorbing failure via defense in depth, and failure modes need to be prioritized to take care of the most likely and highest impact risks. The first technique is the most generally useful.

Latency 52

Latency Engineering Systems Hardware 52

Nick Desaulniers

APRIL 6, 2020

” Distracted in a conference grand ballroom, above what might be the largest mall in the world or at least Bangkok, a blank QEMU session has me seriously questioning my life choices. halt (); Some sort of very early exception handler; better to sit busy in an infinite loop than run off and destroy hardware or corrupt data, I suppose.

C++ 67

C++ Hardware Code Virtualization 67

Adrian Cockcroft

NOVEMBER 11, 2019

A resilient system continues to operate successfully in the presence of failures. The system needs to maintain a safety margin that is capable of absorbing failure via defense in depth, and failure modes need to be prioritized to take care of the most likely and highest impact risks. The first technique is the most generally useful.

Latency 53

Latency Engineering Systems Hardware 53

SQL Performance

NOVEMBER 1, 2019

This way your memory limit is based on how much you spent on licensing, not some arbitrary limit decided in a conference room somewhere 5 years ago. I also feel like Enterprise customers tend to already have more and better hardware to throw at the problem than smaller shops can afford. Parting Thoughts.

Servers 40

Servers Hardware Database Availability 40

Brendan Gregg

SEPTEMBER 25, 2021

CLI tools The Cassandra systems were EC2 virtual machine (Xen) instances. As a Xen guest, this profile was gathered using perf(1) and the kernel's software cpu-clock soft interrupts, not the hardware NMI. Microbenchmark os::javaTimeMillis() on both systems. Running this on the two systems saw similar results.

Speed 40

Speed Java AWS Virtualization 40

Smashing Magazine

JANUARY 15, 2019

For the last three years, the SustainableUX conference has seen experts in web sustainability sharing their knowledge across an array of web-based disciplines. how much data does the browser have to download to display your website) and resource usage of the hardware serving and receiving the website. These include data transfer (i.e.

Website Performance 73

Website Performance Website Energy Performance 73

Brendan Gregg

SEPTEMBER 5, 2017

What follows are topics that may be of interest to anyone looking to migrate their systems and skillset: scan these to find topics that interest you. ## ZFS ZFS is available for Linux via the [zfsonlinux] and [OpenZFS] projects, and more recently was included in Canonical's Ubuntu Linux distribution: Ubuntu Xenial 16.04 LTS (April 2016).

Virtualization 76

Virtualization AWS Engineering Hardware 76

John McCalpin

JANUARY 22, 2018

Dr. Damon McDougall gave a short presentation on this study at the IXPUG 2018 Fall Conference ( pdf ) — I originally wrote these notes to help organize my thoughts as we were preparing the IXPUG presentation, and later decided that the extra details contained here are interesting enough for me to post it. cmpl $1000000000, %eax.

Latency 40

Latency Hardware Code Testing 40

Smashing Magazine

AUGUST 22, 2021

The reasons why are quite complex, and I’ve written several academic papers on the topic, as well as talked about it in a conference. Passing this data is done using kernel APIs (system calls), which involves a certain amount of overhead per API call. We can also expect QUIC-specific hardware to become available.

Performance 106

Performance Network Latency Servers 106