Brendan Gregg

APRIL 14, 2022

offer letter logo (2014) flame graphs (2014) eBPF tools (2014-2019) PMC analysis (2017) my pandemic-abandoned desk (2020); office wall I joined Netflix in 2014, a company at the forefront of cloud computing with an attractive work culture. It was the most challenging job among those I interviewed for. Thank you.

Java 52

Java Cache Cloud AWS 52

Brendan Gregg

JULY 2, 2021

There's an arms race to add [eBPF] (BPF) to commercial observability products, and in this post I'll describe how to quickly do that. People like to show me their BPF observability products after they have prototyped or built them, but I often wish I had given them advice before they started. You might not even like BPF.

Open Source 40

Open Source Latency Cache Energy 40

Brendan Gregg

OCTOBER 15, 2019

At a Linux conference in Düsseldorf, 2014, I saw a talk by kernel maintainer [Steven Rostedt] that answered both questions. I'd also recommend watching: - BPF at Facebook by Alexei Starovoitov ( youtube ) - The ubiquity but also the necessity of eBPF as a technology to keep the kernel relevant by David S.

C++ 77

C++ Engineering Technology Technology 77

Brendan Gregg

FEBRUARY 28, 2023

Titus, the Netflix container management platform, is now open source,” [link] , Apr 2018 [Cutress 19] Dr. DDR6: Here's What to Expect in RAM Modules,” [link] , Nov 2020 [Salter 20] Jim Salter, “Western Digital releases new 18TB, 20TB EAMR drives,” [link] , Jul 2020 [Spier 20] Martin Spier, Brendan Gregg, et al.,

Performance 52

Performance Latency Cache Virtualization 52

Brendan Gregg

OCTOBER 14, 2019

At a Linux conference in Düsseldorf, 2014, I saw a talk by kernel maintainer [Steven Rostedt] that answered both questions. I'd also recommend watching: - BPF at Facebook by Alexei Starovoitov ( youtube ) - The ubiquity but also the necessity of eBPF as a technology to keep the kernel relevant by David S.

C++ 52

C++ Engineering Technology Technology 52

Brendan Gregg

MARCH 16, 2024

" I pictured how this is used to walk stack traces in my BPF book. Stack Frame with Base Pointer ( x86-64 ABI ) Figure 2-6: Frame Pointer-based Stack Walking ( BPF book ) This stack-walking technique is commonly used by external profilers and debuggers, including Linux perf and eBPF, and ultimately visualized by flame graphs.

Java 145

Java Cache Google Hardware 145

The Netflix TechBlog

FEBRUARY 20, 2019

Introducing BCC / eBPF visualisations The Netflix performance engineering team (especially my colleague Brendan) has been contributing to eBPF since its beginning in 2014, including developing many open source performance tools for BCC such as execsnoop, biosnoop, biotop, ext4slower, tcplife, runqlat, and more.

Performance 211

Performance Latency Open Source Metrics 211

Brendan Gregg

MAY 16, 2017

When I joined Netflix in April 2014, we had over 40 million subscribers in 41 countries. In the last three years I developed the ftrace-based [perf-tools] and used them to solve many problems, which I wrote about in [lwn.net] and spoke about at [LISA 2014]. We are now in 190 countries and just crossed [100 million subscribers]!

Java 75

Java Entertainment Engineering Scalability 75

Brendan Gregg

SEPTEMBER 5, 2017

Here's some output from my zfsdist tool, in bcc/BPF, which measures ZFS latency as a histogram on Linux: # zfsdist. The final piece was enhanced BPF (aka eBPF: enhanced Berkeley Packet Filter), which provided the custom in-kernel programmability needed for an advanced tracer, created by Alexei Starovoitov (now at Facebook).

Virtualization 76

Virtualization AWS Engineering Hardware 76

Brendan Gregg

AUGUST 8, 2017

And here's an excerpt from [Linux] today (include/linux/sched/loadavg.h): #define EXP_1 1884 /* 1/exp(5sec/1min) as fixed-point */ #define EXP_5 2014 /* 1/exp(5sec/5min) */ #define EXP_15 2037 /* 1/exp(5sec/15min) */. EXPFF: EXP 0.920043902 ;C = 1 MIN EXP 0.983471344 ;C = 5 MIN EXP 0.994459811 ;C = 15 MIN. 3 tar termb$ iostat -x 60. [.]

Latency 111

Latency Metrics C++ Systems 111