Brendan Gregg

JULY 2, 2021

There's an arms race to add [eBPF] (BPF) to commercial observability products, and in this post I'll describe how to quickly do that. People like to show me their BPF observability products after they have prototyped or built them, but I often wish I had given them advice before they started. You might not even like BPF.

Open Source 40

Open Source Latency Cache Energy 40

Brendan Gregg

MAY 16, 2017

I previously wrote about Netflix in [2015] and [2016], and if you are interested in what it's like to work here, I already covered much in those posts. I also worked with Alexei Starovoitov (now at Facebook) on enhanced BPF for tracing, and developed many [bcc tools] that use BPF. I've now worked at Netflix for over three years.

Java 75

Java Entertainment Engineering Scalability 75

Brendan Gregg

FEBRUARY 28, 2023

Titus, the Netflix container management platform, is now open source,” [link] , Apr 2018 [Cutress 19] Dr. DDR6: Here's What to Expect in RAM Modules,” [link] , Nov 2020 [Salter 20] Jim Salter, “Western Digital releases new 18TB, 20TB EAMR drives,” [link] , Jul 2020 [Spier 20] Martin Spier, Brendan Gregg, et al.,

Performance 52

Performance Latency Cache Virtualization 52

The Netflix TechBlog

NOVEMBER 22, 2019

December 3 11:30am-12:30pm NFX 208 Netflix’s container journey to bare metal Amazon EC2 Andrew Spyker , Compute Platform Engineering Manager Abstract : In 2015, Netflix started supporting containers as part of their compute platform. The talk also includes examples of using these tools in the Amazon Elastic Compute Cloud (Amazon EC2) cloud.

AWS 37

AWS Entertainment Open Source Benchmarking 37

Brendan Gregg

AUGUST 26, 2021

A quick look at the killsnoop(8) source reminded me that I wrote it back in 2015, which is practically ancient in eBPF years. bpftrace tool from my BPF book traces the signal:signal_generate tracepoint, which should catch every type of generated signal, including tgkill(2). A crash happened, but killsnoop(8) didn't see it. signals.bt

Storage 145

Storage C++ Processing Database 145

Brendan Gregg

MAY 28, 2021

Back in 2015 we'd have BPF (iovisor) meetups in Santa Clara and most contributors would be there in person, with some having travelled. But I'm now excited about my new adventure: Doing an advanced tech role remotely from Australia. I know others who have also left the Bay Area or are planning to.

Google 40

Google Engineering Open Source AWS 40

Brendan Gregg

MAY 8, 2021

Update: Some modern drives after 2015 are sealed with [helium].) biolatency (this is my BPF tool from [bcc]), printing 60-second histograms, per disk (-D): # biolatency -D 60 1 Tracing block device I/O. Such hard drives have air ports, and air filters, to equalize pressure with the outside air. Hit Ctrl-C to end. Hit Ctrl-C to end.

Performance 40

Performance Latency Speed Systems 40

Brendan Gregg

AUGUST 26, 2021

A quick look at the killsnoop(8) source reminded me that I wrote it back in 2015, which is practically ancient in eBPF years. bpftrace tool from my BPF book traces the signal:signal_generate tracepoint, which should catch every type of generated signal, including tgkill(2). A crash happened, but killsnoop(8) didn't see it. signals.bt

Storage 40

Storage C++ Processing Database 40

Brendan Gregg

AUGUST 8, 2017

rc5-virtual (bgregg-xenial-bpf-i-0b7296777a2585be1) 08/01/2017 _x86_64_ (8 CPU) 10:15:51 PM UID PID %usr %system %guest %CPU CPU Command 10:16:51 PM 0 18468 2.85 It spends several minutes mostly blocked on disk reads. Here are the stats, collected from three different terminal windows: terma$ pidstat -p `pgrep -x tar` 60. termc$ uptime.

Latency 111

Latency Metrics C++ Systems 111