Brendan Gregg

JULY 4, 2021

AWS Graviton2); for memory with the arrival of DDR5 and High Bandwidth Memory (HBM) on-processor; for storage including new uses for 3D Xpoint as a 3D NAND accelerator; for networking with the rise of QUIC and eXpress Data Path (XDP); and so on. I also wrote about these topics in detail for my recent [Systems Performance 2nd Edition] book.

Performance 140

Performance Latency Hardware Storage 140

The Netflix TechBlog

NOVEMBER 29, 2023

HDR was launched at Netflix in 2016 and the number of titles available in HDR has been growing ever since. Separately, our invention of Dynamically Optimized ( DO ) encoding helps achieve optimized bitrate-quality tradeoffs depending on the complexity of the content.

Open Source 252

Open Source Software Engineering Internet Internet 252

Brendan Gregg

FEBRUARY 28, 2023

This talk originated from my updates to [Systems Performance 2nd Edition], and this was the first time I've given this talk in person! CXL in a way allows a custom memory controller to be added to a system, to increase memory capacity, bandwidth, and overall performance. Ford, et al., “TCP

Performance 110

Performance Latency Cache Virtualization 110

High Scalability

AUGUST 13, 2018

Decentralized systems will continue to lose to centralized systems until there's a driver requiring decentralization to deliver a clearly superior consumer experience. For all the idealistic technical reasons I laid out long ago in Building Super Scalable Systems: Blade Runner Meets Autonomic Computing In The Ambient Cloud.

Internet 116

Internet Internet Google Scalability 116

Dynatrace

AUGUST 10, 2019

Having the ability to monitor the performance and availability of your organization’s internal applications—in addition to your organization’s customer-facing applications—from within your corporate network is an important benefit of synthetic monitoring. Windows 2016 Server (or a supported Linux distribution ).

Monitoring 104

Monitoring Hardware Tuning Mobile 104

The Netflix TechBlog

MAY 1, 2019

surround audio in 2010, and began streaming Dolby Atmos in 2016 , but wanted to bring studio quality sound to our members around the world. We want your experience to be brilliant even if you aren’t listening with a state-of-the-art home theater system. We began streaming 5.1 Until now, we’ve only used adaptive streaming for video.

Engineering 274

Engineering Network Media Entertainment 274

Alex Russell

MARCH 6, 2021

Thanks to progress in networks and browsers (but not devices), a more generous global budget cap has emerged for sites constructed the "modern" way: ~100KiB of HTML/CSS/fonts and ~300-350KiB of JS (compressed) is the new rule-of-thumb limit for at least the next year or two. Modern network performance and availability.

Performance 124

Performance Network Cache Metrics 124

ACM Sigarch

APRIL 26, 2018

The new DGX-2, which achieves 2 petaFLOPS in the system, combines 16 fully interconnected GPUS with 10x the deep learning performance. They use the graph as the basic representation for many AI-related algorithms, including neural network, Bayesian network, Markov Field, and some other emerging methods. The new GV100 packs 7.4

Hardware 99

Hardware Artificial Intelligence Architecture Google 99

Brendan Gregg

DECEMBER 22, 2019

Miller (networking maintainer) recently asked if I still used my "noisy typewriter", which I had turned on a little too loud during a [LISA 2016] demo. Masanori Misono has already ported [Tetris to bpftrace] (originally by Tomoki Sekiyama): It reads keystrokes by tracing the pty_write() kernel function. The source is: #!

C++ 102

C++ Education Java AWS 102

Brendan Gregg

JULY 4, 2021

AWS Graviton2); for memory with the arrival of DDR5 and High Bandwidth Memory (HBM) on-processor; for storage including new uses for 3D Xpoint as a 3D NAND accelerator; for networking with the rise of QUIC and eXpress Data Path (XDP); and so on. I also wrote about these topics in detail for my recent [Systems Performance 2nd Edition] book.

Performance 52

Performance Latency Hardware Storage 52

All Things Distributed

MAY 19, 2017

We decided to use our inventive powers to design and build a new database engine that would give database systems such as MySQL and PostgreSQL reliability and performance at scale, at a level that could serve even the most demanding OLTP applications.

Database 120

Database Design Cloud Storage 120

Brendan Gregg

FEBRUARY 28, 2023

This talk originated from my updates to Systems Performance 2nd Edition , and this was the first time I've given this talk in person! CXL in a way allows a custom memory controller to be added to a system, to increase memory capacity, bandwidth, and overall performance. Ford, et al., “TCP

Performance 52

Performance Latency Cache Virtualization 52

Brendan Gregg

OCTOBER 8, 2018

Created by Alastair Robertson , bpftrace is an open source high-level tracing front-end that lets you analyze systems in custom ways. eBPF (extended Berkeley Packet Filter) is in the Linux kernel and is the new hotness in systems engineering. It's shaping up to be a DTrace version 2.0: bcc is powerful but laborious to program.

C++ 110

C++ Virtualization Programming Latency 110

All Things Distributed

JUNE 20, 2017

As well as AWS Regions, we also have 21 AWS Edge Network Locations in Asia Pacific. Since then, AWS has added two more PoPs in Hong Kong, the latest in 2016. It's an entertainment website where users can post content or "memes" that they find amusing and share them across social media networks.

AWS 149

AWS Logistics Cloud Social Media 149

Brendan Gregg

DECEMBER 21, 2019

Miller (networking maintainer) recently asked if I still used my "noisy typewriter," which I had turned on a little too loud during a [LISA 2016] demo. Masanori Misono has already ported [Tetris to bpftrace] (originally by Tomoki Sekiyama): It reads keystrokes by tracing the pty_write() kernel function. The source is: #!

C++ 52

C++ Education Java AWS 52

Brendan Gregg

DECEMBER 21, 2019

Miller (networking maintainer) recently asked if I still used my "noisy typewriter," which I had turned on a little too loud during a [LISA 2016] demo. Masanori Misono has already ported [Tetris to bpftrace] (originally by Tomoki Sekiyama): It reads keystrokes by tracing the pty_write() kernel function. The source is: #!

C++ 52

C++ Education Java AWS 52

Percona

APRIL 12, 2023

Instead of relational (SQL) databases defined primarily through a hierarchy of related sets via tables and columns, their non-relational structure used a system of collections and documents. 2016: The company adds service-loaded MongoDB Professional to its mix. They came up with a horizontally scalable NoSQL database.

Open Source 79

Open Source Programming Database Servers 79

Adrian Cockcroft

JANUARY 29, 2023

There is no way to model how much more traffic you can send to that system before it exceeds it’s SLA. This is unfortunate, because we’d really like to be able to build systems that have an SLA that we can share with the consumers of our interfaces, and be able to measure how well we are doing.

Lambda 98

Lambda Latency Cache C++ 98

Brendan Gregg

NOVEMBER 29, 2017

In this configuration, the AMI and boot is paravirt (PV), the kernel is making hypercalls instead of privileged instructions, and the system is using paravirt network and storage drivers. But not all workloads: some are network bound (proxies) and storage bound (databases). ## 5. AWS called this [enhanced networking].

Virtualization 111

Virtualization AWS Hardware Storage 111

Testsigma

JANUARY 7, 2022

In 2016, Google made it clear that since mobile traffic is more than all else, mobile-friendly websites will be prioritised when a user searches on mobile. From mobile applications to chatbots to predictive systems, AI is growing its foot in every direction. 2015 saw the trend of scriptless testing and IoT focussed methodologies.

Artificial Intelligence 89

Artificial Intelligence IoT Software Software 89

O'Reilly

OCTOBER 29, 2019

To an electrical engineer, however, feedback systems are filters and amplifiers. That sound is amplified and comes out of the speaker system. The feedback system is a filter: it amplifies one set of frequencies; and everything else disappears. Network propaganda works in much the same way as microphones and music.

Social Media 53

Social Media Media Network Energy 53

Percona

SEPTEMBER 19, 2023

This blog post will walk you through how MySQL’s InnoDB redo log archiving uses PXB to manage the redo log files on the busiest systems and look at how a new feature released in PXB version 8.0.34-29 Enabling redo log archiving on the server requires setting a value for the innodb_redo_log_archive_dirs system variable.

C++ 101

C++ Servers Storage Speed 101

SQL Server According to Bob

JULY 13, 2016

And the Operating System environment for Windows Server has become more robust but has introduced more features and complexity. Today we are introducing the new Basic Installer experience for SQL Server 2016 Express , SQL Server 2016 Developer , and SQL Server 2016 Evaluation Editions.

Servers 40

Servers Media Engineering Virtualization 40

Brendan Gregg

SEPTEMBER 5, 2017

What follows are topics that may be of interest to anyone looking to migrate their systems and skillset: scan these to find topics that interest you. ## ZFS ZFS is available for Linux via the [zfsonlinux] and [OpenZFS] projects, and more recently was included in Canonical's Ubuntu Linux distribution: Ubuntu Xenial 16.04 LTS (April 2016).

Virtualization 76

Virtualization AWS Engineering Hardware 76

O'Reilly

APRIL 23, 2024

Perhaps a more direct way to say this in the context of economic value creation is that companies such as Amazon and Google and Facebook had developed a set of remarkable advances in networked and data-enabled market coordination. But over time, something went very wrong. Cory Doctorow calls this the “enshittification” of Big Tech platforms.

Social Media 108

Social Media Google Internet Internet 108

Alex Russell

SEPTEMBER 12, 2016

Instead, this article focuses on the strictest common criteria to ensure broad installability as of late 2016. Apps on the homescreen: Should load instantly, regardless of network state. This isn’t to say that they need to function fully offline, but they must put their own UI on screen without requiring a network round trip.

Mobile 52

Mobile Network Education Design 52

Alex Russell

MAY 9, 2022

Only teams that master their systems can make intentional trade-offs. Panglossian nonsense. --> Organisations that serve their tools tread water no matter how advanced their technology, while groups that intentionally manage systems can succeed on any stack. [1]. Performance isn't a binary and there are no silver bullets.

Performance 86

Performance Latency Metrics Engineering 86

Brendan Gregg

OCTOBER 7, 2018

Created by Alastair Robertson , bpftrace is an open source high-level tracing front-end that lets you analyze systems in custom ways. eBPF (extended Berkeley Packet Filter) is in the Linux kernel and is the new hotness in systems engineering. It's shaping up to be a DTrace version 2.0: bcc is powerful but laborious to program.

C++ 40

C++ Virtualization Programming Latency 40

Alex Russell

OCTOBER 22, 2017

We constrain ourselves to a real-world baseline device + network configuration to measure progress. Budgets are scaled to a benchmark network & device. Tools and CI systems help them monitor progress & prevent regressions. The median user is on a slow network. Global Ground-Truth. Where to start?

Performance 68

Performance Benchmarking Network Mobile 68

Alex Russell

MARCH 31, 2022

De-duplication can prevent multiple copies on disk, but we're out of luck on reducing network costs. Late in the drafting of this post, it was pointed out to me by Frederik Braun that Brad Hill had considered the downsides of a site-populated SRI addressable cache back in 2016. Shared caches must make sense within this value system.

Cache 82

Cache Government Traffic Network 82

The Morning Paper

SEPTEMBER 29, 2019

No link to a public version control system (e.g. that means any third-party package can access the file system and network etc. The change in the slope of the transitive dependencies line around 2016 is speculated to be due to the fallout from the left-pad incident. For npm packages running outside of the browser on Node.js

Code 76

Code Strategy Website Network 76

O'Reilly Software

APRIL 4, 2018

Are there inherent time relationships in the messages that need to be preserved as they travel across the system? In 2016, Apache Spark introduced Structured Streaming , a new streaming engine based on the SparkSQL abstractions and runtime optimizations. Clean and reprocess it? Do I need to preserve ordering? At least once?

Architecture 80

Architecture Scalability Google Operating System 80

Testsigma

FEBRUARY 12, 2021

The Internet of Things is generally referred to as IoT which encompasses computers, cars, houses or some other technological system related. billion in 2016. Millions of different mobile apps are listed in various play stores of different operating systems. Better and quicker services are urgently needed today.

Artificial Intelligence 92

Artificial Intelligence Software Software IoT 92

Smashing Magazine

JANUARY 15, 2019

This includes the work done by the server, the client and the intermediary communications networks that transmit data between the two. All of the major browsers provide developer tools that allow us to measure network activity. In 2016, O’Reilly published “ Designing For Sustainability ” by Tim Frick. Large preview ).

Website Performance 73

Website Performance Website Energy Performance 73

Brendan Gregg

NOVEMBER 28, 2017

In this configuration, the AMI and boot is paravirt (PV), the kernel is making hypercalls instead of privileged instructions, and the system is using paravirt network and storage drivers. But not all workloads: some are network bound (proxies) and storage bound (databases). ## 5. AWS called this [enhanced networking].

Virtualization 40

Virtualization AWS Hardware Storage 40

Alex Russell

APRIL 29, 2021

After 20 years of neck-in-neck competition, often starting from common code lineages, there just isn't that much left to wring out of the system. Consistent improvement is the name of the game, and it can still have positive impacts, particularly as users lean on the system more heavily over time. Some relatively minor additions (e.g.

Media 145

Media Games Education Engineering 145

Testsigma

DECEMBER 8, 2019

It has been since 2016 that a major IT firm WIPRO has been pushing for the application of hyper-automation to all their processes in their own firm. DevOps has been around for about a decade now and has become a crucial automation process that enables organizations to deploy changes to a system in production quickly and efficiently.

IoT 78

IoT Software Software Artificial Intelligence 78