The Netflix TechBlog

OCTOBER 19, 2020

If we had an ID for each streaming session then distributed tracing could easily reconstruct session failure by providing service topology, retry and error tags, and latency measurements for all service calls. Using simple lookup indices in Cassandra gives us the ability to maintain acceptable read latencies while doing heavy writes.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

Dynatrace

APRIL 5, 2021

These include website hosting, database management, backup and restore, IoT capabilities, e-commerce solutions, app development tools and more, with new services released regularly. Lambda’s toolbox of automated processes helps developers streamline to build fast, robust, and scalable applications on accelerated timelines.

Lambda 180

Lambda AWS Serverless Hardware 180

InfoQ

DECEMBER 27, 2023

LinkedIn announced that it would be moving to gRPC with Protocol Buffers for the inter-service communication in its microservices platform, where previously an open-source Rest.li framework was used with JSON as a primary serialization format.

Open Source 143

Open Source Latency Tuning Scalability 143

The Netflix TechBlog

NOVEMBER 22, 2019

To sustain this data growth at Netflix, it has deployed open-source software Ceph using AWS services to achieve the required SLOs of some of the post-production workflows. Netflix runs dozens of stateful services on AWS under strict sub-millisecond tail-latency requirements, which brings unique challenges.

AWS 100

AWS Entertainment Open Source Benchmarking 100

The Netflix TechBlog

NOVEMBER 22, 2019

To sustain this data growth at Netflix, it has deployed open-source software Ceph using AWS services to achieve the required SLOs of some of the post-production workflows. Netflix runs dozens of stateful services on AWS under strict sub-millisecond tail-latency requirements, which brings unique challenges.

AWS 100

AWS Entertainment Open Source Benchmarking 100

Brendan Gregg

JULY 4, 2021

It's an exciting time for developments in computer performance, not just for the BPF technology (which I often [write about]) but also for processors with 3D stacking and cloud vendor CPUs (e.g., Ford, et al., “TCP on Upcoming Sapphire Rapids CPUs,” [link] Oct 2020 - [Liu 20] Linda Liu, “Samsung QVO vs EVO vs PRO: What’s the Difference?

Performance 140

Performance Latency Hardware Storage 140

InfoQ

JANUARY 29, 2024

The team at RevenueCat created an open-source memcache client that provides several advanced features. RevenueCat extensively uses caching to improve the availability and performance of its product API while ensuring consistency. The company shared its techniques to deliver the platform, which can handle over 1.2 By Rafal Gancarz

Cache 107

Cache Open Source Availability Performance 107

Scalegrid

MARCH 22, 2024

Deploying a multi-cloud environment involves a concerted effort in planning, negotiation with vendors, skill development, and understanding the advanced use cases of a multi-cloud approach for tailored industry solutions and compliance. They can also bolster uptime and limit latency issues or potential downtimes.

Strategy 130

Strategy Cloud Government Innovation 130

Brendan Gregg

FEBRUARY 28, 2023

At USENIX SREcon22 APAC I gave the opening keynote on the future of computer performance, rounding up the latest developments and making predictions of where I see things heading. This talk originated from my updates to [Systems Performance 2nd Edition], and this was the first time I've given this talk in person! Ford, et al., “TCP

Performance 110

Performance Latency Cache Virtualization 110

The Netflix TechBlog

MARCH 13, 2020

SVT-AV1: open-source AV1 encoder and decoder by Andrey Norkin , Joel Sole , Mariana Afonso , Kyle Swanson, Agata Opalach , Anush Moorthy , Anne Aaron SVT-AV1 is an open-source AV1 codec implementation hosted on GitHub [link] under a BSD + patent license. As seen below, SVT-AV1 demonstrates 16.5%

Open Source 154

Open Source Efficiency C++ Speed 154

Dynatrace

JULY 6, 2020

As adoption rates for Microsoft Azure continue to skyrocket, Dynatrace is developing a deeper integration with the platform to provide even more value to organizations that run their businesses on Azure or use it as a part of their multi-cloud strategy. Deeper visibility and more precise answers. Get a comprehensive view of your batch jobs.

Azure 148

Azure Cloud Big Data Virtualization 148

Brendan Gregg

FEBRUARY 28, 2023

At USENIX SREcon22 APAC I gave the opening keynote on the future of computer performance, rounding up the latest developments and making predictions of where I see things heading. This talk originated from my updates to Systems Performance 2nd Edition , and this was the first time I've given this talk in person! Ford, et al., “TCP

Performance 52

Performance Latency Cache Virtualization 52

Brendan Gregg

AUGUST 18, 2019

bpftrace is a new open source tracer for Linux for analyzing production performance problems and troubleshooting software. It was created by Alastair Robertson, a talented UK-based developer who has previously won various coding competitions. For smaller environments, it can be of more use helping eliminate latency outliers.

Latency 68

Latency C++ Cache Programming 68

Brendan Gregg

JULY 4, 2021

It's an exciting time for developments in computer performance, not just for the BPF technology (which I often [write about]) but also for processors with 3D stacking and cloud vendor CPUs (e.g., Ford, et al., “TCP on Upcoming Sapphire Rapids CPUs,” [link] Oct 2020 - [Liu 20] Linda Liu, “Samsung QVO vs EVO vs PRO: What’s the Difference?

Performance 52

Performance Latency Hardware Storage 52

Brendan Gregg

JULY 2, 2021

opensnoop Files opened table. biolatency Disk I/O latency histogram heat map. runqlat CPU scheduler latency heat map. Here are the top ten tools you can run and present as a generic BPF observability dashboard, along with suggested visualizations: Tool Shows Visualization. execsnoop New processes (via exec(2)) table.

Latency 145

Latency Cache Energy Systems 145

Testsigma

AUGUST 24, 2020

Continuous Testing is the process of testing at all stages of software development – one after the another- without any human intervention. Continuous Testing makes it possible to eliminate testing as a bottleneck for faster software development and delivery. Continuous Testing is key to faster delivery of Agile products to the market.

Testing 69

Testing Open Source Scalability Latency 69

The Netflix TechBlog

NOVEMBER 22, 2019

To sustain this data growth at Netflix, it has deployed open-source software Ceph using AWS services to achieve the required SLOs of some of the post-production workflows. Netflix runs dozens of stateful services on AWS under strict sub-millisecond tail-latency requirements, which brings unique challenges.

AWS 37

AWS Entertainment Open Source Benchmarking 37

The Morning Paper

SEPTEMBER 5, 2019

For response times (latencies) reporting a simple metric such as ‘average’ is next to useless. Instead we want to understand what’s happening at different latency percentiles (e.g For reasons we’ll see shortly though, none of those were good enough for Datadog, so they developed their own sketching data structure, DDSketch.

Latency 52

Latency Metrics Open Source Java 52

Adrian Cockcroft

NOVEMBER 18, 2019

We’re excited to let you know that we have an Open Source track at re:Invent this year! Learn about what’s in it: [link] Some selected sessions: OPN201 Change is coming to robotics development: The shift to ROS 2 ?? Brendan Gregg tours BPF tracing, with open source tools & examples for EC2 instance analysis.

Open Source 40

Open Source AWS Lambda Serverless 40

Adrian Cockcroft

NOVEMBER 18, 2019

We’re excited to let you know that we have an Open Source track at re:Invent this year! Learn about what’s in it: [link] Some selected sessions: OPN201 Change is coming to robotics development: The shift to ROS 2 ?? Brendan Gregg tours BPF tracing, with open source tools & examples for EC2 instance analysis.

Open Source 40

Open Source AWS Lambda Serverless 40

VoltDB

JUNE 29, 2017

Most existing adtech infrastructure simply can not achieve the required latency. VoltDB provides the necessary technology to achieve the latency required by header bidding. Open source analytics tools can give you this information – but not in real-time. Another adtech infrastructure problem is capacity.

Performance 40

Performance Hardware Latency Infrastructure 40

VoltDB

JUNE 29, 2017

Most existing adtech infrastructure simply can not achieve the required latency. VoltDB provides the necessary technology to achieve the latency required by header bidding. Open source analytics tools can give you this information – but not in real-time. Another adtech infrastructure problem is capacity.

Performance 40

Performance Hardware Latency Infrastructure 40

Adrian Cockcroft

JULY 4, 2023

The rest of this blog post is an edited version of the conversation I had with ChatGPT, that also acts as documentation of the code I’ve open sourced. At that time I developed a function called as.peaks — which takes a histogram, and fits normally distributed peaks to it, and shared that code on github.

Development 52

Development Latency C++ Open Source 52

Abhishek Tiwari

JULY 12, 2018

Data Pipeline A data pipeline is a software that ingests data from multiple sources, transforms it and finally makes it available to internal or external products. In recent times, in order to gain valuable insights or to develop the data-driven products companies such as Netflix, Spotify, Uber, AirBnB have built internal data pipelines.

Latency 63

Latency Analytics Scalability Engineering 63

HammerDB

MAY 5, 2019

Note that the main developer of HammerDB is Intel employee (#IAMINTEL) however HammerDB is a personal open source project and HammerDB has no optimization whatsoever for a database running on any particular architecture. In the recent MySQL 8.0.16

Testing 48

Testing Tuning Latency Storage 48

KeyCDN

AUGUST 15, 2019

In the context of web development, performance testing entails using software tools to simulate how an application runs under specific circumstances. Wait time: Sometimes called average latency, wait time refers the amount of time a request spends in a queue before it gets processed. What is Performance Testing?

Testing Tools 55

Testing Tools Best Practices Performance Testing Testing 55

Brendan Gregg

JULY 2, 2021

If you're using an open source monitoring platform, first check if it already has a BPF agent. opensnoop Files opened table 3. biolatency Disk I/O latency histogram heat map 5. runqlat CPU scheduler latency heat map 10. Think of this as "version 1": A starting point that's pretty useful.

Open Source 40

Open Source Latency Cache Energy 40

Scalegrid

MARCH 28, 2024

Introduction Caching serves a dual purpose in web development – speeding up client requests and reducing server load. Redis Revealed: An Overview Redis, a renowned open-source, in-memory remote dictionary server, stands out for its diverse data structures and advanced features.

Cache 130

Cache Storage Scalability Architecture 130

Brendan Gregg

OCTOBER 8, 2018

Created by Alastair Robertson , bpftrace is an open source high-level tracing front-end that lets you analyze systems in custom ways. It is being developed for BSD, too, where BPF originated. Since I helped developed bpftrace, I'm aware of how fresh my own code is and how likely I introduced bugs. I wrote seeksize.d

C++ 110

C++ Virtualization Programming Latency 110

The Morning Paper

MARCH 10, 2020

Therefore, using the benchmarking results of YCSB as guidance for production might cause some misleading results… To address this issue, we propose a key-range based modelling and develop a benchmark that can better emulate the workloads of real-world key-value stores.

Benchmarking 59

Benchmarking Storage Cache Open Source 59

Dynatrace

MAY 13, 2022

Apache Cassandra is an open-source, distributed, NoSQL database. It also removes the need for developers and database administrators to manage infrastructure or update database versions. Below is an example Dynatrace problem card, which shows how a spike in Cassandra write latency impacts your application.

Azure 237

Azure Latency Metrics Infrastructure 237

O'Reilly

JANUARY 16, 2024

For years I had an aspirational goal of unifying all of my backups into one central directory tree, akin to the concept of a monorepo in software development. I’ve recently been brainstorming ideas for how to design such a system and how to deal with the practical challenges of scaling and maintenance.

Entertainment 76

Entertainment Internet Internet Design 76

Abhishek Tiwari

DECEMBER 27, 2017

faster development and deployment cycles. The good news is all of these challenges including performance can be tackled and in several cases, there are new developments to overcome them. To address this, the open source community is now trying to develop Kubernetes custom schedulers specifically optimised for big data workloads.

Big Data 63

Big Data Storage Benchmarking Hardware 63

The Symphonia

MAY 22, 2018

I also rewrote the section on Startup Latency since Cold Starts are one of the big “FUD” areas of Serverless. Also there’s been a lot of open source updates, including from Amazon and Microsoft. I also wrote an article on just this area last year. Deployment and packaging has also got much better.

Serverless 40

Serverless Architecture Lambda Azure 40

Scalegrid

JUNE 4, 2020

DigitalOcean is a cost-effective cloud provider that caters to, and is widely adopted by the developer community. Compare Latency. lower latency compared to DigitalOcean for PostgreSQL. Now, let’s take a look at the throughput and latency performance of our comparison. ScaleGrid PostgreSQL provides on average 42.3%

Database 230

Database Latency Benchmarking Performance 230

The Netflix TechBlog

FEBRUARY 20, 2019

Today we are excited to announce latency heatmaps and improved container support for our on-host monitoring solution?—?Vector?—?to Vector is open source and in use by multiple companies. Remotely view real-time process scheduler latency and tcp throughput with Vector and eBPF What is Vector? to the broader community.

Performance 211

Performance Latency Open Source Metrics 211

Percona

JULY 17, 2023

Developers and DBAs like MongoDB’s ease-of-use. DBAs and developers appreciate its combination of flexibility, scalability, and performance. DBAs and developers appreciate its combination of flexibility, scalability, and performance. It ranks No. 5 among all database management systems and No.

Open Source 73

Open Source Database Scalability Software 73