The Netflix TechBlog

OCTOBER 19, 2020

If we had an ID for each streaming session then distributed tracing could easily reconstruct session failure by providing service topology, retry and error tags, and latency measurements for all service calls. Using simple lookup indices in Cassandra gives us the ability to maintain acceptable read latencies while doing heavy writes.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

Brendan Gregg

JULY 4, 2021

It's an exciting time for developments in computer performance, not just for the BPF technology (which I often [write about]) but also for processors with 3D stacking and cloud vendor CPUs (e.g., Ford, et al., “TCP on Upcoming Sapphire Rapids CPUs,” [link] Oct 2020 - [Liu 20] Linda Liu, “Samsung QVO vs EVO vs PRO: What’s the Difference?

Performance 140

Performance Latency Hardware Storage 140

The Netflix TechBlog

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! To sustain this data growth at Netflix, it has deployed open-source software Ceph using AWS services to achieve the required SLOs of some of the post-production workflows.

AWS 37

AWS Entertainment Open Source Benchmarking 37

High Scalability

MARCH 22, 2019

Hell, many of these providers are just providing open source API compatibility with custom-built backends! What happens when no new open source comes out of the smaller companies, and the big-3 decide they don't really need or want to play nice anymore? They'll learn a lot and love you even more.5 We achieve 5.5

Internet 134

Internet Internet Scalability Wireless 134

DZone

OCTOBER 31, 2019

Improve AWS performance! AWS is the #1 cloud provider for open-source database hosting, and the go-to cloud for MySQL deployments.

AWS 36

AWS Latency Performance Open Source 36

Brendan Gregg

FEBRUARY 28, 2023

My personal opinion is that I don't see a widespread need for more capacity given horizontal scaling and servers that can already exceed 1 Tbyte of DRAM; bandwidth is also helpful, but I'd be concerned about the increased latency for adding a hop to more memory. Ford, et al., “TCP

Performance 110

Performance Latency Cache Virtualization 110

Brendan Gregg

JULY 4, 2021

It's an exciting time for developments in computer performance, not just for the BPF technology (which I often [write about]) but also for processors with 3D stacking and cloud vendor CPUs (e.g., Ford, et al., “TCP on Upcoming Sapphire Rapids CPUs,” [link] Oct 2020 - [Liu 20] Linda Liu, “Samsung QVO vs EVO vs PRO: What’s the Difference?

Performance 52

Performance Latency Hardware Storage 52

Brendan Gregg

FEBRUARY 28, 2023

My personal opinion is that I don't see a widespread need for more capacity given horizontal scaling and servers that can already exceed 1 Tbyte of DRAM; bandwidth is also helpful, but I'd be concerned about the increased latency for adding a hop to more memory. Ford, et al., “TCP

Performance 52

Performance Latency Cache Virtualization 52

Adrian Cockcroft

NOVEMBER 18, 2019

We’re excited to let you know that we have an Open Source track at re:Invent this year! AWS Developer Relations on how the shift from Robot Operating System (ROS) 1 to ROS 2 will change the landscape for all robot lovers. Brendan Gregg tours BPF tracing, with open source tools & examples for EC2 instance analysis.

Open Source 40

Open Source AWS Lambda Serverless 40

Adrian Cockcroft

NOVEMBER 18, 2019

We’re excited to let you know that we have an Open Source track at re:Invent this year! AWS Developer Relations on how the shift from Robot Operating System (ROS) 1 to ROS 2 will change the landscape for all robot lovers. Brendan Gregg tours BPF tracing, with open source tools & examples for EC2 instance analysis.

Open Source 40

Open Source AWS Lambda Serverless 40

Percona

DECEMBER 11, 2023

A quick configuration change may do the trick in improving the performance of your AWS RDS for MySQL instance. DLVs are particularly advantageous for databases with large allocated storage, high I/O per second (IOPS) requirements, or latency-sensitive workloads. Who can benefit from DLV? and later v10 versions MySQL: 8.0.28

AWS 95

AWS Benchmarking Performance Traffic 95

Scalegrid

NOVEMBER 25, 2019

ScyllaDB is an open-source distributed NoSQL data store, reimplemented from the popular Apache Cassandra database. ScyllaDB offers significantly lower latency which allows you to process a high volume of data with minimal delay. percentile latency is up to 11X better than Cassandra on AWS EC2 bare metal.

Big Data 187

Big Data Database Open Source Azure 187

VoltDB

JUNE 29, 2017

Most existing adtech infrastructure simply can not achieve the required latency. VoltDB provides the necessary technology to achieve the latency required by header bidding. And while it’s easy to say “spin up more AWS servers”, you’re only adding to your cloud provider bill. Another adtech infrastructure problem is capacity.

Performance 40

Performance Hardware Latency Infrastructure 40

VoltDB

JUNE 29, 2017

Most existing adtech infrastructure simply can not achieve the required latency. VoltDB provides the necessary technology to achieve the latency required by header bidding. And while it’s easy to say “spin up more AWS servers”, you’re only adding to your cloud provider bill. Another adtech infrastructure problem is capacity.

Performance 40

Performance Hardware Latency Infrastructure 40

The Symphonia

MAY 22, 2018

I also rewrote the section on Startup Latency since Cold Starts are one of the big “FUD” areas of Serverless. Also there’s been a lot of open source updates, including from Amazon and Microsoft. and products like AWS Fargate have made this distinction murkier still. I also wrote an article on just this area last year.

Serverless 40

Serverless Architecture Lambda Azure 40

Scalegrid

DECEMBER 16, 2019

MongoDB is the #3 open source database and the #1 NoSQL database in the world. So, what makes DigitalOcean a compelling alternative to other popular MongoDB hosting providers like AWS and Azure ? DigitalOcean, along with AWS and Azure, offer a 99.99% uptime SLA, but while that sounds close to perfect, that 0.01% equates to 52.6

Azure 187

Azure AWS Latency Database 187

The Morning Paper

MAY 12, 2019

An open-source benchmark suite for microservices and their hardware-software implications for cloud & edge systems Gan et al., It’s a pretty impressive effort to pull together and make available in open source (not yet available as I write this) such a suite, and I’m sure explains much of the long list of 24 authors on this paper.

Open Source 61

Open Source Hardware Benchmarking Systems 61

The Netflix TechBlog

FEBRUARY 20, 2019

Today we are excited to announce latency heatmaps and improved container support for our on-host monitoring solution?—?Vector?—?to Vector is open source and in use by multiple companies. Remotely view real-time process scheduler latency and tcp throughput with Vector and eBPF What is Vector? to the broader community.

Performance 211

Performance Latency Open Source Metrics 211

Percona

JULY 1, 2023

Now that Database-as-a-service (DBaaS) is in high demand, there are multiple questions regarding AWS services that cannot always be answered easily: When should I use Aurora and when should I use RDS MySQL ? Amazon Aurora is a proprietary, cloud-native, fully managed relational database service developed by Amazon Web Services (AWS).

AWS 52

AWS Database Serverless Storage 52

Dynatrace

MAY 17, 2023

OpenTelemetry , the open source observability tool, has become the go-to standard for instrumenting custom applications to collect observability telemetry data. Monitoring DNS query time is important for understanding network latency, ensuring that services are available, troubleshooting issues, and optimizing application performance.

Metrics 171

Metrics Database Monitoring Network 171

Dynatrace

OCTOBER 1, 2021

In these modern environments, every hardware, software, and cloud infrastructure component and every container, open-source tool, and microservice generates records of every activity. Open-source solutions, such as OpenTelemetry , provide a de facto standard for collecting telemetry data in cloud settings.

Metrics 363

Metrics Open Source Monitoring Infrastructure 363

High Scalability

SEPTEMBER 8, 2018

No, I don’t think that is because AWS is earning a 355x margin on DynamoDB! To meet user-defined goals for performance (request latency) and cost, the monitoring service tracks and adjusts resources to workload changes. You get all the multicore Anna performance you want, but you don’t pay for what you don’t need.

Storage 199

Storage Performance AWS Efficiency 199

All Things Distributed

JULY 20, 2015

This architecture affords Amazon ECS high availability, low latency, and high throughput because the data store is never pessimistically locked. As you can see, the latency remains relatively jitter-free despite large fluctuations in the cluster size. Hailo was founded in 2011 and has been built on AWS since Day 1.

Latency 166

Latency Architecture AWS Open Source 166

High Scalability

DECEMBER 21, 2018

2) Cloud will decentralise in terms of provision not power i.e. Amazon will "invade" more of those holdouts with AWS Outpost. kellabyte : “Open source” infrastructure companies are a giant s**t show right now. Me : Nothing special. 1) Enterprise data centres will continue to close. 3) Serverless will rocket.

Internet 137

Internet Internet Scalability Serverless 137

Smashing Magazine

AUGUST 3, 2023

For Performance Monitoring Many of you reading this may already be familiar with Next.js, but it is a popular open-source JavaScript framework that allows us to monitor our website’s performance in real-time. Addressing Many Pages With Rare Visits Adopting the AWS CloudFront CDN significantly improved our website’s performance.

Performance 74

Performance Cache Traffic Metrics 74

All Things Distributed

NOVEMBER 21, 2017

Redis's microsecond latency has made it a de facto choice for caching. Four years ago, as part of our AWS fast data journey, we introduced Amazon ElastiCache for Redis , a fully managed, in-memory data store that operates at microsecond latency. TB of in-memory capacity in a single cluster. Atomic slot migration.

Games 113

Games Retail Latency Education 113

Percona

MAY 2, 2023

Also, we will take a look at our open-source backup utility custom-built to help avoid costs and proprietary software – Percona Backup for MongoDB or PBM. It also supports storing the backups on filesystem and AWS S3 buckets. root@ip-172-31-37-92 tmp]# time mongo < hot.js 8 connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb

Best Practices 52

Best Practices Database Storage Servers 52

The Netflix TechBlog

DECEMBER 17, 2019

Therefore, dumps are needed to capture the full state of a source. There are several open source CDC projects, often using the same underlying libraries, database APIs, and protocols. There are services at Netflix that use RDBMS kind of databases such as MySQL or PostgreSQL via AWS RDS.

Database 197

Database Traffic Transportation Open Source 197

The Netflix TechBlog

DECEMBER 17, 2019

Therefore, dumps are needed to capture the full state of a source. There are several open source CDC projects, often using the same underlying libraries, database APIs, and protocols. There are services at Netflix that use RDBMS kind of databases such as MySQL or PostgreSQL via AWS RDS.

Database 212

Database Traffic Transportation Open Source 212

The Netflix TechBlog

AUGUST 13, 2020

It usually has dependencies, talks to other services, and lives in different AWS regions. For example, a latency increase is less critical than error rate increase and some error codes are less critical than others. We start with Spinnaker , our open source delivery platform. Client metrics and QoE changes.

Monitoring 210

Monitoring Tuning Traffic Metrics 210

Smashing Magazine

AUGUST 20, 2021

It was a battle of not only proprietary vs open source but also static vs dynamic. You could create and update blog posts, all content was straight HTML — open-source WYSIWYG editors weren’t available at the time, and Markdown didn’t come about until 2004. We can see all the bones of modern Jamstack CMSs here.

Ecommerce 139

Ecommerce Website Government Internet 139

All Things Distributed

AUGUST 20, 2015

When using relational databases, traversing relationships requires expensive table JOIN operations, causing significantly increased latency as table size and query complexity grow. Like many AWS innovations, the desire to build a solution for a scalable graph database came from Amazon’s retail business. Enter graph databases.

Database 134

Database Logistics Availability Social Media 134

All Things Distributed

OCTOBER 12, 2016

Three years ago, as part of our AWS Fast Data journey we introduced Amazon ElastiCache for Redis , a fully managed in-memory data store that operates at sub-millisecond latency. All these capabilities are available to customers at no additional charge, and maintain open-source Redis compatibility. Redis and Fast Data. “Now

Scalability 131

Scalability Analytics Cache AWS 131

Adrian Cockcroft

APRIL 5, 2023

This proposal seeks to define a standard for real-time carbon and energy data as time-series data that would be accessed alongside and synchronized with the existing throughput, utilization and latency metrics that are provided for the components and applications in computing environments.

Energy 52

Energy Metrics Cloud Operating System 52

Percona

APRIL 17, 2023

The CFQ works well for many general use cases but lacks latency guarantees. The deadline excels at latency-sensitive use cases ( like databases ), and noop is closer to no schedule at all. Percona’s open source database monitoring tool monitors the health of your database infrastructure and helps you improve MongoDB performance.

Best Practices 52

Best Practices Design Tuning Database 52

Adrian Cockcroft

NOVEMBER 11, 2019

Collecting some critical metrics at one second intervals, with a total observability latency of ten seconds or less matches the human attention span much better. This is why most AWS regions have three availability zones. Try to measure your mean time to respond (MTTR) for incidents. that are used across multiple applications.

Latency 52

Latency Engineering Systems Hardware 52

Adrian Cockcroft

NOVEMBER 11, 2019

Collecting some critical metrics at one second intervals, with a total observability latency of ten seconds or less matches the human attention span much better. This is why most AWS regions have three availability zones. Try to measure your mean time to respond (MTTR) for incidents. that are used across multiple applications.

Latency 53

Latency Engineering Systems Hardware 53