The Netflix TechBlog

OCTOBER 28, 2021

I developed many batch and real-time data pipelines using open source technologies for AOL Advertising and eBay. Over the years, I followed the big data open-source community and Netflix tech blogs closely, and learned a lot about Netflix’s innovative engineering solutions and active contributions to the open-source ecosystem.

Data Engineering 173

Data Engineering Engineering Big Data Software Engineering 173

The Netflix TechBlog

OCTOBER 19, 2020

Troubleshooting a session in Edgar When we started building Edgar four years ago, there were very few open-source distributed tracing systems that satisfied our needs. Our tactical approach was to use Netflix-specific libraries for collecting traces from Java-based streaming services until open source tracer libraries matured.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

Dynatrace

MARCH 24, 2023

Container technology enables organizations to efficiently develop cloud-native applications or to modernize legacy applications to take advantage of cloud services. Originally created by Google, Kubernetes was donated to the CNCF as an open source project. How does container orchestration work?

Infrastructure 186

Infrastructure Open Source Operating System Cloud 186

Percona

MARCH 7, 2023

One approach is to separate compute and storage to allow for independent scaling. It’s an open source alternative to AWS Aurora Postgres that utilizes a serverless architecture. It’s an open source alternative to AWS Aurora Postgres that utilizes a serverless architecture. So, what is Neon?

Serverless 111

Serverless Storage Open Source Architecture 111

Percona

DECEMBER 27, 2023

We want adopters of our technology to be able to deploy their critical open source databases and applications across any public or private cloud environment. Consequently, they might miss out on the benefits of integrating security into the SDLC, such as enhanced efficiency, speed, and quality in software delivery.

Cloud 80

Cloud Open Source Strategy Best Practices 80

The Morning Paper

NOVEMBER 5, 2019

File systems unfit as distributed storage backends: lessons from 10 years of Ceph evolution Aghayev et al., In this case, the assumption that a distributed storage backend should clearly be layered on top of a local file system. What is a distributed storage backend? SOSP’19. This is not surprising in hindsight.

Storage 64

Storage Systems Hardware Efficiency 64

Percona

FEBRUARY 15, 2023

A bloom filter is a space-efficient way of storing information about a list of keys. LSM storage engines like MyRocks are very different from the more common B-Tree-based storage engines like InnoDB. Bloom filter is an important feature improving performance of LSM storage engines.

Storage 128

Storage Tuning Cache Engineering 128

Percona

JUNE 22, 2023

A monitoring tool like Percona Monitoring and Management (PMM) is a popular choice among open source options for effectively monitoring MySQL performance. Query performance Query performance is a key performance indicator (KPI) in MySQL, as it measures the efficiency and speed of query execution. Hint Hint, ProxySQL helps.

Performance 104

Performance Monitoring Traffic Database 104

Percona

MAY 1, 2023

PostgreSQL performance optimization aims to improve the efficiency of a PostgreSQL database system by adjusting configurations and implementing best practices to identify and resolve bottlenecks, improve query speed, and maximize database throughput and responsiveness. What is PostgreSQL performance tuning?

Tuning 52

Tuning Efficiency Database Performance 52

Dynatrace

DECEMBER 15, 2022

In this blog, we share three log ingestion strategies from the field that demonstrate how building up efficient log collection can be environment-agnostic by using our generic log ingestion application programming interface (API). Set up the configuration on the same port as specified for source data, in this example 5140.

Strategy 180

Strategy AWS Open Source Transportation 180

Percona

OCTOBER 18, 2023

Migrating a proprietary database to open source is a major decision that can significantly affect your organization. Advantages of migrating to open source For many reasons mentioned earlier, organizations are increasingly shifting towards open source databases for their data management needs.

Open Source 92

Open Source Database Hardware Strategy 92

Dynatrace

DECEMBER 18, 2023

Another source reported a 43% overall increase in Prometheus adoption for the last six months of 2021. As the two first graduates of the open source project incubator, the Cloud Native Computing Foundation (CNCF), Kubernetes and Prometheus are usually deployed together. What is Prometheus?

Metrics 178

Metrics Engineering Energy Tuning 178

Percona

JUNE 30, 2023

Percona, a leading provider of open-source database software and services, announced the general availability of Percona Operator for PostgreSQL version 2. The solution is 100% open source and automates the deployment and management of PostgreSQL clusters on Kubernetes. In version 1.x,

Availability 94

Availability Open Source Azure Hardware 94

Dynatrace

MAY 26, 2021

From May 17 to May 18, 2021, the Open-Source Engineering team at Dynatrace attended the virtual observability conference, o11yfest. Trace-based sampling can help you save storage costs. This can help you save money in storage costs in the long run. Dynatrace news. Seek support from vendors on your o11y journey.

AWS 212

AWS Strategy Storage Metrics 212

Percona

JUNE 4, 2023

MariaDB is a popular SQL open source relational database management system that originated as a fork of MySQL after MySQL was acquired by Sun Microsystems in 2008 and later Oracle in 2010. It continues to be developed and maintained by the open source community and remains free to use under the GNU General Public License.

Open Source 52

Open Source Servers Storage Scalability 52

Percona

AUGUST 17, 2023

PostgreSQL is open source relational database management software. It has evolved steadily, however, during 25 years as an open source project. Another reason for PostgreSQL popularity (perhaps the biggest): It’s open source and free to use, distribute, and modify. What is PostgreSQL used for?

Open Source 78

Open Source Healthcare Social Media Database 78

Scalegrid

JUNE 22, 2020

MySQL is the all-time number one open source database in the world, and a staple in RDBMS space. ScaleGrid provides 30% more storage on average vs. DigitalOcean for MySQL at the same affordable price. In this benchmark, we measure MySQL throughput in terms of queries per second (QPS) to measure our query efficiency.

Database 217

Database Benchmarking Latency Performance 217

Dynatrace

AUGUST 5, 2022

For nonurgent messages, texting is a more efficient approach. Consumers store messages in a queue — usually in a buffer or on a storage medium — until they can process and delete them. The concept is like text messaging — a feature most mobile phone users understand. If you need to send a message, you can call the person.

Monitoring 200

Monitoring Serverless Programming Speed 200

Dynatrace

AUGUST 5, 2022

For nonurgent messages, texting is a more efficient approach. Consumers store messages in a queue — usually in a buffer or on a storage medium — until they can process and delete them. The concept is like text messaging — a feature most mobile phone users understand. If you need to send a message, you can call the person.

Monitoring 199

Monitoring Serverless Programming Speed 199

Percona

JANUARY 19, 2023

In this blog post, we will review key topics to consider for managing large datasets more efficiently in MySQL. Redundant indexes: It is known that accessing rows by fetching an index is more efficient than through a table scan in most cases. It can help us to save costs on storage and backup times.

Open Source 88

Open Source Storage Database Big Data 88

Dynatrace

MAY 26, 2021

From May 17 to May 18, 2021, the Open-Source Engineering team at Dynatrace attended the virtual observability conference, o11yfest. Trace-based sampling can help you save storage costs. This can help you save money in storage costs in the long run. Dynatrace news. Seek support from vendors on your o11y journey.

AWS 147

AWS Strategy Storage Metrics 147

The Netflix TechBlog

NOVEMBER 22, 2019

4:45pm-5:45pm NFX 209 File system as a service at Netflix Kishore Kasi , Senior Software Engineer Abstract : As Netflix grows in original content creation, its need for storage is also increasing at a rapid pace. Technology advancements in content creation and consumption have also increased its data footprint.

AWS 100

AWS Entertainment Open Source Benchmarking 100

The Netflix TechBlog

NOVEMBER 22, 2019

4:45pm-5:45pm NFX 209 File system as a service at Netflix Kishore Kasi , Senior Software Engineer Abstract : As Netflix grows in original content creation, its need for storage is also increasing at a rapid pace. Technology advancements in content creation and consumption have also increased its data footprint.

AWS 100

AWS Entertainment Open Source Benchmarking 100

Dynatrace

DECEMBER 16, 2022

Without open standards, platforms promote lock-in and hinder innovation. With open standards, developers can take a Lego-like approach to application development, which makes delivery more efficient. AI-powered precise answers and timely insights with ad-hoc analytics. Empower your ecosystem with extensibility.

Innovation 197

Innovation Software Software Cloud 197

All Things Distributed

MAY 19, 2017

In many, high-throughput, OLTP style applications the database plays a crucial role to achieve scale, reliability, high-performance and cost efficiency. We believe the central constraint in high throughput data processing has moved from compute and storage to the network.

Database 117

Database Design Cloud Storage 117

Adrian Cockcroft

NOVEMBER 26, 2023

SUS101: Sustainability innovation in AWS Global Infrastructure AWS is determined to make the cloud the cleanest and most energy-efficient way to run customers’ infrastructure and business. This includes providing the efficient, resilient services AWS customers expect, while minimizing their environmental footprint.

AWS 52

AWS Energy IoT Best Practices 52

CSS - Tricks

SEPTEMBER 21, 2020

There has got to be a balance between the propagation and duplicative storage as far as the savings that would be realized by the efficiency of saving requests. For example, I can build a little area that opens and closes in HTML with a <details> element. Energy source used by the data centre 4. I kinda doubt it.

Website 97

Website Energy Storage Efficiency 97

Percona

SEPTEMBER 19, 2023

Now, since backup speed and compactness are important for busy, important databases, Percona’s open source physical backup solution – Percona XtraBackup (PXB) , takes into account all these aspects and benefits MySQL world with its exceptional capabilities! Taking backup off-peak time while the system saturation is lower.

C++ 105

C++ Servers Storage Speed 105

Percona

MARCH 30, 2023

The use of open source databases has increased steadily in recent years. Past trepidation — about perceived vulnerabilities and performance issues — has faded as decision makers realize what an “open source database” really is and what it offers. What is an open source database?

Open Source 75

Open Source Database Storage Scalability 75

The Netflix TechBlog

JULY 30, 2019

and beyond By Anoop Panicker and Kishore Banala Conductor is a workflow orchestration engine developed and open-sourced by Netflix. Many of the Netflix Content and Studio Engineering services rely on Conductor for efficient processing of their business flows. The Netflix Media Database (NMDB) is one such example.

Lambda 189

Lambda Media Open Source Metrics 189

Percona

JULY 25, 2023

After the “data dictionary” (DD) engine and DD cache are initialized on a server, the Storage Engines can ask for a table definition. 28, xtrabackup –prepare is faster and memory efficient with the dictionary cache. This search tuple (key) is used to find the record to perform the undo operation.

Cache 86

Cache Benchmarking Servers Design 86

Brendan Gregg

FEBRUARY 28, 2023

Make sure your system can handle next-generation DRAM,” [link] Nov 2011 - [Hruska 12] Joel Hruska, “The future of CPU scaling: Exploring options on the cutting edge,” [link] Feb 2012 - [Gregg 13] Brendan Gregg, “Blazing Performance with Flame Graphs,” [link] 2013 - [Shimpi 13] Anand Lal Shimpi, “Seagate to Ship 5TB HDD in 2014 using Shingled Magnetic (..)

Performance 110

Performance Latency Cache Virtualization 110

Percona

JANUARY 4, 2024

Managing storage and performance efficiently in your MySQL database is crucial, and general tablespaces offer flexibility in achieving this. In contrast to the single system tablespace that holds system tables by default, general tablespaces are user-defined storage containers for multiple InnoDB tables.

Storage 88

Storage Engineering Database Hardware 88

Scalegrid

MARCH 28, 2024

This article will explore how they handle data storage and scalability, perform in different scenarios, and, most importantly, how these factors influence your choice. Redis Revealed: An Overview Redis, a renowned open-source, in-memory remote dictionary server, stands out for its diverse data structures and advanced features.

Cache 130

Cache Storage Scalability Architecture 130

Dynatrace

JUNE 6, 2023

Kubernetes has become the leading container orchestration platform for organizations adopting open source solutions to manage, scale, and automate application deployment. Kubernetes is an open source container orchestration platform for managing, automating, and scaling containerized applications. What is Kubernetes?

Open Source 192

Open Source Storage Analytics Innovation 192

The Netflix TechBlog

NOVEMBER 22, 2019

4:45pm-5:45pm NFX 209 File system as a service at Netflix Kishore Kasi , Senior Software Engineer Abstract : As Netflix grows in original content creation, its need for storage is also increasing at a rapid pace. This talk explores the journey, learnings, and improvements to performance analysis, efficiency, reliability, and security.

AWS 37

AWS Entertainment Open Source Benchmarking 37

The Netflix TechBlog

MAY 21, 2019

There are several problems efficient test scheduling could help us solve: Quickly detect a regression in the integration of the Netflix SDK on a consumer electronic or MVPD (multichannel video programming distributor) device. Lerner, as a web-service, relies on Amazon Web Services (AWS) and Netflix’s Open Source Software (OSS) tools.

Testing 163

Testing AWS Lambda Network 163