The Netflix TechBlog

DECEMBER 21, 2020

At this scale, we can gain a significant amount of performance and cost benefits by optimizing the storage layout (records, objects, partitions) as the data lands into our warehouse. We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits.

Storage 203

Storage Latency Efficiency Data Engineering 203

Adrian Cockcroft

NOVEMBER 26, 2023

The Pantheon in Rome — Extremely sustainable architecture — photo by Adrian I wrote a medium post after AWS re:Invent 2022 summarizing the (lack of) news and all the talks related to Sustainability. Explore an implementation of this architecture with PVH, the parent company of Tommy Hilfiger and Calvin Klein.

AWS 52

AWS Energy IoT Best Practices 52

HammerDB

MAY 5, 2019

Note that the main developer of HammerDB is Intel employee (#IAMINTEL) however HammerDB is a personal open source project and HammerDB has no optimization whatsoever for a database running on any particular architecture. in the directory storage/innobase/ut the following section has UT_RELAX_CPU defined to call the PAUSE instruction on x86.

Testing 48

Testing Tuning Latency Storage 48

Adrian Cockcroft

JUNE 21, 2022

I was mostly coding in C, tuning FORTRAN, and when I needed to do a lot of data analysis of benchmark results used the S-PLUS statistics language, that is the predecessor to R. At one point we had two conflicting high availability products from different teams, who weren’t cooperating. He had it up and running on Wednesday.

Tuning 52

Tuning Benchmarking Engineering C++ 52

Percona

JANUARY 19, 2023

It can help us to save costs on storage and backup times. While MySQL can handle large data sets, it is always recommended to keep only the used data in the databases, as this will make data access more efficient, and also will help to save costs on storage and backups. It is available under a paid subscription.

Open Source 91

Open Source Storage Database Big Data 91

The Morning Paper

JUNE 13, 2019

As we saw with the SOAP paper last time out, even with a fixed model variant and hardware there are a lot of different ways to map a training workload over the available hardware. Different hardware architectures (CPUs, GPUs, TPUs, FPGAs, ASICs, …) offer different performance and cost trade-offs.

Hardware 58

Hardware Latency Serverless Energy 58

The Netflix TechBlog

MARCH 14, 2023

Specifically, we will dive into the architecture that powers search capabilities for studio applications at Netflix. We must quickly surface the most stand-out highlights from the titles available on our service in the form of images and videos in the member experience. First, we must provide the content that will bring them joy.

Media 291

Media Innovation Energy Architecture 291

Dynatrace

JUNE 27, 2023

The increasing complexity of cloud service architectures requires a rock-solid understanding of the activity, health status, and security of cloud services. Logs complement out-of-the-box metrics and enable automated actions for responding to availability, security, and other service events.

AWS 197

AWS Cloud Lambda Analytics 197

Abhishek Tiwari

JULY 12, 2018

Data Pipeline A data pipeline is a software that ingests data from multiple sources, transforms it and finally makes it available to internal or external products. If tuned for performance, there is a good change reliability is compromised - and vice versa. In a nutshell, a data pipeline is a distributed system.

Latency 63

Latency Analytics Scalability Engineering 63

Dynatrace

FEBRUARY 16, 2023

Grail combines the big-data storage of a data warehouse with the analytical flexibility of a data lake. Now, that same full-spectrum value is available at the massive scale of the Dynatrace Grail data lakehouse. To solve this problem, Dynatrace launched Grail, its causational data lakehouse , in 2022.

Analytics 185

Analytics Innovation Metrics Database 185

Dynatrace

FEBRUARY 9, 2022

Cloud vendors such as Amazon Web Services (AWS), Microsoft, and Google provide a wide spectrum of serverless services for compute and event-driven workloads, databases, storage, messaging, and other purposes. AI-powered automation and deep, broad observability for serverless architectures. Stay tuned for updates. Dynatrace news.

Serverless 186

Serverless Azure Lambda AWS 186

The Netflix TechBlog

OCTOBER 27, 2020

At Netflix, we also heavily embrace a microservice architecture that emphasizes separation of concerns. The KV DAL allows applications to use a well-defined and storage engine agnostic HTTP/gRPC key-value data interface that in turn decouples applications from hard to maintain and backwards-incompatible datastore APIs.

Latency 243

Latency Storage Big Data Tuning 243

Percona

APRIL 7, 2023

If we were to select the most important MySQL setting, if we were given a freshly installed MySQL or Percona Server for MySQL and could only tune a single MySQL variable, which one would it be? MySQL comes pre-configured to be conservative instead of making the most of the resources available in the server. Why is that?

Tuning 138

Tuning Cache Servers Benchmarking 138

Dynatrace

DECEMBER 9, 2020

Cloud-native technologies and microservice architectures have shifted technical complexity from the source code of services to the interconnections between services. Heterogeneous cloud-native microservice architectures can lead to visibility gaps in distributed traces. Dynatrace news. What’s next?

Java 232

Java Traffic Strategy Architecture 232

The Netflix TechBlog

JUNE 13, 2023

By collecting and analyzing key performance metrics of the service over time, we can assess the impact of the new changes and determine if they meet the availability, latency, and performance requirements. They enable us to further fine-tune and configure the system, ensuring the new changes are integrated smoothly and seamlessly.

Traffic 279

Traffic Metrics Systems Strategy 279

Scalegrid

JUNE 4, 2020

On average, ScaleGrid provides over 30% more storage vs. DigitalOcean for PostgreSQL at the same affordable price. ScaleGrid for PostgreSQL is architectured to leverage-high performance SSD disks on DigitalOcean, and is finely tuned and optimized to achieve the best performance on DigitalOcean infrastructure. High Availability.

Database 230

Database Latency Benchmarking Performance 230

The Netflix TechBlog

NOVEMBER 2, 2021

The Reloaded system is a well-matured and scalable system, but its monolithic architecture can slow down rapid innovation. Here, based on the video length, the throughput and latency requirements, available scale etc., The quality results are now available to the caller via the getQuality endpoint. via bug fixes). 4c & 5.

Media 171

Media Innovation Metrics Latency 171

Dynatrace

JULY 6, 2022

Virtualization has revolutionized system administration by making it possible for software to manage systems, storage, and networks. With the self-service features and an everything-as-code architecture, labor requirements will significantly decrease and SRE best practices will emerge. Design, implement, and tune effective SLOs.

Best Practices 186

Best Practices Open Source Tuning Infrastructure 186

Dynatrace

NOVEMBER 15, 2021

Log monitoring, log analysis, and log analytics are more important than ever as organizations adopt more cloud-native technologies, containers, and microservices-based architectures. Log analysis can reveal potential bottlenecks and inefficient configurations so teams can fine-tune system performance. Dynatrace news. What are logs?

Analytics 208

Analytics Monitoring Artificial Intelligence DevOps 208

Percona

APRIL 17, 2023

The main objective of this post is to share my experience over the past years tuning MongoDB and centralize the diverse sources that I crossed in this journey in a unique place. There is an issue with this, which causes the OS to swap even with memory available. Spoiler alert: This post focuses on MongoDB 3.6.X 25.84 - Total 21.04

Best Practices 52

Best Practices Design Tuning Database 52

All Things Distributed

OCTOBER 8, 2014

The best part is that we are also significantly expanding the free tier many of you already enjoy by increasing the storage to 25 GB and throughput to 200 million requests per month. We designed DynamoDB to operate with at least 99.999% availability. In 2012, we launched Amazon DynamoDB, the successor to Amazon Dynamo.

Availability 145

Availability Performance Scalability AWS 145

All Things Distributed

MARCH 7, 2013

As I sat down with the DynamoDB team to review our progress over the last year, I realized that DynamoDB had surpassed even my own expectations for how easily applications could achieve massive scale and high availability with DynamoDB. This rapid adoption has allowed us to benefit from the scale economies inherent in our architecture.

Ecommerce 139

Ecommerce Storage Scalability Hardware 139

All Things Distributed

NOVEMBER 28, 2012

s architecture and underlying platform are also optimized to deliver high performance for data warehousing workloads. Redshift has a massively parallel processing (MPP) architecture, which enables it to distribute and parallelize queries across multiple low cost nodes. Amazon Redshiftâ??s Amazon Redshiftâ??s Parallelism isnâ??t

Cloud 164

Cloud Storage Architecture Scalability 164

SQL Performance

APRIL 27, 2020

The basic tier provides up to 5 DTUs with standard storage. The standard tier supports from 10 up to 3000 DTUs with standard storage and the premium tier supports 125 up to 4000 DTUs with premium storage, which is orders of magnitude faster than standard storage. vCore Pricing Tier. GB per vCore. HyperScale Database.

Azure 55

Azure Database Serverless Hardware 55

All Things Distributed

NOVEMBER 29, 2017

Though the AWS Cloud gives you access to the storage and processing power required for ML, the process for building, training, and deploying ML models has unique challenges that often block successful use of this powerful new technology. Today, we are announcing the general availability of Amazon SageMaker.

Tuning 94

Tuning AWS Scalability Infrastructure 94

Official Blog - World Web Technology

MAY 12, 2023

Stay tuned. The front-end of the website is what you see and interact on the display on the screen and the backend is the logical part of the app that has components such as storage, coding, content management, etc. When developers use the front-end and back-end together, it is called coupled CMS architecture.

Website 52

Website Development Scalability Architecture 52

HammerDB

JANUARY 5, 2019

Some opinions claim that “Benchmarks are meaningless”, “benchmarks are irrelevant” or “benchmarks are nothing like your real applications” However for others “Benchmarks matter,” as they “account for the processing architecture and speed, memory, storage subsystems and the database engine.”

Benchmarking 40

Benchmarking Database Social Media Scalability 40

All Things Distributed

MARCH 28, 2011

What used to be only available in physical formats now often has digital equivalents and this digitalization is driving great new innovations. A key part of the Cloud Drive architecture is a Metadata Service that allows customers to quickly search and organize their digital collections within Cloud Drive. Order in the Chaos.

AWS 138

AWS Cloud Storage Internet 138

HammerDB

OCTOBER 12, 2018

As is also the case this limitation is at the database level (especially the storage engine) rather than the hardware level. InnoDB is the storage engine that will deliver the best OLTP throughput and should be chosen for this test. . GHz available cpufreq governors: performance powersave . HammerDB difference from Sysbench.

Best Practices 40

Best Practices Scalability Performance C++ 40

Smashing Magazine

SEPTEMBER 27, 2019

All the code mentioned here is publicly available, and you can get to it whenever you want. There are two repositories available for you to play with: Frankenstein TodoMVC. It contains several branches, each of which represents a particular migration direction between applications, available in the first repository.

Storage 67

Storage Testing Processing Code 67

SQL Server According to Bob

JANUARY 11, 2020

Stable Media Stable media is often confused with physical storage. SQL Server defines stable media as storage that can survive system restart or common failure. Stable media is commonly physical disk storage, but other devices and certain caching facilities qualify as well.

Servers 40

Servers Cache Media Hardware 40

Percona

NOVEMBER 3, 2023

Rather than listing the concepts, function calls, etc, available in Citus, which frankly is a bit boring, I’m going to explore scaling out a database system starting with a single host. A future blog will continue my exploration into Citus by scaling out pgbench into other architectures.

Database 107

Database Benchmarking Latency C++ 107

High Scalability

NOVEMBER 25, 2019

It is limited by the disk space; it can’t expand storage elastically; it chokes if you run few I/O intensive processes or try collaborating with 100 other users. Over time, costs for S3 and GCS became reasonable and with Egnyte’s storage plugin architecture, our customers can now bring in any storage backend of their choice.

Architecture 52

Architecture Cache Azure Storage 52