The Netflix TechBlog

NOVEMBER 15, 2022

Rachel Kelley (AWS), Ranjit Raju (AWS) Rendering is core to the the VFX process VFX studios around the world create amazing imagery for Netflix productions. Every VFX studio has a slightly different architecture and workflow, and a one-size-fits-all solution often isn’t enough to bridge the gap.

Cloud 282

Cloud Entertainment AWS Infrastructure 282

All Things Distributed

APRIL 5, 2012

Empowering innovation is at the heart of everything we do at Amazon Web Services (AWS). I often get to meet, discuss, and learn from innovators how they are using AWS to deliver transformative applications to their users, customers and partners. Troy: We moved our service from internal servers to AWS. And how about you Troy?

Innovation 146

Innovation AWS Education Network 146

Brendan Gregg

FEBRUARY 28, 2023

Make sure your system can handle next-generation DRAM,” [link] , Nov 2011 [Hruska 12] Joel Hruska, “The future of CPU scaling: Exploring options on the cutting edge,” [link] , Feb 2012 [Gregg 13] Brendan Gregg, “Blazing Performance with Flame Graphs,” [link] , 2013 [Shimpi 13] Anand Lal Shimpi, “Seagate to Ship 5TB HDD in 2014 using Shingled Magnetic (..)

Performance 52

Performance Latency Cache Virtualization 52

Adrian Cockcroft

JUNE 21, 2022

I became the Sun UK local specialist in performance and hardware, and as Sun transitioned from a desktop workstation company to sell high end multiprocessor servers I was helping customers find and fix scalability problems. We had specializations in hardware, operating systems, databases, graphics, etc. He hasn’t changed.

Tuning 52

Tuning Benchmarking Engineering C++ 52

Brendan Gregg

NOVEMBER 29, 2017

Hardware virtualization for cloud computing has come a long way, improving performance using technologies such as VT-x, SR-IOV, VT-d, NVMe, and APICv. At Netflix, we've been using these technologies as they've been made available for instance types in the AWS EC2 cloud. I'd expect between 0.1%

Virtualization 111

Virtualization AWS Hardware Storage 111

Scalegrid

JANUARY 8, 2024

Understanding Multi-Cloud and Hybrid Cloud Cloud computing has revolutionized the IT industry, offering a host of advantages including cost-effectiveness, increased agility, and access to cutting-edge hardware. In this scenario, two notable models – multi-cloud and hybrid cloud have emerged. But what do these entail?

Strategy 147

Strategy Cloud Scalability Artificial Intelligence 147

Percona

JULY 1, 2023

Now that Database-as-a-service (DBaaS) is in high demand, there are multiple questions regarding AWS services that cannot always be answered easily: When should I use Aurora and when should I use RDS MySQL ? Amazon Aurora is a proprietary, cloud-native, fully managed relational database service developed by Amazon Web Services (AWS).

AWS 52

AWS Database Serverless Storage 52

Dynatrace

AUGUST 11, 2022

Cloud providers then manage physical hardware, virtual machines, and web server software management. Cloud providers such as Google, Amazon Web Services, and Microsoft also followed suit with frameworks such as Google Cloud Functions , AWS Lambda , and Microsoft Azure Functions. FaaS vs. monolithic architectures.

Development 243

Development Serverless Best Practices Lambda 243

Scalegrid

MAY 13, 2020

In this blog post, we explain what Greenplum is, and break down the Greenplum architecture, advantages, major use cases, and how to get started. Greenplum Database is an open-source , hardware-agnostic MPP database for analytics, based on PostgreSQL and developed by Pivotal who was later acquired by VMware. The Greenplum Architecture.

Big Data 321

Big Data Database Artificial Intelligence Open Source 321

Dynatrace

DECEMBER 8, 2021

You may be using serverless functions like AWS Lambda , Azure Functions , or Google Cloud Functions, or a container management service, such as Kubernetes. In contrast to modern software architecture, which uses distributed microservices, organizations historically structured their applications in a pattern known as “monolithic.”

Cloud 220

Cloud Serverless Hardware Operating System 220

Brendan Gregg

NOVEMBER 28, 2017

Hardware virtualization for cloud computing has come a long way, improving performance using technologies such as VT-x, SR-IOV, VT-d, NVMe, and APICv. At Netflix, we've been using these technologies as they've been made available for instance types in the AWS EC2 cloud. I'd expect between 0.1%

Virtualization 40

Virtualization AWS Hardware Storage 40

High Scalability

OCTOBER 24, 2022

Each cloud-native evolution is about using the hardware more efficiently. You can even go old school and use non cloud-native architectures. For example, AWS created Nitro. Nitro is a revolutionary combination of purpose-built hardware and software designed to provide performance and security. It has been done.

Cloud 187

Cloud Hardware Infrastructure Innovation 187

Dynatrace

NOVEMBER 15, 2021

Log monitoring, log analysis, and log analytics are more important than ever as organizations adopt more cloud-native technologies, containers, and microservices-based architectures. Driving this growth is the increasing adoption of hyperscale cloud providers (AWS, Azure, and GCP) and containerized microservices running on Kubernetes.

Analytics 202

Analytics Monitoring Artificial Intelligence DevOps 202

All Things Distributed

MARCH 11, 2016

The epoch of AWS is the launch of Amazon S3 on March 14, 2006, now almost 10 years ago. Given that AWS is a pioneer in building and operating these services world-wide, these lessons have been of crucial importance to our business. This is a given, whether you are using the highest quality hardware or lowest cost components.

AWS 144

AWS Hardware Retail Virtualization 144

KeyCDN

FEBRUARY 22, 2023

The goal of WebAssembly is to execute at native speeds by taking advantage of common hardware features available on a variety of platforms. With cloud-based infrastructure, organizations can easily scale their web applications to handle increased traffic or demand without the need for expensive hardware upgrades.

Artificial Intelligence 111

Artificial Intelligence Development Serverless Website 111

The Netflix TechBlog

FEBRUARY 9, 2021

This requires an asset storage solution. Asset Storage We refer to asset storage and management simply as asset management. We can leverage high performance VMs in AWS to generate the assets. Here’s what the final architecture looked like. First, asset generation is CPU intensive and bursty.

Engineering 165

Engineering Storage Latency Entertainment 165

Percona

JULY 26, 2023

MongoDB is a non-relational document database that provides support for JSON-like storage. To make this process work more efficiently and ensure a smooth failover, it is important to have the same hardware configuration on all the nodes of the replica set. MongoDB is popular with developers as it is easy to get started with.

Database 81

Database Servers Storage Strategy 81

All Things Distributed

OCTOBER 8, 2014

The best part is that we are also significantly expanding the free tier many of you already enjoy by increasing the storage to 25 GB and throughput to 200 million requests per month. More than a decade ago, Amazon embarked on a mission to build a distributed system that challenged conventional methods of data storage and querying.

Availability 145

Availability Performance Scalability AWS 145

All Things Distributed

NOVEMBER 15, 2010

Building general purpose architectures has always been hard; there are often so many conflicting requirements that you cannot derive an architecture that will serve all, so we have often ended up focusing on one side of the requirements that allow you to serve that area really well. From CPU to GPU. General Purpose GPU programming.

AWS 136

AWS Latency Programming Architecture 136

Percona

APRIL 17, 2023

The swap issue is explained in the excellent article by Jeremy Cole at the Swap Insanity and NUMA Architecture. For example: Read/Write tickets WiredTiger uses tickets to control the number of read / write operations simultaneously processed by the storage engine. The report focuses on MySQL, but it is valid for MongoDB as well.

Best Practices 52

Best Practices Design Tuning Database 52

HammerDB

JANUARY 5, 2019

Some opinions claim that “Benchmarks are meaningless”, “benchmarks are irrelevant” or “benchmarks are nothing like your real applications” However for others “Benchmarks matter,” as they “account for the processing architecture and speed, memory, storage subsystems and the database engine.”

Benchmarking 40

Benchmarking Database Social Media Scalability 40

All Things Distributed

JANUARY 18, 2012

Amazon DynamoDB stores data on Solid State Drives (SSDs) and replicates it synchronously across multiple AWS Availability Zones in an AWS Region to provide built-in high availability and data durability. These architectural discussions culminated in Amazon DynamoDB, a new NoSQL service that we are excited to release today.

Scalability 167

Scalability Database Ecommerce Latency 167

The Morning Paper

MAY 28, 2019

We don’t talk about energy as often as we probably should on this blog, but it’s certainly true that our data centres and various IT systems consume an awful lot of it. The goal is to produce a low-energy hardware classifier for embedded applications doing local processing of sensor data. An end-to-end architecture.

Energy 52

Energy Hardware Efficiency Architecture 52

Percona

NOVEMBER 3, 2023

PostgreSQL Cluster One coordinator node citus-coord-01 Three worker nodes citus1 citus2 citus3 Hardware AWS Instance Ubuntu Server 20.04, SSD volume type 64-bit (x86) c5.xlarge A future blog will continue my exploration into Citus by scaling out pgbench into other architectures.

Database 106

Database Benchmarking Latency C++ 106

High Scalability

NOVEMBER 25, 2019

It is limited by the disk space; it can’t expand storage elastically; it chokes if you run few I/O intensive processes or try collaborating with 100 other users. Egnyte is a secure Content Collaboration and Data Governance platform, founded in 2007 when Google drive wasn't born and AWS S3 was cost-prohibitive. Offline access.

Architecture 52

Architecture Cache Azure Storage 52

Smashing Magazine

JANUARY 6, 2020

On the other hand, we have hardware constraints on memory and CPU due to JavaScript parsing times (we’ll talk about them in detail later). Still, after all these years, keeping progressive enhancement as the guiding principle of your front-end architecture and deployment is a safe bet. Use progressive enhancement as a default.

Performance 145

Performance Cache Network Metrics 145

Smashing Magazine

JANUARY 11, 2021

Defining The Environment Choosing a framework, baseline performance cost, Webpack, dependencies, CDN, front-end architecture, CSR, SSR, CSR + SSR, static rendering, prerendering, PRPL pattern. Still, after all these years, keeping progressive enhancement as the guiding principle of your front-end architecture and deployment is a safe bet.

Performance 145

Performance Cache Media Metrics 145

Smashing Magazine

JANUARY 7, 2019

On the other hand, we have hardware constraints on memory and CPU due to JavaScript parsing times (we’ll talk about them in detail later). Keeping progressive enhancement as the guiding principle of your front-end architecture and deployment is a safe bet. Consider using PRPL pattern and app shell architecture.

Performance 103

Performance Cache Metrics Network 103