The Netflix TechBlog

NOVEMBER 15, 2022

Rachel Kelley (AWS), Ranjit Raju (AWS) Rendering is core to the the VFX process VFX studios around the world create amazing imagery for Netflix productions. Rendering on AWS provides the flexibility to control how quickly a project is completed. By: Peter Cioni (Netflix), Alex Schworer (Netflix), Mac Moore (Conductor Tech.),

Cloud 282

Cloud Entertainment AWS Infrastructure 282

Scalegrid

JANUARY 8, 2024

Real-world examples like Spotify’s multi-cloud strategy for cost reduction and performance, and Netflix’s hybrid cloud setup for efficient content streaming and creation, illustrate the practical applications of each model. In this scenario, two notable models – multi-cloud and hybrid cloud have emerged. But what do these entail?

Strategy 147

Strategy Cloud Scalability Artificial Intelligence 147

Percona

JULY 1, 2023

Now that Database-as-a-service (DBaaS) is in high demand, there are multiple questions regarding AWS services that cannot always be answered easily: When should I use Aurora and when should I use RDS MySQL ? Amazon Aurora is a proprietary, cloud-native, fully managed relational database service developed by Amazon Web Services (AWS).

AWS 52

AWS Database Serverless Storage 52

Percona

FEBRUARY 24, 2023

Figure 1: PMM Home Dashboard From the Amazon Web Services (AWS) documentation , an instance is considered over-provisioned when at least one specification of your instance, such as CPU, memory, or network, can be sized down while still meeting the performance requirements of your workload and no specification is under-provisioned.

Cloud 123

Cloud Serverless AWS Storage 123

The Netflix TechBlog

MARCH 8, 2021

They need specialized hardware, access to petabytes of images, and digital content creation applications with controlled licenses. Below is a broad technical overview of how to go from an AWS instance to a Netflix Workstation. They could need a GPU when doing graphics-intensive work or extra large storage to handle file management.

Entertainment 306

Entertainment Storage Hardware Open Source 306

Dynatrace

AUGUST 11, 2022

Cloud providers then manage physical hardware, virtual machines, and web server software management. Cloud providers such as Google, Amazon Web Services, and Microsoft also followed suit with frameworks such as Google Cloud Functions , AWS Lambda , and Microsoft Azure Functions. How does function as a service work?

Development 247

Development Serverless Best Practices Lambda 247

Dynatrace

NOVEMBER 3, 2022

Serverless container offerings such as AWS Fargate enable companies to manage and modify containers while abstracting server layers to offer customization without increased complexity. IaaS provides direct access to compute resources such as servers, storage, and networks. In FaaS environments, providers manage all the hardware.

Serverless 216

Serverless Azure Hardware Transportation 216

Dynatrace

JULY 31, 2020

Dynatrace Managed is the turn-key solution for organizations that want to enjoy all the conveniences of a SaaS solution—for example, ease-of-use, no operational overhead, and fast release cycles—while keeping their data on-premise to comply with regulatory requirements. Dynatrace Managed now available on the Google Cloud Platform.

Cloud 206

Cloud Availability Azure Google 206

Scalegrid

MAY 13, 2020

Greenplum Database is an open-source , hardware-agnostic MPP database for analytics, based on PostgreSQL and developed by Pivotal who was later acquired by VMware. High performance, query optimization, open source and polymorphic data storage are the major Greenplum advantages. Polymorphic Data Storage. Greenplum Advantages.

Big Data 321

Big Data Database Artificial Intelligence Open Source 321

Dynatrace

NOVEMBER 15, 2021

Driving this growth is the increasing adoption of hyperscale cloud providers (AWS, Azure, and GCP) and containerized microservices running on Kubernetes. Logs can include data about user inputs, system processes, and hardware states. For example: Lack of end-to-end observability. billion in 2020 to $4.1 What is log analytics?

Analytics 205

Analytics Monitoring Artificial Intelligence DevOps 205

All Things Distributed

MARCH 11, 2016

The epoch of AWS is the launch of Amazon S3 on March 14, 2006, now almost 10 years ago. Given that AWS is a pioneer in building and operating these services world-wide, these lessons have been of crucial importance to our business. This is a given, whether you are using the highest quality hardware or lowest cost components.

AWS 144

AWS Hardware Retail Virtualization 144

High Scalability

OCTOBER 24, 2022

Each cloud-native evolution is about using the hardware more efficiently. Examples of traditional natural monopolies are dams, railroads, power plants, water supply, and sewer services. For example, AWS created Nitro. Would Nitro have been invented if AWS was restricted to being a platform provider?

Cloud 187

Cloud Hardware Infrastructure Innovation 187

All Things Distributed

OCTOBER 7, 2015

In AWS’ quest to enable the best data storage options for engineers, we have built several innovative database solutions like Amazon RDS, Amazon RDS for Aurora, Amazon DynamoDB, and Amazon Redshift. However, the data infrastructure to collect, store and process data is geared toward developers (e.g., Big data challenges.

Cloud 137

Cloud Big Data AWS Analytics 137

KeyCDN

FEBRUARY 22, 2023

The goal of WebAssembly is to execute at native speeds by taking advantage of common hardware features available on a variety of platforms. With cloud-based infrastructure, organizations can easily scale their web applications to handle increased traffic or demand without the need for expensive hardware upgrades.

Artificial Intelligence 111

Artificial Intelligence Development Serverless Website 111

The Netflix TechBlog

FEBRUARY 9, 2021

This requires an asset storage solution. Asset Storage We refer to asset storage and management simply as asset management. We can leverage high performance VMs in AWS to generate the assets. To support the demands of our stakeholders we designed a Domain Specific Language (DSL) that we call an asset generation recipe.

Engineering 165

Engineering Storage Latency Entertainment 165

Percona

JULY 26, 2023

MongoDB is a non-relational document database that provides support for JSON-like storage. For example, MongoDB 5.0 To make this process work more efficiently and ensure a smooth failover, it is important to have the same hardware configuration on all the nodes of the replica set.

Database 83

Database Servers Storage Strategy 83

Percona

APRIL 17, 2023

For example, on a 128GB memory host, this can allow up to 38.4GB of dirty pages. It is possible to remount on the fly; For example, in order to remount /mnt/db/ with these options: mount -oremount,rw,noatime/mnt/db It is necessary to add/edit the corresponding line in /etc/fstab for the option to persist on reboots.

Best Practices 52

Best Practices Design Tuning Database 52

The Morning Paper

MARCH 3, 2020

It takes you through the thinking processes and engineering practices behind the design of a key part of the control plane for AWS Elastic Block Storage (EBS): the Physalia database that stores configuration information. For Physalia, and for AWS more generally, the guiding principle is minimise the blast radius. NSDI’20.

Database 106

Database AWS Network Design 106

All Things Distributed

OCTOBER 8, 2014

The best part is that we are also significantly expanding the free tier many of you already enjoy by increasing the storage to 25 GB and throughput to 200 million requests per month. More than a decade ago, Amazon embarked on a mission to build a distributed system that challenged conventional methods of data storage and querying.

Availability 145

Availability Performance Scalability AWS 145

The Morning Paper

MAY 28, 2019

We don’t talk about energy as often as we probably should on this blog, but it’s certainly true that our data centres and various IT systems consume an awful lot of it. The goal is to produce a low-energy hardware classifier for embedded applications doing local processing of sensor data. accuracy, but now at just 7.8nJ/prediction.

Energy 52

Energy Hardware Efficiency Architecture 52

HammerDB

JANUARY 5, 2019

Some opinions claim that “Benchmarks are meaningless”, “benchmarks are irrelevant” or “benchmarks are nothing like your real applications” However for others “Benchmarks matter,” as they “account for the processing architecture and speed, memory, storage subsystems and the database engine.”

Benchmarking 40

Benchmarking Database Social Media Scalability 40

All Things Distributed

NOVEMBER 15, 2010

For example, the most fundamental abstraction trade-off has always been latency versus throughput. The early GPU systems were very vendor specific and mostly consisted of graphic operators implemented in hardware being able to operate on data streams in parallel. Also more details can be found on the AWS Developer blog.

AWS 136

AWS Latency Programming Architecture 136

All Things Distributed

JANUARY 18, 2012

Amazon DynamoDB stores data on Solid State Drives (SSDs) and replicates it synchronously across multiple AWS Availability Zones in an AWS Region to provide built-in high availability and data durability. Customers can typically achieve average service-side in the single-digit milliseconds. History of NoSQL at Amazon â??

Scalability 167

Scalability Database Ecommerce Latency 167

Brendan Gregg

SEPTEMBER 5, 2017

Linux has been adding tracing technologies over the years: kprobes (kernel dynamic tracing), uprobes (user-level dynamic tracing), tracepoints (static tracing), and perf_events (profiling and hardware counters). There are some extreme cases, where a lack of proper driver support on Solaris can have Linux run 10x faster. So there'll be 4.14-rc1

Virtualization 76

Virtualization AWS Engineering Hardware 76

SQL Performance

JANUARY 22, 2020

To demonstrate the potential performance impacts of incorrectly typed table-valued parameters with SQL Server, we are going to create an example user-defined table type with the following structure: CREATE TYPE dbo. PharmacyData AS TABLE. (

Servers 55

Servers Azure Hardware Network 55

Dynatrace

AUGUST 5, 2019

In this role, I am leading a global team that works closely with our strategic partners such as AWS, Microsoft, Google, Pivotal, Red Hat and others. Lift & Shift is where you basically just move physical or virtual hosts to the cloud – essentially you just run your host on somebody else’s hardware. These examples include e.g:

Cloud 31

Cloud Traffic Database Network 31

Adrian Cockcroft

NOVEMBER 11, 2019

This discussion focuses on hardware, software and operational failure modes. Systems that have an identifiable capacity trend, for example are filling up disk space at a predictable rate, have a “time to live” (TTL) that can be calculated. This is why most AWS regions have three availability zones. Drift into Failure ].

Latency 52

Latency Engineering Systems Hardware 52

Adrian Cockcroft

NOVEMBER 11, 2019

This discussion focuses on hardware, software and operational failure modes. Systems that have an identifiable capacity trend, for example are filling up disk space at a predictable rate, have a “time to live” (TTL) that can be calculated. This is why most AWS regions have three availability zones. Drift into Failure ].

Latency 53

Latency Engineering Systems Hardware 53

Percona

JANUARY 17, 2019

On multi-core machines – which is the majority of the hardware nowadays – and in the cloud, we have multiple cores available for use. AWS Aurora (based on MySQL 5.6) now has a version which will support parallelism for SELECT queries (utilizing the read capacity of storage nodes underneath the Aurora cluster).

Cache 48

Cache C++ AWS Airlines 48

High Scalability

NOVEMBER 25, 2019

It is limited by the disk space; it can’t expand storage elastically; it chokes if you run few I/O intensive processes or try collaborating with 100 other users. Egnyte is a secure Content Collaboration and Data Governance platform, founded in 2007 when Google drive wasn't born and AWS S3 was cost-prohibitive. Recommendations.

Architecture 52

Architecture Cache Azure Storage 52

Smashing Magazine

JANUARY 6, 2020

Run performance experiments and measure outcomes — both on mobile and on desktop (for example, with Google Analytics ). Rather than focusing on full page loading time (via onLoad and DOMContentLoaded timings, for example), prioritize page loading as perceived by your customers. What does it mean? Large preview ).

Performance 145

Performance Cache Network Metrics 145

Smashing Magazine

JANUARY 11, 2021

For example, to make performance more tangible, you could expose the revenue performance impact by showing the correlation between the conversion rate and time to application load, as well as rendering performance. Run performance experiments and measure outcomes — both on mobile and on desktop (for example, with Google Analytics ).

Performance 145

Performance Cache Media Metrics 145

Smashing Magazine

JANUARY 7, 2019

One way or another, rather than focusing on full page loading time (via onLoad and DOMContentLoaded timings, for example), prioritize page loading as perceived by your customers. On the other hand, we have hardware constraints on memory and CPU due to JavaScript parsing times (we’ll talk about them in detail later).

Performance 103

Performance Cache Metrics Network 103