The Netflix TechBlog

MAY 21, 2019

We thought that reinforcement learning would be a promising approach that could provide great flexibility in the training process. Likewise it has very low requirements on the initial amount of training data. We needed a way to run agents and tie the runs to the internal infrastructure for analytics, reporting, and visualizations.

Testing 163

Testing AWS Lambda Network 163

Percona

DECEMBER 27, 2023

Oftentimes, it is a pillar of modern infrastructure strategy to avoid cloud vendor lock-in. Standardization and collaboration are key to sharing common knowledge and patterns across teams and infrastructures. Nearly every vendor at Kubecon and every person we spoke to had some form of a multi-cloud requirement or strategy.

Cloud 80

Cloud Open Source Strategy Best Practices 80

Percona

JUNE 22, 2023

As such, fault tolerance is more expensive to implement because it requires dedicated infrastructure that completely mirrors the primary system. Components of high availability infrastructure Multiple copies of data Data redundancy helps prevent data loss due to hardware or software failures.

Availability 108

Availability Database Open Source Hardware 108

Percona

OCTOBER 18, 2023

Flexibility and scalability Open source databases provide much greater flexibility regarding customization and configuration. Are you looking to enhance performance, improve scalability, cut expenses, or gain access to specific features you don’t currently have? Does anyone on my team require further training before we start?

Open Source 93

Open Source Database Hardware Strategy 93

IO River

NOVEMBER 2, 2023

The code is declarative, which means you specify what you want the final configuration to look like, and Terraform ensures the infrastructure matches that desired state.‍So, ‍Terraform is a revolution in the way we handle infrastructure. Background‍The idea of managing infrastructure through code wasn't initiated by HashiCorp.

Open Source 52

Open Source DevOps AWS Azure 52

Percona

APRIL 10, 2023

It is the most stable, scalable, and secure open-source MySQL distribution, with two download options: one based on Percona Server for MySQL and one based on Percona XtraDB Cluster. It is an enhanced, source-available, and highly-scalable database that is a fully-compatible, drop-in replacement for MongoDB 6.0.5 Community Edition.

Open Source 78

Open Source Servers AWS Database 78

Testsigma

OCTOBER 16, 2020

As cloud-based frameworks require no expertise, installation time or effort and do not depend on a robust internal infrastructure, your tests can be up and running in just a matter of a few minutes. With SaaS, you don’t need to invest in internal infrastructure. Ensuring whether the platform is reliable and scalable 5.

Best Practices 81

Best Practices Virtualization Cloud Architecture 81

IO River

NOVEMBER 2, 2023

The code is declarative, which means you specify what you want the final configuration to look like, and Terraform ensures the infrastructure matches that desired state.‍So, What Exactly is Terraform?‍Terraform Terraform is a revolution in the way we handle infrastructure. This level of granularity is not found in UI-managed systems.

Open Source 40

Open Source DevOps AWS Azure 40

The Netflix TechBlog

NOVEMBER 6, 2023

In order to train the model on internal training data (video clips with aligned text descriptions), we implemented a scalable version on Ray Train and switched to a more performant video decoding library. These models are trained on large amounts of image-caption pairs via in-batch contrastive learning.

Media 225

Media Social Media Tuning Efficiency 225

The Netflix TechBlog

FEBRUARY 13, 2023

Our goal in building a media-focused ML infrastructure is to reduce the time from ideation to productization for our media ML practitioners. Media Feature Storage: Amber Storage Media feature computation tends to be expensive and time-consuming. We accomplish this by paving the path to: Accessing and processing media data (e.g.

Media 290

Media Storage Infrastructure Systems 290

Brendan Gregg

APRIL 30, 2018

LISA originally stood for "Large Installation System Administration," where "large" meant systems with more than a gigabyte of storage, or with more than 100 users. Submit #lisa18 talk, training, panel, and demo proposals by May 24 [link]. Have something to say on the present & future of #ops? Hope to see you in Nashville!

DevOps 43

DevOps Network Best Practices Programming 43

Brendan Gregg

APRIL 29, 2018

LISA originally stood for "Large Installation System Administration," where "large" meant systems with more than a gigabyte of storage, or with more than 100 users. Submit #lisa18 talk, training, panel, and demo proposals by May 24 [link]. Have something to say on the present & future of #ops? Hope to see you in Nashville!

DevOps 40

DevOps Network Best Practices Programming 40

The Netflix TechBlog

APRIL 26, 2022

We built Axion primarily to remove any training-serving skew and make offline experimentation faster. We make sure there is no training/serving skew by using the same data and the code for online and offline feature generation. Our machine learning models train on several weeks of data.

Storage 187

Storage Design Scalability Latency 187

The Netflix TechBlog

DECEMBER 3, 2019

About two years ago, we, at our newly formed Machine Learning Infrastructure team started asking our data scientists a question: “What is the hardest thing for you as a data scientist at Netflix?” Our job as a Machine Learning Infrastructure team would therefore not be mainly about enabling new technical feats.

Open Source 232

Open Source AWS Infrastructure Energy 232

The Netflix TechBlog

DECEMBER 3, 2019

About two years ago, we, at our newly formed Machine Learning Infrastructure team started asking our data scientists a question: “What is the hardest thing for you as a data scientist at Netflix?” Our job as a Machine Learning Infrastructure team would therefore not be mainly about enabling new technical feats.

Open Source 134

Open Source AWS Infrastructure Energy 134

High Scalability

OCTOBER 24, 2022

Cloud Infrastructure Services -- An analysis of potentially anti-competitive practices by Professor Frédéric Jenny. There is no structural separation as there is with a train service. What some consider infrastructure or platform is just another cloud service. A group advocating for fair licensing. It's a choice.

Cloud 187

Cloud Hardware Infrastructure Innovation 187

All Things Distributed

DECEMBER 13, 2016

In November 2015, Amazon Web Services announced that it would launch a new AWS infrastructure region in the United Kingdom. Today, I'm happy to announce that the AWS Europe (London) Region, our 16th technology infrastructure region globally, is now generally available for use by customers worldwide.

AWS 166

AWS Cloud Artificial Intelligence IoT 166

All Things Distributed

NOVEMBER 29, 2017

Though the AWS Cloud gives you access to the storage and processing power required for ML, the process for building, training, and deploying ML models has unique challenges that often block successful use of this powerful new technology. The challenges begin with collecting, cleaning, and formatting training data.

Tuning 94

Tuning AWS Scalability Infrastructure 94

High Scalability

NOVEMBER 25, 2019

It is limited by the disk space; it can’t expand storage elastically; it chokes if you run few I/O intensive processes or try collaborating with 100 other users. Over time, costs for S3 and GCS became reasonable and with Egnyte’s storage plugin architecture, our customers can now bring in any storage backend of their choice.

Architecture 52

Architecture Cache Azure Storage 52