The Netflix TechBlog

MARCH 6, 2019

Mounting object storage in Netflix’s media processing platform By Barak Alon (on behalf of Netflix’s Media Cloud Engineering team) MezzFS (short for “Mezzanine File System”) is a tool we’ve developed at Netflix that mounts cloud objects as local files via FUSE. Netflix operates in multiple AWS regions. Regional caching? —?Netflix

Media 214

Media Storage Processing Cache 214

Dynatrace

OCTOBER 6, 2023

Buckets are similar to folders, a physical storage location. Debug-level logs, which also generate high volumes and have a shorter lifespan or value period than other logs, could similarly benefit from dedicated storage. Suppose a single Grail environment is central storage for pre-production and production systems.

Artificial Intelligence 266

Artificial Intelligence Analytics Metrics Storage 266

Adrian Cockcroft

NOVEMBER 26, 2023

The Pantheon in Rome — Extremely sustainable architecture — photo by Adrian I wrote a medium post after AWS re:Invent 2022 summarizing the (lack of) news and all the talks related to Sustainability. This includes providing the efficient, resilient services AWS customers expect, while minimizing their environmental footprint.

AWS 52

AWS Energy IoT Best Practices 52

The Netflix TechBlog

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! To sustain this data growth at Netflix, it has deployed open-source software Ceph using AWS services to achieve the required SLOs of some of the post-production workflows.

AWS 100

AWS Entertainment Open Source Benchmarking 100

The Netflix TechBlog

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! To sustain this data growth at Netflix, it has deployed open-source software Ceph using AWS services to achieve the required SLOs of some of the post-production workflows.

AWS 100

AWS Entertainment Open Source Benchmarking 100

Dynatrace

SEPTEMBER 17, 2019

What we need though is data that is automatically connected with context for k8s administrators, architects and developers to better understand what is going in the part of k8s that’s relevant to them and how does it impact those parts that are relevant for their colleagues. PostgreSQL & Elastic for data storage. REDIS for caching.

AWS 128

AWS Monitoring Ecommerce Lambda 128

Dynatrace

MAY 26, 2021

The second focused on the OTel community, with more technical talks by representatives from companies like AWS, Elastic, and more. To improve their alerting system, the team decided to move to SLO/SLI-based alerting, which require deeper knowledge of the application but are more helpful than generalized metric alerts.

AWS 218

AWS Strategy Storage Metrics 218

Percona

JULY 1, 2023

Now that Database-as-a-service (DBaaS) is in high demand, there are multiple questions regarding AWS services that cannot always be answered easily: When should I use Aurora and when should I use RDS MySQL ? Amazon Aurora is a proprietary, cloud-native, fully managed relational database service developed by Amazon Web Services (AWS).

AWS 52

AWS Database Serverless Storage 52

Dynatrace

MAY 26, 2021

The second focused on the OTel community, with more technical talks by representatives from companies like AWS, Elastic, and more. To improve their alerting system, the team decided to move to SLO/SLI-based alerting, which require deeper knowledge of the application but are more helpful than generalized metric alerts.

AWS 147

AWS Strategy Storage Metrics 147

Dynatrace

AUGUST 24, 2020

Kubernetes simplifies the operation and development of distributed applications by streamlining the deployment of containerized workloads and distributing them over a set of nodes. Dynatrace AWS monitoring gives you an overview of the resources that are used in your AWS infrastructure along with their historical usage.

Analytics 246

Analytics Infrastructure AWS Operating System 246

All Things Distributed

JUNE 10, 2016

Building your applications with only managed components has become very popular, and AWS Lambda plays a crucial role in that. The Mobile Backend reference architecture demonstrates how to use AWS Lambda along with other services to build a serverless backend for a mobile application. about teletext.io are read eagerly around the globe.

Lambda 107

Lambda Serverless AWS Architecture 107

Dynatrace

OCTOBER 11, 2021

In a time when modern microservices are easier to deploy, GCF, like its counterparts AWS Lambda and Microsoft Azure Functions , gives development teams an agility boost for delivering value to their customers quickly with low overhead costs. GCF operates under the following three tenets: Simplify the developer experience.

Google 227

Google Cloud Serverless Azure 227

All Things Distributed

MAY 10, 2016

Building your applications with only managed components has become very popular, and AWS Lambda plays a crucial role in that. The Mobile Backend reference architecture demonstrates how to use AWS Lambda along with other services to build a serverless backend for a mobile application. about teletext.io are read eagerly around the globe.

Lambda 105

Lambda Serverless AWS Architecture 105

Dynatrace

AUGUST 21, 2023

How OpenTelemetry works Observability data is the stock-in-trade of OpenTelemetry: Logs, metrics, and traces. OpenTelemetry works by providing developers with APIs, SDKs, and tools to instrument their code and collect telemetry data such as logs, metrics, and traces. But one blind spot remained.

Open Source 202

Open Source Analytics Lambda Metrics 202

Scalegrid

MARCH 14, 2024

Public Cloud Infrastructure Third-party providers run public cloud services, delivering a broad array of offerings like computing power, storage solutions, and network capabilities that enhance the functionality of a hybrid cloud architecture. We will examine each of these elements in more detail.

Strategy 130

Strategy Cloud Artificial Intelligence Infrastructure 130

The Netflix TechBlog

OCTOBER 19, 2020

Our distributed tracing infrastructure is grouped into three sections: tracer library instrumentation, stream processing, and storage. We earned the trust of our engineers by developing empathy for their operational burden and by focusing on providing efficient tracer library integrations in runtime environments.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

Scalegrid

DECEMBER 16, 2019

It’s a cross-platform document-oriented database that uses JSON-like documents with schema, and is leveraged broadly across startup apps up to enterprise-level businesses developing modern apps. So, what makes DigitalOcean a compelling alternative to other popular MongoDB hosting providers like AWS and Azure ?

Azure 187

Azure AWS Latency Database 187

Dynatrace

NOVEMBER 15, 2021

Driving this growth is the increasing adoption of hyperscale cloud providers (AWS, Azure, and GCP) and containerized microservices running on Kubernetes. Log monitoring is a process by which developers and administrators continuously observe logs as they’re being recorded. billion in 2020 to $4.1 What is log monitoring?

Analytics 214

Analytics Monitoring Artificial Intelligence DevOps 214

Dynatrace

DECEMBER 8, 2021

You may be using serverless functions like AWS Lambda , Azure Functions , or Google Cloud Functions, or a container management service, such as Kubernetes. Modern operating systems provide capabilities to observe and report various metrics about the applications running. The last aspect is the centralization of compute.

Cloud 231

Cloud Serverless Hardware Operating System 231

Dynatrace

JANUARY 16, 2023

Most Kubernetes clusters in the cloud (73%) are built on top of managed distributions from the hyperscalers like AWS Elastic Kubernetes Service (EKS), Azure Kubernetes Service (AKS), or Google Kubernetes Engine (GKE). In general, metrics collectors and providers are most common, followed by log and tracing projects.

Open Source 347

Open Source Java Operating System Programming 347

Dynatrace

AUGUST 24, 2020

Kubernetes simplifies the operation and development of distributed applications by streamlining the deployment of containerized workloads and distributing them over a set of nodes. Metrics to find out how the behavior of a system has changed over time . Dynatrace news.

Analytics 135

Analytics Infrastructure AWS Operating System 135

Dynatrace

NOVEMBER 23, 2020

While speeding up development processes and reducing complexity does make the lives of Kubernetes operators easier, the inherent abstraction and automation can lead to new types of errors that are difficult to find, troubleshoot, and prevent. Configuring storage in Kubernetes is more complex than using a file system on your host.

Virtualization 162

Virtualization Infrastructure Monitoring Cloud 162

The Netflix TechBlog

JULY 30, 2019

and beyond By Anoop Panicker and Kishore Banala Conductor is a workflow orchestration engine developed and open-sourced by Netflix. Our cluster size ranges from 12–18 instances of AWS EC2 m4.4xlarge instances, typically running at ~30% capacity. *?—?Cassandra Cassandra persistence module is a partial implementation.

Lambda 189

Lambda Media Open Source Metrics 189

The Netflix TechBlog

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! To sustain this data growth at Netflix, it has deployed open-source software Ceph using AWS services to achieve the required SLOs of some of the post-production workflows.

AWS 37

AWS Entertainment Open Source Benchmarking 37

All Things Distributed

APRIL 9, 2015

Details on the AWS Blog. AWS has been offering a range of storage solutions: objects, block storage, databases, archiving, etc. When we designed Amazon EFS we decided to build along the AWS principles: Elastic, scalable, highly available, consistent performance, secure, and cost-effective. Details on the AWS Blog.

Lambda 122

Lambda Cloud IoT AWS 122

The Netflix TechBlog

OCTOBER 15, 2019

To address these challenges, we developed Delta. Thus, ensuring the atomicity of writes across different storage technologies remains a challenging problem for applications [3]. Delta Delta has been developed to address the limitations of existing solutions for data synchronization, and also allows to enrich data on the fly.

Transportation 192

Transportation Architecture Processing Storage 192

All Things Distributed

JUNE 21, 2018

The answer for me is simple: Developers want their applications to be well architected and scale effectively. The days of the one-size-fits-all monolithic database are behind us, and developers are now building highly distributed applications using a multitude of purpose-built databases. The opposite is true.

Database 167

Database AWS Games Latency 167

The Netflix TechBlog

MAY 21, 2019

Lerner, as a web-service, relies on Amazon Web Services (AWS) and Netflix’s Open Source Software (OSS) tools. Lerner uses AWS services to store binary versions of the agents, agent configurations, and training data. Lerner uses AWS services to store binary versions of the agents, agent configurations, and training data.

Testing 163

Testing AWS Lambda Network 163

The Morning Paper

FEBRUARY 6, 2020

The key ingredients of Cloudburst are a highly-scalable key-value store for persistent state ( Anna ), local caches co-located with function execution environments, and cache-consistency protocols to preserve developer sanity while data is moved in and out of those caches. Otherwise the scheduler picks an executor at random. Evaluation.

Lambda 98

Lambda Serverless Cache Latency 98

Adrian Cockcroft

APRIL 5, 2023

This proposal seeks to define a standard for real-time carbon and energy data as time-series data that would be accessed alongside and synchronized with the existing throughput, utilization and latency metrics that are provided for the components and applications in computing environments.

Energy 52

Energy Metrics Cloud Operating System 52

Percona

NOVEMBER 3, 2023

PostgreSQL Cluster One coordinator node citus-coord-01 Three worker nodes citus1 citus2 citus3 Hardware AWS Instance Ubuntu Server 20.04, SSD volume type 64-bit (x86) c5.xlarge psql pgbench <<_eof1_ qecho adding node citus3. select citus_add_node('citus3', 5432); qecho rebalancing shards across TWO nodes.

Database 101

Database Benchmarking Latency C++ 101

All Things Distributed

AUGUST 22, 2011

Today AWS has launched Amazon ElastiCache , a new service that makes it easy to add distributed in-memory caching to any application. Given the widespread use of caching in many of the applications in the AWS Cloud, a caching service had been high on the request list of our customers. Driving Storage Costs Down for AWS Customers.

Cloud 117

Cloud Cache AWS Storage 117

Percona

DECEMBER 20, 2018

With the Percona Database Performance Blog, Percona staff and leadership work hard to provide the open source community with insights, technical support, predictions and metrics around multiple open source database software technologies. MySQL stored procedures, functions and triggers are tempting constructs for application developers.

Database 40

Database Tuning Performance AWS 40

All Things Distributed

JANUARY 18, 2012

s Dynamo technology , which was one of the first non-relational databases developed at Amazon. With Amazon DynamoDB, developers scaling cloud-based applications can start small with just the capacity they need and then increase the request capacity of a given table as their app grows in popularity. History of NoSQL at Amazon â??

Scalability 167

Scalability Database Ecommerce Latency 167

High Scalability

NOVEMBER 25, 2019

It is limited by the disk space; it can’t expand storage elastically; it chokes if you run few I/O intensive processes or try collaborating with 100 other users. Egnyte is a secure Content Collaboration and Data Governance platform, founded in 2007 when Google drive wasn't born and AWS S3 was cost-prohibitive. Recommendations.

Architecture 52

Architecture Cache Azure Storage 52

HammerDB

JANUARY 5, 2019

Some opinions claim that “Benchmarks are meaningless”, “benchmarks are irrelevant” or “benchmarks are nothing like your real applications” However for others “Benchmarks matter,” as they “account for the processing architecture and speed, memory, storage subsystems and the database engine.”

Benchmarking 40

Benchmarking Database Social Media Scalability 40

The Morning Paper

MAY 28, 2019

We don’t talk about energy as often as we probably should on this blog, but it’s certainly true that our data centres and various IT systems consume an awful lot of it. So it’s interesting to see a paper using nano-Joules per prediction as an evaluation metric. One such possible representation is pure analog signalling.

Energy 52

Energy Hardware Efficiency Architecture 52