The Netflix TechBlog

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! To sustain this data growth at Netflix, it has deployed open-source software Ceph using AWS services to achieve the required SLOs of some of the post-production workflows.

AWS 100

AWS Entertainment Open Source Benchmarking 100

The Netflix TechBlog

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! To sustain this data growth at Netflix, it has deployed open-source software Ceph using AWS services to achieve the required SLOs of some of the post-production workflows.

AWS 100

AWS Entertainment Open Source Benchmarking 100

All Things Distributed

NOVEMBER 12, 2018

Today, I'm happy to announce that the AWS GovCloud (US-East) Region, our 19th global infrastructure Region, is now available for use by customers in the US. With this launch, AWS now provides 57 Availability Zones, with another 12 zones and four Regions in Bahrain, Cape Town, Hong Kong SAR, and Stockholm expected to come online by 2020.

AWS 117

AWS Healthcare Cloud Government 117

Scalegrid

MARCH 14, 2024

Understanding Hybrid Cloud Strategy A hybrid cloud merges the capabilities of public and private clouds into a singular, coherent system. In practice, a hybrid cloud operates by melding resources and services from multiple computing environments, which necessitates effective coordination, orchestration, and integration to work efficiently.

Strategy 130

Strategy Cloud Artificial Intelligence Infrastructure 130

Scalegrid

JANUARY 8, 2024

This article delves into the specifics of how AI optimizes cloud efficiency, ensures scalability, and reinforces security, providing a glimpse at its transformative role without giving away extensive details. Using AI for Enhanced Cloud Operations The integration of AI in cloud computing is enhancing operational efficiency in several ways.

Artificial Intelligence 147

Artificial Intelligence Cloud Scalability Analytics 147

Adrian Cockcroft

MAY 6, 2023

Then they tried to scale it to cope with high traffic and discovered that some of the state transitions in their step functions were too frequent, and they had some overly chatty calls between AWS lambda functions and S3. His first edition in 2015 was foundational, and he updated it in 2021 with a second edition.

Serverless 145

Serverless Lambda Best Practices Traffic 145

Scalegrid

JANUARY 12, 2024

Simply put, it’s the set of computational tasks that cloud systems perform, such as hosting databases, enabling collaboration tools, or running compute-intensive algorithms. This article analyzes cloud workloads, delving into their forms, functions, and how they influence the cost and efficiency of your cloud infrastructure.

Cloud 130

Cloud Virtualization Storage Efficiency 130

Percona

JULY 1, 2023

Now that Database-as-a-service (DBaaS) is in high demand, there are multiple questions regarding AWS services that cannot always be answered easily: When should I use Aurora and when should I use RDS MySQL ? Amazon Aurora is a proprietary, cloud-native, fully managed relational database service developed by Amazon Web Services (AWS).

AWS 52

AWS Database Serverless Storage 52

The Netflix TechBlog

OCTOBER 19, 2020

which is difficult when troubleshooting distributed systems. If we had an ID for each streaming session then distributed tracing could easily reconstruct session failure by providing service topology, retry and error tags, and latency measurements for all service calls.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

Adrian Cockcroft

JANUARY 20, 2023

on Myths and Legends of High Performance Computing  — it’s a somewhat light-hearted look at some of the same issues by the leader of the team that built the Fugaku system I mention below. Jack Dongarra talked about the scores, and pointed out the low efficiency on some important workloads. Next generation architectures will use CXL3.0

Architecture 52

Architecture Latency Benchmarking AWS 52

Adrian Cockcroft

NOVEMBER 18, 2019

AWS Developer Relations on how the shift from Robot Operating System (ROS) 1 to ROS 2 will change the landscape for all robot lovers. OPN205-R Contributing to the AWS Construct Library (repeats) Using and loving the AWS Cloud Development Kit and want to help make it better? Alejandra Olvera-Novack?—?AWS

Open Source 40

Open Source AWS Lambda Serverless 40

Adrian Cockcroft

NOVEMBER 18, 2019

AWS Developer Relations on how the shift from Robot Operating System (ROS) 1 to ROS 2 will change the landscape for all robot lovers. OPN205-R Contributing to the AWS Construct Library (repeats) Using and loving the AWS Cloud Development Kit and want to help make it better? Alejandra Olvera-Novack?—?AWS

Open Source 40

Open Source AWS Lambda Serverless 40

Dynatrace

OCTOBER 1, 2021

As dynamic systems architectures increase in complexity and scale, IT teams face mounting pressure to track and respond to conditions and issues across their multi-cloud environments. An advanced observability solution can also be used to automate more processes, increasing efficiency and innovation among Ops and Apps teams.

Metrics 363

Metrics Open Source Monitoring Infrastructure 363

The Morning Paper

FEBRUARY 6, 2020

Given the simplicity and economic appeal of FaaS, it is interesting to explore designs that preserve the autoscaling and operational benefits of current offerings, while adding performant, cost-efficient and consistent shared state and communication. Oh, and there’s a scheduler too of course to keep all the plates spinning. Consistency.

Lambda 98

Lambda Serverless Cache Latency 98

The Netflix TechBlog

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! To sustain this data growth at Netflix, it has deployed open-source software Ceph using AWS services to achieve the required SLOs of some of the post-production workflows.

AWS 37

AWS Entertainment Open Source Benchmarking 37

Brendan Gregg

FEBRUARY 28, 2023

This talk originated from my updates to [Systems Performance 2nd Edition], and this was the first time I've given this talk in person! CXL in a way allows a custom memory controller to be added to a system, to increase memory capacity, bandwidth, and overall performance. Ford, et al., “TCP

Performance 110

Performance Latency Cache Virtualization 110

High Scalability

MARCH 22, 2019

µs of replication latency on lossy Ethernet, which is faster than or comparable to specialized replication systems that use programmable switches, FPGAs, or RDMA.". They'll learn a lot and love you even more.5 5 billion : weekly visits to Apple App store; $500m : new US exascale computer; $1.7 We achieve 5.5

Internet 134

Internet Internet Scalability Wireless 134

Adrian Cockcroft

APRIL 18, 2018

In reality, in any non-trivial installation, there are multiple tools collecting, storing and displaying overlapping sets of metrics from many types of systems and different levels of abstraction. What if your monitoring systems fail? How do you even know when a monitoring system has failed?

Monitoring 40

Monitoring Systems Virtualization Metrics 40

All Things Distributed

NOVEMBER 8, 2012

Werner Vogels weblog on building scalable and robust distributed systems. Improving the Cloud - More Efficient Queuing with SQS. For example, AWS customers use SQS for asynchronous communication pipelines, buffer queues for databases, asynchronous work queues, and moving latency out of highly responsive requests paths.

Efficiency 107

Efficiency Cloud Games Scalability 107

Dynatrace

SEPTEMBER 30, 2021

Cloud services platforms like AWS, Azure, and GCP are reshaping how organizations deliver value to their customers, making cloud migration an increasingly attractive option for running applications. This can fundamentally transform how they work, make processes more efficient, and improve the overall customer experience.

Cloud 166

Cloud Traffic Best Practices Strategy 166

Dotcom-Montior

OCTOBER 6, 2021

To think about it another way, site reliability engineering is where the traditional IT role, or system administration role, and DevOps meet. In a traditional IT environment, organizations may have had a team of system administrators managing complex systems. What Does a Site Reliability Engineer Do? Monitoring. On-call Support.

Engineering 67

Engineering DevOps Monitoring Google 67

Brendan Gregg

FEBRUARY 28, 2023

This talk originated from my updates to Systems Performance 2nd Edition , and this was the first time I've given this talk in person! CXL in a way allows a custom memory controller to be added to a system, to increase memory capacity, bandwidth, and overall performance. Ford, et al., “TCP

Performance 52

Performance Latency Cache Virtualization 52

All Things Distributed

APRIL 17, 2013

Werner Vogels weblog on building scalable and robust distributed systems. While DynamoDB already allows you to perform low-latency queries based on your tableâ??s This gives you the ability to perform richer queries while still meeting the low-latency demands of responsive, scalable applications. All Things Distributed.

Cloud 116

Cloud Latency Games Scalability 116

IO River

NOVEMBER 2, 2023

‍Think of a situation where you're asked to build a service in AWS that distributes static content to your users. Your first option is to use its native CDN - Amazon CloudFront, as it seamlessly integrates with all of your other AWS Services that you're using to build your service.‍For What is vendor lock-in?‍Think

Lambda 52

Lambda AWS Cache Website 52

Adrian Cockcroft

APRIL 5, 2023

This proposal seeks to define a standard for real-time carbon and energy data as time-series data that would be accessed alongside and synchronized with the existing throughput, utilization and latency metrics that are provided for the components and applications in computing environments.

Energy 52

Energy Metrics Cloud Operating System 52

The Morning Paper

JUNE 23, 2019

Coupled with stateless application servers to execute business logic and a database-like system to provide persistent storage, they form a core component of popular data center service archictectures. Why are developers using RInK systems as part of their design? We’ve seen similar high marshalling overheads in big data systems too.)

Cache 79

Cache Latency Google Lambda 79

Smashing Magazine

AUGUST 3, 2023

This is why the async and deferred attributes are crucial, as they ensure an efficient, seamless web browsing experience. This proximity significantly reduces the time it takes for the data to travel from the server to the user, thus reducing latency and improving load times. Time to First Byte over time. LCP in seconds.

Performance 74

Performance Cache Traffic Metrics 74

cdemi

FEBRUARY 20, 2017

Serverless systems are still in their infancy. Cost - Serverless Computing is more cost-efficient than having a fixed quantity of servers. Whether you choose Azure Functions or AWS Lambda, you cannot easily switch to another. Amazon: AWS Lambda. Advantages and Disadvantages of Serverless. Advantages. IBM: OpenWhisk.

Serverless 64

Serverless Architecture Lambda Azure 64

IO River

NOVEMBER 2, 2023

Think of a situation where you're asked to build a service in AWS that distributes static content to your users. Later on, your service might require security, and you might use AWS Web Application Firewall, which directly integrates with Amazon CloudFront to help secure your service. What is vendor lock-in?‍Think

Lambda 52

Lambda AWS Cache Website 52

VoltDB

JUNE 1, 2017

Telcos and Fortune 500 corporations used to spend years defining custom systems, down to the last cable. These systems were designed to have a lifetime of half a decade or more, and rapidly changing hardware meant that the initial deployment had to be sized for 5-7 years out. The “Public Private Cloud” folks.

Hardware 56

Hardware Latency Virtualization Cloud 56

Percona

APRIL 17, 2023

In this blog post, we will discuss the best practices on the MongoDB ecosystem applied at the Operating System (OS) and MongoDB levels. Operating System (OS) settings Swappiness Swappiness is a Linux kernel setting that influences the behavior of the Virtual Memory manager when it needs to allocate a swap, ranging from 0-100.

Best Practices 52

Best Practices Design Tuning Database 52

The Morning Paper

MAY 28, 2019

We don’t talk about energy as often as we probably should on this blog, but it’s certainly true that our data centres and various IT systems consume an awful lot of it. One efficient way of doing that in analog hardware is the use of current-starved inverters. Boosted race trees for low energy classification Tzimpragos et al.,

Energy 52

Energy Hardware Efficiency Architecture 52

VoltDB

JUNE 29, 2017

This allows publishers to establish a “premium” auction instead of the typical real-time bidding system. Most existing adtech infrastructure simply can not achieve the required latency. VoltDB provides the necessary technology to achieve the latency required by header bidding. Another adtech infrastructure problem is capacity.

Performance 40

Performance Hardware Latency Infrastructure 40

VoltDB

JUNE 29, 2017

This allows publishers to establish a “premium” auction instead of the typical real-time bidding system. Most existing adtech infrastructure simply can not achieve the required latency. VoltDB provides the necessary technology to achieve the latency required by header bidding. Another adtech infrastructure problem is capacity.

Performance 40

Performance Hardware Latency Infrastructure 40

VoltDB

JUNE 1, 2017

Telcos and Fortune 500 corporations used to spend years defining custom systems, down to the last cable. These systems were designed to have a lifetime of half a decade or more, and rapidly changing hardware meant that the initial deployment had to be sized for 5-7 years out. The “Public Private Cloud” folks.

Hardware 40

Hardware Latency Virtualization Cloud 40

The Netflix TechBlog

DECEMBER 21, 2020

By Anupom Syam Background At Netflix, our current data warehouse contains hundreds of Petabytes of data stored in AWS S3 , and each day we ingest and create additional Petabytes. We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits.

Storage 203

Storage Latency Efficiency Data Engineering 203

Brendan Gregg

SEPTEMBER 5, 2017

What follows are topics that may be of interest to anyone looking to migrate their systems and skillset: scan these to find topics that interest you. ## ZFS ZFS is available for Linux via the [zfsonlinux] and [OpenZFS] projects, and more recently was included in Canonical's Ubuntu Linux distribution: Ubuntu Xenial 16.04 Hit Ctrl-C to end. ^C

Virtualization 76

Virtualization AWS Engineering Hardware 76