The Netflix TechBlog

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! To sustain this data growth at Netflix, it has deployed open-source software Ceph using AWS services to achieve the required SLOs of some of the post-production workflows.

AWS 100

AWS Entertainment Open Source Benchmarking 100

Dynatrace

MARCH 18, 2024

For example, Amazon Web Services (AWS) charges for data transfer between Amazon EC2 instances within the same region. Suboptimal architecture design. Poorly designed cloud solutions can become costly over time. Hyperscaler cloud service providers such as AWS, Microsoft Azure, and Google Cloud Platform can do this, too.

Best Practices 225

Best Practices Strategy Cloud AWS 225

Dynatrace

MARCH 10, 2021

This week Dynatrace achieved Amazon Web Services (AWS) Machine Learning Competency status in the new Applied Artificial Intelligence (Applied AI) category. The designation reflects AWS’ recognition that Dynatrace has demonstrated deep experience and proven customer success building AI-powered solutions on AWS.

AWS 142

AWS Cloud Artificial Intelligence Innovation 142

All Things Distributed

OCTOBER 17, 2016

The Ohio Region is the fifth AWS region in the US. It brings the worldwide total of AWS Availability Zones (AZs) to 38, and the number of regions globally to 14. The pace of expansion at AWS is accelerating, and Ohio is our third region launch this year. We are also glad to offer low transfer rates between both US East Regions.

AWS 127

AWS Cloud Infrastructure Analytics 127

All Things Distributed

MAY 19, 2017

Soon after the launch of Amazon Relation Dabase Service (RDS) AWS customers were giving us feedback that they would love to migrate to RDS but what they would love even more was if we could also unshackle them from the high-cost, punitive licensing schemes that came with the proprietary databases.

Database 117

Database Design Cloud Storage 117

All Things Distributed

JULY 9, 2020

When I think about how Amazon’s globally connected distribution network has changed in the last decade alone, it’s incredible. Some of their equipment was designed before the internet even existed. Manufacturing, in particular, has always captivated my attention in this respect.

Artificial Intelligence 150

Artificial Intelligence AWS IoT Internet 150

Dynatrace

DECEMBER 8, 2022

DevOps platform engineers are responsible for cloud platform availability and performance, as well as the efficiency of virtual bandwidth, routers, switches, virtual private networks, firewalls, and network management. Amazon Web Services (AWS). Automated DevOps throughout AWS hybrid-cloud environments. Kubernetes.

DevOps 184

DevOps Engineering Testing Open Source 184

Scalegrid

JANUARY 11, 2024

Native tools, such as CloudWatch from AWS, have more extensive integrations and provide comprehensive protection for data in the cloud compared to their third-party counterparts. For example, Blumira offers a third-party solution while AWS provides its own with CloudWatch.

Best Practices 130

Best Practices Cloud Monitoring Strategy 130

All Things Distributed

MAY 22, 2013

Amazon Redshift and Designing for Security. these are core principles for every AWS service. A lot of the power of AWS comes from the fact that we can invest in expertise in these sorts of areas and spread the benefits across all our customers. Amazon Redshift starts with the security foundation underlying all AWS services.

Design 110

Design AWS Hardware Media 110

Adrian Cockcroft

JANUARY 17, 2022

The need for systems to be resilient is still increasing, and chaos engineering tools and techniques are developing as a key way to validate that resilience is working as designed. For example AWS launched an instance type with 800 Gbits/s of network bandwith in 2021.

Serverless 98

Serverless Hardware AWS Architecture 98

Dynatrace

OCTOBER 28, 2021

In 2009, the purveyor of online videos migrated to AWS cloud infrastructure to deliver its entertainment to a growing audience. A disruption spanning hardware, networks, and cloud infrastructure can require input and participation from network and infrastructure architects, risk experts, security teams, and even procurement officers.

Engineering 218

Engineering Entertainment Cloud Infrastructure 218

All Things Distributed

JANUARY 23, 2012

Expanding the Cloud - The AWS Storage Gateway. Today Amazon Web Services has launched the AWS Storage Gateway, making the power of secure and reliable cloud storage accessible from customersâ?? The Amazon Virtual Private Cloud extends on-premises compute with all the power of AWS, making it elastic, scalable and highly reliable.

Storage 115

Storage AWS Virtualization Cloud 115

The Netflix TechBlog

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! To sustain this data growth at Netflix, it has deployed open-source software Ceph using AWS services to achieve the required SLOs of some of the post-production workflows.

AWS 37

AWS Entertainment Open Source Benchmarking 37

All Things Distributed

MARCH 2, 2011

Expanding the Cloud - Introducing the AWS Asia Pacific (Tokyo) Region. Today Amazon Web Services is expanding its world-wide coverage with the launch of a new AWS Region located in Tokyo, Japan. The advanced Asia Pacific network infrastructure also makes the AWS Tokyo Region a viable low-latency option for customers from South Korea.

AWS 112

AWS Cloud Games Latency 112

High Scalability

OCTOBER 19, 2018

four petabytes : added to Internet Archive per year; 60,000 : patents donated by Microsoft to the Open Invention Network; 30 million : DuckDuckGo daily searches; 5 seconds : Google+ session length; 1 trillion : ARM device goal; $40B : Softbank investment in 5G; 30 : Happy Birthday IRC!; They'll love it and you'll be their hero forever.

Internet 123

Internet Internet Scalability Lambda 123

All Things Distributed

MARCH 5, 2012

Driving Compute Cost Down for AWS Customers. AWS today announced a substantial price drop from March 1, 2012 for many of the Amazon EC2, Amazon RDS, and Amazon ElastiCache instances types around the world. Driving costs down for our customers is part of the DNA of Amazon and therefore also part of the DNA of AWS. Comments ().

AWS 110

AWS Innovation Scalability Cache 110

The Netflix TechBlog

MAY 21, 2019

The device ecosystem is rich with partners ranging from Silicon-on-Chip (SoC) manufacturers, Original Design Manufacturer (ODM) and Original Equipment Manufacturer (OEM) vendors. Solving the above problems could help Netflix and our Partners save time and money during the entire lifecycle of device design, build, test, and certification.

Testing 163

Testing AWS Lambda Network 163

All Things Distributed

AUGUST 16, 2011

Expanding the Cloud - The AWS GovCloud (US) Region. Today AWS announced the launch of the AWS GovCloud (US) Region. The concept of regions gives AWS customers control over the placement of their resources and services. The Cloud First strategy is most visible with new Federal IT programs, which are all designed to be â??Cloud

AWS 111

AWS Government Big Data Cloud 111

All Things Distributed

JULY 7, 2011

Expanding the Cloud - AWS Import/Export Support for Amazon EBS. The AWS Import/Export team has announced today that they have expanded their functionality significantly by adding Import into Amazon EBS. AWS Import/Export transfers data off of storage devices using Amazons high-speed internal network and bypassing the Internet.

AWS 96

AWS Cloud Storage Internet 96

All Things Distributed

SEPTEMBER 25, 2017

Today, I am excited to announce plans for Amazon Web Services (AWS) to bring an infrastructure Region to the Middle East! Based in Bahrain, this will be the first Region for AWS in the Middle East. This news marks the 22nd AWS Region we have announced globally. 2017 continues a busy year for AWS in the Middle East.

Cloud 152

Cloud Education Energy Government 152

The Netflix TechBlog

OCTOBER 19, 2020

Now let’s look at how we designed the tracing infrastructure that powers Edgar. Reconstructing a streaming session was a tedious and time consuming process that involved tracing all interactions (requests) between the Netflix app, our Content Delivery Network (CDN), and backend microservices.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

Abhishek Tiwari

JULY 22, 2023

For instance, Sentinel policies can enforce rules such as "All AWS S3 buckets must have server-side encryption enabled" or "Only specific IAM roles can access certain resources." Conftest : Conftest is a tool designed to enforce policies on configuration files. Checkov: Checkov is a static code analysis tool.

AWS 52

AWS Best Practices Infrastructure Traffic 52

All Things Distributed

APRIL 11, 2018

Last year at re:Invent, we launched AWS Fargate , and today, I want to explore how Fargate fundamentally changes the landscape of container technology. This is the vision behind AWS Fargate. With AWS Fargate, you can provision resources to run your containers at a much finer grain than with an EC2 instance. Getting Started.

Cloud 148

Cloud AWS Innovation Traffic 148

Brendan Gregg

JULY 4, 2021

AWS Graviton2); for memory with the arrival of DDR5 and High Bandwidth Memory (HBM) on-processor; for storage including new uses for 3D Xpoint as a 3D NAND accelerator; for networking with the rise of QUIC and eXpress Data Path (XDP); and so on.

Performance 140

Performance Latency Hardware Storage 140

Scalegrid

JANUARY 8, 2024

Infrastructure Excellence ScaleGrid’s infrastructure is designed to facilitate hosting in your cloud account and provides cost-saving options with AWS or Azure Reserved Instances or GCP. This results in faster response times and reduced network traffic, enhancing the overall efficiency and effectiveness of cloud services.

Artificial Intelligence 147

Artificial Intelligence Cloud Scalability Analytics 147

Dynatrace

SEPTEMBER 15, 2020

As an Amazon Web Services (AWS) Advanced Technology Partner, Dynatrace easily integrates with AWS to help you stay on top of the dynamics of your enterprise cloud environment?. We’re therefore excited to announce that Dynatrace has received the AWS Outposts Service Ready designation. What is AWS Outposts?

AWS 195

AWS Design Storage Cloud 195

Dynatrace

JUNE 27, 2023

Many AWS services and third party solutions use AWS S3 for log storage. We hear from our customers how important it is to have a centralized, quick, and powerful access point to analyze these logs; hence we’re making it easier to ingest AWS S3 logs and leverage Dynatrace Log Management and Analytics powered by Grail.

AWS 191

AWS Cloud Lambda Analytics 191

Percona

APRIL 26, 2023

So in case of a network failure, we’d have to retry a recent chunk rather than the whole file. The final design The downloader starts the number of workers, which equals the concurrency (number of CPU cores by default). And we would have bottlenecks in case of big collections. The idea is to download these chunks concurrently.

Speed 81

Speed Storage AWS Azure 81

All Things Distributed

NOVEMBER 26, 2013

As I discussed in my re:Invent keynote earlier this month, I am now happy to announce the immediate availability of Amazon RDS Cross Region Read Replicas , which is another important enhancement for our customers using or planning to use multiple AWS Regions to deploy their applications. Cross Region Read Replicas are available for MySQL 5.6

Cloud 145

Cloud AWS Traffic Latency 145

All Things Distributed

JANUARY 19, 2011

AWS Elastic Beanstalk: A Quick and Simple Way into the Cloud. Additionally customers are not restricted to AWS services; they can mix-and-match services from other providers to best meet their needs. There are some excellent platforms running on AWS that do precisely this; Ruby on Rails developers have Heroku. Comments ().

AWS 115

AWS Cloud Java Operating System 115

Dynatrace

NOVEMBER 23, 2020

Kubernetes Dashboard as a part of a comprehensive monitoring solution This article introduces the key building blocks to consider when designing your Kubernetes monitoring solution. When designing and running modern, scalable, and distributed applications, Kubernetes seems to be the solution for all your needs. Cloud platform layer.

Virtualization 159

Virtualization Infrastructure Monitoring Cloud 159

HammerDB

APRIL 19, 2023

As database performance is heavily influenced by the performance of storage, network, memory, and processors, we must understand the upper limit of these key components. For example, if you are buying the latest Amazon memory-optimized EC2 instance (R7iz), the AWS page ( [link] ) tells us the following: Up to 3.9 4.22 %usr 38.40

Performance 54

Performance Benchmarking Cache Storage 54

The Netflix TechBlog

MAY 26, 2020

Service Segmentation: The ease of the cloud deployments has led to the organic growth of multiple AWS accounts, deployment practices, interconnection practices, etc. Without having network visibility, it’s not possible to improve our reliability, security and capacity posture. A flow log record represents a network flow in the VPC.

Network 150

Network Tuning AWS Big Data 150

Brendan Gregg

FEBRUARY 28, 2023

Titus, the Netflix container management platform, is now open source,” [link] Apr 2018 - [Cutress 19] Dr. DDR6: Here's What to Expect in RAM Modules,” [link] Nov 2020 - [Salter 20] Jim Salter, “Western Digital releases new 18TB, 20TB EAMR drives,” [link] Jul 2020 - [Spier 20] Martin Spier, Brendan Gregg, et al.,

Performance 110

Performance Latency Cache Virtualization 110

All Things Distributed

DECEMBER 18, 2012

After the AWS re: Invent conference I spent two weeks in Europe for the last customer visits of the year. Although there are still a few very exciting AWS news updates to happen this year. Jul 6 - End-To-End Arguments in System Design , by J. Aug 11 - " On the Naming and Binding of Network Destinations ", Saltzer, J.

Operating System 144

Operating System Cache AWS Scalability 144

The Morning Paper

AUGUST 29, 2019

If you’re moving an OLAP workload to the cloud (AWS in the context of this paper), what DBMS setup should you go with? We focused on OLAP-oriented parallel data warehouse products available for AWS and restricted our attention to commercially available systems. The design space. VLDB’19. Key findings. The last word.

Architecture 76

Architecture Cloud Storage Serverless 76

The Morning Paper

MARCH 8, 2020

This paper describes the design decisions behind the Snowflake cloud-based data warehouse. This paper presents Snowflake design and implementation along with a discussion on how recent changes in cloud infrastructure (emerging hardware, fine-grained billing, etc.) S3 on AWS). From shared-nothing to disaggregation. Elasticity.

Storage 112

Storage Engineering Cache Serverless 112