The Netflix TechBlog

OCTOBER 19, 2020

a Netflix member via Twitter This is an example of a question our on-call engineers need to answer to help resolve a member issue?—?which Our distributed tracing infrastructure is grouped into three sections: tracer library instrumentation, stream processing, and storage. Storage: don’t break the bank!

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

All Things Distributed

JUNE 10, 2016

Building your applications with only managed components has become very popular, and AWS Lambda plays a crucial role in that. I see a tremendous interest in examples how to build such applications, and articles such as " The Serverless Start-Up - Down With Servers! The architecture can be created with an AWS CloudFormation template.

Lambda 107

Lambda Serverless AWS Architecture 107

All Things Distributed

MAY 10, 2016

Building your applications with only managed components has become very popular, and AWS Lambda plays a crucial role in that. I see a tremendous interest in examples how to build such applications, and articles such as " The Serverless Start-Up - Down With Servers! The architecture can be created with an AWS CloudFormation template.

Lambda 105

Lambda Serverless AWS Architecture 105

High Scalability

JUNE 22, 2018

And if you know anyone looking for a simple book that uses lots of pictures and lots of examples to explain the cloud, then please recommend my new book: Explain the Cloud Like I'm 10. using them to respond to storage events on s3 or database events or auth events is super easy and powerful. Do you like this sort of Stuff?

Internet 113

Internet Internet Scalability Artificial Intelligence 113

All Things Distributed

AUGUST 20, 2012

Werner Vogels weblog on building scalable and robust distributed systems. Managing Cold Storage with Amazon Glacier. With the introduction of Amazon Glacier , IT organizations now have a solution that removes the headaches of digital archiving and provides extremely low cost storage. All Things Distributed. Comments ().

Storage 122

Storage Cloud AWS Media 122

All Things Distributed

NOVEMBER 13, 2014

When AWS launched, it changed how developers thought about IT services: What used to take weeks or months of purchasing and provisioning turned into minutes with Amazon EC2. Capital-intensive storage solutions became as simple as PUTting and GETting objects in Amazon S3. Our answer is a new compute service called AWS Lambda.

Lambda 136

Lambda AWS Cloud Scalability 136

All Things Distributed

MARCH 5, 2012

Werner Vogels weblog on building scalable and robust distributed systems. Driving Compute Cost Down for AWS Customers. AWS today announced a substantial price drop from March 1, 2012 for many of the Amazon EC2, Amazon RDS, and Amazon ElastiCache instances types around the world. For example, the popular m1.small

AWS 110

AWS Innovation Scalability Cache 110

Scalegrid

JANUARY 9, 2024

As more companies move away from traditional on-premises data centers, they enter into an era where scalability, flexibility, and cost-effectiveness become possible through various services offered by different providers in the market today. This enabled them to effectively manage and safeguard their air quality data.

Strategy 130

Strategy Cloud Best Practices Government 130

All Things Distributed

AUGUST 16, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Expanding the Cloud - The AWS GovCloud (US) Region. Today AWS announced the launch of the AWS GovCloud (US) Region. The concept of regions gives AWS customers control over the placement of their resources and services. All Things Distributed.

AWS 111

AWS Government Big Data Cloud 111

The Netflix TechBlog

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! To sustain this data growth at Netflix, it has deployed open-source software Ceph using AWS services to achieve the required SLOs of some of the post-production workflows.

AWS 37

AWS Entertainment Open Source Benchmarking 37

All Things Distributed

JUNE 29, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Driving Bandwidth Cost Down for AWS Customers. For example, when our retail customers contributed to create larger economies of scale for Amazon.com, we used the savings to lower pricing such that our customers could also benefit. All Things Distributed.

AWS 104

AWS Retail Innovation Strategy 104

Dynatrace

NOVEMBER 23, 2020

Nevertheless, there are related components and processes, for example, virtualization infrastructure and storage systems (see image below), that can lead to problems in your Kubernetes infrastructure. Configuring storage in Kubernetes is more complex than using a file system on your host. The Kubernetes experience.

Virtualization 159

Virtualization Infrastructure Monitoring Cloud 159

HammerDB

APRIL 19, 2023

As database performance is heavily influenced by the performance of storage, network, memory, and processors, we must understand the upper limit of these key components. For example, if you are buying the latest Amazon memory-optimized EC2 instance (R7iz), the AWS page ( [link] ) tells us the following: Up to 3.9

Performance 54

Performance Benchmarking Cache Storage 54

All Things Distributed

SEPTEMBER 25, 2012

Werner Vogels weblog on building scalable and robust distributed systems. Amazon RDS Provisioned IOPS is ideal for mission-critical online transaction processing (OLTP) workloads that require a high performance storage option with consistent IOPS, within a narrow band of tolerance. Provisioned IOPS storage in RDS. Comments ().

Cloud 117

Cloud AWS Storage Database 117

Dynatrace

JUNE 27, 2023

Many AWS services and third party solutions use AWS S3 for log storage. We hear from our customers how important it is to have a centralized, quick, and powerful access point to analyze these logs; hence we’re making it easier to ingest AWS S3 logs and leverage Dynatrace Log Management and Analytics powered by Grail.

AWS 191

AWS Cloud Lambda Analytics 191

All Things Distributed

AUGUST 17, 2011

Werner Vogels weblog on building scalable and robust distributed systems. As some of you may remember I was pretty excited when Amazon Simple Storage Service (S3) released its website feature such that I could serve this weblog completely from S3. Driving Storage Costs Down for AWS Customers. All Things Distributed.

Servers 126

Servers Social Media AWS Website 126

All Things Distributed

SEPTEMBER 6, 2013

This spring I travelled through Europe for the AWS Global Summit series. Europe is a continent with much diversity and for each country there are great AWS customer examples to tell. Here are some great examples from different industries each with unique use cases. To reach those goals, Shell in 2010 began using AWS.

Cloud 129

Cloud Energy AWS Healthcare 129

All Things Distributed

APRIL 28, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Expanding the Cloud - Opening the AWS Asia Pacific (Singapore) Region. Today Amazon Web Services has taken another important step in serving customers worldwide: the AWS Asia Pacific (Singapore) Region is now launched. All Things Distributed. Comments ().

AWS 61

AWS Cloud Latency Storage 61

All Things Distributed

DECEMBER 3, 2009

Werner Vogels weblog on building scalable and robust distributed systems. Expanding the Cloud - New AWS Region: US-West (Northern California). We have expanded the AWS footprint in the US and starting today a new AWS Region is available for use: US-West (Northern California). Driving Storage Costs Down for AWS Customers.

AWS 60

AWS Cloud Latency Storage 60

The Netflix TechBlog

DECEMBER 21, 2020

By Anupom Syam Background At Netflix, our current data warehouse contains hundreds of Petabytes of data stored in AWS S3 , and each day we ingest and create additional Petabytes. We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits.

Storage 203

Storage Latency Efficiency Data Engineering 203

Testsigma

AUGUST 23, 2021

Scalability at its best. Let us take an example, a mobile app for e-learning is released on Android devices. They facilitate the storage of huge data and have automated monitoring of the systems on the cloud. Monitoring the security pitfalls, e.g. AWS CloudWatch. e.g. AWS Macie. Multi-Factor Authorization(MFA).

Cloud 74

Cloud Testing AWS Testing Tools 74

All Things Distributed

AUGUST 18, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Many of our Big-Data customers already saw a big drop in their AWS bill last month when the cost of incoming bandwidth was dropped to $0.00. However, this cannot be done without efficient, scalable data analytics. Countdown to What is Next in AWS.

Big Data 111

Big Data Analytics AWS Cloud 111

All Things Distributed

APRIL 5, 2012

Werner Vogels weblog on building scalable and robust distributed systems. Empowering innovation is at the heart of everything we do at Amazon Web Services (AWS). I often get to meet, discuss, and learn from innovators how they are using AWS to deliver transformative applications to their users, customers and partners.

Innovation 146

Innovation AWS Education Network 146

All Things Distributed

MARCH 20, 2012

Werner Vogels weblog on building scalable and robust distributed systems. s AWS Elastic Beanstalk announcement of PHP and Git support reminded me of the post where I mentioned that we want to let a thousand platforms bloom on AWS. Some might ask why AWS would want a thousand platforms. A Thousand Platforms. Comments ().

AWS 104

AWS Scalability Java Storage 104

All Things Distributed

MAY 24, 2011

Werner Vogels weblog on building scalable and robust distributed systems. allthingsdistributed.com) point to same location where for example www.allthingsdistributed.com is pointing to jump through complex redirect hoops. Many AWS customers use Elastic Load Balancers (ELB) to front their Internet facing applications.

Internet 114

Internet Internet AWS Scalability 114

Scalegrid

JANUARY 8, 2024

Key Takeaways Multi-cloud involves using services from multiple cloud providers to gain flexibility and reduce vendor lock-in, while hybrid cloud combines private and public cloud resources to balance control and scalability. The reason lies in the unique advantages that come from utilizing a hybrid cloud.

Strategy 147

Strategy Cloud Scalability Artificial Intelligence 147

All Things Distributed

JANUARY 11, 2011

Werner Vogels weblog on building scalable and robust distributed systems. One wireless company for example has an international plan that will charge you $25 per month for 50MB after which they will charge you $20 per MB. Theyre not even an AWS customer :-). Driving Storage Costs Down for AWS Customers.

Wireless 107

Wireless AWS Internet Internet 107

High Scalability

APRIL 16, 2019

Let’s use an example to review how our framework deals with split-brain scenarios in the cluster. At ScaleGrid, we offer highly available hosting for MySQL on AWS and MySQL on Azure that is implemented based on the concepts explained in this blog series. Please visit the ScaleGrid Console for a free trial of our solutions.

Availability 149

Availability Network Azure AWS 149

Dynatrace

JANUARY 31, 2024

The exponential growth of data volume—including observability, security, software lifecycle, and business data—forces organizations to deal with cost increases while providing flexible, robust, and scalable ingest. OpenPipeline high-performance filtering and preprocessing provides full ingest and storage control for the Dynatrace platform.

Analytics 189

Analytics Processing Transportation Storage 189

All Things Distributed

DECEMBER 5, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Route 53 has the business properties that you have come to expect from an AWS service: fully self-service and programmable, with transparent pay-as-you-go pricing and no minimum usage commitments. All Things Distributed. Comments (). Some fundamentals on Naming.

Cloud 117

Cloud Internet Internet AWS 117

Dotcom-Montior

OCTOBER 6, 2021

Site reliability engineers are focused with creating scalable and reliable software systems, so this also includes ensuring that development work is efficient and reliable, so when the finished product is ready for production, there are no surprises. What Does a Site Reliability Engineer Do? Performance. Monitoring. Incident Response.

Engineering 67

Engineering DevOps Monitoring Google 67

All Things Distributed

JULY 13, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Cluster Compute Instances for Amazon EC2 change the HPC game for two important reasons: Access to scalable on-demand capacity. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. Comments ().

Cloud 118

Cloud AWS Automotive Latency 118

The Morning Paper

JUNE 16, 2019

We’ll get to that, but first let’s look at some examples of privacy policies and how they can be expressed. For example: An aggregation policy could be used to rewrite any matching aggregation into a differentially-private version. The challenging thing of course, is efficiently maintaining all of these parallel universes.

Database 90

Database Cache Benchmarking Efficiency 90

Dynatrace

OCTOBER 6, 2023

Buckets are similar to folders, a physical storage location. For example, a separate bucket could be used for detailed logs from Dynatrace Synthetic nodes. Debug-level logs, which also generate high volumes and have a shorter lifespan or value period than other logs, could similarly benefit from dedicated storage.

Artificial Intelligence 255

Artificial Intelligence Analytics Metrics Storage 255

All Things Distributed

APRIL 23, 2010

Werner Vogels weblog on building scalable and robust distributed systems. As examples of such services I always use Twillio (voice &sms) and Simplegeo (location), but it is time to start building out my knowledge of all the different services that are in the ecosystem. Driving Storage Costs Down for AWS Customers.

AWS 60

AWS Storage Cloud Big Data 60

Dynatrace

DECEMBER 22, 2019

AWS offers a broad set of global, cloud-based services including computing, storage, networking, Internet of Things (IoT), and many others. You can use these services in combinations that are tailored to help your business move faster, lower IT costs, and support scalability. Monitor and understand additional AWS services.

AWS 135

AWS Metrics IoT Storage 135

Dynatrace

DECEMBER 22, 2019

AWS offers a broad set of global, cloud-based services including computing, storage, networking, Internet of Things (IoT), and many others. You can use these services in combinations that are tailored to help your business move faster, lower IT costs, and support scalability. Monitor and understand additional AWS services.

AWS 132

AWS Metrics IoT Storage 132