The Netflix TechBlog

OCTOBER 19, 2020

We earned the trust of our engineers by developing empathy for their operational burden and by focusing on providing efficient tracer library integrations in runtime environments. An additional implication of a lenient sampling policy is the need for scalable stream processing and storage infrastructure fleets to handle increased data volume.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

All Things Distributed

NOVEMBER 8, 2012

Werner Vogels weblog on building scalable and robust distributed systems. Improving the Cloud - More Efficient Queuing with SQS. The Amazon Simple Queue Service (SQS) is a highly scalable, reliable and elastic queuing service that just works. Similarly, AWS customers have been using SQS in interesting ways. Comments ().

Efficiency 104

Efficiency Cloud Games Scalability 104

All Things Distributed

JUNE 29, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Driving Bandwidth Cost Down for AWS Customers. Any work we can do to improve over these dimensions generates long term benefits for AWS customers. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway.

AWS 104

AWS Retail Innovation Strategy 104

Dynatrace

OCTOBER 7, 2021

Amazon Web Services (AWS), offers a wide range of serverless solutions. To get a better understanding of AWS serverless, we’ll first explore the basics of serverless architectures, review AWS serverless offerings, and explore common use cases. Scalability. Finally, there’s scalability. Data Store.

Serverless 286

Serverless AWS Lambda Storage 286

Percona Community

JANUARY 7, 2020

At re:Invent in Las Vegas in December 2019, AWS announced the public preview of RDS Proxy, a fully managed database proxy that sits between your application and RDS.

AWS 52

AWS Scalability Database Efficiency 52

The Netflix TechBlog

MAY 21, 2019

There are several problems efficient test scheduling could help us solve: Quickly detect a regression in the integration of the Netflix SDK on a consumer electronic or MVPD (multichannel video programming distributor) device. We wanted the to tool be available as a standalone library as well as scalable API service.

Testing 163

Testing AWS Lambda Network 163

All Things Distributed

APRIL 17, 2013

Werner Vogels weblog on building scalable and robust distributed systems. This gives you the ability to perform richer queries while still meeting the low-latency demands of responsive, scalable applications. database performance and scalability are critical for delivering new services to our 34+ million readers on any device.

Cloud 114

Cloud Latency Games Scalability 114

All Things Distributed

AUGUST 18, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Many of our Big-Data customers already saw a big drop in their AWS bill last month when the cost of incoming bandwidth was dropped to $0.00. However, this cannot be done without efficient, scalable data analytics. Countdown to What is Next in AWS.

Big Data 111

Big Data Analytics AWS Cloud 111

Percona

OCTOBER 19, 2023

Welcome to the first installment of our series: Scalable Solutions with Percona Distribution for PostgreSQL. In this guide, we will demonstrate how to establish a Citus database spanning multiple nodes by using Percona Distribution for PostgreSQL. This tutorial will use Ubuntu 20.04 and Percona Distribution for PostgreSQL 15.

Scalability 85

Scalability Database Open Source Traffic 85

Scalegrid

JANUARY 8, 2024

Key Takeaways Multi-cloud involves using services from multiple cloud providers to gain flexibility and reduce vendor lock-in, while hybrid cloud combines private and public cloud resources to balance control and scalability. Thus making it an ideal choice for businesses seeking a successful implementation of their multi-cloud strategy.

Strategy 147

Strategy Cloud Scalability Artificial Intelligence 147

Scalegrid

MAY 13, 2020

Greenplum Database is a massively parallel processing (MPP) SQL database that is built and based on PostgreSQL. Greenplum Database is an open-source , hardware-agnostic MPP database for analytics, based on PostgreSQL and developed by Pivotal who was later acquired by VMware. What is an MPP Database?

Big Data 321

Big Data Database Artificial Intelligence Open Source 321

Percona

JULY 1, 2023

Now that Database-as-a-service (DBaaS) is in high demand, there are multiple questions regarding AWS services that cannot always be answered easily: When should I use Aurora and when should I use RDS MySQL ? What we should really compare is the MySQL and Aurora database engines provided by Amazon RDS.

AWS 52

AWS Database Serverless Storage 52

Scalegrid

MARCH 14, 2024

This approach allows companies to combine the security and control of private clouds with public clouds’ scalability and innovation potential. The public cloud’s ability to scale efficiently enables ‘cloudbursting’ when demand spikes without requiring businesses to overprovision their own infrastructures.

Strategy 130

Strategy Cloud Artificial Intelligence Infrastructure 130

ScaleOut Software

JANUARY 9, 2020

To keep operations efficient and cost-effective, it’s important to be able to quickly respond to issues as they occur and efficiently verify their resolution. It’s not enough just to pick out interesting events from an aggregated data stream and then send them to a database for offline analysis using Spark.

Logistics 52

Logistics Retail Analytics Azure 52

ScaleOut Software

JANUARY 9, 2020

To keep operations efficient and cost-effective, it’s important to be able to quickly respond to issues as they occur and efficiently verify their resolution. It’s not enough just to pick out interesting events from an aggregated data stream and then send them to a database for offline analysis using Spark.

Logistics 52

Logistics Retail Analytics Azure 52

ScaleOut Software

JANUARY 9, 2020

To keep operations efficient and cost-effective, it’s important to be able to quickly respond to issues as they occur and efficiently verify their resolution. It’s not enough to just pick out interesting events from an aggregated data stream and then send them to a database for offline analysis using Spark.

Logistics 52

Logistics Retail Analytics Azure 52

Dynatrace

APRIL 1, 2021

Versatile, feature-rich cloud computing environments such as AWS, Microsoft Azure, and GCP have been a game-changer. Keeping track of performance, response time, and efficiency can be cumbersome, especially when teams use a multicloud strategy that spans cloud environments and on-premises systems. The benefit is scalability.

Azure 175

Azure AWS DevOps Cloud 175

Scalegrid

JANUARY 12, 2024

Simply put, it’s the set of computational tasks that cloud systems perform, such as hosting databases, enabling collaboration tools, or running compute-intensive algorithms. This article analyzes cloud workloads, delving into their forms, functions, and how they influence the cost and efficiency of your cloud infrastructure.

Cloud 130

Cloud Virtualization Storage Efficiency 130

High Scalability

JUNE 29, 2018

kellabyte : LOL at racking up an AWS bill of $140,000 in 4 hours of compute time. kellabyte : Recently I got to work on a project that really stressed Amazon AWS scalability. We spun up a cluster of 100,000 AWS instances multiple times. They'll love you even more. potential immigrants can actually attend! Don't stop here.

Scalability 131

Scalability Internet Internet Wireless 131

All Things Distributed

DECEMBER 13, 2009

Werner Vogels weblog on building scalable and robust distributed systems. Spot Instances are an innovation that is made possible by the unparalleled economies of scale created by the tremendous growth of the AWS Infrastructure Services. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway.

Cloud 63

Cloud AWS Storage Innovation 63

All Things Distributed

JULY 14, 2015

I am excited to share with you that today we are expanding DynamoDB with streams, cross-region replication, and database triggers. In traditional database architectures, database engines often run a small search engine or data warehouse engines on the same hardware as the database. Let me expand on each one of them.

Database 167

Database Lambda AWS IoT 167

All Things Distributed

JANUARY 18, 2012

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Today is a very exciting day as we release Amazon DynamoDB , a fast, highly reliable and cost-effective NoSQL database service designed for internet scale applications.

Scalability 167

Scalability Database Ecommerce Latency 167

All Things Distributed

JUNE 21, 2018

A common question that I get is why do we offer so many database products? To do this, they need to be able to use multiple databases and data models within the same application. Seldom can one database fit the needs of multiple distinct use cases. Seldom can one database fit the needs of multiple distinct use cases.

Database 167

Database AWS Games Latency 167

All Things Distributed

MARCH 19, 2018

At AWS, we continue to strive to enable builders to build cutting-edge technologies faster in a secure, reliable, and scalable fashion. Now, thousands of customers are trying Amazon SageMaker and building ML models on top of their data lakes in AWS. In machine learning, more is usually more. Acceleration and distribution.

Scalability 159

Scalability Hardware AWS Tuning 159

Percona

FEBRUARY 24, 2023

In the Home Dashboard of PMM, a low CPU utilization on any of the database services that are being monitored could mean that the server is inactive or over-provisioned. From AWS documentation , Amazon EBS is an easy-to-use, scalable, high-performance block-storage service designed for Amazon EC2.

Cloud 119

Cloud Serverless AWS Storage 119

Dynatrace

MAY 5, 2020

Dynatrace is an AWS Advanced Technology Partner, so it’s easy to integrate Dynatrace with AWS to help you stay on top of the dynamics of your enterprise cloud environment?. In addition to automatic full-stack monitoring, Dynatrace provides comprehensive support for a wide range of AWS services. Dynatrace news.

Design 143

Design AWS Education Innovation 143

Percona

DECEMBER 18, 2023

Percona is a leading provider of unbiased, performance-first, open source database solutions that allow organizations to easily, securely, and affordably maintain business agility, minimize risks, and stay competitive, free from vendor lock-in. Today’s post includes those releases and updates that have come out since December 4, 2023.

Monitoring 87

Monitoring Open Source Servers Scalability 87

All Things Distributed

FEBRUARY 18, 2013

Werner Vogels weblog on building scalable and robust distributed systems. Expanding the Cloud - Introducing AWS OpsWorks, a Powerful Application Management Solution. As with all the AWS Application Management services AWS OpsWorks is provided at no additional charge. AWS OpsWorks Unique Features. Comments ().

AWS 157

AWS Cloud Architecture Infrastructure 157

All Things Distributed

OCTOBER 12, 2016

Fast Data is an emerging industry term for information that is arriving at high volume and incredible rates, faster than traditional databases can manage. Three years ago, as part of our AWS Fast Data journey we introduced Amazon ElastiCache for Redis , a fully managed in-memory data store that operates at sub-millisecond latency.

Scalability 130

Scalability Analytics Cache AWS 130

The Netflix TechBlog

DECEMBER 21, 2020

By Anupom Syam Background At Netflix, our current data warehouse contains hundreds of Petabytes of data stored in AWS S3 , and each day we ingest and create additional Petabytes. We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits.

Storage 203

Storage Latency Efficiency Data Engineering 203

The Netflix TechBlog

MARCH 25, 2019

Building and Scaling Data Lineage at Netflix to Improve Data Infrastructure Reliability, and Efficiency By: Di Lin , Girish Lingappa , Jitender Aswani Imagine yourself in the role of a data-inspired decision maker staring at a metric on a dashboard about to make a critical business decision but pausing to ask a question?—?“Can

Infrastructure 219

Infrastructure Big Data Transportation Architecture 219

Dynatrace

NOVEMBER 10, 2022

To take full advantage of the scalability, flexibility, and resilience of cloud platforms, organizations need to build or rearchitect applications around a cloud-native architecture. Taken together, these features enable organizations to build software that is more scalable, reliable, and flexible than traditionally built software.

Cloud 162

Cloud Architecture Serverless Infrastructure 162

All Things Distributed

AUGUST 20, 2015

It opens up the possibility to enjoy the value that graph databases bring to relationship-centric use cases, without worrying about managing the underlying storage. Developers need efficient methods to store, traverse, and query these relationships. Enter graph databases. Graph databases at Amazon.

Database 134

Database Logistics Availability Social Media 134

The Netflix TechBlog

OCTOBER 27, 2020

The processed data is typically stored as data warehouse tables in AWS S3. Therefore, we must efficiently move data from the data warehouse to a global, low-latency and highly-reliable key-value store. As the paved path for moving data to key-value stores, Bulldozer provides a scalable and efficient no-code solution.

Latency 243

Latency Storage Big Data Tuning 243

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. In databases like MySQL and PostgreSQL, transaction logs are the source of CDC events. Designed with High Availability in mind.

Database 212

Database Traffic Transportation Open Source 212

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. In databases like MySQL and PostgreSQL, transaction logs are the source of CDC events. Designed with High Availability in mind.

Database 197

Database Traffic Transportation Open Source 197

The Netflix TechBlog

OCTOBER 18, 2022

As Big data and ML became more prevalent and impactful, the scalability, reliability, and usability of the orchestrating ecosystem have increasingly become more important for our data scientists and the company. Motivation Scalability and usability are essential to enable large-scale workflows and support a wide range of use cases.

Java 202

Java Scalability Traffic Architecture 202