The Netflix TechBlog

OCTOBER 18, 2022

As Big data and ML became more prevalent and impactful, the scalability, reliability, and usability of the orchestrating ecosystem have increasingly become more important for our data scientists and the company. In this blog, we use step and job interchangeably. It is hard to support super large workflows in general.

Java 202

Java Scalability Traffic Architecture 202

All Things Distributed

AUGUST 19, 2011

Job Openings in AWS - Senior Leader in Database Services. This week it is an opening for senior leaders with AWS Database Services. AWS Database Services is responsible for setting the database strategy and delivering distributed structured storage services to our AWS customers. blog comments powered by Disqus.

AWS 75

AWS Database Storage Scalability 75

The Netflix TechBlog

DECEMBER 21, 2020

We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits. This article will list some of the use cases of AutoOptimize, discuss the design principles that help enhance efficiency, and present the high-level architecture.

Storage 203

Storage Latency Efficiency Data Engineering 203

Dynatrace

JUNE 1, 2020

The variables that can impact the performance of an application vary; from coding errors or ‘bugs’ in the software, database slowdowns, hosting and network performance, to operating system and device type support. appeared first on Dynatrace blog. From APM to full-stack monitoring. You can download the report here.

Monitoring 132

Monitoring Performance Social Media Artificial Intelligence 132

Percona

JANUARY 14, 2019

What if we use ClickHouse (which is a columnar analytical database) as our main datastore? Well, typically, an analytical database is not a replacement for a transactional or key/value datastore. Although such databases can be very efficient with counts and averages, some queries will be slow or simply non existent. Processed 4.15

Database 40

Database Analytics Blockchain Healthcare 40

All Things Distributed

NOVEMBER 15, 2016

To scale to a larger number of users and support the growth in data volume spurred by social media, web, mobile, IoT, ad-tech, and ecommerce workloads, these tools require customers to invest in even more infrastructure to maintain performance. AutoGraph : Picking the right visualization is not easy, and there is lot of science behind it.

Analytics 152

Analytics Availability Media Social Media 152

All Things Distributed

FEBRUARY 17, 2011

Because my blog is almost completely static content I wanted to run in this very simple configuration since the launch of Amazon S3. It would allow the blog to be powered by the incredible scale and reliability of Amazon S3 with a minimum of effort from my side. if you want to see what this blogs error page looks like.

AWS 134

AWS Website Storage Servers 134

Highly Scalable

MARCH 1, 2012

NoSQL databases are often compared by various non-functional criteria, such as scalability, performance, and consistency. At the same time, NoSQL data modeling is not so well studied and lacks the systematic theory found in relational databases. Document databases advance the BigTable model offering two significant improvements.

Database 279

Database Ecommerce Efficiency Engineering 279

All Things Distributed

AUGUST 17, 2011

A good example is the comments section on this blog; a few lines of JavaScript and these pages have a dynamic nature with comments, trackbacks and social media discussion showing up as they happen. But I really want a setup that allows me to thinker with the blog where ever I am (e.g. at 30,000 feet). Which suits me just fine.

Servers 126

Servers Social Media AWS Website 126

All Things Distributed

AUGUST 16, 2011

The Cloud First strategy is most visible with new Federal IT programs, which are all designed to be â??Cloud Government and Big Data. One particular early use case for AWS GovCloud (US) will be massive data processing and analytics. blog comments powered by Disqus. Driving down the cost of Big-Data analytics.

AWS 111

AWS Government Big Data Cloud 111

All Things Distributed

AUGUST 22, 2011

There are many success stories about the effectiveness of caching in many different scenarios; next to helping applications achieving fast and predictable performance, it often protects databases from requests bursts and brownouts under overload conditions. blog comments powered by Disqus. Driving down the cost of Big-Data analytics.

Cloud 117

Cloud Cache AWS Storage 117

All Things Distributed

APRIL 27, 2011

Given that I have frequently written about many of these technologies on this blog I asked investor relations to be allowed to reprint it here. To our shareowners: Random forests, naïve Bayesian estimators, RESTful services, gossip protocols, eventual consistency, data sharding, anti-entropy, Byzantine quorum, erasure coding, vector clocks.

Technology 151

Technology Technology AWS Storage 151

All Things Distributed

DECEMBER 14, 2011

Please also visit the AWS developer blog for more great stories from our South American customers. blog comments powered by Disqus. he posts material that doesnt belong on this blog or on twitter. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Contact Info. Werner Vogels.

AWS 114

AWS Latency Storage Big Data 114

All Things Distributed

MARCH 24, 2011

Up to 200 developers and designers will get together to hack up interesting applications using the Internets APIs and SDKs. It is likely that the Amazon Web Services will be used by many of the participants for their compute, storage, database and other cloud resource needs. blog comments powered by Disqus. Happy Hacking!

AWS 93

AWS Internet Internet Storage 93

All Things Distributed

JULY 7, 2011

blog comments powered by Disqus. he posts material that doesnt belong on this blog or on twitter. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. Contact Info.

AWS 96

AWS Cloud Storage Internet 96

All Things Distributed

MARCH 28, 2011

Amazon S3 is used by enterprises of all sizes and is designed to handle scaling extremely well; it stores hundreds of billions of objects and easily performs several hundreds of thousands of storage transaction a second. blog comments powered by Disqus. he posts material that doesnt belong on this blog or on twitter.

AWS 138

AWS Cloud Storage Internet 138

All Things Distributed

SEPTEMBER 29, 2010

Like many folks who started down the path of using real-time micro-blogging services (read: twitter ) the convenience of those platforms has made that sharing happens there instead of on the blogs we (used to) keep. That was a mistake which I will try to correct in the coming months as Ill get back into blogging mode.

AWS 69

AWS Storage Big Data Scalability 69

All Things Distributed

FEBRUARY 27, 2011

While cleaning out the digital attic I ran into this drawing that Hugh MacLeod (aka "gapingvoid") made for me in reponse to a storm-in-a-teacup about blogging Amazon. blog comments powered by Disqus. he posts material that doesnt belong on this blog or on twitter. Job Openings in AWS - Senior Leader in Database Services.

AWS 60

AWS Storage Big Data Scalability 60

All Things Distributed

AUGUST 20, 2012

The service redundantly stores data in multiple facilities and on multiple devices within each facility, as Amazon Glacier is designed to provide average annual durability of 99.999999999% for each item stored. Data is retrieved by scheduling a job, which typically completes within 3 to 5 hours. designed for 11 ninesâ??)

Storage 122

Storage Cloud AWS Media 122

All Things Distributed

JUNE 29, 2011

For more details see the announcement , the details pages of the services at [link] , and the posting on the AWS developer blog. blog comments powered by Disqus. he posts material that doesnt belong on this blog or on twitter. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications.

AWS 104

AWS Retail Innovation Strategy 104

All Things Distributed

MARCH 2, 2011

blog comments powered by Disqus. he posts material that doesnt belong on this blog or on twitter. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. Contact Info.

AWS 112

AWS Cloud Games Latency 112

All Things Distributed

FEBRUARY 25, 2011

When a new customer is onboarded, the ISV has to spin up a collection of AWS resources to run their web-servers, app-servers and databases in a multi-AZ (availability zone) setting to achieve high-availability. For more information on AWS CloudFormation see their detail page and read more on the AWS developer blog. Contact Info.

AWS 121

AWS Cloud Scalability Storage 121

Testsigma

MAY 7, 2020

IBM Big Data and Analytics Hub website cited a case study, where a US insurance company was estimating 15% of their testing efforts to be just test data collection for the backend system and the frontend system. For testing purposes, usually, a mix of static and dynamic data is needed. Copy production data i.

Testing 60

Testing Storage Database Processing 60

All Things Distributed

JANUARY 19, 2011

Flexibility is one of the key principles of Amazon Web Services - developers can select any programming language and software package, any operating system, any middleware and any database to build systems and applications that meet their requirements. More information on the launch can be found on the AWS developer blog. Contact Info.

AWS 115

AWS Cloud Java Scalability 115

All Things Distributed

JANUARY 11, 2011

blog comments powered by Disqus. he posts material that doesnt belong on this blog or on twitter. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. Contact Info.

Wireless 107

Wireless AWS Internet Internet 107

All Things Distributed

DECEMBER 9, 2010

By supporting such large object sizes, Amazon S3 better enables a variety of interesting big data use cases. For more information on Multipart Upload and managing large objects in Amazon S3 , see Jeff Barrs blog posts on Amazon S3 Multipart Upload and Large Object Support as well as the Amazon S3 Developer Guide. Contact Info.

AWS 80

AWS Big Data Scalability Storage 80

All Things Distributed

MAY 24, 2011

More information can found on the Elastic Load Balancing and the Route 53 detail pages and on the two blog posts on the AWS developer blog. blog comments powered by Disqus. he posts material that doesnt belong on this blog or on twitter. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications.

Internet 114

Internet Internet AWS Scalability 114

All Things Distributed

JULY 11, 2011

To get started using Spot or for more details visit the Amazon EC2 Spot Instance web page, the AWS developer blog , and the EC2 Release Notes. blog comments powered by Disqus. he posts material that doesnt belong on this blog or on twitter. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications.

AWS 85

AWS Storage Cloud Big Data 85

All Things Distributed

NOVEMBER 19, 2010

Big news this week was of course the launch of Cluster GPU instances for Amazon EC2. There were blog posts by Jeff Barr The Cluster GPU Instance and James Hamilton HPC in the Cloud with GPGPUs , as well as my background posting: Expanding the Cloud - Adding the Incredible Power of the Amazon EC2 Cluster GPU Instances. Contact Info.

AWS 67

AWS Cloud Benchmarking Storage 67

All Things Distributed

JULY 13, 2010

Cluster Computer Instances for Amazon EC2 are a new instance type specifically designed for High Performance Computing applications. Other industries using Amazon EC2 for HPC-style workloads include pharmaceuticals, oil exploration, industrial and automotive design, media and entertainment, and more. blog comments powered by Disqus.

Cloud 118

Cloud AWS Automotive Latency 118

All Things Distributed

JULY 3, 2011

blog comments powered by Disqus. he posts material that doesnt belong on this blog or on twitter. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. Contact Info.

AWS 67

AWS Storage Big Data Cloud 67

All Things Distributed

APRIL 23, 2010

blog comments powered by Disqus. he posts material that doesnt belong on this blog or on twitter. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. Contact Info.

AWS 60

AWS Storage Cloud Big Data 60

All Things Distributed

MAY 18, 2010

Its use cases span the whole storage spectrum; from enterprise database backup to media distribution, from consumer file storage to genomics datasets, from image serving for websites to telemetry storage for NASA. The same goes for durability; core to the design of S3 is that we go to great lengths to never, ever lose a single bit.

Storage 71

Storage Cloud AWS Scalability 71

All Things Distributed

FEBRUARY 24, 2010

Amazon SimpleDB has launched today with a new set of features giving the customer more control over which consistency and concurrency models to use in their database operations. These new features will make it easier to transition those applications to SimpleDB that are designed with traditional database tools in mind.

AWS 69

AWS Latency Database Scalability 69

All Things Distributed

DECEMBER 5, 2010

We have designed Route 53 to propagate updates very quickly and give the customer the tools to find out when all changes have been propagated. If you want to learn more about Route 53 visit [link] and read the blog post at the AWS Developer weblog. blog comments powered by Disqus. Driving down the cost of Big-Data analytics.

Cloud 117

Cloud Internet Internet AWS 117

All Things Distributed

SEPTEMBER 30, 2010

blog comments powered by Disqus. he posts material that doesnt belong on this blog or on twitter. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. Contact Info.

AWS 75

AWS Storage Cloud Best Practices 75

All Things Distributed

APRIL 19, 2010

As Dan Rayburn wrote on his Business of Video blog, "Amazon will be in the drivers seat to own the market for small and medium sized content owners who need simple delivery at a great price.". blog comments powered by Disqus. he posts material that doesnt belong on this blog or on twitter. Contact Info. Werner Vogels.

Media 60

Media AWS Storage Big Data 60