The Netflix TechBlog

MARCH 25, 2019

Now, imagine yourself in the role of a software engineer responsible for a micro-service which publishes data consumed by few critical customer facing services (e.g. You are about to make structural changes to the data and want to know who and what downstream to your service will be impacted.

Infrastructure 219

Infrastructure Big Data Transportation Architecture 219

The Netflix TechBlog

OCTOBER 18, 2022

As Big data and ML became more prevalent and impactful, the scalability, reliability, and usability of the orchestrating ecosystem have increasingly become more important for our data scientists and the company. Meson was based on a single leader architecture with high availability.

Java 202

Java Scalability Traffic Architecture 202

Dynatrace

JUNE 29, 2022

To drive better outcomes using hybrid cloud architectures, it helps to understand their benefits—and how to orchestrate them seamlessly. What is hybrid cloud architecture? Hybrid cloud architecture is a computing environment that shares data and applications on a combination of public clouds and on-premises private clouds.

Infrastructure 187

Infrastructure Cloud Azure AWS 187

Dynatrace

JULY 6, 2022

As organizations continue to adopt multicloud strategies, the complexity of these environments grows, increasing the need to automate cloud engineering operations to ensure organizations can enforce their policies and architecture principles. Big data automation tools. appeared first on Dynatrace blog.

Artificial Intelligence 194

Artificial Intelligence Tuning Strategy Big Data 194

Uber Engineering

JANUARY 15, 2020

Uber leverages real-time analytics on aggregate data to improve the user experience across our products, from fighting fraudulent behavior on Uber Eats to forecasting demand on our platform. .

Engineering 115

Engineering Analytics Big Data Database 115

ACM Sigarch

APRIL 8, 2019

Computer architecture is an important and exciting field of computer science, which enables many other fields (eg. big-data processing, machine learning, quantum computing, and so on). For those of us who pursued computer architecture as a career, this is well understood. Why is that? Should we be alarmed as a community?

Architecture 74

Architecture Open Source Hardware Software Engineering 74

The Netflix TechBlog

NOVEMBER 3, 2023

Democratizing Stream Processing @ Netflix By Guil Pires , Mark Cho , Mingliang Liu , Sujay Jain Data powers much of what we do at Netflix. On the Data Platform team, we build the infrastructure used across the company to process data at scale.

Processing 265

Processing Engineering Infrastructure Latency 265

Uber Engineering

AUGUST 14, 2019

Once identified, … The post Less is More: Engineering Data Warehouse Efficiency with Minimalist Design appeared first on Uber Engineering Blog. In our experience, optimizing for operational efficiency requires answering one key question: for which tables does the maintenance cost supersede utility?

Efficiency 99

Efficiency Engineering Design Storage 99

Dynatrace

JULY 6, 2020

Our customers have frequently requested support for this first new batch of services, which cover databases, big data, networks, and computing. See the health of your big data resources at a glance. Azure HDInsight supports a broad range of use cases including data warehousing, machine learning, and IoT analytics.

Azure 150

Azure Cloud Big Data Virtualization 150

Uber Engineering

APRIL 5, 2018

Three years ago, Uber Engineering adopted Hadoop as the storage ( HDFS ) and compute ( YARN ) infrastructure for our organization’s big data analysis.

Systems 109

Systems Big Data Storage Infrastructure 109

Dynatrace

MARCH 14, 2022

AIOps (or “AI for IT operations”) uses artificial intelligence so that big data can help IT teams work faster and more effectively. Over the past few years, the company has undergone a digital transformation, migrating to a hybrid, cloud-native environment built on Amazon Web Services and a microservices architecture.

Healthcare 232

Healthcare Artificial Intelligence Innovation Strategy 232

Dynatrace

OCTOBER 14, 2021

Gartner defines AIOps as the combination of “big data and machine learning to automate IT operations processes, including event correlation, anomaly detection, and causality determination.” But what is AIOps, exactly? And how can it support your organization? What is AIOps? Challenges of traditional AIOps. AIOps use cases.

Artificial Intelligence 218

Artificial Intelligence DevOps Innovation Metrics 218

Percona

JANUARY 19, 2023

16 years ago, our founder Peter Zaitsev covered this topic and some of the points described there are still valid, and we will cover more on this blog. With disks being faster nowadays and CPU and memory resources being cheaper, we could easily say MySQL can handle TBs of data with good performance.

Open Source 84

Open Source Storage Database Big Data 84

Uber Engineering

DECEMBER 17, 2019

Uber’s busy 2019 included our billionth delivery of an Uber Eats order, 24 million miles covered by bike and scooter riders on our platform, and trips to top destinations such as the Empire State Building, the Eiffel Tower, and the … The post Uber’s Data Platform in 2019: Transforming Information to Intelligence appeared first on Uber Engineering (..)

Engineering 126

Engineering Big Data Infrastructure Analytics 126

All Things Distributed

NOVEMBER 15, 2016

The reality is that many traditional BI solutions are built on top of legacy desktop and on-premises architectures that are decades old. They require teams of data engineers to spend months building complex data models and synthesizing the data before they can generate their first report. Enter Amazon QuickSight.

Analytics 152

Analytics Availability Media Social Media 152

The Netflix TechBlog

DECEMBER 21, 2020

This article will list some of the use cases of AutoOptimize, discuss the design principles that help enhance efficiency, and present the high-level architecture. Some of the optimizations are prerequisites for a high-performance data warehouse. We will publish a follow-up blog post about AutoAnalyze in the future.

Storage 203

Storage Latency Efficiency Data Engineering 203

Uber Engineering

DECEMBER 10, 2019

Michelangelo , Uber’s machine learning (ML) platform, powers machine learning model training across various use cases at Uber, such as forecasting rider demand , fraud detection , food discovery and recommendation for Uber Eats , and improving the accuracy of … The post Productionizing Distributed XGBoost to Train Deep Tree Models with Large (..)

Engineering 104

Engineering Big Data Architecture 104

All Things Distributed

APRIL 27, 2011

Given that I have frequently written about many of these technologies on this blog I asked investor relations to be allowed to reprint it here. To our shareowners: Random forests, naïve Bayesian estimators, RESTful services, gossip protocols, eventual consistency, data sharding, anti-entropy, Byzantine quorum, erasure coding, vector clocks.

Technology 151

Technology Technology AWS Storage 151

Rigor

JANUARY 13, 2020

When it comes to web content, you can easily find what you need through many different paths, from search engines and social media to playlists and blogs, jumping from one source to another with just a tap of a finger. Information Architecture. Web Performance Daybook-Volume-2. Professional Website Performance.

Performance 75

Performance Social Media Website Website Performance 75

Uber Engineering

OCTOBER 30, 2018

Cluster management, a common software infrastructure among technology companies, aggregates compute resources from a collection of physical hosts into a shared resource pool, amplifying compute power and allowing for the flexible use of data center hardware.

Hardware 103

Hardware Infrastructure Engineering Technology 103

ACM Sigarch

NOVEMBER 1, 2019

This blog post gives a glimpse of the computer systems research papers presented at the USENIX Annual Technical Conference (ATC) 2019, with an emphasis on systems that use new hardware architectures. As a consequence, the vast majority of the papers in the past has usually focused on conventional X86 or GPU-accelerated architectures.

Architecture 12

Architecture Hardware Cache Storage 12

All Things Distributed

DECEMBER 8, 2016

It adopted Amazon Redshift, Amazon EMR and AWS Lambda to power its data warehouse, big data, and data science applications, supporting the development of product features at a fraction of the cost of competing solutions. Kik Interactive is a Canadian chat platform with hundreds of millions of users around the globe.

AWS 155

AWS Cloud Lambda Innovation 155

All Things Distributed

NOVEMBER 19, 2010

Big news this week was of course the launch of Cluster GPU instances for Amazon EC2. There were blog posts by Jeff Barr The Cluster GPU Instance and James Hamilton HPC in the Cloud with GPGPUs , as well as my background posting: Expanding the Cloud - Adding the Incredible Power of the Amazon EC2 Cluster GPU Instances. Contact Info.

AWS 67

AWS Cloud Benchmarking Storage 67

All Things Distributed

NOVEMBER 15, 2010

Building general purpose architectures has always been hard; there are often so many conflicting requirements that you cannot derive an architecture that will serve all, so we have often ended up focusing on one side of the requirements that allow you to serve that area really well. From CPU to GPU.

AWS 136

AWS Latency Programming Architecture 136

Uber Engineering

OCTOBER 30, 2018

Cluster management, a common software infrastructure among technology companies, aggregates compute resources from a collection of physical hosts into a shared resource pool, amplifying compute power and allowing for the flexible use of data center hardware.

Hardware 62

Hardware Infrastructure Engineering Technology 62

All Things Distributed

SEPTEMBER 30, 2010

Visiting future customers is equally exiting as you get a change to understand their current architecture, if it is a migration, and how they plan to exploit cloud services in their new setup. blog comments powered by Disqus. he posts material that doesnt belong on this blog or on twitter. Contact Info. Werner Vogels.

AWS 75

AWS Storage Cloud Best Practices 75

All Things Distributed

MARCH 28, 2011

A key part of the Cloud Drive architecture is a Metadata Service that allows customers to quickly search and organize their digital collections within Cloud Drive. If you are an engineer interested in working on Amazon Cloud Drive and related technologies the team has a number of openings and would love to talk to you! Contact Info.

AWS 138

AWS Cloud Storage Internet 138

All Things Distributed

JULY 13, 2010

Additionally, many high-end HPC applications take advantage of knowing their in-house hardware platforms to achieve major speedup by exploiting the specific processor architecture. Cluster Computer Instances are similar to other Amazon EC2 instances but have been specifically engineered to provide high performance compute and networking.

Cloud 118

Cloud AWS Automotive Latency 118

All Things Distributed

DECEMBER 5, 2010

There is more than one Werner Vogels in this world and although I never get emails, snail mail or phones calls for any of my peers, I am sure they are somewhat frustrated if they type in our name in a search engine :-). If you want to learn more about Route 53 visit [link] and read the blog post at the AWS Developer weblog.

Cloud 117

Cloud Internet Internet AWS 117