Uber Engineering

AUGUST 3, 2018

Data powers Uber’s global marketplace, enabling more reliable and seamless user experiences across our products for riders, … The post Databook: Turning Big Data into Knowledge with Metadata at Uber appeared first on Uber Engineering Blog.

Big Data 110

Big Data Transportation Engineering Storage 110

Dynatrace

OCTOBER 4, 2022

Log management and analytics is an essential part of any organization’s infrastructure, and it’s no secret the industry has suffered from a shortage of innovation for several years. Several pain points have made it difficult for organizations to manage their data efficiently and create actual value.

Analytics 231

Analytics Artificial Intelligence Storage Serverless 231

Dynatrace

MARCH 24, 2023

Containers enable developers to package microservices or applications with the libraries, configuration files, and dependencies needed to run on any infrastructure, regardless of the target system environment. Mesos supports several container orchestration engines and can launch Docker containers independently of the Docker daemon.

Infrastructure 182

Infrastructure Open Source Operating System Cloud 182

Dynatrace

JANUARY 16, 2023

The study analyzes factual Kubernetes production data from thousands of organizations worldwide that are using the Dynatrace Software Intelligence Platform to keep their Kubernetes clusters secure, healthy, and high performing. Kubernetes infrastructure models differ between cloud and on-premises. Kubernetes moved to the cloud in 2022.

Open Source 340

Open Source Java Operating System Programming 340

Dynatrace

JULY 6, 2022

As organizations continue to adopt multicloud strategies, the complexity of these environments grows, increasing the need to automate cloud engineering operations to ensure organizations can enforce their policies and architecture principles. Big data automation tools. Batch process automation.

Artificial Intelligence 185

Artificial Intelligence Tuning Strategy Big Data 185

The Netflix TechBlog

JUNE 7, 2021

At much less than 1% of CPU and memory on the instance, this highly performant sidecar provides flow data at scale for network insight. Challenges The cloud network infrastructure that Netflix utilizes today consists of AWS services such as VPC, DirectConnect, VPC Peering, Transit Gateways, NAT Gateways, etc and Netflix owned devices.

Network 325

Network Transportation AWS Cloud 325

Dynatrace

DECEMBER 15, 2022

ITOps is an IT discipline involving actions and decisions made by the operations team responsible for an organization’s IT infrastructure. Besides the traditional system hardware, storage, routers, and software, ITOps also includes virtual components of the network and cloud infrastructure. What is ITOps? ITOps vs. AIOps.

Artificial Intelligence 198

Artificial Intelligence DevOps Hardware Virtualization 198

Uber Engineering

APRIL 5, 2018

Three years ago, Uber Engineering adopted Hadoop as the storage ( HDFS ) and compute ( YARN ) infrastructure for our organization’s big data analysis.

Systems 109

Systems Big Data Storage Infrastructure 109

ScaleOut Software

JULY 19, 2021

We are increasingly surrounded by intelligent IoT devices, which have become an essential part of our lives and an integral component of business and industrial infrastructures. Today’s streaming analytics architectures are not equipped to make sense of this rapidly changing information and react to it as it arrives.

IoT 78

IoT Analytics Big Data Architecture 78

Uber Engineering

AUGUST 14, 2019

Once identified, … The post Less is More: Engineering Data Warehouse Efficiency with Minimalist Design appeared first on Uber Engineering Blog. In our experience, optimizing for operational efficiency requires answering one key question: for which tables does the maintenance cost supersede utility?

Efficiency 103

Efficiency Engineering Design Storage 103

Dynatrace

MARCH 14, 2022

AIOps (or “AI for IT operations”) uses artificial intelligence so that big data can help IT teams work faster and more effectively. Over the past few years, the company has undergone a digital transformation, migrating to a hybrid, cloud-native environment built on Amazon Web Services and a microservices architecture.

Healthcare 225

Healthcare Artificial Intelligence Innovation Strategy 225

High Scalability

APRIL 14, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! this is going to be a challenging journey for any backend engineer! Learn to balance architecture trade-offs and design scalable enterprise-level software. Please apply here.

Education 105

Education Software Engineering Scalability Engineering 105

High Scalability

MAY 12, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! this is going to be a challenging journey for any backend engineer! Learn to balance architecture trade-offs and design scalable enterprise-level software. Please apply here.

Education 102

Education Software Engineering Scalability Engineering 102

High Scalability

APRIL 28, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! this is going to be a challenging journey for any backend engineer! Learn to balance architecture trade-offs and design scalable enterprise-level software. Please apply here.

Education 102

Education Software Engineering Scalability Engineering 102

High Scalability

MAY 26, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! this is going to be a challenging journey for any backend engineer! Created by former senior-level AWS engineers of 15 years. Try out their platform. Please apply here.

Education 100

Education AWS Software Engineering Scalability 100

Dynatrace

OCTOBER 14, 2021

Gartner defines AIOps as the combination of “big data and machine learning to automate IT operations processes, including event correlation, anomaly detection, and causality determination.” The second challenge with traditional AIOps centers around the data processing cycle. But what is AIOps, exactly? What is AIOps?

Artificial Intelligence 208

Artificial Intelligence DevOps Innovation Metrics 208

All Things Distributed

JUNE 26, 2016

In June 2015, Amazon Web Services announced that it would launch a new AWS infrastructure region in India. Market innovators and change agents need a comprehensive infrastructure platform that can reliably scale on-demand. Advanced problem solving that connects big data with machine learning.

AWS 90

AWS Cloud Healthcare Blockchain 90

All Things Distributed

DECEMBER 8, 2016

Earlier this year, Amazon Web Services (AWS) announced it would launch a new AWS infrastructure region in Montreal, Quebec. The new Canada (Central) Region offers a robust suite of infrastructure, management, and developer services that can enable innovators to deploy market-leading applications. in the coming year. Performance.

AWS 155

AWS Cloud Lambda Innovation 155

Uber Engineering

OCTOBER 30, 2018

Cluster management, a common software infrastructure among technology companies, aggregates compute resources from a collection of physical hosts into a shared resource pool, amplifying compute power and allowing for the flexible use of data center hardware.

Hardware 103

Hardware Infrastructure Engineering Technology 103

The Netflix TechBlog

MARCH 25, 2019

Building and Scaling Data Lineage at Netflix to Improve Data Infrastructure Reliability, and Efficiency By: Di Lin , Girish Lingappa , Jitender Aswani Imagine yourself in the role of a data-inspired decision maker staring at a metric on a dashboard about to make a critical business decision but pausing to ask a question?—?“Can

Infrastructure 219

Infrastructure Big Data Transportation Architecture 219

Uber Engineering

DECEMBER 17, 2019

Uber’s busy 2019 included our billionth delivery of an Uber Eats order, 24 million miles covered by bike and scooter riders on our platform, and trips to top destinations such as the Empire State Building, the Eiffel Tower, and the … The post Uber’s Data Platform in 2019: Transforming Information to Intelligence appeared first on Uber Engineering (..)

Engineering 130

Engineering Big Data Infrastructure Analytics 130

Uber Engineering

OCTOBER 30, 2018

Cluster management, a common software infrastructure among technology companies, aggregates compute resources from a collection of physical hosts into a shared resource pool, amplifying compute power and allowing for the flexible use of data center hardware.

Hardware 62

Hardware Infrastructure Engineering Technology 62

All Things Distributed

SEPTEMBER 6, 2013

Shell leverages AWS for big data analytics to help achieve these goals. Shell''s scientists, especially the geophysicists and drilling engineers, frequently use cloud computing to run models. In addition, its robust architecture supports ten times as many scientists, all working simultaneously.

Cloud 129

Cloud Energy AWS Healthcare 129

Brendan Gregg

APRIL 30, 2018

USENIX’s LISA conference is the premier event for topics in production system engineering. And we believe that everyone has valuable stories to share: Your tales of trial and error, analysis and workarounds, or architecture migrations, can happen at any company. ## Submit Your Work! Join us for 3 days in Nashville at LISA'18.

DevOps 43

DevOps Network Best Practices Programming 43

Brendan Gregg

APRIL 29, 2018

USENIX’s LISA conference is the premier event for topics in production system engineering. And we believe that everyone has valuable stories to share: Your tales of trial and error, analysis and workarounds, or architecture migrations, can happen at any company. ## Submit Your Work! Join us for 3 days in Nashville at LISA'18.

DevOps 40

DevOps Network Best Practices Programming 40

All Things Distributed

JULY 13, 2010

Customers with complex computational workloads such as tightly coupled, parallel processes, or with applications that are very sensitive to network performance, can now achieve the same high compute and networking performance provided by custom-built infrastructure while benefiting from the elasticity, flexibility and cost advantages of Amazon EC2.

Cloud 118

Cloud AWS Automotive Latency 118

The Netflix TechBlog

MAY 26, 2020

Cloud Network Insight is a suite of solutions that provides both operational and analytical insight into the Cloud Network Infrastructure to address the identified problems. As with any sustainable engineering design, focusing on simplicity is very important. And excellent logging is needed for debugging purposes and supportability.

Network 150

Network Tuning AWS Big Data 150

All Things Distributed

DECEMBER 5, 2010

There is more than one Werner Vogels in this world and although I never get emails, snail mail or phones calls for any of my peers, I am sure they are somewhat frustrated if they type in our name in a search engine :-). DNS is an absolutely critical piece of the internet infrastructure. Driving down the cost of Big-Data analytics.

Cloud 117

Cloud Internet Internet AWS 117

The Netflix TechBlog

OCTOBER 27, 2020

By Tianlong Chen and Ioannis Papapanagiotou Netflix has more than 195 million subscribers that generate petabytes of data everyday. Data scientists and engineers collect this data from our subscribers and videos, and implement data analytics models to discover customer behaviour with the goal of maximizing user joy.

Latency 243

Latency Storage Big Data Tuning 243

The Netflix TechBlog

DECEMBER 21, 2020

This article will list some of the use cases of AutoOptimize, discuss the design principles that help enhance efficiency, and present the high-level architecture. Some of the optimizations are prerequisites for a high-performance data warehouse.

Storage 203

Storage Latency Efficiency Data Engineering 203

All Things Distributed

NOVEMBER 15, 2010

Building general purpose architectures has always been hard; there are often so many conflicting requirements that you cannot derive an architecture that will serve all, so we have often ended up focusing on one side of the requirements that allow you to serve that area really well. From CPU to GPU.

AWS 136

AWS Latency Programming Architecture 136

The Netflix TechBlog

SEPTEMBER 10, 2019

Our A/B tests range across UI, algorithms, messaging, marketing, operations, and infrastructure changes. Instead of relying on engineers to productionize scientific contributions, we’ve made a strategic bet to build an architecture that enables data scientists to easily contribute.

Metrics 215

Metrics Architecture Infrastructure Innovation 215

All Things Distributed

NOVEMBER 15, 2016

As I mentioned, we live in a world where massive volumes of data are being generated, every day, from connected devices, websites, mobile apps, and customer applications running on top of AWS infrastructure. Put simply, data is not always readily available and accessible to organizational end users. Enter Amazon QuickSight.

Analytics 152

Analytics Availability Media Social Media 152

The Netflix TechBlog

NOVEMBER 3, 2023

Democratizing Stream Processing @ Netflix By Guil Pires , Mark Cho , Mingliang Liu , Sujay Jain Data powers much of what we do at Netflix. On the Data Platform team, we build the infrastructure used across the company to process data at scale.

Processing 265

Processing Engineering Infrastructure Latency 265