The Morning Paper

OCTOBER 26, 2020

Helios: hyperscale indexing for the cloud & edge , Potharaju et al., On the surface this is a paper about fast data ingestion from high-volume streams, with indexing to support efficient querying. These two narratives of reference architecture and ingestion/indexing system are interwoven throughout the paper. PVLDB’20.

Cloud 104

Cloud Big Data Latency Architecture 104

Abhishek Tiwari

DECEMBER 27, 2017

Kubernetes has emerged as go to container orchestration platform for data engineering teams. In 2018, a widespread adaptation of Kubernetes for big data processing is anitcipated. Organisations are already using Kubernetes for a variety of workloads [1] [2] and data workloads are up next. Key challenges. Performance.

Big Data 63

Big Data Storage Benchmarking Hardware 63

Dynatrace

APRIL 20, 2023

With more automated approaches to log monitoring and log analysis, however, organizations can gain visibility into their applications and infrastructure efficiently and with greater precision—even as cloud environments grow. Further, business leaders must often determine whether the data is relevant for the business and if they can afford it.

Analytics 177

Analytics Infrastructure Storage Efficiency 177

Dynatrace

OCTOBER 4, 2022

While data lakes and data warehousing architectures are commonly used modes for storing and analyzing data, a data lakehouse is an efficient third way to store and analyze data that unifies the two architectures while preserving the benefits of both. What is a data lakehouse? Data warehouses.

Artificial Intelligence 197

Artificial Intelligence Storage Analytics Government 197

The Netflix TechBlog

JANUARY 13, 2022

By Vikram Srivastava and Marcelo Mayworm Netflix has one of the most complex data platforms in the cloud on which our data scientists and engineers run batch and streaming workloads. This has led to a dramatic reduction in the time it takes to detect issues in hardware or bugs in recently rolled out data platform software.

Big Data 238

Big Data Infrastructure Metrics Hardware 238

Dynatrace

OCTOBER 4, 2022

Limited data availability constrains value creation. Modern IT environments — whether multicloud, on-premises, or hybrid-cloud architectures — generate exponentially increasing data volumes. Stop worrying about log data ingest and storage — start creating value instead.

Analytics 223

Analytics Artificial Intelligence Storage Serverless 223

The Netflix TechBlog

JUNE 7, 2021

At much less than 1% of CPU and memory on the instance, this highly performant sidecar provides flow data at scale for network insight. Challenges The cloud network infrastructure that Netflix utilizes today consists of AWS services such as VPC, DirectConnect, VPC Peering, Transit Gateways, NAT Gateways, etc and Netflix owned devices.

Network 325

Network Transportation AWS Cloud 325

Dynatrace

MARCH 24, 2023

By embracing public cloud and hybrid cloud computing environments, IT teams can further accelerate development and automate software deployment and management. Container technology enables organizations to efficiently develop cloud-native applications or to modernize legacy applications to take advantage of cloud services.

Infrastructure 176

Infrastructure Open Source Operating System Cloud 176

Dynatrace

DECEMBER 15, 2022

As more organizations adopt cloud-native technologies, traditional approaches to IT operations have been evolving. Complex cloud computing environments are increasingly replacing traditional data centers. In fact, Gartner estimates that 80% of enterprises will shut down their on-premises data centers by 2025.

Artificial Intelligence 191

Artificial Intelligence DevOps Hardware Virtualization 191

The Morning Paper

MAY 14, 2019

Seer: leveraging big data to navigate the complexity of performance debugging in cloud microservices Gan et al., Seer is an online system that observes the behaviour of cloud applications (using the DeathStarBench microservices for the evaluation) and predicts when QoS violations may be about to occur. ASPLOS’19.

Big Data 45

Big Data Cloud Performance Hardware 45

Dynatrace

JULY 6, 2022

Scripts and procedures usually focus on a particular task, such as deploying a new microservice to a Kubernetes cluster, implementing data retention policies on archived files in the cloud, or running a vulnerability scanner over code before it’s deployed. Big data automation tools. How does IT automation work?

Artificial Intelligence 178

Artificial Intelligence Tuning Strategy Big Data 178

Dynatrace

MARCH 14, 2022

As patient care continues to evolve, IT teams have accelerated this shift from legacy, on-premises systems to cloud technology to more build, test, and deploy software, and fuel healthcare innovation. exemplifies this trend, where cloud transformation and artificial intelligence are popular topics. Emerging security threats.

Healthcare 218

Healthcare Artificial Intelligence Innovation Strategy 218

Dynatrace

FEBRUARY 16, 2023

Grail data lakehouse delivers massively parallel processing for answers at scale Modern cloud-native computing is constantly upping the ante on data volume, variety, and velocity. To solve this problem, Dynatrace launched Grail, its causational data lakehouse , in 2022. Kubernetes makes spans longer,” Ortner explains.

Analytics 176

Analytics Innovation Metrics Database 176

Dynatrace

OCTOBER 17, 2022

Artificial intelligence for IT operations, or AIOps, combines big data and machine learning to provide actionable insight for IT teams to shape and automate their operational strategy. Modern IT operations involve observing networks, cloud resources and applications, endpoint devices, and more. AIOps use cases.

DevOps 185

DevOps Artificial Intelligence Healthcare Innovation 185

ScaleOut Software

JULY 19, 2021

Today’s streaming analytics architectures are not equipped to make sense of this rapidly changing information and react to it as it arrives. This data is also periodically uploaded to a data lake for offline batch analysis that calculates key statistics and looks for big trends that can help optimize operations.

IoT 78

IoT Analytics Big Data Architecture 78

The Netflix TechBlog

MARCH 25, 2019

Building and Scaling Data Lineage at Netflix to Improve Data Infrastructure Reliability, and Efficiency By: Di Lin , Girish Lingappa , Jitender Aswani Imagine yourself in the role of a data-inspired decision maker staring at a metric on a dashboard about to make a critical business decision but pausing to ask a question?—?“Can

Infrastructure 219

Infrastructure Big Data Transportation Architecture 219

Dynatrace

OCTOBER 14, 2021

Gartner defines AIOps as the combination of “big data and machine learning to automate IT operations processes, including event correlation, anomaly detection, and causality determination.” Only deterministic AIOps technology enables fully automated cloud operations across the entire enterprise development lifecycle.

Artificial Intelligence 201

Artificial Intelligence DevOps Innovation Metrics 201

ACM Sigarch

MAY 31, 2023

Introduction Memory systems are evolving into heterogeneous and composable architectures. Heterogeneous and Composable Memory (HCM) offers a feasible solution for terabyte- or petabyte-scale systems, addressing the performance and efficiency demands of emerging big-data applications. The recently announced CXL3.0

Latency 52

Latency Hardware Cache Architecture 52

All Things Distributed

SEPTEMBER 6, 2013

Given that I am originally from the Netherlands I have, of course, a special interest in how Dutch companies are using our cloud services. . But it is not just Dutch entrepreneurs who build their business in the cloud, also traditional Dutch enterprises are moving to the cloud to improve their agility and cost-effectiveness.

Cloud 129

Cloud Energy AWS Healthcare 129

ScaleOut Software

NOVEMBER 24, 2020

However, telematics architectures face challenges in responding to telemetry in real time. Current Telematics Architecture. The volume of incoming telemetry challenges current telematics systems to keep up and quickly make sense of all the data. Challenges for Current Architectures. Solution: Real-Time Digital Twins.

Analytics 52

Analytics Architecture Scalability Software Architecture 52

All Things Distributed

JULY 13, 2010

Expanding the Cloud - Cluster Compute Instances for Amazon EC2. Today, Amazon Web Services took very an important step in unlocking the advantages of cloud computing for a very important application area. Unlocking the benefits of the cloud for the HPC community. Expanding the Cloud - The AWS Storage Gateway. Comments ().

Cloud 118

Cloud AWS Automotive Latency 118

All Things Distributed

DECEMBER 5, 2010

Expanding the Cloud with DNS - Introducing Amazon Route 53. While registrars manage the namespace in the DNS naming architecture, DNS servers are used to provide the mapping between names and the addresses used to identify an access point. Expanding the Cloud - The AWS Storage Gateway. Expanding the Cloud â??

Cloud 117

Cloud Internet Internet AWS 117

ScaleOut Software

APRIL 3, 2020

What’s missing is a flexible, fast, and easy-to-use software system that can be quickly adapted to track these assets in real time and provide immediate answers for logistics managers. These questions can be answered using the latest data as it streams in from the field. What are real-time digital twins and why are they useful here?

Logistics 52

Logistics Analytics Scalability Cloud 52

ScaleOut Software

APRIL 3, 2020

What’s missing is a flexible, fast, and easy-to-use software system that can be quickly adapted to track these assets in real time and provide immediate answers for logistics managers. These questions can be answered using the latest data as it streams in from the field. What are real-time digital twins and why are they useful here?

Logistics 52

Logistics Analytics Scalability Cloud 52

All Things Distributed

DECEMBER 20, 2017

Developments like cloud computing, the internet of things, artificial intelligence, and machine learning are proving that IT has (again) become a strategic business driver. Marketers use big data and artificial intelligence to find out more about the future needs of their customers. More than mere support.

Artificial Intelligence 79

Artificial Intelligence Social Media Logistics AWS 79

All Things Distributed

AUGUST 19, 2011

This team is constantly rethinking the assumptions behind how traditional databases were built and constantly working on building the right database architectures suited for the Cloud environment. For more information: Head of Software Development  . Expanding the Cloud - The AWS Storage Gateway. Expanding the Cloud â??

AWS 75

AWS Database Storage Scalability 75

ScaleOut Software

JUNE 15, 2020

Instead, most applications just sift through the telemetry for patterns that might indicate exceptional conditions and forward the bulk of incoming messages to a data lake for offline scrubbing with a big data tool such as Spark. Maintain State Information for Each Data Source.

Analytics 52

Analytics IoT Lambda Big Data 52

Brendan Gregg

APRIL 30, 2018

Today's LISA attracts attendees working on all sizes of production systems, and its attendees include sysadmins, systems engineers, SREs, DevOps engineers, software engineers, IT managers, security engineers, network administrators, researchers, students, and more.

DevOps 43

DevOps Network Best Practices Programming 43

SQL Server According to Bob

FEBRUARY 15, 2018

Best practices on Building a Big Data Analytics Solution – Michael Rys. If you want to learn about Azure Data Lake, there is no one better. If anything just come to see me explain the architecture which is an amazing innovative piece of software. Modernizing ETL with Azure Data Lake – Michael Rys.

Engineering 40

Engineering Azure Best Practices Servers 40

Brendan Gregg

APRIL 29, 2018

Today's LISA attracts attendees working on all sizes of production systems, and its attendees include sysadmins, systems engineers, SREs, DevOps engineers, software engineers, IT managers, security engineers, network administrators, researchers, students, and more.

DevOps 40

DevOps Network Best Practices Programming 40

ScaleOut Software

JUNE 15, 2020

Instead, most applications just sift through the telemetry for patterns that might indicate exceptional conditions and forward the bulk of incoming messages to a data lake for offline scrubbing with a big data tool such as Spark. Maintain State Information for Each Data Source.

Analytics 52

Analytics IoT Lambda Big Data 52

All Things Distributed

APRIL 27, 2011

To our shareowners: Random forests, naïve Bayesian estimators, RESTful services, gossip protocols, eventual consistency, data sharding, anti-entropy, Byzantine quorum, erasure coding, vector clocks. Look inside a current textbook on software architecture, and youll find few patterns that we dont apply at Amazon.

Technology 151

Technology Technology AWS Storage 151

All Things Distributed

NOVEMBER 15, 2010

Expanding the Cloud - Adding the Incredible Power of the Amazon EC2 Cluster GPU Instances. These trade-offs have even impacted the way the lowest level building blocks in our computer architectures have been designed. All Things Distributed. Werner Vogels weblog on building scalable and robust distributed systems. Comments ().

AWS 136

AWS Latency Programming Architecture 136

All Things Distributed

NOVEMBER 15, 2016

Previously, I wrote about Amazon QuickSight , a new service targeted at business users that aims to simplify the process of deriving insights from a wide variety of data sources quickly, easily, and at a low cost. QuickSight is a very fast, cloud-powered, business intelligence service for the 1/10th the cost of old-guard BI solutions.

Analytics 152

Analytics Availability Media Social Media 152