DZone

OCTOBER 23, 2023

Reference data sources don’t always require ACID transaction capabilities, rather need support for fast read queries often based on simple data access patterns, and event-driven architecture to ensure the target systems remain up-to-date.

AWS 266

AWS Performance Database Architecture 266

Dynatrace

JANUARY 4, 2024

Traditional analytics and AI systems rely on statistical models to correlate events with possible causes. Causal AI is a different approach that goes beyond event correlations to understand the underlying reasons for trends and patterns. That’s where causal AI can help. Causal AI is particularly effective in observability.

Government 205

Government Analytics Benchmarking Storage 205

Strategic Tech

FEBRUARY 15, 2023

FThis article describes a pattern we have observed and applied in multi-team-scope architecture modernization initiatives, the Architecture Modernization Enabling Team (AMET). An AMET is a type of architecture enabling team that coordinates and upskills all teams and stakeholders in the modernization initiative.

Architecture 97

Architecture Innovation Airlines Technology 97

InfoQ

FEBRUARY 1, 2024

Peter Lawrey discusses achieving vertical scalability by minimizing accidental complexity and using an event-driven architecture. By Peter Lawrey

Scalability 73

Scalability Efficiency Systems Architecture 73

Uber Engineering

SEPTEMBER 23, 2021

With this new ability came new challenges that needed to be solved at Uber, such as systems for ad auctions, bidding, attribution, reporting, and more. This article focuses on how we … The post Real-Time Exactly-Once Ad Event Processing with Apache Flink, Kafka, and Pinot appeared first on Uber Engineering Blog.

Processing 145

Processing Engineering Systems Architecture 145

Strategic Tech

SEPTEMBER 30, 2020

One of the biggest time costs in software development is understanding how a system works. Systems are getting more complex yet our ability to understand them doesn’t seem to be growing at the same rate. As we continuously develop software systems, the complexity slowly increases and we don’t fully realise it.

Architecture 75

Architecture Design Programming Software 75

The Netflix TechBlog

NOVEMBER 14, 2023

Late-arriving data is essentially delayed data due to system retries, network delays, batch processing schedules, system outages, delayed upstream workflows, or reconciliation in source systems. Using fixed lookback windows to always reprocess data, assuming that most late-arriving events will occur within that window.

Data Engineering 264

Data Engineering Engineering Processing Games 264

Dynatrace

JANUARY 22, 2024

While traditional AI relies on finding correlations in data, causal AI aims to determine the precise underlying mechanisms that drive events and outcomes. That’s why causal AI use cases abound for organizations looking to build more reliable and transparent AI systems. However, correlation does not imply causation.

Artificial Intelligence 212

Artificial Intelligence Healthcare Retail Government 212

Strategic Tech

APRIL 1, 2022

I’ve just created a new kata which you and your team/friends can use to practice your architecture and domain-driven design skills. This kata is split into four sections that address different aspects of architecting software systems. First with a domain quiz, and then by slicing up the provided event storm into domains.

Architecture 52

Architecture Software Architecture Strategy Design 52

DZone

NOVEMBER 7, 2023

In the labyrinth of data-driven architectures, the challenge of data integration—fusing data from disparate sources into a coherent, usable form — stands as one of the cornerstones. Systems were not equipped to handle multiple tasks simultaneously. However, it would be a mistake to dismiss batch processing as an antiquated approach.

Processing 177

Processing Architecture Technology Technology 177

Scalegrid

AUGUST 16, 2021

Data-driven applications span a wide breadth of complexity, from simple microservices to real-time event-driven systems under significant load. Modern application architectures such as the JAMstack enforce the separation […]. Guest post by Ben Hagan from PolyScale.ai

Cache 279

Cache DevOps Architecture Systems 279

The Netflix TechBlog

MAY 4, 2023

Behind the scenes, a myriad of systems and services are involved in orchestrating the product experience. These backend systems are consistently being evolved and optimized to meet and exceed customer and product expectations. This blog series will examine the tools, techniques, and strategies we have utilized to achieve this goal.

Traffic 339

Traffic Latency Tuning Systems 339

The Netflix TechBlog

APRIL 27, 2023

Engineers want their alerting system to be realtime, reliable, and actionable. A few years ago, we were paged by our SRE team due to our Metrics Alerting System falling behind — critical application health alerts reached engineers 45 minutes late! In other words, false positives are bad but false negatives are the absolute worst!

Storage 288

Storage Cache Metrics Database 288

DZone

SEPTEMBER 11, 2023

False positives are instances where the system incorrectly identifies a regular event as an anomaly, leading to unnecessary investigative efforts and operational delays. Overview of the System Architecture Data Stream This is the continuous flow of data that the engine reviews.

Engineering 161

Engineering Healthcare Architecture Monitoring 161

DZone

SEPTEMBER 14, 2023

Such a gigantic log analysis system is part of their cybersecurity management. For the need of real-time monitoring, threat tracing, and alerting, they require a log analytic system that can automatically collect, store, analyze, and visualize logs and event records.

Analytics 124

Analytics Storage Architecture Scalability 124

All Things Distributed

AUGUST 17, 2012

Werner Vogels weblog on building scalable and robust distributed systems. Back-to-Basics Weekend Reading - Staged Event-Driven Architecture. Staged Event-Driven Architecture. Several of the principles from this paper have made it into systems I have since built. All Things Distributed. Comments ().

Architecture 98

Architecture AWS Internet Internet 98

Dynatrace

FEBRUARY 26, 2024

During the Dynatrace annual user conference, Perform , the Partner Summit session returned, welcoming Global System Integrators (GSIs), Hyperscalers, and Cloud Solutions Partners to participate.

Artificial Intelligence 199

Artificial Intelligence IoT Programming Innovation 199

Strategic Tech

JULY 21, 2020

Most organisations go through an architecture modernisation effort at some point as their systems drift into a state of intolerable maintenance costs and they diverge too far from modern technological advances. What architecture will be optimal for enabling that business vision? How are we going to deliver the new architecture?

Architecture 98

Architecture Design Software Architecture Strategy 98

Dynatrace

JANUARY 23, 2024

Perform serves yearly as the marquis Dynatrace event to unveil new announcements, learn about new uses and best practices, and meet with peers and partners alike. What will the new architecture be? More so than ever before, organizations are investing in cloud migration and cloud modernization to lower total cost of ownership (TCO).

Cloud 217

Cloud Artificial Intelligence Performance AWS 217

Dynatrace

MAY 2, 2024

Additionally, blind spots in cloud architecture are making it increasingly difficult for organizations to balance application performance with a robust security posture. As organizations train generative AI systems with critical data, they must be aware of the security and compliance risks.

Artificial Intelligence 130

Artificial Intelligence Cloud Best Practices DevOps 130

O'Reilly Software

JUNE 12, 2018

The benefits of modeling data as events as a mechanism to evolve our software systems. In fact, before we even had the word microservices in our lexicon, back when it was just good old-fashioned service-oriented architecture, we were talking about data: how to access it, where it lives, who “owns” it.

Database 72

Database Cache Architecture Latency 72

All Things Distributed

JUNE 10, 2016

If you are looking for more examples there are the Lambda Serverless Reference Architectures that can serve as the blueprint for building your own serverless applications. Mobile Backend Serverless Reference Architecture. Real-time File Processing Serverless Reference Architecture. IoT Backend Serverless Reference Architecture.

Lambda 109

Lambda Serverless AWS Architecture 109

Dynatrace

SEPTEMBER 5, 2023

Additionally, monitoring allows for proactive maintenance and optimization, leading to improved system performance and user experience. An extension-built topology model allows you to easily navigate between all the entities that make up your database servers or cluster architecture and review specific statistics in their proper context.

Database 206

Database Metrics Monitoring Servers 206

Dynatrace

NOVEMBER 17, 2023

Dynatrace observability architecture can be classified into three layers: Orchestration (Dynatrace) CI/CD toolset (Jenkins / Chef / Puppet / Bamboo, etc.) Log reading can be facilitated by installing OneAgent or employing a log ingestion system to ingest the logs into Dynatrace.

Innovation 228

Innovation Architecture Code Processing 228

All Things Distributed

MAY 10, 2016

If you are looking for more examples there are the Lambda Serverless Reference Architectures that can serve as the blueprint for building your own serverless applications. Mobile Backend Serverless Reference Architecture. Real-time File Processing Serverless Reference Architecture. IoT Backend Serverless Reference Architecture.

Lambda 105

Lambda Serverless AWS Architecture 105

Dynatrace

DECEMBER 21, 2023

Evaluating these on three levels—data center, host, and application architecture (plus code)—is helpful. You might optimize your cooling system or move your data center to a colder region with reduced cooling demands. Application architectures might not be conducive to rehosting. Unfortunately, it’s not that simple.

Cloud 214

Cloud Energy Best Practices Cache 214

O'Reilly Software

FEBRUARY 5, 2019

Watch highlights from expert talks covering cloud-native programming, software architecture career advice, and more. People from across the software architecture world are coming together in New York for the O'Reilly Software Architecture Conference. Below you'll find links to highlights from the event.

Software Architecture 100

Software Architecture Architecture Software Software 100

The Netflix TechBlog

OCTOBER 15, 2019

Delta is an eventual consistent, event driven, data synchronization and enrichment platform. Another thread or process is constantly polling events from the log table and writes them to one or multiple datastores, optionally removing events from the log table after acknowledged by all datastores.

Transportation 192

Transportation Architecture Processing Storage 192

Dynatrace

MAY 24, 2023

Dynatrace is proud to provide deep monitoring support for Azure Linux as a container host operating system (OS) platform for Azure Kubernetes Services (AKS) to enable customers to operate efficiently and innovate faster. Visibility of the lifecycle of k8s objects (pods, services, nodes) and associated events is needed. Performance.

Azure 231

Azure Monitoring Operating System Virtualization 231

Dynatrace

APRIL 19, 2023

The new architecture enables more granularity in permission management and provides the dynamics necessary to serve modern access management use cases. We’ve replaced the role-based permission system with security policies that provide fine-grained control over service access and introduce new dynamic controls for user authorization.

Storage 214

Storage Analytics Systems Architecture 214

The Netflix TechBlog

AUGUST 30, 2021

In the Device Management Platform, this is achieved by having device updates be event-sourced through the control plane to the cloud so that NTS will always have the most up-to-date information about the devices available for testing. The RAE is configured to be effectively a router that devices under test (DUTs) are connected to.

Latency 213

Latency Traffic Transportation Hardware 213

Dynatrace

FEBRUARY 8, 2021

Also, these modern, cloud-native architectures produce an immense volume, velocity, and variety of data. Logs and events play an essential role in this mix; they include critical information which can’t be found anywhere else, like details on transactions, processes, users and environment changes.

Cloud 261

Cloud Azure Analytics Open Source 261

The Netflix TechBlog

FEBRUARY 18, 2022

To this end, we developed a Rapid Event Notification System (RENO) to support use cases that require server initiated communication with devices in a scalable and extensible manner. In this blog post, we will give an overview of the Rapid Event Notification System at Netflix and share some of the learnings we gained along the way.

Systems 334

Systems Traffic Architecture Mobile 334

O'Reilly Software

JUNE 13, 2019

Experts explore software architecture security, design heuristics, Next Architecture, and more. Experts from across the software architecture world are coming together in San Jose for the O'Reilly Software Architecture Conference. Below you'll find links to highlights from the event. Next Architecture.

Software Architecture 89

Software Architecture Architecture Software Software 89

Particular Software

JUNE 7, 2017

You can attempt to avoid this bitter fate by using architectural patterns like event-driven architecture. When you build a system of discrete services that communicate via events, you limit the complexity of each service by reducing coupling. What do I mean by event? Often, commands and events come in pairs.

Architecture 68

Architecture Code Processing Systems 68

Dynatrace

JANUARY 31, 2024

GenAI is prone to erratic behavior due to unforeseen data scenarios or underlying system issues. Figure 1: Sample RAG architecture While this approach significantly improves the response quality of GenAI applications, it also introduces new challenges.

Cache 209

Cache Azure Infrastructure Monitoring 209

Dynatrace

MAY 3, 2022

To adapt, many are turning to AIOps and other automation technologies to solve the complex issues that accompany cloud-native architecture. While shifting to a multicloud model offers many advantages, the change also introduces more data and systems to track and a complex matrix of systems to manage.

Cloud 182

Cloud DevOps Architecture Technology 182