InfoQ

JULY 7, 2023

Facundo Agriel dives into the architecture of Magic Pocket, some early key design patterns, and the challenges of operating such a system at this scale. By Facundo Agriel

Systems 52

Systems Storage Architecture Design 52

InfoQ

JUNE 16, 2023

Datadog created a dedicated data ingestion architecture offering exactly-once semantics for their third-generation event store, Husky. The event-driven architecture (EDA) can accommodate bursts in traffic in the multi-tenant platform with reasonable ingestion latency and acceptable operational costs. By Rafal Gancarz

Architecture 55

Architecture Scalability Latency Traffic 55

All Things Distributed

MAY 19, 2017

We decided to use our inventive powers to design and build a new database engine that would give database systems such as MySQL and PostgreSQL reliability and performance at scale, at a level that could serve even the most demanding OLTP applications.

Database 117

Database Design Cloud Storage 117

Dynatrace

JUNE 9, 2023

When extended to include application security practices, DevOps becomes DevSecOps and includes security-focused criteria, such as security by design and making security a critical release criterion. Research has found that 99% of organizations have embraced a multicloud architecture.

DevOps 177

DevOps Analytics Storage Metrics 177

The Morning Paper

AUGUST 29, 2019

Choosing a cloud DBMS: architectures and tradeoffs Tan et al., As it is infeasible to test every OLAP system runnable on AWS, we chose widely-used systems that represented a variety of architectures and cost models. The design space. VLDB’19. Key findings. Serverless o?erings Query performance.

Architecture 76

Architecture Cloud Storage Serverless 76

Scalegrid

FEBRUARY 8, 2024

A distributed storage system is foundational in today’s data-driven landscape, ensuring data spread over multiple servers is reliable, accessible, and manageable. Understanding distributed storage is imperative as data volumes and the need for robust storage solutions rise.

Storage 130

Storage Systems Big Data Azure 130

Uber Engineering

AUGUST 14, 2019

Maintaining Uber’s large-scale data warehouse comes with an operational cost in terms of ETL functions and storage. Once identified, … The post Less is More: Engineering Data Warehouse Efficiency with Minimalist Design appeared first on Uber Engineering Blog.

Efficiency 103

Efficiency Engineering Design Storage 103

ACM Sigarch

AUGUST 3, 2023

(Editor’s Note: This post was submitted as a rebuttal to Andrew Chien’s July 24 SIGARCH Blog Post ) The recent post “ Why Embodied Carbon is a poor Architecture Design metric, and Operational Carbon remains an important Problem ” by Prof. Unlike Prof. Chien’s argument: It costs only $1.4B Therefore, the $1.4B

Energy 98

Energy Architecture Hardware Metrics 98

Scalegrid

JANUARY 5, 2024

It was initially designed to mitigate the limitations of file management systems, including slow operations, inadequate security, and substantial data redundancy. It comprises a collection of interrelated data and a set of software tools that aid in the access, processing, and management of data.

Efficiency 147

Efficiency Storage Database Scalability 147

Strategic Tech

MAY 24, 2019

Over the past few years I have focused much of my learning and work choices around learning about the design of sociotechnical systems?—?how how to design software architectures and organise teams around them. As part of good design, we should consider the effects of changing all of these variables.

Design 45

Design Software Architecture Strategy Technology 45

DZone

JUNE 12, 2019

The workload could refer to anything from an increase in users, storage, or a number of transactions. An architect should focus on designing a scalable software architecture from the early phase of the product life cycle.

Scalability 100

Scalability Software Architecture Storage Architecture 100

Dynatrace

MAY 26, 2021

In the keynote by Christina Yakomin and Steve Prazenica from Vanguard, the presenters recounted their journey from a monolith with alert-based incident reporting and no positive health signals to an observable microservice architecture. Trace-based sampling can help you save storage costs. So, the design of OTel allows for flexibility.

AWS 205

AWS Strategy Storage Metrics 205

VoltDB

JULY 10, 2018

This goal has been attempted to be addressed from the beginning of time: think of Object Oriented Programming, Service Oriented Architecture, Enterprise Service Bus and now Microservices. VoltDB is a distributed in-memory database that is designed to power real-time intelligent decisions on fast moving streaming data.

Architecture 40

Architecture Scalability Ecommerce Latency 40

VoltDB

JULY 10, 2018

This goal has been attempted to be addressed from the beginning of time: think of Object Oriented Programming, Service Oriented Architecture, Enterprise Service Bus and now Microservices. VoltDB is a distributed in-memory database that is designed to power real-time intelligent decisions on fast moving streaming data.

Architecture 40

Architecture Scalability Ecommerce Latency 40

InfoQ

DECEMBER 29, 2023

Zendesk reduced its data storage costs by over 80% by migrating from DynamoDB to a tiered storage solution using MySQL and S3. The company considered different storage technologies and decided to combine the relational database and the object store to strike a balance between querybility and scalability while keeping the costs down.

Storage 134

Storage Scalability Database Technology 134

Scalegrid

JANUARY 8, 2024

AI algorithms embedded in cloud architecture automate repetitive processes, streamlining workloads and reducing the chance of human error. With a multi-cloud architecture, Scalegrid offers the flexibility and competitive edge necessary for AI applications in the rapidly evolving tech environment.

Artificial Intelligence 147

Artificial Intelligence Cloud Scalability Analytics 147

Dynatrace

JUNE 7, 2023

Kubernetes also gives developers freedom of choice when selecting operating systems, container runtimes, storage engines, and other key elements for their Kubernetes environments. Without having to worry about underlying infrastructure concerns, such as storage, security, and lifecycle management, developers can focus on writing code.

Open Source 199

Open Source Social Media Infrastructure Operating System 199

High Scalability

NOVEMBER 14, 2018

Five years ago when Google published The Datacenter as a Computer: Designing Warehouse-Scale Machines it was a manifesto declaring the world of computing had changed forever. The world is still changing, so Google published a new edition: The Datacenter as a Computer: Designing Warehouse-Scale Machines, Third Edition.

Design 54

Design Education Google Internet 54

Dynatrace

MAY 26, 2021

In the keynote by Christina Yakomin and Steve Prazenica from Vanguard, the presenters recounted their journey from a monolith with alert-based incident reporting and no positive health signals to an observable microservice architecture. Trace-based sampling can help you save storage costs. So, the design of OTel allows for flexibility.

AWS 147

AWS Strategy Storage Metrics 147

Dynatrace

DECEMBER 15, 2022

ITOps refers to the process of acquiring, designing, deploying, configuring, and maintaining equipment and services that support an organization’s desired business outcomes. Besides the traditional system hardware, storage, routers, and software, ITOps also includes virtual components of the network and cloud infrastructure.

Artificial Intelligence 195

Artificial Intelligence DevOps Hardware Virtualization 195

The Netflix TechBlog

DECEMBER 21, 2020

At this scale, we can gain a significant amount of performance and cost benefits by optimizing the storage layout (records, objects, partitions) as the data lands into our warehouse. We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits.

Storage 203

Storage Latency Efficiency Data Engineering 203

ScaleOut Software

JUNE 29, 2018

Having just concluded participation in another In-Memory Computing Summit , it has become even more clear to me that the key to mainstream adoption of in-memory computing software platforms is architecture — the root of a platform’s value to applications. These priorities tend to push the architecture to the back burner.

Software Architecture 40

Software Architecture Architecture Software Software 40

ScaleOut Software

JUNE 29, 2018

Having just concluded participation in another In-Memory Computing Summit , it has become even more clear to me that the key to mainstream adoption of in-memory computing software platforms is architecture — the root of a platform’s value to applications. These priorities tend to push the architecture to the back burner.

Software Architecture 40

Software Architecture Architecture Software Software 40

Dynatrace

APRIL 19, 2023

Creating an ecosystem that facilitates data security and data privacy by design can be difficult, but it’s critical to securing information. When organizations focus on data privacy by design, they build security considerations into cloud systems upfront rather than as a bolt-on consideration. Why perform exclusion at two points?

Design 184

Design Storage Programming Analytics 184

Percona

DECEMBER 4, 2023

Percona software is designed for peak performance, uncompromised security, limitless scalability, and disaster-proofed availability. Release highlights include: Docker images are now available for x86_64 architectures. Download Percona Distribution for PostgreSQL 16.1 Percona Distribution for PostgreSQL 15.5

Open Source 79

Open Source Azure Servers Programming 79

Dynatrace

SEPTEMBER 13, 2023

This is a set of best practices and guidelines that help you design and operate reliable, secure, efficient, cost-effective, and sustainable systems in the cloud. Storing frequently accessed data in faster storage, usually in-memory caching, improves data retrieval speed and overall system performance. Beyond

AWS 240

AWS Efficiency Azure Cloud 240

Dynatrace

AUGUST 8, 2019

One of our design principles is to make sure that Dynatrace is as easy to set up and use as possible. In short, over the last several months, we’ve completely rebuilt the architecture of the OneAgent installer for Windows. Smaller network and Dynatrace cluster storage footprint. Dynatrace news. msi-based deployment.

Efficiency 117

Efficiency Network Storage Architecture 117

Brendan Gregg

FEBRUARY 28, 2023

Titus, the Netflix container management platform, is now open source,” [link] Apr 2018 - [Cutress 19] Dr. DDR6: Here's What to Expect in RAM Modules,” [link] Nov 2020 - [Salter 20] Jim Salter, “Western Digital releases new 18TB, 20TB EAMR drives,” [link] Jul 2020 - [Spier 20] Martin Spier, Brendan Gregg, et al.,

Performance 110

Performance Latency Cache Virtualization 110

Smashing Magazine

SEPTEMBER 28, 2022

On design systems, UX, web performance and CSS/JS. Caching partially stores your data and is not used as permanent storage. Using the cache as permanent storage is an anti-pattern. Common Websocket Architecture. Event Sourcing Architecture. There needs to be an underlying architectural pattern that can support it.

Cache 125

Cache Performance Servers Social Media 125

Percona

DECEMBER 27, 2023

While integrating AI and machine learning into cloud-native architectures, there’s an increasing demand from users for AI to be open and collaborative. Well, here we are – with storage autoscaling for databases in Kubernetes, slated for release in Q1, 2024 after a year of hard work.

Cloud 79

Cloud Open Source Strategy Best Practices 79

Adrian Cockcroft

NOVEMBER 26, 2023

The Pantheon in Rome — Extremely sustainable architecture — photo by Adrian I wrote a medium post after AWS re:Invent 2022 summarizing the (lack of) news and all the talks related to Sustainability. Explore an implementation of this architecture with PVH, the parent company of Tommy Hilfiger and Calvin Klein.

AWS 52

AWS Energy IoT Best Practices 52

Dynatrace

JANUARY 23, 2024

While app-centric serverless approaches abstract some of the complexities of cloud-native architecture, as the analyst firm Forrester notes , the next frontier for serverless adoption is at the edge. Organizations increasingly struggle with the challenge of monitoring the explosion of microservices and tools that come with these environments.

Performance 179

Performance DevOps Innovation Artificial Intelligence 179

The Netflix TechBlog

MARCH 10, 2020

Leveraging Hexagonal Architecture We needed to support the ability to swap data sources without impacting business logic , so we knew we needed to keep them decoupled. We decided to build our app based on principles behind Hexagonal Architecture and Uncle Bob’s Clean Architecture. Entities are the domain objects (e.g.,

Architecture 345

Architecture Transportation Java Strategy 345

All Things Distributed

OCTOBER 17, 2016

As with all AWS AZs, the AZs in Ohio each have redundant power, networking, and connectivity, which are designed to be resilient to issues in another AZ. We hope this will be helpful for customers who want to implement backup or disaster recovery architectures and need to transfer large amounts of data between these regions.

AWS 127

AWS Cloud Infrastructure Analytics 127

Percona

AUGUST 17, 2023

PostgreSQL has powerful and advanced features, including asynchronous replication, full-text searches of the database, and native support for JSON-style storage, key-value storage, and XML. It has evolved steadily, however, during 25 years as an open source project. And because you’re covered, you’re also free.

Open Source 78

Open Source Healthcare Social Media Database 78

Percona

JANUARY 19, 2023

Make sure you design the data types correctly while planning for the future growth of the table. It can help us to save costs on storage and backup times. MyRocks: MyRocks is a storage engine developed by Facebook and made open source. It was developed for optimizing data storage and access for big data sets.

Open Source 87

Open Source Storage Database Big Data 87

InfoQ

JUNE 26, 2023

These features are burst capacity, hierarchical partition keys, serverless container storage of 1 TB, and priority-based execution. Microsoft has recently unveiled several new features for Azure Cosmos DB to enhance cost efficiency, boost performance, and increase elasticity. By Steef-Jan Wiggers

Azure 86

Azure Serverless Performance Storage 86