ScaleOut Software

AUGUST 18, 2018

In previous blogs , we have explored the power of the digital twin model for stateful stream-processing. Digital twins are software abstractions that track the behavior of individual devices in IoT applications. They combine an event handling function with state information about each device.

IoT 40

IoT Ecommerce Analytics Latency 40

cdemi

DECEMBER 4, 2016

Modern software usually involves running tasks that invoke services. The service retrieves the messages from the queue and processes them. Maximizing Scalability : Number of Queues and Services can be varied to meet Demand. If the service is subjected to intermittent heavy loads, it can cause performance or reliability issues.

Design 47

Design Azure Scalability Latency 47

O'Reilly Software

OCTOBER 23, 2018

Likewise, the term "architect" didn't enter popular usage to describe a role in the software field until the late 1990s. It's a given that we must design a system, including a local software architecture, that actually runs, that is "solid." the user experience and design) and within the software construction itself.

Strategy 100

Strategy Architecture Technology Technology 100

cdemi

MAY 6, 2016

I have decided to write a series on some common and modern Design Patterns that are useful in today's horizontally scalable (such as cloud-hosted) applications. The Competing Consumer Pattern enables a system to process multiple messages concurrently to optimize throughput, to improve scalability and availability.

Design 40

Design Cache Scalability Architecture 40

O'Reilly

OCTOBER 19, 2021

This is both frustrating for companies that would prefer making ML an ordinary, fuss-free value-generating function like software engineering, as well as exciting for vendors who see the opportunity to create buzz around a new category of enterprise software. What does a modern technology stack for streamlined ML processes look like?

DevOps 138

DevOps Software Engineering Infrastructure Open Source 138

Percona

MARCH 21, 2023

Tenants Multi-tenancy is a software architecture pattern where a single instance of a software application serves multiple tenants, allowing them to share resources like storage, processing power, and memory while maintaining separate, secure access to their respective data.

Serverless 125

Serverless Software Architecture Virtualization Database 125

The Netflix TechBlog

MAY 19, 2023

A cleanup process to prune stale relationships from the database. The Beginning of SpiceDB Caveats The SpiceDB community had already explored integrating SpiceDB with Open Policy Agent (OPA) and concluded it strayed too far from Zanzibar’s core promise of global horizontal scalability with strong consistency.

Cache 249

Cache Google Open Source Systems 249

The Netflix TechBlog

DECEMBER 3, 2019

mainly because of mundane reasons related to software engineering. We heard many stories about difficulties related to data access and basic data processing. While a typical machine learning workflow running on Metaflow touches only a small shard of this warehouse, it can still process terabytes of data.

Open Source 232

Open Source AWS Infrastructure Energy 232

The Netflix TechBlog

DECEMBER 3, 2019

mainly because of mundane reasons related to software engineering. We heard many stories about difficulties related to data access and basic data processing. While a typical machine learning workflow running on Metaflow touches only a small shard of this warehouse, it can still process terabytes of data.

Open Source 134

Open Source AWS Infrastructure Energy 134

cdemi

MAY 8, 2016

It may be beneficial to have the application prepopulate the cache as part of the startup process. An item in the data store may be changed at any time by an external process, and this change might not be reflected in the cache until the next time the item is loaded into the cache. Consistency.

Cache 54

Cache Design C++ Processing 54

O'Reilly Software

APRIL 4, 2018

Considerations for setting the architectural foundations for a fast data platform. We live in the era of the connected experience, where our daily interactions with the world can be digitized, collected, processed, and analyzed to generate valuable insights. Accumulating all this data to process overnight is not an option anymore.

Architecture 80

Architecture Scalability Google Operating System 80

All Things Distributed

APRIL 27, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Look inside a current textbook on software architecture, and youll find few patterns that we dont apply at Amazon. The storage systems weve pioneered demonstrate extreme scalability while maintaining tight control over performance, availability, and cost.

Technology 151

Technology Technology AWS Storage 151

Strategic Tech

JULY 18, 2018

There are thousands of ways we can shape the software systems we build and organise our teams around them. The products themselves are software systems which grow harder and harder to reason about as they scale and age. Lack of accountability leads to unmaintainable software that grows harder and harder to change.

Design 40

Design Architecture Software Architecture Systems 40

All Things Distributed

NOVEMBER 15, 2010

Werner Vogels weblog on building scalable and robust distributed systems. From financial processing and traditional oil & gas exploration HPC applications to integrating complex 3D graphics into online and mobile applications, the applications of GPU processing appear to be limitless. All Things Distributed. Comments ().

AWS 136

AWS Latency Programming Architecture 136

cdemi

DECEMBER 7, 2016

Building on the Queue-Based Load Leveling Pattern , the Competing Consumer Pattern enables a system to process multiple messages concurrently to optimize throughput, to improve scalability and availability, and to balance the workload. Systems that use the Competing Consumer Pattern should ensure that message processing is idempotent.

Design 40

Design Scalability Systems Processing 40