ScaleOut Software

AUGUST 18, 2018

In previous blogs , we have explored the power of the digital twin model for stateful stream-processing. Digital twins are software abstractions that track the behavior of individual devices in IoT applications. They combine an event handling function with state information about each device.

IoT 45

IoT Ecommerce Analytics Latency 45

cdemi

DECEMBER 4, 2016

The service retrieves the messages from the queue and processes them. Requests from a number of different tasks, which can be generated at a highly variable rate , can be passed to the service through the same message queue and will be processed at a constant rate that the service can work at.

Design 47

Design Azure Scalability Latency 47

cdemi

DECEMBER 7, 2016

Building on the Queue-Based Load Leveling Pattern , the Competing Consumer Pattern enables a system to process multiple messages concurrently to optimize throughput, to improve scalability and availability, and to balance the workload. Systems that use the Competing Consumer Pattern should ensure that message processing is idempotent.

Design 40

Design Scalability Systems Processing 40

ScaleOut Software

AUGUST 18, 2018

In previous blogs , we have explored the power of the digital twin model for stateful stream-processing. Digital twins are software abstractions that track the behavior of individual devices in IoT applications. They combine an event handling function with state information about each device.

IoT 40

IoT Ecommerce Analytics Latency 40

Strategic Tech

JULY 18, 2018

?? Primary Heuristic 2: Align with Business Domain Unless we’re a startup that is constantly pivoting, macro parts of our business processes will remain stable over long periods of time. Putting Heuristics into Practice We still don’t have a flowchart yet for designing sociotechnical systems but we’re bringing more structure to the process.

Design 40

Design Architecture Software Architecture Systems 40

cdemi

MAY 6, 2016

I have decided to write a series on some common and modern Design Patterns that are useful in today's horizontally scalable (such as cloud-hosted) applications. The Competing Consumer Pattern enables a system to process multiple messages concurrently to optimize throughput, to improve scalability and availability.

Design 40

Design Cache Scalability Architecture 40

O'Reilly Software

OCTOBER 23, 2018

It's a given that we must design a system, including a local software architecture, that actually runs, that is "solid." The ones I focus on most are scalability, availability, maintainability, manageability, monitorability, extensibility, interoperability, portability, security, and performance. Solid doesn't mean inflexible.

Strategy 100

Strategy Architecture Technology Technology 100

The Netflix TechBlog

MAY 19, 2023

A cleanup process to prune stale relationships from the database. The Beginning of SpiceDB Caveats The SpiceDB community had already explored integrating SpiceDB with Open Policy Agent (OPA) and concluded it strayed too far from Zanzibar’s core promise of global horizontal scalability with strong consistency.

Cache 249

Cache Google Open Source Systems 249

O'Reilly

OCTOBER 19, 2021

While there isn’t an authoritative definition for the term, it shares its ethos with its predecessor, the DevOps movement in software engineering: by adopting well-defined processes, modern tooling, and automated workflows, we can streamline the process of moving from development to robust production deployments.

DevOps 138

DevOps Software Engineering Infrastructure Open Source 138

The Netflix TechBlog

DECEMBER 3, 2019

mainly because of mundane reasons related to software engineering. We heard many stories about difficulties related to data access and basic data processing. While a typical machine learning workflow running on Metaflow touches only a small shard of this warehouse, it can still process terabytes of data.

Open Source 232

Open Source AWS Infrastructure Energy 232

cdemi

MAY 8, 2016

It may be beneficial to have the application prepopulate the cache as part of the startup process. An item in the data store may be changed at any time by an external process, and this change might not be reflected in the cache until the next time the item is loaded into the cache. Consistency.

Cache 54

Cache Design C++ Processing 54

The Netflix TechBlog

DECEMBER 3, 2019

mainly because of mundane reasons related to software engineering. We heard many stories about difficulties related to data access and basic data processing. While a typical machine learning workflow running on Metaflow touches only a small shard of this warehouse, it can still process terabytes of data.

Open Source 134

Open Source AWS Infrastructure Energy 134

All Things Distributed

APRIL 27, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Look inside a current textbook on software architecture, and youll find few patterns that we dont apply at Amazon. The storage systems weve pioneered demonstrate extreme scalability while maintaining tight control over performance, availability, and cost.

Technology 151

Technology Technology AWS Storage 151

All Things Distributed

NOVEMBER 15, 2010

Werner Vogels weblog on building scalable and robust distributed systems. From financial processing and traditional oil & gas exploration HPC applications to integrating complex 3D graphics into online and mobile applications, the applications of GPU processing appear to be limitless. All Things Distributed. Comments ().

AWS 136

AWS Latency Programming Architecture 136