InfoQ

JANUARY 29, 2024

The team at RevenueCat created an open-source memcache client that provides several advanced features. RevenueCat extensively uses caching to improve the availability and performance of its product API while ensuring consistency. The company shared its techniques to deliver the platform, which can handle over 1.2 By Rafal Gancarz

Cache 104

Cache Open Source Availability Performance 104

Brendan Gregg

FEBRUARY 28, 2023

My personal opinion is that I don't see a widespread need for more capacity given horizontal scaling and servers that can already exceed 1 Tbyte of DRAM; bandwidth is also helpful, but I'd be concerned about the increased latency for adding a hop to more memory. Ford, et al., “TCP

Performance 110

Performance Latency Cache Virtualization 110

Dynatrace

JULY 6, 2020

It enables you to use popular open-source frameworks such as Hadoop, Spark, and Kafka in Azure cloud environments. This means that you can improve performance, scale your application, and enable complex application architectures like IaaS and PaaS, on premise + cloud, or multi-cloud hybrid environments.

Azure 147

Azure Cloud Big Data Virtualization 147

The Netflix TechBlog

MARCH 13, 2020

SVT-AV1: open-source AV1 encoder and decoder by Andrey Norkin , Joel Sole , Mariana Afonso , Kyle Swanson, Agata Opalach , Anush Moorthy , Anne Aaron SVT-AV1 is an open-source AV1 codec implementation hosted on GitHub [link] under a BSD + patent license. We welcome more contributors to the project.

Open Source 154

Open Source Efficiency C++ Speed 154

Brendan Gregg

JULY 2, 2021

opensnoop Files opened table. biolatency Disk I/O latency histogram heat map. runqlat CPU scheduler latency heat map. The architecture is: While the bpftrace binary is installed on all the target systems, the bpftrace tools (text files) live on a web server and are pushed out when needed.

Latency 145

Latency Cache Energy Systems 145

Brendan Gregg

FEBRUARY 28, 2023

My personal opinion is that I don't see a widespread need for more capacity given horizontal scaling and servers that can already exceed 1 Tbyte of DRAM; bandwidth is also helpful, but I'd be concerned about the increased latency for adding a hop to more memory. Ford, et al., “TCP

Performance 52

Performance Latency Cache Virtualization 52

HammerDB

MAY 5, 2019

Note that the main developer of HammerDB is Intel employee (#IAMINTEL) however HammerDB is a personal open source project and HammerDB has no optimization whatsoever for a database running on any particular architecture. In the recent MySQL 8.0.16

Testing 48

Testing Tuning Latency Storage 48

Scalegrid

MARCH 28, 2024

Key Takeaways Redis offers complex data structures and additional features for versatile data handling, while Memcached excels in simplicity with a fast, multi-threaded architecture for basic caching needs. However, Redis, with its single-threaded architecture, may encounter bottlenecks with large numbers of concurrent connections.

Cache 130

Cache Storage Scalability Architecture 130

Testsigma

AUGUST 24, 2020

Maintaining test data in a central repository and then accessing it from different test environments may ease this problem but may introduce other issues like network latency in tests. The right combination of open source and paid test frameworks/tools can be used rather than relying solely on open-source test frameworks/tools.

Testing 69

Testing Open Source Scalability Latency 69

Abhishek Tiwari

JULY 12, 2018

Here are 8 fallacies of data pipeline The pipeline is reliable Topology is stateless Pipeline is infinitely scalable Processing latency is minimum Everything is observable There is no domino effect Pipeline is cost­-effective Data is homogeneous The pipeline is reliable The inconvenient truth is that pipeline is not reliable.

Latency 63

Latency Analytics Scalability Engineering 63

The Morning Paper

FEBRUARY 23, 2020

Our approach differs substantially by (1) providing economic incentives for data to be contributed and integrated into existing schemas, (2) offering a SQL interface instead of graph based approaches, (3) including the computational and storage infrastructure in the architectural vision. An embodiment for structured data for IoT.

Blockchain 94

Blockchain Internet Internet IoT 94

CSS - Tricks

JULY 25, 2019

Lighthouse is an open source project run by a dedicated team from Google Chrome. If you’re interested in a high-level overview of Lighthouse architecture, read this guide from the official repository. Estimated Input Latency. Estimated Input Latency. What is Google Lighthouse? They are: Time to Interactive ( TTI ).

Google 71

Google Engineering Speed Mobile 71

Uber Engineering

MARCH 12, 2017

With the evolution of storage formats like Apache Parquet and Apache ORC and query engines like Presto and Apache Impala , the Hadoop ecosystem has the potential to become a general-purpose, unified serving layer for workloads that can tolerate latencies … The post Hudi: Uber Engineering’s Incremental Processing Framework on Apache Hadoop appeared (..)

Processing 105

Processing Latency Storage Engineering 105

Brendan Gregg

JULY 2, 2021

If you're using an open source monitoring platform, first check if it already has a BPF agent. opensnoop Files opened table 3. biolatency Disk I/O latency histogram heat map 5. runqlat CPU scheduler latency heat map 10. Think of this as "version 1": A starting point that's pretty useful.

Open Source 40

Open Source Latency Cache Energy 40

Dynatrace

MAY 13, 2022

Apache Cassandra is an open-source, distributed, NoSQL database. Because of its scalability and distributed architecture, thousands of companies trust it to run their cloud and hybrid-based workloads at high availability without compromising performance. Add your visualization to your dashboards for easy access and sharing.

Azure 236

Azure Latency Metrics Infrastructure 236

Abhishek Tiwari

DECEMBER 27, 2017

For instance, scatter/gather pattern can be used to implement a MapReduce like batch processing architecture on top of Kubernetes. To address this, the open source community is now trying to develop Kubernetes custom schedulers specifically optimised for big data workloads.

Big Data 63

Big Data Storage Benchmarking Hardware 63

Dynatrace

OCTOBER 1, 2021

As dynamic systems architectures increase in complexity and scale, IT teams face mounting pressure to track and respond to conditions and issues across their multi-cloud environments. Organizations usually implement observability using a combination of instrumentation methods including open-source instrumentation tools, such as OpenTelemetry.

Metrics 363

Metrics Open Source Monitoring Infrastructure 363

Scalegrid

JUNE 4, 2020

As an open source database, it’s a highly popular choice for enterprise applications looking to modernize their infrastructure and reduce their total cost of ownership, along with startup and developer applications looking for a powerful, flexible and cost-effective database to work with. Compare Latency. Compare Pricing.

Database 230

Database Latency Benchmarking Performance 230

The Morning Paper

MAY 12, 2019

An open-source benchmark suite for microservices and their hardware-software implications for cloud & edge systems Gan et al., It’s a pretty impressive effort to pull together and make available in open source (not yet available as I write this) such a suite, and I’m sure explains much of the long list of 24 authors on this paper.

Open Source 61

Open Source Hardware Benchmarking Systems 61

The Netflix TechBlog

FEBRUARY 20, 2019

Today we are excited to announce latency heatmaps and improved container support for our on-host monitoring solution?—?Vector?—?to Vector is open source and in use by multiple companies. Remotely view real-time process scheduler latency and tcp throughput with Vector and eBPF What is Vector? to the broader community.

Performance 211

Performance Latency Open Source Metrics 211

Percona

JULY 17, 2023

Not really open source : Even the MongoDB Community version is not open source; it’s source-available and is under the SSPL license (introduced by MongoDB itself). Organizations have access to the source code, allowing them to make changes and enhancements as needed. Couchbase — No.

Open Source 74

Open Source Database Scalability Software 74

Dynatrace

JUNE 25, 2021

As a strong supporter of open source and open standards, I’m aware that the wide availability of standards, open-source tools, and some newly coined terms are causing a lot of confusion. I also serve as co-chair of the W3C Distributed Tracing working group , which works on standards like W3C Trace Context.

Open Source 199

Open Source Monitoring Google Systems 199

The Netflix TechBlog

JULY 26, 2021

Netflix is known for its loosely coupled microservice architecture and with a global studio footprint, surfacing and connecting the data from microservices into a studio data catalog in real time has become more important than ever. Most of the business views created on top of the Iceberg tables can tolerate a few minutes of latency.

Big Data 253

Big Data Government Analytics Processing 253

The Netflix TechBlog

NOVEMBER 2, 2021

It has become a de facto standard for perceptual quality measurements within Netflix and, thanks to its open-source nature , throughout the video industry. The Reloaded system is a well-matured and scalable system, but its monolithic architecture can slow down rapid innovation. via bug fixes). We call this system Cosmos.

Media 171

Media Innovation Metrics Latency 171

Percona

NOVEMBER 9, 2023

At its core, Kubernetes (often abbreviated as K8s) is an open source tool that automates the deployment, scaling, and management of containerized applications. This makes it much easier for organizations to transition to open source solutions, reduce costs, and avoid vendor lock-in.

Efficiency 100

Efficiency Cloud Healthcare Open Source 100

Dynatrace

DECEMBER 22, 2021

Metrics are measures of critical system values, such as CPU utilization or average write latency to persistent storage. They are particularly important in distributed systems, such as microservices architectures. Observability platforms are becoming essential as the complexity of cloud-native architectures increases.

Artificial Intelligence 177

Artificial Intelligence Metrics Architecture DevOps 177

All Things Distributed

JULY 20, 2015

Today, I want to explore the Amazon ECS architecture and what this architecture enables. This architecture affords Amazon ECS high availability, low latency, and high throughput because the data store is never pessimistically locked. At first, the team looked to a few open-source solutions (e.g.,

Latency 166

Latency Architecture AWS Open Source 166

The Morning Paper

OCTOBER 26, 2020

Helios also serves as a reference architecture for how Microsoft envisions its next generation of distributed big-data processing systems being built. These two narratives of reference architecture and ingestion/indexing system are interwoven throughout the paper. Why do we need a new reference architecture? Emphasis mine ).

Cloud 104

Cloud Big Data Latency Architecture 104

High Scalability

SEPTEMBER 8, 2018

RISELabs , those wonderfully innovative folks over at Berkeley, have uplifted their Anna datatabase —a shared-nothing, thread-per-core architecture to achieve lightning-fast speeds by avoiding all coordination mechanisms—to become cloud-aware. While database neogenesis has slowed down considerably, it has not gone necrotic.

Storage 199

Storage Performance AWS Efficiency 199

The Netflix TechBlog

DECEMBER 17, 2019

Therefore, dumps are needed to capture the full state of a source. There are several open source CDC projects, often using the same underlying libraries, database APIs, and protocols. We want to support these systems as a source so that they can provide their data for further consumption. DBLog High Level Architecture.

Database 197

Database Traffic Transportation Open Source 197

The Netflix TechBlog

DECEMBER 17, 2019

Therefore, dumps are needed to capture the full state of a source. There are several open source CDC projects, often using the same underlying libraries, database APIs, and protocols. We want to support these systems as a source so that they can provide their data for further consumption. DBLog High Level Architecture.

Database 212

Database Traffic Transportation Open Source 212

The Netflix TechBlog

JULY 12, 2019

Old Gatekeeper Architecture This model had several problems associated with it: This process was completely I/O bound and put a lot of load on upstream systems. New Gatekeeper Architecture With this model, liveness evaluation is conceptually separated from the data retrieval from upstream systems.

Cache 183

Cache Architecture Latency Engineering 183

Percona

JANUARY 24, 2023

Individual processes generate WAL records, and latency is very crucial for transactions. Configured checkpoint frequency, the chance of checkpoints triggered by WAL generation, storage performance, acceptable CPU overhead, type of CPU architecture, and many other factors. The actual benefit of compression depends on many factors.

Database 115

Database Benchmarking Open Source Latency 115

The Netflix TechBlog

AUGUST 13, 2020

For example, a latency increase is less critical than error rate increase and some error codes are less critical than others. We start with Spinnaker , our open source delivery platform. Client metrics and QoE changes. Alerts triggered by our alerting platform. A healthy Netflix service enables us to entertain the world.

Monitoring 210

Monitoring Tuning Traffic Metrics 210

Smashing Magazine

AUGUST 3, 2023

For Performance Monitoring Many of you reading this may already be familiar with Next.js, but it is a popular open-source JavaScript framework that allows us to monitor our website’s performance in real-time. This allowed users to fetch assets from a geographically closer source, substantially reducing processing time.

Performance 74

Performance Cache Traffic Metrics 74

Adrian Cockcroft

APRIL 5, 2023

This proposal seeks to define a standard for real-time carbon and energy data as time-series data that would be accessed alongside and synchronized with the existing throughput, utilization and latency metrics that are provided for the components and applications in computing environments.

Energy 52

Energy Metrics Cloud Operating System 52

All Things Distributed

AUGUST 20, 2015

When using relational databases, traversing relationships requires expensive table JOIN operations, causing significantly increased latency as table size and query complexity grow. Titan has a pluggable storage architecture, using existing NoSQL databases as underlying storage for the graph data. Enter graph databases.

Database 134

Database Logistics Availability Social Media 134