n0derunner

SEPTEMBER 15, 2023

Background For the past few months engineers from Nutanix have been participating in the MLPerftm Storage benchmark which is designed to measure the storage performance required for ML training workloads. appeared first on n0derunner.

Benchmarking 59

Benchmarking Storage Servers Engineering 59

InfoQ

MAY 15, 2023

A horizontally scalable exabyte-scale blob storage system which operates out of multiple regions, Magic Pocket is used to store all of Dropbox’s data. Adopting SMR technology and erasure codes, the system has extremely high durability guarantees but is cheaper than operating in the cloud. By Facundo Agriel

Storage 111

Storage Systems Scalability Cloud 111

Dynatrace

MAY 7, 2024

Fluent Bit is a telemetry agent designed to receive data (logs, traces, and metrics), process or modify it, and export it to a destination. However, you can also use Fluent Bit as a processor because you can perform various actions on the data. However, Fluent Bit was designed to be lightweight, multi-threaded, and run on edge devices.

Best Practices 248

Best Practices IoT Metrics Storage 248

Dynatrace

OCTOBER 17, 2023

In the recently published Gartner® “ Critic al Capabilities for Application Performance Monitoring and Observability,” Dynatrace scored highest for the IT Operations Use Case (4.15/5) This is accomplished by using service monitoring and anomaly detection for early-warning notifications of performance issues.” 5) in the Gartner report.

Monitoring 230

Monitoring Artificial Intelligence Performance Analytics 230

The Morning Paper

MARCH 8, 2020

Building an elastic query engine on disaggregated storage , Vuppalapati, NSDI’20. This paper describes the design decisions behind the Snowflake cloud-based data warehouse. have altered the many assumptions that guided the design and optimization of the Snowflake system. From shared-nothing to disaggregation.

Storage 112

Storage Engineering Cache Serverless 112

InfoQ

OCTOBER 14, 2023

times the storage per node than before” without a price change. Google recently announced various improvements to Cloud Spanner, its distributed, decoupled relational database service with a “50% increase in throughput and 2.5 By Steef-Jan Wiggers

Google 100

Google Storage Cloud Database 100

Dynatrace

JANUARY 23, 2024

These are the goals of AI observability and data observability, a key theme at Dynatrace Perform 2024 , the observability provider’s annual conference, which takes place in Las Vegas from January 29 to February 1, 2024. Join us at Dynatrace Perform 2024 , either on-site or virtuall y, to explore these themes further.

Performance 190

Performance DevOps Innovation Artificial Intelligence 190

Percona

DECEMBER 11, 2023

A quick configuration change may do the trick in improving the performance of your AWS RDS for MySQL instance. Here, we will discuss a notable new feature in Amazon RDS, the Dedicated Log Volume (DLV), that has been introduced to boost database performance. Who can benefit from DLV? and later v10 versions MySQL: 8.0.28

AWS 95

AWS Benchmarking Performance Traffic 95

Scalegrid

MAY 13, 2020

It’s architecture was specially designed to manage large-scale data warehouses and business intelligence workloads by giving you the ability to spread your data out across a multitude of servers. Greenplum uses an MPP database design that can help you develop a scalable, high performance deployment. At a glance – TLDR.

Big Data 321

Big Data Database Artificial Intelligence Open Source 321

Dynatrace

JANUARY 17, 2024

An AI observability strategy—which monitors IT system performance and costs—may help organizations achieve that balance. AI requires more compute and storage. Training AI data is resource-intensive and costly, again, because of increased computational and storage requirements. AI performs frequent data transfers.

Strategy 226

Strategy Artificial Intelligence Storage Cloud 226

The Morning Paper

JANUARY 28, 2020

Narrowing the gap between serverless and its state with storage functions , Zhang et al., Shredder is " a low-latency multi-tenant cloud store that allows small units of computation to be performed directly within storage nodes. " High performance. SoCC’19. "Narrowing Introducing Shredder.

Serverless 99

Serverless Storage Latency Cloud 99

Brendan Gregg

JULY 4, 2021

It's an exciting time for developments in computer performance, not just for the BPF technology (which I often [write about]) but also for processors with 3D stacking and cloud vendor CPUs (e.g., This was a chance to talk about other things I've been working on, such as the present and future of hardware performance.

Performance 140

Performance Latency Hardware Storage 140

Dynatrace

JANUARY 31, 2024

Organizations need to ensure their solutions meet security and privacy requirements through certified high-performance filtering, masking, routing, and encryption technologies while remaining easy to configure and operate. Such transformations can reduce storage costs by 99%. Addressing security requirements.

Analytics 197

Analytics Processing Transportation Storage 197

Scalegrid

MARCH 28, 2024

In this comparison of Redis vs Memcached, we strip away the complexity, focusing on each in-memory data store’s performance, scalability, and unique features. Redis and Memcached both provide high performance with sub-millisecond response times. Choosing between Redis and Memcached hinges on specific application requirements.

Cache 130

Cache Storage Scalability Architecture 130

Percona Community

OCTOBER 23, 2020

Tech that makes our databases perform better. In this blog post we’re going to see those technologies at work to give us awesome block storage performance with flexibility and simple operations. It’s a new generation in storage software, designed for super high speed low latency NVMe devices. Why is SPDK exciting?

Storage 52

Storage Latency Speed Database 52

Dynatrace

OCTOBER 17, 2023

Dynatrace has developed the purpose-built data lakehouse, Grail , eliminating the need for separate management of indexes and storage. All data is readily accessible without storage tiers, such as costly solid-state drives (SSDs). No storage tiers, no archiving or retrieval from archives, and no indexing or reindexing.

Retail 241

Retail Storage Best Practices Architecture 241

Dynatrace

OCTOBER 4, 2022

A data lakehouse addresses these limitations and introduces an entirely new architectural design. This architecture offers rich data management and analytics features (taken from the data warehouse model) on top of low-cost cloud storage systems (which are used by data lakes). High-performance analytics—no indexing required.

Artificial Intelligence 272

Artificial Intelligence Analytics Storage Architecture 272

Dynatrace

NOVEMBER 11, 2022

Therefore, they need an environment that offers scalable computing, storage, and networking. Hyperconverged infrastructure (HCI) is an IT architecture that combines servers, storage, and networking functions into a unified, software-centric platform to streamline resource management. What is hyperconverged infrastructure?

Infrastructure 205

Infrastructure Storage Virtualization Network 205

Particular Software

APRIL 12, 2021

Azure Storage Queues is a basic yet robust queueing service available on the Azure platform. For example, it lacks concepts like topics and subscriptions to build publisher and subscriber topologies—these are capabilities NServiceBus adds on top of Azure Storage Queues. Here’s how it works.

Azure 52

Azure Storage Transportation Java 52

Dynatrace

OCTOBER 7, 2021

This means you no longer have to provision, scale, and maintain servers to run your applications, databases, and storage systems. Instead of worrying about infrastructure management functions, such as capacity provisioning and hardware maintenance, teams can focus on application design, deployment, and delivery. Reliability.

Serverless 285

Serverless AWS Lambda Storage 285

Dynatrace

APRIL 28, 2023

This means compromising between keeping data available as long as possible for analysis while juggling the costs and overhead of storage, archiving, and retrieval. This allows you to design data management and retention policies based on individual requirements, starting from days of retention up to a decade.

Analytics 218

Analytics Storage Infrastructure Metrics 218

Brendan Gregg

FEBRUARY 28, 2023

At USENIX SREcon22 APAC I gave the opening keynote on the future of computer performance, rounding up the latest developments and making predictions of where I see things heading. This talk originated from my updates to [Systems Performance 2nd Edition], and this was the first time I've given this talk in person! Or even on a plane.

Performance 110

Performance Latency Cache Virtualization 110

Percona

APRIL 17, 2023

Note that the intent of tuning the settings is not exclusively about improving performance but also enhancing the high availability and resilience of the MongoDB database. For bare metals, any algorithm among deadline or noop (the performance difference between them is imperceptible) will be better than CFQ.

Best Practices 52

Best Practices Design Tuning Database 52

Dynatrace

OCTOBER 4, 2022

Data warehouses offer a single storage repository for structured data and provide a source of truth for organizations. Unlike data warehouses, however, data is not transformed before landing in storage. A data lakehouse provides a cost-effective storage layer for both structured and unstructured data. Query language.

Artificial Intelligence 213

Artificial Intelligence Storage Analytics Government 213

InfoQ

JUNE 26, 2023

Microsoft has recently unveiled several new features for Azure Cosmos DB to enhance cost efficiency, boost performance, and increase elasticity. These features are burst capacity, hierarchical partition keys, serverless container storage of 1 TB, and priority-based execution. By Steef-Jan Wiggers

Azure 85

Azure Serverless Performance Storage 85

HammerDB

APRIL 19, 2023

Predicting application performance is a difficult art, but an important one when choosing the target deployment environment. In this blog, we aim to call out some key considerations when trying to assess MySQL performance for your application. For storage, FIO is generally used.

Performance 54

Performance Benchmarking Cache Storage 54

All Things Distributed

MAY 19, 2017

In many, high-throughput, OLTP style applications the database plays a crucial role to achieve scale, reliability, high-performance and cost efficiency. In a paper published this week at SIGMOD'17 , the Amazon Aurora team presents the design considerations for the new database engine and how they addressed them.

Database 117

Database Design Cloud Storage 117

Dynatrace

JUNE 6, 2023

Therefore, they are built to be non-persistent by design. Traditional storage solutions were not created to address these requirements, which are common among modern deployments. Traditional storage solutions were not created to address these requirements, which are common among modern deployments.

Open Source 197

Open Source Storage Analytics Innovation 197

The Morning Paper

NOVEMBER 5, 2019

File systems unfit as distributed storage backends: lessons from 10 years of Ceph evolution Aghayev et al., In this case, the assumption that a distributed storage backend should clearly be layered on top of a local file system. What is a distributed storage backend? SOSP’19. This is not surprising in hindsight.

Storage 64

Storage Systems Hardware Efficiency 64

DZone

JANUARY 20, 2023

Do Not Be Misled Designing and implementing a scalable graph database system has never been a trivial task. It has been a norm to perceive that distributed databases use the method of adding cheap PC(s) to achieve scalability (storage and computing) and attempt to store data once and for all on demand.

Scalability 210

Scalability Big Data Hardware Internet 210

Percona

SEPTEMBER 1, 2023

While there is no magic bullet for MySQL performance tuning, there are a few areas that can be focused on upfront that can dramatically improve the performance of your MySQL installation. What are the Benefits of MySQL Performance Tuning? Experiencing database performance issues?

Tuning 52

Tuning Database Performance Hardware 52

Scalegrid

MARCH 8, 2024

Use the ‘–log-bin’ option to designate a base name for the binary log files. Following creation, best practices dictate carefully transferring the secure backup file and its associated binary log SQL files to your designated recovery host. Such bloated logs could also become problematic in terms of performance implications.

Database 162

Database Strategy Servers Best Practices 162

Dynatrace

AUGUST 11, 2022

Before an organization moves to function as a service, it’s important to understand how it works, its benefits and challenges, its effect on scalability, and why cloud-native observability is essential for attaining peak performance. Infrastructure as a service (IaaS) handles compute, storage, and network resources. What is FaaS?

Development 255

Development Serverless Best Practices Lambda 255

Scalegrid

JANUARY 12, 2024

Simply put, it’s the set of computational tasks that cloud systems perform, such as hosting databases, enabling collaboration tools, or running compute-intensive algorithms. Storage is a critical aspect to consider when working with cloud workloads. What is workload in cloud computing?

Cloud 130

Cloud Virtualization Storage Efficiency 130

Scalegrid

JANUARY 5, 2024

It was initially designed to mitigate the limitations of file management systems, including slow operations, inadequate security, and substantial data redundancy. It optimizes performance through monitoring tools and expert support, offers scalable solutions, and ensures high data security with end-to-end encryption and compliance adherence.

Efficiency 147

Efficiency Storage Database Scalability 147

InfoQ

JULY 7, 2023

Facundo Agriel dives into the architecture of Magic Pocket, some early key design patterns, and the challenges of operating such a system at this scale. By Facundo Agriel

Systems 52

Systems Storage Architecture Design 52

All Things Distributed

APRIL 20, 2021

I recently posted about Amazon S3 and how it’s evolved over the last 15 years since we launched the service in 2006 as “storage for the internet.” ” We built S3 because we knew customers wanted to store backups, videos, and images for applications like e-commerce web sites.

Internet 218

Internet Internet Storage Design 218