DZone

AUGUST 6, 2021

Most log management solutions store log data in a database and enable search by storing an index of the data. On a small scale, this isn’t problematic but when dealing with large-scale deployments, organizations end up using lots of computing, storage, and human resources just to manage their indexes.

Storage 189

Storage Database Systems Efficiency 189

Dynatrace

JUNE 9, 2023

ln a world driven by macroeconomic uncertainty, businesses increasingly turn to data-driven decision-making to stay agile. They’re unleashing the power of cloud-based analytics on large data sets to unlock the insights they and the business need to make smarter decisions. All of these factors challenge DevOps maturity.

DevOps 188

DevOps Analytics Storage Metrics 188

Dynatrace

SEPTEMBER 8, 2022

Log management is an organization’s rules and policies for managing and enabling the creation, transmission, analysis, storage, and other tasks related to IT systems’ and applications’ log data. Log monitoring enables the collection of log data, and log analysis promotes intelligent, data-driven decision making.

Systems 189

Systems Cloud Analytics DevOps 189

Dynatrace

SEPTEMBER 5, 2023

Technology and operations teams work to ensure that applications and digital systems work seamlessly and securely. Predictive AI uses statistical algorithms and other advanced machine learning techniques to anticipate what might happen next in a system. What is predictive AI? Proactive resource allocation. Capacity planning.

Artificial Intelligence 266

Artificial Intelligence DevOps Analytics Engineering 266

Dynatrace

OCTOBER 14, 2022

Log data provides a unique source of truth for debugging applications, optimizing infrastructure, and investigating security incidents. This contextualization of log data enables AI-powered problem detection and root cause analysis at scale. Dynamic landscape and data handling requirements result in manual work.

Storage 232

Storage Government Network Infrastructure 232

DZone

SEPTEMBER 22, 2022

Alluxio workers are responsible for managing local resources, and they store data as blocks. Users can allocate different storage tiers as the resources for Alluxio workers, including MEM/SSD/HDD, which are further composed of directories. In this article, we analyze the policies of block allocation from the source code.

Code 246

Code Virtualization Systems Storage 246

Scalegrid

JANUARY 19, 2024

When a MongoDB rollback happens, it can cause trouble to your data integrity and system consistency. Understanding how to address a rollback is critical for minimizing potential data loss and maintaining seamless operations. The consequences of rollbacks are significant and should not be underestimated.

Database 130

Database Network Servers Monitoring 130

The Netflix TechBlog

OCTOBER 28, 2021

Data Engineers of Netflix?—?Interview Interview with Pallavi Phadnis This post is part of our “ Data Engineers of Netflix ” series, where our very own data engineers talk about their journeys to Data Engineering @ Netflix. Pallavi Phadnis is a Senior Software Engineer on the Product Data Science and Engineering team.

Data Engineering 173

Data Engineering Engineering Big Data Software Engineering 173

Dynatrace

JANUARY 23, 2024

AI data analysis can help development teams release software faster and at higher quality. So how can organizations ensure data quality, reliability, and freshness for AI-driven answers and insights? And how can they take advantage of AI without incurring skyrocketing costs to store, manage, and query data?

Performance 191

Performance DevOps Innovation Artificial Intelligence 191

DZone

NOVEMBER 7, 2023

This is about how a cyber security service provider built its log storage and analysis system (LSAS) and realized 3X data writing speed, 7X query execution speed, and visualized management. It also provides data management and file-tracking services.

Storage 216

Storage Speed Systems 216

Scalegrid

FEBRUARY 8, 2024

A distributed storage system is foundational in today’s data-driven landscape, ensuring data spread over multiple servers is reliable, accessible, and manageable. This guide delves into how these systems work, the challenges they solve, and their essential role in businesses and technology.

Storage 130

Storage Systems Big Data Azure 130

The Netflix TechBlog

OCTOBER 15, 2019

Part I: Overview Andreas Andreakis , Falguni Jhaveri , Ioannis Papapanagiotou , Mark Cho , Poorna Reddy , Tongliang Liu Overview It is a commonly observed pattern for applications to utilize multiple datastores where each is used to serve a specific need such as storing the canonical form of data (MySQL etc.), caching (Memcached etc.),

Transportation 192

Transportation Architecture Processing Storage 192

The Morning Paper

MARCH 8, 2020

Building an elastic query engine on disaggregated storage , Vuppalapati, NSDI’20. This paper describes the design decisions behind the Snowflake cloud-based data warehouse. have altered the many assumptions that guided the design and optimization of the Snowflake system. From shared-nothing to disaggregation.

Storage 112

Storage Engineering Cache Serverless 112

Dynatrace

SEPTEMBER 23, 2020

Recent improvements in OneAgent runtime-data handling. Operating Systems are not always set up in the same way. Storage mount points in a system might be larger or smaller, local or remote, with high or low latency, and various speeds. Storage and network transfer of files is a measurable cost. See details below.

Storage 150

Storage Latency Operating System Network 150

The Netflix TechBlog

APRIL 27, 2023

Engineers want their alerting system to be realtime, reliable, and actionable. A few years ago, we were paged by our SRE team due to our Metrics Alerting System falling behind — critical application health alerts reached engineers 45 minutes late! In other words, false positives are bad but false negatives are the absolute worst!

Storage 288

Storage Cache Metrics Database 288

Dynatrace

OCTOBER 17, 2023

In today’s data-driven landscape, businesses are grappling with an unprecedented surge in data volume. The data generated by applications to support operational teams has reached overwhelming proportions. This data overload also prevents customer-centric pricing models as users consider cost-effective technology platforms.

Retail 242

Retail Storage Best Practices Architecture 242

Dynatrace

APRIL 24, 2024

To transparently manage expectations and maintain trust with our customers, we expanded the Dynatrace SLA beyond accessing the user interface to cover the full range of relevant product categories, such as processing and retaining incoming data, accessing and working with data, and triggering automations.

Azure 245

Azure Infrastructure Metrics AWS 245

The Morning Paper

NOVEMBER 5, 2019

File systems unfit as distributed storage backends: lessons from 10 years of Ceph evolution Aghayev et al., In this case, the assumption that a distributed storage backend should clearly be layered on top of a local file system. What is a distributed storage backend? SOSP’19.

Storage 64

Storage Systems Hardware Efficiency 64

Dynatrace

OCTOBER 23, 2023

Hyper-V plays a vital role in ensuring the reliable operations of data centers that are based on Microsoft platforms. Microsoft Hyper-V is a virtualization platform that manages virtual machines (VMs) on Windows-based systems. Optimize resource allocation, identify bottlenecks, and improve overall system performance.

Efficiency 219

Efficiency Virtualization Hardware Performance 219

Dynatrace

APRIL 19, 2023

We’ve replaced the role-based permission system with security policies that provide fine-grained control over service access and introduce new dynamic controls for user authorization. The managed policies always reflect the latest changes in the Dynatrace platform and its underlying permission system and, therefore, can’t be customized.

Storage 214

Storage Analytics Systems Architecture 214

Dynatrace

JANUARY 22, 2024

While traditional AI relies on finding correlations in data, causal AI aims to determine the precise underlying mechanisms that drive events and outcomes. That’s why causal AI use cases abound for organizations looking to build more reliable and transparent AI systems. Causal AI use cases can complement other types of AI.

Artificial Intelligence 212

Artificial Intelligence Healthcare Retail Government 212

High Scalability

JANUARY 11, 2022

from a client it performs two parallel operations: i) persisting the action in the data store ii) publish the action in a streaming data store for a pub-sub model. User Feed Service, Media Counter Service) read the actions from the streaming data store and performs their specific tasks. System Components. Data Models.

Design 334

Design Media Storage Logistics 334

Dynatrace

JULY 29, 2022

Log analytics is the process of viewing, interpreting, and querying log data so developers and IT teams can quickly detect and resolve application and system issues. At the same time, log analytics can present challenges as data volumes explode, particularly in traditional environments that lack end-to-end observability solutions.

Analytics 220

Analytics Storage Retail Monitoring 220

DZone

DECEMBER 14, 2021

Metadata synchronization (sync) is a core feature in Alluxio that keeps files and directories consistent with their source of truth in under-storage systems, thus making it simple for users to reason the data retrieved from Alluxio. Meanwhile, understanding the internal process is important in order to tune the performance.

Design 244

Design Storage Tuning Systems 244

Dynatrace

NOVEMBER 9, 2023

Nutanix overview dashboard The extension automatically gathers real-time performance data from your Nutanix clusters to monitor resource usage, cluster health, and more, all in one place. Storage container metrics Track the usage and performance of storage containers to optimize resource allocation.

Virtualization 198

Virtualization Storage Metrics Monitoring 198

DZone

JANUARY 31, 2024

SIEM systems enable early detection of security threats and suspicious activities by analyzing vast amounts of log data in real time. Correlation Engine: SIEM systems analyze and correlate the collected data to identify patterns, anomalies, and potential security incidents.

Storage 136

Storage Data Engineering Network Infrastructure 136

Dynatrace

DECEMBER 18, 2023

For busy site reliability engineers, ensuring system reliability, scalability, and overall health is an imperative that’s getting harder to achieve in ever-expanding, cloud-native, container-based environments. To get a more granular look into telemetry data, many analysts rely on custom metrics using Prometheus.

Metrics 182

Metrics Engineering Energy Tuning 182

DZone

JUNE 15, 2023

In order to gain insight into these problems, we gather a range of metrics and logs to monitor the utilization of system resources such as CPU, memory, and application-specific latencies. It is worth noting that this data collection process does not impact the performance of the application.

Latency 246

Latency Storage Strategy Metrics 246

Dynatrace

JULY 29, 2022

Log analytics is the process of viewing, interpreting, and querying log data so developers and IT teams can quickly detect and resolve application and system issues. At the same time, log analytics can present challenges as data volumes explode, particularly in traditional environments that lack end-to-end observability solutions.

Analytics 186

Analytics Storage Retail Monitoring 186

Dynatrace

APRIL 8, 2024

If system failures occur, teams must resolve them quickly and resolutely. Maximizing a strong foundation of IT infrastructure performance and security means effectively utilizing all the data that these systems produce to drive risk management processes and decision-making. Automated issue resolution.

Analytics 206

Analytics Infrastructure Efficiency Technology 206

Dynatrace

MARCH 15, 2024

Syslog is the go-to protocol that delivers infrastructure administrators, network engineers, and security team logs that tell them all they need to know about their systems’ delivery, performance, availability, and security. Syslog is a protocol with clear specifications that require a dedicated syslog server.

Open Source 205

Open Source Infrastructure Network Government 205

The Netflix TechBlog

MAY 4, 2023

Behind the scenes, a myriad of systems and services are involved in orchestrating the product experience. These backend systems are consistently being evolved and optimized to meet and exceed customer and product expectations. This technique facilitates validation on multiple fronts.

Traffic 339

Traffic Latency Tuning Systems 339

The Netflix TechBlog

OCTOBER 19, 2020

which is difficult when troubleshooting distributed systems. Troubleshooting a session in Edgar When we started building Edgar four years ago, there were very few open-source distributed tracing systems that satisfied our needs. Traces collected from various microservices are ingested in a stream processing manner into the data store.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

DZone

JANUARY 23, 2022

Building resilient systems requires comprehensive error management. Errors could occur in any part of the system / or its ecosystem and there are different ways of handling these e.g. Datacenter - data center failure where the whole DC could become unavailable due to power failure, network connectivity failure, environmental catastrophe, etc.

Hardware 243

Hardware DevOps Network Storage 243

DZone

JANUARY 20, 2023

Having a distributed and scalable graph database system is highly sought after in many enterprise scenarios. Do Not Be Misled Designing and implementing a scalable graph database system has never been a trivial task.

Scalability 201

Scalability Big Data Hardware Internet 201

All Things Distributed

MARCH 23, 2021

Back then, Amazon was ~2% of its size today, and was growing faster than traditional IT systems could support. We had to rethink everything previously known about building scalable systems. Storage was one of our biggest pain points, and the traditional systems we used just weren’t fitting the needs of the Amazon.com retail business.

Ecommerce 216

Ecommerce Retail Storage Scalability 216

Smashing Magazine

SEPTEMBER 28, 2022

Five Data-Loading Patterns To Improve Frontend Performance. Five Data-Loading Patterns To Improve Frontend Performance. Data loading patterns are an essential part of your application as they will determine which parts of your application are directly usable by visitors. But isn’t waiting for the data the point?

Cache 125

Cache Performance Servers Social Media 125