Scalegrid

JANUARY 5, 2024

Enhanced data security, better data integrity, and efficient access to information. This article cuts through the complexity to showcase the tangible benefits of DBMS, equipping you with the knowledge to make informed decisions about your data management strategies. What are the key advantages of DBMS?

Efficiency 147

Efficiency Storage Database Scalability 147

Dynatrace

JANUARY 23, 2024

AI data analysis can help development teams release software faster and at higher quality. So how can organizations ensure data quality, reliability, and freshness for AI-driven answers and insights? And how can they take advantage of AI without incurring skyrocketing costs to store, manage, and query data?

Performance 190

Performance DevOps Innovation Artificial Intelligence 190

The Morning Paper

MARCH 8, 2020

Building an elastic query engine on disaggregated storage , Vuppalapati, NSDI’20. This paper describes the design decisions behind the Snowflake cloud-based data warehouse. An increasingly large fraction of data in modern workloads comes from less predictable and highly variable sources.

Storage 112

Storage Engineering Cache Serverless 112

Percona

FEBRUARY 10, 2023

In the cloud era, however, developers and operation engineers started fully embracing automation tools making their job significantly easier. ViewBlock , a blockchain explorer, uses the Percona Operator for MongoDB to store critical data. In our lab we will use AWS EKS with a standard storage class.

Storage 108

Storage Blockchain AWS Cloud 108

Uber Engineering

AUGUST 14, 2019

Maintaining Uber’s large-scale data warehouse comes with an operational cost in terms of ETL functions and storage. Once identified, … The post Less is More: Engineering Data Warehouse Efficiency with Minimalist Design appeared first on Uber Engineering Blog.

Efficiency 99

Efficiency Engineering Design Storage 99

DZone

OCTOBER 24, 2023

The ELK stack is an abbreviation for Elasticsearch, Logstash, and Kibana, which offers the following capabilities: Elasticsearch: a scalable search and analytics engine with a log analytics tool and application-formed database, perfect for data-driven applications.

Analytics 267

Analytics Storage Infrastructure Scalability 267

Dynatrace

MARCH 23, 2020

A decade ago, while working for a large hosting provider, I led a team that was thrown into turmoil over the purchasing of server and storage hardware in preparation for a multi-million dollar super-bowl ad campaign. Our procurement decisions were based on trace data that was pulled from a handful of fragmented monitoring solutions.

Google 195

Google Engineering Metrics Hardware 195

High Scalability

JANUARY 11, 2022

Machine Learning Engineer at Amazon and has led several machine-learning initiatives across the Amazon ecosystem. from a client it performs two parallel operations: i) persisting the action in the data store ii) publish the action in a streaming data store for a pub-sub model. The MultiPart/Form-Data contains a series of parts.

Design 334

Design Media Storage Logistics 334

DZone

MAY 5, 2023

This blog post explores the benefits of combining both open-source frameworks, shows unique differentiators of Flink versus Kafka, and discusses when to use a Kafka-native streaming engine like Kafka Streams instead of Flink. The Tremendous Adoption of Apache Kafka and Apache Flink Apache Kafka became the de facto standard for data streaming.

Open Source 246

Open Source Storage Innovation Engineering 246

The Netflix TechBlog

APRIL 27, 2023

Engineers want their alerting system to be realtime, reliable, and actionable. A few years ago, we were paged by our SRE team due to our Metrics Alerting System falling behind — critical application health alerts reached engineers 45 minutes late! Data expressions define what data needs to be sourced in order to evaluate a query.

Storage 288

Storage Cache Metrics Database 288

DZone

NOVEMBER 27, 2023

An open-source distributed SQL query engine, Trino is widely used for data analytics on distributed data storage. Optimizing Trino to make it faster can help organizations achieve quicker insights and better user experiences, as well as cut costs and improve infrastructure efficiency and scalability.

Tuning 162

Tuning Speed Performance Open Source 162

Dynatrace

MAY 7, 2024

Fluent Bit is a telemetry agent designed to receive data (logs, traces, and metrics), process or modify it, and export it to a destination. Fluent Bit can serve as a proxy before you send data to Dynatrace or similar. However, you can also use Fluent Bit as a processor because you can perform various actions on the data.

Best Practices 248

Best Practices IoT Metrics Storage 248

Dynatrace

NOVEMBER 6, 2023

Cloud-native workloads on edge devices are gaining momentum among organizations as they extend the hybrid cloud closer to the data source and end users at the edge. The challenge of cloud-native observability at the enterprise edge In aggregate, connected devices generate huge volumes of data.

Retail 263

Retail Analytics Storage Cloud 263

DZone

JANUARY 31, 2024

SIEM systems enable early detection of security threats and suspicious activities by analyzing vast amounts of log data in real time. Correlation Engine: SIEM systems analyze and correlate the collected data to identify patterns, anomalies, and potential security incidents.

Storage 145

Storage Data Engineering Network Infrastructure 145

Dynatrace

JULY 29, 2022

Log analytics is the process of viewing, interpreting, and querying log data so developers and IT teams can quickly detect and resolve application and system issues. At the same time, log analytics can present challenges as data volumes explode, particularly in traditional environments that lack end-to-end observability solutions.

Analytics 219

Analytics Storage Retail Monitoring 219

Percona

APRIL 14, 2023

A shared characteristic in most (if not all) databases, be them traditional relational databases like Oracle, MySQL, and PostgreSQL or some kind of NoSQL-style database like MongoDB, is the use of a caching mechanism to keep (a copy of) part of the data in memory. So, how do you know if your hot data is in memory? MySQL does.

Cache 90

Cache Efficiency Database Monitoring 90

Dynatrace

NOVEMBER 28, 2023

In the previous installment of this blog series , we explored how to set up Dynatrace as a build-stage orchestrator to effectively address the challenges faced by Site Reliability Engineers (SREs). Davis AI can leverage this data to enable predictive analysis. This approach effectively combats configuration drift.

Traffic 265

Traffic Best Practices Strategy Engineering 265

The Netflix TechBlog

OCTOBER 19, 2020

a Netflix member via Twitter This is an example of a question our on-call engineers need to answer to help resolve a member issue?—?which We needed to increase engineering productivity via distributed request tracing. Traces collected from various microservices are ingested in a stream processing manner into the data store.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

The Netflix TechBlog

NOVEMBER 29, 2023

A vital aspect of such development is subjective testing with HDR encodes in order to generate training data. In spite of reaching higher qualities than the fixed ladder, the HDR-DO ladder, on average, occupies only 58% of the storage space compared to fixed-bitrate ladder. Yes, we are committed to supporting the open-source community.

Open Source 252

Open Source Software Engineering Internet Internet 252

Percona

MARCH 21, 2023

We want to improve our ability to replicate your data and limit replication to row-based events securely, wherein we do not have control over the source(s). The replica must impose data access limitations on the replicated stream because changes may get past the security barrier separating the source and replica in some configurations.

Storage 86

Storage Open Source Database Engineering 86

Dynatrace

JULY 29, 2022

Log analytics is the process of viewing, interpreting, and querying log data so developers and IT teams can quickly detect and resolve application and system issues. At the same time, log analytics can present challenges as data volumes explode, particularly in traditional environments that lack end-to-end observability solutions.

Analytics 185

Analytics Storage Retail Monitoring 185

Smashing Magazine

SEPTEMBER 28, 2022

Five Data-Loading Patterns To Improve Frontend Performance. Five Data-Loading Patterns To Improve Frontend Performance. Data loading patterns are an essential part of your application as they will determine which parts of your application are directly usable by visitors. But isn’t waiting for the data the point?

Cache 127

Cache Performance Servers Social Media 127

Dynatrace

MARCH 15, 2024

Syslog is the go-to protocol that delivers infrastructure administrators, network engineers, and security team logs that tell them all they need to know about their systems’ delivery, performance, availability, and security. This will enable you to use Dynatrace ActiveGate to ingest syslog data.

Open Source 205

Open Source Infrastructure Network Government 205

Percona

JANUARY 5, 2023

Space constraint has been an endless and painstaking process for many of us, especially in systems that have a high number of transactions and data growth exceeding hundreds of GBs in a matter of days. MySQL won’t shrink tablespace and storage won’t be released to the file system. What is partitioning? 1 mysql mysql 9.0G

Traffic 60

Traffic Engineering Speed Storage 60

Dynatrace

MARCH 24, 2023

Problems include provisioning and deployment; load balancing; securing interactions between containers; configuration and allocation of resources such as networking and storage; and deprovisioning containers that are no longer needed. Mesos also supports other orchestration engines, including Kubernetes and Docker Swarm.

Infrastructure 190

Infrastructure Open Source Operating System Cloud 190

Percona

OCTOBER 13, 2023

A common practice among DBAs and developers is to copy table data and.frm files from the data dictionary. In the case of InnoDB tablespaces, this information is incorporated into the tablespace itself, creating a fusion of metadata and data primarily to enhance performance. In MySQL 8.0, ibd files), general tablespace files (.ibd

Storage 92

Storage Database Engineering Open Source 92

Percona

APRIL 20, 2023

Data-at-rest encryption is essential for compliance with regulations that require the protection of sensitive data. It is also critical for securing sensitive data and avoiding data breaches. PostgreSQL does not natively support Transparent Data Encryption (TDE). apiVersion: storage.k8s.io/v1

Storage 120

Storage AWS Cloud Database 120

DZone

JULY 20, 2021

Taking a proactive and efficient approach to Kubernetes cluster monitoring can help engineering teams identify and predict many critical problems like CPU outage, memory outage, storage issues well in advance of these issues taking a toll on a business.

Monitoring 246

Monitoring Storage Efficiency Engineering 246

InfoQ

DECEMBER 29, 2023

Zendesk reduced its data storage costs by over 80% by migrating from DynamoDB to a tiered storage solution using MySQL and S3. By Rafal Gancarz

Storage 134

Storage Scalability Database Technology 134

Dynatrace

FEBRUARY 2, 2023

Should business data be part of your observability solution? Technology and business leaders express increasing interest in integrating business data into their IT observability strategies, citing the value of effective collaboration between business and IT.

Analytics 206

Analytics Metrics DevOps Storage 206

Percona

MARCH 7, 2023

One approach is to separate compute and storage to allow for independent scaling. By separating storage and compute, Neon replaces the PostgreSQL storage layer with data nodes, and compute nodes are distributed across a cluster of nodes. Compute nodes are stateless PostgreSQL nodes backed by the Neon storage engine.

Serverless 109

Serverless Storage Open Source Architecture 109

Percona

FEBRUARY 15, 2023

Bloom filters are an essential component of an LSM-based database engine like MyRocks. With MyRocks/RocksDB, data is stored in a set of large SST files. LSM storage engines like MyRocks are very different from the more common B-Tree-based storage engines like InnoDB.

Storage 126

Storage Tuning Cache Engineering 126

The Netflix TechBlog

DECEMBER 21, 2020

By Anupom Syam Background At Netflix, our current data warehouse contains hundreds of Petabytes of data stored in AWS S3 , and each day we ingest and create additional Petabytes. We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits.

Storage 203

Storage Latency Efficiency Data Engineering 203

Dynatrace

AUGUST 30, 2023

Dynatrace’s collaboration with Google addresses these needs by providing simple, scalable, and innovative data acquisition for comprehensive analysis and troubleshooting. Here we asked Davis, the Dynatrace AI engine , to correlate CPU usage against other signals. Agent logs security.

Google 225

Google Cloud Innovation Infrastructure 225

Percona

JULY 13, 2023

With the rise of globalization, the need to store and process multilingual data has become essential. The first two rows contain simple text data, while the third row contains an emoji. Table: To ensure proper storage of emojis, let’s create the table columns using the utf8mb4 character set. version as a game-changer.

Database 92

Database Games Open Source Engineering 92

Dynatrace

MAY 26, 2021

From May 17 to May 18, 2021, the Open-Source Engineering team at Dynatrace attended the virtual observability conference, o11yfest. A recurring theme was Site Reliability Engineering. Trace-based sampling can help you save storage costs. This can help you save money in storage costs in the long run. We second that.

AWS 217

AWS Strategy Storage Metrics 217

Scalegrid

JANUARY 20, 2020

Unlike cloning or replication, this allows you to maintain an active, point-in-time copy of your production data. Unlike a static clone, this data imports on a set schedule so your follower cluster is always in sync with your production cluster. Note: ScaleGrid implements follower clusters using storage snapshots.

Analytics 185

Analytics Storage Database Systems 185