The Morning Paper

NOVEMBER 5, 2019

File systems unfit as distributed storage backends: lessons from 10 years of Ceph evolution Aghayev et al., In this case, the assumption that a distributed storage backend should clearly be layered on top of a local file system. What is a distributed storage backend? SOSP’19.

Storage 64

Storage Systems Hardware Efficiency 64

The Netflix TechBlog

APRIL 27, 2023

Engineers want their alerting system to be realtime, reliable, and actionable. A few years ago, we were paged by our SRE team due to our Metrics Alerting System falling behind — critical application health alerts reached engineers 45 minutes late! The internals here are outside the scope of this blog post.

Storage 288

Storage Cache Metrics Database 288

Dynatrace

APRIL 19, 2023

In Part 1 of this blog post series , we introduced the concept of security policies in preparation for upcoming enhancements to the Dynatrace platform. We’ve replaced the role-based permission system with security policies that provide fine-grained control over service access and introduce new dynamic controls for user authorization.

Storage 215

Storage Analytics Systems Architecture 215

Particular Software

MAY 22, 2023

Is this the best way to build a system, though? For example, you might need the data to be denormalized for performance, or the warehouse data might exist on a physically different system that can’t participate in a database join. ViewModel composition techniques also allow flexibility in terms of data storage technology.

Systems 98

Systems Cache Strategy Database 98

Dynatrace

DECEMBER 5, 2023

Streamline privacy requirements with flexible retention periods Data retention is a critical aspect of data handling, and it’s not just about privacy compliance—it’s about having the flexibility to optimize data storage times in Grail for your Dynatrace use cases. Grail buckets function like folders in a file system.

Storage 170

Storage Healthcare Best Practices Speed 170

Dynatrace

JULY 1, 2022

Operating systems support. We’re informing you that Dynatrace Managed version 1.250 is going to be the last one that you can upgrade to on the unsupported operating systems. Storage service upgrade to Cassandra 3.11.13. The post Dynatrace Managed release notes version 1.244 appeared first on Dynatrace blog.

Operating System 209

Operating System Storage Metrics Systems 209

The Netflix TechBlog

MAY 4, 2023

Behind the scenes, a myriad of systems and services are involved in orchestrating the product experience. These backend systems are consistently being evolved and optimized to meet and exceed customer and product expectations. This blog series will examine the tools, techniques, and strategies we have utilized to achieve this goal.

Traffic 339

Traffic Latency Tuning Systems 339

The Netflix TechBlog

SEPTEMBER 24, 2021

Our previous tech blog Packaging award-winning shows with award-winning technology detailed our packaging technology deployed on the streaming side. Lastly, the packager kicks in, adding a system layer to the asset, making it ready to be consumed by the clients. Uploading and downloading data always come with a penalty, namely latency.

Cloud 237

Cloud Media Storage Cache 237

Dynatrace

AUGUST 1, 2022

You quickly realize that it will take ages to fill up the overprovisioned database storage. Two days later, your database runs out of storage in the middle of the night. Unfortunately, you did not have any monitoring or alerting in place, and you stopped checking the system daily because you have lost faith in your idea.

Storage 193

Storage Database Network Metrics 193

The Netflix TechBlog

JUNE 13, 2023

This is where large-scale system migrations come into play. Our previous blog post presented replay traffic testing — a crucial instrument in our toolkit that allows us to implement these transformations with precision and reliability. Canaries and sticky canaries are valuable tools in the system migration process.

Traffic 279

Traffic Metrics Systems Strategy 279

Dynatrace

JULY 29, 2022

Log analytics is the process of viewing, interpreting, and querying log data so developers and IT teams can quickly detect and resolve application and system issues. Cold storage and rehydration. Cold storage and rehydration. Data that organizations may need to access only once a quarter or year can reside in cold storage.

Analytics 220

Analytics Storage Retail Monitoring 220

Dynatrace

OCTOBER 6, 2023

Custom data buckets for faster queries, increased control, and custom retention periods The first layer in the Grail data model consists of buckets and tables (and views for entities, which is outside this blog post’s scope). Buckets are similar to folders, a physical storage location. There is a default bucket for each table.

Artificial Intelligence 266

Artificial Intelligence Analytics Metrics Storage 266

DZone

OCTOBER 11, 2022

Making Google’s CalDAV and CardDAV APIs available for everyone ( Google Developers Blog). Linux System Mining with Python ( Javalobby – The heart of the Java developer community). Email Reveals Google App Engine Search API About Ready For Preview Release, Charges Planned For Storage, Operations ( TechCrunch). Java EE 7 is Final.

Java 244

Java Best Practices Google Analytics 244

Uber Engineering

APRIL 5, 2018

Three years ago, Uber Engineering adopted Hadoop as the storage ( HDFS ) and compute ( YARN ) infrastructure for our organization’s big data analysis.

Systems 109

Systems Big Data Storage Infrastructure 109

Scalegrid

MAY 13, 2020

In this blog post, we explain what Greenplum is, and break down the Greenplum architecture, advantages, major use cases, and how to get started. High performance, query optimization, open source and polymorphic data storage are the major Greenplum advantages. Typically an MPP system has one leader node and one or many compute nodes.

Big Data 321

Big Data Database Artificial Intelligence Open Source 321

Dynatrace

JULY 29, 2022

Log analytics is the process of viewing, interpreting, and querying log data so developers and IT teams can quickly detect and resolve application and system issues. Cold storage and rehydration. Cold storage and rehydration. Data that organizations may need to access only once a quarter or year can reside in cold storage.

Analytics 186

Analytics Storage Retail Monitoring 186

Dynatrace

OCTOBER 17, 2023

Moreover, the system lacks flexibility, imposing strict schemas that administrators and developers must adhere to avoid additional costs. Dynatrace has developed the purpose-built data lakehouse, Grail , eliminating the need for separate management of indexes and storage. The majority of costs are associated with data querying.

Retail 242

Retail Storage Best Practices Architecture 242

Dynatrace

OCTOBER 4, 2022

Teams have introduced workarounds to reduce storage costs. Additionally, efforts such as lowered data retention times, two-tiered storage systems, shaky index management, sampled data, and data pipelines reduce the overall amount of stored data. Stop worrying about log data ingest and storage — start creating value instead.

Analytics 240

Analytics Artificial Intelligence Storage Serverless 240

Dynatrace

OCTOBER 7, 2021

This means you no longer have to provision, scale, and maintain servers to run your applications, databases, and storage systems. Speed is next; serverless solutions are quick to spin up or down as needed, and there are no delays due to limited storage or resource access. AWS offers four serverless offerings for storage.

Serverless 286

Serverless AWS Lambda Storage 286

Percona

DECEMBER 20, 2018

Let’s look at some of the most popular Percona Database Performance Blog posts in 2018. With the Percona Database Performance Blog, Percona staff and leadership work hard to provide the open source community with insights, technical support, predictions and metrics around multiple open source database software technologies.

Database 40

Database Tuning Performance AWS 40

Dynatrace

OCTOBER 4, 2022

This architecture offers rich data management and analytics features (taken from the data warehouse model) on top of low-cost cloud storage systems (which are used by data lakes). This decoupling ensures the openness of data and storage formats, while also preserving data in context. Retain data. Start using Grail now.

Artificial Intelligence 272

Artificial Intelligence Analytics Storage Architecture 272

The Netflix TechBlog

MAY 5, 2021

Netflix Drive relies on a data store that will be the persistent storage layer for assets, and a metadata store which will provide a relevant mapping from the file system hierarchy to the data store entities. 2 , are the file system interface, the API interface, and the metadata and data stores. A sample manifest file.

Media 174

Media Storage Architecture Cloud 174

The Netflix TechBlog

NOVEMBER 29, 2023

As noted in an earlier blog post , we began developing an HDR variant of VMAF; let’s call it HDR-VMAF. In spite of reaching higher qualities than the fixed ladder, the HDR-DO ladder, on average, occupies only 58% of the storage space compared to fixed-bitrate ladder. Krasula, A. Choudhury, S. Malfait, A.

Open Source 252

Open Source Software Engineering Internet Internet 252

Dynatrace

NOVEMBER 20, 2023

The path to achieving unprecedented productivity and software innovation through ChatGPT and other generative AI – blog Paired with causal AI, organizations can increase the impact and safer use of ChatGPT and other generative AI technologies. What is causal AI? What is predictive AI? What is AIOps?

AWS 214

AWS Artificial Intelligence Analytics Cloud 214

Percona

MARCH 7, 2023

One approach is to separate compute and storage to allow for independent scaling. By separating storage and compute, Neon replaces the PostgreSQL storage layer with data nodes, and compute nodes are distributed across a cluster of nodes. Architecture A Neon installation consists of compute nodes and the Neon storage engine.

Serverless 106

Serverless Storage Open Source Architecture 106

All Things Distributed

JANUARY 23, 2012

Werner Vogels weblog on building scalable and robust distributed systems. Expanding the Cloud - The AWS Storage Gateway. Today Amazon Web Services has launched the AWS Storage Gateway, making the power of secure and reliable cloud storage accessible from customersâ?? s storage infrastructure. Comments ().

Storage 115

Storage AWS Virtualization Cloud 115

The Netflix TechBlog

OCTOBER 19, 2020

which is difficult when troubleshooting distributed systems. In our previous blog post we introduced Edgar, our troubleshooting tool for streaming sessions. Troubleshooting a session in Edgar When we started building Edgar four years ago, there were very few open-source distributed tracing systems that satisfied our needs.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

The Netflix TechBlog

MARCH 14, 2023

We implemented a batch processing system for users to submit their requests and wait for the system to generate the output. This limited pilot system greatly reduced the time spent by our users to manually analyze the content. Maintaining disparate systems posed a challenge. Processing took several hours to complete.

Media 291

Media Innovation Energy Architecture 291

Scalegrid

NOVEMBER 29, 2023

This blog post will explore some of the major challenges in database management that businesses can expect, especially when transitioning from traditional database servers. A DBaaS automates several processes, such as using, erasing, and spinning up storage without interventions from IT staff. Your data will always be intact.

Healthcare 130

Healthcare Hardware Database Scalability 130

Dynatrace

DECEMBER 18, 2023

For busy site reliability engineers, ensuring system reliability, scalability, and overall health is an imperative that’s getting harder to achieve in ever-expanding, cloud-native, container-based environments. But often, we use additional services and solutions within our environment for backups, storage, networking, and more.

Metrics 183

Metrics Engineering Energy Tuning 183

All Things Distributed

AUGUST 20, 2012

Werner Vogels weblog on building scalable and robust distributed systems. Managing Cold Storage with Amazon Glacier. With the introduction of Amazon Glacier , IT organizations now have a solution that removes the headaches of digital archiving and provides extremely low cost storage. All Things Distributed. Comments ().

Storage 122

Storage Cloud AWS Media 122

Dynatrace

AUGUST 11, 2022

Infrastructure as a service (IaaS) handles compute, storage, and network resources. Because a third party manages part of the infrastructure, IT teams give up a measure of control over system architecture. FaaS drills down even deeper to scale specific aspects of storage, compute, or other services. But how does FaaS fit in?

Development 256

Development Serverless Best Practices Lambda 256

Dynatrace

SEPTEMBER 23, 2020

This blog post highlights a group of improvements that were released with OneAgent version 1.199. Operating Systems are not always set up in the same way. Storage mount points in a system might be larger or smaller, local or remote, with high or low latency, and various speeds. See details below. See details below.

Storage 150

Storage Latency Operating System Network 150

Dynatrace

APRIL 8, 2024

This blog explores how vertically integrated risk management solutions that use AI and automation enable unparalleled visibility, control, and efficiency for risk management in banking. If system failures occur, teams must resolve them quickly and resolutely. Automated issue resolution. Break down data silos.

Analytics 206

Analytics Infrastructure Efficiency Technology 206

Percona

DECEMBER 11, 2023

A Dedicated Log Volume (DLV) is a specialized storage volume designed to house database transaction logs separately from the volume containing the database tables. DLVs are particularly advantageous for databases with large allocated storage, high I/O per second (IOPS) requirements, or latency-sensitive workloads.

AWS 93

AWS Benchmarking Performance Traffic 93

Dynatrace

DECEMBER 16, 2021

The amount of recordable data is broad, and ranges from basic user browser and web page information to detailed technical information about the system Log4j 2 is running on. How Log4j 2 can be exploited depends on the specifics of the affected system. It logs messages from software, and searches for errors afterwards.

Internet 262

Internet Internet Games Java 262

Scalegrid

JANUARY 20, 2020

Follower clusters are a ScaleGrid feature that allows you to keep two independent database systems (of the same type) in sync. In this blog post, we will cover the top three scenarios to leverage follower clusters for your application. This helps reduce the load on the source system. This is where follower clusters come in.

Analytics 185

Analytics Storage Database Systems 185