Scalegrid

MAY 13, 2020

In this blog post, we explain what Greenplum is, and break down the Greenplum architecture, advantages, major use cases, and how to get started. It’s architecture was specially designed to manage large-scale data warehouses and business intelligence workloads by giving you the ability to spread your data out across a multitude of servers.

Big Data 321

Big Data Database Artificial Intelligence Open Source 321

Uber Engineering

AUGUST 14, 2019

Maintaining Uber’s large-scale data warehouse comes with an operational cost in terms of ETL functions and storage. Once identified, … The post Less is More: Engineering Data Warehouse Efficiency with Minimalist Design appeared first on Uber Engineering Blog.

Efficiency 103

Efficiency Engineering Design Storage 103

Dynatrace

OCTOBER 17, 2023

Dynatrace has developed the purpose-built data lakehouse, Grail , eliminating the need for separate management of indexes and storage. All data is readily accessible without storage tiers, such as costly solid-state drives (SSDs). No storage tiers, no archiving or retrieval from archives, and no indexing or reindexing.

Retail 233

Retail Storage Best Practices Architecture 233

Dynatrace

OCTOBER 4, 2022

A data lakehouse addresses these limitations and introduces an entirely new architectural design. This architecture offers rich data management and analytics features (taken from the data warehouse model) on top of low-cost cloud storage systems (which are used by data lakes). Grail is built for such analytics, not storage.

Artificial Intelligence 264

Artificial Intelligence Analytics Storage Architecture 264

Percona

APRIL 17, 2023

In this blog post, we will discuss the best practices on the MongoDB ecosystem applied at the Operating System (OS) and MongoDB levels. For example: Read/Write tickets WiredTiger uses tickets to control the number of read / write operations simultaneously processed by the storage engine. By default, it uses 50% of the memory + 1 GB.

Best Practices 52

Best Practices Design Tuning Database 52

The Netflix TechBlog

NOVEMBER 29, 2023

As noted in an earlier blog post , we began developing an HDR variant of VMAF; let’s call it HDR-VMAF. Bitrate versus quality comparison HDR-VMAF is designed to be format-agnostic — it measures the perceptual quality of HDR video signal regardless of its container format, for example, Dolby Vision or HDR10. The graphic below (Fig.

Open Source 252

Open Source Software Engineering Internet Internet 252

Dynatrace

NOVEMBER 20, 2023

The path to achieving unprecedented productivity and software innovation through ChatGPT and other generative AI – blog Paired with causal AI, organizations can increase the impact and safer use of ChatGPT and other generative AI technologies. What is causal AI? What is predictive AI? What is AIOps?

AWS 206

AWS Artificial Intelligence Analytics Cloud 206

The Morning Paper

NOVEMBER 5, 2019

File systems unfit as distributed storage backends: lessons from 10 years of Ceph evolution Aghayev et al., In this case, the assumption that a distributed storage backend should clearly be layered on top of a local file system. What is a distributed storage backend? SOSP’19. This is not surprising in hindsight.

Storage 64

Storage Systems Hardware Efficiency 64

Percona

MARCH 29, 2023

Compression in any database is necessary as it has many advantages, like storage reduction, data transmission time, etc. Storage reduction alone results in significant cost savings, and we can save more data in the same space. In this blog, we will discuss both data and network-level compression offered in MongoDB.

Storage 112

Storage Network Open Source Latency 112

Dynatrace

AUGUST 11, 2022

Infrastructure as a service (IaaS) handles compute, storage, and network resources. Consider a monolithic application, for example, designed to perform a host of functions. Microservices, on the other hand, make it possible to quickly scale up a single aspect of an application, such as storage or compute use.

Development 247

Development Serverless Best Practices Lambda 247

Percona

DECEMBER 11, 2023

A Dedicated Log Volume (DLV) is a specialized storage volume designed to house database transaction logs separately from the volume containing the database tables. DLVs are particularly advantageous for databases with large allocated storage, high I/O per second (IOPS) requirements, or latency-sensitive workloads.

AWS 101

AWS Benchmarking Performance Traffic 101

All Things Distributed

JANUARY 23, 2012

Expanding the Cloud - The AWS Storage Gateway. Today Amazon Web Services has launched the AWS Storage Gateway, making the power of secure and reliable cloud storage accessible from customersâ?? With the launch of the AWS Storage Gateway our customers can now integrate their on-premises IT environment with AWSâ??s

Storage 115

Storage AWS Virtualization Cloud 115

Percona

MAY 29, 2023

pgBackrest on the Main site streams backups and Write Ahead Logs (WALs) to the object storage. Once installed, configure the Custom Resource manifest so that pgBackrest starts using the Object Storage of your choice. In this blog post, we saw how Kubernetes and Operators can simplify your DR design.

Storage 122

Storage Cloud Network Traffic 122

All Things Distributed

AUGUST 20, 2012

Managing Cold Storage with Amazon Glacier. With the introduction of Amazon Glacier , IT organizations now have a solution that removes the headaches of digital archiving and provides extremely low cost storage. All Things Distributed. Werner Vogels weblog on building scalable and robust distributed systems. Expanding the Cloud â??

Storage 122

Storage Cloud AWS Media 122

Percona

JANUARY 30, 2023

As Ibrar Ahmed noted in his blog post on Transparent Database Encryption (TDE). Instead, it seems PostgreSQL Developers are of the opinion that encryption is a storage-level problem and is better solved on the filesystem or block device level. It does not require encryption capabilities on the storage level to provide data security.

Database 126

Database Storage Government Open Source 126

Dynatrace

NOVEMBER 11, 2020

Readers of this blog surely appreciate the earlier enhancements we announced related to enterprise-scale life-cycle management of OneAgents via REST API , easier configurability of OneAgent settings , and the recent improvements to OneAgent security. Easier rollout thanks to log storage best practices.

Best Practices 193

Best Practices Storage Java Benchmarking 193

Strategic Tech

MAY 24, 2019

Over the past few years I have focused much of my learning and work choices around learning about the design of sociotechnical systems?—?how how to design software architectures and organise teams around them. As part of good design, we should consider the effects of changing all of these variables.

Design 45

Design Software Architecture Strategy Technology 45

Dynatrace

MAY 26, 2021

This blog dives into what we learned by listening to experts in the field and the key takeaways we gathered from the talks. Trace-based sampling can help you save storage costs. This can help you save money in storage costs in the long run. OTel is designed to ensure flexibility and stability.

AWS 208

AWS Strategy Storage Metrics 208

The Netflix TechBlog

OCTOBER 28, 2021

During earlier years of my career, I primarily worked as a backend software engineer, designing and building the backend systems that enable big data analytics. The efficiency achieved in processing and storage layers brought us big savings on computing and storage costs.

Data Engineering 173

Data Engineering Engineering Big Data Software Engineering 173

Scalegrid

JULY 17, 2020

For the purposes of this blog, when we refer to JSON support in PostgreSQL, we will refer to JSONB going forward. JSONB storage has some drawbacks vs. traditional columns: PostreSQL does not store column statistics for JSONB columns. JSONB storage results in a larger storage footprint.

Storage 321

Storage Database Efficiency Processing 321

Percona

NOVEMBER 2, 2023

This blog post will outline the top three questions raised during the session and provide thorough responses from our team of experts. The bad On the other hand, Kubernetes is not designed to handle large databases or mainframes. However, Kubernetes does introduce additional layers, particularly in storage and networking.

Open Source 88

Open Source Hardware Storage Database 88

Percona

FEBRUARY 24, 2023

Benefits of Graviton2 Processors Best price performance for a broad range of workloads Extensive software support Enhanced security for cloud applications Available with managed AWS services Best performance per watt of energy used in Amazon EC2 Storage Continuing with the AWS example, choosing the right storage option will be key to performance.

Cloud 122

Cloud Serverless AWS Storage 122

ACM Sigarch

AUGUST 3, 2023

(Editor’s Note: This post was submitted as a rebuttal to Andrew Chien’s July 24 SIGARCH Blog Post ) The recent post “ Why Embodied Carbon is a poor Architecture Design metric, and Operational Carbon remains an important Problem ” by Prof. Chien’s blog post points to a paper by Bashir et al. Unlike Prof.

Energy 98

Energy Architecture Hardware Metrics 98

Official Blog - World Web Technology

NOVEMBER 27, 2023

You may have to put off learning web development and design. You can make a great website without coding or advanced web design skills. Anyone can use a good website builder, even if they don’t know web design. It depends on the best practices of web design. The news is positive. A website builder is an alternative.

Website 59

Website Development Mobile Storage 59

Scalegrid

NOVEMBER 29, 2023

This blog post will explore some of the major challenges in database management that businesses can expect, especially when transitioning from traditional database servers. A DBaaS automates several processes, such as using, erasing, and spinning up storage without interventions from IT staff. Your data will always be intact.

Healthcare 130

Healthcare Hardware Database Scalability 130

Dynatrace

JANUARY 23, 2024

Why growing AI adoption requires an AI observability strategy – blog While AI adoption brings operational efficiency and innovation for organizations, it also introduces the potential for runaway AI costs. Platform engineering: Empowering key Kubernetes use cases with Dynatrace – blog Digital transformation continues surging forward.

Performance 182

Performance DevOps Innovation Artificial Intelligence 182

The Netflix TechBlog

OCTOBER 19, 2020

In our previous blog post we introduced Edgar, our troubleshooting tool for streaming sessions. Now let’s look at how we designed the tracing infrastructure that powers Edgar. Our distributed tracing infrastructure is grouped into three sections: tracer library instrumentation, stream processing, and storage.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

Dynatrace

SEPTEMBER 8, 2023

In part 1 of this blog series , we explored the concept of business observability, its significance, and how real-time visibility aids in making informed decisions. To achieve this, we’ll create a Grail bucket specifically designed to retain data for a duration of 10 years (3,657 days).

Database 266

Database Artificial Intelligence Metrics Monitoring 266

Brendan Gregg

JULY 4, 2021

AWS Graviton2); for memory with the arrival of DDR5 and High Bandwidth Memory (HBM) on-processor; for storage including new uses for 3D Xpoint as a 3D NAND accelerator; for networking with the rise of QUIC and eXpress Data Path (XDP); and so on.

Performance 140

Performance Latency Hardware Storage 140

Dynatrace

MAY 26, 2021

This blog dives into what we learned by listening to experts in the field and the key takeaways we gathered from the talks. Trace-based sampling can help you save storage costs. This can help you save money in storage costs in the long run. OTel is designed to ensure flexibility and stability.

AWS 147

AWS Strategy Storage Metrics 147

Percona

AUGUST 9, 2023

This upgrade has vastly improved the management and storage of metadata, resulting in better reliability and scalability for various database objects. In this blog post, we will explore the intricacies of MySQL 8’s Transaction Data Dictionary, its advantages, and its real-life application using practical examples.

Database 79

Database Storage Scalability Games 79

Percona

AUGUST 29, 2023

In this blog post, we will explore the immense value of provisioning PostgreSQL clusters with Percona Operator by using bootstrap capabilities: Start the cluster with init SQL script Bootstrap the cluster from the existing cluster or backup Getting started You need to have the Percona Operator for PostgreSQL deployed.

Storage 98

Storage Database Media Open Source 98

All Things Distributed

FEBRUARY 23, 2011

Free at Last - A Fully Self-Sustained Blog Running in Amazon S3. In a follow up to the last blog post I have removed the last two dependencies this blog had on running a server somewhere: comments are now served by Disqus and search is now handled by Bing. blog comments powered by Disqus. Comments (). Contact Info.

AWS 78

AWS Storage Big Data Servers 78

Dynatrace

SEPTEMBER 7, 2022

With Amazon Web Services, the main sources from which to ingest logs—Simple Storage Service, or S3, and CloudWatch —come with an additional cost. The post Dynatrace adds support for VPC Flow Logs to Kinesis Data Firehose appeared first on Dynatrace blog. What is Amazon VPC Flow Logs to Kinesis Data Firehose, and why is it important?

Traffic 212

Traffic AWS Network Cloud 212

The Netflix TechBlog

DECEMBER 21, 2020

At this scale, we can gain a significant amount of performance and cost benefits by optimizing the storage layout (records, objects, partitions) as the data lands into our warehouse. We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits.

Storage 203

Storage Latency Efficiency Data Engineering 203

Dynatrace

NOVEMBER 22, 2021

EC2 is Amazon’s Infrastructure-as-a-service (IaaS) compute platform designed to handle any workload at scale. With EC2, Amazon manages the basic compute, storage, networking infrastructure and virtualization layer, and leaves the rest for you to manage: OS, middleware, runtime environment, data, and applications. Amazon EC2.

Best Practices 208

Best Practices AWS Monitoring Serverless 208

Dynatrace

JULY 6, 2022

Virtualization has revolutionized system administration by making it possible for software to manage systems, storage, and networks. Design, implement, and tune effective SLOs. The post 5 SRE best practices you can implement today appeared first on Dynatrace blog. This number will likely increase as the SRE discipline matures.

Best Practices 184

Best Practices Open Source Tuning Infrastructure 184