article thumbnail

Leveraging Infrastructure as Code for Data Engineering Projects: A Comprehensive Guide

DZone

However, with the emergence of Infrastructure as Code (IaC) practices, data engineers can now automate infrastructure provisioning, deployment, and management, ensuring reliability, scalability, and reproducibility.

article thumbnail

Article: Magic Pocket: Dropbox’s Exabyte-Scale Blob Storage System

InfoQ

Adopting SMR technology and erasure codes, the system has extremely high durability guarantees but is cheaper than operating in the cloud. A horizontally scalable exabyte-scale blob storage system which operates out of multiple regions, Magic Pocket is used to store all of Dropbox’s data. By Facundo Agriel

Storage 110
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

AWS NoSQL Performance Lab Using Python

DZone

NoSQL databases emerge as ideal candidates to meet these requirements, and cloud platforms such as AWS offer managed and highly resilient data ecosystems. In this article, I am not going to determine which AWS NoSQL database is better: the concept of a better database only exists within a specific purposeful context.

AWS 266
article thumbnail

Optimization of I/O Workloads by Profiling in Python

DZone

Profiling is a crucial step in this process as it helps identify the parts of the code that are most resource-intensive. Do they involve disk I/O, such as file read/write operations, network I/O, which includes data transmission over a network, or database I/O, comprising database interactions?

Network 177
article thumbnail

Mastering Kubernetes deployments with Keptn: a comprehensive guide to enhanced visibility

Dynatrace

In this article, we’ll explore these challenges in detail and introduce Keptn, an open source project that addresses these issues, enhancing Kubernetes observability for smoother and more efficient deployments. External dependencies Many applications rely on external services, such as databases, APIs, or third-party services.

article thumbnail

No code automation testing: When to use and when not to use

Testsigma

Having a coded approach to test automation has its benefits. No code automation testing is an alternate approach that might be suitable for you and give you ROI sooner, providing few conditions are met. In this article, we will discuss the scenarios when you can use no code automation testing and when you should not use it.

Code 108
article thumbnail

Tips and Tricks for Writing Linux BPF Applications With libbpf

DZone

At the beginning of 2020, when I used the BCC tools to analyze our database performance bottlenecks, and pulled the code from the GitHub, I accidentally discovered that there was an additional libbpf-tools directory in the BCC project. It was fun and I learned a lot.

Database 179