All Things Distributed

AUGUST 24, 2012

Werner Vogels weblog on building scalable and robust distributed systems. Which makes this week a good moment to read up on some of the historical work around the costs of data engineering. In 1997 Jim revisted his caculations with the help of Goetz Graefe , and it details the impact of 10 years of hardware and pricing progress.

Storage 108

Storage Hardware AWS Data Engineering 108

Abhishek Tiwari

JULY 12, 2018

Unfortunately, building data pipelines remains a daunting, time-consuming, and costly activity. Not everyone is operating at Netflix or Spotify scale data engineering function. Often companies underestimate the necessary effort and cost involved to build and maintain data pipelines.

Latency 63

Latency Analytics Scalability Engineering 63

Abhishek Tiwari

DECEMBER 27, 2017

Kubernetes has emerged as go to container orchestration platform for data engineering teams. In 2018, a widespread adaptation of Kubernetes for big data processing is anitcipated. Organisations are already using Kubernetes for a variety of workloads [1] [2] and data workloads are up next. Native frameworks.

Big Data 63

Big Data Storage Benchmarking Hardware 63

All Things Distributed

OCTOBER 7, 2015

Today, I am excited to share with you a brand new service called Amazon QuickSight that aims to simplify the process of deriving insights from a wide variety of data sources in a fast and affordable manner. QuickSight is a fast, cloud native, scalable, business intelligence service for the 1/10th the cost of old-guard BI solutions.

Cloud 137

Cloud Big Data AWS Analytics 137

O'Reilly

JULY 15, 2020

Technical roles represented in the “Other” category include IT managers, data engineers, DevOps practitioners, data scientists, systems engineers, and systems administrators. Combined, technology verticals—software, computers/hardware, and telecommunications—account for about 35% of the audience (Figure 2).

Database 135

Database Architecture Education Systems 135