The Netflix TechBlog

JULY 15, 2021

Data Engineers of Netflix?—?Interview Interview with Kevin Wylie This post is part of our “Data Engineers of Netflix” series, where our very own data engineers talk about their journeys to Data Engineering @ Netflix. Kevin, what drew you to data engineering?

Data Engineering 189

Data Engineering Engineering Entertainment Big Data 189

DZone

JANUARY 13, 2023

Introduction Traditionally, SQL has been used for relational databases and data warehouses. However, in recent years there has been an exponential increase in the amount of data that connected systems produce, which has brought about a need for new ways to store and analyze such information.

IoT 174

IoT Analytics Database Metrics 174

All Things Distributed

NOVEMBER 15, 2016

They require teams of data engineers to spend months building complex data models and synthesizing the data before they can generate their first report. Let’s walk through some of the core experiences of QuickSight that make it so easy to set up, connect to your data sources, and build visualizations in minutes.

Analytics 152

Analytics Availability Media Social Media 152

The Netflix TechBlog

MARCH 5, 2019

While our engineering teams have and continue to build solutions to lighten this cognitive load (better guardrails, improved tooling, …), data and its derived products are critical elements to understanding, optimizing and abstracting our infrastructure. What will be the cost of rolling out the winning cell of an AB test to all users?

Infrastructure 182

Infrastructure Cloud Scalability AWS 182

All Things Distributed

OCTOBER 7, 2015

However, the data infrastructure to collect, store and process data is geared toward developers (e.g., In AWS’ quest to enable the best data storage options for engineers, we have built several innovative database solutions like Amazon RDS, Amazon RDS for Aurora, Amazon DynamoDB, and Amazon Redshift.

Cloud 137

Cloud Big Data AWS Analytics 137

The Netflix TechBlog

DECEMBER 3, 2022

Optionally, this step can use the Write-Audit-Publish pattern to ensure that data is correct before it is made available to the rest of the company. See example below: - template: id: wap type: wap tables: - ${CATALOG}/${DATABASE}/${TABLE} write_jobs: - job: id: write type: Spark spark: script: $S3{./src/sparksql_write.sql}

Best Practices 358

Best Practices Code Testing Data Engineering 358

Abhishek Tiwari

DECEMBER 27, 2017

ETL refers to extract, transform, load and it is generally used for data warehousing and data integration. ETL is a product of the relational database era and it has not evolved much in last decade. There are several emerging data trends that will define the future of ETL in 2018.

Big Data 49

Big Data Artificial Intelligence Storage Hardware 49