Remove Data Engineering Remove Software Remove Software Engineering Remove Testing
article thumbnail

Bringing Software Engineering Rigor to Data

DZone

In software engineering, we've learned that building robust and stable applications has a direct correlation with overall organization performance. The data community is striving to incorporate the core concepts of engineering rigor found in software communities but still has further to go. Posted with permission.

article thumbnail

Our First Netflix Data Engineering Summit

The Netflix TechBlog

Engineers from across the company came together to share best practices on everything from Data Processing Patterns to Building Reliable Data Pipelines. The result was a series of talks which we are now sharing with the rest of the Data Engineering community! In this video, Sr.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Data Engineers of Netflix?—?Interview with Samuel Setegne

The Netflix TechBlog

Data Engineers of Netflix?—?Interview Interview with Samuel Setegne Samuel Setegne This post is part of our “Data Engineers of Netflix” interview series, where our very own data engineers talk about their journeys to Data Engineering @ Netflix. What drew you to Netflix?

article thumbnail

AI meets operations

O'Reilly

On one hand, ops groups are in a good position to do this; they’re already heavily invested in testing, monitoring, version control, reproducibility, and automation. This has important implications for testing. In the last two decades, a tremendous amount of work has been done on testing and building test suites.

article thumbnail

Incremental Processing using Netflix Maestro and Apache Iceberg

The Netflix TechBlog

IPS enables users to continue to use the data processing patterns with minimal changes. Introduction Netflix relies on data to power its business in all phases. Users configure the workflow to read the data in a window (e.g. data arrives too late to be useful). past 3 hours or 10 days).

article thumbnail

Organise your engineering teams around the work by reteaming

Abhishek Tiwari

Warehouse engineering squad - managing software services related inventory, stocktake, dispatch, allocation, transfer, robotics, etc. Customer experience engineering squad - focus on end-to-end customer life-cycle, marketing, targeting, personalisation, loyalty, etc. You want to move fast. How is that even possible?

article thumbnail

Microservices Adoption in 2020

O'Reilly

Success with microservices means owning the software lifecycle. Most (74%) respondents say their teams own the build-test-deploy-maintain phases of the software lifecycle. Software engineers comprise the survey audience’s single largest cluster, over one quarter (27%) of respondents (Figure 1).

Database 135