article thumbnail

1. Streamlining Membership Data Engineering at Netflix with Psyberg

The Netflix TechBlog

By Abhinaya Shetty , Bharath Mummadisetty At Netflix, our Membership and Finance Data Engineering team harnesses diverse data related to plans, pricing, membership life cycle, and revenue to fuel analytics, power various dashboards, and make data-informed decisions.

article thumbnail

Bringing Software Engineering Rigor to Data

DZone

In software engineering, we've learned that building robust and stable applications has a direct correlation with overall organization performance. The data community is striving to incorporate the core concepts of engineering rigor found in software communities but still has further to go.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Secrets Detection: Optimizing Filter Processes

DZone

While increasing both the precision and the recall of our secrets detection engine, we felt the need to keep a close eye on speed. So it wasn’t a surprise to find that our engine had the same problem: more power, less speed. In a gearbox, if you want to increase torque, you need to decrease speed.

article thumbnail

Supporting Diverse ML Systems at Netflix

The Netflix TechBlog

Without these integrations, projects would be stuck at the prototyping stage, or they would have to be maintained as outliers outside the systems maintained by our engineering teams, incurring unsustainable operational overhead. Importantly, all the use cases were engineered by practitioners themselves.

Systems 226
article thumbnail

3. Psyberg: Automated end to end catch up

The Netflix TechBlog

This helps overwrite data only when required and minimizes unnecessary reprocessing. As seen above, by chaining these Psyberg workflows, we could automate the catchup for late-arriving data from hours 2 and 6. The Data Engineer does not need to perform any manual intervention in this case and can thus focus on more important things!

Tuning 244
article thumbnail

A Day in the Life of an Experimentation and Causal Inference Scientist @ Netflix

The Netflix TechBlog

At Netflix, our data scientists span many areas of technical specialization, including experimentation, causal inference, machine learning, NLP, modeling, and optimization. Together with data analytics and data engineering, we comprise the larger, centralized Data Science and Engineering group.

Analytics 207
article thumbnail

How Our Paths Brought Us to Data and Netflix

The Netflix TechBlog

—?and what the role entails by Julie Beckley & Chris Pham This Q&A provides insights into the diverse set of skills, projects, and culture within Data Science and Engineering (DSE) at Netflix through the eyes of two team members: Chris Pham and Julie Beckley. Photo from a team curling offsite?—?There’s There’s us to the right!

Analytics 223