High Scalability

MARCH 24, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems : large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc.

Education 105

Education Software Engineering Engineering Big Data 105

High Scalability

MARCH 3, 2020

Level up on in-demand technologies and prep for your interviews on Educative.io, featuring popular courses like the bestselling Grokking the System Design Interview. Etleap is analyst-friendly , enterprise-grade ETL-as-a-service , built for Redshift and Snowflake data warehouses and S3/Glue data lakes.

Education 114

Education Games Engineering Java 114

High Scalability

MARCH 17, 2020

Level up on in-demand technologies and prep for your interviews on Educative.io, featuring popular courses like the bestselling Grokking the System Design Interview. Etleap is analyst-friendly , enterprise-grade ETL-as-a-service , built for Redshift and Snowflake data warehouses and S3/Glue data lakes.

Education 105

Education Engineering Java Software Engineering 105

High Scalability

FEBRUARY 18, 2020

Level up on in-demand technologies and prep for your interviews on Educative.io, featuring popular courses like the bestselling Grokking the System Design Interview. Etleap is analyst-friendly , enterprise-grade ETL-as-a-service , built for Redshift and Snowflake data warehouses and S3/Glue data lakes.

Education 113

Education Games Engineering Java 113

High Scalability

FEBRUARY 9, 2020

Level up on in-demand technologies and prep for your interviews on Educative.io, featuring popular courses like the bestselling Grokking the System Design Interview. Etleap is analyst-friendly , enterprise-grade ETL-as-a-service , built for Redshift and Snowflake data warehouses and S3/Glue data lakes.

Education 108

Education Games Engineering Java 108

The Agile Manager

JULY 31, 2022

There are shadow IT teams of developers or data engineers that spring up in areas like operations or marketing because the captive IT function is slow, if not outright incapable, of responding to internal customer demand. These structures don’t just ignore feedback loops, they are designed to repress feedback.

Programming 59

Programming Engineering Data Engineering Innovation 59

The Netflix TechBlog

DECEMBER 3, 2022

One of the main reasons this feature exists is just like with food samples, to give you “a taste” of the production quality ETL code that you could encounter inside the Netflix data ecosystem. " , country_code STRING COMMENT "Country code of the playback session." Let’s review the transformation steps below.

Best Practices 358

Best Practices Code Testing Data Engineering 358

High Scalability

JANUARY 22, 2020

Level up on in-demand technologies and prep for your interviews on Educative.io, featuring popular courses like the bestselling Grokking the System Design Interview. Etleap is analyst-friendly , enterprise-grade ETL-as-a-service , built for Redshift and Snowflake data warehouses and S3/Glue data lakes.

Education 22

Education Java Software Engineering Engineering 22

High Scalability

NOVEMBER 12, 2019

Grokking the System Design Interview is a popular course on Educative.io (taken by 20,000+ people) that's widely considered the best System Design interview resource on the Internet. Etleap is analyst-friendly , enterprise-grade ETL-as-a-service , built for Redshift and Snowflake data warehouses and S3/Glue data lakes.

Education 22

Education Java Software Engineering Engineering 22

The Netflix TechBlog

FEBRUARY 9, 2022

any business logic code in a raw (e.g. JAR) form to be executed as part of the user defined data pipeline. data pipeline ?—?a DAG) for the purpose of transforming data using some business logic. The design of a cron job is simple, you take some system command, you pick the schedule to run it on and you are done.

Storage 201

Storage Data Engineering Testing Code 201

High Scalability

OCTOBER 29, 2019

Grokking the System Design Interview is a popular course on Educative.io (taken by 20,000+ people) that's widely considered the best System Design interview resource on the Internet. Etleap is analyst-friendly , enterprise-grade ETL-as-a-service , built for Redshift and Snowflake data warehouses and S3/Glue data lakes.

Education 22

Education Java Software Engineering Engineering 22

High Scalability

FEBRUARY 6, 2020

Level up on in-demand technologies and prep for your interviews on Educative.io, featuring popular courses like the bestselling Grokking the System Design Interview. Etleap is analyst-friendly , enterprise-grade ETL-as-a-service , built for Redshift and Snowflake data warehouses and S3/Glue data lakes.

Education 21

Education Java Software Engineering Engineering 21

High Scalability

JANUARY 7, 2020

Level up on in-demand technologies and prep for your interviews on Educative.io, featuring popular courses like the bestselling Grokking the System Design Interview. Etleap is analyst-friendly , enterprise-grade ETL-as-a-service , built for Redshift and Snowflake data warehouses and S3/Glue data lakes.

Education 21

Education Java Software Engineering Engineering 21

High Scalability

DECEMBER 12, 2019

Level up on in-demand technologies and prep for your interviews on Educative.io, featuring popular courses like the bestselling Grokking the System Design Interview. Etleap is analyst-friendly , enterprise-grade ETL-as-a-service , built for Redshift and Snowflake data warehouses and S3/Glue data lakes.

Education 21

Education Java Software Engineering Engineering 21

The Symphonia

JUNE 19, 2018

This was a good, balanced, assessment, and it was fascinating to see Pulumi’s “code over template” approach. This type of talk is close to my heart since I strongly feel we need to do a lot of deep thinking (and subsequent knowledge sharing) of design and architecture patterns with Serverless. I wholeheartedly agree.

Serverless 40

Serverless AWS DevOps Open Source 40

Abhishek Tiwari

JULY 12, 2018

Unfortunately, building data pipelines remains a daunting, time-consuming, and costly activity. Not everyone is operating at Netflix or Spotify scale data engineering function. Often companies underestimate the necessary effort and cost involved to build and maintain data pipelines.

Latency 63

Latency Analytics Scalability Engineering 63

O'Reilly

FEBRUARY 18, 2020

This year’s growth in Python usage was buoyed by its increasing popularity among data scientists and machine learning (ML) and artificial intelligence (AI) engineers. The shift to cloud native design is transforming both software architecture and infrastructure and operations. Coincidence? This follows a 3% drop in 2018.

Software Architecture 134

Software Architecture DevOps Data Engineering Architecture 134

The Netflix TechBlog

NOVEMBER 20, 2023

These challenges are currently addressed in suboptimal and less cost efficient ways by individual local teams to fulfill the needs, such as Lookback: This is a generic and simple approach that data engineers use to solve the data accuracy problem. Users configure the workflow to read the data in a window (e.g.

Processing 210

Processing Big Data Efficiency Engineering 210

The Netflix TechBlog

OCTOBER 18, 2022

It is a general-purpose workflow orchestrator that provides a fully managed workflow-as-a-service (WAAS) to the data platform at Netflix. It serves thousands of users, including data scientists, data engineers, machine learning engineers, software engineers, content producers, and business analysts, for various use cases.

Java 202

Java Scalability Traffic Architecture 202

Abhishek Tiwari

FEBRUARY 26, 2020

There are no hard-and-fast rules to figure out interdependency between technology architecture and engineering organization but below is what I think can really work well for product startup. At this stage you can use CI/CD tooling provide by source code hosting platform (Github, Gitlab) or a SaaS solution like CircleCI.

Technology 81

Technology Technology Architecture Engineering 81

Abhishek Tiwari

JULY 20, 2019

One thing stand-out to me is being intentional and practical about your engineering organisation design. First and foremost, being intentional about organisation design requires good and honest discussions about all possible option. Design your teams to stay small and iterate fast. You want to move fast.

Engineering 61

Engineering Retail Airlines Healthcare 61

Testsigma

NOVEMBER 28, 2019

There are hundreds of tools through which the automation code can be written in different programming languages. Udacity Udacity provides nanodegree programs on all automation languages like C++, Machine Learning, Data engineer, Robotics and more. All the tracks are covered completely online.

Website 53

Website Testing Programming Automotive 53

O'Reilly

MARCH 2, 2020

It’s not about getting software developers to write code faster. Perhaps the appropriate yardstick for AI projects is the experiment itself, not the code committed to git.). This has to do with the concept of bounded context from Domain Driven Design.). Key survey results: The C-suite is engaged with data quality.

Artificial Intelligence 21

Artificial Intelligence Software Architecture Programming Infrastructure 21

The Netflix TechBlog

SEPTEMBER 10, 2019

Our data scientists faced numerous challenges in our previous infrastructure. Complex business logic was embedded directly into the ETL pipelines by data engineers. In order to replicate results, scientists had to delve deep into the data, code, and documentation.

Metrics 215

Metrics Architecture Infrastructure Innovation 215

The Netflix TechBlog

JANUARY 11, 2022

To directly support great decision-making throughout the company, there are a number of data science teams at Netflix that partner directly with Product Managers, engineering teams, and other business units to design, execute, and learn from experiments. Curious to learn about what it’s like to be a Data Engineer at Netflix?

Innovation 243

Innovation Metrics Engineering Testing 243