High Scalability

MAY 26, 2020

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! this is going to be a challenging journey for any backend engineer! Who's Hiring?

Education 100

Education AWS Software Engineering Scalability 100

High Scalability

MARCH 30, 2020

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! this is going to be a challenging journey for any backend engineer! Who's Hiring?

Education 105

Education Software Engineering Engineering Big Data 105

High Scalability

MARCH 24, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems : large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc.

Education 105

Education Software Engineering Engineering Big Data 105

High Scalability

MARCH 3, 2020

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Apply here. Advertise your job here! Cool Products and Services.

Education 114

Education Games Engineering Java 114

High Scalability

MARCH 17, 2020

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Apply here. Advertise your job here! Cool Products and Services.

Education 105

Education Engineering Java Software Engineering 105

High Scalability

FEBRUARY 18, 2020

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Apply here. Advertise your job here! Cool Products and Services.

Education 113

Education Games Engineering Java 113

High Scalability

FEBRUARY 9, 2020

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Apply here. Advertise your job here! Cool Products and Services.

Education 108

Education Games Engineering Java 108

The Netflix TechBlog

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! Technology advancements in content creation and consumption have also increased its data footprint. We explore all the systems necessary to make and stream content from Netflix.

AWS 100

AWS Entertainment Open Source Benchmarking 100

The Netflix TechBlog

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! Technology advancements in content creation and consumption have also increased its data footprint. We explore all the systems necessary to make and stream content from Netflix.

AWS 100

AWS Entertainment Open Source Benchmarking 100

Abhishek Tiwari

JULY 12, 2018

In recent times, in order to gain valuable insights or to develop the data-driven products companies such as Netflix, Spotify, Uber, AirBnB have built internal data pipelines. If built correctly, data pipelines can offer strategic advantages to the business. In a nutshell, a data pipeline is a distributed system.

Latency 63

Latency Analytics Scalability Engineering 63

All Things Distributed

AUGUST 24, 2012

Werner Vogels weblog on building scalable and robust distributed systems. Which makes this week a good moment to read up on some of the historical work around the costs of data engineering. All Things Distributed. Back-to-Basics Weekend Reading - The 5 Minute Rule. By Werner Vogels on 24 August 2012 04:00 PM.

Storage 108

Storage Hardware AWS Data Engineering 108

The Netflix TechBlog

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! Technology advancements in content creation and consumption have also increased its data footprint. We’ve compiled our speaking events below so you know what we’ve been working on.

AWS 37

AWS Entertainment Open Source Benchmarking 37

Abhishek Tiwari

DECEMBER 27, 2017

Kubernetes has emerged as go to container orchestration platform for data engineering teams. In 2018, a widespread adaptation of Kubernetes for big data processing is anitcipated. Organisations are already using Kubernetes for a variety of workloads [1] [2] and data workloads are up next. Native frameworks.

Big Data 63

Big Data Storage Benchmarking Hardware 63

High Scalability

JANUARY 22, 2020

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Apply here. Advertise your job here! Cool Products and Services.

Education 22

Education Java Software Engineering Engineering 22

High Scalability

NOVEMBER 12, 2019

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Apply here. Stateful JavaScript Apps. Generous free tier.

Education 22

Education Java Software Engineering Engineering 22

High Scalability

FEBRUARY 6, 2020

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Apply here. Advertise your job here! Cool Products and Services.

Education 21

Education Java Software Engineering Engineering 21

High Scalability

OCTOBER 29, 2019

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Apply here. Stateful JavaScript Apps. Generous free tier.

Education 22

Education Java Software Engineering Engineering 22

High Scalability

JANUARY 7, 2020

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Apply here. Advertise your job here! Cool Products and Services.

Education 21

Education Java Software Engineering Engineering 21

High Scalability

DECEMBER 12, 2019

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Apply here. Advertise your job here! Cool Products and Services.

Education 21

Education Java Software Engineering Engineering 21

InfoQ

JULY 20, 2023

Microsoft recently announced the general availability (GA) of Azure Managed Lustre, a managed file system for high-performance computing (HPC) and AI workloads. By Steef-Jan Wiggers

Azure 40

Azure Availability Systems Performance 40

The Netflix TechBlog

SEPTEMBER 18, 2020

Analytics at Netflix: Who We Are and What We Do An Introduction to Analytics and Visualization Engineering at Netflix by Molly Jackman & Meghana Reddy Explained: Season 1 (Photo Credit: Netflix) Across nearly every industry, there is recognition that data analytics is key to driving informed business decision-making.

Analytics 240

Analytics Engineering Data Engineering Tuning 240

The Netflix TechBlog

OCTOBER 18, 2022

As Big data and ML became more prevalent and impactful, the scalability, reliability, and usability of the orchestrating ecosystem have increasingly become more important for our data scientists and the company. Due to its popularity, the number of workflows managed by the system has grown exponentially.

Java 202

Java Scalability Traffic Architecture 202

The Netflix TechBlog

MARCH 25, 2019

Now, imagine yourself in the role of a software engineer responsible for a micro-service which publishes data consumed by few critical customer facing services (e.g. You are about to make structural changes to the data and want to know who and what downstream to your service will be impacted.

Infrastructure 219

Infrastructure Big Data Transportation Architecture 219

The Netflix TechBlog

DECEMBER 21, 2020

Use cases We found several use cases where a system like AutoOptimize can bring tons of value. Some of the optimizations are prerequisites for a high-performance data warehouse. Sometimes Data Engineers write downstream ETLs on ingested data to optimize the data/metadata layouts to make other ETL processes cheaper and faster.

Storage 203

Storage Latency Efficiency Data Engineering 203

Abhishek Tiwari

JULY 7, 2023

The DAG Model and the Misconception Data pipelines are commonly implemented as Directed Acyclic Graphs (DAGs), where data flows through a series of processing steps, with each step represented as a node and the dependencies between steps represented as edges.

Logistics 52

Logistics Transportation Scalability Data Engineering 52

O'Reilly

JULY 15, 2020

Almost one-third (29%) of respondents say their employers are migrating or implementing a majority of their systems (over 50%) using microservices. Software engineers comprise the survey audience’s single largest cluster, over one quarter (27%) of respondents (Figure 1). Adopters are betting big on microservices. What does that mean?

Database 135

Database Architecture Education Systems 135