High Scalability

APRIL 14, 2020

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! Learn to balance architecture trade-offs and design scalable enterprise-level software.

Education 105

Education Software Engineering Scalability Engineering 105

High Scalability

MAY 12, 2020

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! Learn to balance architecture trade-offs and design scalable enterprise-level software.

Education 102

Education Software Engineering Scalability Engineering 102

High Scalability

APRIL 28, 2020

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! Learn to balance architecture trade-offs and design scalable enterprise-level software.

Education 102

Education Software Engineering Scalability Engineering 102

High Scalability

MAY 26, 2020

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! Created by former senior-level AWS engineers of 15 years. Who's Hiring? Apply here.

Education 100

Education AWS Software Engineering Scalability 100

High Scalability

MARCH 24, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems : large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc.

Education 105

Education Software Engineering Engineering Big Data 105

High Scalability

MARCH 30, 2020

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! The result is a process that doesn't get you the best employees you could.

Education 105

Education Software Engineering Engineering Big Data 105

High Scalability

MARCH 3, 2020

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. The result is a process that doesn't get you the best employees you could.

Education 114

Education Games Engineering Java 114

High Scalability

MARCH 17, 2020

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. The result is a process that doesn't get you the best employees you could.

Education 105

Education Engineering Java Software Engineering 105

High Scalability

FEBRUARY 18, 2020

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. The result is a process that doesn't get you the best employees you could.

Education 113

Education Games Engineering Java 113

High Scalability

FEBRUARY 9, 2020

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. The result is a process that doesn't get you the best employees you could.

Education 108

Education Games Engineering Java 108

Dynatrace

JUNE 15, 2021

And the last sentence of the email was what made me want to share this story publicly, as it’s a testimonial to how modern software engineering and operations should make you feel. Our dynamic growth in engineering also made us invest in continuous training for new and existing hires.

AWS 182

AWS Traffic Architecture Azure 182

The Netflix TechBlog

NOVEMBER 20, 2023

by Jun He , Yingyi Zhang , and Pawan Dixit Incremental processing is an approach to process new or changed data in workflows. The key advantage is that it only incrementally processes data that are newly added or updated to a dataset, instead of re-processing the complete dataset.

Processing 210

Processing Big Data Efficiency Engineering 210

ACM Sigarch

APRIL 8, 2019

big-data processing, machine learning, quantum computing, and so on). Her current work focuses on hardware/software co-design for extremely large-scale deep learning training. Lena Olson is a Software Engineer at Google. . About the Authors: Newsha Ardalani is a Research Scientist at Baidu AI Lab.

Architecture 74

Architecture Open Source Hardware Software Engineering 74

SQL Server According to Bob

JANUARY 24, 2017

In my last post I highlighted the marriage of PE and ELF images within the same process space to build SQL Server on Linux. Many of you have used WinDbg , Visual Studio or other other debugging utilities but none of these are designed for PE and ELF in the same process space. Target can be a live process or a dump file.).

Servers 40

Servers Open Source Software Engineering Processing 40

The Morning Paper

APRIL 16, 2019

We would like students to consider the performance characteristics of their systems, and our tests check that their designs attain reasonable run-time performance. In addition, another lesson here is that once we accept model checking is necessary, we need to design systems that are amenable to model checking.

Systems 43

Systems Efficiency Testing Strategy 43

O'Reilly

OCTOBER 19, 2021

This is both frustrating for companies that would prefer making ML an ordinary, fuss-free value-generating function like software engineering, as well as exciting for vendors who see the opportunity to create buzz around a new category of enterprise software. All ML projects are software projects.

DevOps 138

DevOps Software Engineering Infrastructure Open Source 138

The Netflix TechBlog

DECEMBER 3, 2019

mainly because of mundane reasons related to software engineering. We heard many stories about difficulties related to data access and basic data processing. By design, Metaflow is a deceptively simple Python library: Data scientists can structure their workflow as a Directed Acyclic Graph of steps, as depicted above.

Open Source 232

Open Source AWS Infrastructure Energy 232

The Netflix TechBlog

APRIL 26, 2022

We built Axion primarily to remove any training-serving skew and make offline experimentation faster. We will share how its design has evolved over the years and the lessons learned while building it. To understand Axion’s design, we need to know the various components that interact with it.

Storage 187

Storage Design Scalability Latency 187

Strategic Tech

JANUARY 22, 2024

external consultants or training) Preference for waiting for the problems to arise (indicating the need for an AMET) or forming an AMET early to prevent issues arising Confidence level that modernization will actually happen (sometimes it is just too unclear and lacking priority) How quickly do you want/need to modernize?

Architecture 98

Architecture Metrics Logistics Strategy 98

The Netflix TechBlog

DECEMBER 3, 2019

mainly because of mundane reasons related to software engineering. We heard many stories about difficulties related to data access and basic data processing. By design, Metaflow is a deceptively simple Python library: Data scientists can structure their workflow as a Directed Acyclic Graph of steps, as depicted above.

Open Source 134

Open Source AWS Infrastructure Energy 134

The Netflix TechBlog

OCTOBER 18, 2022

These include ETL pipelines, ML model training workflows, batch jobs, etc. For example, a workflow to backfill hourly data for the past five years can lead to 43800 jobs (24 * 365 * 5), each of which processes data for an hour. All the requests are processed via distributed queues for message passing.

Java 202

Java Scalability Traffic Architecture 202

The Netflix TechBlog

OCTOBER 16, 2019

Gutenberg allows browsing older versions of data for use cases such as debugging, rapid mitigation of data related incidents, and re-training of machine-learning models. This post is a high level overview of the design and architecture of Gutenberg. for example to train machine-learned models.

Cache 187

Cache Architecture Metrics Java 187

Abhishek Tiwari

JULY 20, 2019

One thing stand-out to me is being intentional and practical about your engineering organisation design. First and foremost, being intentional about organisation design requires good and honest discussions about all possible option. Specialisation could be around products, business process, or technologies. Certainly not.

Engineering 61

Engineering Retail Airlines Healthcare 61

O'Reilly

MAY 19, 2020

Live online training, videos, books, certification prep, and more, from O’Reilly and our partner publishers. More than one-third have adopted site reliability engineering (SRE); slightly less have developed production AI services. Designing a survey inevitably entails making a spate of methodological trade-offs.

Cloud 141

Cloud Serverless AWS Architecture 141