High Scalability

APRIL 28, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! T riplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Try out their platform. Apply here.

Education 102

Education Software Engineering Scalability Engineering 102

High Scalability

MAY 26, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! T riplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Try out their platform. Apply here.

Education 100

Education AWS Software Engineering Scalability 100

The Morning Paper

FEBRUARY 18, 2020

Many of the software engineering discipline and controls need to be brought over into an ML context. The following chart breaks down features in three main areas: training and auditing, serving and deployment, and data management, across six systems. But model inference migrating into the DBMS is a bolder prediction.

Government 95

Government Software Engineering Storage Database 95

O'Reilly

NOVEMBER 12, 2019

For the inaugural O’Reilly survey on serverless architecture adoption, we were pleasantly surprised at the high level of response: more than 1,500 respondents from a wide range of locations, companies, and industries participated. As noted earlier, the majority of survey respondents are software engineers.

Serverless 87

Serverless Architecture FinTech Infrastructure 87

O'Reilly Software

JANUARY 12, 2018

2018 marks the fourth year of O’Reilly’s Software Architecture Conference , a software engineering event focused on providing hands-on training experiences for technologists at all levels of an organization—from experienced developers up through CTOs. Building evolutionary software architecture.

Technology 75

Technology Technology Engineering Software Architecture 75

Adrian Cockcroft

FEBRUARY 13, 2023

Provides comparison of inference workload on P4dn GPU instances vs. AWS Trainium saving 92% energy and 90% cost, and training workload on P4dn vs. AWS Inferentia 2.6x shorter training time, saving 54% energy and 75% cost. SUS302 Optimizing architectures for sustainability — Katja Philipp AWS SA and Szymon Kochanski AWS SA.

AWS 64

AWS Energy Architecture Programming 64

O'Reilly

FEBRUARY 2, 2020

First, the behavior of an AI application depends on a model , which is built from source code and training data. You need a repository for models and for the training data. And that’s essential: if you need to understand how your model behaves, and you don’t have the training data, you’re sunk.

Software Architecture 68

Software Architecture Monitoring Software Engineering Architecture 68

Strategic Tech

JANUARY 22, 2024

Architecture modernization initiatives are strategic efforts involving many teams, usually for many months or years. An AMET is an architecture Enabling Team that helps to coordinate and upskill all teams and stakeholders involved in a modernization initiative. They need a more loosely coupled architecture and empowered teams.

Architecture 98

Architecture Metrics Logistics Strategy 98

The Morning Paper

FEBRUARY 16, 2020

In Software 1.0 But in Software 2.0 the majority of our effort goes into curating training data, i.e., specification-by-example of what the system should do. The existing system was based on a rule-based architecture with hand-crafted rules and had been in production since 2013. " Switching to a Software 2.0

Systems 84

Systems Design Software Software 84

All Things Distributed

SEPTEMBER 4, 2017

And thirdly, an "algorithmic revolution" has taken place, meaning it is now possible to train trillions of algorithms simultaneously, making the whole machine learning process much faster. For the past 20 years, thousands of software engineers at Amazon have been working on ML. Machine Leaning at Amazon.

Artificial Intelligence 149

Artificial Intelligence Transportation Education Airlines 149

Brendan Gregg

APRIL 30, 2018

Today's LISA attracts attendees working on all sizes of production systems, and its attendees include sysadmins, systems engineers, SREs, DevOps engineers, software engineers, IT managers, security engineers, network administrators, researchers, students, and more. Hope to see you in Nashville!

DevOps 43

DevOps Network Best Practices Programming 43

Brendan Gregg

APRIL 29, 2018

Today's LISA attracts attendees working on all sizes of production systems, and its attendees include sysadmins, systems engineers, SREs, DevOps engineers, software engineers, IT managers, security engineers, network administrators, researchers, students, and more. Hope to see you in Nashville!

DevOps 40

DevOps Network Best Practices Programming 40

O'Reilly

OCTOBER 19, 2021

This is both frustrating for companies that would prefer making ML an ordinary, fuss-free value-generating function like software engineering, as well as exciting for vendors who see the opportunity to create buzz around a new category of enterprise software. Software Architecture. This approach is not novel.

DevOps 138

DevOps Software Engineering Open Source Infrastructure 138

The Netflix TechBlog

DECEMBER 3, 2019

mainly because of mundane reasons related to software engineering. On the other hand, very few data scientists feel strongly about the nature of the data warehouse, the compute platform that trains and scores their models, or the workflow scheduler. The steps can be arbitrary Python code.

Open Source 232

Open Source AWS Infrastructure Energy 232

The Netflix TechBlog

OCTOBER 18, 2022

These include ETL pipelines, ML model training workflows, batch jobs, etc. Meson was based on a single leader architecture with high availability. Similarly, ML model training workflows usually consist of tens of thousands of training jobs within a single workflow. Figure 1 shows the high-level architecture.

Java 202

Java Scalability Traffic Architecture 202

The Netflix TechBlog

APRIL 26, 2022

We built Axion primarily to remove any training-serving skew and make offline experimentation faster. Figure 1: Netflix ML Architecture Fact: A fact is data about our members or videos. We make sure there is no training/serving skew by using the same data and the code for online and offline feature generation.

Storage 187

Storage Design Scalability Latency 187

The Netflix TechBlog

DECEMBER 3, 2019

mainly because of mundane reasons related to software engineering. On the other hand, very few data scientists feel strongly about the nature of the data warehouse, the compute platform that trains and scores their models, or the workflow scheduler. The steps can be arbitrary Python code.

Open Source 134

Open Source AWS Infrastructure Energy 134

The Netflix TechBlog

OCTOBER 16, 2019

By Ammar Khaku Introduction In a microservice architecture such as Netflix’s, propagating datasets from a single source to multiple downstream destinations can be challenging. This post is a high level overview of the design and architecture of Gutenberg. for example to train machine-learned models.

Cache 187

Cache Architecture Metrics Java 187

Abhishek Tiwari

JULY 20, 2019

The engineering organisation described may not work for you because of a team of 8-10 people is still a very big overhead. In this model, software architecture and code ownership is a reflection of the organisational model. Is it possible to draw inspiration from outside of software engineering? Probably yes.

Engineering 61

Engineering Retail Airlines Healthcare 61

The Netflix TechBlog

NOVEMBER 20, 2023

Whether in analyzing A/B tests, optimizing studio production, training algorithms, investing in content acquisition, detecting security breaches, or optimizing payments, well structured and accurate data is foundational. IPS enables users to continue to use the data processing patterns with minimal changes.

Processing 210

Processing Big Data Efficiency Engineering 210

O'Reilly

MAY 19, 2020

Live online training, videos, books, certification prep, and more, from O’Reilly and our partner publishers. More than one-third have adopted site reliability engineering (SRE); slightly less have developed production AI services. For this audience, SRE’s future is brighter than AI’s, however. Respondent Demographics.

Cloud 141

Cloud Serverless AWS Architecture 141

O'Reilly

JULY 15, 2020

Software engineers comprise the survey audience’s single largest cluster, over one quarter (27%) of respondents (Figure 1). If you combine the different architectural roles—i.e., software and systems architects, technical leads—architects represent almost 28% of the sample. Respondent Demographics.

Database 135

Database Architecture Education Systems 135