High Scalability

MAY 12, 2020

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! Learn to balance architecture trade-offs and design scalable enterprise-level software.

Education 102

Education Software Engineering Scalability Engineering 102

High Scalability

APRIL 28, 2020

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! Learn to balance architecture trade-offs and design scalable enterprise-level software.

Education 102

Education Software Engineering Scalability Engineering 102

High Scalability

MAY 26, 2020

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! Created by former senior-level AWS engineers of 15 years. Who's Hiring? Apply here.

Education 100

Education AWS Software Engineering Scalability 100

The Morning Paper

FEBRUARY 18, 2020

The key insight this led to is that ML models are software , derived from data. ML learning systems combine the characteristics of software, e.g. the need for CI/CD pipelines, and of data, e.g. the need to track lineage. Many of the software engineering discipline and controls need to be brought over into an ML context.

Government 95

Government Software Engineering Storage Database 95

O'Reilly Software

JANUARY 12, 2018

2018 marks the fourth year of O’Reilly’s Software Architecture Conference , a software engineering event focused on providing hands-on training experiences for technologists at all levels of an organization—from experienced developers up through CTOs. Building evolutionary software architecture.

Technology 75

Technology Technology Engineering Software Architecture 75

O'Reilly

FEBRUARY 2, 2020

First, the behavior of an AI application depends on a model , which is built from source code and training data. You need a repository for models and for the training data. And that’s essential: if you need to understand how your model behaves, and you don’t have the training data, you’re sunk.

Software Architecture 68

Software Architecture Monitoring Software Engineering Architecture 68

Adrian Cockcroft

FEBRUARY 13, 2023

Discussion of the water positive goal and optimizations that let AWS return water directly for irrigation rather than to the sewage system. Provides comparison of inference workload on P4dn GPU instances vs. AWS Trainium saving 92% energy and 90% cost, and training workload on P4dn vs. AWS Inferentia 2.6x

AWS 64

AWS Energy Architecture Programming 64

O'Reilly

NOVEMBER 12, 2019

For the inaugural O’Reilly survey on serverless architecture adoption, we were pleasantly surprised at the high level of response: more than 1,500 respondents from a wide range of locations, companies, and industries participated. As noted earlier, the majority of survey respondents are software engineers.

Serverless 88

Serverless Architecture FinTech Infrastructure 88

Strategic Tech

JANUARY 22, 2024

Architecture modernization initiatives are strategic efforts involving many teams, usually for many months or years. An AMET is an architecture Enabling Team that helps to coordinate and upskill all teams and stakeholders involved in a modernization initiative. They need a more loosely coupled architecture and empowered teams.

Architecture 98

Architecture Metrics Logistics Strategy 98

All Things Distributed

SEPTEMBER 4, 2017

And thirdly, an "algorithmic revolution" has taken place, meaning it is now possible to train trillions of algorithms simultaneously, making the whole machine learning process much faster. For the past 20 years, thousands of software engineers at Amazon have been working on ML. Machine Leaning at Amazon.

Artificial Intelligence 147

Artificial Intelligence Transportation Education Airlines 147

Brendan Gregg

APRIL 30, 2018

USENIX’s LISA conference is the premier event for topics in production system engineering. We both have had long careers supporting system administration, and LISA has always felt like a homecoming, reuniting with old friends while welcoming newcomers. Submit #lisa18 talk, training, panel, and demo proposals by May 24 [link].

DevOps 43

DevOps Network Best Practices Programming 43

Brendan Gregg

APRIL 29, 2018

USENIX’s LISA conference is the premier event for topics in production system engineering. We both have had long careers supporting system administration, and LISA has always felt like a homecoming, reuniting with old friends while welcoming newcomers. Submit #lisa18 talk, training, panel, and demo proposals by May 24 [link].

DevOps 40

DevOps Network Best Practices Programming 40

O'Reilly

OCTOBER 19, 2021

This is both frustrating for companies that would prefer making ML an ordinary, fuss-free value-generating function like software engineering, as well as exciting for vendors who see the opportunity to create buzz around a new category of enterprise software. The new category is often called MLOps. This approach is not novel.

DevOps 138

DevOps Software Engineering Infrastructure Open Source 138

The Netflix TechBlog

OCTOBER 18, 2022

These include ETL pipelines, ML model training workflows, batch jobs, etc. Due to its popularity, the number of workflows managed by the system has grown exponentially. The scheduler on-call has to closely monitor the system during non-business hours. Meson was based on a single leader architecture with high availability.

Java 202

Java Scalability Traffic Architecture 202

The Netflix TechBlog

APRIL 26, 2022

We built Axion primarily to remove any training-serving skew and make offline experimentation faster. Figure 1: Netflix ML Architecture Fact: A fact is data about our members or videos. We make sure there is no training/serving skew by using the same data and the code for online and offline feature generation.

Storage 187

Storage Design Scalability Latency 187

The Netflix TechBlog

OCTOBER 16, 2019

By Ammar Khaku Introduction In a microservice architecture such as Netflix’s, propagating datasets from a single source to multiple downstream destinations can be challenging. Dataset propagation At Netflix we use an in-house dataset pub/sub system called Gutenberg. for example to train machine-learned models.

Cache 187

Cache Architecture Metrics Java 187

Abhishek Tiwari

JULY 20, 2019

Store engineering squad - focus on software and systems required for the storefront including point-of-sales system, promotions, etc. ERP engineering squad - supply chain planning, purchase order management, product lifecycle management, merchandise planning, etc. You want to move fast. How is that even possible?

Engineering 61

Engineering Retail Airlines Healthcare 61

O'Reilly

JULY 15, 2020

Almost one-third (29%) of respondents say their employers are migrating or implementing a majority of their systems (over 50%) using microservices. Success with microservices means owning the software lifecycle. Most (74%) respondents say their teams own the build-test-deploy-maintain phases of the software lifecycle.

Database 135

Database Architecture Education Systems 135

O'Reilly

MAY 19, 2020

Live online training, videos, books, certification prep, and more, from O’Reilly and our partner publishers. Software engineers represent the largest cohort, comprising almost 20% of all respondents (see Figure 1 ). Technical leads and architects (about 11%) are next, followed by software and systems architects (9+%).

Cloud 141

Cloud Serverless AWS Architecture 141