High Scalability

APRIL 14, 2020

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! Learn to balance architecture trade-offs and design scalable enterprise-level software.

Education 105

Education Software Engineering Scalability Engineering 105

High Scalability

MAY 12, 2020

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! Learn to balance architecture trade-offs and design scalable enterprise-level software.

Education 102

Education Software Engineering Scalability Engineering 102

High Scalability

JANUARY 11, 2022

The streaming data store makes the system extensible to support other use-cases (e.g. FUN FACT : In this talk , Rodrigo Schmidt, director of engineering at Instagram talks about the different challenges they have faced in scaling the data infrastructure at Instagram. System Components. Streaming Data Model.

Design 334

Design Media Storage Logistics 334

High Scalability

APRIL 28, 2020

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! Learn to balance architecture trade-offs and design scalable enterprise-level software.

Education 102

Education Software Engineering Scalability Engineering 102

High Scalability

MAY 26, 2020

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! Created by former senior-level AWS engineers of 15 years. Who's Hiring? Apply here.

Education 100

Education AWS Software Engineering Scalability 100

High Scalability

MARCH 30, 2020

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! Learn how to train interviewers so everyone's on the same page. Who's Hiring?

Education 105

Education Software Engineering Engineering Big Data 105

High Scalability

MARCH 24, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems : large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc.

Education 105

Education Software Engineering Engineering Big Data 105

Dynatrace

JUNE 15, 2021

Software reliability and resiliency don’t just happen by simply moving your software to a modern stack, or by moving your workloads to the cloud. The fact is, Reliability and Resiliency must be rooted in the architecture of a distributed system. Dynatrace news. Fact #2: No significant impact on Dynatrace Users.

AWS 186

AWS Traffic Architecture Azure 186

The Morning Paper

FEBRUARY 18, 2020

The key insight this led to is that ML models are software , derived from data. ML learning systems combine the characteristics of software, e.g. the need for CI/CD pipelines, and of data, e.g. the need to track lineage. Many of the software engineering discipline and controls need to be brought over into an ML context.

Government 95

Government Software Engineering Storage Database 95

High Scalability

MARCH 17, 2020

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Learn how to train interviewers so everyone's on the same page. Apply here.

Education 105

Education Engineering Java Software Engineering 105

High Scalability

MARCH 3, 2020

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Learn how to train interviewers so everyone's on the same page. Apply here.

Education 114

Education Games Engineering Java 114

High Scalability

FEBRUARY 18, 2020

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Learn how to train interviewers so everyone's on the same page. Apply here.

Education 113

Education Games Engineering Java 113

High Scalability

FEBRUARY 9, 2020

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Learn how to train interviewers so everyone's on the same page. Apply here.

Education 108

Education Games Engineering Java 108

O'Reilly

FEBRUARY 2, 2020

First, the behavior of an AI application depends on a model , which is built from source code and training data. You need a repository for models and for the training data. And that’s essential: if you need to understand how your model behaves, and you don’t have the training data, you’re sunk.

Software Architecture 68

Software Architecture Monitoring Software Engineering Architecture 68

O'Reilly Software

JANUARY 12, 2018

2018 marks the fourth year of O’Reilly’s Software Architecture Conference , a software engineering event focused on providing hands-on training experiences for technologists at all levels of an organization—from experienced developers up through CTOs. Building evolutionary software architecture.

Technology 75

Technology Technology Engineering Software Architecture 75

Tasktop

JULY 7, 2021

Senior Systems Administrator, Michelle Santos, embraced a different type of cloud at Golden Ears Provincial Park in BC. Sébastien Coquelin, Staff Software Engineer, and his family found themselves dwarfed by the trees on Vancouver Island. Software Engineer, Nolan Mullins, hiked the Rockies with red wine for sustinence.

Energy 52

Energy Software Engineering Education Engineering 52

O'Reilly

NOVEMBER 12, 2019

We suspect this points to a general drift toward software teams taking more responsibility for infrastructure, and increasingly, enabled by serverless options. As noted earlier, the majority of survey respondents are software engineers. Industries of survey respondents. Organization size of survey respondents.

Serverless 88

Serverless Architecture FinTech Infrastructure 88

The Morning Paper

APRIL 16, 2019

Teaching rigorous distributed systems with efficient model checking Michael et al., It describes the labs environment, DSLabs , developed at the University of Washington to accompany a course in distributed systems. Enabling students to build running performant versions of all of those systems in the time available is one challenge.

Systems 43

Systems Efficiency Testing Strategy 43

The Agile Manager

JANUARY 31, 2014

The growth in demand for software has consistently outpaced the growth in the supply of software developers. Each major expansion in software development - automation (60s), productivity (80s), internet (90s), mobile (00s) - has been additive to the total stock of software in the world. It's worth looking at why.

Software 73

Software Software Development Government 73

Strategic Tech

SEPTEMBER 29, 2018

finding good software engineers takes so long and requires so much effort… but it doesn’t have to. Improving your hiring fortunes is not just about optimising your hiring process, it’s about making systemic changes to your organisation. Hiring is so hard?—?finding Contact me if you are interested or would like to know more.

Software Engineering 40

Software Engineering Open Source Engineering Java 40

Adrian Cockcroft

FEBRUARY 13, 2023

Discussion of the water positive goal and optimizations that let AWS return water directly for irrigation rather than to the sewage system. Provides comparison of inference workload on P4dn GPU instances vs. AWS Trainium saving 92% energy and 90% cost, and training workload on P4dn vs. AWS Inferentia 2.6x

AWS 64

AWS Energy Architecture Programming 64

ACM Sigarch

APRIL 8, 2019

However, we often hear anecdotes that the number of prospective graduate students applying to computer architecture/systems is small and shrinking. Her current work focuses on hardware/software co-design for extremely large-scale deep learning training. Lena Olson is a Software Engineer at Google. . Why is that?

Architecture 74

Architecture Open Source Hardware Software Engineering 74

SQL Server According to Bob

JUNE 9, 2017

This table shows a capture from sys.dm_exec_requests on my system. The hotspot causes a convoy/train to occur on the page latch (UP) for the PFS page. Note: Adding multiple TEMPDB files resolves the train/convoy. The cleanup activity created a convey/train of latch activity on the PFS page in TEMPDB, leading to blocking.

Servers 40

Servers Software Engineering Engineering Software 40

All Things Distributed

SEPTEMBER 4, 2017

And thirdly, an "algorithmic revolution" has taken place, meaning it is now possible to train trillions of algorithms simultaneously, making the whole machine learning process much faster. For the past 20 years, thousands of software engineers at Amazon have been working on ML. Machine Leaning at Amazon.

Artificial Intelligence 147

Artificial Intelligence Transportation Education Airlines 147

Brendan Gregg

APRIL 30, 2018

USENIX’s LISA conference is the premier event for topics in production system engineering. We both have had long careers supporting system administration, and LISA has always felt like a homecoming, reuniting with old friends while welcoming newcomers. Submit #lisa18 talk, training, panel, and demo proposals by May 24 [link].

DevOps 43

DevOps Network Best Practices Programming 43

Brendan Gregg

APRIL 29, 2018

USENIX’s LISA conference is the premier event for topics in production system engineering. We both have had long careers supporting system administration, and LISA has always felt like a homecoming, reuniting with old friends while welcoming newcomers. Submit #lisa18 talk, training, panel, and demo proposals by May 24 [link].

DevOps 40

DevOps Network Best Practices Programming 40

O'Reilly

OCTOBER 19, 2021

This is both frustrating for companies that would prefer making ML an ordinary, fuss-free value-generating function like software engineering, as well as exciting for vendors who see the opportunity to create buzz around a new category of enterprise software. All ML projects are software projects.

DevOps 138

DevOps Software Engineering Infrastructure Open Source 138

Strategic Tech

JANUARY 22, 2024

external consultants or training) Preference for waiting for the problems to arise (indicating the need for an AMET) or forming an AMET early to prevent issues arising Confidence level that modernization will actually happen (sometimes it is just too unclear and lacking priority) How quickly do you want/need to modernize?

Architecture 98

Architecture Metrics Logistics Strategy 98

The Netflix TechBlog

OCTOBER 18, 2022

These include ETL pipelines, ML model training workflows, batch jobs, etc. Due to its popularity, the number of workflows managed by the system has grown exponentially. The scheduler on-call has to closely monitor the system during non-business hours. A large number of batch workflows run daily to serve various business needs.

Java 202

Java Scalability Traffic Architecture 202

The Netflix TechBlog

APRIL 26, 2022

We built Axion primarily to remove any training-serving skew and make offline experimentation faster. We make sure there is no training/serving skew by using the same data and the code for online and offline feature generation. Our machine learning models train on several weeks of data. What’s our end goal?

Storage 187

Storage Design Scalability Latency 187

Abhishek Tiwari

JULY 20, 2019

Warehouse engineering squad - managing software services related inventory, stocktake, dispatch, allocation, transfer, robotics, etc. Customer experience engineering squad - focus on end-to-end customer life-cycle, marketing, targeting, personalisation, loyalty, etc. You want to move fast. How is that even possible?

Engineering 61

Engineering Retail Airlines Healthcare 61

The Netflix TechBlog

OCTOBER 16, 2019

Dataset propagation At Netflix we use an in-house dataset pub/sub system called Gutenberg. Gutenberg allows browsing older versions of data for use cases such as debugging, rapid mitigation of data related incidents, and re-training of machine-learning models. for example to train machine-learned models.

Cache 187

Cache Architecture Metrics Java 187

O'Reilly

JULY 15, 2020

Almost one-third (29%) of respondents say their employers are migrating or implementing a majority of their systems (over 50%) using microservices. Success with microservices means owning the software lifecycle. Most (74%) respondents say their teams own the build-test-deploy-maintain phases of the software lifecycle.

Database 135

Database Architecture Education Systems 135

O'Reilly

JULY 20, 2021

Most recently, IBM used Flickr photos that were shared under a Creative Commons license to train facial recognition algorithms. You will find that the paradigms you choose for other parties won’t align with the expectations for children, and modifying your software to accommodate children is difficult or impossible.

Google 121

Google Games Technology Technology 121

O'Reilly

MAY 19, 2020

Live online training, videos, books, certification prep, and more, from O’Reilly and our partner publishers. Software engineers represent the largest cohort, comprising almost 20% of all respondents (see Figure 1 ). Technical leads and architects (about 11%) are next, followed by software and systems architects (9+%).

Cloud 141

Cloud Serverless AWS Architecture 141

The Agile Manager

SEPTEMBER 25, 2009

Remember, although the Marines put all recruits through a rigorous 12-week training program, every single one of those recruits is still just an entry-level soldier at its conclusion. It can take years to train and groom people for more challenging roles. Why should you expect different for software engineers?

Automotive 40

Automotive Software Engineering Programming Metrics 40