High Scalability

APRIL 28, 2020

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! Learn to balance architecture trade-offs and design scalable enterprise-level software.

Education 102

Education Software Engineering Scalability Engineering 102

High Scalability

MARCH 30, 2020

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! Learn how to train interviewers so everyone's on the same page. Who's Hiring?

Education 105

Education Software Engineering Engineering Big Data 105

High Scalability

MARCH 24, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems : large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc.

Education 105

Education Software Engineering Engineering Big Data 105

High Scalability

MAY 26, 2020

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). this is going to be a challenging journey for any backend engineer! Created by former senior-level AWS engineers of 15 years. Who's Hiring? Apply here.

Education 100

Education AWS Software Engineering Scalability 100

The Morning Paper

FEBRUARY 18, 2020

The key insight this led to is that ML models are software , derived from data. ML learning systems combine the characteristics of software, e.g. the need for CI/CD pipelines, and of data, e.g. the need to track lineage. Many of the software engineering discipline and controls need to be brought over into an ML context.

Government 95

Government Software Engineering Storage Database 95

High Scalability

MARCH 17, 2020

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Learn how to train interviewers so everyone's on the same page. Apply here.

Education 105

Education Engineering Java Software Engineering 105

High Scalability

MARCH 3, 2020

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Learn how to train interviewers so everyone's on the same page. Apply here.

Education 114

Education Games Engineering Java 114

High Scalability

FEBRUARY 18, 2020

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Learn how to train interviewers so everyone's on the same page. Apply here.

Education 113

Education Games Engineering Java 113

High Scalability

FEBRUARY 9, 2020

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Learn how to train interviewers so everyone's on the same page. Apply here.

Education 108

Education Games Engineering Java 108

O'Reilly

FEBRUARY 2, 2020

First, the behavior of an AI application depends on a model , which is built from source code and training data. You need a repository for models and for the training data. And that’s essential: if you need to understand how your model behaves, and you don’t have the training data, you’re sunk.

Software Architecture 68

Software Architecture Monitoring Software Engineering Architecture 68

O'Reilly Software

JANUARY 12, 2018

2018 marks the fourth year of O’Reilly’s Software Architecture Conference , a software engineering event focused on providing hands-on training experiences for technologists at all levels of an organization—from experienced developers up through CTOs. Building evolutionary software architecture.

Technology 75

Technology Technology Engineering Software Architecture 75

Tasktop

JULY 7, 2021

Senior Systems Administrator, Michelle Santos, embraced a different type of cloud at Golden Ears Provincial Park in BC. Sébastien Coquelin, Staff Software Engineer, and his family found themselves dwarfed by the trees on Vancouver Island. Software Engineer, Nolan Mullins, hiked the Rockies with red wine for sustinence.

Energy 52

Energy Software Engineering Education Engineering 52

Adrian Cockcroft

FEBRUARY 13, 2023

Discussion of the water positive goal and optimizations that let AWS return water directly for irrigation rather than to the sewage system. Provides comparison of inference workload on P4dn GPU instances vs. AWS Trainium saving 92% energy and 90% cost, and training workload on P4dn vs. AWS Inferentia 2.6x

AWS 64

AWS Energy Architecture Programming 64

Strategic Tech

SEPTEMBER 29, 2018

finding good software engineers takes so long and requires so much effort… but it doesn’t have to. Improving your hiring fortunes is not just about optimising your hiring process, it’s about making systemic changes to your organisation. Hiring is so hard?—?finding Contact me if you are interested or would like to know more.

Software Engineering 40

Software Engineering Open Source Engineering Java 40

The Morning Paper

JULY 7, 2019

Software engineering for machine learning: a case study Amershi et al., More specifically, we’ll be looking at the results of an internal study with over 500 participants designed to figure out how product development and software engineering is changing at Microsoft with the rise of AI and ML. ICSE’19.

Software Engineering 92

Software Engineering Engineering Software Software 92

SQL Server According to Bob

JUNE 9, 2017

This table shows a capture from sys.dm_exec_requests on my system. The hotspot causes a convoy/train to occur on the page latch (UP) for the PFS page. Note: Adding multiple TEMPDB files resolves the train/convoy. The cleanup activity created a convey/train of latch activity on the PFS page in TEMPDB, leading to blocking.

Servers 40

Servers Software Engineering Engineering Software 40

All Things Distributed

SEPTEMBER 4, 2017

And thirdly, an "algorithmic revolution" has taken place, meaning it is now possible to train trillions of algorithms simultaneously, making the whole machine learning process much faster. For the past 20 years, thousands of software engineers at Amazon have been working on ML. Machine Leaning at Amazon.

Artificial Intelligence 149

Artificial Intelligence Transportation Education Airlines 149

Brendan Gregg

APRIL 30, 2018

USENIX’s LISA conference is the premier event for topics in production system engineering. We both have had long careers supporting system administration, and LISA has always felt like a homecoming, reuniting with old friends while welcoming newcomers. Submit #lisa18 talk, training, panel, and demo proposals by May 24 [link].

DevOps 43

DevOps Network Best Practices Programming 43

Brendan Gregg

APRIL 29, 2018

USENIX’s LISA conference is the premier event for topics in production system engineering. We both have had long careers supporting system administration, and LISA has always felt like a homecoming, reuniting with old friends while welcoming newcomers. Submit #lisa18 talk, training, panel, and demo proposals by May 24 [link].

DevOps 40

DevOps Network Best Practices Programming 40

The Agile Manager

JANUARY 31, 2014

Yet we continue to find new applications for software: it is increasingly a product differentiator (embedded systems) or a product category of its own (social networking). Still, people investing in software assets will not take "can't find competent people" for an answer. The risk of spectacular software failure will increase.

Software 73

Software Software Development Government 73

J. Paul Reed

MARCH 14, 2019

The 737Max and Why Software Engineers Might Want to Pay Attention As someone with a bit of a reputation for talking about aviation and software development and operations , I’ve been asked about the 737Max repeatedly over the past week. the part under control of the automatic system?—?can

Software Engineering 75

Software Engineering Engineering Software Software 75

O'Reilly

OCTOBER 19, 2021

This is both frustrating for companies that would prefer making ML an ordinary, fuss-free value-generating function like software engineering, as well as exciting for vendors who see the opportunity to create buzz around a new category of enterprise software. The new category is often called MLOps. This approach is not novel.

DevOps 138

DevOps Software Engineering Infrastructure Open Source 138

DZone

FEBRUARY 27, 2021

As a Software Engineer, the mind is trained to seek optimizations in every aspect of development and ooze out every bit of available CPU Resource to deliver a performing application. They still will win for mission-critical or real-time systems, which need performance over these parameters.

Java 207

Java C++ Benchmarking Programming 207

Strategic Tech

JANUARY 22, 2024

external consultants or training) Preference for waiting for the problems to arise (indicating the need for an AMET) or forming an AMET early to prevent issues arising Confidence level that modernization will actually happen (sometimes it is just too unclear and lacking priority) How quickly do you want/need to modernize?

Architecture 98

Architecture Metrics Logistics Strategy 98

The Netflix TechBlog

OCTOBER 18, 2022

These include ETL pipelines, ML model training workflows, batch jobs, etc. Due to its popularity, the number of workflows managed by the system has grown exponentially. The scheduler on-call has to closely monitor the system during non-business hours. A large number of batch workflows run daily to serve various business needs.

Java 202

Java Scalability Traffic Architecture 202

The Netflix TechBlog

APRIL 26, 2022

We built Axion primarily to remove any training-serving skew and make offline experimentation faster. We make sure there is no training/serving skew by using the same data and the code for online and offline feature generation. Our machine learning models train on several weeks of data. What’s our end goal?

Storage 187

Storage Design Scalability Latency 187

Dynatrace

JUNE 15, 2021

The fact is, Reliability and Resiliency must be rooted in the architecture of a distributed system. And the last sentence of the email was what made me want to share this story publicly, as it’s a testimonial to how modern software engineering and operations should make you feel.

AWS 192

AWS Traffic Architecture Azure 192

The Morning Paper

APRIL 16, 2019

Teaching rigorous distributed systems with efficient model checking Michael et al., It describes the labs environment, DSLabs , developed at the University of Washington to accompany a course in distributed systems. Enabling students to build running performant versions of all of those systems in the time available is one challenge.

Systems 43

Systems Efficiency Testing Strategy 43

O'Reilly

NOVEMBER 12, 2019

More than a fifth of the respondents work in the software industry—skewing results toward the concerns of software companies, and helping explain the preponderance of those with software engineering roles. As noted earlier, the majority of survey respondents are software engineers.

Serverless 88

Serverless Architecture FinTech Infrastructure 88

The Netflix TechBlog

OCTOBER 16, 2019

Dataset propagation At Netflix we use an in-house dataset pub/sub system called Gutenberg. Gutenberg allows browsing older versions of data for use cases such as debugging, rapid mitigation of data related incidents, and re-training of machine-learning models. for example to train machine-learned models.

Cache 187

Cache Architecture Metrics Java 187

Abhishek Tiwari

JULY 20, 2019

Store engineering squad - focus on software and systems required for the storefront including point-of-sales system, promotions, etc. ERP engineering squad - supply chain planning, purchase order management, product lifecycle management, merchandise planning, etc. product) don't change over a long period.

Engineering 61

Engineering Retail Airlines Healthcare 61

O'Reilly

JULY 15, 2020

Almost one-third (29%) of respondents say their employers are migrating or implementing a majority of their systems (over 50%) using microservices. Success with microservices means owning the software lifecycle. Most (74%) respondents say their teams own the build-test-deploy-maintain phases of the software lifecycle.

Database 135

Database Architecture Education Systems 135

O'Reilly

JULY 20, 2021

Most recently, IBM used Flickr photos that were shared under a Creative Commons license to train facial recognition algorithms. There may be alarm systems. There have been cases of harassment, intimidation, and domestic abuse by people whose access should have been revoked: for example, an ex-partner turning off the heating system.

Google 121

Google Games Technology Technology 121

O'Reilly

MAY 19, 2020

Live online training, videos, books, certification prep, and more, from O’Reilly and our partner publishers. Software engineers represent the largest cohort, comprising almost 20% of all respondents (see Figure 1 ). Technical leads and architects (about 11%) are next, followed by software and systems architects (9+%).

Cloud 141

Cloud Serverless AWS Architecture 141

The Agile Manager

SEPTEMBER 25, 2009

Remember, although the Marines put all recruits through a rigorous 12-week training program, every single one of those recruits is still just an entry-level soldier at its conclusion. It can take years to train and groom people for more challenging roles. Why should you expect different for software engineers?

Automotive 40

Automotive Software Engineering Programming Metrics 40

ACM Sigarch

APRIL 8, 2019

However, we often hear anecdotes that the number of prospective graduate students applying to computer architecture/systems is small and shrinking. Her current work focuses on hardware/software co-design for extremely large-scale deep learning training. Lena Olson is a Software Engineer at Google. . Why is that?

Architecture 74

Architecture Open Source Hardware Software Engineering 74