Software engineering for machine learning: a case study

The Morning Paper

Software engineering for machine learning: a case study Amershi et al., Respondents were well spread across data and applied science (42%), software engineering (32%), program management (17%), research (7%) and other (1%).

Team building in software engineering

Tasktop

This can be a challenge for engineering teams: engineers can be less social and more competitive than non-engineers. We’re typically heads-down, and it seems any interaction we have is just business: problem solving, design discussions, code review comments, story planning. The post Team building in software engineering appeared first on Tasktop Blog. EngineeringWhen you come in to work, do your teammates greet you? Are jokes told on your team?

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

From Student to Junior Software Engineer at Tasktop

Tasktop

The beginning of my experience as a Junior Software Engineer on one of Tasktop’s ‘Integrations Teams’ marked a definitive transition in the way I learned and practiced computer science and software development. I was drawn to Tasktop because of its strong impact on the software development industry. Software Engineers, was introduced to the fundamental concepts and tools on which Tasktop is built. Looking for a career in software development?

The 737Max and Why Software Engineers Might Want to Pay Attention

J. Paul Reed

The 737Max and Why Software Engineers Might Want to Pay Attention As someone with a bit of a reputation for talking about aviation and software development and operations , I’ve been asked about the 737Max repeatedly over the past week.

Migrating a privacy-safe information extraction system to a Software 2.0 design

The Morning Paper

Migrating a privacy-safe information extraction system to a software 2.0 design , Sheng, CIDR’20. This is a comparatively short (7 pages) but very interesting paper detailing the migration of a software system to a ‘Software 2.0’ ’ design.

The ‘Chocolate Sauce’ Design Heuristic

Strategic Tech

A trip to the supermarket can teach you a lot about designing software systems and shaping teams to build them… I was recently in need of some chocolate sauce. The designers of the supermarket had to decide where the chocolate sauce should live.

How Netflix microservices tackle dataset pub-sub

The Netflix TechBlog

This post is a high level overview of the design and architecture of Gutenberg. An important point to note is that Gutenberg is not designed as an eventing system?—?it Gutenberg, however, is designed for publishing and consuming an entire immutable view of a dataset.

Scaling symbolic evaluation for automated verification of systems code with Serval

The Morning Paper

Serval is a framework for developing automated verifiers of systems software. To invest in formally modelling something, you really need to have a very good idea of what the software needs to do. Uncategorized Formal methods Software Engineering

Code 72

How do committees invent?

The Morning Paper

We’re back in 1968, a time when it was taken for granted that before building a system, it was necessary to design it. " Designs are produced by people, and the set of people working on a design are part of a design organisation. The definition of design itself is quite interesting: That kind of intellectual activity which creates a whole from its diverse parts may be called the design of a system. Uncategorized Software Engineering

Declarative assembly of web applications from pre-defined concepts

The Morning Paper

component-based software development, reusable software catalogues etc. Then there’s the changing demographic of software building. If the thing you’re trying to build doesn’t fit with our framework, you probably have a design flaw! Uncategorized Software Engineering

Teaching rigorous distributed systems with efficient model checking

The Morning Paper

We would like students to consider the performance characteristics of their systems, and our tests check that their designs attain reasonable run-time performance. and used to taking small incremental steps, fail to appreciate the need for some degree of up-front design. (Of

Three key checklists and remedies for trustworthy analysis of online controlled experiments at scale

The Morning Paper

Last time out we looked at machine learning at Microsoft , where we learned among other things that using an online controlled experiment (OCE) approach to rolling out changes to ML-centric software is important. Experiment design analysis checklist. Uncategorized Software Engineering

Visualising Sociotechnical Architecture with DDD and Team Topologies

Strategic Tech

I’ve been disappointed for a long time with the way in which companies organise software development teams. I remember as a young, naive software developer, I assumed there would be structured processes and patterns similar to those used for designing a software architecture.

New Metadata-Only Column Changes in SQL Server 2016

SQL Performance

If we write an ALTER COLUMN statement without NULL or NOT NULL in that case, the engine helpfully assumes we meant NOT NULL because the identity property is not allowed on nullable columns. The ALTER TABLE. ALTER COLUMN command is very powerful.

Uncovering Hidden Business Rules with DDD Aggregates

Strategic Tech

In software, reacting to unforeseen circumstances in real-time is not possible. The gap between defining business requirements and translating them into software needs to be minimised in order to prevent this category of problems. They don’t understand software, they.

Core Domain Patterns

Strategic Tech

How we spend our time and apply our resources when developing software systems is possibly the most fundamental and difficult challenge. A natural tendency for software engineers is to gravitate towards the most technically interesting challenges. Time and resources are limited.

Book Review: Pro JavaScript Design Patterns

Tim Kadlec

The books reviewed will all be somehow related to web development or design so you will never hear me tell you how much I enjoyed Stephen King’s Dark Tower series or Napoleon’s Pyramids by William Dietrich… except for right now of course. Pro JavaScript Design Patterns is written by Ross Harmes and Dustin Diaz. Ross is a front-end engineer from Yahoo! Dustin works for Google as a user interface engineer.

Sponsored Post: InterviewCamp.io, Scrapinghub, Fauna, Sisu, Educative, PA File Sight, Etleap, Triplebyte, Stream

High Scalability

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems : large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc. Who's Hiring? InterviewCamp.io

Sponsored Post: InterviewCamp.io, Scrapinghub, Fauna, Sisu, Educative, PA File Sight, Etleap, Triplebyte, Stream

High Scalability

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems : large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc. Who's Hiring? InterviewCamp.io

Post: InterviewCamp.io, Scrapinghub, Fauna, Sisu, Educative, PA File Sight, Etleap, Triplebyte, Stream

High Scalability

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems : large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc. Who's Hiring? InterviewCamp.io

Post: InterviewCamp.io, Scrapinghub, Fauna, Sisu, Educative, PA File Sight, Etleap, Triplebyte, Stream

High Scalability

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems : large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc. Who's Hiring? InterviewCamp.io

Stuff The Internet Says On Scalability For October 5th, 2018

High Scalability

Because the device in question is a high-speed unit designed to process a high volume of ballots for an entire county, hacking just one of these machines could enable anattacker to flip the Electoral College and determine the outcome of a presidential election?. Hey, wake up!

Post: InterviewCamp.io, Scrapinghub, Fauna, Sisu, Educative, PA File Sight, Etleap, Triplebyte, Stream

High Scalability

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems : large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc. Who's Hiring? InterviewCamp.io

Post: Scrapinghub, Fauna, Sisu, Educative, PA File Sight, Etleap, Triplebyte, Stream

High Scalability

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems : large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc. this is going to be a challenging journey for any backend engineer! Who's Hiring?

Tackling the Pipeline Problem in the Architecture Research Community

ACM Sigarch

Her current work focuses on hardware/software co-design for extremely large-scale deep learning training. Lena Olson is a Software Engineer at Google. . Disclaimer: Newsha is a Research Scientist at Baidu and Lena is a Software Engineer at Google.

World’s Top Web Performance Leaders To Watch

Rigor

Reading time 16 min Whether you’re a web performance expert, an evangelist for the culture of performance, a web engineer incorporating performance into your process, or someone new to the web performance entirely, you probably identify as curious, excited about new ideas, and always learning.

Millions of tiny databases

The Morning Paper

It takes you through the thinking processes and engineering practices behind the design of a key part of the control plane for AWS Elastic Block Storage (EBS): the Physalia database that stores configuration information. Engineering decisions involve making lots of trade-offs.

Snap: a microkernel approach to host networking

The Morning Paper

It’s been clear for a while that software designed explicitly for the data center environment will increasingly want/need to make different design trade-offs to e.g. general-purpose systems software that you might install on your own machines. The little engine that could.

Building and Scaling Data Lineage at Netflix to Improve Data Infrastructure Reliability, and…

The Netflix TechBlog

Now, imagine yourself in the role of a software engineer responsible for a micro-service which publishes data consumed by few critical customer facing services (e.g. Design a flexible data model ? —?Represent

Open-Sourcing Metaflow, a Human-Centric Framework for Data Science

The Netflix TechBlog

mainly because of mundane reasons related to software engineering. They know that feature engineering is critical for many models, so they want to stay in control of model inputs and feature engineering logic. It leverages elasticity of the cloud by design?—?both

How It Works: SQL Server Lock Partitioning

SQL Server According to Bob

The lock manager (LockMgr) is designed to track partitioned and non-partitioned locks. The lock manager design allows lock types benefiting from partitioning to co-exist with non-partitioned lock types and increased scalability. Bob Dorr – Principal Software Engineer SQL Server

Post: Essilen Research, Fauna, Sisu, Educative, PA File Sight, Etleap, Triplebyte, Stream

High Scalability

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. T riplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Who's Hiring? Are you looking to improve how you hire technical talent ?

Post: Essilen Research, Fauna, Sisu, Educative, PA File Sight, Etleap, Triplebyte, Stream

High Scalability

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. T riplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Who's Hiring? Are you looking to improve how you hire technical talent ?

Post: Fauna, Sisu, Educative, PA File Sight, Etleap, Triplebyte, Stream

High Scalability

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. T riplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Who's Hiring?

Post: Essilen Research, Fauna, Sisu, Educative, PA File Sight, Etleap, Triplebyte, Stream

High Scalability

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. T riplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Who's Hiring? Are you looking to improve how you hire technical talent ?

Sponsored Post: InMemory.Net, Triplebyte, Etleap, Stream, Scalyr

High Scalability

Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. For heads of IT/Engineering responsible for building an analytics infrastructure , Etleap is an ETL solution for creating perfect data pipelines from day one. Unlike older enterprise solutions, Etleap doesn’t require extensive engineering work to set up, maintain, and scale. Who's Hiring?

Sponsored Post: Etleap, PerfOps, InMemory.Net, Triplebyte, Stream, Scalyr

High Scalability

Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. For heads of IT/Engineering responsible for building an analytics infrastructure , Etleap is an ETL solution for creating perfect data pipelines from day one. Unlike older enterprise solutions, Etleap doesn’t require extensive engineering work to set up, maintain, and scale. Who's Hiring?

Netflix at AWS re:Invent 2019

The Netflix TechBlog

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! December 2 1pm-2pm CMP 326-R Capacity Management Made Easy with Amazon EC2 Auto Scaling Vadim Filanovsky , Senior Performance Engineer & Anoop Kapoor, AWS Abstract :Amazon EC2 Auto Scaling offers a hands-free capacity management experience to help customers maintain a healthy fleet, improve application availability, and reduce costs.

AWS 100

Netflix at AWS re:Invent 2019

The Netflix TechBlog

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! December 2 1pm-2pm CMP 326-R Capacity Management Made Easy with Amazon EC2 Auto Scaling Vadim Filanovsky , Senior Performance Engineer & Anoop Kapoor, AWS Abstract :Amazon EC2 Auto Scaling offers a hands-free capacity management experience to help customers maintain a healthy fleet, improve application availability, and reduce costs.

AWS 100

Sponsored Post: InMemory.Net, Triplebyte, Etleap, Stream, Scalyr

High Scalability

Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. For heads of IT/Engineering responsible for building an analytics infrastructure , Etleap is an ETL solution for creating perfect data pipelines from day one. Unlike older enterprise solutions, Etleap doesn’t require extensive engineering work to set up, maintain, and scale. Who's Hiring?

Open-Sourcing Metaflow, a Human-Centric Framework for Data Science

The Netflix TechBlog

mainly because of mundane reasons related to software engineering. They know that feature engineering is critical for many models, so they want to stay in control of model inputs and feature engineering logic. It leverages elasticity of the cloud by design?—?both

Sponsored Post: InMemory.Net, Triplebyte, Etleap, Stream, Scalyr

High Scalability

Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. For heads of IT/Engineering responsible for building an analytics infrastructure , Etleap is an ETL solution for creating perfect data pipelines from day one. Unlike older enterprise solutions, Etleap doesn’t require extensive engineering work to set up, maintain, and scale. Who's Hiring?

Trade-offs under pressure: heuristics and observations of teams resolving internet service outages (Part 1)

The Morning Paper

In summary, software engineers find themselves in a very unenviable position when attempting to resolve an outage with their service. Preventative design is not enough.