article thumbnail

Designing Instagram

High Scalability

FUN FACT : In this talk , Rodrigo Schmidt, director of engineering at Instagram talks about the different challenges they have faced in scaling the data infrastructure at Instagram. We will use a graph database such as Neo4j to store the information. Sample Queries supported by Graph Database. System Components. API Design.

Design 334
article thumbnail

Cloudy with a high chance of DBMS: a 10-year prediction for enterprise-grade ML

The Morning Paper

Many of the software engineering discipline and controls need to be brought over into an ML context. The following chart breaks down features in three main areas: training and auditing, serving and deployment, and data management, across six systems. But model inference migrating into the DBMS is a bolder prediction.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

USENIX LISA 2018: CFP Now Open

Brendan Gregg

Today's LISA attracts attendees working on all sizes of production systems, and its attendees include sysadmins, systems engineers, SREs, DevOps engineers, software engineers, IT managers, security engineers, network administrators, researchers, students, and more. Hope to see you in Nashville!

DevOps 43
article thumbnail

USENIX LISA 2018: CFP Now Open

Brendan Gregg

Today's LISA attracts attendees working on all sizes of production systems, and its attendees include sysadmins, systems engineers, SREs, DevOps engineers, software engineers, IT managers, security engineers, network administrators, researchers, students, and more. Hope to see you in Nashville!

DevOps 40
article thumbnail

MLOps and DevOps: Why Data Makes It Different

O'Reilly

This is both frustrating for companies that would prefer making ML an ordinary, fuss-free value-generating function like software engineering, as well as exciting for vendors who see the opportunity to create buzz around a new category of enterprise software. The new category is often called MLOps. This approach is not novel.

DevOps 138
article thumbnail

Orchestrating Data/ML Workflows at Scale With Netflix Maestro

The Netflix TechBlog

These include ETL pipelines, ML model training workflows, batch jobs, etc. Similarly, ML model training workflows usually consist of tens of thousands of training jobs within a single workflow. A large number of batch workflows run daily to serve various business needs. It is hard to support super large workflows in general.

Java 202
article thumbnail

Microservices Adoption in 2020

O'Reilly

Software engineers comprise the survey audience’s single largest cluster, over one quarter (27%) of respondents (Figure 1). software and systems architects, technical leads—architects represent almost 28% of the sample. Use of a Central, Managed Database. Are our respondents succeeding with decoupled databases?

Database 135