Dynatrace

NOVEMBER 22, 2021

With EC2, Amazon manages the basic compute, storage, networking infrastructure and virtualization layer, and leaves the rest for you to manage: OS, middleware, runtime environment, data, and applications. Amazon EC2. EC2 is Amazon’s Infrastructure-as-a-service (IaaS) compute platform designed to handle any workload at scale.

Best Practices 207

Best Practices AWS Monitoring Serverless 207

The Netflix TechBlog

NOVEMBER 22, 2019

4:45pm-5:45pm NFX 209 File system as a service at Netflix Kishore Kasi , Senior Software Engineer Abstract : As Netflix grows in original content creation, its need for storage is also increasing at a rapid pace. In this session, we present our human-centric design principles that enable the autonomy our engineers enjoy.

AWS 100

AWS Entertainment Open Source Benchmarking 100

The Netflix TechBlog

NOVEMBER 22, 2019

4:45pm-5:45pm NFX 209 File system as a service at Netflix Kishore Kasi , Senior Software Engineer Abstract : As Netflix grows in original content creation, its need for storage is also increasing at a rapid pace. In this session, we present our human-centric design principles that enable the autonomy our engineers enjoy.

AWS 100

AWS Entertainment Open Source Benchmarking 100

Dynatrace

JANUARY 23, 2024

The discipline shows promise: According to Gartner, 80% of software engineering organizations “will establish platform teams as internal providers of reusable services, components, and tools for application delivery” by 2026.

Performance 182

Performance DevOps Innovation Artificial Intelligence 182

The Morning Paper

FEBRUARY 18, 2020

Many of the software engineering discipline and controls need to be brought over into an ML context. Flock treats ML models as software artefacts derived from data. Models themselves must be subject to scrutiny with their storage and querying/scoring secured and auditable.

Government 95

Government Software Engineering Storage Database 95

SQL Server According to Bob

SEPTEMBER 9, 2019

The lock manager uses a hash table that is accessed based on the LockResource, where the LockResource is byte array for opaque storage. Bob Dorr – Principal Software Engineer SQL Server. The actual rows need not remain stored in buffer pool, but the rows are transactionaly protected.

Servers 103

Servers Database Scalability Tuning 103

Dotcom-Montior

OCTOBER 6, 2021

A site reliability engineer, or SRE, is a role that that encompasses aspects of both software engineering and operations/infrastructure. The term site reliability engineering first came into existence at Google in 2003 when a site reliability team was created. At that time, the team was made up of software engineers.

Engineering 67

Engineering DevOps Monitoring Google 67

O'Reilly

NOVEMBER 12, 2019

More than a fifth of the respondents work in the software industry—skewing results toward the concerns of software companies, and helping explain the preponderance of those with software engineering roles. As noted earlier, the majority of survey respondents are software engineers.

Serverless 88

Serverless Architecture FinTech Infrastructure 88

SQL Server According to Bob

DECEMBER 18, 2018

Device level flushing may have an impact on your I/O caching, read ahead or other behaviors of the storage system. Neal, Matt, and others from Windows Storage, Windows Azure Storage, Windows Hyper-V, … validating Windows behaviors. · Any storage device that can survive a power outage. Starting with the Linux 4.18

Servers 90

Servers Media Cache Storage 90

SQL Server According to Bob

NOVEMBER 8, 2016

There are two ways to take advantage of non-volatile storage (NVDIMM) for SQL Server (Direct access or Block Device storage.). Block storage is what you think of today as disk access. SQL works with the operating system to detect the non-volatile storage devices and leverage them for the tail of the log.

Cache 40

Cache Servers Media Storage 40

All Things Distributed

DECEMBER 18, 2012

Aug 24 - The Five-Minute Rule Ten Years Later, and Other Computer Storage Rules of Thumb , Jim Gray and Goetz Graefe, ACM SIGMOD Record 26 (4): 63â??68, Sep 7 - Adaptive load sharing in homogeneous distributed systems , D Eager, ED Lazowska and J Zahorjan - IEEE transactions on software engineering, 1986. Lorie, G.F. Nijssen ed.,

Operating System 144

Operating System Cache AWS Scalability 144

The Netflix TechBlog

NOVEMBER 22, 2019

4:45pm-5:45pm NFX 209 File system as a service at Netflix Kishore Kasi , Senior Software Engineer Abstract : As Netflix grows in original content creation, its need for storage is also increasing at a rapid pace. In this session, we present our human-centric design principles that enable the autonomy our engineers enjoy.

AWS 37

AWS Entertainment Open Source Benchmarking 37

The Morning Paper

MARCH 3, 2020

It takes you through the thinking processes and engineering practices behind the design of a key part of the control plane for AWS Elastic Block Storage (EBS): the Physalia database that stores configuration information. Engineering decisions involve making lots of trade-offs. This paper is a real joy to read.

Database 106

Database AWS Network Design 106

SQL Server According to Bob

JUNE 3, 2016

Ryan Stonecipher – Principle SQL Server Software Engineer. Bob Dorr – Principal SQL Server Escalation Engineer. 32GB RAM, 4 Core Hyper-threaded enabled 2.8Ghz, SSD Storage. DEMO – It Just Runs: DBCC CheckDB. The DBCC CheckDB demonstration loads a table and demonstrates the performance improvement.

Hardware 40

Hardware Cache Servers Software Engineering 40

Brendan Gregg

APRIL 30, 2018

LISA originally stood for "Large Installation System Administration," where "large" meant systems with more than a gigabyte of storage, or with more than 100 users.

DevOps 43

DevOps Network Best Practices Programming 43

The Morning Paper

NOVEMBER 19, 2019

Local-first apps keep their data in local storage on each device, but the data is also synchronised across all the devices on which a user works. Operations can be handled by reading and writing to the local file system, with data synchronisation happening in the background. It should work across multiple devices.

Cloud 83

Cloud Software Software Internet 83

SQL Server According to Bob

MAY 19, 2019

For example, Azure SQL Database Edge can already use TSQL Predict Machine Learning (ML) and Column Storage. Bob Dorr – Principal Software Engineer SQL Server. . Some features are still being worked on for IoT Edge but our goal is to provide the same SQL Server feature set.

IoT 61

IoT Servers Development Cache 61

The Morning Paper

OCTOBER 22, 2019

Software engineering for machine learning ’), is the lack of documentation, metadata, and provenance. are tied for most commonly used data storage and manipulation solutions. Once into the EDA process, another major challenge, that we’ve also looked at in the context of machine learning pipelines (e.g.

Programming 40

Programming Software Engineering Database Analytics 40

Brendan Gregg

APRIL 29, 2018

LISA originally stood for "Large Installation System Administration," where "large" meant systems with more than a gigabyte of storage, or with more than 100 users.

DevOps 40

DevOps Network Best Practices Programming 40

Smashing Magazine

MAY 17, 2021

I founded Instant Domain Search in 2005 and kept it as a side-hustle while I worked on a Y Combinator company (Snipshot, W06), before working as a software engineer at Facebook. Since we don’t need to run real-time queries on the data, we stream it into Google BigQuery’s streaming API for storage. Large preview ).

Google 93

Google Analytics Serverless Metrics 93

The Morning Paper

JANUARY 26, 2020

Reverb also take a snapshot of the client’s local storage (e.g. Within the browser Reverb rewrites JavaScript code to inject instrumentation that runs during each read or write to JavaScript or DOM state, as well al logging nondeterministic events such as mouse clicks and timer firings. cookies) at the start of logging.

Programming 86

Programming Servers Network Latency 86

SQL Performance

APRIL 17, 2020

When compressed, fixed-length string columns do not store trailing blanks, making them just as efficient as variable-length string columns from a storage point of view. Some of this may be due to somewhat out-of-date considerations like space usage. This is the case when using ALTER COLUMN to change from integer to smallint for example.

Servers 63

Servers Database Testing Efficiency 63

SQL Server According to Bob

AUGUST 8, 2016

The local SSD used for Buffer Pool Extension is WARM because it is commonly faster than going to the stable storage (COLD, Disk.). Bob Dorr – Principal Software Engineer SQL Server. The concept is fast access to the data on the page. The closest to the executing code is RAM.

Cache 40

Cache Energy Database Servers 40

SQL Server According to Bob

MARCH 27, 2017

Invoke generate-core.sh – capture system information – paldumper capture – compress all captured files Storage location: /var/opt/mssql/log (core*). Microsoft Engineer Processing Flow. Also refer to ulimit and /proc/sys/kernel/core_pattern for limitations and storage locations. Exception Processing Flow.

Servers 40

Servers Virtualization Storage Design 40

The Morning Paper

DECEMBER 3, 2019

Each instance is a full stack service in its own right, with front-end GUI components, a back-end server and data storage, and all the code necessary to coordinate them. Concepts are embodied as microservices: A Déjà Vu application is a set of concept instances that run in parallel.

Open Source 52

Open Source Programming Speed Software 52

The Netflix TechBlog

APRIL 26, 2022

The first version of our logger library optimized for storage by deduplicating facts and optimized for network i/o using different compression methods for each fact. Since we were optimizing at the logging level for storage and performance, we had less data and metadata to play with to optimize the query performance.

Storage 187

Storage Design Scalability Latency 187

The Morning Paper

APRIL 16, 2019

During the ten week course, students implement four different assignments: an exactly-once RPC protocol; a primary-backup system; Paxos; and a scalable, transactional key-value storage system. 175 undergraduates a year currently go through this course.

Systems 43

Systems Efficiency Testing Strategy 43

The Netflix TechBlog

DECEMBER 3, 2019

mainly because of mundane reasons related to software engineering. both for compute and storage. We were expecting to hear answers related to large-scale data and models, and maybe issues related to modern GPUs. Instead, we heard stories about projects where getting the first version to production took surprisingly long?—?mainly

Open Source 232

Open Source AWS Infrastructure Energy 232

The Netflix TechBlog

FEBRUARY 9, 2021

This requires an asset storage solution. Asset Storage We refer to asset storage and management simply as asset management. Generating a single localized asset is a big achievement, but we still need to store the asset somewhere and have the ability to search for it. First, asset generation is CPU intensive and bursty.

Engineering 165

Engineering Storage Latency Entertainment 165

The Netflix TechBlog

MAY 14, 2021

The watermarking functionality, at the start, was a simple offering with various Google Drive integrations for storage and links. In late 2019, our team started integrating it with the rest of the ecosystem by writing a thin Java Domain graph service (DGS) to wrap the asynchronous watermarking functionality that was then in Ruby on Rails.

Traffic 160

Traffic Latency Java Google 160

The Netflix TechBlog

DECEMBER 3, 2019

mainly because of mundane reasons related to software engineering. both for compute and storage. We were expecting to hear answers related to large-scale data and models, and maybe issues related to modern GPUs. Instead, we heard stories about projects where getting the first version to production took surprisingly long?—?mainly

Open Source 134

Open Source AWS Infrastructure Energy 134

SQL Server According to Bob

DECEMBER 31, 2018

Using SQL Server Management Studio (SSMS) and storing the events in a table requires reading the events, streaming to table storage followed by an order by event sequence query. Bob Dorr – Principal Software Engineer SQL Server. Large traces can encounter the 32-bit SSMS memory limitations.

Efficiency 40

Efficiency Code Servers Cache 40

SQL Server According to Bob

JANUARY 11, 2020

​​ The offline formatting ​​ approach saves storage space, ​​ reduces CPU ​​ and the impact on performance. Bob Dorr​​ –​​ Principal Software Engineer SQL Server.

Network 40

Network Servers Processing Software Engineering 40

Abhishek Tiwari

NOVEMBER 3, 2018

In the CMS selection process, developer experience is not a factor, although successful implementation and ongoing maintenance require developer friendly tooling and support for modern software engineering practices. Most of cloud object/blob storage services have native support for static site hosting.

Systems 63

Systems Cache Website Network 63

The Netflix TechBlog

NOVEMBER 3, 2022

When a new leader is elected it loads all data from external storage. Overview The figure below depicts a simplified high-level architecture of a single Titus cluster (a.k.a cell): Titus Job Coordinator is a leader elected process managing the active state of the system. Active data includes jobs and tasks that are currently running.

Cache 224

Cache Latency Traffic Systems 224

O'Reilly

MAY 19, 2020

. • More than one-third have adopted site reliability engineering (SRE); slightly less have developed production AI services. Software engineers represent the largest cohort, comprising almost 20% of all respondents (see Figure 1 ). For this audience, SRE’s future is brighter than AI’s, however. Respondent Demographics.

Cloud 141

Cloud Serverless AWS Architecture 141