The Morning Paper

MARCH 8, 2020

Building an elastic query engine on disaggregated storage , Vuppalapati, NSDI’20. This paper presents Snowflake design and implementation along with a discussion on how recent changes in cloud infrastructure (emerging hardware, fine-grained billing, etc.) But the ephemeral storage service for intermediate data is not based on S3.

Storage 112

Storage Engineering Cache Serverless 112

Dynatrace

DECEMBER 16, 2020

Access to source code repositories is limited on both the network and the user level. Source code management systems are only accessible from within the Dynatrace corporate network. Remote access to the Dynatrace corporate network requires multi-factor authentication (MFA). No manual, error-prone steps are involved.

Software 230

Software Software Development AWS 230

Scalegrid

JANUARY 12, 2024

While managing cloud workloads offers numerous benefits, it also presents several challenges such as security risks, compliance issues, and resource optimization, which can be addressed effectively with tools like ScaleGrid, offering features like encryption, disaster recovery, and real-time resource optimization for diverse databases.

Cloud 130

Cloud Virtualization Storage Efficiency 130

Percona

FEBRUARY 24, 2023

Let’s take a look at how to get the benefits you need while spending less, based on the recommendations presented by Dani Guzmán Burgos, our Percona Monitoring and Management (PMM) Tech Lead, on this webinar (now available on demand) hosted in November last year. These color-coded states on panels are available in PMM 2.32.0

Cloud 121

Cloud Serverless AWS Storage 121

Scalegrid

JANUARY 8, 2024

Challenges of Multi-Cloud Although multi-cloud has its benefits, it also presents some obstacles. The company utilizes public cloud resources such as Amazon Web Services (AWS) for streaming purposes and maintains on-premises infrastructure for content creation and processing.

Strategy 147

Strategy Cloud Scalability Artificial Intelligence 147

Dynatrace

MAY 6, 2021

Contact Dynatrace ONE if you wish to enable Cluster-side screenshot storage on pre-1.216 fresh-installed Clusters. To better present default values, we changed the position of session replay permissions in group details page. . Consumption and storage now returned correctly when using Environment API with paging. APM-292756).

Operating System 227

Operating System AWS Metrics Storage 227

Dynatrace

JUNE 26, 2020

Virginia (AWS) ?, California (AWS), San Jose (Azure), Texas (Azure), Ohio (AWS), Toronto (Azure) ?, London (AWS), London (Azure), Frankfurt (AWS) ?, Sydney (AWS) ? Hong Kong (Azure), Tokyo (Azure), Sao Paulo (AWS). Storage and management of credentials via the Synthetic Monitoring credential vault.

Monitoring 254

Monitoring Azure AWS Traffic 254

Brendan Gregg

DECEMBER 31, 2017

My last talk for 2017 was at AWS re:Invent, on "How Netflix Tunes EC2 Instances for Performance," an updated version of my [2014] talk. Storage I/O. Networking. John Bennett (networking) co-presented How Netflix Monitors Applications in Near Real-Time with Amazon Kinesis. File System. tcp_tw_reuse = 1 net.ipv4.ip_local_port_range

Tuning 61

Tuning AWS Best Practices Network 61

Brendan Gregg

JULY 4, 2021

AWS Graviton2); for memory with the arrival of DDR5 and High Bandwidth Memory (HBM) on-processor; for storage including new uses for 3D Xpoint as a 3D NAND accelerator; for networking with the rise of QUIC and eXpress Data Path (XDP); and so on.

Performance 140

Performance Latency Hardware Storage 140

Brendan Gregg

DECEMBER 30, 2017

My last talk for 2017 was at AWS re:Invent, on "How Netflix Tunes EC2 Instances for Performance," an updated version of my [2014] talk. Storage I/O. Networking. John Bennett (networking) co-presented How Netflix Monitors Applications in Near Real-Time with Amazon Kinesis. File System. tcp_tw_reuse = 1 net.ipv4.ip_local_port_range

Tuning 52

Tuning AWS Best Practices Network 52

All Things Distributed

MAY 19, 2017

Soon after the launch of Amazon Relation Dabase Service (RDS) AWS customers were giving us feedback that they would love to migrate to RDS but what they would love even more was if we could also unshackle them from the high-cost, punitive licensing schemes that came with the proprietary databases.

Database 117

Database Design Cloud Storage 117

All Things Distributed

JULY 16, 2012

Register for AWS re: Invent. The first annual AWS user and partner conference will be held November 27-29 at The Venetian in Las Vega. It is shaping up to be a great event with many Amazonians, partners and customers presenting in well over 150 sessions. Werner Vogels weblog on building scalable and robust distributed systems.

AWS 83

AWS Big Data Media Storage 83

The Netflix TechBlog

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! To sustain this data growth at Netflix, it has deployed open-source software Ceph using AWS services to achieve the required SLOs of some of the post-production workflows.

AWS 37

AWS Entertainment Open Source Benchmarking 37

All Things Distributed

AUGUST 16, 2011

Expanding the Cloud - The AWS GovCloud (US) Region. Today AWS announced the launch of the AWS GovCloud (US) Region. The concept of regions gives AWS customers control over the placement of their resources and services. All Things Distributed. Werner Vogels weblog on building scalable and robust distributed systems.

AWS 111

AWS Government Big Data Cloud 111

Smashing Magazine

NOVEMBER 8, 2018

Sharing Data Among Multiple Servers Through AWS S3. Sharing Data Among Multiple Servers Through AWS S3. A simple upload and save operation presents no issues. But since this article is about connecting an application running on the server with AWS S3, we don’t consider this solution. Leonardo Losoviz.

AWS 44

AWS Servers Storage Cloud 44

Brendan Gregg

JULY 4, 2021

AWS Graviton2); for memory with the arrival of DDR5 and High Bandwidth Memory (HBM) on-processor; for storage including new uses for 3D Xpoint as a 3D NAND accelerator; for networking with the rise of QUIC and eXpress Data Path (XDP); and so on.

Performance 52

Performance Latency Hardware Storage 52

IO River

NOVEMBER 2, 2023

It’s not just limited to cloud resources like AWS and Azure; Terraform is versatile, extending its capabilities to key performance areas like Content Delivery Network (CDN) management, ensuring efficient content delivery and optimal user experience.‍Started ‍Terraform, on the other hand, presents a stark contrast.

Open Source 52

Open Source DevOps AWS Azure 52

The Netflix TechBlog

DECEMBER 21, 2020

By Anupom Syam Background At Netflix, our current data warehouse contains hundreds of Petabytes of data stored in AWS S3 , and each day we ingest and create additional Petabytes. We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits.

Storage 203

Storage Latency Efficiency Data Engineering 203

The Netflix TechBlog

JULY 30, 2019

In this blog, we would like to present the latest updates to Conductor, address some of the frequently asked questions and thank the community for their contributions. Our cluster size ranges from 12–18 instances of AWS EC2 m4.4xlarge instances, typically running at ~30% capacity. *?—?Cassandra

Lambda 189

Lambda Media Open Source Metrics 189

Brendan Gregg

FEBRUARY 28, 2023

Titus, the Netflix container management platform, is now open source,” [link] Apr 2018 - [Cutress 19] Dr. [link] 2021 - [Trader 21] Tiffany Trader, “Cerebras Second-Gen 7nm Wafer Scale Engine Doubles AI Performance Over First-Gen Chip ,” [link] Apr 2021 - [Ghigoff 21] Yoann Ghigoff, et al., "BMC:

Performance 110

Performance Latency Cache Virtualization 110

Percona

APRIL 17, 2023

For example: UUID=f41e390f-835b-4223-a9bb-9b45984ddf8d / xfs rw,noatime,attr2,inode64,noquota 0 0 Unix ulimit settings Most UNIX-like operating systems, including Linux and macOS, provide ways to limit and control the usage of system resources such as threads, files, and network connections on a per-process and per-user basis.

Best Practices 52

Best Practices Design Tuning Database 52

Adrian Cockcroft

APRIL 5, 2023

I published a paper and presentation in 2006 that discusses the complexities of measuring CPU utilization. The underlying data is available to the cloud vendors themselves, so they would need to supply a one minute, per instance, energy use value as an AWS CloudWatch metric or equivalent for other cloud providers.

Energy 52

Energy Metrics Cloud Operating System 52

IO River

NOVEMBER 2, 2023

In fact, Amazon Web Services (AWS) paved the way with its service, CloudFormation, which introduced the concept of Infrastructure as Code (IaC). Taking inspiration from AWS's CloudFormation, HashiCorp introduced Terraform with a broader vision. Terraform, on the other hand, presents a stark contrast.

Open Source 40

Open Source DevOps AWS Azure 40

The Symphonia

JUNE 19, 2018

This summer also marks the 4-yearly event that is La Copa Mundial (we only get Telemundo in my apartment, not Fox Sports Network) but since the good old US of A are absent from the men’s World Cup this year, football fever is distinctly frigid. Some in-depth and unusual client work (want to encrypt a petabyte of S3 storage?

Serverless 40

Serverless AWS DevOps Open Source 40

Adrian Cockcroft

JUNE 21, 2022

The early days at Sun Cambridge were special, I absorbed a lot about networking and the technical side of the role from my fellow systems engineer Martin Baines, and we were driving all over the region in cool company cars (I had a Citroen BX 16V) selling a really hot product. He hasn’t changed.

Tuning 52

Tuning Benchmarking Engineering C++ 52

Brendan Gregg

FEBRUARY 28, 2023

Titus, the Netflix container management platform, is now open source,” [link] , Apr 2018 [Cutress 19] Dr. [link] , 2021 [Trader 21] Tiffany Trader, “Cerebras Second-Gen 7nm Wafer Scale Engine Doubles AI Performance Over First-Gen Chip ,” [link] , Apr 2021 [Ghigoff 21] Yoann Ghigoff, et al.,

Performance 52

Performance Latency Cache Virtualization 52

Enprowess

JANUARY 12, 2022

Other benefits: It has other benefits like a Quicker launch to the market, Easier distribution, saving device power and storage, seamless maintenance, and updating. The dark mode is a significant consideration of every web development at present. Serverless Computing – AWS Lambda – Amazon Web Services. Image Source.

Blockchain 59

Blockchain IoT Development Serverless 59

The Morning Paper

MAY 28, 2019

We don’t talk about energy as often as we probably should on this blog, but it’s certainly true that our data centres and various IT systems consume an awful lot of it. So decision trees are essentially networks of threshold functions. Boosted race trees for low energy classification Tzimpragos et al., ASPLOS’19.

Energy 52

Energy Hardware Efficiency Architecture 52

Adrian Cockcroft

NOVEMBER 11, 2019

This is why most AWS regions have three availability zones. When there’s three ways to succeed, we still have two ways to succeed when a failure is present, and if data is corrupted, we can tell which of the three is the odd one out. There is no need to retry and no extra time taken when a failure is present.

Latency 52

Latency Engineering Systems Hardware 52

Adrian Cockcroft

NOVEMBER 11, 2019

This is why most AWS regions have three availability zones. When there’s three ways to succeed, we still have two ways to succeed when a failure is present, and if data is corrupted, we can tell which of the three is the odd one out. There is no need to retry and no extra time taken when a failure is present.

Latency 53

Latency Engineering Systems Hardware 53

Brendan Gregg

SEPTEMBER 5, 2017

At my employer we sometimes use SR-IOV for direct network interface access, and NVMe for direct disk access. Just from a quick search, see this presentation PDF , which digs into oops message components. Most of the changes since rc7 are actually networking fixes, the bulk of them to various drivers.

Virtualization 76

Virtualization AWS Engineering Hardware 76

Smashing Magazine

JANUARY 11, 2021

Networking, HTTP/2, HTTP/3 OCSP stapling, EV/DV certificates, packaging, IPv6, QUIC, HTTP/3. If you don’t have a device at hand, emulate mobile experience on desktop by testing on a throttled 3G network (e.g. Moto G4) on a slow 3G network, emulated at 400ms RTT and 400kbps transfer speed. 300ms RTT, 1.6 Mbps down, 0.8

Performance 145

Performance Cache Media Metrics 145

Smashing Magazine

JANUARY 6, 2020

If you don’t have a device at hand, emulate mobile experience on desktop by testing on a throttled 3G network (e.g. To make the performance impact more visible, you could even introduce 2G Tuesdays or set up a throttled 3G/4G network in your office for faster testing. 300ms RTT, 1.6 Mbps down, 0.8

Performance 145

Performance Cache Network Metrics 145

Smashing Magazine

JANUARY 7, 2019

If you don’t have a device at hand, emulate mobile experience on desktop by testing on a throttled network (e.g. To make the performance impact more visible, you could even introduce 2G Tuesdays or set up a throttled 3G network in your office for faster testing. 150ms RTT, 1.5 Mbps down, 0.7 Mbps up) with a throttled CPU (5× slowdown).

Performance 103

Performance Cache Metrics Network 103

All Things Distributed

FEBRUARY 17, 2011

New AWS feature: Run your website from Amazon S3. Since a few days ago this weblog serves 100% of its content directly out of the Amazon Simple Storage Service (S3) without the need for a web server to be involved. All of this can be done from the AWS console as well as with the AWS SDKs. Countdown to What is Next in AWS.

AWS 134

AWS Website Storage Servers 134

The Netflix TechBlog

NOVEMBER 3, 2022

In the time since it was first presented as an advanced Mesos framework, Titus has transparently evolved from being built on top of Mesos to Kubernetes, handling an ever-increasing volume of containers. This blog post presents how our current iteration of Titus deals with high API call volumes by scaling out horizontally.

Cache 224

Cache Latency Traffic Systems 224

The Netflix TechBlog

FEBRUARY 9, 2021

We need to be able to easily determine what imagery is present for a given platform, region, and language. The homepage needs to load in a reasonable amount of time, even in poor network conditions. This requires an asset storage solution. Asset Storage We refer to asset storage and management simply as asset management.

Engineering 165

Engineering Storage Latency Entertainment 165