Brendan Gregg

FEBRUARY 28, 2023

This talk originated from my updates to [Systems Performance 2nd Edition], and this was the first time I've given this talk in person! CXL in a way allows a custom memory controller to be added to a system, to increase memory capacity, bandwidth, and overall performance. Ford, et al., “TCP

Performance 110

Performance Latency Cache Virtualization 110

All Things Distributed

AUGUST 16, 2012

Werner Vogels weblog on building scalable and robust distributed systems. The best practices that we are collecting in the AWS Economics Center are there to help our customers get a total view on their IT cost such that they can accurately compare on-premise and cloud. All Things Distributed. Comments (). Total Cost of Ownership.

AWS 118

AWS Hardware Traffic Best Practices 118

The Morning Paper

MARCH 8, 2020

This paper presents Snowflake design and implementation along with a discussion on how recent changes in cloud infrastructure (emerging hardware, fine-grained billing, etc.) have altered the many assumptions that guided the design and optimization of the Snowflake system. S3 on AWS). From shared-nothing to disaggregation.

Storage 112

Storage Engineering Cache Serverless 112

All Things Distributed

MARCH 11, 2015

AWS is enabling innovations in areas such as healthcare, automotive, life sciences, retail, media, energy, robotics that it is mind boggling and humbling. For example many of the Internet of Things innovations that we have seen come to life in the past years on AWS all have a significant analytics components to it.

Analytics 135

Analytics Cloud Healthcare Retail 135

cdemi

FEBRUARY 20, 2017

Let's talk about the elephant in the room; Serverless doesn't really mean that there are no Software or Hardware servers. Serverless systems are still in their infancy. Cost - Serverless Computing is more cost-efficient than having a fixed quantity of servers. Amazon: AWS Lambda. Advantages. Disadvantages. IBM: OpenWhisk.

Serverless 64

Serverless Architecture Lambda Azure 64

VoltDB

JUNE 1, 2017

Hardware Optimizers” want to get the maximum utilization out of hardware. Telcos and Fortune 500 corporations used to spend years defining custom systems, down to the last cable. Private Clouds made of commodity hardware are perceived as the logical solution to this problem. The “Public Private Cloud” folks.

Hardware 56

Hardware Latency Virtualization Cloud 56

VoltDB

JUNE 1, 2017

Hardware Optimizers” want to get the maximum utilization out of hardware. Telcos and Fortune 500 corporations used to spend years defining custom systems, down to the last cable. Private Clouds made of commodity hardware are perceived as the logical solution to this problem. The “Public Private Cloud” folks.

Hardware 40

Hardware Latency Virtualization Cloud 40

Brendan Gregg

FEBRUARY 28, 2023

This talk originated from my updates to Systems Performance 2nd Edition , and this was the first time I've given this talk in person! CXL in a way allows a custom memory controller to be added to a system, to increase memory capacity, bandwidth, and overall performance. Ford, et al., “TCP

Performance 52

Performance Latency Cache Virtualization 52

VoltDB

JUNE 29, 2017

This allows publishers to establish a “premium” auction instead of the typical real-time bidding system. However, increasing hardware capacity doesn’t really solve the problem, and it introduces new ones. And while it’s easy to say “spin up more AWS servers”, you’re only adding to your cloud provider bill.

Performance 40

Performance Hardware Latency Infrastructure 40

VoltDB

JUNE 29, 2017

This allows publishers to establish a “premium” auction instead of the typical real-time bidding system. However, increasing hardware capacity doesn’t really solve the problem, and it introduces new ones. And while it’s easy to say “spin up more AWS servers”, you’re only adding to your cloud provider bill.

Performance 40

Performance Hardware Latency Infrastructure 40

All Things Distributed

DECEMBER 20, 2017

The benefit for customers: Authorized users can view this data and therefore manage their inventories across different sites, making the maintenance processes much more efficient. Breuninger, a fashion department store chain steeped in tradition, has recognized this and relies on a self-developed e-commerce platform in the AWS Cloud.

Artificial Intelligence 79

Artificial Intelligence Social Media Logistics AWS 79

Brendan Gregg

NOVEMBER 29, 2017

Hardware virtualization for cloud computing has come a long way, improving performance using technologies such as VT-x, SR-IOV, VT-d, NVMe, and APICv. At Netflix, we've been using these technologies as they've been made available for instance types in the AWS EC2 cloud.

Virtualization 111

Virtualization AWS Hardware Storage 111

Scalegrid

JANUARY 8, 2024

Real-world examples like Spotify’s multi-cloud strategy for cost reduction and performance, and Netflix’s hybrid cloud setup for efficient content streaming and creation, illustrate the practical applications of each model. In this scenario, two notable models – multi-cloud and hybrid cloud have emerged. What is Hybrid Cloud?

Strategy 147

Strategy Cloud Scalability Artificial Intelligence 147

Percona

JULY 1, 2023

Now that Database-as-a-service (DBaaS) is in high demand, there are multiple questions regarding AWS services that cannot always be answered easily: When should I use Aurora and when should I use RDS MySQL ? Amazon Aurora is a proprietary, cloud-native, fully managed relational database service developed by Amazon Web Services (AWS).

AWS 52

AWS Database Serverless Storage 52

Dynatrace

JULY 8, 2019

When we wanted to add a location, we had to ship hardware and get someone to install that hardware in a rack with power and network. Hardware was outdated. Fixed hardware is a single point of failure – even when we had redundant machines. Keep hardware and browsers updated at all times. Sound easy?

Cloud 202

Cloud Hardware Testing Logistics 202

Dynatrace

NOVEMBER 3, 2022

The containerization craze has continued for enterprises, with benefits such as portability, efficiency, and scalability. These containers are software packages that include all the relevant dependencies needed to run software on any system. In FaaS environments, providers manage all the hardware. million in 2020. CaaS vs. FaaS.

Serverless 221

Serverless Azure Hardware Transportation 221

All Things Distributed

DECEMBER 12, 2018

In April 2017, Amazon Web Services announced that it would launch a new AWS infrastructure region Region in Sweden. Today, I'm happy to announce that the AWS Europe (Stockholm) Region, our 20th Region globally, is now generally available for use by customers. Public sector.

AWS 124

AWS Cloud Games Serverless 124

High Scalability

OCTOBER 24, 2022

Each cloud-native evolution is about using the hardware more efficiently. Anyone can rent a rack in a colo and stand up a system. Of course not, but let's ignore that very few organizations in the world have the technological know-how to create such managed services, especially without low level control of the entire system.

Cloud 187

Cloud Hardware Infrastructure Innovation 187

Dynatrace

NOVEMBER 15, 2021

Driving this growth is the increasing adoption of hyperscale cloud providers (AWS, Azure, and GCP) and containerized microservices running on Kubernetes. A log is a detailed, timestamped record of an event generated by an operating system, computing environment, application, server, or network device. billion in 2020 to $4.1

Analytics 213

Analytics Monitoring Artificial Intelligence DevOps 213

The Netflix TechBlog

MARCH 8, 2021

They need specialized hardware, access to petabytes of images, and digital content creation applications with controlled licenses. Below is a broad technical overview of how to go from an AWS instance to a Netflix Workstation. We needed a system that could manage hundreds to one-day thousands of workstations.

Entertainment 306

Entertainment Storage Hardware Open Source 306

Brendan Gregg

NOVEMBER 28, 2017

Hardware virtualization for cloud computing has come a long way, improving performance using technologies such as VT-x, SR-IOV, VT-d, NVMe, and APICv. At Netflix, we've been using these technologies as they've been made available for instance types in the AWS EC2 cloud.

Virtualization 40

Virtualization AWS Hardware Storage 40

Scalegrid

MAY 13, 2020

Greenplum Database is an open-source , hardware-agnostic MPP database for analytics, based on PostgreSQL and developed by Pivotal who was later acquired by VMware. The MPP system leverages a shared-nothing architecture to handle multiple operations in parallel. Typically an MPP system has one leader node and one or many compute nodes.

Big Data 321

Big Data Database Artificial Intelligence Open Source 321

The Netflix TechBlog

APRIL 29, 2019

takes place in Amazon Web Services (AWS), whereas everything that happens afterwards (i.e., Various software systems are needed to design, build, and operate this CDN infrastructure, and a significant number of them are written in Python. We have created streams of events from a number of systems that get unified into a single tool.

Open Source 279

Open Source Network Infrastructure Big Data 279

All Things Distributed

MARCH 11, 2016

The epoch of AWS is the launch of Amazon S3 on March 14, 2006, now almost 10 years ago. Given that AWS is a pioneer in building and operating these services world-wide, these lessons have been of crucial importance to our business. Build evolvable systems. Systems need to keep running even if the “house is on fire.”

AWS 144

AWS Hardware Retail Virtualization 144

All Things Distributed

JULY 14, 2015

In this blog post, I will explain how these three new capabilities empower you to build applications with distributed systems architecture and create responsive, reliable, and high-performance applications using DynamoDB that work at any scale. DynamoDB Streams simplifies and improves this design pattern with a distributed systems approach.

Database 167

Database Lambda AWS IoT 167

The Morning Paper

MARCH 1, 2020

Firecracker is the virtual machine monitor (VMM) that powers AWS Lambda and AWS Fargate, and has been used in production at AWS since 2018. The first version of AWS Lambda was built using Linux containers. This tradeoff is unacceptable to public infrastructure providers, who need both strong security and minimal overhead.

Serverless 101

Serverless Virtualization Lambda AWS 101

Dynatrace

OCTOBER 1, 2021

As dynamic systems architectures increase in complexity and scale, IT teams face mounting pressure to track and respond to conditions and issues across their multi-cloud environments. An advanced observability solution can also be used to automate more processes, increasing efficiency and innovation among Ops and Apps teams.

Metrics 363

Metrics Open Source Monitoring Infrastructure 363

All Things Distributed

MARCH 19, 2018

At AWS, we continue to strive to enable builders to build cutting-edge technologies faster in a secure, reliable, and scalable fashion. Now, thousands of customers are trying Amazon SageMaker and building ML models on top of their data lakes in AWS. In machine learning, more is usually more. Data always keeps being accrued.

Scalability 159

Scalability Hardware AWS Tuning 159

KeyCDN

FEBRUARY 22, 2023

Chatbots and virtual assistants Chatbots and virtual assistants are becoming more common on websites and web applications as they provide an efficient and convenient way for users to interact with a business. There are several popular cloud-based platforms for web development and deployment, such as AWS , Azure , and Google Cloud Platform.

Artificial Intelligence 111

Artificial Intelligence Development Serverless Website 111

O'Reilly

NOVEMBER 28, 2023

That pricing won’t be sustainable, particularly as hardware shortages drive up the cost of building infrastructure. How will AI adopters react when the cost of renting infrastructure from AWS, Microsoft, or Google rises? Users also need to know how to create a prompt for an AI system that will generate a useful answer.

Open Source 106

Open Source Artificial Intelligence Google Infrastructure 106

Testsigma

MAY 19, 2021

Thinking back on how SDLC started and what it is today, the only reasons for its success can be accounted to efficiency, speed and most importantly automation – DevOps and cloud-based solutions can be considered major contributors here (after all DevOps is 41% less time-consuming than traditional ops ). . Centralizes the Processes.

DevOps 59

DevOps Cloud Artificial Intelligence Infrastructure 59

Adrian Cockcroft

NOVEMBER 13, 2023

In the simplest case, you have a growing workload, and you optimize it to run more efficiently so that you don’t need to buy or rent additional hardware, so your carbon footprint stays the same, but the carbon per transaction or operation is going down. they are never really idle. I’ve written before about how to tune out retry storms.

Energy 52

Energy Cloud Traffic Transportation 52

The Morning Paper

MAY 28, 2019

We don’t talk about energy as often as we probably should on this blog, but it’s certainly true that our data centres and various IT systems consume an awful lot of it. The goal is to produce a low-energy hardware classifier for embedded applications doing local processing of sensor data. Introducing race logic.

Energy 52

Energy Hardware Efficiency Architecture 52

Percona

JULY 26, 2023

To make this process work more efficiently and ensure a smooth failover, it is important to have the same hardware configuration on all the nodes of the replica set. This is more efficient than $regex but needs you to prepare in advance by adding text indexes to your data sets.

Database 80

Database Servers Storage Strategy 80

All Things Distributed

JULY 20, 2015

To be robust and scalable, this key/value store needs to be distributed for durability and availability, to protect against network partitions or hardware failures. Hailo was founded in 2011 and has been built on AWS since Day 1. containers stopping and starting). At first, the team looked to a few open-source solutions (e.g.,

Latency 166

Latency Architecture AWS Open Source 166

All Things Distributed

NOVEMBER 15, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Now that our ability to generate higher and higher clock rates has stalled and CPU architectural improvements have shifted focus towards multiple cores, we see that it is becoming harder to efficiently use these computer systems. All Things Distributed.

AWS 136

AWS Latency Programming Architecture 136

Percona

APRIL 17, 2023

In this blog post, we will discuss the best practices on the MongoDB ecosystem applied at the Operating System (OS) and MongoDB levels. Operating System (OS) settings Swappiness Swappiness is a Linux kernel setting that influences the behavior of the Virtual Memory manager when it needs to allocate a swap, ranging from 0-100.

Best Practices 52

Best Practices Design Tuning Database 52