Brendan Gregg

DECEMBER 30, 2017

My last talk for 2017 was at AWS re:Invent, on "How Netflix Tunes EC2 Instances for Performance," an updated version of my [2014] talk. Our team looks after the BaseAMI, kernel tuning, OS performance tools and profilers, and self-service tools like Vector. File System. Networking. Networking. Virtual Memory.

Tuning 52

Tuning AWS Best Practices Network 52

Scalegrid

OCTOBER 24, 2019

AWS is the #1 cloud provider for open-source database hosting, and the go-to cloud for MySQL deployments. While many AWS users default to their managed database solution, Amazon RDS, there are alternatives available that can improve your MySQL performance on AWS through advanced customization options and unlimited EC2 instance type support.

AWS 183

AWS Latency Performance Benchmarking 183

Dynatrace

NOVEMBER 15, 2021

Driving this growth is the increasing adoption of hyperscale cloud providers (AWS, Azure, and GCP) and containerized microservices running on Kubernetes. A log is a detailed, timestamped record of an event generated by an operating system, computing environment, application, server, or network device. billion in 2020 to $4.1

Analytics 214

Analytics Monitoring Artificial Intelligence DevOps 214

The Netflix TechBlog

JUNE 14, 2023

The AB experiment results hinted that GraphQL’s correctness was not up to par with the legacy system. Watch our Chaos Engineering talk from AWS Reinvent to learn more about Sticky Canaries. We set up a client-side AB experiment that tested Falcor versus GraphQL and reported coarse-grained quality of experience metrics ( QoE ).

Traffic 353

Traffic Latency Cache Metrics 353

Scalegrid

JANUARY 11, 2024

Native tools, such as CloudWatch from AWS, have more extensive integrations and provide comprehensive protection for data in the cloud compared to their third-party counterparts. For example, Blumira offers a third-party solution while AWS provides its own with CloudWatch.

Best Practices 130

Best Practices Cloud Monitoring Strategy 130

All Things Distributed

NOVEMBER 30, 2016

Last week, I wrote a blog about helping the machine learning scientist community select the right deep learning framework from among many we support on AWS such as MxNet, TensorFlow, Caffe, etc. in ML and neural networks) and access to vast amounts of data.

AWS 165

AWS Lambda Artificial Intelligence Mobile 165

The Netflix TechBlog

APRIL 29, 2019

Open Connect Open Connect is Netflix’s content delivery network (CDN). takes place in Amazon Web Services (AWS), whereas everything that happens afterwards (i.e., video streaming) takes place in the Open Connect network. takes place in Amazon Web Services (AWS), whereas everything that happens afterwards (i.e.,

Open Source 279

Open Source Network Infrastructure Big Data 279

Dynatrace

OCTOBER 28, 2021

In 2009, the purveyor of online videos migrated to AWS cloud infrastructure to deliver its entertainment to a growing audience. The science has spawned specialized chaos engineers who dedicate themselves to disrupting cloud software and the on-prem systems they interact with to make them resilient. The history of chaos engineering.

Engineering 231

Engineering Entertainment Cloud Infrastructure 231

Dynatrace

MAY 17, 2023

Think about items such as general system metrics (for example, CPU utilization, free memory, number of services), the connectivity status, details of our web server, or even more granular in-application tasks like database queries. OneAgent is the native telemetry data collector and monitoring solution of Dynatrace.

Metrics 176

Metrics Monitoring Database Network 176

The Netflix TechBlog

OCTOBER 19, 2020

which is difficult when troubleshooting distributed systems. Reconstructing a streaming session was a tedious and time consuming process that involved tracing all interactions (requests) between the Netflix app, our Content Delivery Network (CDN), and backend microservices.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

Percona

DECEMBER 11, 2023

For example, you can choose the public cloud storage type – gp3, io2, etc, or set file system. This is done without downtime, but replication might introduce additional load on the primary node and the network. Regional disks Regional disks are available at Azure and Google Cloud but not yet at AWS.

Storage 100

Storage Strategy Cloud Azure 100

Dynatrace

MAY 1, 2020

The investment continues—we’re anticipating an upcoming release of the AWS Graviton2 processor , which has already been announced to be significantly more powerful than its predecessor. Network measurements with per-interface and per-process resolution. Network metrics are also collected for detected processes.

Java 133

Java Hardware Tuning Metrics 133

The Netflix TechBlog

SEPTEMBER 24, 2021

Lastly, the packager kicks in, adding a system layer to the asset, making it ready to be consumed by the clients. It is worth pointing out that cloud processing is always subject to variable network conditions. There are existing distributed file systems for the cloud as well as off-the-shelf FUSE modules for S3.

Cloud 237

Cloud Media Storage Cache 237

HammerDB

APRIL 19, 2023

Therefore, before we attempt to measure our database performance, we should know the system or cloud instance to be tested in detail. As database performance is heavily influenced by the performance of storage, network, memory, and processors, we must understand the upper limit of these key components. Operating System: Ubuntu 22.04

Performance 54

Performance Benchmarking Cache Storage 54

Scalegrid

JULY 29, 2019

MySQL is a free open source relational database management system that is leveraged across a majority of WordPress sites, and allows you to query your data such as posts, pages, images, user profiles, and more. Managing a database is hard, as it needs continuous updating, tuning, and monitoring to ensure the performance of your website.

Database 166

Database Cloud Website Azure 166

All Things Distributed

MARCH 19, 2018

At AWS, we continue to strive to enable builders to build cutting-edge technologies faster in a secure, reliable, and scalable fashion. Now, thousands of customers are trying Amazon SageMaker and building ML models on top of their data lakes in AWS. In machine learning, more is usually more. Data always keeps being accrued.

Scalability 159

Scalability Hardware AWS Tuning 159

Adrian Cockcroft

JUNE 21, 2022

Photo by Adrian I spent six years at Cambridge Consultants, building some interesting systems, managing our Sun workstations and learning a lot, but by then Sun had opened a sales office across the street, and I wanted to find out what they were going to release next, before everyone else. He hasn’t changed.

Tuning 52

Tuning Benchmarking Engineering C++ 52

Percona

APRIL 17, 2023

In this blog post, we will discuss the best practices on the MongoDB ecosystem applied at the Operating System (OS) and MongoDB levels. The main objective of this post is to share my experience over the past years tuning MongoDB and centralize the diverse sources that I crossed in this journey in a unique place. 25.84 - Total 21.04

Best Practices 52

Best Practices Design Tuning Database 52

Dynatrace

JULY 23, 2020

Of course, we could define a static threshold for each disk within the IT system. This release extends auto-adaptive baselines to the following generic metric sources, all in the context of Dynatrace Smartscape topology: Built-in OneAgent infrastructure monitoring metrics (host, process, network, etc.). Custom log metrics.

Metrics 216

Metrics Innovation Strategy Monitoring 216

O'Reilly

JANUARY 17, 2023

You’ll try this with a few other algorithms, and their respective tuning parameters–maybe even break out TensorFlow to build a custom neural net along the way–and the winning model will be the one that heads to production. The second is the experienced ML professional who really knows how to build and tune models.

Tuning 128

Tuning Open Source AWS Software 128

All Things Distributed

OCTOBER 2, 2017

With these requirements in mind, and a willingness to question the status quo, a small group of distributed systems experts came together and designed a horizontally scalable distributed database that would scale out for both reads and writes to meet the long-term needs of our business. This was the genesis of the Amazon Dynamo database.

Internet 128

Internet Internet AWS Performance 128

All Things Distributed

NOVEMBER 29, 2017

Though the AWS Cloud gives you access to the storage and processing power required for ML, the process for building, training, and deploying ML models has unique challenges that often block successful use of this powerful new technology. However, many developers find them difficult to build and deploy.

Tuning 94

Tuning AWS Scalability Infrastructure 94

The Netflix TechBlog

DECEMBER 21, 2020

By Anupom Syam Background At Netflix, our current data warehouse contains hundreds of Petabytes of data stored in AWS S3 , and each day we ingest and create additional Petabytes. Use cases We found several use cases where a system like AutoOptimize can bring tons of value. Also, respond to ad-hoc requests created manually by end-users.

Storage 203

Storage Latency Efficiency Data Engineering 203

Percona

APRIL 12, 2023

Instead of relational (SQL) databases defined primarily through a hierarchy of related sets via tables and columns, their non-relational structure used a system of collections and documents. They came up with a horizontally scalable NoSQL database. Then came the coup de grace — the aforementioned stack-invasive SSPL.

Open Source 80

Open Source Programming Database Servers 80

Percona

JULY 2, 2023

In setting up our demonstration system, we use pg_basebackup and continuous archiving. Database Performance Tuning : By optimizing backup schedules, monitoring for errors, or looking for ways to reduce backup storage needs, DBAs are always working to improve the efficiency of backups and restores.

Strategy 52

Strategy Database Storage Open Source 52

All Things Distributed

NOVEMBER 28, 2012

Werner Vogels weblog on building scalable and robust distributed systems. They contain large amounts of locally attached storage on multiple spindles and are connected by a minimally oversubscribed 10 Gigabit Ethernet network. Loading, monitoring, tuning, taking backups, and recovering from faults are complex and time-consuming tasks.

Cloud 164

Cloud Storage Architecture Scalability 164

Brendan Gregg

SEPTEMBER 5, 2017

What follows are topics that may be of interest to anyone looking to migrate their systems and skillset: scan these to find topics that interest you. ## ZFS ZFS is available for Linux via the [zfsonlinux] and [OpenZFS] projects, and more recently was included in Canonical's Ubuntu Linux distribution: Ubuntu Xenial 16.04 LTS (April 2016).

Virtualization 76

Virtualization AWS Engineering Hardware 76

High Scalability

NOVEMBER 25, 2019

He and his colleagues spend their productive hours scaling large distributed file systems. Egnyte is a secure Content Collaboration and Data Governance platform, founded in 2007 when Google drive wasn't born and AWS S3 was cost-prohibitive. Java used to power core file system code. You can reach him at @kpatelwork. Introduction.

Architecture 52

Architecture Cache Azure Storage 52

O'Reilly

NOVEMBER 28, 2023

Even with cloud-based foundation models like GPT-4, which eliminate the need to develop your own model or provide your own infrastructure, fine-tuning a model for any particular use case is still a major undertaking. How will AI adopters react when the cost of renting infrastructure from AWS, Microsoft, or Google rises?

Open Source 106

Open Source Artificial Intelligence Google Infrastructure 106