Adrian Cockcroft

JANUARY 17, 2022

photo taken by Adrian Cockcroft A year ago I did a talk at re:Invent called Architecture Trends and Topics for 2021 , so I thought it was worth seeing how they played out and updating them for the coming year. There were five trends and topics for 2021, Serverless First, Chaos Engineering, Wardley Mapping, Huge Hardware, Sustainability.

Serverless 98

Serverless Hardware AWS Architecture 98

The Netflix TechBlog

JANUARY 13, 2022

Batch Pensive Batch Pensive Architecture Batch workflows in the data platform run using a Scheduler service that launches containers on the Netflix container management platform called Titus to run workflow steps. In the future, we are looking to automate this process.

Big Data 238

Big Data Infrastructure Metrics Hardware 238

Percona

JULY 1, 2023

Now that Database-as-a-service (DBaaS) is in high demand, there are multiple questions regarding AWS services that cannot always be answered easily: When should I use Aurora and when should I use RDS MySQL ? Amazon Aurora is a proprietary, cloud-native, fully managed relational database service developed by Amazon Web Services (AWS).

AWS 52

AWS Database Serverless Storage 52

Dynatrace

NOVEMBER 22, 2022

Cloud service providers, such as Amazon Web Services (AWS) , can offer infrastructure with five-nines availability by deploying in multiple availability zones and replicating data between regions. Complicating the situation further, increasingly connected services are pushing more data processing to the edge.

Infrastructure 191

Infrastructure Availability Systems Retail 191

Dynatrace

NOVEMBER 15, 2021

Log monitoring, log analysis, and log analytics are more important than ever as organizations adopt more cloud-native technologies, containers, and microservices-based architectures. Driving this growth is the increasing adoption of hyperscale cloud providers (AWS, Azure, and GCP) and containerized microservices running on Kubernetes.

Analytics 218

Analytics Monitoring Artificial Intelligence DevOps 218

Dynatrace

AUGUST 12, 2020

As companies strive to innovate and deliver faster, modern software architecture is evolving at near the speed of light. It allows for the breaking up of heavy monolithic architectures into multiple serverless “functions.” Understand and optimize your architecture. Dynatrace news. Azure Functions in a nutshell.

Serverless 241

Serverless Azure Monitoring Lambda 241

Dynatrace

JUNE 15, 2023

Cloud application security is a combination of policies, processes, and controls that aim to reduce the risk of exposing cloud-based applications to compromise or failure from external or internal threats. This process can delay deployments or cause developers to skip security testing completely to meet project deadlines.

Cloud 185

Cloud Open Source Architecture Monitoring 185

Scalegrid

JANUARY 8, 2024

Understanding Multi-Cloud and Hybrid Cloud Cloud computing has revolutionized the IT industry, offering a host of advantages including cost-effectiveness, increased agility, and access to cutting-edge hardware. In this scenario, two notable models – multi-cloud and hybrid cloud have emerged. But what do these entail?

Strategy 147

Strategy Cloud Scalability Artificial Intelligence 147

Dynatrace

JANUARY 16, 2023

Most Kubernetes clusters in the cloud (73%) are built on top of managed distributions from the hyperscalers like AWS Elastic Kubernetes Service (EKS), Azure Kubernetes Service (AKS), or Google Kubernetes Engine (GKE). Cloud-hosted Kubernetes clusters are on par to overtake on-premises deployments in 2023.

Open Source 349

Open Source Java Operating System Programming 349

Dynatrace

OCTOBER 1, 2021

As dynamic systems architectures increase in complexity and scale, IT teams face mounting pressure to track and respond to conditions and issues across their multi-cloud environments. An advanced observability solution can also be used to automate more processes, increasing efficiency and innovation among Ops and Apps teams.

Metrics 363

Metrics Open Source Monitoring Infrastructure 363

The Morning Paper

MAY 12, 2019

An open-source benchmark suite for microservices and their hardware-software implications for cloud & edge systems Gan et al., A typical architecture diagram for one of these services looks like this: Suitably armed with a set of benchmark microservices applications, the investigation can begin! Hardware implications.

Open Source 61

Open Source Hardware Benchmarking Systems 61

Dynatrace

MAY 1, 2020

The investment continues—we’re anticipating an upcoming release of the AWS Graviton2 processor , which has already been announced to be significantly more powerful than its predecessor. Network measurements with per-interface and per-process resolution. Network metrics are also collected for detected processes.

Java 134

Java Hardware Tuning Metrics 134

Dynatrace

SEPTEMBER 30, 2021

Cloud services platforms like AWS, Azure, and GCP are reshaping how organizations deliver value to their customers, making cloud migration an increasingly attractive option for running applications. A key step in digital transformation is migrating from traditional on-prem IT processes to adopting cloud services.

Cloud 172

Cloud Traffic Best Practices Strategy 172

Dynatrace

AUGUST 12, 2020

As companies strive to innovate and deliver faster, modern software architecture is evolving at near the speed of light. It allows for the breaking up of heavy monolithic architectures into multiple serverless “functions.” Understand and optimize your architecture. Dynatrace news. Azure Functions in a nutshell.

Serverless 144

Serverless Azure Monitoring Lambda 144

KeyCDN

FEBRUARY 22, 2023

The goal of WebAssembly is to execute at native speeds by taking advantage of common hardware features available on a variety of platforms. With cloud-based infrastructure, organizations can easily scale their web applications to handle increased traffic or demand without the need for expensive hardware upgrades.

Artificial Intelligence 111

Artificial Intelligence Development Serverless Website 111

All Things Distributed

MARCH 11, 2016

The epoch of AWS is the launch of Amazon S3 on March 14, 2006, now almost 10 years ago. Given that AWS is a pioneer in building and operating these services world-wide, these lessons have been of crucial importance to our business. This is a given, whether you are using the highest quality hardware or lowest cost components.

AWS 144

AWS Hardware Retail Virtualization 144

Percona

NOVEMBER 3, 2023

About the cluster Following a step-by-step process, the objective is to create a four-node cluster consisting of: PostgreSQL version 15 Citus extension (I’ll be using version 11, but there are newer ones available.) psql pgbench <<_eof1_ qecho adding node citus3.

Database 105

Database Benchmarking Latency C++ 105

Brendan Gregg

NOVEMBER 28, 2017

Hardware virtualization for cloud computing has come a long way, improving performance using technologies such as VT-x, SR-IOV, VT-d, NVMe, and APICv. At Netflix, we've been using these technologies as they've been made available for instance types in the AWS EC2 cloud. I'd expect between 0.1% small (thanks @cperciva ). ## 3.

Virtualization 40

Virtualization AWS Hardware Storage 40

Dynatrace

OCTOBER 28, 2021

But with the complexity that comes with digital transformation and cloud-native architecture, teams need a way to make sure applications can withstand the “chaos” of production. In 2009, the purveyor of online videos migrated to AWS cloud infrastructure to deliver its entertainment to a growing audience. Dynatrace news.

Engineering 234

Engineering Entertainment Cloud Infrastructure 234

All Things Distributed

NOVEMBER 15, 2016

Previously, I wrote about Amazon QuickSight , a new service targeted at business users that aims to simplify the process of deriving insights from a wide variety of data sources quickly, easily, and at a low cost. As a native offering from AWS, QuickSight comes deeply integrated with AWS data sources such as Amazon Redshift, RDS, and S3.

Analytics 152

Analytics Availability Media Social Media 152

High Scalability

NOVEMBER 25, 2019

Your Laptop has a filesystem used by hundreds of processes. It is limited by the disk space; it can’t expand storage elastically; it chokes if you run few I/O intensive processes or try collaborating with 100 other users. On prem data processing. SOA architecture based on REST APIs. This is the current architecture.

Architecture 52

Architecture Cache Azure Storage 52

All Things Distributed

AUGUST 16, 2012

The best practices that we are collecting in the AWS Economics Center are there to help our customers get a total view on their IT cost such that they can accurately compare on-premise and cloud. The AWS Economics Center now features the TCO of Web Applications whitepaper. Total Cost of Ownership and the Return on Agility. Comments ().

AWS 118

AWS Hardware Traffic Best Practices 118

The Netflix TechBlog

FEBRUARY 9, 2021

We can leverage high performance VMs in AWS to generate the assets. However, it would be cost-inefficient to leverage this same hardware for lightweight and more consistent traffic patterns that an asset management service requires. Here’s what the final architecture looked like. First, the fields can be coded by hand.

Engineering 165

Engineering Storage Latency Entertainment 165

Dotcom-Montior

NOVEMBER 11, 2020

those resources now belong to cloud providers, such as AWS Lambda, Google Cloud Platform, Microsoft Azure, and others. In serverless architecture, when applications are developed, they are typically composed of many different services. Other benefits to serverless architecture include the following: Cost.

Serverless 57

Serverless Monitoring Lambda Latency 57

All Things Distributed

NOVEMBER 15, 2010

From financial processing and traditional oil & gas exploration HPC applications to integrating complex 3D graphics into online and mobile applications, the applications of GPU processing appear to be limitless. Because of its focus on latency, the generic CPU yielded rather inefficient system for graphics processing.

AWS 136

AWS Latency Programming Architecture 136

All Things Distributed

SEPTEMBER 6, 2013

This spring I travelled through Europe for the AWS Global Summit series. Europe is a continent with much diversity and for each country there are great AWS customer examples to tell. To reach those goals, Shell in 2010 began using AWS. Shell leverages AWS for big data analytics to help achieve these goals.

Cloud 129

Cloud Energy AWS Healthcare 129

The Morning Paper

MAY 28, 2019

We don’t talk about energy as often as we probably should on this blog, but it’s certainly true that our data centres and various IT systems consume an awful lot of it. The goal is to produce a low-energy hardware classifier for embedded applications doing local processing of sensor data. An end-to-end architecture.

Energy 52

Energy Hardware Efficiency Architecture 52

Brendan Gregg

FEBRUARY 28, 2023

Make sure your system can handle next-generation DRAM,” [link] Nov 2011 - [Hruska 12] Joel Hruska, “The future of CPU scaling: Exploring options on the cutting edge,” [link] Feb 2012 - [Gregg 13] Brendan Gregg, “Blazing Performance with Flame Graphs,” [link] 2013 - [Shimpi 13] Anand Lal Shimpi, “Seagate to Ship 5TB HDD in 2014 using Shingled Magnetic (..)

Performance 110

Performance Latency Cache Virtualization 110

The Morning Paper

MARCH 8, 2020

When I think about cloud-native architectures, I think about disaggregation (enabling each resource type to scale independently), fine-grained units of resource allocation (enabling rapid response to changing workload demands, i.e. elasticity), and isolation (keeping tenants apart). joins) during query processing. S3 on AWS).

Storage 112

Storage Engineering Cache Serverless 112

All Things Distributed

JULY 13, 2010

Customers with complex computational workloads such as tightly coupled, parallel processes, or with applications that are very sensitive to network performance, can now achieve the same high compute and networking performance provided by custom-built infrastructure while benefiting from the elasticity, flexibility and cost advantages of Amazon EC2.

Cloud 118

Cloud AWS Automotive Latency 118

All Things Distributed

APRIL 5, 2012

Empowering innovation is at the heart of everything we do at Amazon Web Services (AWS). I often get to meet, discuss, and learn from innovators how they are using AWS to deliver transformative applications to their users, customers and partners. Troy: We moved our service from internal servers to AWS. hands freeÃ?

Innovation 146

Innovation AWS Education Network 146

HammerDB

JANUARY 5, 2019

Some opinions claim that “Benchmarks are meaningless”, “benchmarks are irrelevant” or “benchmarks are nothing like your real applications” However for others “Benchmarks matter,” as they “account for the processing architecture and speed, memory, storage subsystems and the database engine.”

Benchmarking 40

Benchmarking Database Social Media Scalability 40

Testsigma

MAY 19, 2021

A process that was developed to automate the processes from build creation to deployment and maybe everything in between. Cloud computing solutions work as a catalyst in this process due to the easily available high processing power of the cloud, resulting in faster deployments. Centralizes the Processes.

DevOps 59

DevOps Cloud Artificial Intelligence Infrastructure 59

All Things Distributed

DECEMBER 20, 2017

The benefit for customers: Authorized users can view this data and therefore manage their inventories across different sites, making the maintenance processes much more efficient. Breuninger, a fashion department store chain steeped in tradition, has recognized this and relies on a self-developed e-commerce platform in the AWS Cloud.

Artificial Intelligence 79

Artificial Intelligence Social Media Logistics AWS 79

Percona

APRIL 17, 2023

The swap issue is explained in the excellent article by Jeremy Cole at the Swap Insanity and NUMA Architecture. The report focuses on MySQL, but it is valid for MongoDB as well. Unfortunately, MongoDB is not NUMA-aware, and because of this, MongoDB can allocate memory unevenly, leading to the swap issue even with memory available.

Best Practices 52

Best Practices Design Tuning Database 52

All Things Distributed

JUNE 21, 2012

Amazon DynamoDB is the fastest growing new service in the history of AWS. In the five months since it launched in January, DynamoDB , our fast and scalable NoSQL database service, has been setting AWS growth records. By Werner Vogels on 21 June 2012 09:00 AM. Comments (). DynamoDB came along at just the right time.

Games 142

Games Scalability AWS Database 142

Adrian Cockcroft

JUNE 21, 2022

I became the Sun UK local specialist in performance and hardware, and as Sun transitioned from a desktop workstation company to sell high end multiprocessor servers I was helping customers find and fix scalability problems. We had specializations in hardware, operating systems, databases, graphics, etc. He hasn’t changed.

Tuning 52

Tuning Benchmarking Engineering C++ 52