O'Reilly

SEPTEMBER 15, 2021

In June 2021, we asked the recipients of our Data & AI Newsletter to respond to a survey about compensation. The average salary for data and AI professionals who responded to the survey was $146,000. We didn’t use the data from these respondents; in practice, discarding this data had no effect on the results.

Azure 145

Azure Social Media Programming AWS 145

Scalegrid

APRIL 16, 2020

Since our BYOC plans are hosted through your own AWS or Azure account, all cloud instances, backups and data transfer costs are paid directly through your cloud provider. While this is a good way to get a rough estimate, your monthly cloud costs will indeed vary based on the amount of backups performed and your data transfer activity.

Cloud 242

Cloud Azure AWS Database 242

Dynatrace

AUGUST 5, 2022

Additionally, a message queue can smooth out spiky workloads by enabling the producers and consumers to work at a consistent pace without losing data. In this scenario, message queues coordinate large numbers of microservices, which operate autonomously without the need to provision virtual machines or allocate hardware resources.

Monitoring 204

Monitoring Serverless Programming Speed 204

Dynatrace

AUGUST 5, 2022

Additionally, a message queue can smooth out spiky workloads by enabling the producers and consumers to work at a consistent pace without losing data. In this scenario, message queues coordinate large numbers of microservices, which operate autonomously without the need to provision virtual machines or allocate hardware resources.

Monitoring 204

Monitoring Serverless Programming Speed 204

ACM Sigarch

OCTOBER 25, 2019

When it comes to hardware support to mitigate software security issues, there is a significant gap between what is available in products today and known solutions. Acceleration—Adding hardware support to reduce the runtime overheads of security features. Approaches Towards Security.

Hardware 98

Hardware Virtualization Architecture Games 98

Dynatrace

NOVEMBER 29, 2022

As a result, teams must verify their data first because the triggering condition could be based on discovered data that is stale. Using Dynatrace Cloud Automation, teams can easily mitigate data breaches and security violations. Hence there are far fewer chances for false positives. Register now!

Infrastructure 176

Infrastructure Code Cloud DevOps 176

Dynatrace

APRIL 15, 2022

Organizations hit this cloud operations wall when replacing static virtual machines with dynamic container orchestration and expanding to multicloud environments. “This enabled us to fundamentally improve the application, remove performance bottlenecks, and also give us the data we needed to understand how to migrate into the cloud.”

Cloud 181

Cloud Infrastructure Innovation Monitoring 181

Abhishek Tiwari

DECEMBER 27, 2017

Kubernetes has emerged as go to container orchestration platform for data engineering teams. In 2018, a widespread adaptation of Kubernetes for big data processing is anitcipated. Organisations are already using Kubernetes for a variety of workloads [1] [2] and data workloads are up next. better cluster resource utilization.

Big Data 63

Big Data Storage Benchmarking Hardware 63

The Netflix TechBlog

NOVEMBER 9, 2022

While we understand it’s virtually impossible to achieve a linear increase in throughput as the number of vCPUs grow, a near-linear increase is attainable. a contiguous chunk of data (typically 64 bytes on x86 systems) transferred to and from the cache. to a larger AWS instance size, from m5.4xl (16 vCPUs) to m5.12xl (48 vCPUs).

Hardware 363

Hardware Cache Performance Latency 363

ACM Sigarch

MAY 31, 2023

Heterogeneous and Composable Memory (HCM) offers a feasible solution for terabyte- or petabyte-scale systems, addressing the performance and efficiency demands of emerging big-data applications. Also, we cannot expect the programmer to know data placement and handle different memory latency and bandwidth in heterogeneous memory.

Latency 52

Latency Hardware Cache Architecture 52

Abhishek Tiwari

JULY 12, 2018

Building data pipelines can offer strategic advantages to the business. Often companies underestimate the necessary effort and cost involved to build and maintain data pipelines. Data pipeline initiatives are generally unfinished projects. In this post, we will discuss why you should avoid building data pipelines in first place.

Latency 63

Latency Analytics Scalability Engineering 63

Brendan Gregg

JULY 4, 2021

AWS Graviton2); for memory with the arrival of DDR5 and High Bandwidth Memory (HBM) on-processor; for storage including new uses for 3D Xpoint as a 3D NAND accelerator; for networking with the rise of QUIC and eXpress Data Path (XDP); and so on. I also wrote about these topics in detail for my recent [Systems Performance 2nd Edition] book.

Performance 140

Performance Latency Hardware Storage 140

All Things Distributed

DECEMBER 20, 2017

How companies can use ideas from mass production to create business with data. In this way, designers are part of an ecosystem in which the functionalities of simulations, data and people come together, enabling them to develop better products faster. Value creation through data. Strategically, IT doesn't matter.

Artificial Intelligence 79

Artificial Intelligence Social Media Logistics AWS 79

The Morning Paper

MAY 27, 2019

Last week we saw the benefits of rethinking memory and pointer models at the hardware level when it came to object storage and compression ( Zippads ). The protections are hardware implemented and cannot be forged in software. At hardware reset the boot code is granted maximally permissive architectural capabilities.

C++ 61

C++ Hardware Virtualization Benchmarking 61

ACM Sigarch

APRIL 26, 2018

They introduced the architecture of coarse grain reconfigurable array (CGRA) for statically scheduled data flow computing in HOTCHIPS’17 and its software stack of compiler and linker in ICCAD’17. HPU: Holographic Processing Unit (HPU) is the specific hardware of Microsoft’s Hololens. FPU: Floating Processing Unit (FPU).

Hardware 99

Hardware Artificial Intelligence Architecture Google 99

Sutter's Mill

FEBRUARY 13, 2023

This meant that nearly all of the C++23 release cycle, and the entire “development” phase of the cycle, was done virtually via Zoom with many hundreds of telecons from 2020 through 2022. The first pandemic-cancelled in-person meeting would have been the first meeting of the three-year C++23 cycle.

C++ 132

C++ Hardware Virtualization Design 132

Adrian Cockcroft

FEBRUARY 9, 2023

The layers of platforms start at the bottom with hardware choices such as which CPU architectures and vendors you want to use. The virtualization and networking platform could be datacenter based, with something like VMware, or cloud based using one of the cloud providers such as AWS EC2.

Engineering 137

Engineering Serverless Lambda AWS 137

Nick Desaulniers

MAY 30, 2016

Data Models. What these programmers are seeing is known as data models. Programmer 1 one on a 64b x86-64 OSX machine had an LP64 data model where longs (L), (larger long longs,) and pointers (P) are 64b, but ints were 32b. Data model. There are older data models such as LP32 (Windows 3.1, sizeof(int). sizeof(long).

C++ 59

C++ Programming Code Hardware 59

Brendan Gregg

FEBRUARY 28, 2023

Make sure your system can handle next-generation DRAM,” [link] Nov 2011 - [Hruska 12] Joel Hruska, “The future of CPU scaling: Exploring options on the cutting edge,” [link] Feb 2012 - [Gregg 13] Brendan Gregg, “Blazing Performance with Flame Graphs,” [link] 2013 - [Shimpi 13] Anand Lal Shimpi, “Seagate to Ship 5TB HDD in 2014 using Shingled Magnetic (..)

Performance 110

Performance Latency Cache Virtualization 110

HammerDB

OCTOBER 23, 2023

The data shows a scripted automated workload running a number of back to back tests each time with an increasing number of virtual users. Gbits/sec We also test both local port and socket connections and found that a local port provided the best comparative data for both MySQL and MariaDB. On MySQL, we saw a 1.5X

Benchmarking 62

Benchmarking Database Network C++ 62

The Morning Paper

MARCH 8, 2020

This paper describes the design decisions behind the Snowflake cloud-based data warehouse. This paper presents Snowflake design and implementation along with a discussion on how recent changes in cloud infrastructure (emerging hardware, fine-grained billing, etc.) From shared-nothing to disaggregation. joins) during query processing.

Storage 112

Storage Engineering Cache Serverless 112

DZone

JULY 19, 2023

To address this need, the integration of cloud computing and virtualization has emerged as a groundbreaking solution as these technologies boast scalability and flexibility, entirely transforming the operational landscape. Alongside the transition to the cloud, Enel embraced virtualization to maximize the utilization of its IT resources.

Virtualization 130

Virtualization Scalability Energy Cloud 130

VoltDB

JUNE 1, 2017

It was – like the hypothetical movie I describe above – more than a little bit odd, as you could leave a session discussing ever more abstract layers of virtualization and walk into one where they emphasized the critical importance of pinning a network interface to a specific VM for optimal performance.

Hardware 56

Hardware Latency Virtualization Cloud 56

HammerDB

APRIL 19, 2023

Regardless of whether the computing platform to be evaluated is on-prem, containerized, virtualized, or in the cloud, it is crucial to consider several essential factors. From this data, it can be argued that HammerDB is a better candidate to test MySQL database engine and CPU performance. cpu_utilization_%_in_kernel_mode 29.70

Performance 54

Performance Benchmarking Cache Storage 54

Smashing Magazine

OCTOBER 27, 2021

Nowadays, hardware and software are designed to conduct eye-tracking studies for marketing , UX , psychological and medical research , gaming , and several other use cases. Another area that has been showing huge potential is eye-tracking in the context of virtual reality. Source: Nielsen Norman Group ) ( Large preview ).

Mobile 113

Mobile Website Hardware Social Media 113

VoltDB

JUNE 1, 2017

It was – like the hypothetical movie I describe above – more than a little bit odd, as you could leave a session discussing ever more abstract layers of virtualization and walk into one where they emphasized the critical importance of pinning a network interface to a specific VM for optimal performance.

Hardware 40

Hardware Latency Virtualization Cloud 40

Brendan Gregg

JULY 4, 2021

AWS Graviton2); for memory with the arrival of DDR5 and High Bandwidth Memory (HBM) on-processor; for storage including new uses for 3D Xpoint as a 3D NAND accelerator; for networking with the rise of QUIC and eXpress Data Path (XDP); and so on. I also wrote about these topics in detail for my recent [Systems Performance 2nd Edition] book.

Performance 52

Performance Latency Hardware Storage 52

Brendan Gregg

FEBRUARY 28, 2023

Make sure your system can handle next-generation DRAM,” [link] , Nov 2011 [Hruska 12] Joel Hruska, “The future of CPU scaling: Exploring options on the cutting edge,” [link] , Feb 2012 [Gregg 13] Brendan Gregg, “Blazing Performance with Flame Graphs,” [link] , 2013 [Shimpi 13] Anand Lal Shimpi, “Seagate to Ship 5TB HDD in 2014 using Shingled Magnetic (..)

Performance 52

Performance Latency Cache Virtualization 52

All Things Distributed

JANUARY 23, 2012

The Amazon Virtual Private Cloud extends on-premises compute with all the power of AWS, making it elastic, scalable and highly reliable. Data written to these volumes is maintained on your on-premises storage hardware while being asynchronously backed up to AWS, where it is stored in Amazon S3 in the form of Amazon EBS snapshots.

Storage 115

Storage AWS Virtualization Cloud 115

VoltDB

JUNE 19, 2017

Currently, most applications are hosted in a small number of data centers (1-5), each of which is in charge of a geographic region. With the 5G revolution, operators will need to manage hundreds of edge deployments, and maintain the physical space and hardware to achieve 1ms of latency. These have to communicate with each other.

Latency 40

Latency Retail Virtualization Hardware 40

VoltDB

JUNE 19, 2017

Currently, most applications are hosted in a small number of data centers (1-5), each of which is in charge of a geographic region. With the 5G revolution, operators will need to manage hundreds of edge deployments, and maintain the physical space and hardware to achieve 1ms of latency. These have to communicate with each other.

Latency 40

Latency Retail Virtualization Hardware 40

The Morning Paper

FEBRUARY 18, 2020

When it comes to leveraging ML in enterprise applications, especially in regulated environments, the level of scrutiny for data handling, model fairness, user privacy, and debuggability will be substantially higher than in the first wave of ML applications. The key insight this led to is that ML models are software , derived from data.

Government 95

Government Software Engineering Storage Database 95

KeyCDN

JANUARY 23, 2020

RUM gathers data from real life users to analyze application performance. A wide range of users with different operating systems, browsers, hardware configurations and other variables provides a wide sample size that helps developers discover as many issues as possible. What is real user monitoring (RUM)? Basic server metrics.

Strategy 72

Strategy Metrics Monitoring Servers 72

Testsigma

JULY 3, 2021

More control: While performing on-premise testing, organizations have more control over configurations, setup, hardware, and software. Data security: Since all the tools and machines are deployed within the premises — data is quite secure. This may require virtual and physical storage spaces, data centers, etc.

Cloud 57

Cloud Testing Storage DevOps 57

ACM Sigarch

FEBRUARY 23, 2018

Engineers, like economists, deal with large amounts of data and pride ourselves on our clinical ability to analyze and solve complex problems. Virtualization, for instance, was being addressed by IBM in the 1960s. I recall heated discussions 20+ years ago on whether speculative data in caches improves or degrades cache performance.

Architecture 47

Architecture Cache Hardware Virtualization 47

HammerDB

DECEMBER 5, 2020

A full understanding of why this is important requires some knowledge of the evolution of database hardware and software. For TPC-C this meant enough available spindles to reduce I/O latency and for TPC-H enough bandwidth for data throughput. This was both expensive and time consuming to configure.

C++ 40

C++ Benchmarking Virtualization Open Source 40

All Things Distributed

SEPTEMBER 4, 2017

ML is that part of AI that describes rules and recognizes patterns from large amounts of data in order to predict future data. Both concepts are virtually omnipresent and at the top of most buzzword rankings. As a result, there is a critical mass of data available. Because three things are coming together.

Artificial Intelligence 147

Artificial Intelligence Transportation Education Airlines 147