Expert insights. Personalized for you.
database big data performance benchmarking performance analysis redis alternative ec2 image ec2stack hardware news keydbWe've always been excited about Arm so when Amazon offered us early access to their new Arm-based instances we jumped at the chance to see what they could do. MORE
Hardware infrastructure. This is a guest post by Hugues Alary , Lead Engineer at Betabrand , a retail clothing company and crowdfunding platform, based in San Francisco. This article was originally published here. Early infrastructure. Rackspace. The scalability and maintainability issue. MORE
HPU: Holographic Processing Unit (HPU) is the specific hardware of Microsoft’s Hololens. SPU: Stream Processing Unit (SPU) is related to the specialized hardware to process the data streams of video. MORE
Cluster management, a common software infrastructure among technology companies, aggregates compute resources from a collection of physical hosts into a shared resource pool, amplifying compute power and allowing for the flexible use of data center hardware. MORE
Adaptive Loading is a pattern for delivering a fast core experience to all users (including low-end devices) where you progressively add high-end-only features, if a user's network and hardware can handle it MORE
Byte-addressable non-volatile memory,) NVM will fundamentally change the way hardware interacts, the way operating systems are designed, and the way applications operate on data. " Uncategorized Hardware Operating SystemsA persistent problem: managing pointers in NVM Bittman et al., MORE
The goal behind “real-time” containers is enabling your most important containers – those with mission-critical requirements around time-sensitive performance and reliability – to share the same hardware as non-real-time containers. MORE
This is the first post in a series of posts on different approaches to systems security especially as they apply to hardware and architectural security. Naively securing this system would require a large amount of trust; “guns and guards”, trusted personnel, and trusted software and hardware. MORE
The award recognizes "an individual or set of individuals for the development of a software or hardware system whose technical contributions have had significant impact on the theory or practice of large-scale data management systems MORE
Most Intel microprocessors support “HyperThreading” (Intel’s trademark for their implementation of “simultaneous multithreading”) — which allows the hardware to support (typically) two “Logical Processors” for each physical core. Last year I was trying to diagnose a mild slowdown in a code, and wanted to be able to use the hardware performance counters to divide processor activity into four categories: Neither Logical Processor active. MORE
Uncategorized Algorithms and data structures HardwareEfficient lock-free durable sets Zuriel et al., OOPSLA’19. Given non-volatile memory (NVRAM), the naive hope for persistence is that it would be a no-op: what happens in memory, stays in memory. MORE
When we wanted to add a location, we had to ship hardware and get someone to install that hardware in a rack with power and network. Hardware was outdated. Fixed hardware is a single point of failure – even when we had redundant machines. Dynatrace news. MORE
… to realize these insights, hardware needs to access data at object granularity and must have control over pointers between objects. Hotpads is a hardware-managed hierarchy of scratchpad-like memories called pads. Uncategorized Hardware Operating Systems MORE
According to Dr. Bandwidth, performance analysis has two recurring themes: How fast should this code (or “simple” variations on this code) run on this hardware? This can start with either a “top-down” or “bottom-up” approach, but in complex codes running on complex hardware, what is really required is both approaches — iterated until the interactions between all the components are understood. The networking hardware. MORE
A Dynatrace Managed cluster may lack the necessary hardware to process all the additional incoming data. The new ALR algorithm gives you more precise AI answers and optimized hardware utilization. All this automatically and with the same hardware. Dynatrace news. MORE
It can also retrieve information about hardware and software from the OS. Learn how to use Telegraf on Windows. MORE
Hardware performance counter results for a simple benchmark code calling Intel’s optimized DGEMM implementation for this processor (from the Intel MKL library) show that about 20% of the dynamic instruction count consists of instructions that are not packed SIMD operations (i.e., This is an uninspiring fraction of peak performance that would normally suggest significant inefficiencies in either the hardware or software. MORE
Cache Coherence Implementations Computer Architecture Computer Hardware Performance memory bandwidth memory latency Xeon PhiThe Xeon Phi x200 (Knights Landing) has a lot of modes of operation (selected at boot time), and the latency and bandwidth characteristics are slightly different for each mode. MORE
It seems very likely that the hardware has to be able to merge these two load operations into a single L1 Data Cache access to keep the rate of cache accesses from being the performance bottleneck. But 2 32-bit loads is only 1/8 of a natural 512-bit cache access, and it seems unlikely that the hardware can merge cache accesses across multiple cycles. Algorithms Computer Architecture Computer Hardware Performance arithmetic high performance computing microprocessors MORE
Computer Architecture Computer Hardware Performance cache DRAM high performance computing memory bandwidth memory latency STREAM benchmark“Memory Bandwidth and System Balance in HPC Systems” If you are planning to attend the SuperComputing 2016 conference in Salt Lake City next month, be sure to reserve a spot on your calendar for my talk on Wednesday afternoon (4:15pm-5:00pm). MORE
C++ Hardware Software Development Talks & EventsScott Meyers, Andrei Alexandrescu and I are continuing to craft and announce the technical program for C++ and Beyond (C&B) 2011 , and two more sessions are now posted. All talks are brand-new material created specifically for C&B 2011. Here are short blurbs; follow the links for longer descriptions. MORE
We know that getting full computational performance out of most machines—nearly all desktops and laptops, most game consoles, and the newest smartphones—already means harnessing local parallel hardware, mainly in the form of multicore CPU processing. You can expect the above keynote to be, well, keynote-y… oriented toward software product features and of course AMD’s hardware, with plenty of forward-looking industry vision style material. MORE
The division by a power of two ( / (2 N )) can be implemented as a right shift if we are working with unsigned integers, which compiles to single instruction: that is possible because the underlying hardware uses a base 2. Not all instructions on modern processors cost the same. Additions and subtractions are cheaper than multiplications which are themselves cheaper than divisions. For this reason, compilers frequently replace division instructions by multiplications. MORE
An important concern in optimizing the hardware platform is hardware components that restrict performance, known as bottlenecks. General DBA Performance Tuning hardwareQuite often, the problem isn’t correcting performance bottlenecks as much as it is identifying them in the first place. Start with obtaining a performance baseline. You monitor the server over time so that you can determine Server average […]. MORE
This data can include hardware statistics, such as measures of CPU or memory consumed over time. Summary There is a multitude of database metrics that we can collect and use to help us understand database and server resource consumption, as well as overall usage. MORE
Portable: It allows shipping a single EXE that can use any combination of GPU vendors’ hardware. The initial implementation uses DirectCompute and supports all devices that are DX11 capable; DirectCompute is just an implementation detail of the first release, and the model can (and I expect will) be implemented to directly talk to any interesting hardware. More to come… C++ Concurrency Hardware Microsoft Software Development Talks & Events MORE
" I was in the search for a straightforward tool that shows me hardware "VM" metrics and allows me to check the logs without ssh into the VM and ideally tool should be Open Source. Following on my previous article. MORE
Don’t trust hardware or software; then you can build trustworthy hardware and software. Hardware Software DevelopmentJames Hamilton on how to write reliable software in a world where anything that can fail, will fail. MORE
Now welcome to the hardware jungle. For the first time in the history of computing, mainstream hardware is no longer a single-processor von Neumann machine, and never will be again. Concurrency Hardware Opinion & Editorial Software Development MORE
EWR is the ratio of bytes issue by the iMC divided by the number of bytes actually written to the 3D-XPoint media (as measured by the DIMM’s hardware counters). Uncategorized Hardware PerformanceAn empirical guide to the behavior and use of scalable persistent memory , Yang et al., MORE
SQL Server Performance Tuning can be a difficult assignment, especially when working with a massive database where even the minor change can raise a significant impact on the existing query performance. MORE
Now welcome to the hardware jungle. Concurrency Hardware Software Development Talks & Events WebLast month in Kansas City I gave a talk on “Welcome to the Jungle,” based on my recent essay of the same name (sequel to “The Free Lunch Is Over”) concerning the turn to mainstream heterogeneous distributed computing and the end of Moore’s Law. MORE
The technology industry has made leaps and bounds in the last decade- in fact, so much that it’s hard to make sure all the new software and hardware available is safe and of good quality. MORE
An open-source benchmark suite for microservices and their hardware-software implications for cloud & edge systems Gan et al., Hardware implications. ASPLOS’19. MORE
Greenplum Database is an open-source , hardware-agnostic MPP database for analytics, based on PostgreSQL and developed by Pivotal who was later acquired by VMware. Greenplum Database is a massively parallel processing (MPP) SQL database that is built and based on PostgreSQL. MORE
Our Premium High Availability comes with the following features: Active-active deployment model for optimum hardware utilization. Save on costs for hardware and network bandwidth to optimize total cost of ownership. Dynatrace news. MORE
Mainstream hardware – many kinds of parallelism: What’s the relationship among multi-core CPUs, hardware threads , SIMD vector units (Intel SSE and AVX , ARM Neon ), and GPGPU (general-purpose computation on GPUs, which I covered at C++ and Beyond 2011 )? Task and data parallelism: What’s the difference between task parallelism and data parallelism, which kind of of hardware does each allow you to exploit, and why? MORE
Cluster management, a common software infrastructure among technology companies, aggregates compute resources from a collection of physical hosts into a shared resource pool, amplifying compute power and allowing for the flexible use of data center hardware. Architecture Apache Hadoop Apache Spark Big Data Capacity Planning Cassandra Cluster Management Data Center Hardware MySQL Peloton Redis Uber Uber Engineering Unified Resource Scheduler Workload Cluster MORE
The goal is to produce a low-energy hardware classifier for embedded applications doing local processing of sensor data. Race logic has four primary operations that are easy to implement in hardware: MAX, MIN, ADD-CONSTANT, and INHIBIT. Uncategorized Hardware Machine Learning MORE
Now welcome to the hardware jungle. Cloud Concurrency Hardware Software Development Talks & EventsWhile visiting Facebook earlier this month, I gave a shorter version of my “Welcome to the Jungle” talk, based on the eponymous WttJ article. MORE
Architecture Capacity Capacity Safety Hardware Capacity Planning Reliability site reliability engineering SRE Uber Eng Uber EngineeringReliability engineering teams at Uber build the tools, libraries, and infrastructure that enable engineers to operate our thousands of microservices reliably at scale. MORE
atomic<> Weapons: The C++11 Memory Model and Modern Hardware. We’ll include clear answers to several FAQs: “how do the compiler and hardware cooperate to remember how to respect these rules?”, “what is a race condition?”, C++ Hardware Software Development Talks & Events MORE
When it comes to hardware support to mitigate software security issues, there is a significant gap between what is available in products today and known solutions. Acceleration—Adding hardware support to reduce the runtime overheads of security features. MORE
Hardware Related Topics SQL Shack
APRIL 14, 2020
SQL Server Performance Tuning can be a difficult assignment, especially when working with a massive database where even the minor change can raise a significant impact on the existing query performance.
SQL Server Performance
MARCH 12, 2019
An important concern in optimizing the hardware platform is hardware components that restrict performance, known as bottlenecks. General DBA Performance Tuning hardwareQuite often, the problem isn’t correcting performance bottlenecks as much as it is identifying them in the first place. Start with obtaining a performance baseline. You monitor the server over time so that you can determine Server average […].
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
The Morning Paper
MAY 12, 2019
An open-source benchmark suite for microservices and their hardware-software implications for cloud & edge systems Gan et al., Hardware implications. ASPLOS’19.
Sutter's Mill
JUNE 29, 2012
atomic<> Weapons: The C++11 Memory Model and Modern Hardware. We’ll include clear answers to several FAQs: “how do the compiler and hardware cooperate to remember how to respect these rules?”, “what is a race condition?”, C++ Hardware Software Development Talks & Events
John McCalpin
SEPTEMBER 17, 2018
Most Intel microprocessors support “HyperThreading” (Intel’s trademark for their implementation of “simultaneous multithreading”) — which allows the hardware to support (typically) two “Logical Processors” for each physical core. Last year I was trying to diagnose a mild slowdown in a code, and wanted to be able to use the hardware performance counters to divide processor activity into four categories: Neither Logical Processor active.
DZone
MAY 14, 2020
database big data performance benchmarking performance analysis redis alternative ec2 image ec2stack hardware news keydbWe've always been excited about Arm so when Amazon offered us early access to their new Arm-based instances we jumped at the chance to see what they could do.
ACM Sigarch
APRIL 26, 2018
HPU: Holographic Processing Unit (HPU) is the specific hardware of Microsoft’s Hololens. SPU: Stream Processing Unit (SPU) is related to the specialized hardware to process the data streams of video.
The Morning Paper
MAY 23, 2019
… to realize these insights, hardware needs to access data at object granularity and must have control over pointers between objects. Hotpads is a hardware-managed hierarchy of scratchpad-like memories called pads. Uncategorized Hardware Operating Systems
Uber Engineering
MARCH 7, 2019
Architecture Capacity Capacity Safety Hardware Capacity Planning Reliability site reliability engineering SRE Uber Eng Uber EngineeringReliability engineering teams at Uber build the tools, libraries, and infrastructure that enable engineers to operate our thousands of microservices reliably at scale.
The Morning Paper
MARCH 17, 2020
EWR is the ratio of bytes issue by the iMC divided by the number of bytes actually written to the 3D-XPoint media (as measured by the DIMM’s hardware counters). Uncategorized Hardware PerformanceAn empirical guide to the behavior and use of scalable persistent memory , Yang et al.,
The Morning Paper
DECEMBER 1, 2019
Uncategorized Algorithms and data structures HardwareEfficient lock-free durable sets Zuriel et al., OOPSLA’19. Given non-volatile memory (NVRAM), the naive hope for persistence is that it would be a no-op: what happens in memory, stays in memory.
Uber Engineering
OCTOBER 30, 2018
Cluster management, a common software infrastructure among technology companies, aggregates compute resources from a collection of physical hosts into a shared resource pool, amplifying compute power and allowing for the flexible use of data center hardware. Architecture Apache Hadoop Apache Spark Big Data Capacity Planning Cassandra Cluster Management Data Center Hardware MySQL Peloton Redis Uber Uber Engineering Unified Resource Scheduler Workload Cluster
ACM Sigarch
OCTOBER 4, 2018
This is the first post in a series of posts on different approaches to systems security especially as they apply to hardware and architectural security. Naively securing this system would require a large amount of trust; “guns and guards”, trusted personnel, and trusted software and hardware.
The Morning Paper
MAY 28, 2019
The goal is to produce a low-energy hardware classifier for embedded applications doing local processing of sensor data. Race logic has four primary operations that are easy to implement in hardware: MAX, MIN, ADD-CONSTANT, and INHIBIT. Uncategorized Hardware Machine Learning
Uber Engineering
OCTOBER 30, 2018
Cluster management, a common software infrastructure among technology companies, aggregates compute resources from a collection of physical hosts into a shared resource pool, amplifying compute power and allowing for the flexible use of data center hardware.
John McCalpin
AUGUST 1, 2018
According to Dr. Bandwidth, performance analysis has two recurring themes: How fast should this code (or “simple” variations on this code) run on this hardware? This can start with either a “top-down” or “bottom-up” approach, but in complex codes running on complex hardware, what is really required is both approaches — iterated until the interactions between all the components are understood. The networking hardware.
Sutter's Mill
FEBRUARY 26, 2012
Don’t trust hardware or software; then you can build trustworthy hardware and software. Hardware Software DevelopmentJames Hamilton on how to write reliable software in a world where anything that can fail, will fail.
High Scalability
APRIL 8, 2019
Hardware infrastructure. This is a guest post by Hugues Alary , Lead Engineer at Betabrand , a retail clothing company and crowdfunding platform, based in San Francisco. This article was originally published here. Early infrastructure. Rackspace. The scalability and maintainability issue.
John McCalpin
OCTOBER 16, 2016
Computer Architecture Computer Hardware Performance cache DRAM high performance computing memory bandwidth memory latency STREAM benchmark“Memory Bandwidth and System Balance in HPC Systems” If you are planning to attend the SuperComputing 2016 conference in Salt Lake City next month, be sure to reserve a spot on your calendar for my talk on Wednesday afternoon (4:15pm-5:00pm).
John McCalpin
JANUARY 22, 2018
Hardware performance counter results for a simple benchmark code calling Intel’s optimized DGEMM implementation for this processor (from the Intel MKL library) show that about 20% of the dynamic instruction count consists of instructions that are not packed SIMD operations (i.e., This is an uninspiring fraction of peak performance that would normally suggest significant inefficiencies in either the hardware or software.
John McCalpin
NOVEMBER 5, 2016
It seems very likely that the hardware has to be able to merge these two load operations into a single L1 Data Cache access to keep the rate of cache accesses from being the performance bottleneck. But 2 32-bit loads is only 1/8 of a natural 512-bit cache access, and it seems unlikely that the hardware can merge cache accesses across multiple cycles. Algorithms Computer Architecture Computer Hardware Performance arithmetic high performance computing microprocessors
John McCalpin
DECEMBER 6, 2016
Cache Coherence Implementations Computer Architecture Computer Hardware Performance memory bandwidth memory latency Xeon PhiThe Xeon Phi x200 (Knights Landing) has a lot of modes of operation (selected at boot time), and the latency and bandwidth characteristics are slightly different for each mode.
Addy Osmani
DECEMBER 29, 2019
Adaptive Loading is a pattern for delivering a fast core experience to all users (including low-end devices) where you progressively add high-end-only features, if a user's network and hardware can handle it
Sutter's Mill
DECEMBER 29, 2011
Now welcome to the hardware jungle. For the first time in the history of computing, mainstream hardware is no longer a single-processor von Neumann machine, and never will be again. Concurrency Hardware Opinion & Editorial Software Development
SQL Shack
OCTOBER 2, 2019
This data can include hardware statistics, such as measures of CPU or memory consumed over time. Summary There is a multitude of database metrics that we can collect and use to help us understand database and server resource consumption, as well as overall usage.
Sutter's Mill
JUNE 21, 2012
Now welcome to the hardware jungle. Cloud Concurrency Hardware Software Development Talks & EventsWhile visiting Facebook earlier this month, I gave a shorter version of my “Welcome to the Jungle” talk, based on the eponymous WttJ article.
Sutter's Mill
APRIL 23, 2012
Now welcome to the hardware jungle. Concurrency Hardware Software Development Talks & Events WebLast month in Kansas City I gave a talk on “Welcome to the Jungle,” based on my recent essay of the same name (sequel to “The Free Lunch Is Over”) concerning the turn to mainstream heterogeneous distributed computing and the end of Moore’s Law.
QAMentor
JUNE 25, 2019
The technology industry has made leaps and bounds in the last decade- in fact, so much that it’s hard to make sure all the new software and hardware available is safe and of good quality.
Sutter's Mill
APRIL 6, 2011
We know that getting full computational performance out of most machines—nearly all desktops and laptops, most game consoles, and the newest smartphones—already means harnessing local parallel hardware, mainly in the form of multicore CPU processing. You can expect the above keynote to be, well, keynote-y… oriented toward software product features and of course AMD’s hardware, with plenty of forward-looking industry vision style material.
Dynatrace
JULY 8, 2019
When we wanted to add a location, we had to ship hardware and get someone to install that hardware in a rack with power and network. Hardware was outdated. Fixed hardware is a single point of failure – even when we had redundant machines. Dynatrace news.
Sutter's Mill
MAY 28, 2012
Mainstream hardware – many kinds of parallelism: What’s the relationship among multi-core CPUs, hardware threads , SIMD vector units (Intel SSE and AVX , ARM Neon ), and GPGPU (general-purpose computation on GPUs, which I covered at C++ and Beyond 2011 )? Task and data parallelism: What’s the difference between task parallelism and data parallelism, which kind of of hardware does each allow you to exploit, and why?
Sutter's Mill
JUNE 16, 2011
Portable: It allows shipping a single EXE that can use any combination of GPU vendors’ hardware. The initial implementation uses DirectCompute and supports all devices that are DX11 capable; DirectCompute is just an implementation detail of the first release, and the model can (and I expect will) be implemented to directly talk to any interesting hardware. More to come… C++ Concurrency Hardware Microsoft Software Development Talks & Events
Daniel Lemire
FEBRUARY 8, 2019
The division by a power of two ( / (2 N )) can be implemented as a right shift if we are working with unsigned integers, which compiles to single instruction: that is possible because the underlying hardware uses a base 2. Not all instructions on modern processors cost the same. Additions and subtractions are cheaper than multiplications which are themselves cheaper than divisions. For this reason, compilers frequently replace division instructions by multiplications.
Dynatrace
MAY 14, 2020
A Dynatrace Managed cluster may lack the necessary hardware to process all the additional incoming data. The new ALR algorithm gives you more precise AI answers and optimized hardware utilization. All this automatically and with the same hardware. Dynatrace news.
DZone
APRIL 7, 2020
The goal behind “real-time” containers is enabling your most important containers – those with mission-critical requirements around time-sensitive performance and reliability – to share the same hardware as non-real-time containers.
Sutter's Mill
APRIL 14, 2011
C++ Hardware Software Development Talks & EventsScott Meyers, Andrei Alexandrescu and I are continuing to craft and announce the technical program for C++ and Beyond (C&B) 2011 , and two more sessions are now posted. All talks are brand-new material created specifically for C&B 2011. Here are short blurbs; follow the links for longer descriptions.
ACM Sigarch
OCTOBER 25, 2019
When it comes to hardware support to mitigate software security issues, there is a significant gap between what is available in products today and known solutions. Acceleration—Adding hardware support to reduce the runtime overheads of security features.
DZone
APRIL 30, 2020
" I was in the search for a straightforward tool that shows me hardware "VM" metrics and allows me to check the logs without ssh into the VM and ideally tool should be Open Source. Following on my previous article.
Scalegrid
MAY 13, 2020
Greenplum Database is an open-source , hardware-agnostic MPP database for analytics, based on PostgreSQL and developed by Pivotal who was later acquired by VMware. Greenplum Database is a massively parallel processing (MPP) SQL database that is built and based on PostgreSQL.
DZone
DECEMBER 3, 2019
It can also retrieve information about hardware and software from the OS. Learn how to use Telegraf on Windows.
Dynatrace
JUNE 25, 2020
Our Premium High Availability comes with the following features: Active-active deployment model for optimum hardware utilization. Save on costs for hardware and network bandwidth to optimize total cost of ownership. Dynatrace news.
All Things Distributed
JULY 4, 2019
The award recognizes "an individual or set of individuals for the development of a software or hardware system whose technical contributions have had significant impact on the theory or practice of large-scale data management systems
130 
Let's personalize your content