It’s time to upgrade the PTC System Monitor (PSM)!

Dynatrace

As a PSM system administrator, you’ve relied on AppMon as a preconfigured APM tool for detecting, diagnosing, and repairing problems that impact the operational health of your Windchill application suite. There’s a more efficient way with Dynatrace. Dynatrace news.

Efficient Enterprise Testing — Integration Tests (Part Three)

DZone

Efficiency is everything! This part of the series will show how to verify our applications with code-level as well as system-level integration tests. performance junit integration testing system testing enterprise testing

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Teaching rigorous distributed systems with efficient model checking

The Morning Paper

Teaching rigorous distributed systems with efficient model checking Michael et al., It describes the labs environment, DSLabs , developed at the University of Washington to accompany a course in distributed systems. During the ten week course, students implement four different assignments: an exactly-once RPC protocol; a primary-backup system; Paxos; and a scalable, transactional key-value storage system. A visual debugger/system explorer.

Fine-grained, secure and efficient data provenance on blockchain systems

The Morning Paper

Fine-grained, secure and efficient data provenance on blockchain systems Ruan et al., That’s hard to do in today’s blockchain systems for two reasons: Provenance can only be determined by querying and replaying all on-chain transactions, which is inefficient and an offline activity. They demonstrate that provenance queries are efficient, and the system incurs small storage overhead.

Best Practices for Efficient Log Management and Monitoring

DZone

With so much flux and complexity across a cloud-native system, it's important to have robust monitoring and logging in place to control and manage the inevitable chaos. performance monitoring apm log management log efficient log management and monitoring log management best practices log storageWhen managing cloud-native applications, it's essential to have end-to-end visibility into what's happening at any given time.

Optimizing Database Performance and Efficiency

DZone

It's easy for modern, distributed, high-scale applications to hide database performance and efficiency problems. Optimizing performance of such complex systems at scale requires some skill, but more importantly it requires a sound strategy and good observability, because you can't optimize what you can't measure.

How Ably Efficiently Implemented Consistent Hashing

High Scalability

You can view the original article—H ow to implement consistent hashing efficiently —on Ably's blog. In this article, we’ll understand what consistent hashing is all about and why it is an essential tool in scalable distributed system architectures. Further, we’ll look at data structures that can be used to implement this algorithm efficiently at scale.

Ensuring Performance, Efficiency, and Scalability of Digital Transformation

Alex Podelko

Computing System Congestion Management Using Exponential Smoothing Forecasting by James Brady, State of Nevada. – System performance management is an important topic – and James is going to share a practical method for it. System Performance Estimation, Evaluation, and Decision (SPEED) by Kingsum Chow, Yingying Wen, Alibaba. Marrying Artificial Intelligence and Automation to Drive Operational Efficiencies by Priyanka Arora, Asha Somayajula, Subarna Gaine, Mastercard.

How to Create a Simple and Efficient PHP Cache

DZone

In this article, I’ll show you how to make a simple PHP caching system for your web pages. When working on PHP websites made from scratch and without a framework, speed can often be an issue. Caching is extremely useful in order to speed up PHP webpages. tutorial performance cache page caching

Cache 138

Optimizing Database Performance and Efficiency

Baron Schwartz

It’s easy for modern, distributed, high-scale applications to hide database performance and efficiency problems. Optimizing performance of such complex systems at scale requires some skill, but more importantly it requires a sound strategy and good observability, because you can’t optimize what you can’t measure.

Benchmarking spreadsheet systems

The Morning Paper

Benchmarking spreadsheet systems Rahman et al., They often freeze during computation, and are unable to import datasets well below the size limits posed by current spreadsheet systems. The other systems avoid this recomputation, but are slower than Excel for value-only datasets. Excel is more efficient if the data is sorted. Preprint. A recent TwThread drew my attention to this pre-print paper.

Orbital edge computing: nano satellite constellations as a new class of computer system

The Morning Paper

Orbital edge computing: nanosatellite constellations as a new class of computer system , Denby & Lucia, ASPLOS’20. Only space system architects don’t call it request-response, they call it a ‘ bent-pipe architecture.’. Nanosatellite systems have a GSD of around 3.0m/px.

Wireless attacks on aircraft instrument landing systems

The Morning Paper

Wireless attacks on aircraft instrument landing systems Sathaye et al., Today’s paper is a good reminder of just how important it is becoming to consider cyber threat models in what are primary physical systems, especially if you happen to be flying on an aeroplane – which I am right now as I write this! The first fully operational Instrument Landing System (ILS) for planes was deployed in 1932. USENIX Security Symposium 2019.

Machine learning systems are stuck in a rut

The Morning Paper

Machine learning systems are stuck in a rut Barham & Isard, HotOS’19. In this paper we argue that systems for numerical computing are stuck in a local basin of performance and programmability. Systems researchers are doing an excellent job improving the performance of 5-year old benchmarks, but gradually making it harder to explore innovative machine learning research ideas.

One SQL to rule them all: an efficient and syntactically idiomatic approach to management of streams and tables

The Morning Paper

One SQL to rule them all: an efficient and syntactically idiomatic approach to management of streams and tables Begoli et al., The watermark is maintained by the system. As an example, one area that caught my eye was the SQL standard definition that time in SQL queries is fixed at query time (either to the current time, or to a specified fixed time using AS OF SYSTEM TIME ).

File systems unfit as distributed storage backends: lessons from ten years of Ceph evolution

The Morning Paper

File systems unfit as distributed storage backends: lessons from 10 years of Ceph evolution Aghayev et al., In this case, the assumption that a distributed storage backend should clearly be layered on top of a local file system. Ceph is a widely-used, open-source distributed file system that followed this convention [of building on top of a local file system] for a decade. ” Ten years of building on local file systems.

Viewing and Sorting XEvents Efficiently (Code Samples) – XEProfiler

SQL Server According to Bob

using System; using System.Collections.Generic; using System.Drawing; using System.IO; using System.Linq; using System.Threading; using System.Threading.Tasks; using System.Windows.Forms; using Microsoft.SqlServer.XEvent.Linq; namespace XEProfiler { public partial class xeProfilerForm : Form { // Event locator index needs // private class LocationInfo { public QueryableXEventData Stream { get; private set; }.

PyTorch-BigGraph: a large-scale graph embedding system

The Morning Paper

PyTorch-BigGraph: a large-scale graph embedding system Lerer et al., Efficient negative sampling for large graphs. To increase memory efficiency on large graphs, we observe that a single batch of sampled source or destination nodes can be reused to construct multiple negative examples. The edge scores for a batch can be computed via an efficient batched matrix multiplication. SysML’19.

The #1 ServiceNow Integration Use Case for Enterprise Software Delivery Efficiency

Tasktop

Teams waste tremendous amounts of time “swivel-chairing” between systems, trying to keep records up-to-date in two places and sending emails and chat messages to follow up on details and statuses. The fact that teams use different systems of record and different workflows makes collaboration and communication between IT support staff and developers particularly challenging, and often results in longer resolution times. .

Alignment Efficiency: When to Sacrifice Speed for Greater Alignment

Strategic Tech

To address this challenge, I’d like to introduce you to the concept of Alignment Efficiency. I’ll show you some example scenarios to help you understand this concept, and I’ll demonstrate sociotechnical architecture patterns you can apply in your organisation to optimise your alignment efficiency. Any Formula 1 fans reading will likely be familiar with aerodynamic efficiency?—?adding Alignment Efficiency is inspired by aerodynamic efficiency.

Who monitors the monitoring systems?

Adrian Cockcroft

In reality, in any non-trivial installation, there are multiple tools collecting, storing and displaying overlapping sets of metrics from many types of systems and different levels of abstraction. These monitoring systems provide critical observability capabilities that are needed to successfully configure, deploy, debug and troubleshoot installations and applications. What if your monitoring systems fail? How do you even know when a monitoring system has failed?

Approaches to System Security: Using Cryptographic Techniques to Minimize Trust

ACM Sigarch

This is the first post in a series of posts on different approaches to systems security especially as they apply to hardware and architectural security. In this post, we will consider the use of mathematics/cryptography as an approach to improving systems security. The class of techniques described in this blog post, which we broadly refer to as applied hardware and architecture cryptography, apply proven cryptographic techniques to strengthen systems.

Improving the Cloud - More Efficient Queuing with SQS - All Things.

All Things Distributed

Werner Vogels weblog on building scalable and robust distributed systems. Improving the Cloud - More Efficient Queuing with SQS. Historically, messaging has been an important building block for building highly reliable distributed systems. s e-commerce platform, messaging systems have always been a key part of our service-oriented architecture to build an asynchronous communication pipeline between different services. All Things Distributed.

The Challenges and Traps of Architecting Sociotechnical Systems

Strategic Tech

If we a achieve a loosely-coupled, well-encapsulated architecture with an organizational structure to match we can achieve better delivery performance… and substantially grow the size of the engineering organization and increase productivity linearly” — Nicole Forsgren and Jez Humble in Accelerate From personal experiences, I’m sure we’ve all learned that getting the boundaries right in sociotechnical systems is extremely important yet monstrously difficult.

Corporate Middle Management as an Autopoietic System

The Agile Manager

[T]he aim of such systems is ultimately to produce themselves: their own organization and identity is their most important product. -- Gareth Morgan, Images of Organization , p. This is in contrast to allopoietic systems, which use components (raw materials such as silicon and plastic) to generate something (mobile phones and computers) which are distinct from the thing that created it (the factory where they are made). The system thus organizes its environment as part of itself.

Maximizing fun (and profit) in your distributed systems

Particular Software

Based on our experience running business systems in production, we know we need to monitor our theme park to make sure it's working properly. We can use this data to extrapolate when we need to upgrade the electrical system, add a new water pipe, add more bays to our carpark, or commission more trucks to haul away our trash. How many CPU cycles is a system using? Infrastructure monitoring tools generally treat systems as "black boxes" that consume resources.

PostgreSQL Connection Pooling: Part 1 – Pros & Cons

Scalegrid

On modern Linux systems, the difference in overhead between forking a process and creating a thread is much lesser than it used to be. Using a connection pool in each module is hardly efficient: Even with a relatively small number of modules, and a small pool size in each, you end up with a lot of server processes. A long time ago, in a galaxy far far away, ‘threads’ were a programming novelty rarely used and seldom trusted.

Content Management Systems of the Future: Headless, JAMstack, ADN and Functions at the Edge

Abhishek Tiwari

Recently I was asked about content management systems (CMS) of the future - more specifically how they are evolving in the era of microservices, APIs, and serverless computing. Raw content data along with templates are version controlled using Git or similar versioning systems. They were using a very legacy CMS or more precisely a document management system which was painful to work with.

Software-defined far memory in warehouse scale computers

The Morning Paper

” This paper describes a “far memory” system that has been in production deployment at Google since 2016. But for this to be realized in a practical manner, the system has to (1) be able to accurately control its aggressiveness to minimize the impact on application performance, and (2) be resilient to the variation of cold memory behavior across different machines,clusters, and jobs. The far memory system has been deployed in production since 2016.

Cluster Diagnostics: Troubleshoot Cluster Issues Using Only SQL Queries

DZone

Ideally, a TiDB cluster should always be efficient and problem-free. performance sql troubleshooting database administration distributed system tidbTiDB is an open-source, distributed SQL database that supports Hybrid Transactional/Analytical Processing (HTAP) workloads. It should be stable, load-balanced, and have a reliable rate of queries per second (QPS). There shouldn't be any jitters (either in the cluster or on disk), and no hotspots, slow queries, or network fluctuations.

Evolution of Netflix Conductor:

The Netflix TechBlog

Many of the Netflix Content and Studio Engineering services rely on Conductor for efficient processing of their business flows. External Payload Storage External payload storage was implemented to prevent the usage of Conductor as a data persistence system and to reduce the pressure on its backend datastore. Workflow Status Listener Conductor can be configured to publish notifications to external systems or queues upon completion/termination of workflows.

Lambda 163

More Manageable, Efficient Code Through 5S

Tim Kadlec

One way of combating this is by implementing the 5S System. The 5S System is actually a Japanese improvement process originally developed for the manufacturing industry. Each of the five words when translated to English began with ’S’ hence we call it the 5S System. Like many good philosophies however, the 5S System can apply to a variety of topics. By using the system applied to coding, we can make our code more efficient and much easier to maintain.

Seamless offloading of web app computations from mobile device to edge clouds via HTML5 Web Worker migration

The Morning Paper

The Mobile Web Worker (MWW) System. The Mobile Web Worker (MWW) System introduces a new client-side Mobile Web Worker Manager component which is responsible for managing web workers, including their migration when this is estimated to be beneficial. The current system assumes an application specific regression model is available on the servers which can predict processing time given the current parameters of the job (e.g. Uncategorized Distributed Systems

Firecracker: lightweight virtualization for serverless applications

The Morning Paper

Firekube exists because none of the existing alternatives (virtualisation, containers or language-specific vms) met the combined needs of multi-tenant efficiency and strong isolation in the AWS environment. This approach puts some limitations on packing efficiency, and also necessitates a container trade-off between security and code compatibility based on the types of syscalls containers are allowed to make. Uncategorized Containers Operating Systems

Mergeable replicated data types – Part II

The Morning Paper

How do you efficiently store, compute, and retrieve the LCA given two concurrent versions? The short summary here is that “ diff experiments show that the version control-inspired replication model can be efficiently supported for common data structures by transmitting succinct diffs over the network rather than entire versions.” Uncategorized Algorithms and data structures Distributed SystemsMergeable replicated data types – part II Kaki et al.,

Cloudburst: stateful functions-as-a-service

The Morning Paper

Given the simplicity and economic appeal of FaaS, it is interesting to explore designs that preserve the autoscaling and operational benefits of current offerings, while adding performant, cost-efficient and consistent shared state and communication. Cloudburst has four key components: function executors, caches, function schedulers, and a resource management system. Uncategorized Distributed SystemsCloudburst: stateful functions-as-a-service , Sreekanti et al., arXiv 2020.

Cache 64

A Brief Guide of xPU for AI Accelerators

ACM Sigarch

The new DGX-2, which achieves 2 petaFLOPS in the system, combines 16 fully interconnected GPUS with 10x the deep learning performance. Compared with Google Pixel 1, the HDR photography is accelerated by 5x and the power efficiency increased by 10x. ZPU : is a small, portable CPU core by a Norwegian company Zylin AS to run supervisory code in electronic systems that include an FPGA.

Back-to-Basics Weekend Reading - Join Processing in Relational.

All Things Distributed

Werner Vogels weblog on building scalable and robust distributed systems. It is very hard to implement the join operation efficiently as there any many unknowns in the execution of the operation. All Things Distributed. Back-to-Basics Weekend Reading - Join Processing in Relational Databases. By Werner Vogels on 12 April 2013 04:00 AM. Permalink. Comments (). Joins are one of the fundamental relational database query operations.

A tale of two abstractions: the case for object space

The Morning Paper

…software operating on persistent data structures requires "global" pointers that remain valid after a process terminates, while hardware requires that a diverse set of devices all have the same mappings they need for bulk transfers to and from memory, and that they be able to do so for a potentially heterogeneous memory system. Both abstractions must be implemented in a way that is efficient using existing hardware. Uncategorized Operating Systems

Compress objects, not cache lines: an object-based compressed memory hierarchy

The Morning Paper

Pads are designed to store variable sized objects efficiently, and a key feature is that they transfer objects across pad levels implicitly (just like cache levels) based on memory accesses. Uncategorized Hardware Operating SystemsCompress objects, not cache lines: an object-based compressed memory hierarchy Tsai & Sanchez, ASPLOS’19. Last time out we saw how Google have been able to save millions of dollars though memory compression enabled via zswap.

Cache 61

A case for lease-based, utilitarian resource management on mobile devices

The Morning Paper

LeaseOS borrows the concept of a lease from distributed systems, but with a rather nice twist, and is able to reduce power wastage by 92% with no disruption to application experience and no changes required to the apps themselves. If an app has been using resources efficiently, the lease manager increases the lease term, reverting back to the 5-second lease on any sign of misbehaviour. Uncategorized mobile Operating Systems

Seer: leveraging big data to navigate the complexity of performance debugging in cloud microservices

The Morning Paper

Last time around we looked at the DeathStarBench suite of microservices-based benchmark applications and learned that microservices systems can be especially latency sensitive, and that hotspots can propagate through a microservices architecture in interesting ways. Seer is an online system that observes the behaviour of cloud applications (using the DeathStarBench microservices for the evaluation) and predicts when QoS violations may be about to occur.

Sustainability versus Efficiency

The Agile Manager

In so doing, they're not focused on sustainability, they're focused on efficiency. Efficiency requires everything to stay the same. We luxuriate in the simplicity of holding everything else constant when we focus solely on efficiency. When we pursue efficiency, we're focused on where the business is right now. In efficiency-centric businesses, it’s not uncommon to find people doing substantially the same things that people were doing 10 years earlier.