INDUSTRY INSIGHTS YOUR PEERS ARE READING
Scaling development processes. This is a guest post by Hugues Alary , Lead Engineer at Betabrand , a retail clothing company and crowdfunding platform, based in San Francisco. This article was originally published here. Early infrastructure. Rackspace. Hardware infrastructure. MORE
This article we help distinguish between process metrics, external metrics and PurePaths (traces). If you already have the OneAgent installed, and you’d like to bring in additional process metrics into Dynatrace , the OneAgent plugin is a good fit. Dynatrace news. MORE
The obvious impact of this is that in the event of a master failure, the slave will not be up-to-date as its SQL thread is still processing the events in the relay log. This will delay the failover process as our framework expects the slave to be fully up-to-date before it can be promoted. To address this issue, we enable multi-threaded slaves with the option slave_parallel_workers to set the number of parallel SQL threads to process events in the relay logs. MORE
APU: Accelerated Processing Unit is the AMD’s Fusion architecture that integrates both CPU and GPU on the same die. BPU: Brain Processing Unit is the design of the AI chips by Horizon Robotics. CPU: Central Processing Unit. FPU: Floating Processing Unit (FPU). MORE
Batch processing of data is an efficient way of processing large volumes of data where data is collected, processed and then batch results are produced. Batch processing can be applied in many use cases. tutorial performance spring boot spring batch batch processing MORE
Why Process groups are not perfect. Dynatrace detects automatically identical processes and puts them under the same process group. Dynatrace will create a single process group called backend-API with 2 process instances underneath (host-A and host-B). MORE
The post How DevOps Testing can Enhance the Application Development Process? We have entered the digital age where technology is flourishing everywhere and is playing an integral role in our lives. Mobile applications are one great. MORE
In the previous posts in this series, we've seen some examples of long-running processes , how to model them, and where to store the state. So how can we ensure that our long-running process doesn't get into an inconsistent state if something fails along the way? MORE
In data processing it seems, all roads eventually lead back to SQL! In particular, we need to take care to separate the event time from the processing time (which could be some arbitrary time later). Uncategorized Datastores Stream processing MORE
Kill the PostgreSQL process. Manual intervention was required to start the PostgreSQL process again. Stop the PostgreSQL process. Manual intervention was required to start the PostgreSQL process again. Stop the repmgrd process. Kill the PostgreSQL process. Manual intervention was required to start the PostgreSQL process again. Stop the PostgreSQL process and bring it back immediately after health check expiry. Stop the repmgr process. MORE
Back-to-Basics Weekend Reading - Join Processing in Relational Databases. In 1992 Priti Mishra and Margaret Eich conducted a survey on what was achieved until then in Join Processing and described in details the algorithms, the implementation complexity and the performance. MORE
The time spent on a repetitive configuration process before completing a purchase order can negatively impact your software product’s time to value. MORE
print ( 'Hello from process {} out of {}'. In parallel programming with MPI, we need the so-called communicator , which is a group of processes that can talk to each other. To identify the processes with that group, each process is assigned a rank that is unique within the communicator. It also makes sense to know the total number of processes, which is often referred to as the size of the communicator. master process. print ( 'Process {} sent data:'. MORE
At present, according to the best software testing companies, there are various automated software testing … The post Tips to Enhance the Efficiency of Software Testing Process appeared first on QA Mentor Blog. Software Testing Test Process & Best Practices MORE
I love software process. Process can do wonders for that half. My passion is finding ways to build software faster, but better technology is only half the battle. The other half is a people and organization problem. MORE
For this test, I am using an Intel (skylake) processing and GCC 8.1. Not all instructions on modern processors cost the same. Additions and subtractions are cheaper than multiplications which are themselves cheaper than divisions. For this reason, compilers frequently replace division instructions by multiplications. Roughly speaking, it works in this manner. Suppose that you want to divide a variable n by a constant d. You have that n/d = n * (2 N / d ) / (2 N ). MORE
Its goal is to assign running processes to time slices of the CPU in a “fair” way. In this way a user-space process defines a “fence” within which CFS operates for each container. This user-space process is a Titus subsystem called titus-isolate which works as follows. MORE
Software Testing Test Process & Best PracticesAny software needs to go through various types of tests to assure that it has the required competitive edge. MORE
Visual testing is the automated process of detecting and reviewing visual UI changes. The visuals and UI of applications are critical parts of how our users use software, but often teams are still relying on slow, error-prone manual testing processes. MORE
With the evolution of storage formats like Apache Parquet and Apache ORC and query engines like Presto and Apache Impala , the Hadoop ecosystem has the potential to become a general-purpose, unified serving layer for workloads that can tolerate latencies … The post Hudi: Uber Engineering’s Incremental Processing Framework on Apache Hadoop appeared first on Uber Engineering Blog. MORE
Architecture Open Source Analytics Analytics Dashboard Apache Calcite Apache Flink Apache Samza Apache Storm App AthenaX Bill Lui Data Data Analytics Event Stream Processing Haohui Mai Kafka Michelangelo Mobile Naveen Cherukuri OS Restaurant Manager SQL Streaming Analytics Structured Query Language Uber Data UberEATS UberPOOL YARNUber facilitates seamless and more enjoyable user experiences by channeling data from a variety of real-time sources. MORE
As you all know security, privacy, and protection of our customer’s data is our number one priority and as such we work very closely with regulators to ensure that customers can be assured that they are getting the right protections when processing and storing data in the AWS. The media alert below that went out today gives the details: European Union Data Protection Authorities Approve Amazon Web Services’ Data Processing Agreement. MORE
Architecture 180 Days of Change Avro Batch Processing Configurability Dead Letter Queues Dead-Lettering DLQs Driver Engineering Driver Injury Protection Program Go Insurance Engineering Kafka Microservice Ning Xia Reliability Reliable Reprocessing SLAs Uber Uber EngineeringIn distributed systems, retries are inevitable. MORE
In this three-part blog series, we’ll share the following three common problem scenarios that you can easily solve by building an automated self-healing system with Dynatrace and xMatters Flow Designer: Process crash. As a first use case, let’s explore how your DevOps teams can prevent a process crash from taking down services across an organization—in five easy steps. Use case #1: Prevent a process crash from taking down services across an organization. Dynatrace news. MORE
For me, it was kinda fun to peel back each layer and see how it impacted the next part of the problem, and I hope my detailing it has taught people a thing or two in the process. MORE
Keptn is currently leveraging Knative and installs Knative as well as other depending components such as Prometheus during the default keptn installation process. In my case, both prometheus.knative-monitoring pods jumped in Process CPU and I/O request bytes. Dynatrace news. MORE
Last week I wrote about the performance consequences of inadvertently loading gdi32.dll into processes that are created and destroyed at very high rates. This is the first sign of the performance problems that will happen during process destruction. Process destruction (2.3 MORE
As you all know security, privacy, and protection of our customer’s data is our number one priority and as such we work very closely with regulators to ensure that customers can be assured that they are getting the right protections when processing and storing data in the AWS. The media alert below that went out today gives the details: European Union Data Protection Authorities Approve Amazon Web Services’ Data Processing Agreement. MORE
Improved Oracle process recognition. Previously, all Oracle processes were represented by one Oracle process group and one Oracle process group instance on each host. Starting with OneAgent version 1.173, each Oracle process group will represent a single Oracle SID (unique identifier for every Oracle DB instance). The SID is part of the process group name, and is extracted from process names (Unix) or service description (Windows). Dynatrace news. MORE
Windows has long had a reputation for slow file operations and slow process creation. This weeks’ blog post covers a technique you can use to make process creation on Windows grow slower over time (with no limit), in a way that will be untraceable for most users! MORE
A few months ago I wrote a blog post on How to Capture Per Process Metrics in PMM. Since that time, Nick Cabatoff has made a lot of improvements to Process Exporter and I’ve improved the Grafana Dashboard to match. Processes by Disk IO. MORE
Traditional stream-processing and complex event processing systems, such as Apache Storm and Software AG’s Apama , have focused on extracting interesting patterns from incoming data with stateless applications. MORE
Subtitle: Making Windows Slower Part 3: Process Destruction. Process destruction was slow, serialized, and was blocking the system input queue, leading to repeated short mouse-movement hangs when building Chrome. Every Windows process contains several default GDI object handles. MORE
delivering a large amount of business value in the process. Old Gatekeeper Architecture This model had several problems associated with it: This process was completely I/O bound and put a lot of load on upstream systems. Improved debuggability and visibility into liveness processing. MORE
Mounting object storage in Netflix’s media processing platform By Barak Alon (on behalf of Netflix’s Media Cloud Engineering team) MezzFS (short for “Mezzanine File System”) is a tool we’ve developed at Netflix that mounts cloud objects as local files via FUSE. MezzFS?—?Mounting MORE
The shortcomings and drawbacks of batch-oriented data processing were widely recognized by the Big Data community quite a long time ago. It became clear that real-time query processing and in-stream processing is the immediate need in many practical applications. MORE
At Uber’s scale, even a one percent increase in the rate of sign-ups to first trips (the driver conversion rate) carries a … The post Maximizing Process Performance with Maze, Uber’s Funnel Visualization Platform appeared first on Uber Engineering Blog. At Uber, we spend a considerable amount of resources making the driver sign-up experience as easy as possible. MORE
In previous blogs , we have explored the power of the digital twin model for stateful stream-processing. Digital twins are software abstractions that track the behavior of individual devices in IoT applications. They combine an event handling function with state information about each device. MORE
These are exciting times in the evolution of stream-processing. As we have seen in previous blogs , the digital twin model offers a breakthrough approach to structuring stateful stream-processing applications. It represents a big step forward for building stream-processing applications. MORE
To help equip you for the ongoing process of optimization and the life of debugging ahead of you, we’ve gathered a list of the best tools to monitor the JVM in both development and production environments. java open source performance monitoring jvm process java performance process monitoring MORE
Zombies probably won’t consume 32 GB of your memory like they did to me, but zombie processes do exist, and I can help you find them and make sure that developers fix them. He’d even written a tool that would dump a list of zombie processes – their names and counts. MORE
We’re thrilled to announce that we’ve added the Image Processing feature! How Does Image Processing Work? The Image Processing feature is available on all Pull Zones. This will improve the image processing performance and decrease the overall cost. MORE
Architecture Open Source Cadence Customer Obsession Platform Customer Support Natural Language Processing NLP Orchestration Engine Ticket Routing Ticket Routing WorkflowEvery day, Uber users around the world initiate customer support tickets through our Customer Obsession Platform. MORE
While there are still quite a lot of cases where it is still applicable, it needs to evolve into more sophisticated processes tightly integrated with development and other parts of performance engineering. Yes, the tools and process should be easier for non-experts to incorporate some performance testing into continuous development process. But then I recalled that installing LoadRunner was a real process – sometimes taking a half day, with a lot of choices to make. MORE
Processing Related Topics DZone
AUGUST 5, 2019
In the previous posts in this series, we've seen some examples of long-running processes , how to model them, and where to store the state. So how can we ensure that our long-running process doesn't get into an inconsistent state if something fails along the way?
DZone
AUGUST 2, 2019
Batch processing of data is an efficient way of processing large volumes of data where data is collected, processed and then batch results are produced. Batch processing can be applied in many use cases. tutorial performance spring boot spring batch batch processing
The Netflix TechBlog
MARCH 6, 2019
Mounting object storage in Netflix’s media processing platform By Barak Alon (on behalf of Netflix’s Media Cloud Engineering team) MezzFS (short for “Mezzanine File System”) is a tool we’ve developed at Netflix that mounts cloud objects as local files via FUSE. MezzFS?—?Mounting
Highly Scalable
AUGUST 20, 2013
The shortcomings and drawbacks of batch-oriented data processing were widely recognized by the Big Data community quite a long time ago. It became clear that real-time query processing and in-stream processing is the immediate need in many practical applications.
Kovair
JUNE 20, 2019
The post How DevOps Testing can Enhance the Application Development Process? We have entered the digital age where technology is flourishing everywhere and is playing an integral role in our lives. Mobile applications are one great.
DZone
APRIL 18, 2019
To help equip you for the ongoing process of optimization and the life of debugging ahead of you, we’ve gathered a list of the best tools to monitor the JVM in both development and production environments. java open source performance monitoring jvm process java performance process monitoring
Randon ASCII
OCTOBER 15, 2018
Windows has long had a reputation for slow file operations and slow process creation. This weeks’ blog post covers a technique you can use to make process creation on Windows grow slower over time (with no limit), in a way that will be untraceable for most users!
Percona
SEPTEMBER 18, 2018
A few months ago I wrote a blog post on How to Capture Per Process Metrics in PMM. Since that time, Nick Cabatoff has made a lot of improvements to Process Exporter and I’ve improved the Grafana Dashboard to match. Processes by Disk IO.
Randon ASCII
FEBRUARY 11, 2018
Zombies probably won’t consume 32 GB of your memory like they did to me, but zombie processes do exist, and I can help you find them and make sure that developers fix them. He’d even written a tool that would dump a list of zombie processes – their names and counts.
ScaleOut Software
MAY 4, 2018
Traditional stream-processing and complex event processing systems, such as Apache Storm and Software AG’s Apama , have focused on extracting interesting patterns from incoming data with stateless applications.
Professor Beekums
JULY 16, 2017
I love software process. Process can do wonders for that half. My passion is finding ways to build software faster, but better technology is only half the battle. The other half is a people and organization problem.
Tasktop
JULY 29, 2019
The time spent on a repetitive configuration process before completing a purchase order can negatively impact your software product’s time to value.
Uber Engineering
AUGUST 16, 2018
At Uber’s scale, even a one percent increase in the rate of sign-ups to first trips (the driver conversion rate) carries a … The post Maximizing Process Performance with Maze, Uber’s Funnel Visualization Platform appeared first on Uber Engineering Blog. At Uber, we spend a considerable amount of resources making the driver sign-up experience as easy as possible.
Uber Engineering
MARCH 12, 2017
With the evolution of storage formats like Apache Parquet and Apache ORC and query engines like Presto and Apache Impala , the Hadoop ecosystem has the potential to become a general-purpose, unified serving layer for workloads that can tolerate latencies … The post Hudi: Uber Engineering’s Incremental Processing Framework on Apache Hadoop appeared first on Uber Engineering Blog.
KeyCDN
MAY 14, 2019
We’re thrilled to announce that we’ve added the Image Processing feature! How Does Image Processing Work? The Image Processing feature is available on all Pull Zones. This will improve the image processing performance and decrease the overall cost.
Scalegrid
MARCH 12, 2019
Kill the PostgreSQL process. Manual intervention was required to start the PostgreSQL process again. Stop the PostgreSQL process. Manual intervention was required to start the PostgreSQL process again. Stop the repmgrd process. Kill the PostgreSQL process. Manual intervention was required to start the PostgreSQL process again. Stop the PostgreSQL process and bring it back immediately after health check expiry. Stop the repmgr process.
The Morning Paper
JULY 2, 2019
In data processing it seems, all roads eventually lead back to SQL! In particular, we need to take care to separate the event time from the processing time (which could be some arbitrary time later). Uncategorized Datastores Stream processing
All Things Distributed
APRIL 12, 2013
Back-to-Basics Weekend Reading - Join Processing in Relational Databases. In 1992 Priti Mishra and Margaret Eich conducted a survey on what was achieved until then in Join Processing and described in details the algorithms, the implementation complexity and the performance.
Scalegrid
JANUARY 8, 2019
The obvious impact of this is that in the event of a master failure, the slave will not be up-to-date as its SQL thread is still processing the events in the relay log. This will delay the failover process as our framework expects the slave to be fully up-to-date before it can be promoted. To address this issue, we enable multi-threaded slaves with the option slave_parallel_workers to set the number of parallel SQL threads to process events in the relay logs.
All Things Distributed
MARCH 31, 2015
As you all know security, privacy, and protection of our customer’s data is our number one priority and as such we work very closely with regulators to ensure that customers can be assured that they are getting the right protections when processing and storing data in the AWS. The media alert below that went out today gives the details: European Union Data Protection Authorities Approve Amazon Web Services’ Data Processing Agreement.
Randon ASCII
DECEMBER 3, 2018
Subtitle: Making Windows Slower Part 3: Process Destruction. Process destruction was slow, serialized, and was blocking the system input queue, leading to repeated short mouse-movement hangs when building Chrome. Every Windows process contains several default GDI object handles.
Randon ASCII
DECEMBER 9, 2018
Last week I wrote about the performance consequences of inadvertently loading gdi32.dll into processes that are created and destroyed at very high rates. This is the first sign of the performance problems that will happen during process destruction. Process destruction (2.3
ScaleOut Software
SEPTEMBER 20, 2018
These are exciting times in the evolution of stream-processing. As we have seen in previous blogs , the digital twin model offers a breakthrough approach to structuring stateful stream-processing applications. It represents a big step forward for building stream-processing applications.
QAMentor
DECEMBER 25, 2018
Software Testing Test Process & Best PracticesAny software needs to go through various types of tests to assure that it has the required competitive edge.
All Things Distributed
MARCH 31, 2015
As you all know security, privacy, and protection of our customer’s data is our number one priority and as such we work very closely with regulators to ensure that customers can be assured that they are getting the right protections when processing and storing data in the AWS. The media alert below that went out today gives the details: European Union Data Protection Authorities Approve Amazon Web Services’ Data Processing Agreement.
ScaleOut Software
AUGUST 18, 2018
In previous blogs , we have explored the power of the digital twin model for stateful stream-processing. Digital twins are software abstractions that track the behavior of individual devices in IoT applications. They combine an event handling function with state information about each device.
Uber Engineering
MARCH 28, 2019
Architecture Open Source Cadence Customer Obsession Platform Customer Support Natural Language Processing NLP Orchestration Engine Ticket Routing Ticket Routing WorkflowEvery day, Uber users around the world initiate customer support tickets through our Customer Obsession Platform.
Uber Engineering
FEBRUARY 16, 2018
Architecture 180 Days of Change Avro Batch Processing Configurability Dead Letter Queues Dead-Lettering DLQs Driver Engineering Driver Injury Protection Program Go Insurance Engineering Kafka Microservice Ning Xia Reliability Reliable Reprocessing SLAs Uber Uber EngineeringIn distributed systems, retries are inevitable.
Uber Engineering
OCTOBER 9, 2017
Architecture Open Source Analytics Analytics Dashboard Apache Calcite Apache Flink Apache Samza Apache Storm App AthenaX Bill Lui Data Data Analytics Event Stream Processing Haohui Mai Kafka Michelangelo Mobile Naveen Cherukuri OS Restaurant Manager SQL Streaming Analytics Structured Query Language Uber Data UberEATS UberPOOL YARNUber facilitates seamless and more enjoyable user experiences by channeling data from a variety of real-time sources.
Alex Podelko
AUGUST 18, 2019
While there are still quite a lot of cases where it is still applicable, it needs to evolve into more sophisticated processes tightly integrated with development and other parts of performance engineering. Yes, the tools and process should be easier for non-experts to incorporate some performance testing into continuous development process. But then I recalled that installing LoadRunner was a real process – sometimes taking a half day, with a lot of choices to make.
High Scalability
APRIL 8, 2019
Scaling development processes. This is a guest post by Hugues Alary , Lead Engineer at Betabrand , a retail clothing company and crowdfunding platform, based in San Francisco. This article was originally published here. Early infrastructure. Rackspace. Hardware infrastructure.
Dynatrace
JULY 11, 2019
Keptn is currently leveraging Knative and installs Knative as well as other depending components such as Prometheus during the default keptn installation process. In my case, both prometheus.knative-monitoring pods jumped in Process CPU and I/O request bytes. Dynatrace news.
Daniel Lemire
FEBRUARY 8, 2019
For this test, I am using an Intel (skylake) processing and GCC 8.1. Not all instructions on modern processors cost the same. Additions and subtractions are cheaper than multiplications which are themselves cheaper than divisions. For this reason, compilers frequently replace division instructions by multiplications. Roughly speaking, it works in this manner. Suppose that you want to divide a variable n by a constant d. You have that n/d = n * (2 N / d ) / (2 N ).
CSS Wizardry
AUGUST 13, 2019
For me, it was kinda fun to peel back each layer and see how it impacted the next part of the problem, and I hope my detailing it has taught people a thing or two in the process.
Dynatrace
AUGUST 14, 2019
Improved Oracle process recognition. Previously, all Oracle processes were represented by one Oracle process group and one Oracle process group instance on each host. Starting with OneAgent version 1.173, each Oracle process group will represent a single Oracle SID (unique identifier for every Oracle DB instance). The SID is part of the process group name, and is extracted from process names (Unix) or service description (Windows). Dynatrace news.
Dynatrace
JULY 10, 2019
This article we help distinguish between process metrics, external metrics and PurePaths (traces). If you already have the OneAgent installed, and you’d like to bring in additional process metrics into Dynatrace , the OneAgent plugin is a good fit. Dynatrace news.
PDC
AUGUST 2, 2019
print ( 'Hello from process {} out of {}'. In parallel programming with MPI, we need the so-called communicator , which is a group of processes that can talk to each other. To identify the processes with that group, each process is assigned a rank that is unique within the communicator. It also makes sense to know the total number of processes, which is often referred to as the size of the communicator. master process. print ( 'Process {} sent data:'.
The Netflix TechBlog
JULY 12, 2019
delivering a large amount of business value in the process. Old Gatekeeper Architecture This model had several problems associated with it: This process was completely I/O bound and put a lot of load on upstream systems. Improved debuggability and visibility into liveness processing.
Dynatrace
APRIL 24, 2019
Why Process groups are not perfect. Dynatrace detects automatically identical processes and puts them under the same process group. Dynatrace will create a single process group called backend-API with 2 process instances underneath (host-A and host-B).
ACM Sigarch
APRIL 26, 2018
APU: Accelerated Processing Unit is the AMD’s Fusion architecture that integrates both CPU and GPU on the same die. BPU: Brain Processing Unit is the design of the AI chips by Horizon Robotics. CPU: Central Processing Unit. FPU: Floating Processing Unit (FPU).
DZone
MAY 28, 2019
Visual testing is the automated process of detecting and reviewing visual UI changes. The visuals and UI of applications are critical parts of how our users use software, but often teams are still relying on slow, error-prone manual testing processes.
Dynatrace
JULY 25, 2019
In this three-part blog series, we’ll share the following three common problem scenarios that you can easily solve by building an automated self-healing system with Dynatrace and xMatters Flow Designer: Process crash. As a first use case, let’s explore how your DevOps teams can prevent a process crash from taking down services across an organization—in five easy steps. Use case #1: Prevent a process crash from taking down services across an organization. Dynatrace news.
The Netflix TechBlog
JUNE 4, 2019
Its goal is to assign running processes to time slices of the CPU in a “fair” way. In this way a user-space process defines a “fence” within which CFS operates for each container. This user-space process is a Titus subsystem called titus-isolate which works as follows.
264 
Let's personalize your content