DZone

JANUARY 26, 2024

In the ever-changing realm of modern business practices, ensuring the successful execution of batch jobs is vital for activities like data processing, system upkeep, and the general workflow of the organization. Understanding Batch Systems and Challenges Batch jobs involve sequences of operations executed without direct user involvement.

Systems 169

Systems Efficiency Innovation Processing 169

Dynatrace

MARCH 12, 2024

However, business processes can be inefficient, broken, or violate Service Level Objectives (SLOs) even when the underlying system’s health is good; a process is greater than the sum of its parts. Locked away in disparate systems, difficult to access, and with different formats, business data can be days or weeks old.

Processing 253

Processing Monitoring Analytics C++ 253

SQL Server According to Bob

DECEMBER 3, 2019

I have previously tested and blogged about the NTFS, sparse, file system limitation error 665: [link] when running DBCC or using Snapshot databases with SQL Server. In theory an XFS file can allocate 2 to the 52nd number of blocks so if 1/2 of those are holes then 2 to the 26th is the possible fragment count the file could achieve.

Servers 45

Servers Systems Storage Database 45

Scalegrid

MARCH 8, 2024

It relies heavily on binary log files to record the incremental changes needed for recovery. Monitoring and managing binary log activities, such as file retention, are essential for effective recovery. The restoration capability is primarily dependent on binary log files.

Database 162

Database Strategy Servers Best Practices 162

All Things Distributed

NOVEMBER 17, 2012

Werner Vogels weblog on building scalable and robust distributed systems. Back-to-Basics Weekend Reading - The Andrew File System. I am bringing with me a paper with one of first distributed systems that had actually see wide-spread commercial deployment. Scale and performance in a distributed file system , John H.

Systems 86

Systems Government Cache Scalability 86

DZone

DECEMBER 28, 2023

Here I am going to show the power of SQL Loader + Unix Script utility where multiple data files can be loaded by the SQL loader with automated shell scripts. This would be useful while dealing with large chunks of data and when data needs to be moved from one system to another system.

Programming 161

Programming Systems Availability Processing 161

DZone

NOVEMBER 28, 2023

Using open-source technology is a good idea when dealing with Automated Clearing House (ACH) batch processing, the primary system that agencies use for electronic funds transfer (EFT). Because the structure of an ACH file is very rarely modified by the Federal Reserve, once the technology is developed, very little maintenance is required.

Open Source 237

Open Source Technology Technology Processing 237

Scalegrid

FEBRUARY 8, 2024

A distributed storage system is foundational in today’s data-driven landscape, ensuring data spread over multiple servers is reliable, accessible, and manageable. This guide delves into how these systems work, the challenges they solve, and their essential role in businesses and technology.

Storage 130

Storage Systems Big Data Azure 130

DZone

NOVEMBER 7, 2023

This is about how a cyber security service provider built its log storage and analysis system (LSAS) and realized 3X data writing speed, 7X query execution speed, and visualized management. Log Storage and Analysis Platform In this use case, the LSAS collects system logs from its enterprise users, scans them, and detects viruses.

Storage 216

Storage Speed Systems 216

CSS Wizardry

OCTOBER 16, 2023

If you wanted to cache a file ‘forever’, you’d probably use a Cache-Control header like this: Cache-Control: max-age=31536000 This instructs any cache that it may store and reuse a response for one year (60 seconds × 60 minutes × 24 hours × 365 days = 31,536,000 seconds ). Let’s say we have a simple source CSS file called button.css.

Cache 187

Cache C++ Servers Internet 187

DZone

AUGUST 4, 2022

For the longest time, hosting static files on CDNs was the de facto standard for performance tuning website pages. Most developers and administrators think that adding a CDN-hosted static file improves performance. Not only did it have performance benefits, but it was also convenient for developers.

Cache 293

Cache Tuning Storage Website 293

DZone

JUNE 6, 2023

Sometimes, it is necessary to examine the behavior of a system to determine which process has utilized its resources, such as memory or CPU time. These resources are often scarce and may not be easily replenished, making it important for the system to record its status in a file. This is atop.

Network 257

Network Monitoring Systems Processing 257

DZone

FEBRUARY 1, 2023

For instance, Data Lake Storage Gen2 offers scale, file-level security, and file system semantics. Azure Data Lake Storage Gen1 and Azure Blob Storage's capabilities are combined in Data Lake Storage Gen2.

Azure 243

Azure Storage Big Data Analytics 243

DZone

DECEMBER 14, 2021

Metadata synchronization (sync) is a core feature in Alluxio that keeps files and directories consistent with their source of truth in under-storage systems, thus making it simple for users to reason the data retrieved from Alluxio. Meanwhile, understanding the internal process is important in order to tune the performance.

Design 244

Design Storage Tuning Systems 244

DZone

SEPTEMBER 13, 2023

In the realm of Linux and operating systems, disk imaging plays a vital role in various scenarios. They serve as a convenient means to store and distribute entire file systems, preserving their integrity and structure. One powerful tool available in the Linux world for creating disk images is mkisofs. What Is mkisofs?

Systems 141

Systems Media Operating System Availability 141

DZone

JANUARY 25, 2024

Extended Berkeley Packet Filter (eBPF) is a programming technology designed for the Linux operating system (OS) kernel space, enabling developers to create efficient, secure, and non-intrusive programs.

Monitoring 161

Monitoring Programming Operating System Efficiency 161

DZone

MARCH 24, 2023

Overview If you have ever downloaded a new Linux distribution ISO image, you may have wondered how to access the contents inside the image prior to repartitioning your disk and installing the operating system onto your local disk. In Linux and other UNIX-like systems, it is possible to use a regular file as a block device.

Virtualization 130

Virtualization Operating System Systems Availability 130

DZone

MARCH 9, 2023

Monitoring is a small aspect of our operational needs; configuring, monitoring, and checking the configuration of tools such as Fluentd and Fluentbit can be a bit frustrating, particularly if we want to validate more advanced configuration that does more than simply lift log files and dump the content into a solution such as OpenSearch.

Monitoring 245

Monitoring Testing Open Source Speed 245

The Netflix TechBlog

MARCH 7, 2024

The Machine Learning Platform (MLP) team at Netflix provides an entire ecosystem of tools around Metaflow , an open source machine learning infrastructure framework we started, to empower data scientists and machine learning practitioners to build and manage a variety of ML systems. ETL workflows), as well as downstream (e.g.

Systems 226

Systems Media Cache Open Source 226

Scalegrid

FEBRUARY 1, 2024

MySQL is a popular open-source relational database management system for online applications and data warehousing. However, data loss is always possible due to hardware malfunction, software defects, or other unforeseen circumstances, just like with any computer system. To back up all databases, the –all-databases option is used.

Best Practices 130

Best Practices Storage Strategy Database 130

DZone

OCTOBER 9, 2021

As requests go through a distributed software system, Spring Cloud Sleuth allows you to aggregate and track log entries. In a monolithic system, you can generally use the thread ID to filter the log, because all requests are simply logged to the same log file. There is no single log file.

Cloud 262

Cloud Servers Software Software 262

Dynatrace

MARCH 15, 2024

Syslog is the go-to protocol that delivers infrastructure administrators, network engineers, and security team logs that tell them all they need to know about their systems’ delivery, performance, availability, and security. Syslog is a protocol with clear specifications that require a dedicated syslog server.

Open Source 203

Open Source Infrastructure Network Government 203

Brendan Gregg

MARCH 23, 2024

Given how slow the system is, installing this package could take several minutes. Since the server owners are now in the SRE chatroom to help with the outage, you ask: "how do you install system packages? " Maybe this system is too slow with the performance issue? "Linux Crisis Tools" in SysPerf 2.)

Servers 145

Servers Traffic Network Systems 145

Dynatrace

JANUARY 23, 2024

Figure 1: XML external entity inclusion attack via man-In-the-middle Another scenario would be an XML response containing a pointer to read the contents of a local file on the client’s machine as shown in Figure 2. A possibility to make this exploit successful is tricking the client to connect to an attacker-controlled server. > <!DOCTYPE

Java 208

Java Servers Code Software 208

DZone

APRIL 11, 2023

It provides built-in connectors for various data sources such as databases, file systems, cloud storage, and more. Ingesting Data With Azure Data Factory Azure Data Factory is a cloud-based data integration service enabling you to ingest data from various sources into a cloud-based data lake or warehouse.

Azure 246

Azure Analytics Storage Cloud 246

Scalegrid

JANUARY 5, 2024

If you’re considering a database management system, understanding these benefits is crucial. Understanding Database Management Systems (DBMS) A Database Management System (DBMS) assists users in creating and managing databases. In a DBMS, data serves as the material which the system stores, structures, and controls.

Efficiency 147

Efficiency Storage Database Scalability 147

Scalegrid

NOVEMBER 23, 2023

Configure the PostgreSQL hostname by editing configuration files and restarting the server, with secure storage of connection details to enhance security. The host variable can be found within specified code strings dedicated to creating/modifying content present on that particular remotely located Postgresql platform system.

Database 130

Database Servers Storage Network 130

CSS Wizardry

SEPTEMBER 28, 2023

There’s some subtle complexity designed to prevent people gaming the system, which is exactly what we’re trying to do. penaltyFactor is the factor by which upscaling will count against us, given by min(displaySize, naturalSize) / displaySize , where: naturalSize is the pixel area of the image file in question.

Games 275

Games Ecommerce Cache Systems 275

DZone

MARCH 19, 2021

Sparse files are files with “holes”. File holes do not take up any physical space as the file system does not allocate any disk blocks for a hole until data is written into it. Virtual machine images are examples of spare files. Reading a hole returns a null byte.

Storage 219

Storage Virtualization Systems Performance 219

DZone

JANUARY 20, 2023

It holds responsibility for providing configuration information, naming, synchronization, and group services over large clusters in distributed systems. zNodes The key concept of ZooKeeper is the zNode, which can be acted either as files or directories.

Servers 152

Servers Systems 152

The Netflix TechBlog

NOVEMBER 13, 2023

In this blog post, we will introduce speech and music detection as an enabling technology for a variety of audio applications in Film & TV, as well as introduce our speech and music activity detection (SMAD) system which we recently published as a journal article in EURASIP Journal on Audio, Speech, and Music Processing.

Architecture 224

Architecture Energy Benchmarking Processing 224

Dynatrace

AUGUST 18, 2022

Whether deployment of a new system, a new release, or a rolling patch update, I wanted it done consistently and successfully every time. Because of the quantity of “proper noun solutions” I experienced in my System Administrator career, I wasn’t previously aware of the full spectrum of capabilities in Dynatrace.

Games 294

Games Monitoring Infrastructure DevOps 294

Percona

MAY 10, 2023

I have seen many Linux Performance engineers looking at the “IOWait” portion of CPU usage as something to indicate whenever the system is I/O-bound. IOWait is completely gone and now this system does not look I/O-bound at all! Because of this accounting, other interesting behaviors are possible.

Cache 144

Cache Monitoring Open Source Processing 144

Dynatrace

APRIL 14, 2022

With a critical CVSS rating of 9.8 , Spring4Shell leaves affected systems vulnerable to remote code execution (RCE). Figure 3: HTML file of our demo app showing the variable ${message} mapped from our DemoObject. 2 Set the prefix and suffix of the file to store it on a server. 3 Write arbitrary payload into the created file.

Java 218

Java Servers Code Open Source 218

Dynatrace

JULY 25, 2022

Building on the success of DevOps practices, GitOps is a relatively new way to manage infrastructure through code and automation, around a single Git repository (or a storage system for all the changes and files that relate to a given project). Development teams use GitOps to specify their infrastructure requirements in code.

DevOps 227

DevOps Infrastructure Speed Cloud 227

Dynatrace

MARCH 24, 2023

Containers enable developers to package microservices or applications with the libraries, configuration files, and dependencies needed to run on any infrastructure, regardless of the target system environment. This means organizations are increasingly using Kubernetes not just for running applications, but also as an operating system.

Infrastructure 188

Infrastructure Open Source Operating System Cloud 188

Scalegrid

FEBRUARY 15, 2024

Configuring MongoDB server for SSL/TLS involves modifying the mongod.conf file with SSL settings, including specifying paths to.pem files and enabling certificate validation to ensure only trusted clients can connect. The process continues by combining the self-signed CA certificate and private key into a single.pem file.

Servers 130

Servers Best Practices Government Network 130