Brendan Gregg

MAY 28, 2021

I've moved from the San Francisco Bay Area to Sydney, Australia, where I will continue the best job so far of my career: Performance engineering at Netflix. I'm grateful for the support of Netflix engineering management, Netflix HRBPs, and others for helping to make this happen.

Google 144

Google Open Source Engineering AWS 144

Abhishek Tiwari

APRIL 27, 2021

It seriously turns me off when organisations are unable to differentiate between a role and a job. Talking about this confusion, Scrum Product Owner probably one of the most misunderstood roles these days which unfortunately many organisations advertise as a job. A job describes what one does daily.

Design 52

Design 52

DZone

DECEMBER 27, 2023

Welcome to the first post in our exciting series on mastering offline data pipeline's best practices, focusing on the potent combination of Apache Airflow and data processing engines like Hive and Spark. This post focuses on elevating our data engineering game, streamlining your data workflows, and significantly cutting computing costs.

Best Practices 130

Best Practices Data Engineering Big Data Games 130

DZone

AUGUST 1, 2022

If you’ve used Prometheus in the past for metrics collection/monitoring, then you’re probably familiar with Prometheus recording rules. As the Prometheus documentation says, recording rules allow you to precompute frequently needed or computationally expensive expressions and save their result as a new set of time series.

Speed 130

Speed Metrics Monitoring Performance 130

Professor Beekums

NOVEMBER 14, 2021

This is a conversation I had when I was interviewing a few years ago: Interviewer: “Yeah, a lot of candidates actually have trouble with this problem because it never shows up in the software we write.” ” Me: “…so why is it part of the interview?”

Performance 138

Performance Software Software 138

Dynatrace

MARCH 6, 2024

Find out how your LPARs are doing overall, which jobs are consuming the most resources, and which users are holding up your output queues. Maybe you’d like to know when your jobs are in a waiting status, when an application is spawning too few or too many job instances, or when all the disks in your pool are being utilized evenly.

Monitoring 242

Monitoring Infrastructure Metrics Analytics 242

The Netflix TechBlog

MARCH 4, 2024

In this blog post, we present our project on Auto Remediation, which integrates the currently used rule-based classifier with an ML service and aims to automatically remediate failed jobs without human intervention. Therefore, the operational cost increases linearly with the number of failed jobs. Fully automated.

Tuning 210

Tuning Efficiency Big Data Engineering 210

SQL Shack

JULY 14, 2020

In this article, I am going to explain how we can automate the index and statistics maintenance of Azure SQL Database using an Elastic Job Agent. The Microsoft SQL Server Agent component is used to automate the various tasks including database maintenance. But in Azure, the SQL Server Agent is only available in the Azure […].

Azure 64

Azure Database Servers Availability 64

Professor Beekums

MARCH 19, 2017

Job hopping is not considered an admirable trait, at least by employers. There is not much for them to like about a developer that leaves after a short tenure. Hiring people is an expensive process since it requires the expense of recruiters as well as time spent by the existing development team for interviews.

Software 60

Software Software Processing Development 60

Dynatrace

NOVEMBER 17, 2023

Within this workflow, a designated task, powered by JavaScript, initiates the build stage of the pipeline job, commencing a new build. The below screenshot shows the code initiating the Jenkins job. During the job pipeline execution, logs are extracted and pushed to Grail.

Innovation 231

Innovation Architecture Code Processing 231

InfoQ

DECEMBER 31, 2019

At Airbnb, they move time consuming, resource intensive tasks over to asynchronous background jobs to improve scalability. The job scheduling system has become a very important component and they have therefore built Dynein, a distributed delayed job queueing service and scheduler.

Scalability 52

Scalability Design Systems Architecture 52

SQL Shack

DECEMBER 4, 2019

In this article, we will talk about how to track enabled or disabled SQL jobs in SQL Server using T-SQL. Users with the Sysadmin role have the default permissions to modify the information of any jobs in SQL Server. If a user is not in this role and wants access to this activity, then the […].

Servers 56

Servers 56

Dynatrace

FEBRUARY 8, 2024

It is this huge moment for all of us in our careers and in how we do our jobs,” she said. But approximately 60% are concerned about job loss, stress, and other issues, given the impact of AI on the workforce. Knowing this, job descriptions and hiring methods need to adjust to build diverse teams. “Be

Artificial Intelligence 271

Artificial Intelligence Technology Technology Strategy 271

Dynatrace

DECEMBER 18, 2023

The data is examined and can fall into one of the below two categories: If there are errors in the telemetry data If there are errors in the telemetry data, the task immediately marks the stage as problematic and triggers the process to stop the ongoing job. This task also retrieves additional job metadata to include in the generated message.

Testing 259

Testing Best Practices Performance Testing Engineering 259

Professor Beekums

JULY 9, 2017

When is a developer’s job done?” Their job is done when code is written. My first job out of college had 3 month code freezes before each release. This may sound like a silly question. It seems obvious at first. Developers write code. I used to have this view. That 3 months was for the QA team to fully test things.

Development 58

Development Code Testing 58

Dynatrace

JANUARY 30, 2024

Is artificial intelligence (AI) here to steal government employees’ jobs? There is a lot of concern about AI taking jobs away from humans. Can embracing AI really make life easier? However, during Episode 74 of the Tech Transforms podcast , we discussed the roles that AI can fill.

Artificial Intelligence 210

Artificial Intelligence Government Innovation Tuning 210

Uber Engineering

MAY 16, 2018

In our push to empower our engineers to author more accurate alerts, Uber’s Observability Applications team sought to introduce alert … The post Engineering a Job-based Forecasting Workflow for Observability Anomaly Detection appeared first on Uber Engineering Blog.

Engineering 81

Engineering Monitoring Availability Systems 81

Professor Beekums

MARCH 12, 2017

There is way too much focus on developers improving their “interviewing skills” and not enough on interviewers doing a better job themselves. Also if a person’s job is to write software, they should be good at algorithms! One of the problems interviewers have is they ask candidates to solve classic computer science problems.

Software 67

Software Software Development Performance 67

Brendan Gregg

MAY 28, 2021

I've moved from the San Francisco Bay Area to Sydney, Australia, where I will continue the best job so far of my career: Performance engineering at Netflix. I also had prior jobs in the Bay Area where I'd be in the office most days past midnight, so compared to that this is like a health retreat!)

Google 40

Google Engineering Open Source AWS 40

DZone

JANUARY 10, 2024

Many of today’s hottest jobs didn’t exist at the turn of the millennium. Another relatively new job role in demand is that of a Site Reliability Engineer or SRE. But despite being new, the job role adds a lot of value to an organization. But despite being new, the job role adds a lot of value to an organization.

Social Media 141

Social Media DevOps Media Engineering 141

DZone

JANUARY 26, 2024

In the ever-changing realm of modern business practices, ensuring the successful execution of batch jobs is vital for activities like data processing, system upkeep, and the general workflow of the organization. Understanding Batch Systems and Challenges Batch jobs involve sequences of operations executed without direct user involvement.

Systems 169

Systems Efficiency Innovation Processing 169

Professor Beekums

JULY 14, 2016

Job descriptions for developer roles rarely tell you what they should. I’ve looked at close to a thousand job descriptions and have helped write a few. Job description writers, including me, have a tendency to come up with something “safe”. We fill job descriptions with text that sounds great, but says nothing tangible.

Development 49

Development 49

Professor Beekums

SEPTEMBER 30, 2019

I think a much simpler explanation is that hiring managers are trying to make their jobs easier. I’m going to give a benefit of a doubt and not assume this practice is due to some sinister motive.

Testing 42

Testing Processing 42

Dynatrace

MARCH 21, 2024

Siloed tools are among the chief complaints that lead to IT operations problems and job dissatisfaction. To put this into context, the survey found 49% of IT teams feel burned out by war rooms, 46% have missed personal time during evenings and weekends, and 21% have considered a change in job role or career due to added stress.

Games 196

Games Innovation Strategy Monitoring 196

The Netflix TechBlog

NOVEMBER 14, 2023

This stored data is then available for the subsequent LOAD.FACT_TABLE job in the workflow to utilize and for analysis and debugging purposes. Audits We run different audits on the uncommitted Iceberg snapshot created as part of the job run. Leveraging Psyberg metadata, we can identify the cohort of data involved as part of the job run.

Processing 226

Processing Data Engineering Efficiency Analytics 226

DZone

FEBRUARY 4, 2024

If you are a site reliability engineer (SRE) for a large Kubernetes-powered application, optimizing resources and performance is a daunting job.

Scalability 284

Scalability Engineering Performance 284

The Netflix TechBlog

DECEMBER 14, 2023

Start/Stop/Continue for optimizing complex ETL jobs Judit Lantos, Data Engineer, Member Experience Data Engineering, shares a case study to demonstrate an effective approach for optimizing complex ETL jobs.

Data Engineering 259

Data Engineering Engineering Software Engineering Best Practices 259

Dynatrace

MARCH 26, 2024

What makes you come into the day and say, ‘I love this job. This is the job I want to be doing,’ and ‘I can’t imagine doing anything else’?” Hamilton is directly contributing to educating and supporting the government’s work to attract and retain talent. asked Dr. Hamilton. The mission is super important.

Government 191

Government Artificial Intelligence Energy Education 191

DZone

FEBRUARY 9, 2024

These users are typically IT admins who require these privileges to perform their job duties, such as system administrators, database administrators , and network engineers. A privileged user is someone who has been granted elevated permissions to access certain data, applications, or systems within an organization.

Strategy 183

Strategy Architecture Network Database 183

Dynatrace

SEPTEMBER 21, 2023

The job of the platform team is to define how the environments are built and where they run, and to make sure they’re always available in an easy way.” It is a more active approach to collaboration that understands developers’ needs and takes deliberate steps to make their jobs easier. “No Safdar said.

Games 244

Games Engineering DevOps Education 244

Dynatrace

OCTOBER 25, 2022

The sidecar container remains active even when the job is already completed. Using Dynatrace to analyze the details of the pod, the K8s operator understands that, while the job container has already exited successfully, the sidecar proxy injected by the service mesh is still running, which prevents the pod from stopping and being deleted.

Efficiency 243

Efficiency Best Practices DevOps Open Source 243

The Netflix TechBlog

JANUARY 13, 2022

These steps launch jobs on clusters running Apache Spark and Presto via Genie. Pensive collects logs for the failed jobs launched by the step from the relevant data platform components and then extracts the stack traces. But our job is nowhere near done. If it crosses a threshold, Atlas sends a notification to Streaming Pensive.

Big Data 238

Big Data Infrastructure Metrics Hardware 238

DZone

AUGUST 3, 2022

The selection of the right test automation framework can greatly simplify the job of the developer working on the test code. The framework features can be exploited to write tests that perform the job with minimal implementation. to choose from.

Code 246

Code Testing Performance Development 246

The Netflix TechBlog

NOVEMBER 14, 2023

Step 2 : Stop all impacted workflows and downstream jobs (such as the sequential load ETL) and patch the missed data in the fact tables. Step 3 : Identify the number of partitions to be rerun for the sequential stateful load jobs to account for the delayed data and rerun them from the impacted date-hour.

Data Engineering 264

Data Engineering Engineering Processing Games 264

DZone

JULY 25, 2023

My job was to build an application to track orders for his company. When we arrived at the office that first day, he helped me to a nearby desk supporting a Compaq Portable computer. On top of the system was a book focused on using dBase II. My uncle knew I had an interest in computers, and I couldn’t wait to get started.

Scalability 245

Scalability Cloud Systems Development 245

The Polyglot Developer

JANUARY 30, 2018

Job hunting can be one of the most time and energy consuming tasks we have to do. In the following article I will try to provide you with some tips and tricks when applying and preparing for a job interview as a Software Engineer. The post TPDP Episode #16: Job Hunting As A Software Engineer appeared first on The Polyglot Developer.

Software Engineering 40

Software Engineering Engineering Software Software 40

Professor Beekums

MARCH 6, 2022

In 2014, I had been hearing a lot of talk about Go. It sounded like a really exciting language that I wanted to try. Having learned to code in Java and spending my early career working in Java, I was used to complex inheritence hierarchies and many many layers of abstraction. The notion of NOT having inheritence was so out there.

Java 55

Java Code 55