All Things Distributed

AUGUST 18, 2011

Driving down the cost of Big-Data analytics. The Amazon Elastic MapReduce (EMR) team announced today the ability to seamlessly use Amazon EC2 Spot Instances with their service, significantly driving down the cost of data analytics in the cloud. The posting on the AWS developer blog also has some more background.

Big Data 111

Big Data Analytics AWS Cloud 111

The Netflix TechBlog

OCTOBER 28, 2021

Netflix’s unique work culture and petabyte-scale data problems are what drew me to Netflix. During earlier years of my career, I primarily worked as a backend software engineer, designing and building the backend systems that enable big data analytics. How’s data engineering similar and different from software engineering?

Data Engineering 173

Data Engineering Engineering Big Data Software Engineering 173

O'Reilly

AUGUST 15, 2023

To follow up on our previous survey about low-code and no-code tools, we decided to run another short survey about tools specifically for software developers—including, but not limited to, GitHub Copilot and ChatGPT. We’re interested in how “developer enablement” tools of all sorts are changing the workplace.

Development 73

Development Programming Speed Open Source 73

High Scalability

SEPTEMBER 14, 2020

This is a guest post by Limor Maayan-Wainstein , a senior technical writer with 10 years of experience writing about cybersecurity, big data, cloud computing, web development, and more. High performance computing (HPC) enables you to solve complex problems which cannot be solved by regular computing.

Cloud 261

Cloud Big Data Virtualization Efficiency 261

The Netflix TechBlog

JULY 15, 2021

I stumbled into data engineering rather than making an intentional career move into the field. I started my career as an application developer with basic familiarity with SQL. I was later hired into my first purely data gig where I was able to deepen my knowledge of big data. What drew you to Netflix?

Data Engineering 189

Data Engineering Engineering Entertainment Big Data 189

DZone

DECEMBER 21, 2020

Applications used in the field of Big Data process huge amounts of information, and this often happens in real time. Naturally, such applications must be highly reliable so that no error in the code can interfere with data processing. It is an open-source framework for distributed processing of large amounts of data.

Code 150

Code Java Big Data Open Source 150

The Morning Paper

MAY 14, 2019

Seer: leveraging big data to navigate the complexity of performance debugging in cloud microservices Gan et al., ASPLOS’19. Seer has now been deployed in the Social Network cluster for over two months, and in this time it has detected 536 upcoming QoS violations (90.6% accuracy) and avoided 495 (84%) of them.

Big Data 45

Big Data Cloud Performance Hardware 45

The Netflix TechBlog

MARCH 4, 2024

Operational automation–including but not limited to, auto diagnosis, auto remediation, auto configuration, auto tuning, auto scaling, auto debugging, and auto testing–is key to the success of modern data platforms.

Tuning 210

Tuning Efficiency Big Data Engineering 210

Scalegrid

MAY 13, 2020

Greenplum Database is an open-source , hardware-agnostic MPP database for analytics, based on PostgreSQL and developed by Pivotal who was later acquired by VMware. This feature-packed database provides powerful and rapid analytics on data that scales up to petabyte volumes. What Exactly is Greenplum? At a glance – TLDR.

Big Data 321

Big Data Database Artificial Intelligence Open Source 321

DZone

DECEMBER 28, 2020

The introduction of innovative technologies has brought the newest updates in software testing, development, design, and delivery. Nowadays, Big Data tests mainly include data testing, paving the way for the Internet of Things to become the center point. Besides, AI and ML seem to reach a new level.

Software 186

Software Software Testing Big Data 186

Dynatrace

JULY 6, 2022

Developing automation takes time. This kind of automation can support key IT operations, such as infrastructure, digital processes, business processes, and big-data automation. Big data automation tools. Automating routine IT tasks eliminates the human element—and the potential mistakes that come with it.

Artificial Intelligence 185

Artificial Intelligence Tuning Strategy Big Data 185

Uber Engineering

JUNE 1, 2023

Uber runs more than 100K big data workloads per day using Apache Spark–at that scale it’s crucial to write optimized apps. The Delivery Data Solutions team built Spark Analysers, a real-time system to catch anti-patterns in the Spark application at Uber scale, helping Uber developers optimize their apps.

Big Data 68

Big Data Systems Development 68

Dynatrace

MARCH 24, 2023

By embracing public cloud and hybrid cloud computing environments, IT teams can further accelerate development and automate software deployment and management. Container technology enables organizations to efficiently develop cloud-native applications or to modernize legacy applications to take advantage of cloud services.

Infrastructure 182

Infrastructure Open Source Operating System Cloud 182

Dynatrace

DECEMBER 15, 2022

We’ll discuss how the responsibilities of ITOps teams changed with the rise of cloud technologies and agile development methodologies. Adding application security to development and operations workflows increases efficiency. So, what is ITOps? What is ITOps? CloudOps teams are one step further in the digital supply chain.

Artificial Intelligence 198

Artificial Intelligence DevOps Hardware Virtualization 198

KeyCDN

OCTOBER 2, 2018

In the world of web development, those who become experts usually do so by learning from their predecessors. Reading and following the right web development blogs makes it much easier to get a solid education. That’s why we’ve compiled an exhaustive list of web development blogs and newsletters to make this process easier.

Development 75

Development Website Design Code 75

Dynatrace

JULY 6, 2020

As adoption rates for Microsoft Azure continue to skyrocket, Dynatrace is developing a deeper integration with the platform to provide even more value to organizations that run their businesses on Azure or use it as a part of their multi-cloud strategy. See the health of your big data resources at a glance. Azure Front Door.

Azure 146

Azure Cloud Big Data Virtualization 146

Dynatrace

APRIL 11, 2023

The focus on bringing various organizational teams together—such as development, business, and security teams — makes sense as observability data, security data, and business event data coalesce in these cloud-native environments. As organizations develop new applications, vulnerabilities will continue to emerge.

Cloud 183

Cloud DevOps Open Source Retail 183

Dynatrace

JANUARY 16, 2023

The immense growth of Kubernetes presents new security challenges in runtime and increased complexity in hardening CI/CD pipelines in development. Big data : To store, search, and analyze large datasets, 32% of organizations use Elasticsearch. Andreas Berger, Dynatrace Senior Principal Application Security.

Open Source 340

Open Source Java Operating System Programming 340

The Netflix TechBlog

JUNE 1, 2021

clinical data was often small enough to fit into memory on an average computer and only in rare cases would its computation require any technical ingenuity or massive computing power. There was not enough scope to explore the distributed and large-scale computing challenges that usually come with big data processing.

Data Engineering 130

Data Engineering Engineering Big Data Healthcare 130

Dynatrace

MARCH 14, 2022

With containers, microservices, applications, and other components that are constantly broken down and rebuilt as part of the software development lifecycle (SDLC), IT teams struggle to identify true issues and deliver software that enables doctors and nurses to deliver care. Today’s enterprise environments are dynamic and complex.

Healthcare 225

Healthcare Artificial Intelligence Innovation Strategy 225

Dynatrace

SEPTEMBER 15, 2020

In my recent Performance Clinic with Stefano Doni , CTO & Co-Founder of Akamas , I made the statement, “Application development and release cycles today are measured in days, instead of months. Supported technologies include cloud services, big data, databases, OS, containers, and application runtimes like the JVM.

Performance 183

Performance Java Metrics Cloud 183

The Netflix TechBlog

SEPTEMBER 18, 2020

I bring my breadth of big data tools and technologies while Julie has been building statistical models for the past decade. My work is typically developed in R or Python. [Chris] Julie and I joined the Streaming DSE team at Netflix a few years ago and have been close colleagues and friends since then. Do they cause less errors?

Analytics 223

Analytics Education Innovation Engineering 223

High Scalability

APRIL 14, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems : large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc.

Education 105

Education Software Engineering Scalability Engineering 105

High Scalability

MAY 12, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems : large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc.

Education 102

Education Software Engineering Scalability Engineering 102

High Scalability

APRIL 28, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems : large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc.

Education 102

Education Software Engineering Scalability Engineering 102

High Scalability

MARCH 24, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems : large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc.

Education 105

Education Software Engineering Engineering Big Data 105

High Scalability

MAY 26, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems : large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc.

Education 100

Education AWS Software Engineering Scalability 100

High Scalability

MARCH 30, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems : large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc.

Education 105

Education Software Engineering Engineering Big Data 105

Percona

JANUARY 19, 2023

Some of them are: MySQL Cluster: MySQL NDB Cluster is an in-memory database clustering solution developed by Oracle for MySQL. MyRocks: MyRocks is a storage engine developed by Facebook and made open source. It was developed for optimizing data storage and access for big data sets.

Open Source 90

Open Source Storage Database Big Data 90

Alex Podelko

DECEMBER 19, 2019

which would be great to attend to keep up with recent developments and their impact on my area. How is DevOps changing the Modern Software Development Landscape? , – Today’s hottest question for development – how we build performance engineering into continuous integration. a Panel Discussion.

Efficiency 100

Efficiency Artificial Intelligence Scalability Performance 100

All Things Distributed

SEPTEMBER 5, 2013

Over the past few years, two important trends that have been disrupting the database industry are mobile applications and big data. The explosive growth in mobile devices and mobile apps is generating a huge amount of data, which has fueled the demand for big data services and for high scale databases.

Big Data 98

Big Data Mobile Latency Database 98

The Netflix TechBlog

MARCH 2, 2021

I started working at a local payment processing company after graduation, where I built survival models to calculate lifetime value and experimented with them on our brand new big data stack. I was doing data science without realizing it. Data scientists can take on any aspect of an experimentation project.

Analytics 207

Analytics C++ Innovation Engineering 207

DZone

DECEMBER 14, 2020

In industries like banking, there's a real need for real-time data. Introduction We are living in the cloud age, which means developers do not need to set up their own local machine or dedicate on-premise environment to implement, test and run their application. They just create a VM machine in the cloud to perform those tasks.

Cloud 177

Cloud Testing Development Performance 177

All Things Distributed

JUNE 20, 2017

Today we have local teams in Hong Kong to help customers of all sizes as they move to AWS, including account managers, solutions architects, business developers, partner managers, professional services consultants, technology evangelists, start-up community developers, and more.

AWS 146

AWS Logistics Cloud Social Media 146

The Netflix TechBlog

JUNE 7, 2021

By Alok Tiagi , Hariharan Ananthakrishnan , Ivan Porto Carrero and Keerti Lakshminarayan Netflix has developed a network observability sidecar called Flow Exporter that uses eBPF tracepoints to capture TCP flows at near real time.

Network 325

Network Transportation AWS Cloud 325

All Things Distributed

SEPTEMBER 29, 2016

We have launched three points of presence, with two in Paris and one in Marseille, and also opened offices in the country, employing account managers, solutions architects, trainers, Business Development and Professional Services teams, as well as other job functions. Allez, rendez-vous à Paris – Une nouvelle région AWS arrive en France !

AWS 166

AWS IoT Internet Internet 166

O'Reilly

JANUARY 13, 2020

” Carol Willing , a member of the Python Steering Council and a core developer of CPython, also celebrates these projects—like the Binder service, which promotes reproducible research by creating an executable environment from your Jupyter Notebooks—particularly as they expand beyond their initial aims.

Programming 130

Programming Java Google C++ 130