Remove Big Data Remove Network Remove Scalability Remove Tuning
article thumbnail

Write Optimized Spark Code for Big Data Applications

DZone

Apache Spark is a powerful open-source distributed computing framework that provides a variety of APIs to support big data processing. In addition, pySpark applications can be tuned to optimize performance and achieve better execution time, scalability, and resource utilization.

Big Data 173
article thumbnail

How Netflix uses eBPF flow logs at scale for network insight

The Netflix TechBlog

By Alok Tiagi , Hariharan Ananthakrishnan , Ivan Porto Carrero and Keerti Lakshminarayan Netflix has developed a network observability sidecar called Flow Exporter that uses eBPF tracepoints to capture TCP flows at near real time. Without having network visibility, it’s difficult to improve our reliability, security and capacity posture.

Network 325
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Optimizing data warehouse storage

The Netflix TechBlog

Orient: Gather tuning parameters for a particular table that changed. AutoOptimize relies on some of the Iceberg specific features such as snapshot and atomic operations to perform the optimizations in an accurate and scalable manner. AutoAnalyze In short, AutoAnalyze finds the best tuning/configuration parameters for a table.

Storage 203
article thumbnail

USENIX LISA 2018: CFP Now Open

Brendan Gregg

Apart from networking, attending conferences like LISA in person is an effective way to upgrade your skills: you can block out work interruptions and absorb new knowledge that's been neatly summarized into sessions. We first met each other at LISA, in addition to making many other important industry connections over the years.

DevOps 43
article thumbnail

USENIX LISA 2018: CFP Now Open

Brendan Gregg

Apart from networking, attending conferences like LISA in person is an effective way to upgrade your skills: you can block out work interruptions and absorb new knowledge that's been neatly summarized into sessions. We first met each other at LISA, in addition to making many other important industry connections over the years.

DevOps 40
article thumbnail

World’s Top Web Performance Leaders To Watch

Rigor

One of the top players in web performance, Ilya is a web performance engineer at Google, co-chair of the W3C Web Performance Working Group , and author of High Performance Browser Networking. He has a keen interest in web technologies, performance tuning, security, and the practical use of technology. Doug Sillars. Doug Sillars.

article thumbnail

Should You Use ClickHouse as a Main Operational Database?

Percona

However, ClickHouse is super efficient for timeseries and provides “sharding” out of the box (scalability beyond one node). Currently, an issue has been opened to make the “tailing” based on the primary key much faster: slow order by primary key with small limit on big data.