article thumbnail

Medallion Architecture: Efficient Batch and Stream Processing Data Pipelines With Azure Databricks and Delta Lake

DZone

This article explores the concepts of Medallion Architecture and demonstrates how to implement batch and stream processing pipelines using Azure Databricks and Delta Lake. Detailed code samples and explanations will be provided to illustrate each implementation step.

Azure 246
article thumbnail

What they don't tell you about migrating a message-based system to the cloud

Particular Software

If you search for “how to migrate to the cloud”, there are reams of articles that encourage you to understand your system, evaluate cloud providers, choose the right messaging service, and manage security and compliance. Now go rewrite all your code… we’ll wait…are you done yet? Have you chosen a cloud provider?

Cloud 126
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Breaking data silos: Liquid Reply’s journey to custom API observability with OpenTelemetry and Dynatrace

Dynatrace

Data is proliferating in separate silos from containers and Kubernetes to open source APIs and software to serverless compute services, such as AWS and Azure. The approach described in the article uses the OpenTelemetry Collector to convert the OTLP protocol in JSON format to the OTLP/HTTP format.

article thumbnail

AWS re:Invent 2021 shines light on cloud-native observability

Dynatrace

. “Because of the uncertainty of the times and the likely realities of the ‘new normal,’ more and more organizations are now charting the course for their journeys toward cloud computing and digital transformation,” wrote Gaurav Aggarwal in a Forbes article the impact of COVID-19 on cloud adoption.

AWS 213
article thumbnail

Cloud observability delivers on business value

Dynatrace

According to a recent Forbes articles, Internet users are creating 2.5 According to data cited by McConnell, Amazon Web Services, Microsoft Azure, and Google Cloud Platform grew in the last quarter, ending in June [2023] and jointly delivered almost $50 billion. quintilion bytes of data each day. McConnell said. Cloud modernization.

Cloud 222
article thumbnail

Distributed tracing with W3C Trace Context for improved end-to-end visibility (EAP)

Dynatrace

If you’re interested in learning more about the concepts of distributed tracing, context propagation, and the related challenges, take a look at this excellent article by Alois Reitbauer. Microsoft has already introduced Trace Context support in some of their services, including.NET Azure Functions, API Management, and IoT Hub.

Azure 148
article thumbnail

Architected for resiliency: How Dynatrace withstands data center outages

Dynatrace

This article was inspired by an email I received from Thomas Reisenbichler , Director of Autonomous Cloud Enablement on Friday, June 11 th. Those tests get executed from two locations (Paris and London) hosted by different cloud vendors (Azure & AWS). Fact #4: Multi-node, multi-availability zone deployment architecture.

AWS 189