article thumbnail

Medallion Architecture: Efficient Batch and Stream Processing Data Pipelines With Azure Databricks and Delta Lake

DZone

In today's data-driven world, organizations need efficient and scalable data pipelines to process and analyze large volumes of data. Medallion Architecture provides a framework for organizing data processing workflows into different zones, enabling optimized batch and stream processing.

Azure 246
article thumbnail

Building an Optimized Data Pipeline on Azure Using Spark, Data Factory, Databricks, and Synapse Analytics

DZone

Data processing in the cloud has become increasingly popular due to its scalability, flexibility, and cost-effectiveness. This article will explore how these technologies can be used together to create an optimized data pipeline for data processing in the cloud.

Azure 246
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Microsoft Ignite 2023 guide: AI transformation and Microsoft Azure

Dynatrace

To keep up, organizations are making significant investments to harness this technology and unlock new opportunities to thrive in the era of AI with Microsoft Azure and adjacent technologies. As a Microsoft strategic partner, Dynatrace delivers answers and intelligent automation for cloud-native technologies and Azure.

Azure 130
article thumbnail

Billing Management For Your Next SaaS Idea Using Stripe And Azure Functions

Smashing Magazine

Billing Management For Your Next SaaS Idea Using Stripe And Azure Functions. Billing Management For Your Next SaaS Idea Using Stripe And Azure Functions. The two applications in the aforementioned scenarios share a functional requirement, which is the ability to process the user’s payment. Nwani Victory. Introduction.

Azure 96
article thumbnail

Azure Virtual Machines for SQL Server Usage

SQL Performance

One initial, easy step to moving your SQL Server on-premises workloads to the cloud is using Azure VMs to run your SQL Server workloads in an infrastructure as a service (IaaS) scenario. One important choice you will still have to make is what type and size of Azure virtual machine you want to use for your existing SQL Server workload.

Azure 72
article thumbnail

Breaking data silos: Liquid Reply’s journey to custom API observability with OpenTelemetry and Dynatrace

Dynatrace

Data is proliferating in separate silos from containers and Kubernetes to open source APIs and software to serverless compute services, such as AWS and Azure. The approach described in the article uses the OpenTelemetry Collector to convert the OTLP protocol in JSON format to the OTLP/HTTP format.

article thumbnail

What they don't tell you about migrating a message-based system to the cloud

Particular Software

If you search for “how to migrate to the cloud”, there are reams of articles that encourage you to understand your system, evaluate cloud providers, choose the right messaging service, and manage security and compliance. Migrating a message-based system from on-premises to the cloud is a colossal undertaking. Have you chosen a cloud provider?

Cloud 126