No SAP BTP, no problem: Real-time SAP integration with Databricks

In February 2025, SAP & Databricks announced a landmark partnership to offer SAP Databricks, a natively integrated data + AI service inside the new SAP BDC (Business Data Cloud). SAP BDC itself is anchored within SAP BTP (Business Technology Platform).

SAP Databricks allows customers to run machine learning, generative AI and advanced analytics directly on semantically rich SAP business data, governed by Unity Catalog and shared seamlessly via Delta Sharing (SAP News, Databricks Blog). It represents the art of the possible as it eliminates data copies, curates trusted data products and enables cradle‑to‑grave governance as part of SAP’s broader BTP journey.

But here lies the professional hazard: not every SAP customer is ready to immediately adopt BTP and Business Data Cloud. Licensing models, project funding and organizational readiness mean that for many CIOs, SAP BTP remains a North Star destination, not tomorrow’s reality. Meanwhile, they’re under pressure: supply chain volatility, finance close windows shrinking and auditors watching for tell‑tale signs of gaps in governance. Many firms already run Databricks, feeding it IoT telemetry, Salesforce CRM, Kafka streams and e‑commerce data. They now want to blend S/4HANA ERP or SAP BW on HANA data without waiting for a BTP pivot.

This article is for those CIOs: Leaders looking for a pragmatic glidepath to real‑time SAP → Databricks Lakehouse integration without SAP BTP. We’ll walk through the technical blueprint, explore third‑party integration tools like Fivetran, Informatica and Workato, and show how a fail‑fast but governed approach makes this not just a “tech experiment,” but a competitive weapon.

The CIO’s North Star: The 5‑second SLA

The North Star for ERP analytics is now well defined in the industry: “Change posted in S/4HANA must reflect in Online Analytical Platform (OLAP) systems within ~5 seconds.”

That sounds audacious, but business reality demands it:

• A BSEG/ACDOCA posting that doesn’t reflect in GL analytics until tomorrow can mask liquidity risks.
• A MATDOC stock movement not visible in predictive models may cause halted operations in manufacturing.

The glidepath begins not with boiling the ocean but with fail‑fast pilots: replicate one high‑value domain (GL or order‑to‑cash), prove CDC pipelines work end‑to‑end, nip schema or latency issues in the bud, then scale outward, thereby replicating the mantra of “progress, not perfection.”

Any SAP architect knows the two canonical data surfaces for change data:

• SLT (SAP Landscape Transformation Replication Server): Trigger‑based CDC at the HANA DB level. Logs go into IUUC_LOGTAB, then out via DB or RFC connections.
• Operational Delta Queue (ODQ): Application‑level deltas for CDS Views or classic Extractors. Managed with delta tokens, ensuring cradle‑to‑grave accuracy.

The known‑unknowns like ‘will these pipelines hold up under quarter‑end postings of 200,000 docs/sec?’ pose a real challenge. Finally, the unknown‑unknowns around ABAP customizations that break change pointers or the compliance pivots that shift data residency obligations overnight can give sleepless nights. Smart IT Leaders design for them all — multi‑region failover, lineage built with Unity Catalog and governance so regulators don’t accuse you of “trying to pull a fast one.”

Technical glidepath: From SAP S/4 HANA to Databricks

The integration journey doesn’t need to start with a moonshot. Journey begins with foundation plumbing, then builds toward end‑to‑end streaming, transformation and finally predictive AI.

Sprint 0 is all about foundations

At this stage, SAP Basis and Databricks engineering teams set up the technical scaffolding: configuring SLT to capture deltas from tables like BSEG, activating delta‑enabled CDS views in the ODQ and ensuring encrypted, authorized connectivity between SAP and the target cloud region. The tell‑tale sign of trouble emerges early when log tables balloon disproportionately under load; that needs to be nipped in the bud before streaming overwhelms operations.

Sprint 1 brings streaming capture into play

Instead of lifting and shifting full tables, real‑time change capture streams into Bronze Delta tables. Whether via a native SAP ODP connector into Databricks or a Debezium‑on‑Kafka pipeline, the objective is identical: surface SAP’s transactional heartbeat into the Lakehouse within seconds with measurable KPI such as latency no greater than 5 seconds.

Sprint 2 is where the Medallion architecture shows its power

Bronze provides the raw landing zone, while Silver curates SAP’s intricacies into analytics‑friendly shapes and Gold delivers trusted KPIs for the business. Within this layer, the finance fact tables, product master dimensions and sales order repositories are harmonized while maintaining SAP semantics. If schema drift produces hundreds of evolution events, governance leaders should apply the carrot and stick: Enforce naming standards and lock NUMC keys into string format before analytics teams encounter corrupted joins.

Sprint 3 represents the payoff- BI + AI/ML integration

Now that SAP data lives in Gold, Business users and data scientists can experiment with anomaly detection for disputed invoices or predictive inventory rebalancing. The models themselves, governed with MLflow, can push intelligence back into SAP through lightweight OData services. In effect, we just created a cycle where the ERP is not only the system of record but is also continuously enriched by Lakehouse intelligence.

This pragmatically staged approach proves the mantra: progress, not perfection.

Middleware wildcards

There are multiple ways to skin the cat and while SAP’s native paths are powerful, third‑party integration tools can be game‑changers for many enterprises looking for speed or governance enhancements.

Relying on SAP’s native data surfaces works, but if you seek acceleration or governance benefits beyond what SLT and ODQ provide. Here, middleware vendors enter the scene as wildcards. Choosing between them depends on where the game lies — speed, compliance or process orchestration.

• Fivetran fits organizations that need tangible results in hours, not months. One can quite literally authorize a transport into SAP, configure the connector and watch general ledger entries land in Databricks Bronze the same afternoon. This is why Databricks named Fivetran its 2025 Partner of the Year. The downside? CDC intervals generally measure in minutes, not the sub‑five‑second North Star CIOs dream about. Still, for many, it’s fast enough to prove the art of the possible without bogging down internal teams.
• Informatica, by contrast, appeals to enterprises that cannot afford compliance missteps. Its Intelligent Data Management Cloud emphasizes regulated ingestion, with OData‑based deltas that satisfy even the most conservative auditors in financial services or pharma. By layering lineage, quality and governance capabilities on top, Informatica ensures cradle‑to‑grave oversight.
• Workato shines not as a high‑throughput replication engine but as a process orchestrator. Triggering recipes like “when a goods issue posts in SAP, update Databricks Bronze, notify logistics on Slack and synchronize Salesforce” is where Workato adds value. It acts as a segway between SAP’s transactional reality and cloud agility.
• MuleSoft can serve as a robust API led middleware solution. With its Anypoint Platform, MuleSoft provides a vast library of pre-built connectors and templates for both SAP and Databricks, which can accelerate the development of integration flows. This approach allows companies to quickly expose critical data from SAP systems like S/4HANA and SAP ECC and ingest it into the Databricks Lakehouse. MuleSoft’s role is to act as the “API conductor,” enabling different systems to communicate seamlessly and ensuring secure data exchange, which is crucial for building a unified, composable architecture.

Besides the above 3, there are many others to choose from, such as Boomi, Talend, Azure Data Factory, CGP Dataflow or AWS Glue. SAP Technical teams are best positioned to evaluate its needs against these offerings and narrow down to the right middleware. Middleware may promise agility, but hidden obligations, such as data residency, regulatory perimeter controls, can leave you in peril later.

The decision matrix looks like this in practice: SLT and ODQ excel for native, latency‑sensitive replication; Fivetran delivers fail‑fast speed for pilots; Informatica reassures compliance officers; and Workato enables event‑driven choreography. The right mix depends on whether your organization prizes time to value, governance certainty or process pivot flexibility.

Case study, and patterns CIOs should adopt

Real‑world case work suggests a few best paths forward that consistently yield the best outcomes:

• Feed Bronze via middleware, mature Silver/Gold in Databricks. Let Fivetran or Informatica handle ingestion quirks, then let Databricks execute governance, curation and machine learning.
• Enforce Unity Catalog everywhere. Cradle‑to‑grave lineage not only answers regulators. It empowers IT and business to trust the same single pane of glass.
• Plan hybrid deployments. Whether Fivetran’s hybrid feature or Informatica’s localized processing, hybrid keeps latency low while respecting pivot obligations like GDPR or DORA.

These aren’t nice‑to‑haves; they are pragmatic insurance against professional hazards that surface when you scale real‑time ERP modernization.

Consider Box (Workato – Box Customer Story), the enterprise content management platform. Box faced the common challenge such as seamless flow of critical financial & operational data, often originating in SAP, across its diverse SaaS ecosystem. Box leveraged Workato’s low‑code automation platform to build “recipes” that orchestrated these complex, SAP‑adjacent workflows. For instance, events triggered within SAP can automatically initiate actions in Salesforce, NetSuite or other cloud applications. This meant that a change in a vendor record, a new sales order or a financial close event could instantly update downstream systems, ensuring data consistency and accelerating processes.

This approach allowed Box to:

• Eliminate manual reconciliations & reduce the risk of human error, while freeing up finance teams from tedious, repetitive tasks.
• Accelerate cycle times of critical processes such as order‑to‑cash to procure‑to‑pay, moving faster, thereby directly impacting cash flow and operational responsiveness.
• Ensured data synchronization by automating the flow. Box maintained a consistent view of key business data across its enterprise, preventing data silos and ensuring that analytics platforms

The Box case study demonstrates that even without a full‑scale SAP BTP adoption, firms can achieve significant gains by focusing on event‑driven automation around their SAP core. It’s a clear example of a fail‑fast strategy that delivered immediate, measurable ROI by nipping process inefficiencies in the bud and ensuring cradle‑to‑grave data integrity.

The economics of real-time ERP: Beyond the price tag

When evaluating the investment in real‑time SAP integration, one must look beyond the direct costs of software licenses or cloud compute. The true economic impact lies in the opportunity cost of inaction and the value of accelerated decision‑making.

Consider the direct costs: streaming a few terabytes of SAP data monthly via SLT or middleware might incur minimal egress charges, while Databricks compute for processing and analytics could range from hundreds to a few thousand dollars per month. These are tangible, but often dwarfed by the hidden costs of delayed insights. One CIO, whom I worked with recently, recounted a missed opportunity that resulted in a $1 million loss. In such scenarios, the investment in a Fivetran connector or a Databricks Lakehouse becomes not an expense, but a critical investment against a big hit.

Furthermore, the economic benefits extend to operational efficiency. By automating data flows and enabling real‑time visibility, organizations can nip potential supply chain disruptions in the bud, optimize inventory and accelerate financial closings. This translates into reduced working capital, improved customer satisfaction and a more agile response to market shifts.

The journey to real‑time SAP + Databricks is not a straight line to perfection. Instead, it’s a glidepath defined by continuous improvement and iterative learning. The guiding principle must be progress, not perfection. This means embracing fail‑fast pilots, wherein one starts with a high‑value, manageable domain like General Ledger replication. These early initiatives provide tell‑tale signs of technical or organizational challenges, allowing teams to nip missteps in the bud before they escalate into larger, more costly problems.

The glidepath forward

CIOs and SAP Architects today face a defining moment. The pressure to unlock real-time insights from SAP S/4 HANA data isn’t just an IT ambition, but a business demand. This article offered a bird’s-eye view of that journey. There will always be roadblocks and challenges along the way. However, what separates the leaders from those left in peril is the willingness to act now, learn quickly and build on each success.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?