13.1 — Monitoring Foundations
13.1 What this phase solves
A production Datasphere platform is not successful because its models are correct. It is successful when operators can answer, quickly and with evidence:
- Is data fresh?
- Did the pipeline complete?
- Which task failed?
- Where did the failure begin?
- Is the source unavailable?
- Is the problem transport, connection, replication, transformation, persistence, modeling, authorization, or consumption?
- Is a slow query caused by the source, the Datasphere model, remote federation, persistence, or concurrency?
- Is a discrepancy a data-quality defect or an expected business change?
- Who owns the failed component?
- What is the safe recovery action?
- How do we prove recovery?
- How do we prevent recurrence?
The central principle is:
Monitoring tells you that something happened. Troubleshooting explains why. Operations turns the explanation into a repeatable recovery path.
13.2 Current SAP monitoring model
Current SAP Datasphere documentation provides monitoring capabilities across data-integration tasks and platform operations. The Data Integration Monitor provides monitor views for objects such as remote tables, local tables, views, flows, and task chains. Run details expose execution history, status, scheduling information, and log messages.
Current documentation also provides specialized tools and views including:
- Data Pipeline Analyzer;
- View Analyzer;
- Runtime Metrics and Explain Plan for views;
- Remote Queries monitoring;
- task logs;
- expensive statement logs;
- local and remote table monitoring;
- task-chain run details and retry;
- Data Provisioning Agent monitoring and logs;
- storage and platform monitoring capabilities.
Treat these as complementary tools rather than competing tools.
Official documentation anchors:
- https://help.sap.com/docs/SAP_DATASPHERE
- https://help.sap.com/docs/SAP_DATASPHERE/be5967d099974c69b77f4549425ca4c0/5d4af7620a214872a4dc5286f5e9e997.html
- https://help.sap.com/docs/SAP_DATASPHERE/be5967d099974c69b77f4549425ca4c0/4142201ec1aa49ad89a688a2f1852c.html
- https://help.sap.com/docs/SAP_DATASPHERE/be5967d099974c69b77f4549425ca4c0/e0aeddba00b14be29b5e49b47001d43b.html
- https://help.sap.com/docs/SAP_DATASPHERE/d4f3c5a0bb074d09ae9b42b2b9bd7a08/83a5259b699e4bd4894470adf91192a7.html
- https://help.sap.com/docs/SAP_DATASPHERE/be5967d099974c69b77f4549425ca4c0/806d7f0c45a14f1fb07db0a226b2b822.html
- https://help.sap.com/docs/SAP_DATASPHERE/c8a54ee704e94e15926551293243fd1d/274f2736465c4c48a091c675880502a2.html
13.3 The enterprise observability mental model
Use five monitoring questions:
- Availability — Can the platform and required source systems be reached?
- Execution — Did the data movement or persistence task run?
- Freshness — Is the delivered data current enough?
- Correctness — Is the data complete and semantically correct?
- Performance — Is the workload completing within its expected resource and latency envelope?
A useful operational model is:
SOURCE | v CONNECTION | v ACQUISITION | v TRANSFORMATION | v PERSISTENCE | v SEMANTIC MODEL | v CONSUMPTION | v BUSINESS OUTCOME
Monitoring should cover every boundary.
13.4 Monitoring domains
| Domain | Primary question | Typical evidence |
|---|---|---|
| Platform | Is Datasphere operational? | System and platform monitoring |
| Connection | Can the source be reached? | Connection tests, source logs |
| Remote data | Is federation healthy? | Remote query monitor |
| Replication | Is data arriving? | Remote table monitor |
| Flows | Did movement/transformation complete? | Flow monitor |
| Persistence | Is materialized data current? | View monitor |
| Orchestration | Did dependencies execute? | Task-chain monitor |
| Runtime | Why is a model slow? | Runtime Metrics, View Analyzer |
| Pipeline | Where did data flow fail? | Data Pipeline Analyzer |
| Quality | Is data complete/correct? | Reconciliation and quality checks |
| Security | Is access behaving as designed? | Authorization and audit evidence |
| Operations | Is the service meeting its SLO? | Operational scorecards |
13.5 Monitoring is not the same as logging
A log answers:
What did this execution report?
A monitor answers:
What is the current operational state?
An alert answers:
Which condition requires attention?
An SLO answers:
How reliable does this service need to be?
A runbook answers:
What should the operator do next?
A mature operating model connects them:
Metric | v Threshold | v Alert | v Runbook | v Recovery | v Verification | v RCA | v Prevention
13.6 Data Integration Monitor
The Data Integration Monitor is the operational starting point for many data-movement investigations.
Current SAP documentation describes monitor tabs for task types such as:
- Remote Tables;
- Local Tables;
- Views;
- Flows;
- Task Chains;
- Remote Queries.
The monitor provides execution history and run details. Depending on the object type, the details include status, schedule information, run type, logs, resource information, and execution messages.
Operational rule:
Start with the monitor that owns the failing runtime object before changing the model.