13.7 — Enterprise Observability Architecture
13.81 API-driven operations
A mature platform can separate:
Human diagnosis | v Approved action | v API execution | v Verification
The API is the execution mechanism, not the decision maker.
Use technical users and OAuth appropriately for service-to-service scenarios.
13.82 Observability data model
A useful operational event model contains:
| Field | Purpose |
|---|---|
| object | What ran |
| space | Ownership boundary |
| run_id | Execution identity |
| timestamp | Timeline |
| status | Outcome |
| duration | Performance |
| row_count | Volume |
| watermark | Freshness |
| error_class | Triage |
| owner | Accountability |
| retry_count | Reliability |
| source | Dependency |
Not every field is available from every Datasphere monitor. Where a value is not natively exposed, derive it from controlled operational checks.
13.83 Operational data product
Treat operational metrics as a data product.
Example:
Datasphere runtime events | v Operational model | +--> platform scorecard +--> freshness dashboard +--> failure dashboard +--> capacity dashboard +--> executive KPI
SAP provides monitoring content and monitoring views for supported scenarios; use current product capabilities before building custom telemetry.
13.84 Monitoring content
Current SAP documentation describes SAP Monitoring Content for Data Integration Tasks. It provides preconfigured monitoring views intended for flexible consumption and analytics.
Use standard content first.
Build custom operational reporting when:
- business-specific KPIs are required;
- cross-space reporting is needed;
- SLA/SLO reporting needs custom logic;
- operational data must be combined with business metadata.
Avoid rebuilding standard monitoring functionality without a clear gap.
13.85 Audit versus operational monitoring
Do not confuse audit logs with runtime monitoring.
Audit answers:
Who accessed or changed something?
Operational monitoring answers:
Did the workload execute correctly?
Security forensics may require audit data, while data-pipeline incidents require execution evidence.
Use both when investigating a security-sensitive operational incident.
13.86 Security-aware troubleshooting
A troubleshooting engineer should not receive unrestricted access just because production is broken.
Design:
Operator | v Least-privilege monitoring | +--> task status +--> logs +--> runtime +--> permitted diagnostics
Escalate to privileged roles only when required.
Current SAP documentation defines different privileges for monitoring, executing, scheduling, and analyzing tasks.
13.87 Troubleshooting permissions
When a monitor is missing or a tool cannot be opened, check:
- Global role.
- Scoped role.
- Space assignment.
- Object privilege.
- Runtime privilege.
- Data Integration privilege.
- Data Builder privilege where relevant.
A permissions problem can masquerade as a monitoring problem.
13.88 Production troubleshooting decision tree
Something is wrong
|
v
Is the platform reachable?
/
no yes
| |
platform Is data moving?
incident /
no yes
| |
integration Is data correct?
/
no yes
| |
quality Is it fast?
/
no yes
| |
performance close
13.89 Troubleshooting matrix
| Symptom | First tool | Next evidence |
|---|---|---|
| Task failed | Data Integration Monitor | Task log |
| Chain failed | Task Chain monitor | Failed child |
| Data missing | Data Pipeline Analyzer | Upstream run |
| View slow | Runtime Metrics | View Analyzer |
| Remote query slow | Remote Queries | Source evidence |
| Persistence slow | Views monitor | View Analyzer |
| Agent issue | Agent monitor | Agent logs |
| Stale data | Monitor + watermark | Source freshness |
| Wrong totals | Reconciliation | Pipeline lineage |
| Permission issue | Role/privilege check | Scoped role |
13.90 Common anti-pattern — retry until green
Bad operating behavior:
failure | v retry | v failure | v retry | v green
without understanding why.
This can:
- duplicate data;
- increase source load;
- hide a deterministic defect;
- destroy evidence;
- delay escalation.
Retry is a recovery action, not a diagnostic strategy.
13.91 Common anti-pattern — monitoring only task status
A green task does not prove:
- freshness;
- completeness;
- accuracy;
- semantic correctness;
- performance;
- security.
Use layered operational controls.
13.92 Common anti-pattern — alerting on everything
Too many alerts create noise.
Instead:
critical business signal + actionable threshold + owner + runbook
13.93 Common anti-pattern — manual tribal knowledge
If only one engineer knows how to recover a pipeline, the platform has a resilience problem.
Convert knowledge into:
- runbooks;
- architecture diagrams;
- dependency maps;
- ownership records;
- automated checks.
13.94 Common anti-pattern — deleting evidence
Do not immediately clear logs or rebuild objects after failure.
Preserve:
- run ID;
- timestamps;
- logs;
- screenshots where necessary;
- source evidence;
- configuration version;
- incident timeline.
RCA quality depends on evidence quality.
13.95 Common anti-pattern — fixing the symptom
Example:
dashboard slow | v increase refresh interval
This may hide the issue rather than fix it.
Find whether the real problem is:
- remote federation;
- model complexity;
- persistence;
- data growth;
- source latency;
- concurrency.
13.96 Production monitoring architecture pattern
For a critical finance domain:
S/4HANA | v Remote / Replicated Data | v Acquisition monitoring | v Transformation | v Quality gates | v Persistence | v Analytic Model | v SAC / Excel
Across the flow:
+-------------------------------+ | Freshness + Quality + Runtime | +-------------------------------+ | v Alert / Runbook / RCA
13.97 Architect pattern — business-critical data product
For each data product define:
- owner;
- business purpose;
- source;
- refresh frequency;
- freshness SLO;
- quality rules;
- reconciliation controls;
- monitoring location;
- alert destination;
- recovery method;
- escalation path;
- security classification.
This turns monitoring into a product capability.
13.98 Architect pattern — multiple environments
Use different operational expectations for:
DEV | TEST | PROD
DEV:
- verbose diagnostics;
- manual experimentation;
- relaxed SLO.
TEST:
- representative volume;
- performance validation;
- recovery testing.
PROD:
- controlled changes;
- strict alerting;
- formal ownership;
- business SLOs;
- evidence preservation.
13.99 Architect pattern — multiple spaces
For enterprise spaces:
Domain Space | +--> Acquisition +--> Transformation +--> Semantic +--> Consumption
Monitoring should preserve ownership boundaries while allowing platform-wide operational visibility where authorized.
13.100 Architect pattern — shared platform
A central platform team should provide:
- monitoring standards;
- SLO templates;
- alerting standards;
- runbook templates;
- escalation process;
- capacity reporting;
- platform health reporting.
Domain teams should own their data products and business correctness.
13.101 Hands-on Lab 1 — Monitor a replication task
Objective
Learn to diagnose a remote-table replication execution.
Steps
- Select a non-critical test remote table.
- Run or schedule a controlled replication.
- Open the Data Integration Monitor.
- Inspect the run.
- Record start time, end time, status, and volume indicators.
- Open run details.
- Read the log from the first message.
- Record the final watermark.
- Compare source and target counts.
- Document the result.
Deliverable
Create a one-page operational record:
Object: Run: Status: Runtime: Source watermark: Target watermark: Reconciliation: Conclusion: