Skip to main content

13.2 — Data Integration Monitoring

13.7 Reading a monitor row​

A monitor row should be treated as an operational record.

Ask:

  • Which object ran?
  • Which space owns it?
  • When did it start?
  • When did it finish?
  • Was it manual or scheduled?
  • What was the run type?
  • What is the status?
  • Is there a substatus?
  • Was this the first failure?
  • Was there a retry?
  • Did the runtime change from the normal baseline?

A status without context is not enough for RCA.

13.8 Status versus root cause​

Suppose a task says:

FAILED

That is an observation, not a root cause.

Possible causes:

FAILED | +--> source unavailable +--> credentials expired +--> network interruption +--> schema change +--> invalid transformation +--> memory pressure +--> statement limit +--> authorization +--> target constraint +--> dependency failure

The next action is therefore evidence collection, not immediate retry.

13.9 Task logs​

Task logs provide detailed execution information.

Use task logs to determine:

  • which execution step failed;
  • when it failed;
  • which object was active;
  • whether the error is transient or deterministic;
  • whether the source returned an error;
  • whether the target operation failed;
  • whether a retry is safe.

Do not copy an error message into an incident and stop there.

Translate it into:

symptom affected object failure boundary likely cause evidence recovery prevention

13.10 A disciplined log-reading method​

Read logs in this order:

  1. First failure.
  2. Immediately preceding warnings.
  3. Source or target object.
  4. Error category.
  5. Runtime timestamp.
  6. Retry history.
  7. Dependency status.
  8. Configuration changes near the incident.
  9. Relevant source-system evidence.

The first visible error is not always the root cause, but the earliest causal signal is usually more valuable than the final cascade of failures.

13.11 Task-chain monitoring​

Current SAP documentation allows operators to monitor running and previous task-chain runs and inspect child-task details.

Task-chain monitoring should answer:

  • Did the chain start?
  • Which branch executed?
  • Which task failed first?
  • Which tasks were skipped?
  • Which tasks completed successfully?
  • Did a nested chain fail?
  • Can the failed portion be retried safely?

A task chain is a dependency graph, so troubleshooting must follow the graph rather than only the final status.

13.12 Retry semantics​

Current SAP documentation supports retrying failed task-chain runs. A retry can begin from the failed child or subtask rather than rerunning already successful preceding tasks.

Architectural implication:

Retry behavior is useful only when the tasks are designed to be safely repeatable.

Before retrying, ask:

  • Is the task idempotent?
  • Could partial data already exist?
  • Could a second execution duplicate records?
  • Is the source state unchanged?
  • Did the failure occur before or after a commit?
  • Are downstream consumers already reading the target?

13.13 Safe retry decision tree​

Failure | v Is cause transient? / yes no | | v v Retry Fix cause | v Is task idempotent? / yes no | | v v Retry Reconcile first | v Controlled recovery

13.14 Remote table monitoring​

Remote-table monitoring is important when Datasphere uses replication or other remote-table data-access modes.

Monitor:

  • last successful update;
  • replication status;
  • refresh frequency;
  • volume;
  • memory/disk indicators where provided;
  • execution duration;
  • failure history;
  • source connection health.

A replication task can be technically successful while the business dataset is still stale.

13.15 Remote table freshness​

Define freshness as:

current_time - source_or_delivery_watermark

Do not use task completion time alone.

Example:

Source watermark: 11:55 Datasphere completion: 12:02 Current time: 12:05

Delivery age:

10 minutes

The exact business definition depends on the source and ingestion architecture.

13.16 Monitoring views​

Persisted views can be monitored in the Data Integration Monitor.

For each important persisted view, track:

  • last successful persistence;
  • last failure;
  • runtime;
  • refresh schedule;
  • freshness;
  • row-count changes;
  • memory symptoms;
  • downstream dependencies.

A persisted view should have an owner and a recovery runbook.

13.17 Monitoring flows​

Flows are operational data movement objects.

Monitor:

  • execution status;
  • start and end time;
  • source and target;
  • record counts where available;
  • warnings;
  • runtime;
  • failed operators;
  • target-write behavior;
  • schedule;
  • repeated failures.

Do not assume a flow that completes successfully produced correct data.

13.18 Runtime Metrics for views​

Current Datasphere tooling exposes Runtime Metrics from view editors for performance analysis.

Use Runtime Metrics to investigate:

  • latency;
  • expensive execution;
  • model changes;
  • performance regressions;
  • Explain Plan generation.

Performance troubleshooting should compare a known-good baseline against the current execution.