20.7 Testing, Validation & Reconciliation
Datasphere testing must cover metadata, data, security and runtime behavior.
Test pyramid
Static -> Unit/model -> Integration -> Data reconciliation -> Performance -> UAT
Metadata tests
Verify:
- object exists
- expected type
- dependency graph
- required columns
- semantic annotations
- security artifacts
Data tests
At minimum consider:
- row counts
- key uniqueness
- null thresholds
- domain validity
- freshness
- source-to-target totals
- control totals by business dimension
Reconciliation pattern
For a metric (M):
Variance = Target(M) - Source(M)
Use both absolute and relative thresholds:
Relative variance = abs(Target-Source) / max(abs(Source), epsilon)
Performance tests
Measure representative workloads, not only a successful deployment.
Record:
- execution time
- data volume
- concurrency
- memory pressure where available
- query characteristics
Release gate
Do not promote when critical reconciliation or security tests fail.
Architect principle
A deployment test answers can it run?
A reconciliation test answers did it produce the right result?
You need both.