18.1 Integration Mental Model
1. Start with the workload
Do not start with the connector catalog.
Start with the workload:
Business requirement
|
Freshness
Volume
Latency
Availability
Security
|
Integration pattern
|
Connection / adapter
|
Runtime design
2. Four primary patterns
Federation
Query the source when the consumer needs data.
Best when:
- data is relatively stable,
- source query performance is acceptable,
- low-copy architecture matters,
- freshness must be immediate.
Risk:
- consumer workload becomes source workload.
Replication
Copy source data into Datasphere.
Best when:
- repeated analytics queries are expected,
- source protection matters,
- predictable performance is required,
- history or local transformation is valuable.
Risk:
- storage, freshness management and duplicate data.
ETL or transformation
Extract data, transform it and persist the result.
Best when:
- business logic is complex,
- multiple sources must be harmonized,
- transformation should be separated from source systems.
Push
An external application or ETL platform writes into an Open SQL schema.
Best when:
- the external platform owns extraction,
- a custom integration is required,
- application-driven ingestion is appropriate.
3. Decision matrix
| Requirement | Federation | Replication | ETL | Push |
|---|---|---|---|---|
| Live access | Strong | Depends on freshness | Depends | Depends |
| Source protection | Weaker | Stronger | Stronger | Stronger |
| Complex transformation | Limited | Moderate | Strong | External |
| Predictable query performance | Depends on source | Strong | Strong | Strong |
| Minimal persistence | Strong | Weaker | Weaker | Depends |
4. Anti-pattern
Source
|
"Connect everything"
|
Remote tables everywhere
|
Complex views
|
Source overloaded
A remote table is not automatically the best architecture.
5. Architect exercise
For these workloads choose the primary pattern:
- operational dashboard requiring current inventory,
- monthly finance reporting,
- 5 TB historical analytics,
- customer-service application lookup,
- cross-cloud enterprise data product.
For each decision state why the rejected alternatives are inferior.