Module 4: Table Selection and Initial Load

This module covers selecting tables for replication and performing the initial load process. You'll learn table selection strategies, filtering options, and how to monitor the initial load.

---

1. Understanding Table Replication

1.1 Replication Lifecycle

graph LR
    A[Table Added to MT_ID] --> B[Initial Load Starts]
    B --> C[Full Table Read]
    C --> D[Data Transferred to Target]
    D --> E[Triggers Activated]
    E --> F[Delta Replication Active]
    F -->|Continuous| F
    
    style B fill:#fbbf24
    style F fill:#48bb78

Phases:

1. Selection: Choose tables for replication
2. Preparation: Create logging structures
3. Initial Load: Copy existing data (full table)
4. Trigger Activation: Enable change capture
5. Delta Replication: Continuous synchronization

---

1.2 Table Types and Replication Suitability

Table Type	Replication Fit	Considerations
Master Data (MARA, KNA1)	✅ Excellent	Low change frequency, critical for analytics
Transaction Data (VBAK, BSEG)	✅ Good	High volume, needs performance tuning
Configuration (T000, T001)	⚠️ Selective	Rarely changes, may not need real-time
Temporary (VARI, INDX)	❌ Avoid	User-specific, high churn, low value
System Tables (USR, DDLOG)	❌ Don't Replicate	Security risk, not business data

---

2. Adding Tables to Mass Transfer ID

2.1 Access Table Selection

Transaction: LTRC

1. Select MT_ID: ECC_HANA_01
2. Click tab: "Tables"
3. Click button: "Add Tables"

---

2.2 Table Selection Methods

Method 1: Individual Table Selection

Add Tables Screen:
─────────────────────────────────────
Table Name: [MARA________________]  [Add]

Selected Tables:
☑ MARA - General Material Data
  Fields: 157
  Est. Rows: 1,250,000
  Est. Size: 2.5 GB
  
[☐ Include related tables]
[Add Selected]  [Cancel]

---

Method 2: Pattern-Based Selection

Table Pattern: [V*________________]  [Search]

Matching Tables (125 found):
☐ VBAK - Sales Document Header (850K rows)
☐ VBAP - Sales Document Item (3.2M rows)
☐ VBEP - Sales Document Schedule Line (5.1M rows)
☐ VBFA - Document Flow (12M rows)
...

[Select All]  [Select Visible]  [Add Selected]

Pattern Selection Caution

Using wildcards (*) can add many tables. Review carefully before adding - some tables may be huge or unsuitable for replication.

---

Method 3: Package-Based Selection

Package: [V*________________]  [Search]

SAP Package: VELO (Sales & Distribution)
├── VBAK - Sales Header ✅
├── VBAP - Sales Items ✅
├── VBEP - Schedule Lines ✅
├── VBUK - Sales Status ✅
└── VBUP - Item Status ✅

[Add All Package Tables]

---

Method 4: Import from File/Transport

Import Source:
● File (CSV/Excel)
○ Transport Request
○ Another MT_ID

File Format:
TABLE_NAME, FILTER, TRANSFORMATION
MARA,,
MARC,"WERKS = '1000'",
KNA1,"LAND1 IN ('US','CA','MX')",

[Upload]  [Validate]  [Import]

---

3. Table Configuration Options

3.1 Basic Table Settings

Table Details Screen:

Table Name: VBAK
Description: Sales Document Header Data

Basic Settings:
├── Replication Mode: ● Full Table  ○ Delta Only
├── Initial Load: ☑ Perform Now  ☐ Skip (data exists)
├── Triggers: ☑ Activate after load  ☐ Manual activation
└── Load Priority: ● Normal  ○ High  ○ Low

Advanced Settings:
├── Package Size: [50000] rows
├── Parallel Degree: [4] threads
├── WHERE Clause: [ERDAT >= '20240101'_______]
└── Field Mapping: [Configure Transformations...]

[Save]  [Activate]  [Cancel]

---

3.2 Filtering Data During Replication

WHERE Clause Examples:

Use Case	WHERE Clause	Result
Date Range	ERDAT >= '20200101'	Only recent records
Plant Filter	WERKS IN ('1000','2000')	Specific plants only
Country Filter	LAND1 = 'US'	US customers only
Status Filter	BUKRS = '1000' AND GJAHR >= 2020	Active company/year
Complex	(VKORG = '1000' OR VKORG = '2000') AND VTWEG = '10'	Multiple conditions

Syntax Rules:

• Use ABAP OpenSQL syntax
• Single quotes for literals
• Case-sensitive for string comparisons
• Test in SE16 before applying

Performance Impact

Filters are applied during initial load AND delta replication. Complex filters may slow replication - prefer source-side filtering if possible.

---

3.3 Field Selection and Exclusion

Select Specific Fields:

Table: MARA (Material Master)
Total Fields: 157

Selected Fields (25):
☑ MATNR - Material Number
☑ MTART - Material Type
☑ MATKL - Material Group
☑ MEINS - Base UOM
☐ NTGEW - Net Weight (excluded)
☐ BRGEW - Gross Weight (excluded)
☐ GEWEI - Weight Unit (excluded)
...

[Select All]  [Deselect All]  [Select Key Fields]

Benefits of Field Exclusion:

• ✅ Reduced data volume (30-50% savings)
• ✅ Faster replication
• ✅ Lower network bandwidth
• ✅ Smaller target tables

---

4. Initial Load Process

4.1 Initial Load Configuration

Load Options:

Initial Load Configuration
──────────────────────────────────────
Load Strategy:
● Parallel Load (faster)
  Threads: [8]
  Package Size: [50000] rows
  
○ Sequential Load (safer)
  Single thread, slower but guaranteed order
  
Load Schedule:
● Immediate (now)
○ Scheduled (date/time)
  Date: [________]  Time: [____]
  
Error Handling:
☑ Stop on critical errors
☑ Continue on warnings
☑ Retry failed packages (Max: 3)

Monitoring:
☑ Send email on completion
☑ Create detailed log
Recipient: admin@company.com

[Start Load]  [Schedule]  [Cancel]

---

4.2 Initial Load Execution Flow

sequenceDiagram
    participant SLT as SLT Server
    participant Source as Source ECC
    participant Log as Logging Tables
    participant Target as Target HANA
    
    Note over SLT,Target: Phase 1: Preparation
    SLT->>Source: CREATE LOGGING TABLES
    SLT->>Target: CREATE TARGET TABLES
    SLT->>SLT: Initialize load jobs
    
    Note over SLT,Target: Phase 2: Data Load
    loop For each package
        SLT->>Source: SELECT * FROM MARA<br/>PACKAGE 50000 OFFSET n
        Source-->>SLT: Return data package
        SLT->>SLT: Apply transformations
        SLT->>Target: BULK INSERT INTO MARA
        Target-->>SLT: Confirm write
    end
    
    Note over SLT,Target: Phase 3: Finalization
    SLT->>Source: CREATE TRIGGERS
    SLT->>SLT: Update status to REPLICATING
    SLT-->>SLT: Initial load complete

---

4.3 Monitoring Initial Load Progress

Real-Time Monitoring - Transaction: LTRC

Table: MARA - General Material Data
Status: ⚙️ Loading (Initial Load)

Progress:
█████████████████░░░░░░░░░  65% Complete

Details:
├── Total Rows: 1,250,000
├── Loaded: 812,500
├── Remaining: 437,500
├── Duration: 00:15:32
├── Est. Completion: 00:23:00
├── Speed: 872 rows/sec
└── Errors: 0

Jobs Active:
├── Job 1: Processing package 17/32  ● Running
├── Job 2: Processing package 18/32  ● Running
├── Job 3: Processing package 19/32  ● Running
└── Job 4: Waiting for package      ⏸ Idle

[Pause]  [Stop]  [Refresh]  [Detailed Log]

---

4.4 Initial Load Performance Tuning

Performance Metrics:

Metric	Target	Poor	Good	Excellent
Load Speed	rows/sec	< 500	500-2000	> 2000
Network Latency	ms	> 50	10-50	< 10
CPU Usage	%	> 90	60-80	40-60
Memory Usage	%	> 85	60-80	40-60
Error Rate	%	> 1	0.1-1	< 0.1

---

Optimization Techniques:

graph TD
    A[Slow Initial Load] --> B{Bottleneck?}
    B -->|Source Read| C[Increase parallel jobs<br/>Larger packages]
    B -->|Network| D[Compress data<br/>Reduce field count]
    B -->|Target Write| E[Bulk insert<br/>Disable indexes temp]
    B -->|Transformation| F[Simplify rules<br/>Offload to target]
    
    C --> G[Monitor Impact]
    D --> G
    E --> G
    F --> G
    
    style A fill:#ef4444
    style G fill:#48bb78

Tuning Checklist:

✅ Increase Parallel Jobs:

Before: 4 jobs  →  Load time: 2 hours
After:  12 jobs →  Load time: 45 minutes

✅ Optimize Package Size:

Small tables (< 100K rows): 10,000-20,000
Medium tables (100K-1M): 50,000-100,000
Large tables (> 1M): 100,000-200,000

✅ Disable Target Indexes During Load:

-- Before initial load
ALTER TABLE SLTREPL.MARA DISABLE INDEX ALL;

-- After initial load
ALTER TABLE SLTREPL.MARA REBUILD INDEX ALL;

✅ Use Bulk Insert Mode:

Standard Insert: 500 rows/sec
Bulk Insert: 5,000 rows/sec (10x faster)

---

5. Post-Initial Load Activities

5.1 Verify Data Consistency

Row Count Validation:

-- Source (ECC)
Transaction: SE16
Table: MARA
WHERE: (applied filter if any)
Result: 1,250,000 rows

-- Target (HANA)
SELECT COUNT(*) FROM SLTREPL.MARA;
Result: 1,250,000 rows

✅ Counts match - load successful

---

Data Sampling Validation:

-- Compare sample records
-- Source: Display first 10 materials
Transaction: SE16
Table: MARA
WHERE: MATNR <= '000000000000000010'

-- Target: Query same records
SELECT * FROM SLTREPL.MARA 
WHERE MATNR <= '000000000000000010'
ORDER BY MATNR;

-- Compare key fields manually

---

Checksum Validation (Advanced):

-- Generate checksum on source (custom ABAP program)
SELECT MATNR, MTART, SUM(HASH(MATNR || MTART || MATKL))
FROM MARA
GROUP BY MATNR, MTART;

-- Generate checksum on target
SELECT MATNR, MTART, SUM(HASH(MATNR || MTART || MATKL))
FROM SLTREPL.MARA
GROUP BY MATNR, MTART;

-- Compare results

---

5.2 Trigger Activation

Automatic Activation (Default):

After successful initial load, SLT automatically:

1. Creates database triggers on source table
2. Starts delta replication
3. Updates table status to "Replicating"

Manual Activation:

Transaction: LTRC
Select: MT_ID: ECC_HANA_01
Tab: Tables
Select: MARA
Button: [Activate Triggers]

Trigger Types Created:
├── INSERT Trigger → Captures new records
├── UPDATE Trigger → Captures modifications
└── DELETE Trigger → Captures deletions

Status: ✅ Triggers Active

---

Verify Triggers in Source System:

Transaction: DB02 (Database Performance)

Database → Triggers

Trigger Name: /DMIS/LOG_MARA_I (INSERT)
Table: MARA
Type: AFTER INSERT
Action: INSERT INTO /DMIS/LOG_MARA...

Trigger Name: /DMIS/LOG_MARA_U (UPDATE)
Table: MARA
Type: AFTER UPDATE
Action: INSERT INTO /DMIS/LOG_MARA...

Trigger Name: /DMIS/LOG_MARA_D (DELETE)
Table: MARA
Type: AFTER DELETE  
Action: INSERT INTO /DMIS/DEL_MARA...

---

5.3 Monitor Initial Delta Records

Check First Delta Replications:

Transaction: LTRC
Select: MT_ID: ECC_HANA_01
Tab: Monitor → Real-Time Data Flow

Recent Changes (Last 5 minutes):
┌──────────┬────────┬──────────┬──────────┬─────────┐
│ Time     │ Table  │ Operation│ Records  │ Latency │
├──────────┼────────┼──────────┼──────────┼─────────┤
│ 14:35:22 │ MARA   │ INSERT   │ 3        │ 0.8s    │
│ 14:35:18 │ MARA   │ UPDATE   │ 1        │ 0.5s    │
│ 14:34:55 │ KNA1   │ INSERT   │ 2        │ 1.2s    │
│ 14:34:42 │ VBAK   │ UPDATE   │ 5        │ 0.9s    │
└──────────┴────────┴──────────┴──────────┴─────────┘

✅ Delta replication working correctly

---

6. Handling Large Tables

6.1 Defining "Large Tables"

Category	Row Count	Size	Initial Load Time
Small	< 100K	< 100 MB	< 10 minutes
Medium	100K-1M	100MB-1GB	10-60 minutes
Large	1M-10M	1GB-10GB	1-6 hours
Very Large	> 10M	> 10GB	> 6 hours

---

6.2 Strategies for Large Tables

Strategy 1: Partitioned Load

Table: BSEG (Accounting Document Segment)
Total Rows: 50,000,000
Strategy: Load by fiscal year

Partition 1: WHERE GJAHR = '2022'  (10M rows)
Partition 2: WHERE GJAHR = '2023'  (15M rows)
Partition 3: WHERE GJAHR = '2024'  (15M rows)
Partition 4: WHERE GJAHR = '2025'  (10M rows)

Load each partition as separate table in MT_ID,
then merge in target using HANA views.

---

Strategy 2: Scheduled Overnight Load

Large Table: BSEG
Load Time: 8 hours
Schedule: 22:00 - 06:00 (overnight)

Advantages:
✅ No impact on business hours
✅ Full network bandwidth available
✅ Source system less busy

---

Strategy 3: Initial Load from Backup

graph LR
    A[Source ECC] -->|Create Backup| B[BSEG Flat File]
    B -->|Transfer| C[SLT Server]
    C -->|Load via Bulk Insert| D[Target HANA]
    A -->|Activate Triggers| E[Delta Replication]
    E -->|After Load Complete| D
    
    style B fill:#fbbf24
    style D fill:#48bb78

Steps:

1. Extract table to flat file (CSV/PAR)
2. Transfer file to HANA server
3. Bulk load using HANA IMPORT
4. Activate SLT triggers for delta
5. Reconcile any changes during load window

---

Strategy 4: Skip Initial Load (If Data Exists)

Scenario: Target already has historical data

Configuration:
☐ Perform Initial Load
☑ Activate Triggers Only
☑ Start Delta from NOW

Use Case:
- Migrated data manually before
- Restoring after SLT downtime
- Target has recent backup

---

7. Table Selection Best Practices

7.1 Selection Criteria

Priority Matrix:

graph TD
    A[Table Candidates] --> B{Business Value?}
    B -->|High| C{Change Frequency?}
    B -->|Low| D[Deprioritize]
    
    C -->|High| E[Priority 1:<br/>Real-time critical]
    C -->|Medium| F[Priority 2:<br/>Hourly batch OK]
    C -->|Low| G[Priority 3:<br/>Daily batch OK]
    
    E --> H{Data Volume?}
    F --> H
    G --> H
    
    H -->|Manageable| I[✅ Replicate with SLT]
    H -->|Very Large| J[Consider Hybrid:<br/>SLT + Batch]
    
    style E fill:#48bb78
    style I fill:#48bb78
    style J fill:#fbbf24

---

7.2 Common Table Groups by Module

SAP Sales & Distribution (SD):

Master Data:
├── KNA1 - Customer Master General
├── KNVV - Customer Sales Data
├── MARA - Material Master
└── MARC - Plant Data for Material

Transactional Data:
├── VBAK - Sales Document Header
├── VBAP - Sales Document Item
├── VBEP - Sales Schedule Line
├── VBFA - Document Flow
└── LIKP/LIPS - Delivery Documents

SAP Finance (FI/CO):

Master Data:
├── SKA1 - G/L Account Master
├── LFA1 - Vendor Master
└── KNA1 - Customer Master

Transactional Data:
├── BKPF - Accounting Document Header
├── BSEG - Accounting Document Segment
├── BSID - Customer Open Items
├── BSIK - Vendor Open Items
└── COEP - CO Line Items

SAP Materials Management (MM):

Master Data:
├── MARA - Material Master
├── MARC - Plant Data
├── MARD - Storage Location Data
└── LFA1 - Vendor Master

Transactional Data:
├── EKKO - Purchasing Document Header
├── EKPO - Purchasing Document Item
├── MSEG - Material Document Segment
└── MKPF - Material Document Header

---

7.3 Tables to Avoid

❌ Don't Replicate:

Table	Reason
USR*	User passwords, security risk
VARI	User-specific variants, no business value
INDX	Temporary index tables, high churn
D010*	Development objects, not business data
TPARA	System parameters, security risk
DDLOG	Data dictionary log, system data
Cluster Tables	Complex structure, use table views instead
Pool Tables	Not directly replicable, use views

---

8. Error Handling During Initial Load

8.1 Common Initial Load Errors

Error 1: Table Not Found in Source

Error: Table MARA does not exist in source system
Root Cause: Typo in table name or table doesn't exist
Solution:
  1. Verify table exists: Transaction SE11
  2. Check spelling
  3. Verify in correct client

---

Error 2: Authorization Failure

Error: User SLTUSER has no read authorization for table BSEG
Root Cause: Missing S_TABU_DIS authorization
Solution:
  1. Transaction SU53 in source system
  2. Add authorization for table group
  3. Test in SE16: Display BSEG

---

Error 3: Memory Overflow

Error: STORAGE_PARAMETERS_WRONG_SET
Root Cause: Package size too large for available memory
Solution:
  1. Reduce package size: 50000 → 10000
  2. Increase SLT server memory
  3. Reduce parallel jobs

---

Error 4: Network Timeout

Error: RFC_COMMUNICATION_FAILURE - Connection timeout
Root Cause: Network instability or firewall blocking
Solution:
  1. Check network: ping source_system
  2. Verify firewall rules
  3. Increase RFC timeout: SM59 → Advanced Options

---

Error 5: Target Table Exists

Error: Table SLTREPL.MARA already exists
Root Cause: Previous load attempt or manual creation
Solution:
  Option 1: Drop and recreate
    DROP TABLE SLTREPL.MARA;
  Option 2: Truncate and load
    TRUNCATE TABLE SLTREPL.MARA;
  Option 3: Skip initial load (activate triggers only)

---

8.2 Recovery from Failed Initial Load

Restart Failed Load:

Transaction: LTRC
Select: MT_ID: ECC_HANA_01
Tab: Tables
Select: MARA (Status: ❌ Load Failed)
Button: [Restart Initial Load]

Options:
● Resume from last package
○ Restart from beginning
○ Skip and activate triggers only

[Execute]

---

Cleanup Before Retry:

-- In Target HANA
TRUNCATE TABLE SLTREPL.MARA;

-- In Source (SLT System)
-- Delete logging table records
DELETE FROM /DMIS/LOG_MARA;
DELETE FROM /DMIS/DEL_MARA;

-- Then retry initial load

---

9. Batch Loading Multiple Tables

9.1 Bulk Table Addition

Add Multiple Tables at Once:

Transaction: LTRC
MT_ID: ECC_HANA_01
Tables Tab → Add Multiple Tables

Table List (CSV Format):
TABLE_NAME,WHERE_CLAUSE,PRIORITY
MARA,,HIGH
MARC,"WERKS IN ('1000','2000')",HIGH
KNA1,,MEDIUM
VBAK,"ERDAT >= '20240101'",HIGH
VBAP,"ERDAT >= '20240101'",HIGH
LIKP,,MEDIUM
LIPS,,MEDIUM

[Upload CSV]  [Validate]  [Add All]

---

9.2 Load Sequencing

Define Load Order:

Transaction: LTRC
MT_ID: ECC_HANA_01
Tables Tab → Load Sequence

Priority Groups:
┌─────────────────────────────────────────┐
│ Priority 1 (Load First):               │
│   ├── MARA - Material Master           │
│   ├── KNA1 - Customer Master           │
│   └── LFA1 - Vendor Master             │
│                                         │
│ Priority 2 (Load Second):              │
│   ├── VBAK - Sales Header              │
│   ├── EKKO - PO Header                 │
│   └── MKPF - Material Doc Header       │
│                                         │
│ Priority 3 (Load Last):                │
│   ├── VBAP - Sales Items               │
│   ├── EKPO - PO Items                  │
│   └── MSEG - Material Doc Items        │
└─────────────────────────────────────────┘

Reason: Load master data before transactional data
        to maintain referential integrity

---

Summary

In this module, you learned:

✅ Table selection strategies and methods
✅ Filtering and field selection options
✅ Initial load configuration and execution
✅ Monitoring and optimizing load performance
✅ Post-load verification and trigger activation
✅ Handling large tables and special scenarios
✅ Best practices for table selection
✅ Error handling and recovery procedures
✅ Batch loading multiple tables

---

What's Next?

In Module 5, you'll learn:

• Understanding delta replication mechanisms
• Monitoring real-time data flow
• Handling delta errors
• Performance tuning for delta replication
• Troubleshooting common delta issues

Ready to Proceed

Ensure you have successfully completed at least one initial load and verified data in the target system before moving to Module 5.