07.8 Row-Level Security, Column Masking & ABAC
Fine-grained security begins where object-level permissions stop.
A user can have SELECT on a table and still must not be allowed to see every row or every value. Unity Catalog provides several complementary controls:
- object privileges and ownership decide whether a principal can access an object;
- row filters decide which rows are visible;
- column masks decide which values are visible;
- ABAC policies use governed tags and reusable policies to scale those controls across many assets;
- workspace bindings restrict where governed objects can be used.
Databricks currently recommends ABAC for centralized, scalable row and column controls based on governed tags, while table-level row filters and column masks remain useful for table-specific rules. Dynamic views remain useful when the security boundary is a curated or transformed view rather than simply a filter or mask.
Official references:
1. The security problem
Consider an enterprise customer table:
| customer_id | region | country | customer_name | revenue | |
|---|---|---|---|---|---|
| C001 | EMEA | DE | Alice | alice@example.com | 12000 |
| C002 | AMER | US | Bob | bob@example.com | 18000 |
| C003 | APAC | IN | Carol | carol@example.com | 9000 |
Suppose:
- the EMEA sales team may query the table;
- EMEA users may see only EMEA rows;
- analysts may see customer names;
- analysts must not see full email addresses;
- a privileged data steward may see the unmasked value.
A simple GRANT cannot express all of this.
The architecture becomes:
identity → object permission → fine-grained policy → visible data
This distinction is foundational.
2. Object access is not data visibility
A common mistake is to think:
"If the user has SELECT, the user can see the table."
More precisely:
- the user needs the required privileges to reach the object;
- row filters and masks can further restrict the result;
- ABAC can apply those restrictions dynamically according to policy scope, tags and identity;
- workspace bindings can constrain the environment from which the object is accessible.
Therefore:
GRANT is the baseline; fine-grained security is an additional restriction layer.
ABAC does not replace the base object privileges required to access a table.
3. Three major patterns
Pattern A — Table-level row filter
A row filter is attached directly to one table.
Use it when the rule is specific to a particular table and you do not need centralized reuse.
Conceptually:
CREATE FUNCTION filter_region(region STRING)
RETURNS BOOLEAN
RETURN region = current_user_region();
Then attach the function to the table:
ALTER TABLE main.sales.orders
SET ROW FILTER filter_region ON (region);
The exact function implementation depends on your identity model. The important architectural point is that the function returns a boolean for each row.
If it returns FALSE, that row is not returned.
Pattern B — Table-level column mask
A column mask changes the value returned for a protected column.
For example:
CREATE FUNCTION mask_email(email STRING)
RETURNS STRING
RETURN CASE
WHEN is_account_group_member('pii-clearance') THEN email
ELSE regexp_replace(email, '(^.).*(@.*$)', '$1***$2')
END;
The mask is attached to a column:
ALTER TABLE main.sales.customers
ALTER COLUMN email SET MASK mask_email;
The mask does not grant access. It changes what a permitted user sees.
Pattern C — ABAC
ABAC moves the security rule above the individual table.
The core model is:
governed tags + policy + scope + principal → effective restriction
For example:
- tag:
data_classification=pii - tag:
domain=customer - policy scope: catalog or schema
- mask UDF: reusable enterprise function
- target: all tagged columns
This is the scalable enterprise pattern.
4. Row-level security
Row-level security answers:
Which records can this principal see?
Typical dimensions include:
- region
- country
- business unit
- department
- legal entity
- customer segment
- sales organization
- cost center
- project
- tenant
- data classification
Example
Suppose a user belongs to EMEA.
The policy should transform:
ALL CUSTOMER ROWS
↓
REGION POLICY
↓
EMEA ROWS ONLY
The underlying table remains shared.
This is much more scalable than creating a physical table per region.
Identity-driven filtering
Enterprise designs commonly use one of these patterns:
- identity/group membership;
- identity attributes;
- a governed mapping table;
- a combination of user identity and row attributes.
For many dimensions, mapping tables can be preferable to creating a large number of groups.
Example:
| user | region | department | clearance |
|---|---|---|---|
| alice | EMEA | Finance | masked |
| bob | AMER | Sales | full |
The policy function can evaluate the user's authorization context against the row.
5. Column masking
Column masking answers a different question:
The user may access the row, but what value should they see?
Examples:
- SSN → last four digits
- email → partially redacted
- phone → partially redacted
- bank account → tokenized
- salary → hidden or bucketed
- customer name → pseudonymized
- confidential supplier price → NULL or masked value
A useful distinction:
| Control | Restricts |
|---|---|
| Row filter | rows |
| Column mask | values |
| GRANT | object access |
| Workspace binding | workspace scope |
| ABAC | centralized policy application |
6. Masking is not encryption
Masking and encryption solve different problems.
Encryption protects data at rest or in transit.
Masking controls the value exposed to a querying principal.
For example:
S3 object
↓
encrypted storage
↓
Delta table
↓
Unity Catalog authorization
↓
column mask
↓
user sees redacted value
A masked result is therefore an authorization presentation control, not a replacement for encryption.
7. ABAC mental model
ABAC stands for Attribute-Based Access Control.
In Unity Catalog, governed tags describe attributes of governed objects and policies evaluate those attributes.
A simplified model is:
User / Group
+
Governed Tags
+
Policy Conditions
+
Policy Scope
↓
Effective Security Policy
↓
Row Filter / Column Mask
↓
Query Result
ABAC supports row-filter and column-mask policies and also has newer policy types such as GRANT policies and DENY policies in beta. This course focuses primarily on data visibility because it is the core row/column security problem.
8. Governed tags
Governed tags are an important enterprise design primitive.
Examples:
data_classification = pii
data_classification = confidential
domain = finance
domain = procurement
region = eu
A policy can target objects based on those tags rather than requiring every table owner to manually attach the same rule.
This creates a powerful pattern:
classify once → govern consistently
Tag taxonomy matters
Do not create uncontrolled tag sprawl.
A good taxonomy usually separates:
- classification;
- domain;
- regulatory sensitivity;
- geography;
- ownership;
- retention;
- policy applicability.
Avoid putting actual secrets, credentials or sensitive personal data into tag names or values. Tags are metadata, not a secure secret store.
9. ABAC policy scope
A policy can be attached at an appropriate level in the Unity Catalog hierarchy.
Conceptually:
Metastore
└── Catalog
└── Schema
└── Table
└── Column
A catalog/schema-scoped policy is powerful because it can automatically cover matching assets.
This is one of the biggest differences from table-by-table configuration.
Enterprise principle
Use the narrowest policy scope that provides the required consistency.
Too narrow:
- policy repeated on hundreds of tables;
- difficult lifecycle management.
Too broad:
- accidental coverage of unrelated data;
- harder exception handling.
10. ABAC policy evaluation
At query time, the effective policy depends on several inputs:
Querying identity
+
Group membership
+
Governed tags
+
Policy scope
+
Policy conditions
↓
Policy match
↓
Effective filter / mask
↓
Databricks Runtime enforcement
The same SQL query can therefore return different results for different users.
This is not a bug.
It is the intended security model.
11. Policy conflicts
Enterprise security becomes complicated when multiple policies can affect the same asset.
Examples:
- one policy filters by region;
- another filters by department;
- one policy masks PII;
- another policy applies a different mask.
Do not design policies as if only one rule will ever exist.
Define:
- policy ownership;
- scope;
- precedence/conflict behavior;
- exception process;
- testing;
- auditability;
- change management.
Treat security policy like production code.
12. ABAC vs table-level controls vs dynamic views
| Requirement | Preferred pattern |
|---|---|
| One table, unique rule | Table-level row filter/mask |
| Same rule across many tagged tables | ABAC |
| Security + transformed/curated dataset | Dynamic view |
| Baseline object access | GRANT / ownership |
| Workspace isolation | Workspace bindings |
| Central enterprise policy | ABAC |
| Legacy/simple implementation | Dynamic view or table-level control |
Databricks recommends ABAC when consistent row filtering or column masking must scale across many assets.
Dynamic views remain valuable when the security boundary is itself a curated data product.
13. Dynamic views
A dynamic view can combine multiple operations:
Base tables
↓
JOIN
↓
FILTER
↓
MASK
↓
RESHAPE
↓
Secure view
This is useful when the consumer should not directly access the underlying table.
For example:
- finance analyst gets a curated financial view;
- customer support gets a redacted customer view;
- external reporting gets only approved attributes.
However, dynamic views can become difficult to govern if hundreds of nearly identical views are created.
Use them deliberately.
14. ABAC vs RBAC
RBAC answers:
Which role/group gets access?
ABAC answers:
Given the object's attributes and the policy, what restriction applies?
Enterprise security normally combines them.
Example:
RBAC
Finance Analysts
↓
SELECT on finance catalog
↓
ABAC
PII tag → mask
EU tag → regional filter
↓
Effective dataset
RBAC provides coarse-grained authorization.
ABAC provides scalable fine-grained policy.
15. SAP / BW authorization mapping
This is particularly important when migrating SAP analytics workloads.
SAP/BW mental model
Traditional SAP authorization may involve:
- user roles;
- authorization objects;
- company code;
- controlling area;
- purchasing organization;
- purchasing group;
- plant;
- sales organization.
A Databricks lakehouse can model these as:
identity + business attributes + data tags + policies
Example:
| SAP concept | Lakehouse security concept |
|---|---|
| Company code | row-level attribute |
| Plant | row-level attribute |
| Purchasing org | row-level attribute |
| PII field | governed tag |
| Sensitive financial field | governed tag |
| SAP role | group / principal |
| Authorization object | policy condition |
The migration should not blindly reproduce every SAP role as a Databricks group.
Instead, identify the actual authorization dimensions and model them explicitly.
16. Multi-dimensional authorization
Suppose an enterprise has:
- 8 regions;
- 6 departments;
- 3 clearance levels.
A naïve group model can create a combinatorial explosion.
Instead:
User
↓
Identity attributes
├── region
├── department
└── clearance
↓
Mapping / policy logic
↓
Row filter + column mask
This separates identity management from data policy.
That is an important architect-level design principle.
17. Production architecture
A scalable enterprise pattern looks like this:
Identity Provider
|
Users / Groups
|
v
Unity Catalog
|
+--------------+--------------+
| | |
Object Grants Governed Tags Workspace
| | Binding
+--------------+--------------+
|
ABAC Policies
|
+----------+----------+
| |
Row Filter Column Mask
| |
+----------+----------+
|
Delta Tables
|
+------------+------------+
| | |
BI SQL PySpark
The critical architectural boundary is:
data consumers never receive more data than the policy permits.
18. Performance considerations
Fine-grained security is a correctness feature, but it also participates in query execution.
Important considerations:
- keep policy UDFs simple;
- avoid unnecessarily expensive functions;
- minimize repeated lookups;
- design selective row predicates;
- avoid accidental broad scans;
- test joins and aggregations under the security policy;
- benchmark both privileged and restricted identities.
Security can change the optimizer's freedom because the engine must protect against information leakage.
Therefore:
never benchmark a secure workload only with an unrestricted admin identity.
19. Runtime requirements
Current ABAC row-filter and column-mask capabilities have runtime requirements.
Databricks documents support for:
- serverless compute;
- standard compute on Databricks Runtime 16.4 or later;
- dedicated compute on Databricks Runtime 16.4 or later with the required fine-grained filtering capability.
Older runtimes require careful migration planning.
Always validate the exact runtime and feature compatibility before production rollout.
20. Hands-on Lab 1 — Table-level row filter
Goal
Create a small sales table and expose only the user's authorized region.
Step 1 — Create data
CREATE SCHEMA IF NOT EXISTS main.security_lab;
CREATE OR REPLACE TABLE main.security_lab.sales (
order_id BIGINT,
region STRING,
amount DECIMAL(18,2)
)
USING DELTA;
Step 2 — Add sample rows
INSERT INTO main.security_lab.sales VALUES
(1, 'EMEA', 1000),
(2, 'AMER', 2000),
(3, 'APAC', 3000);
Step 3 — Create the filtering function
Use an identity or group-based function appropriate to your environment.
The learning objective is to ensure that the function returns TRUE only for rows the querying identity is allowed to see.
Step 4 — Attach the row filter
Use ALTER TABLE ... SET ROW FILTER.
Step 5 — Test
Run the same SELECT under:
- EMEA user;
- AMER user;
- security administrator.
Record the expected and actual row sets.
21. Hands-on Lab 2 — Column masking
Create a customer table:
CREATE OR REPLACE TABLE main.security_lab.customers (
customer_id STRING,
customer_name STRING,
email STRING,
phone STRING
)
USING DELTA;
Create a mask function that exposes the full value only to an approved security group.
Then attach the mask to email.
Validate:
- privileged user → full email;
- normal analyst → masked email;
- user without table access → no result at all.
The third case is important:
a mask is not a substitute for SELECT permission.
22. Hands-on Lab 3 — ABAC with governed tags
Design the following policy:
tag: data_classification = pii
↓
ABAC column-mask policy
↓
mask matching columns
Then:
- create the governed tag;
- apply it to an email column;
- create a reusable masking UDF;
- create the policy at schema scope;
- create another table with a similarly tagged email column;
- verify that the same policy applies without manually attaching a second mask.
This demonstrates why ABAC scales better than table-by-table configuration.
23. Hands-on Lab 4 — Region + PII
Build a table containing:
- region;
- department;
- customer;
- email;
- revenue.
Implement:
- row filtering by region;
- masking of email;
- unrestricted access for an approved data steward.
Test at least three identities.
Create a test matrix:
| Identity | Region | Expected | |
|---|---|---|---|
| EMEA analyst | EMEA rows | masked | pass |
| AMER analyst | AMER rows | masked | pass |
| Data steward | all rows | clear | pass |
24. Hands-on Lab 5 — Policy impact analysis
Take an existing Gold table and answer:
- Which columns contain PII?
- Which users should see them?
- Which rows are region-restricted?
- Should the policy be table-specific or catalog/schema-wide?
- Should a dynamic view be used?
- How will the policy be tested?
- How will policy changes be audited?
Document the answer as an architecture decision record.
25. Troubleshooting playbook
Symptom: user sees no rows
Check:
- SELECT privilege;
- USE CATALOG;
- USE SCHEMA;
- workspace binding;
- row-filter policy;
- identity/group membership;
- mapping-table values;
- policy scope.
Symptom: column is always masked
Check:
- mask policy;
- governed tag;
- group membership;
- identity attributes;
- policy exceptions;
- UDF logic.
Symptom: ABAC policy does not apply
Check:
- governed tag exists;
- tag is assigned correctly;
- policy scope covers the object;
- policy principal matches;
- runtime meets requirements;
- UDF privileges are correct;
- policy conditions actually match.
Symptom: query became slower
Compare:
- unrestricted identity;
- restricted identity;
- query plan;
- predicate selectivity;
- UDF complexity;
- joins introduced by mapping logic;
- amount of data scanned.
26. Security testing strategy
Treat fine-grained authorization as a testable product.
Create a matrix:
| Test | Expected |
|---|---|
| analyst queries allowed region | rows returned |
| analyst queries forbidden region | rows absent |
| analyst selects PII | masked |
| steward selects PII | clear |
| unauthorized user accesses table | denied |
| new tagged table appears | policy automatically applies |
| tag removed | policy behavior matches design |
| group membership changes | effective access changes |
Automate these tests where possible.
27. Common anti-patterns
Anti-pattern 1 — Security only in the BI layer
If Tableau, Power BI or another BI tool is the only place filtering happens, another consumer can bypass it.
Security should exist at the governed data layer.
Anti-pattern 2 — One group for every combination
This produces group explosion.
Prefer identity attributes and policy/mapping logic where appropriate.
Anti-pattern 3 — Copying tables for every security audience
This creates duplication, synchronization and governance problems.
Use shared governed tables when policy-based access is sufficient.
Anti-pattern 4 — Hard-coded usernames in SQL
This does not scale.
Use identity/group/attribute-driven controls.
Anti-pattern 5 — Treating masking as encryption
Masking controls visibility. Encryption protects storage/transmission.
Anti-pattern 6 — No negative testing
A security policy that is tested only by an admin is not adequately tested.
Anti-pattern 7 — Overly broad policy scope
A single global policy can become difficult to reason about.
Use deliberate domains and documented exceptions.
28. Architect decision framework
Ask these questions in order:
Question 1
Does the user have baseline object access?
If no → GRANT/ownership/workspace problem.
Question 2
Do rows need filtering?
If yes → row filter.
Question 3
Do values need masking?
If yes → column mask.
Question 4
Is the rule repeated across many assets?
If yes → ABAC.
Question 5
Does the consumer need a curated or reshaped dataset?
If yes → consider a dynamic view.
Question 6
Does workspace location matter?
If yes → workspace binding.
This produces a layered security architecture rather than one oversized mechanism.
29. Enterprise design checklist
Before production:
- Object privileges are defined separately from data restrictions.
- Groups/service principals are used instead of individual grants where appropriate.
- Governed tag taxonomy is documented.
- Sensitive columns are classified.
- Row-filter dimensions are documented.
- Masking rules are documented.
- Policy ownership is assigned.
- Policy scope is justified.
- Exceptions are documented.
- Negative authorization tests exist.
- Runtime compatibility is verified.
- Performance is benchmarked.
- Audit and change processes exist.
- Workspace bindings are aligned with the security model.
- BI-layer security is not the only enforcement point.
30. Interview questions
Beginner
What is row-level security?
A control that restricts which records a permitted user can see.
What is column masking?
A control that changes the value returned for protected columns.
Does masking grant table access?
No. Base object privileges are still required.
Intermediate
ABAC vs RBAC?
RBAC primarily maps roles/groups to privileges; ABAC uses attributes and policies to apply dynamic, scalable restrictions.
When would you use a dynamic view?
When the consumer needs a curated, filtered, masked or reshaped representation rather than direct access to the base table.
Why use governed tags?
They provide a consistent metadata signal that policies can target across many assets.
Architect
How would you secure 5,000 tables with common PII rules?
Use a governed tag taxonomy plus centralized ABAC policies rather than attaching thousands of independent masks.
How would you model SAP authorizations?
Extract the actual business authorization dimensions such as company code, plant or purchasing organization and model them as explicit data/identity attributes instead of blindly recreating every SAP role.
How do you prevent policy bypass?
Enforce security in the governed data platform, control direct storage access, use Unity Catalog privileges and workspace bindings, and test every consumer path.
31. Final mental model
Remember the layers:
1. Identity
↓
2. Workspace boundary
↓
3. Object privilege
↓
4. Governed metadata / tags
↓
5. Fine-grained policy
↓
6. Row filter / column mask
↓
7. Query execution
↓
8. Governed result
The architect's rule is:
Grant access broadly enough for the business use case, then enforce data visibility centrally and consistently at the governed data layer.
For small, table-specific requirements, table-level row filters and masks are appropriate.
For enterprise-wide, repeatable policy, use ABAC.
For curated data products, use dynamic views where they provide a clearer contract.
That is the difference between implementing a filter and designing an enterprise data-security architecture.