02.1 Workspace Architecture
Phase 2 — Platform Architecture
If Control Plane vs Compute Plane explains where Databricks management and execution happen, this tutorial explains the next architectural boundary:
What exactly is a Databricks workspace, what belongs inside it, what belongs outside it, and how should an enterprise decide how many workspaces to create?
A workspace is more than a URL where you open notebooks.
It is an operational boundary for users, collaboration assets, workspace configuration, compute access, and workload execution. Persistent business data should be designed separately through cloud storage and Unity Catalog.
1. The Mental Model You Should Build First
Do not start by memorizing the Databricks UI.
Start with this hierarchy:
DATABRICKS ACCOUNT
Enterprise / tenant boundary
│
┌───────────────┼────────────────┐
│ │ │
▼ ▼ ▼
Identity Workspaces Unity Catalog
Users / Groups Dev / Prod / Governance
Service Principals Region / Domain
│
┌─────────────┼─────────────┐
▼ ▼ ▼
Objects Compute Settings
Notebooks Jobs / SQL ACLs / Policies
Git folders Clusters Secrets
Dashboards Serverless Workspace config
│ │
└──────┬──────┘
▼
Data & AI Assets
│
▼
Cloud Storage / Services
The most important distinction is:
| Boundary | Primary purpose |
|---|---|
| Account | Enterprise-wide administration and shared services |
| Workspace | Operational and collaboration boundary |
| Unity Catalog | Data and AI governance boundary |
| Compute plane | Workload execution boundary |
| Cloud storage | Durable data foundation |
Current Databricks architecture guidance explicitly treats account, workspace, data-governance, and compute-plane concerns as distinct administrative layers.
2. What Is a Workspace?
A Databricks workspace is a deployment in a cloud region that provides an environment for users to access Databricks assets and run workloads.
A workspace typically contains or provides access to:
- Notebooks
- Files
- Git folders
- Jobs
- SQL queries
- Dashboards
- Alerts
- Experiments
- Compute resources
- Workspace-scoped permissions
- Workspace settings
- Policies and configuration
- Connections to governed data
A useful definition
Workspace = collaboration + execution + workspace-scoped administration.
It is not:
- Your data warehouse
- Your cloud object store
- Your Unity Catalog metastore
- A Spark cluster
- The Databricks account
Those concepts interact with a workspace, but they are not interchangeable.
3. Account vs Workspace
This is one of the most important distinctions for enterprise architecture.
Account
The account is the top-level Databricks construct.
Account-level responsibilities can include:
- Identity management
- Users and groups
- Service principals
- Workspace lifecycle
- Unity Catalog metastore management
- Billing and usage
- Account-level policies
- Cross-workspace governance
Workspace
The workspace is where teams actually work.
Workspace-level responsibilities can include:
- Workspace permissions
- Workspace objects
- Compute configuration
- Jobs
- SQL objects
- Git folders
- Workspace settings
- Workspace-scoped policies
Think:
ACCOUNT
│
├── Identity
│ ├── Users
│ ├── Groups
│ └── Service Principals
│
├── Workspace A
│ ├── Notebooks
│ ├── Jobs
│ └── Compute
│
├── Workspace B
│ ├── Notebooks
│ ├── Jobs
│ └── Compute
│
└── Unity Catalog
├── Metastore
├── Catalogs
└── Governance
A single account can contain multiple workspaces, and a Unity Catalog metastore can be attached to multiple workspaces in the same region.
4. Workspace Is an Operational Boundary
A very useful architecture question is:
What should be isolated by workspace, and what should be isolated by Unity Catalog?
Use a workspace when you need a meaningful operational or security boundary.
Examples:
Strong isolation requirement
ACCOUNT
│
┌─────────────┴─────────────┐
▼ ▼
PROD WORKSPACE DEV WORKSPACE
│ │
│ │
Production Engineering
workloads development
Possible reasons:
- Different network boundaries
- Different cloud accounts/VPCs
- Regulatory requirements
- Different administrators
- Production vs development isolation
- Different regional requirements
- Different platform settings
- Different operational risk
Weak isolation requirement
If the requirement is simply:
"Team A should not see Team B's tables."
Do not automatically create another workspace.
A better first question is:
Can Unity Catalog catalogs, schemas, permissions, workspace bindings, and data governance provide the required isolation?
This avoids unnecessary workspace sprawl.
5. The Workspace Split Decision
A mature architecture does not create one workspace per team by default.
Instead, evaluate the following dimensions:
| Driver | May justify separate workspace? |
|---|---|
| Dev / Prod isolation | Yes |
| Regulatory boundary | Often |
| Different cloud region | Often |
| Strong network isolation | Often |
| Different cloud account | Often |
| Separate administrative ownership | Sometimes |
| Separate business unit | Sometimes |
| Different data domain | Usually start with Unity Catalog |
| Small project | Usually no |
| Different notebook folder | No |
| Different table permissions | Usually Unity Catalog |
Architect's rule
Create a new workspace because you need a new operational boundary—not simply because you need another folder.
6. Serverless vs Classic Workspace
Databricks currently distinguishes serverless workspaces and classic workspaces.
Serverless workspace
A serverless workspace is designed to provide a more managed experience, with serverless compute and default storage.
Conceptually:
User
│
▼
Databricks Account
│
▼
Serverless Workspace
│
├── Serverless Compute
├── Workspace Assets
└── Default Storage
The cloud infrastructure for the serverless experience is more managed by Databricks.
Classic workspace
A classic workspace provisions customer-managed cloud resources for workspace storage and compute.
Conceptually:
User
│
▼
Databricks Workspace
│
├── Databricks-managed services
│
└── Customer Cloud Account
├── VPC
├── Compute
├── Storage
└── Network controls
Classic workspaces remain important when you require capabilities such as:
- Custom network architecture
- Private connectivity
- On-premises connectivity
- Specific compliance requirements
- Legacy Spark/RDD workloads
- Specific language/runtime requirements
Databricks currently recommends starting with serverless where it meets the requirements and using classic when specific network, compliance, or workload constraints require it.
7. Important: Serverless Does Not Mean “No Cloud”
This is a common misunderstanding.
Serverless means the infrastructure management model changes.
It does not mean:
"There is no compute."
There is still compute.
There is still storage.
There is still networking.
There are still security controls.
The difference is largely who operates which infrastructure and where it runs.
SERVERLESS
You
│
▼
Databricks services
│
├── Managed compute
├── Managed workspace experience
└── Default storage
versus:
CLASSIC
You
│
▼
Databricks
│
▼
Your cloud account
│
├── VPC
├── Compute resources
├── Workspace storage
└── Network controls
This distinction becomes critical later when we study networking, security, cost and compliance.
8. What Actually Lives in a Workspace?
A workspace is primarily an environment for workspace assets and operational configuration.
Common workspace objects
| Object | Purpose |
|---|---|
| Notebook | Interactive development and execution |
| Git folder | Version-controlled project workspace |
| File | Python, SQL, YAML, Markdown and other small files |
| Job | Orchestration and scheduling |
| Query | Saved SQL |
| Dashboard | BI visualization |
| Alert | Threshold-based notification |
| Experiment | MLflow experiment organization |
| Library | Reusable code/dependencies |
| Compute | Execution resources |
A key architectural point:
Workspace objects are not the same thing as governed business data.
For example:
Workspace
├── Notebook
├── Job
├── Dashboard
└── Git folder
Unity Catalog
├── Catalog
│ └── Schema
│ ├── Table
│ ├── View
│ └── Volume
The workspace consumes and operates on data governed through the data platform.
9. Workspace Storage vs Business Data
This distinction prevents many architecture mistakes.
Databricks workspace architecture includes storage for workspace-related assets and system data.
Examples of workspace-oriented assets include:
- Notebooks
- SQL queries
- Dashboards
- Alerts
- Git folders
- Libraries
- Small configuration files
Business data should instead be designed using the appropriate cloud storage and Unity Catalog governance model.
Wrong mental model
Databricks Workspace
│
└── Everything
├── Customer data
├── Sales data
├── Finance data
└── Notebooks
Better mental model
WORKSPACE
┌─────────────────┐
│ Notebooks │
│ Jobs │
│ Git folders │
│ Dashboards │
│ Workspace config│
└────────┬────────┘
│
│ governed access
▼
UNITY CATALOG
│
┌────────┴────────┐
▼ ▼
Tables Volumes
│ │
└────────┬────────┘
▼
CLOUD STORAGE
Databricks explicitly separates workspace file/system data from customer data objects such as Unity Catalog tables and volumes.
10. Workspace + Unity Catalog
A workspace does not replace Unity Catalog.
Instead:
DATABRICKS ACCOUNT
│
┌─────────┴─────────┐
▼ ▼
WORKSPACE A WORKSPACE B
│ │
└─────────┬─────────┘
▼
UNITY CATALOG
│
METASTORE
│
┌──────────┼──────────┐
▼ ▼ ▼
Catalog A Catalog B Catalog C
│
Schemas
│
Tables / Views / Volumes
This is one of the strongest features of the enterprise architecture:
Workspaces can be operationally separated while data governance remains centralized.
For example:
DEV WORKSPACE ─────┐
│
TEST WORKSPACE ────┼──→ Same regional Metastore
│
PROD WORKSPACE ────┘
The actual access model must still be designed carefully. Shared metastore visibility does not mean every user automatically receives access to every table.
11. Workspace Binding and Data Isolation
Unity Catalog supports additional isolation mechanisms.
A catalog or other governed object can be bound to specific workspaces.
For example:
METASTORE
│
┌───────────┴───────────┐
▼ ▼
finance_prod sandbox
│ │
Workspace PROD Workspace DEV
This gives architects another tool:
- Workspace isolation
- Catalog isolation
- Storage isolation
- Permission isolation
These controls can be combined rather than relying on a single boundary.
12. Workspace and Compute
A workspace is where users configure and access compute, but compute is the execution plane.
For a classic compute workload:
WORKSPACE
│
Start / Configure
│
▼
CLASSIC COMPUTE
│
┌────────┴────────┐
▼ ▼
Driver Executors
│ │
└────────┬────────┘
▼
Data / Services
For serverless:
WORKSPACE
│
▼
SERVERLESS COMPUTE
│
Managed runtime
│
▼
Data / Services
This is why "workspace" and "compute" should never be treated as synonyms.
13. Workspace Administration
There are multiple administrative scopes.
Account administrator
Think:
"I manage the Databricks platform."
Typical responsibilities include:
- Account configuration
- Workspaces
- Identity
- Unity Catalog metastore management
- Account-wide settings
- Billing/usage
Workspace administrator
Think:
"I manage this particular workspace."
Typical responsibilities include:
- Workspace settings
- Workspace users/groups
- Permissions
- Compute policies
- Workspace-specific controls
Metastore administrator
Think:
"I manage centralized data governance."
Responsibilities can include:
- Metastore-level governance
- Storage configuration
- Centralized data controls
- Cross-workspace governance
This separation supports segregation of duties.
14. Workspace Naming Convention
Workspace naming becomes surprisingly important once an enterprise has dozens of workspaces.
A useful pattern is:
{organization}-{environment}-{region}-{purpose}
Examples:
acme-dev-in-central-data
acme-prod-in-central-data
acme-prod-us-east-analytics
acme-prod-eu-west-gdpr
A name should answer:
- Who owns it?
- What environment is it?
- Where is it?
- What is its purpose?
Avoid:
workspace1
workspace2
newworkspace
finalworkspace
finalworkspace2
Names should be boring, predictable and automatable.
15. Dev / Test / Prod Architecture
For a production enterprise, a common starting point is:
ACCOUNT
│
┌──────────┼──────────┐
▼ ▼ ▼
DEV TEST PROD
│ │ │
▼ ▼ ▼
Workspace Workspace Workspace
│ │ │
└──────────┼──────────┘
▼
Unity Catalog
/ Governance
Why separate environments?
- Production stability
- Change isolation
- Different permissions
- Different compute policies
- Different network requirements
- Controlled release process
- Easier rollback
- Compliance
But there is a trade-off.
More workspaces mean:
- More administration
- More automation
- More network configuration
- More monitoring
- More cost attribution complexity
- More deployment coordination
Therefore:
Workspace count should be driven by required isolation, not organizational vanity.
16. Multi-Business-Unit Architecture
Suppose an enterprise has:
- Finance
- Sales
- Supply Chain
- HR
Do not automatically create four workspaces.
A scalable pattern can be:
ACCOUNT
│
Shared Governance
│
┌────────┴────────┐
▼ ▼
DEV WORKSPACE PROD WORKSPACE
│ │
└────────┬────────┘
▼
UNITY CATALOG
│
┌───────────────┼────────────────┐
▼ ▼ ▼
finance sales supply_chain
catalog catalog catalog
Each domain can own its data while the platform team provides common governance and standards.
This pattern is often more scalable than creating a workspace for every small team.
17. Multi-Region Architecture
Region is a major workspace design driver.
Example:
ACCOUNT
│
┌──────────┴──────────┐
▼ ▼
AWS India AWS Europe
│ │
┌─────┴─────┐ ┌─────┴─────┐
▼ ▼ ▼ ▼
DEV PROD DEV PROD
│ │ │ │
└─────┬─────┘ └─────┬─────┘
▼ ▼
Regional UC Regional UC
Metastore Metastore
A common enterprise pattern is one Unity Catalog metastore per region, with cross-region data sharing mechanisms where required.
Region-specific workspaces may be required because of:
- Data residency
- Compliance
- Latency
- Local data sources
- Network architecture
- Disaster recovery
18. Workspace Sprawl — The Hidden Enterprise Problem
Creating workspaces is easy.
Managing 100 workspaces is not.
Imagine:
100 Workspaces
│
├── 100 configurations
├── 100 network relationships
├── 100 sets of permissions
├── 100 monitoring surfaces
├── 100 lifecycle states
└── 100 places for configuration drift
This is why Databricks recommends automation for larger workspace fleets.
Use infrastructure-as-code and APIs for repeatability.
Typical tools include:
- Terraform
- Databricks APIs
- CI/CD
- Declarative Automation Bundles for workload deployment
Architect's rule
If you have many workspaces and provisioning is still manual, the architecture is incomplete.
19. Workspace Architecture and Git
A production workspace should not become the only copy of business-critical code.
Use Git-based development:
Developer
│
▼
GitHub
│
├── dev branch
├── test/release process
└── production artifact
│
▼
Databricks Workspace
│
▼
Jobs / Pipelines
This separates:
- Source control
- Deployment
- Runtime execution
- Data
A workspace is therefore not your source-code repository.
20. Workspace Architecture and Security
Security should be layered.
A useful model is:
Layer 1 — Identity
│
▼
Layer 2 — Workspace access
│
▼
Layer 3 — Compute controls
│
▼
Layer 4 — Unity Catalog permissions
│
▼
Layer 5 — Storage / network controls
│
▼
Layer 6 — Audit / monitoring
No single control is sufficient.
For example:
Giving a user access to a workspace does not mean they should automatically have access to every table.
Likewise:
A user having permission to a table does not mean every compute environment should be able to reach the underlying storage.
Security is a chain.
21. Common Architecture Mistakes
Mistake 1 — One workspace per team
This often creates unnecessary operational complexity.
Better: start with catalogs, schemas and permissions for logical isolation.
Mistake 2 — One workspace for everything
This can become problematic when production, development, regulated workloads and different network boundaries need stronger separation.
Better: define explicit isolation requirements.
Mistake 3 — Treating workspace as storage
A workspace is not the business data lake.
Better: separate workspace assets from governed data.
Mistake 4 — Manual workspace provisioning
This creates configuration drift.
Better: automate with IaC.
Mistake 5 — Using workspace permissions as the only security layer
Workspace access is not equivalent to data authorization.
Better: combine identity, workspace, compute, Unity Catalog, network and storage controls.
Mistake 6 — Creating workspaces without a naming strategy
This becomes painful at scale.
Better: define naming conventions before the second or third workspace.
22. Production Reference Architecture
A mature enterprise design can look like:
ENTERPRISE
│
DATABRICKS ACCOUNT
│
┌────────────────────┼────────────────────┐
│ │ │
Identity Workspace Fleet Governance
│ │ │
Users / Groups ┌─────┼─────┐ Unity Catalog
Service Principals │ │ │ │
DEV TEST PROD │
│ │ │ │
└─────┼─────┘ │
│ │
Compute Plane │
│ │
Spark / SQL / ML │
│ │
└──────────┬──────────┘
▼
Governed Data Assets
│
┌──────────┴──────────┐
▼ ▼
Cloud Storage External Systems
S3 / ADLS / GCS SAP / SaaS / APIs
The architecture is intentionally layered.
Account
Controls the enterprise.
Workspace
Provides an operational environment.
Compute
Executes workloads.
Unity Catalog
Governs data and AI assets.
Cloud storage
Persists the data.
External systems
Provide and consume data.
23. Hands-On Architecture Exercise
Scenario
You are designing Databricks for an enterprise with:
- Development and production environments
- Finance and Sales domains
- AWS
- India and Europe regions
- Strict production isolation
- Central data governance
- GitHub-based CI/CD
Step 1 — Decide workspaces
A reasonable starting point:
India
├── dev
└── prod
Europe
├── dev
└── prod
Step 2 — Decide governance
Use regional Unity Catalog metastores.
India Metastore
├── finance
└── sales
Europe Metastore
├── finance
└── sales
Step 3 — Decide isolation
Production gets stronger controls than development.
Step 4 — Decide code management
GitHub is the source of truth.
Step 5 — Decide provisioning
Use Terraform/API automation rather than manual creation.
Step 6 — Validate
Ask:
- Why do we need four workspaces?
- Could we reduce the count?
- What data must remain regional?
- Which catalogs can be shared?
- Which teams need workspace admin privileges?
- Which compute can access production data?
- How is configuration promoted?
- How is disaster recovery handled?
If you cannot answer these questions, the architecture is not finished.
24. Interview Questions
Beginner
Q1. What is a Databricks workspace?
A workspace is an environment and operational boundary for accessing Databricks assets, collaborating on workloads, configuring workspace-scoped controls, and running compute.
Q2. Is a workspace the same as a cluster?
No. A workspace is an environment; compute is the execution resource used by workloads.
Q3. Is workspace the same as Unity Catalog?
No. Workspace is an operational boundary; Unity Catalog is the governance layer for data and AI assets.
Intermediate
Q4. Why would an enterprise create multiple workspaces?
For meaningful isolation such as SDLC, regulatory boundaries, regions, network requirements, administrative separation, or different platform needs.
Q5. Why not create one workspace per business unit?
Because workspace sprawl increases operational complexity. Logical domain isolation can often be achieved through Unity Catalog catalogs, schemas and permissions.
Q6. Can multiple workspaces use the same Unity Catalog metastore?
Yes, within supported regional/account architecture. This enables centralized governance across linked workspaces.
Architect
Q7. How would you design Dev/Test/Prod?
Use separate workspaces when strong operational, network, security or lifecycle isolation is required, while keeping governance centralized through Unity Catalog where appropriate.
Q8. How would you design multi-region Databricks?
Use regional workspaces aligned with data residency, network and latency requirements, typically with a regional metastore strategy and explicit cross-region sharing requirements.
Q9. How do you prevent workspace sprawl?
Define workspace creation criteria, standardize naming, automate provisioning, use Unity Catalog for logical isolation, and periodically review workspace utilization and ownership.
Q10. What is the difference between workspace isolation and data isolation?
Workspace isolation separates operational environments. Data isolation controls access to data assets. They are complementary rather than interchangeable.
25. Architect's Mental Model
Remember these five statements:
1. Account is the enterprise boundary.
2. Workspace is the operational boundary.
3. Compute plane is the execution boundary.
4. Unity Catalog is the data and AI governance boundary.
5. Cloud storage is the durable data foundation.
Put them together:
ACCOUNT
│
├── IDENTITY
│
├── WORKSPACES
│ │
│ └── COMPUTE
│
└── UNITY CATALOG
│
└── GOVERNED DATA
│
└── CLOUD STORAGE
If you understand this diagram, the later topics become much easier:
- Spark → executes on compute
- PySpark → programs Spark
- SQL → executes through SQL compute
- Delta Lake → manages reliable table state
- Unity Catalog → governs assets
- Jobs → orchestrate workloads
- Auto Loader → ingests files
- Streaming → processes continuous data
- Lakeflow → builds and operates pipelines
- Terraform → provisions platform infrastructure
- CI/CD → promotes code and workloads
26. What You Should Be Able to Explain After This Tutorial
You should now be able to explain:
- What a Databricks workspace is
- Account vs workspace
- Workspace vs compute
- Workspace vs Unity Catalog
- Workspace objects
- Workspace storage vs business data
- Serverless vs classic workspaces
- Why enterprises create multiple workspaces
- When not to create another workspace
- Dev/Test/Prod patterns
- Multi-business-unit patterns
- Multi-region patterns
- Workspace naming
- Workspace administration
- Workspace security
- Workspace automation
- Workspace sprawl
- Workspace + Unity Catalog architecture
Official references
- Databricks — High-level architecture
- Databricks — Workspace strategy
- Databricks — Create and manage workspaces
- Databricks — Workspace UI
- Databricks — Workspace objects
- Databricks — Unity Catalog
Next
The next architecture topic is:
02.2 Workspace Objects & Workspace Storage
We will go deeper into the workspace browser, notebooks, Git folders, workspace files, jobs, dashboards, workspace system data, default storage, and the boundary between workspace assets and governed data.
Foundation first. Architecture second. Implementation third. Optimization fourth. Enterprise design last.
That is the learning pattern we will follow throughout the Databricks course.