Azure Databricks Unity Catalog Scanning

Microsoft Purview can scan Azure Databricks Unity Catalog to ingest technical metadata from the metastore, catalogs, schemas, tables, views and columns.

Scope

A Unity Catalog scan can cover the whole Unity Catalog or be scoped to selected catalogs depending on the configured scan mode and current connector capabilities.

Authentication and permissions

Purview supports several authentication approaches for this connector. The scanning identity needs the Databricks privileges required to read the objects included in scope.

Lineage

Purview can ingest supported table, view and column lineage from Unity Catalog. Lineage extraction depends on the relevant Unity Catalog system lineage tables being available and readable.

Operational considerations

• Confirm SQL warehouse and connection prerequisites where required.
• Scope very large estates appropriately.
• Ensure the scanning identity can read the required catalogs, schemas and tables.
• Ensure all workspaces and assets needed for the intended lineage graph are scanned.
• Review connector limitations before promising complete lineage or classification behaviour.