Skip to content
woodfrog
Data Foundations
Teams on Databricks still using the Hive metastoreUse case on Databricks, no client result claimed

Upgrade to Unity Catalog without breaking a single job.

Your Databricks workspaces still run on the legacy Hive metastore. Access is set per workspace, lineage is missing and new features need Unity Catalog. We upgrade schema by schema, retire what nobody uses and repoint every job, with tests at each step.

Sample deliverable, illustrative data
Hive metastore to Unity Catalog4 of 5 schemas
hive_metastore.salesmain.salesUpgraded
hive_metastore.financemain.financeUpgraded
hive_metastore.hrrestricted.hrUpgraded
hive_metastore.ops_tmpRetired, unused 180 daysRetired
hive_metastore.marketingmain.marketingChecking
Grants rewritten by roleLineage onJobs repointed and tested
Hive metastore in every workspaceUnity Catalog, one governed catalogGrants, lineage and audit in one place
RegisteredDatabricks partner, in Pune
CertifiedDatabricks practitioners on the team
50+Data projects delivered by the team
Week 1Fixed-fee audit
TypeUse case: what we build on Databricks, not a client story
ForTeams on Databricks whose tables, grants and jobs still sit in workspace-level Hive metastores
EngagementWeek 1 audit, then build, fixed scope, fixed fee
Runs onDatabricks on your cloud, with Unity Catalog and Databricks' own upgrade tooling
Before and after

From the old way to the new.

Before
  • A Hive metastore per workspace, with access set separately in each
  • No lineage from a dashboard back to its source table
  • Table ACLs and mount points nobody fully understands
  • New Databricks features that need Unity Catalog, out of reach
After
  • One catalog across workspaces, with access set once by role
  • Lineage from every report back to the source
  • Storage credentials and external locations replacing mounts
  • Serverless, AI/BI and other Unity Catalog features available
The problem

The upgrade everyone postpones, because jobs might break.

Unity Catalog is where Databricks is heading, but the upgrade touches every table, grant, mount and job. Teams put it off because nobody can say which job reads which table, and a missed dependency breaks a report on Monday morning.

Servers in a data centre
Every workspace has its own metastore, and nobody has the full picture
What was broken
  • Grants are scattered across workspaces and table ACLs.
  • Jobs read tables through mount points that will not carry over.
  • Nobody knows which old tables are still read.
  • External tables and managed tables need different handling.
  • The upgrade plan keeps growing as hidden dependencies surface.
You might be here if
  • You have several workspaces with different access rules.
  • Auditors ask who can see what, and the answer takes weeks.
  • You want serverless or AI/BI, and they need Unity Catalog.
  • The upgrade has been on the roadmap for more than a quarter.

Why it mattered. Every month on the Hive metastore keeps access scattered and keeps newer Databricks features out of reach.

The approach

Assess, upgrade schema by schema, then repoint and test.

We start with an assessment of every table, grant, mount and job, using Databricks' own upgrade tooling. Unused tables are retired. The rest move schema by schema, with grants rewritten by role and every job repointed and tested before the old path is removed.

Where it is today
Hive metastore per workspace
Mounts and table ACLs
Jobs on old paths
woodfrog method
Assess everythingTables, grants, mounts, jobs
Upgrade by schemaGrants rewritten by role
Repoint and testThen remove old paths
What changes
One governed catalog
Lineage switched on
Jobs on new paths
Unity Catalog, fully in use
Deliberately not doneA big-bang switch over a weekend. Schemas move one at a time, each tested before the next.
Every grant is written down by role, so access stays explainable after we leave.schematic
The solution

One catalog, every job tested.

What we build
  • An assessment of every table, view, grant, mount and job, with each marked upgrade, retire or rework.
  • Unity Catalog metastore, catalogs and schemas laid out by domain and environment.
  • Storage credentials and external locations replacing mount points.
  • Grants rewritten by role at catalog, schema, table and column level.
  • Every job and notebook repointed to Unity Catalog tables and tested.
  • Lineage and audit logs switched on, with a short guide for your team.
It uses Databricks' own upgrade tooling where it fits and careful manual work where it does not. Nothing is removed from the Hive metastore until the job that read it is tested on the new path.
How it runs
  1. Week 1AuditData profiled, scope and fee fixed
  2. ThenBuildOne area at a time, tested with your users
  3. OngoingOperateQuarterly reviews of quality, access and spend
The impact

What we would measure.

What we would measureNo client values yet
Schemas upgradedCounted against the assessment
Tables retiredUnused, removed rather than moved
Jobs repointed and passingTested before old paths go
Time to answer who can see a tableBefore and after
Grants defined by roleShare of all grants
Plan your Unity Catalog upgrade.A short call, or record a message. A person replies within one business day.
Contact usCertified Services Partner, Claude Partner Network