Using Circle CI for Databricks

I have to use circleCI to deploy jobs in Databricks. So far I have scoured the internet trying to find a piece of information about it, Couldn’t find any. Can anyone help me with this please!?

Hi @dsuryavamshi, and welcome to the CircleCI community!

I would suggest including calls to the relevant Databricks API endpoints in your CircleCI steps; the Databricks authentication token being stored as either a project or context environment variable.

Alternatively, you could opt to use the Databricks CLI, and store the required authentication information in separate environment variables:

  • DATABRICKS_HOST
  • DATABRICKS_USERNAME
  • DATABRICKS_PASSWORD
  • DATABRICKS_TOKEN