Set up a Databricks connector

Capabilities

Resource	Sync	Provision
Accounts
Groups
Roles
Service principals
Workspaces

The Databricks connector supports automatic account provisioning and deprovisioning.

Gather Databricks credentials

Configuring the connector requires you to pass in credentials generated in Databricks. Gather these credentials before you move on.

A user with the Account admin role in each Databricks workspace you want to sync must perform this task.

Look up your Databricks account ID

Ensure that your Databricks user account has the Account admin role and is assigned to each Databricks workspace you want to sync to ConductorOne.

In the Databricks account console, open the menu that appears next to your username in the upper right corner.

Carefully and copy and save account ID.

Generate Databricks credentials

You have three authentication choices when setting up the Databricks connector:

OAuth (syncs info from all Databricks workspaces)
1
Follow the Databricks OAuth authentication documentation to create a service principal and create an OAuth secret.
2
Carefully copy and save the OAuth client ID and secret.
Personal access token (syncs info from a single Databricks workspace)
1
In Databricks, navigate to Settings > Developer > Access tokens and click Manage.
2
Click Generate new token and create a new token.
3
Carefully copy and save the token.
Username and password (syncs info from all Databricks workspaces) You do not need to generate any additional credentials to use this method.

That’s it! Here’s the set of credentials you’ll need when setting up the connector:

Account ID
OAuth client ID
OAuth client secret

Account ID
Personal access token
Workspace ID for the Databricks workspace you’re syncing

Account ID
Username
Password

Next, move on to the instructions for your chosen setup method.

Configure the Databricks connector

To complete this task, you’ll need:

The Connector Administrator or Super Administrator role in ConductorOne
Access to the set of Databricks credentials generated by following the instructions above

Cloud-hosted
Self-hosted

Follow these instructions to use a built-in, no-code connector hosted by ConductorOne.

In ConductorOne, navigate to Admin > Connectors and click Add connector.

Search for Databricks and click Add.

Choose how to set up the new Databricks connector:

Add the connector to a currently unmanaged app (select from the list of apps that were discovered in your identity, SSO, or federation provider that aren’t yet managed with ConductorOne)
Add the connector to a managed app (select from the list of existing managed apps)
Create a new managed app

Set the owner for this connector. You can manage the connector yourself, or choose someone else from the list of ConductorOne users. Setting multiple owners is allowed.If you choose someone else, ConductorOne will notify the new connector owner by email that their help is needed to complete the setup process.

Click Next.

Find the Settings area of the page and click Edit.

Select whether you’re authenticating with OAuth, a Personal access token, or your Username and password.

Paste the account ID you looked up in Step 1 into the Account ID field.

Enter the required OAuth, token, or username and password credentials into the other two fields.

Google Cloud Platform and Azure Databricks customers only: Enter your Databricks account hostname and hostname in the relevant fields.

For more on how to look up these values for GCP, see the GCP Databricks REST API reference and identifiers for workspace objects documentation.
For more on how to look up these values for Azure Databricks, see the the Azure Databricks REST API reference and identifiers for workspace objects documentation.

Click Save.

The connector’s label changes to Syncing, followed by Connected. You can view the logs to ensure that information is syncing.

That’s it! Your Databricks connector is now pulling access data into ConductorOne.

Follow these instructions to use the Databricks connector, hosted and run in your own environment.When running in service mode on Kubernetes, a self-hosted connector maintains an ongoing connection with ConductorOne, automatically syncing and uploading data at regular intervals. This data is immediately available in the ConductorOne UI for access reviews and access requests.

Step 1: Set up a new Databricks connector

In ConductorOne, navigate to Connectors > Add connector.

Search for Baton and click Add.

Choose how to set up the new Databricks connector:

Add the connector to a currently unmanaged app (select from the list of apps that were discovered in your identity, SSO, or federation provider that aren’t yet managed with ConductorOne)
Add the connector to a managed app (select from the list of existing managed apps)
Create a new managed app

Click Next.

In the Settings area of the page, click Edit.

Click Rotate to generate a new Client ID and Secret.Carefully copy and save these credentials. We’ll use them in Step 2.

Step 2: Create Kubernetes configuration files

Create two Kubernetes manifest files for your Databricks connector deployment:

Secrets configuration

# baton-databricks-secrets.yaml
apiVersion: v1
kind: Secret
metadata:
  name: baton-databricks-secrets
type: Opaque
stringData:
  # ConductorOne credentials
  BATON_CLIENT_ID: <ConductorOne client ID>
  BATON_CLIENT_SECRET: <ConductorOne client secret>
  
  # Databricks credentials, option 1
  BATON_ACCOUNT_ID: <Databricks account ID>
  BATON_DATABRICKS_CLIENT_ID: <OAuth client ID>
  BATON_DATABRICKS_CLIENT_SECRET: <OAuth client secret>

  # Databricks credentials, option 2
  BATON_ACCOUNT_ID: <Databricks account ID>
  BATON_WORKSPACE_TOKENS: <Personal access token>
  BATON_WORKSPACES: <Workspace ID for the Databricks workspace you're syncing>

  # Databricks credentials, option 3
  BATON_ACCOUNT_ID: <Databricks account ID>
  BATON_USERNAME: <Username for the Databricks account>
  BATON_PASSWORD: <Password for the Databricks account>

  # Optional: include if you want ConductorOne to provision access using this connector
  BATON_PROVISIONING: true

See the connector’s README or run --help to see all available configuration flags and environment variables.

Deployment configuration

# baton-databricks.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: baton-databricks
  labels:
    app: baton-databricks
spec:
  selector:
    matchLabels:
      app: baton-databricks
  template:
    metadata:
      labels:
        app: baton-databricks
        baton: true
        baton-app: databricks
    spec:
      containers:
      - name: baton-databricks
        image: ghcr.io/conductorone/baton-databricks:latest
        imagePullPolicy: IfNotPresent
        env:
        - name: BATON_HOST_ID
          value: baton-databricks
        envFrom:
        - secretRef:
            name: baton-databricks-secrets

Step 3: Deploy the connector

Create a namespace in which to run ConductorOne connectors (if desired), then apply the secret config and deployment config files.

Check that the connector data uploaded correctly. In ConductorOne, click Applications. On the Managed apps tab, locate and click the name of the application you added the Databricks connector to. Databricks data should be found on the Entitlements and Accounts tabs.

That’s it! Your Databricks connector is now pulling access data into ConductorOne.

Welcome

Build connectors

Configure connectors

Deploy connectors

Upload connector data

Connector library

Set up a Databricks connector

Capabilities

Gather Databricks credentials

Look up your Databricks account ID

Generate Databricks credentials

Configure the Databricks connector

Step 1: Set up a new Databricks connector

Step 2: Create Kubernetes configuration files

Secrets configuration

Deployment configuration

Step 3: Deploy the connector

Welcome

Build connectors

Configure connectors

Deploy connectors

Upload connector data

Connector library

​Capabilities

​Gather Databricks credentials

​Look up your Databricks account ID

​Generate Databricks credentials

​Configure the Databricks connector

​Step 1: Set up a new Databricks connector

​Step 2: Create Kubernetes configuration files

​Secrets configuration

​Deployment configuration

​Step 3: Deploy the connector

Capabilities

Gather Databricks credentials

Look up your Databricks account ID

Generate Databricks credentials

Configure the Databricks connector

Step 1: Set up a new Databricks connector

Step 2: Create Kubernetes configuration files

Secrets configuration

Deployment configuration

Step 3: Deploy the connector