What CoIsland watches in Databricks

Each monitor kind brings its own query and its own card in the notch.

How to monitor Databricks from your Mac’s notch

Databricks emails a failed job to whoever set up its notifications, if anyone did. A CoIsland monitor checks on its own schedule and opens the notch only when something new fails: a job run, a pipeline update, an alert that triggers, or a row your SQL finds.

The job, pipeline and alert kinds read your workspace’s REST API, so they cost nothing and can check every few minutes. The SQL kinds run on a warehouse you pick, and the editor says what that costs before you save.

Each monitor is a file in ~/.coisland/watches:

-- name: Failed nightly ETL
-- kind: databricks.job-runs
-- connector: dbc-a1b2c3d4
-- every: 5m
-- alert: new-rows

job:123456 status:failed,timedout lookback:24h

Connect Databricks covers the token, every field and each kind. The Snowflake connector works the same way for SQL; all connectors share one page.

Connect it in three steps

  1. In your workspace, Settings › Developer › Access tokens: generate a personal access token.

  2. Add a Databricks connector with the workspace URL, the token and, for SQL, a warehouse. Test connector lists the warehouses you can see.

  3. Pick a monitor kind or an example: jobs, pipelines and alerts check every 5 minutes, SQL every 30 unless you change it.

Built for Databricks

  • Jobs, pipelines and alerts are read from the REST API: they never wake a warehouse.
  • SQL is read-only by design: only SELECT, WITH or SHOW ever runs.
  • The editor says what a SQL check costs on the chosen warehouse.
  • The token stays in the macOS Keychain and goes only to your workspace.

Then hand it to an agent

A click on the alert opens the agent in a terminal under it, with the whole alert as its first message.

Databricks questions

How do I get an alert when a Databricks job fails?

Add a Failed job runs monitor, pick the job or leave it empty for every job you can view. Each run that ends failed or timed out reaches the notch once, and its alert page shows the failed task's error and stack trace. It reads the Jobs API and costs nothing.

Does CoIsland keep my SQL warehouse running?

Only the SQL monitors use a warehouse, and each check wakes it if it is stopped; it then stays on until its auto-stop. The editor says how much of the time that keeps it running. A serverless 2X-Small with a short auto-stop, checked every 30 minutes, keeps it cheap.

What if my admin turned off personal access tokens?

Test connector says so. Ask a workspace admin to allow them for you (Settings › Advanced › Personal Access Tokens). Signing in with the Databricks CLI instead is planned.

Can CoIsland change anything in Databricks?

No. It reads job runs, pipeline updates and alerts, and runs only SELECT, WITH or SHOW on a warehouse.

Other connectors

Put your work stack in the notch

Regular price: $59.99 Early bird price: $19.99 once, for 2 Macs. Native for Apple Silicon, macOS 14 and later.

Early bird price until October 25