Langfuse LLM errors, cost and evals in your notch
Your LLM app fails, overspends or gets a thumbs down, and the notch says so. Langfuse's errors, cost, latency and scores reach your Mac, read with a project's key pair on Langfuse Cloud or your own Langfuse.
What CoIsland watches in Langfuse
Each monitor kind brings its own query and its own card in the notch.
-
LLM errors
A generation, span or tool call logged with level ERROR, narrowed by model, name, trace or environment.
-
Cost over a limit
What the project, or each model or trace, cost today or over the last hours goes above a number of dollars.
-
Slow LLM calls
The p95 latency of generations over a window goes above a number of seconds.
-
Eval score below a limit
A numeric score from an evaluator, the API or an annotation comes in under a number.
-
Negative user feedback
A user's feedback score comes in at a value you call negative, with their comment.
How to watch Langfuse from your Mac’s notch
Each monitor is a ring in the notch that counts down to its next check. When a generation fails, the notch opens with its sound and the card reads checkout-assistant › answer · Error · gpt-4o · 2.5s · $1.50 under the error’s message. A click opens the alert in CoIsland with the status message, the tokens, the cost and the user, and Open in Langfuse goes to the trace, on that observation. Connect Langfuse covers every field and kind.
Connect it in three steps
-
Create an API key pair in your Langfuse project's settings (an Owner or Admin can).
-
Add a Langfuse connector with your region, or your own address, and both keys. Test connector names the project.
-
Pick a monitor kind or an example and choose when it alerts.
Built for Langfuse
- All four Langfuse Cloud regions (EU, US, Japan, HIPAA) and self-hosted Langfuse.
- Read only: CoIsland sends GET requests only, though the keys could also write traces.
- Errors and scores alert once each; cost and latency alert as they cross the limit, and again after they came back.
- The secret key stays in the macOS Keychain and goes only to your Langfuse.
Then hand it to an agent
A click on the alert opens the agent in a terminal under it, with the whole alert as its first message.
-
Claude Code
Anthropic's coding agent, in its own CLI.
-
Codex
OpenAI's coding agent, in its own CLI.
-
Cortex Code
Snowflake's coding agent, in its own CLI.
Langfuse questions
Which keys does it need?
A project's public and secret key, from Project Settings › API Keys. Only a project Owner or Admin can create keys; a Member can see the list but not create one.
Does it fit the free Hobby plan?
Yes. Errors and scores use the general and Observations APIs (30 requests a minute on Hobby). Cost and latency use the Metrics API, which Hobby limits to 100 calls a day, so those monitors check hourly by default.
Does it read my prompts and completions?
No. It asks for names, levels, status messages, models, usage, cost and latency, never an observation's input or output.
Other connectors
-
Gmail
Hear about the emails that matter, not every email. Unread mail, chosen senders, labels, Drive comments and invitations reach your Mac's notch, and nothing is marked read.
-
Calendar
The calendars Calendar.app already shows (iCloud, Google, Exchange, subscriptions), or a calendar's secret ICS feed, in your Mac's notch. A countdown to your next meeting, a warning before it starts with one click to join, and news of invitations and changes.
-
Dock badges
One watch over every app that badges its Dock icon. Slack, Microsoft Teams, Outlook, Mail, Messages, Discord, WhatsApp or any other app tells the notch when its count goes up, with no account or token for any of them.
-
Apple Mail
Every account Mail already has, iCloud, Exchange, Gmail or any IMAP, in your Mac's notch. New unread mail in your inboxes, or only from the people and domains you pick, with no password given to CoIsland.
-
GitHub
Monitor GitHub for what needs you, not everything that moves. Reviews, failed Actions runs, releases and security alerts, on github.com or GitHub Enterprise.
-
Vercel
Watch Vercel deployments, stuck builds and project domains. A failed build reaches your Mac's notch with its error, and the alert shows the failing step and the end of the build log.
-
GitLab
Watch GitLab on gitlab.com or a self-managed instance. A review request, a failed pipeline or a new to-do reaches your Mac's notch, and the alert shows the merge request with its description and comments, or the jobs that failed.
-
Cloudflare
Watch Cloudflare Pages deployments, Worker errors and your zones. A failed build reaches your Mac's notch with its failing step, and the alert shows the end of the build log.
-
Render
Watch the deploys of your Render services and whether they are suspended. A failed deploy reaches your Mac's notch with its commit, and a suspended service says who suspended it.
-
Jira
Monitor Jira with the JQL you already write. New matches, status changes, comments, SLAs near a breach and sprint scope reach your Mac's notch, from Jira Cloud or Data Center.
-
Linear
Monitor Linear with its own issue filters. New matches, status changes, issues in triage, SLAs at risk, your mentions and projects going off track reach your Mac's notch.
-
Confluence
Monitor Confluence Cloud without the email digest. Mentions, page edits, new comments, your tasks and any CQL reach your Mac's notch, signed in with the same API token as Jira.
-
Asana
Know what landed on your plate and what is late. Tasks assigned to you, due dates, new tasks in a project and comments on your tasks reach your Mac's notch, read with your own Asana token.
-
ClickUp
Know what is yours and what is late. Tasks assigned to you, their status and due dates, and new tasks in a list reach your Mac's notch, read with your own ClickUp token.
-
Todoist
Know what is due before it is late. Tasks due today or overdue, new tasks in a project, or any filter you already use reach your Mac's notch, read with your own Todoist token.
-
Notion
Know when a Notion row lands on you, a page you care about changes, or someone comments. Read with your own personal access token, straight from your Mac.
-
Shortcut
Know when a story lands on you, moves to review, or an iteration is about to end with work still open. Read with your own Shortcut API token.
-
Airtable
Know when a record starts matching a formula, or someone edits one, without keeping the base open. Read with a personal access token scoped to the bases you pick.
-
Reminders
The reminders Reminders.app already has, iCloud or Exchange, in your Mac's notch. A warning before one is due, another when it is overdue, and news when someone adds to a list you share.
-
Sentry
Monitor Sentry with the searches you already write. New errors, regressions, issues assigned to you, failing crons, sites down and new releases reach your Mac's notch, from sentry.io or your own Sentry.
-
PagerDuty
Know at a glance whether you have to move. Incidents assigned to you, your teams' incidents and your on-call shifts reach your Mac's notch, read with your own PagerDuty token.
-
Datadog
Know when a Datadog monitor fires without living in the dashboard. Monitors in alert, incidents and metric thresholds reach your Mac's notch, read with your own application key.
-
New Relic
Run the NRQL you already trust, and see New Relic's alert issues the moment they open, in your Mac's notch, with your own User key.
-
Snowflake
Monitor Snowflake with read-only SQL. A failed task, a row that breaks a rule or a sudden spike reaches your Mac's notch, and Cortex Code, Claude Code or Codex can take it from there.
-
Databricks
Monitor Databricks from your Mac's notch. A failed job run, a failed pipeline update or a triggered SQL alert reaches you for free, and read-only SQL on a warehouse watches the data itself.
-
dbt
Monitor dbt from your Mac's notch. A job run that fails, a run that takes too long or a source that goes stale reaches you, with the failing step's log one click away.
-
PostHog
Monitor PostHog from your Mac's notch. A HogQL query that finds something, an insight crossing a number or an exception seen for the first time reaches you without opening a dashboard.
-
Supabase
Monitor Supabase from your Mac's notch. A paused or unhealthy project, a new security lint or a row your SQL finds reaches you, with nothing but a personal access token.
-
PostgreSQL
Monitor PostgreSQL from your Mac's notch. A read-only query checks your database on a schedule, and each new row it finds reaches you once.
-
MySQL
Monitor MySQL and MariaDB from your Mac's notch. A read-only query checks your database on a schedule, and each new row it finds reaches you once.
-
ClickHouse
Monitor ClickHouse from your Mac's notch. A read-only query, or a health check on the system tables, runs on a schedule, and each new problem reaches you once.
-
HTTP / JSON
Point CoIsland at any address. A JSON value that changes or crosses a threshold, a new element of a JSON list, or a page that goes down or starts saying "In stock" reaches your Mac's notch.
-
RSS / Atom
Watch any feed. A new GitHub release, a new post on a blog, or a Hacker News story that says your keyword reaches your Mac's notch, with the article one click away.
-
Status pages
Know when a service you depend on is down before your users tell you. Incidents, their updates, components going down and the page's overall status reach your Mac's notch.
-
Shell command
Run a command or a script on your Mac on a schedule. A new line it prints, a new object in its JSON, or a failure reaches the notch once.
-
Stripe
Watch failed payments, disputes, subscriptions, payouts and your balance with a read-only restricted key. A declined card reaches your Mac's notch with the amount, the customer and the bank's reason.
Put your work stack in the notch
Regular price: $59.99 Early bird price: $19.99 once, for 2 Macs. Native for Apple Silicon, macOS 14 and later.
Early bird price until October 25