Connect Langfuse

To connect Langfuse to CoIsland you need one project’s API key pair: a public key and a secret key. CoIsland calls Langfuse’s public API straight from your Mac, with no CoIsland server in between, keeps the secret key in your login Keychain, and only sends GET requests. This page covers the keys, the connector form, the five Langfuse monitor kinds with queries you can paste, the rate limits, and the errors you may meet.

Create an API key pair

Open your project in Langfuse, then Settings › API Keys and Create new API keys. Copy the public key (pk-lf-…) and the secret key (sk-lf-…): the secret key is shown once.

• Who can create keys. Only a project Owner or Admin can create or delete keys. A Member can see the list of keys but not create one: ask an Owner or Admin.
• Keys are not read-only. Langfuse has no read-only key: the same pair your SDK uses to send traces could also write. CoIsland only ever sends GET requests with it. Create a pair just for CoIsland, so you can delete it on its own.
• Project keys, not organization keys. An organization’s keys reach no project here; use a project’s.

Langfuse Cloud runs four regions, each its own service: EU (cloud.langfuse.com), US (us.cloud.langfuse.com), Japan (jp.cloud.langfuse.com) and HIPAA (hipaa.cloud.langfuse.com). Your region is the address you sign in to; keys made in one do not work in another.

Connect Langfuse in CoIsland

Open Settings › Connectors, click +, and choose Langfuse.

Field	What to enter
Connector name	What monitors call it: langfuse. Letters, digits, _ and -.
Cloud region	Your Langfuse Cloud region. Leave it on EU if you sign in at cloud.langfuse.com.
Self-hosted address	Only for your own Langfuse, such as https://langfuse.acme.com: it replaces the region. Any page of it pasted works. http:// is accepted only for localhost.
Public key	The pk-lf-… key.
Secret key	The sk-lf-… key.

Test connector saves nothing. It sends one GET /api/public/projects and reads Connected with the project’s name, its organization and the region. Add Connector writes the region or address and the public key to ~/Library/Application Support/CoIsland/connectors.json, and the secret key to your login Keychain. The pair is sent only to your Langfuse, as HTTP Basic authentication (public key as the user, secret key as the password). Editing the connector with the secret key blank keeps it.

Choose what to watch: the five Langfuse monitor kinds

Open Monitors, add a monitor and choose Langfuse. Each kind is a form; Edit as Text shows the query it writes. A new-items monitor’s first check records what is already there and raises nothing.

LLM errors

Observations logged with level ERROR that started within the lookback (1 hour unless you pick 6h, 24h or 7d), from the Observations API v2, which reads new data in real time. Each alerts once. level:warning takes warnings too. Narrow it by model, observation name, trace name or environment, several with commas. Checks every 5 minutes by default.

lookback:1h model:gpt-4o,gpt-4o-mini environment:production

Cost over a limit

What the project’s observations cost, from the Metrics API v2, over the last 1h, 6h, 24h or 7d, or today since midnight on your Mac, above a number of dollars. per:model or per:trace gives one number each, so each model over the limit alerts on its own. It alerts as the cost crosses the limit, and again after it came back under and crossed again. Checks hourly by default.

window:today above:50
window:24h above:20 per:model

Slow LLM calls

The p95 latency of generations over the window, above a number of seconds (10, 1.5 or 800ms). type:span or type:any measures other observations; per:model, per:name or per:trace gives one number each. Checks hourly by default.

window:1h above:10 per:model

Eval score below a limit

Numeric scores with the name you give, created within the lookback (24 hours unless you pick), whose value is below the limit: an LLM-as-a-judge evaluator, a score your code sends, or an annotation. source:eval, api or annotation narrows by where it came from. Each score alerts once.

score:helpfulness below:0.5 source:eval

Negative user feedback

Scores named user-feedback (or the name you give) whose value is one you call negative: numbers (0), true or false, or categories (thumbs_down), several with commas, all of one kind. Each alerts once, with the user’s comment.

score:user-feedback value:0
score:thumbs value:false

What a Langfuse alert shows

The notch lists an error as checkout-assistant › answer · Error · gpt-4o · 2.5s · $1.50 under its message, a crossing as Model · Above $50.00 · Now $61.20 · today, a low score as Eval · Below 0.5 · production, and feedback as API · Negative · then the first line of the user’s comment. Clicking one opens the alert in CoIsland:

• an error fills in with its trace, model, latency, time to first token, cost, tokens, prompt, user and session, and its whole status message;
• a crossing is computed again, still over or back under, and for the whole project with each model’s number beside it;
• a score shows its value, source, the trace’s name and model, and its comment.

Open in Langfuse goes to the trace (on the observation, for an error or an observation’s score), or to the project for a crossing. CoIsland never reads an observation’s input or output.

What a Langfuse monitor’s watch file looks like

-- name: LLM errors in production
-- kind: langfuse.errors
-- connector: langfuse
-- every: 5m
-- alert: new-rows

lookback:1h environment:production

The kinds are langfuse.errors, langfuse.cost, langfuse.latency, langfuse.scores and langfuse.feedback. Watch files lists every header key.

Rate limits

On Langfuse Cloud, limits apply to the whole organization, per bucket, whatever the project or key. On the free Hobby plan the Observations API v2 and the general API (scores, the project) allow 30 requests a minute, and the Metrics API v2 100 requests a day. An errors or scores check is one request (plus one for its alert page); a cost or latency check is two (the project, then the metric), one of them a Metrics call, and its alert page one or two Metrics calls. That is why cost and latency check hourly by default: two such monitors use 48 Metrics calls a day. Core allows 100 Metrics calls an hour and Pro 500. When Langfuse answers 429, CoIsland backs off for the seconds its Retry-After gives. A self-hosted Langfuse has no such limits.

Troubleshooting Langfuse connector errors

• Langfuse (cloud.langfuse.com) refused the keys. A key was mistyped or deleted, the two keys are not one pair, or the region is not your project’s: keys work only in their region.
• That is the secret key / That is the public key. The two keys were pasted in each other’s field.
• These keys reach no project. They are an organization’s keys; create a project’s.
• Langfuse refused the request: then Langfuse’s reasons, such as a filter it cannot read.
• The Observations API v2 (a self-hosted Langfuse may be too old for it) was not found on this Langfuse. Update your self-hosted Langfuse: these kinds read the v2 Observations and Metrics APIs and the v3 Scores API.
• The address answered with a web page, not Langfuse’s API. The self-hosted address is not your Langfuse’s.
• Rate limited; retry after 30. The next check tries again.