Webhook setup
TIER records PR outcomes from a GitHub webhook. Without it, tierd serve still
captures cost, but it computes no full TIER scores because it has no outcomes.
Configure the webhook in GitHub
Repository (or organization) Settings → Webhooks → Add webhook:
-
Payload URL:
https://<your-server>/webhook/github—<your-server>must be a hostname GitHub can reach from the internet (a public server, or a tunnel such as ngrok or Tailscale funnel in front of a laptop). GitHub cannot reach127.0.0.1; on a laptop, skip the webhook and pick up newly merged PRs withtierd backfillinstead: stopserve, re-runbackfill, then startserveagain (the database takes one writer at a time). -
Content type:
application/json -
Secret: a password you generate once and give to both sides. If you already made
~/.tier/webhook-secretin the README steps, justcatit — don't make a second one. Otherwise make it and keep it in a file:mkdir -m 700 -p ~/.tier [ -s ~/.tier/webhook-secret ] || (umask 077; openssl rand -hex 32 > ~/.tier/webhook-secret) cat ~/.tier/webhook-secret # paste this value into GitHub's Secret box -
Which events? Choose "Let me select individual events" and tick Pull requests, Pushes, and Workflow runs.
TIER processes exactly three event types (
internal/webhook/handler.go):pull_request— a closed + merged PR becomes an outcome (quality1.0). Its timestamp comes frompull_request.merged_at, with receipt time used only when that value is missing, malformed (including zero), or in the future. Delayed or redelivered events therefore land in the merge's reporting window, and the CI and revert observation windows start at the merge time.workflow_run— a CI failure on the merge commit within the 48h window floors that outcome's quality to0.7; a success records a clean CI signal. If you omit "Workflow runs" the CI-fail quality signal is silently disabled — an outcome that broke CI still scores as a clean1.0.push— revert detection applies to pushes to the repository's default branch only and degrades the reverted change's quality: a code-problem (quality) revert floors to0.1, a business-decision (strategic) revert to0.8, within a 60-day window.
Every other event is ignored, so subscribing to more only adds noise.
Configure the secret on the tierd side
Provide the same secret to tierd serve, ideally without putting it on the
command line:
export TIER_WEBHOOK_SECRET=@$HOME/.tier/webhook-secret # read from a file
# or: ./bin/tierd serve --webhook-secret @$HOME/.tier/webhook-secret ...
Fail-closed: if no secret is set, the POST /webhook/github route is not
mounted at all — an unauthenticated webhook would let anyone who can reach the
listener forge merged-PR outcomes or fake revert pushes. You will see this at
startup:
WARN TIER_WEBHOOK_SECRET is not set — POST /webhook/github is disabled (fail closed, #60)
Every delivery is verified with an HMAC-SHA256 signature over the raw body,
compared against the X-Hub-Signature-256 header. A mismatch returns 403.
PR size labels and outcome weight
When a merged PR carries a size label, TIER weighs the outcome from that label
(weight_source='label') instead of the diff-size heuristic. Out of the box it
recognizes GitHub's size/xs..size/xl labels and the bare xs..xl forms,
mapped onto the fixed outcome scale:
| Label (case-insensitive) | Weight |
|---|---|
size/xs, xs |
0.5 |
size/s, s |
1 |
size/m, m |
3 |
size/l, l |
5 |
size/xl, xl |
8 |
A PR whose labels don't match any entry falls through to the git-diff heuristic
(weight_source='git-heuristic') — nothing is lost, but a label is the more
trustworthy signal.
Remapping the label names (outcomes.size_labels)
If your org uses a different naming convention — size: M with a space, size-l,
or an S-prefixed scheme common to pull-request-size bots — remap the names in the
YAML config (tierd serve --config):
outcomes:
size_labels:
"xs": 0.5
"s": 1
"m": 3
"l": 5
"xl": 8
"xxl": 8 # extra names are fine — they just map onto the same fixed scale
Rules:
- Only the names are configurable. Every weight must be one of the fixed scale
0.5, 1, 3, 5, 8so scores stay comparable across orgs — an off-scale value (or a blank label name) makestierd servefail loud at startup. - Matching is case-insensitive.
- A custom table replaces the built-ins (it does not merge) — list every label
you want recognized, including
size/*forms if you still use them. - Absent or
{}keeps the built-in table above unchanged, so existing deployments are unaffected. - A configured match still records
weight_source='label'— no new provenance. - Give
tierd backfillthe same--config. Backfill (rebuilding outcomes from past merged PRs) readsoutcomes.size_labelsfrom its--configfile (#301), so old and live outcomes get the same weights. Without--config, backfill uses the built-in table above and prints a warning saying so. Backfill never re-weighs a PR that is already stored: it skips any merge commit it has seen before.
Work-type labels (#187) are a deliberately fixed taxonomy and are not
configurable; this applies to size labels only.
Delivery semantics
- Signature:
X-Hub-Signature-256: sha256=<hmac>— required; a mismatch is403. - Body bounds (before authentication):
pushbodies may be up to 25 MiB; all other event types are limited to 1 MiB. These limits also apply to chunked requests. Larger bodies return413, are not processed or stored, and incrementtier_webhook_oversized_total{event}. The event label ispush,pull_request,workflow_run, orother; both the event and delivery headers are unauthenticated, even on a signed body. The ERROR log includes the bounded event label, sanitized delivery identifier and size class. One push at a time may grow beyond 1 MiB; contention returns503withRetry-After: 1. The slot remains held through signature verification and processing. - Dedup: replays are deduped by the
X-GitHub-DeliveryGUID (combined with the event type), and merged-PR outcomes carry a durable merge-commit-SHA guard, so a redelivery is a safe no-op. - Failures: a successful (including duplicate) delivery returns
204. A failed one returns503(the database stayed locked past its 5-second wait) or500(any other error, which can also be a lock held past that wait), or times out with no response. After resolving the cause, manually redeliver transient failures: GitHub does not redeliver a failed delivery on its own, so that event is lost until you redeliver it — see Redelivering a failed delivery. Redelivering the same oversized body repeats413; it does not recover the event.
These local bounds do not establish complete delivery. GitHub documents a
separate 25 MB provider payload cap: events exceeding it produce no webhook
payload. Push payloads include at most 2,048 commits. TIER cannot observe
omitted payloads or commits through this endpoint; retrying a local 413 does
not recover provider omissions. See GitHub's
webhook payload limits
and push payload.
Redelivering a failed delivery
GitHub can redeliver only deliveries from the past 3 days, so act promptly. A
redelivery carries the same X-GitHub-Delivery GUID, and tierd
reprocesses it because a failed delivery is never recorded as seen.
In the GitHub UI: open the repository (or organization) Settings → Webhooks, click the TIER webhook's URL, open the Recent Deliveries tab, click a failed delivery (red icon), and click Redeliver.
With the REST API (for example with the gh CLI, signed in as a repository
admin; for an organization webhook, see below):
# 1. The hook id: the number at the end of the webhook's settings page URL
# (.../settings/hooks/<hook_id>), or from this list.
gh api repos/<owner>/<repo>/hooks --jq '.[] | {id, url: .config.url}'
# 2. The failed delivery ids, including timeouts and missing responses.
# Keep ids as bare scalars: object output can round ids above 2^53,
# producing an id that the redelivery endpoint rejects.
gh api --paginate 'repos/<owner>/<repo>/hooks/<hook_id>/deliveries?per_page=100' \
--jq '.[] | select(.status != "OK") | .id'
# 3. Redeliver one of them.
gh api -X POST repos/<owner>/<repo>/hooks/<hook_id>/deliveries/<id>/attempts
For an organization webhook, replace repos/<owner>/<repo> with orgs/<org>.
This needs an organization owner and the admin:org_hook token scope, which a
default gh auth login does not grant: add it with
gh auth refresh -s admin:org_hook.
Verifying it works
You can compute a signature by hand and post a synthetic merged-PR payload
using a private header file, keeping the HMAC out of process arguments
(the key file must be the same file given to tierd serve --webhook-secret):
This writes an outcome for the fake login __tier_webhook_test__ and issue
issue-999999999 into the receiving server's store. The row persists there
and can affect reports; this is not a dry run. Use a test server if you do not
want synthetic data in your live store.
(
set -eu
umask 077
H_WEBHOOK=$(mktemp)
trap 'rm -f "$H_WEBHOOK"' EXIT
PAYLOAD='{"action":"closed","pull_request":{"number":999999999,"merged":true,"merge_commit_sha":"tier-webhook-test-not-a-real-merge","user":{"login":"__tier_webhook_test__"},"body":"closes #999999999","labels":[{"name":"size/m"}],"additions":10,"deletions":2,"changed_files":3}}'
SIG="sha256=$(printf '%s' "$PAYLOAD" | perl -MDigest::SHA=hmac_sha256_hex -e '
local $/;
open my $fh, "<", "$ENV{HOME}/.tier/webhook-secret" or die $!;
my $key = <$fh>;
$key =~ s/[\r\n]+\z//;
die "empty webhook secret\n" unless length $key;
print hmac_sha256_hex(scalar <STDIN>, $key);
')"
[ "$SIG" != "sha256=" ] || { echo "could not compute a signature: check ~/.tier/webhook-secret" >&2; exit 1; }
printf 'X-Hub-Signature-256: %s\n' "$SIG" > "$H_WEBHOOK"
curl -i -X POST https://<your-server>/webhook/github \
-H "X-GitHub-Event: pull_request" \
-H "X-GitHub-Delivery: $(uuidgen)" \
-H @"$H_WEBHOOK" \
-H "Content-Type: application/json" \
-d "$PAYLOAD"
)
# expect HTTP 204
GitHub's "Redeliver" button on any recorded delivery does the same against a real payload.
Troubleshooting
- "PR merged but no outcome recorded." Almost always no issue reference was found on the branch or in the PR body, so the outcome could not be attributed to an issue. Check conventions.md — the branch name or PR body must carry a recognizable issue reference.
- Every delivery returns 403. Signature mismatch: the secret configured in
GitHub differs from
TIER_WEBHOOK_SECRETon the server. - Every delivery returns 404. No secret is set on the server, so the
POST /webhook/githubroute is not mounted at all (fail-closed). SetTIER_WEBHOOK_SECRETand restart.