Peer Learning with TIER: Move Practices, Not Scores

Coach the practice, not the person. Grade the work, not the worker.

TIER helps engineers choose the right model, give it the right context, and spend where it pays. In developer mode, a developer's page is visible to that developer and to the viewers the operator configures: the team's declared manager, an optional skip-level viewer, or whoever a works agreement names. Teammates never see it. Every viewer sees the same page, and the developer can read the log of who viewed it, including administrators. Managers may coach from it; no one may ask for figures beyond it. TIER is not suitable for pay, promotion, appraisal, reviews, improvement plans, discipline or dismissal. The product cannot enforce that; only your policy or works agreement can. Anyone with host access can read the database; these limits apply at the API.

The one rule. A team gets better together by sharing what someone changed and what happened -- a practice -- not by comparing frozen individual numbers. Use name-free team or division cache shares on real data over sealed months in v0.7.0. The developer-mode coaching card ships in v0.7.0 with raw-data retention. The synthetic demo invents its own data in a private database and cannot load yours.

TIER coaches the practice, not the person (see the project README and docs/legal-and-privacy.md). This page shows a team lead how to run peer learning on that principle: agree one practice change, then measure honestly whether it helped. Name-free team cache shares and the v0.7.0 coaching card provide discussion prompts; the card ships with raw-data retention. The synthetic demo previews invented data.

Table of contents

• What this is
• The levers retro
• The before/after practice experiment
• Capture the practice
• Coaching, not appraisal
• What NOT to do

What this is

v0.7.0 availability: Verified coaching cards are available to the subject and permitted viewers with audited reads and raw-data retention; shared reports stay name-free. The separate real-data per-issue report remains future work. tierd demo shows invented views read-only in a private temporary database and cannot load yours.

Practice transfer, not number transfer. When one part of a team has learned to get more yield per $1,000 -- higher cache reuse, less premium-model spend on routine work, less spend that TIER could not link to an issue -- the thing worth spreading is the habit that produced it, described in words, not a leaderboard cell.

The method uses these three surfaces. The coaching card ships in v0.7.0 with raw-data retention:

Who sees what. v0.7.0 developer mode requires active permitted viewer credentials, a nonempty admin token and a named audited operator. The subject and their declared viewers read the same verified, audited coaching card. Team/division reports stay name-free. TIER is built for coaching, not for evaluating developers. Per-person figures are not accurate enough to judge an individual. The synthetic demo shows invented named rows beside the levers. See Coaching, not appraisal.

The levers retro

For a monthly team retro, read the published cache share in GET /api/v1/scores?period=YYYY-MM. In v0.7.0, open Developer card on the dashboard with the subject identifier and that subject's credential. A manager opens each permitted card separately; the developer sees the same card. All views, including administrator views, are logged before the builder runs.

Choose one practice from the four tiles: review model choice on routine work; use scoped sessions, stable prefixes and an instructions file; start from an issue with acceptance criteria; or follow the work-type playbook below. The routine-work tile says “data insufficient” until explicit docs/chore classification exists; the current taxonomy cannot identify those categories. It never infers a task's difficulty or says a cheaper model would have sufficed.

Read the developer's own trend within a work type. A typical range is the nominal 95% CI (with unvalidated coverage) of one pooled cost-per-outcome figure across at least k eligible contributors in one team-wide population, including the subject. Each needs an active viewer credential, at least 50% attributed spend, at least three outcomes in that type and no low-token outcomes. Each must supply 1–50% of both pooled cost and outcomes. Outcomes, rather than per-person means, are pooled. No contributor points, identities or count are shown. The band uses the latest sealed UTC calendar month, ignoring the card's date and repository filters. Its value is rounded to $10 and CI bounds outward to $10. A private input digest is captured with the seal; changes to inputs, hierarchy, aliases, credentials or correction projections withhold the band rather than recomputing it. Legacy seals without that digest have no band. The own trend shows the card's last two complete 14-day windows. Each outcome belongs to its merge fortnight and carries its full-window joined cost and low-token flag. The tripwire counts verified tokens over the 14 days ending at the merge, including tokens before the card window. Separate cache/context and no-issue metrics are labelled as point readings without CIs and are not plotted on the cost axis. Failing any band guard means no band.

The attribution line measures the card's verified spend joined to outcomes, not capture fidelity. Spend on main stays uncovered. Below 50%, the card says “data insufficient” with an information icon and text. It withholds TIER and its CI, all work-type figures, premium-on-routine and the no-issue lever. Verified spend, outcomes, cache-read share and overall premium share remain. “Not on this card” is rounded to ten percentage points; excluded dollars are never published or exactly recoverable from that coarse share.

“One practice to try” selects the largest normalized deterioration among the available own trends: decreased cache-read share, increased no-issue share, or increased cost per outcome within a work type. Every measure uses symmetric relative change: twice the deterioration divided by the sum of the absolute before and after values (zero when both are zero). This dimensionless measure handles transitions from zero consistently. A tie follows the fixed tile order. It is withheld unless both windows clear the attribution floor and each has at least three outcomes. “Check again in 2 weeks” carries before/after n and nominal 95% CIs with unvalidated coverage; an observed change never establishes causation.

The team retro shares practices and name-free reports; teammates never see each other's cards. Agree one experiment from the written practice target, then check the next two-week observation. Evidence links contain only PR and session identifiers, never prompts, code, titles or session content.

1. Fetch the team's sealed month and read its cache share: cache-read tokens divided by all input-side tokens.
2. Treat the lever as a prompt, not a verdict. Ask which workflows could reuse cached context.
3. Compare similar work over finished months. Discuss task mix and context; the pooled team figure is not a cross-type ranking.
4. Treat a cheaper model as a hypothesis. Compare similar work over finished periods, record task mix, acceptance criteria, pricing and capture, and inspect acceptance and repair evidence beside cost. A TIER change cannot establish causation; TIER does not credit merged work to a model or prove equal quality.

Honesty caveat. Yield reflects task mix and context, not raw talent. A senior on legacy work can show a lower lever than a junior on greenfield. Copy practices, never rank people. There is no absolute good/bad band (see docs/rubric.md).

In v0.7.0, only cache_read_share is published as a lever on real-data teams[] rows over sealed months. Every published group, including folded other, needs k distinct counted seats with both captured spend and merged work in that group during the month. Bots and spend-only people cannot lift this floor. Any failing group withholds all levers over the population; k is never below 3. Premium-model and unattributed shares are per-person coaching-card levers only.

The cache share also needs k contributors with cache reads and k without. Count distinct contributors in the row's cost population with positive input-side tokens (input + cache read + cache write), including bots and spend-only people. Zero-input contributors count on neither side. Mixed cached and uncached input belongs to the cache-read side. A failing share is omitted.

Withheld keys are absent, never zero or null. data_quality.team_levers_withheld is true with team_levers_withheld_reason of group_seats_below_k or category_seats_below_k. Residual suppression still withholds the whole response (#593, #864). Detailed model, token-class and work-type tables stay suppressed in team/division mode (#864): differencing work-type totals recovered a 3-person group's spend, and a model used by one person exposed their spend. Live windows and anonymized /scores/compare carry no levers. #937 tracks broader anonymized levers. v0.7.0 ships #1370 subject-bound credentials, a named audited operator, verified spend and fail-closed view logging, plus the coaching card and raw-data retention; the separate per-issue report remains future work.

The before/after practice experiment

For the synthetic demo or name-free sealed reports, explore a practice with GET /api/v1/scores/compare (#277): two half-open windows -- A = before, B = after -- and read the result honestly.

Every delta is B - A. In developer mode, the endpoint reuses the exact windowed computation as /scores, so a compared number never diverges from what /scores reports for the same window; this is a synthetic-demo preview until #826 ships. In team/division mode, it refolds both sealed months with the intersection k floor: a group is named only if it clears k in both months, and each month's residual must also reach k or the whole comparison is withheld. Compared rows can therefore differ from /scores.

Worked example (synthetic demo only; real-data subject-bound developer detail (#1370); coaching card built; release gated on retention). Say the invented team adopted "cheaper model for routine refactors" on CHANGE_DATE. Point the command at the synthetic demo and compare the 8 days before against the 8 days after. The demo's merges sit 7 to 19 days before the day it started, so with that day as S, replace BEFORE_START with S minus 22 days, CHANGE_DATE with S minus 14 days and AFTER_END with S minus 6 days, all as UTC dates:

GET /api/v1/scores/compare?since_a=BEFORE_START&until_a=CHANGE_DATE\
&since_b=CHANGE_DATE&until_b=AFTER_END

# Synthetic demo only; TIER_HOST points at the running tierd demo.
curl -sS "$TIER_HOST/api/v1/scores/compare?since_a=BEFORE_START&until_a=CHANGE_DATE&since_b=CHANGE_DATE&until_b=AFTER_END"

Reading the result honestly is the whole point:

• Each row carries delta_tier, delta_weighted_points, delta_total_cost_usd, and a significant flag.
• significant is a legacy raw non-overlap result, retained for compatibility. It is true only for a developer present and above the floor in both windows with disjoint bounds and, in v0.6.1 (#1233), each row's own attributed-cost share at least 50%; false for withheld bounds. Neither value establishes an effect or a noise threshold. The API labels it unvalidated and the dashboard says “inference not established”, without interval-derived colours.
• In v0.6.1 (#1233), intervals are withheld below 50% of the row's own cost paired with resampled outcomes; exactly 50% passes. When shown they are labelled "excludes uncertainty from unattributed spend". Team and division rows carry none and reveal no floor status.
• Nominal 95% bounds. Coverage is unvalidated: in constructed tests where several outcomes share one issue, they covered the true value less often than 95%. They are descriptive resampling bounds and exclude uncertainty from unattributed spend, incorrect attribution and uncaptured usage.
• Treat a small difference as no evidence either way, and a large one as a prompt to investigate, not a measured effect.
• Watch the honest windowing skew too: a recent window B reads artificially low because cost books up front and outcomes land at merge time. See docs/interpreting-the-number.md before trusting a short after-window.

In team/division mode, compare two closed, sealed months instead:

GET /api/v1/scores/compare?period_a=2026-05&period_b=2026-06

Give both period_a and period_b (A earlier than B), or omit both to select the two latest sealed months. An unsealed month returns 404; finish arming and sealing first. The free-window example above returns 400, with this exact error:

?since_a=, ?until_a=, ?since_b=, ?until_b= not accepted for anonymised reports (#913): they are served per sealed calendar month, so two overlapping windows cannot be differenced to isolate one person's figures. Remove ?since_a=, ?until_a=, ?since_b=, ?until_b= and select two months with ?period_a=YYYY-MM and ?period_b=YYYY-MM (neither: the two latest sealed months)

A valid sealed comparison returns 200 with k-anonymized group deltas (k-anonymity is a two-window intersection: a group must clear the floor in both windows to appear as a named row). The whole comparison is withheld, and says so, when either window's own other row does not reach k (#864), or the comparison's intersection residual does not reach k on either side. One honest caveat there:

In team/division mode, significant is always false. Group aggregates carry no bootstrap CI, so the significance test is simply not computed at group grain -- absence of a significant flag is not evidence of "no effect." In anonymized mode, read the delta as directional only: the sign and rough size of the move, not a pass/fail verdict.

Capture the practice

TIER stores the numbers. Humans store the narrative. The learning does not live in the database -- it lives in the sentence a teammate can read next quarter and copy.

After the experiment, write the practice into the team's own shared doc or runbook -- not into TIER, which has no field for it. A good entry is four lines:

### Practice: cheaper model for routine refactors  (adopted 2026-06-01)
- Change: route bugfix/refactor sessions to the standard model; reserve the
  premium model for design and hard reasoning.
- Signal watched: the same subject’s own work-type trend on the coaching card;
  delta_total_cost_usd on /scores/compare (May vs June).
- Result: premium share fell and delta_tier was positive with significant:false
  after one month. This does not establish that the model change caused the
  difference or that the cheaper model did the work as well. Investigate task
  mix, acceptance, repair evidence, pricing and capture before interpreting it.
- Keep / drop: keep; revisit at the next levers retro.

That entry is what actually transfers between people. The TIER surfaces told you where to look and whether the move cleared noise; your runbook says what to do.

The playbook

For each work type, write down the task, the model/context practice tried, the acceptance criteria and the observation window. Start from an issue, keep the session scoped, reuse stable context, and check cost per outcome alongside n and its CI after two weeks. Keep security, incident, research and other work types separate. Copy the practice in a runbook, without another person's figure. This is the #420 practice playbook; it sets no individual goal or model-switch claim.

Coaching, not appraisal

TIER coaches the practice, not the person. The coaching card is built; release is gated on retention. In developer mode, a developer and their permitted viewers see the same card, with the same caveats. Today, coach from team rows, or preview the per-developer view in tierd demo.

• TIER is built for coaching, not for evaluating developers. Per-person figures are not accurate enough to judge an individual.
• Talk about the habit, not the number. A coaching conversation looks at the practice behind a number (cache reuse, which model for which job, spend with no issue attached), agrees one thing to try, and checks later whether it helped. The number is where the conversation starts, not a grade.
• Keep a developer's numbers between the developer and their manager (and a skip-level manager, if your organization uses one). The team retro uses only team numbers, which name nobody.
• Compare within a work type, never rank. Read a developer's numbers against their own trend within one work type, not against a teammate's.
• The honest caveat travels with every number. Yield reflects task mix and context, not raw talent.

TIER cannot enforce this. It shows numbers; it cannot stop someone from copying one into a review. Only your own written policy, or a works agreement where one applies, can. Write the purpose down before adopting the developer mode with the subject-bound identity controls.

v0.7.0 behavior. Real-data servers publish team/division rows with only the gated cache_read_share lever; developer mode provides verified coaching cards. In the synthetic demo, any API or dashboard reader sees every invented developer's numbers in identifier order, never TIER order. The demo keeps its invented data.

Access and remaining work (from v0.7.0). #1370 supplies subject-bound viewer and upload credentials, a developer-readable fail-closed view log, audited admin reads and immutable event provenance. Shared reports are name-free and the shared-scale named-developer bar panel is removed on real data; the synthetic demo is unchanged. A developer's verified detail is visible only to that developer, their declared manager, an optional skip-level viewer, or declared works-agreement viewers; team membership never grants access. The coaching card requires configured viewer credentials and positive verified spend. Raw-data retention runs at startup and daily (90-day default, 60-day minimum, 365-day ceiling for a full annual coaching cycle). See identity access for commands and restrictions. Every mode stores per-developer identifiers. Real writable developer mode starts age-pruning; once armed, retention also continues in team/division mode. Pruned named history cannot be restored by widening the window; a name-free roll-up remains. Raw webhook payloads have separate retention and are not rewritten by an erase (see privacy.md).

Deployments in co-determined jurisdictions have a legal reason for this too -- see docs/legal-and-privacy.md for works-council, DPIA, and GDPR Article 22 guidance. The anonymized --aggregation team|division modes (k-anonymity floor default 5, hard minimum 3) exist precisely so an organization can run this workflow without ever naming an individual.

What NOT to do

Related

• docs/interpreting-the-number.md -- read the number honestly; the windowing skew.
• docs/legal-and-privacy.md -- the labor-law basis for keeping TIER out of appraisal.
• docs/api-compatibility.md -- the exact /scores and /scores/compare contract.
• docs/rubric.md -- why there is no absolute good/bad band.