Severity scoring

Bots label their own comments, and most of them label too much as important. Limn reads each review comment a bot leaves and gives it its own severity, Nit, Minor, Major or Critical, and up to eight categories, such as correctness, security or testing. A small model does this on your computer. It is not a large language model, it sends nothing anywhere, and it never reads the bot’s own severity label.

#Set it up

1. Open Settings → Bots.
2. Under Bot comment scoring, press Set up (one-time ~240 MB download). This downloads the model and the program that runs it, once.
3. Wait for the section to say Set up. Bot review comments are scored on this machine.

You can also run npx limn-review ml install in a terminal, and npx limn-review ml status to check it.

#It will not slow your computer

Scoring runs in the background at low priority, one comment at a time, using at most half of one processor core. It pauses while your laptop is on battery. A large backlog of old comments therefore takes a while. If you want it faster, or want it to carry on on battery, start Limn with LIMN_ML_CPU_SHARE=1 or LIMN_ML_ON_BATTERY=run. To turn scoring off, start Limn with LIMN_ML_DISABLED=true.

#Where you see it

• On each bot comment on a pull request’s Threads and Activity tabs. Hover over the label to see the model’s confidence and the categories. A label under 50% confidence is dimmed. A walkthrough or summary comment is marked summary, not scored as a finding.
• In Bots Monitoring: the What the bots are flagging strip and the severity columns on the ROI table.

When a bot labels its own comment, Limn shows that label beside its own as “bot said …”, with an arrow when the two disagree. The Inflation column on the ROI table counts how often each bot graded a finding worse than Limn did. A bot that labels nothing shows a dash: no label is silence, not agreement.

#Why it helps

When every bot comment says “important”, none of them is. An independent severity, made the same way for every bot, lets you sort comments by what matters and compare bots fairly. It also feeds the Tune verdict: a bot whose findings are mostly nits is worth narrowing.