In chapter 14, I told you about the SonarCloud dashboard and its red gates. There was a question I had not asked out loud: what did we actually buy?
We moved to SonarCloud's paid plan. I decided it, because I wanted Sonar to be part of the merge gate, not only a dashboard we look at from time to time. This chapter tells what changed, what started badly, and the figures I read today, October 28. I only say what I could check, in the code, in the merged PRs and on SonarCloud's public API.
Before: an analysis that ran on its own
At first we used SonarCloud's automatic analysis. You switch it on with a click, Sonar reads the repository without asking our CI for anything, and it answers on pull requests. It is handy, and we got by with it for a long time.
But it has two limits that I found one after the other. It cannot import test coverage, because the tests run on our side, not at Sonar. And it does not answer on the temporary branches of GitHub's merge queue. That is why I had written in the previous chapter that Sonar “only answered on PRs”. That was true of the automatic analysis, not of Sonar.
- Before (automatic analysis)Sonar reads the repository on its own and answers on the pull request. No coverage, nothing on the merge queue.
- Editor (lint, ratchet)The sonarjs rules answer while we write.
- Tests (lcov, JaCoCo)The test job drops its coverage report.
- CI analysis (after the tests)The shared pipeline's sonar job runs the analysis, imports the coverage and waits for the gate's verdict.
- CI gate (one verdict)One required check sums up the pipeline, Sonar included.
- Merge queue (Sonar answers here too)The CI analysis answers on the queue's temporary branch, then GitHub merges.
What the paid plan changed
Concretely, three things, and they live in the organisation's shared pipeline, not in each repository.
The first is that the analysis is started by our CI, after the tests, with sonar: true in the shared workflow. It waits for the quality gate's answer before giving its verdict. A project's automatic analysis is switched off only once this job is green, so a repository is never left without an analysis.
The second is coverage. The test job drops its report, lcov for JavaScript and TypeScript, JaCoCo for Java, and the Sonar job imports it. That is why Sonar “moved into” the workflow: coverage can only come from an analysis run by the CI.
The third is the merge queue. A CI analysis answers on any branch, gh-readonly-queue/main/... included. I saw it on krizaka-ts: SonarCloud's public API lists that repository's queue branches, each with its gate's verdict. The other repositories move over one by one, following a recipe written in our standards. Until a repository has taken the step, it keeps the rule that a branch must be up to date before merging.
And one small extra, because we thought of it late: a PR that only changes documentation gets a light analysis, with no build, instead of a check that never arrives. That is .github#336.
Why the gates first turned red
I expected everything to go green. It did the opposite.
At the first CI analysis, Sonar had no reference version for the project. It showed “not provided”, so it counted the whole history as new code. Every old defect became the fault of the latest change, and the gates turned red all at once, in repositories that had done nothing new.
Our new-code period is “previous version”: Sonar judges what changed since the last known version. With no version, there is no previous one. The fix is small. .github#354 gives each CI analysis a version, ci- followed by the run number, so the period is the change since the analysis before. Old defects stay visible. They no longer turn the gate red on every PR. In krizaka-ui's analysis history you can see the “not provided” analyses, then the version event, a few hours later.
No version
- Sonar shows “not provided”
- The whole history counts as new code
- Every old defect is blamed on the latest change
- Gates red everywhere at once
A version per analysis
- Each CI analysis is versioned: ci- and the run number
- New code is what changed since the analysis before
- Old defects stay visible
- The gate judges the change
What I keep from it: a tool you buy does not set itself up. We paid for a gate, and the first thing it did was say no to everything.
What the figures say
Here is what I read this morning on the public API, project by project, with the key krizaka_<repository>. I give the numbers as they are; they move from one analysis to the next.
In krizaka-ui, the PR krizaka-ui#781 fixed 37 bugs, among them the sorts without a compare function I mentioned in the previous chapter. The API shows 0 bugs today, and 0 vulnerabilities.
Of the 9 repositories I read, the gate is green in 5: krizaka-com, krizaka-ts, akinaza, yomizaka and orazaka-ai-engine. It is red in 4: krizaka-ui, krizaka-blog, the .github repository and krizaka-build. In all four, the gate fails on new code, first on critical issues, not on old ones. I have not worked around them, and I am not hiding them here.
On coverage, what Sonar receives now: 78.9% for the blog you are reading, 68.5% for krizaka-com, 54.1% for krizaka-build, 39.5% for akinaza and 39.3% for yomizaka. For krizaka-ts the API shows 0.0%: the report is not sent there yet, and our standards note it as a known gap. For krizaka-ui there is no figure at all.
These numbers do not say the code is good. They say where we look, what we do not look at yet, and whether it improves from one week to the next. That is why I reread them the day I write.
Who does what, Sonar or our guards
A question I asked myself when paying: are we doing the same work twice?
My answer is: we try not to. An error should be stopped as early as possible, and Sonar arrives late, after the PR. So whenever a Sonar rule can run in the editor, we run it there. @krizaka/config turns on the eslint-plugin-sonarjs set, and the cognitive complexity threshold is the same as at the gate, 15. For Java, PMD and SpotBugs play that role in mvn verify, and ArchUnit holds the architecture. The ratchets freeze each rule's debt and refuse any increase.
What Sonar keeps for itself, because our guards do not see it: the rules that follow data from end to end, the ones that read our GitHub workflows (injection, permissions), a few Java rules with no local twin, coverage, history and the overview of every repository. Our guards table marks those rows “Sonar only”, without dressing them up.
And there is what Sonar does not replace: secrets, which must be stopped before they enter the history, the rule “a fix brings its guard”, and the tests that tell whether the product does what we promise.
What is planned, not done
There is one SonarCloud feature I would like to try: the Architecture view. It lets you declare the intended architecture, for example “Spring AI only in its adapter” or “no database access in services”, and see the deviations as issues, next to the rest. I wrote the idea down in an issue, after looking at a capture of a “current architecture” view.
Since then, I have settled the order. First a C4 diagram generated from the code: the overview, from the system down to the files. It will also run locally, so I see the code on my machine and not only what is deployed. Only then Sonar's Architecture view, with the intended architecture and its deviations. Nothing is switched on yet. The principle stays simple: never put the same rule in two places. Sonar would give the overview and the history, the build would keep the fast stop.
What I keep from it
We paid for Sonar to see our coverage, judge what changes and speak on the merge queue. That is done, and it can be checked. We did not pay for a green stamp, and today's red gates remind me of it.
The work stayed the same: move the checks as early as possible, and keep Sonar for what only it sees.
