approx. 7 min. reading time
Making project quality visible: turning experience into shared criteria
Written by Arda Müftüoğlu /
September 2026
Table of Contents:
A few months ago, I sat in a status meeting shortly before the acceptance phase. Someone asked the question that always comes up eventually: how are we doing on quality? The first answer was: “going quite well, actually”. It was probably even true. But nobody in the room could have said what “quite well” was based on. The problem is that as long as quality remains implicit knowledge within the team, there’s no shared basis for improving it deliberately. So we wanted to find a way to turn experience and assessment into a shared, visible picture.
We have been here before. Over the past few years we have tried several approaches to central quality checklists, and we learned something from each of them. One lesson was about acceptance: criteria work better when they originate close to actual practice. Another was about scope: no matter how complete a list is, if a large part of it is irrelevant to your project, attention inevitably drops. The current approach builds directly on those lessons, and there is one reason the topic has become more urgent than ever: AI.
Who writes the criteria
The difference this time is simple: the criteria are no longer written by a central function, but by experienced practitioners in each discipline: Engineering, Consulting, Product & CX, Project Management, Data & AI. Each area maintains its own checklists and quality gates in an internal tool.
In practice, that makes a big difference. When a checklist comes from someone who has been doing the job for ten years, the team quickly sees why each item is on it. The questions fit the daily work, use the right language, and reflect situations you recognize from real projects, and that closeness to practice is exactly what earns acceptance.
I constantly mix up the two terms myself, so a quick distinction. For us, a checklist is a kind of ongoing health check for the project, reassessed again and again over its lifetime. Fill it in once at kickoff and never again, and you have a snapshot that is useless two weeks later. A quality gate is something different: a fixed stop in the process where certain things have to be right before work can continue. It is the classic yes-or-no before a release, and a gate like that is only useful if it is sharply drawn: it should force a clear decision without needlessly slowing things down.
So that no project is faced with the same overlong list, a short questionnaire comes first. Fixed price or T&M? Greenfield development or replacing a legacy system? Any regulatory requirements? How big is the team? The answers determine which checklists and gates apply to this project, and only those are shown. The questionnaire is a little annoying at first, but it takes only a few minutes and spares you the later debate about which criteria even apply. The effect is bigger than you might expect. A short list of relevant items gets worked through. A long one where two thirds do not matter eventually gets tuned out, and then it is completely worthless, including the third that would have counted. For project leads, this produces a reliable picture for the first time: instead of a vague “going quite well”, it says something like “mostly green, yellow on risk and estimation”, and that is a sentence you can take into a meeting.
What such a list looks like
It contains questions like:
- Are the right roles filled, and is it clear who decides what?
- Do we know the biggest risks around scope, schedule, budget and dependencies, and do we have a plan B for the big ones?
- Does everyone share the same picture of what “done” means for the release, acceptance criteria included?
- Are the estimates based on capacity, experience and complexity, and do we update them when something shifts?
- Are we measuring actual progress, or just activity?
- Before go-live, are the release criteria, the rollout plan and the support model really confirmed?
These are not boxes you tick in passing. “Do we know our biggest risks?” is not something you answer with a click; you have to take an honest look. And if the estimation item is yellow, you know immediately what to talk about in the next meeting, instead of spending weeks with a nagging feeling that something is off.
Of course, a system like this depends on the ratings being verifiable. A green light on its own proves nothing. For important items we work with evidence and clear criteria, so a rating is more than a snapshot or a personal impression. That matters because people act on these ratings. A wrong green is worse than no rating at all. Those criteria, too, keep being sharpened with experience from real projects, so they become more meaningful with every assessment round.
And then AI enters the picture
AI is fundamentally changing how software gets built, and with it, what we need to look at when we talk about quality. It now supports many tasks in software development, from writing code and tests to reviews and even architecture decisions. What concerns us is whether the result is right for the business domain. Frankly, speed was never what we lacked, and by now it comes almost for free. Judgment does not.
Generated code has a well-known problem: it can look very convincing at first glance. It compiles, the tests are green, and the business logic is still wrong. That is the uncomfortable part. When one machine builds and another machine checks, both may end up with the same blind spot. Somewhere a human has to look at it and ask the questions no green test run can answer: Does this even fit our domain model? Are we watching the right risks? And sometimes the most mundane question is the most important one: Is this really done? AI can suggest an answer to each of these questions. It cannot own the answer. Whether something is right for this domain, this client and this project is a decision, and that decision stays with people. But judgments like that can only be made consistently across many projects if they are anchored to shared criteria. The more AI takes over the building, the more the judgment matters, and the more we need quality to be measurable and verifiable instead of assumed.
For development itself, this changes the role that quality plays in everyday work. It is no longer a stamp that someone from outside applies at the end. The criteria come from experienced developers, so they are written in the language the teams already think in. And because they are visible from day one, you can course-correct early instead of facing an ugly surprise two days before release. That, by the way, is also why I prefer to talk about quality engineering rather than testing. Testing is one part of it. What I mean is the foundation good software is built on. All the more so when a growing share of it is no longer written by hand.
After the first few months, one thing above all has become clear to me: as soon as quality becomes visible, the conversations change. We get to the issues that really need attention much faster. These criteria will probably never be finished. Practice is simply changing too fast for that. So we will have to keep revisiting them. For me, the real value of the approach lies elsewhere: quality becomes visible, and teams can talk far more concretely about what is going well and what is still missing. That conversation is where the quality work happens: people with shared criteria in hand, deciding what is right. No tool takes that decision off our plate, and I would not want it to. If you have tried something similar, tell me about it. I am genuinely interested.
The views expressed in this post are my own and do not necessarily reflect the position of my employer.