AI transformation in an IT project, stage 1: baseline assessment

Before reviewing the repository, establish how the team uses AI while working on this project. In conversations before rollouts, I often hear that the tools are familiar and used every day. Measurement can still show zero or sporadic activity. That need not be deliberate deception. A single chat session or work through a personal account may be enough for the answer "I use it" to feel honest.

That is why a baseline assessment should not rely on declarations alone. They need to be compared with verifiable traces:

• Licence usage statistics. Tools managed by the company often provide usage dashboards. Licences untouched for weeks make it possible to compare declared familiarity with actual activity.
• Requests for tools. Genuine interest leaves traces: access requests, questions about subscription budgets, approval requests for new tools. The absence of such requests is a signal to check alongside usage statistics and conversations with the team.
• A repository review. Working with agents can leave traces in the code: configuration files, versioned context instructions or patterns visible in the change history. Their absence does not prove that no agent was used. It does mean the repository has no versioned workflow that can be confirmed and reproduced.

A gap between declared and measured usage does not prove that the team lacks experience. It is a reason to replace a refresher presentation with a shared task in this repository and observe where the workflow stops.

The second assessment area is the project itself. The point is not the age of the technology or the "legacy" label. The point is how much the system explains about itself and how much work it takes to run it.

Technical debt is best judged by predictability, not by age. Older code with consistent conventions, clear module boundaries and stable interfaces can be easier for an agent than a young project written chaotically. The diagnostic question is: after reading a few modules, can you predict what the rest looks like.

Documentation and informal knowledge determine what an agent can know at all. An agent reads the repository, not the team's memory. Every rule that exists only in people's heads or on private channels is invisible to it. The assessment should identify which decisions exist as code, tests or documents, and which still have to be written down.

The development environment is the simplest practical measure. If a new person needs a week and the help of two colleagues to run the project locally, an agent will stop at the same threshold. The onboarding time of a new team member is a good approximation of the cost of onboarding an agent.

This area also helps test the popular belief that a system is "too hard for AI". I treat it as a hypothesis, not as a fact. In practice, the reported difficulty often does not come from the domain itself. It can be the sum of gaps that can be named and removed one by one: missing configuration, undocumented commands, knowledge outside the repository.

Static checks, tests, and a strong, fast CI are practices considered sound for decades, yet often perceived as excessive demands. The reason is simple. A human compensates for a weak feedback loop with patience, memory and experience. An agent does not have access to the same informal workarounds, so check results are its primary information about the state of the system.

The state of the loop has to be measured in numbers, not adjectives:

• Full CI run time. A run measured in hours does not disqualify a project, but it sets the first goal. In projects I have observed, cutting it by half or two thirds was realistic. For a new system, I treat 5–8 minutes as a practical budget for a full run and around 4 minutes as an ambitious target. This is a measurement reference, not a universal standard for every stack and test scope.
• Unstable tests. A test that sometimes passes and sometimes fails has to be treated as a blocker to remove, not an inconvenience to re-run. A human will run it again and move on. An agent will read the random result as a consequence of its own change and start fixing code that works.
• Local runnability. The key checks must work after a single command, locally, without a hand-built environment. Every step that requires knowledge from outside the repository stops an agent in the middle of its work.

The output of this area is a list of loop gaps with a rough cost of removing each one. This is usually where it is decided whether the first quarter of the transformation delivers product results or becomes a quarter of cleanup.

Skills can be measured with a trial task. Attitudes are harder and more important, because they decide whether the tools will still be used after the rollout ends.

Behind declarations of universal usage there are usually two groups. The first are people who genuinely work with AI but do not talk about it openly, fearing the opinion that a tool writes their code for them. The second are skeptics who made one attempt without any configuration, got a weak result and considered the matter settled. The two groups reinforce each other: the quiet practitioners stay silent, so the only public narrative is the failed experiment.

In my experience, a leadership announcement alone rarely changes this balance. A demonstration on site, in this specific system, performed by someone with a strong technical position has more influence. It provides a result to assess where arguments remain abstract. Such a moment can create temporary tension, especially with people who publicly defended the position that it cannot work here.

A few rules hold up when building consensus. In the rollouts I have observed, unanimity was not a condition for starting. They needed a group of practitioners and a majority willing to judge the results of trials. Demonstration works better than persuasion. If the organization has actually decided to proceed with the transformation, communication should distinguish that direction from the elements still open to discussion, such as pace, tools and safeguards. Questions deserve substantive answers, including an open account of the areas where AI fails. If an employee refuses to use approved tools despite agreed rules, that is an organizational issue requiring a manager's decision, not an ideological dispute for the team forum.

Even a well-prepared repository is not enough when the team does not know the rules for working with code and data, or nobody can accept the cost and risk. This part of the assessment covers the organizational conditions that directly constrain agent work in the project.

The quality of the team's input. An agent's output depends on the quality of its input. If requirements are vague and tickets need clarification at every step, that ambiguity will also carry into the proposed code. Reviewing a dozen recent backlog items says more about process readiness than any declared methodology.

Data and the rules for using it. Before the start, it must be clear what may be passed to a model: code covered by contracts, customer data, contractual clauses, regulatory requirements. A missing policy can stop a rollout even when the project is technically ready, because a responsible team facing ambiguity will choose not to use the tools.

Sponsor and budget. The assessment should identify who decides about the transformation, who funds the licences and the environment work, and what level of risk the organization accepts. Without this, a baseline report remains a document without an addressee.

Metrics. If the organization currently measures neither change lead time nor delivery stability, it will struggle to demonstrate the results of the transformation reliably. Establishing even a simple reference point is part of this stage, because measuring results later in the series needs something to come back to.

The findings have three layers. Project condition shows what an agent can understand and verify on its own. Team practices determine whether the new workflow continues after the first trial. Organizational conditions set the boundaries for data, contracts, budget and accountability. The scope and pace of the first change must fit all three.

A project with a strong feedback loop, for example with stable end-to-end tests covering the behaviour being changed, allows the scope of agent work to expand faster. Automated checks can then catch some mistakes earlier than manual review. The pace should increase in proportion to the coverage, stability and clarity of those controls.

A project in poor shape reverses the order but does not cancel the direction. AI then starts with cleanup and strengthening: shortening the CI run, stabilizing tests, filling in static checks, writing down knowledge available so far only in people's heads. Adding more features to a disordered system grows the very debt that limited the pace in the first place.

Whatever the pace, a good first task can be a substantial but well-bounded backlog item: work postponed for months for lack of time, with clear acceptance criteria and a reversible outcome. Such an opening can improve morale, because the team sees the change working in its favour, and it provides concrete evidence of usefulness.

A few control questions close the assessment:

• Does the repository run locally after one command, how long does a full CI run take, and which tests are unstable?
• How does the team use AI on this project, and can it complete a trial task together?
• Which code and data may be sent to models, who funds the tools, and who accepts the risk of the change?

The answers do not produce a "ready or not ready" score. They define a safe scope for the first change and the gaps to close before agent work expands. That leads into the next stage of the series: strategy and choosing your AI tooling.