Part 3. From adoption to impact: testing AI in real development

From adoption to impact: testing AI in real development

In the second part, we talked about how we approached AI adoption at Attico. It was necessary to understand whether AI could actually make our developers more productive and what kind of impact it would have in our specific environment.

A study published in Management Science showed that there is real potential  —  developers with access to an AI coding assistant completed 26% more tasks on average. At the same time, the results were quite different across the experiments.

For Attico, this was an important signal: we can’t just take someone else’s numbers and expect the same results from our team. We needed to test how AI works with our projects, our processes, our engineering standards, and our quality requirements.

Choosing the team, projects, and tasks

Our work went through three stages:

Setting the goals and success criteria

One of our key hypotheses was simple: AI should significantly reduce development time. We set an initial target of around 30%, which we considered fairly conservative given that industry reports and developer stories mention anything from 10% to 50% or more.

But time savings weren’t the only measure of success. We also wanted to see whether AI could handle more than small, isolated tasks, such as writing a script or generating a piece of code, and actually help complete a full task a developer would normally work on. The result had to be useful in a real project, not just look good in a simplified example.

We didn’t try to run a full scientific experiment with a control group, because that would have meant doing the same task twice  —  manually and with AI  —  which would create extra work and make little sense in real development.

Instead, our AI adoption impact measurement was based on actual project work and developer feedback. The effect varied by task: in some cases, developers estimated that the time was cut in half, and in others, by roughly three times.

So, we can’t say that the 30% hypothesis was strictly proven experimentally. But from a practical and management perspective, the main idea was confirmed: for the right types of tasks, AI can both take on meaningful parts of real development work and noticeably reduce completion time. That was enough for us to keep developing the approach and test it on a larger scale.

Foundation for wider AI adoption

In R&D, adoption happened gradually: first one developer got access, then a few more. Knowledge spread naturally, and there was usually someone who already understood the tools and could help others get started.

That works at a small scale, but not across multiple teams. If every new user needs individual guidance on model selection, task setup, and workflow, onboarding quickly becomes a bottleneck. So, before the wider rollout, we formalized the practices the R&D team had developed.

We documented which models work best for different tasks, how to provide context, structure a task, and break the work into steps. We also described a project analysis stage where AI reviews the codebase first and builds a short working context around key files, components, and their roles, so the project doesn’t have to be explained from scratch every time.

Security was another essential layer. We introduced an AI policy covering allowed use cases, data that must not be shared with external services, and other restrictions. The same documentation explains how to prepare a project for AI, including removing API keys, passwords, and other secrets from source code, even in legacy projects where they may have existed for years.

We also added practical examples. For one real Attico Platform task, we documented the full process: the prompts, the AI responses, where the developer adjusted the approach, and how the task was completed. That can be more useful than a list of recommendations for someone new to this way of working.

The onboarding materials also include a detailed Cursor guide, instructions for installing our tools, and a webinar explaining how the system and its components work. In parallel, we prepared skills and agents for backend, frontend, and Cypress end-to-end testing.

As a result, onboarding became much more predictable. Most basic questions are covered before developers start using the system, and the feedback we now get is usually more specific  —  for example, requests for missing features or suggestions for improvement rather than basic questions about how to use the tools.

For us, that’s a good sign that the foundation is working and that we can scale AI adoption without having to manually support every new user.

Attico’s approach to tools and model selection

We chose Cursor as our main development environment because it lets us switch between different AI models. Since the market changes quickly, we didn’t want to depend on a single vendor or model.

We also choose models based on the stage of the task. Planning requires a stronger model that can understand context and project requirements, while execution based on an agreed plan can often be handled by a simpler one. This helps balance quality, speed, and cost.

Another tool we use is Context7, an MCP server that gives AI access to up-to-date technical documentation. It allows agents to check information about Drupal modules, APIs, and third-party tools instead of relying only on what the model already knows, reducing the risk of outdated or incorrect answers.

At the same time, we standardize only what needs to be consistent. Everyone follows the same security rules and basic workflow principles: for example, personal data and credentials must not be shared with AI, and planning and execution should be treated as separate stages.

Beyond that, developers have flexibility in how they provide context, structure tasks, and work with AI. Different projects may also require different security, context, or review approaches.

So, our principle is simple: standardize what affects security, quality, and predictable results, while leaving enough flexibility for teams to adapt and improve the workflow.

Engineering standards for AI

Working with AI made it especially clear that many engineering rules exist only in the heads of experienced developers. These things may seem obvious to a person, but AI needs them to be clearly explained.

So, alongside the experiment, we also started to formalize our coding standards and development process. This includes Drupal Coding Standards, common approaches to typical tasks, architectural principles, and practices that the team has developed over the years.

The goal is not to describe every possible situation. It’s to define the key principles that make the difference between code that simply works and a solution that meets our quality and architecture standards.

AI pushes us to turn the engineering experience we’ve built up over time from implicit knowledge into clear, reusable rules that an agent can follow.

What the experiment showed: AI adoption impact

We didn’t collect formal metrics as part of a controlled study. Instead, we based our evaluation of the AI adoption impact on interviews with the developers who took part in the experiment. They shared where AI helped, where it caused problems, and how it affected their work overall.

Based on their estimates, development time was reduced by around 30–70% on tasks where AI was a good fit. The sample was small, so we don’t see these numbers as a universal benchmark.

The main takeaway was practical: the team saw the benefits of AI adoption in real development, and the developers confirmed that AI could be used not only for small, individual tasks, but also for real development tasks.

That was enough for us to move from the R&D experiment to a wider rollout on commercial projects.

Attico’s next steps in AI adoption

We’ve used AI on our first commercial projects for about two months, and now our focus is shifting from rollout to further improvement.

We want the system to need less developer input: better understand project details during planning, make fewer mistakes, and require fewer changes.

At the same time, we’re expanding AI use in development and exploring its use in QA.

Another focus is metrics on commercial projects. They should help us measure both the technical impact and business results.

We’re also improving onboarding, turning common questions from developers and project managers into clearer documentation and a simpler process for new team members.

Defining success for the current stage

Looking at roughly one year from when we started this research, success at this stage would mean AI is widely used in developers’ daily work.

Not every team needs to use it in exactly the same way. What matters is that developers feel confident working with AI and follow the principles we defined for our AI-enabled organization.

At the same time, we don’t expect the transformation to be “finished” in a year. AI is developing too quickly, so we’ll need to keep adapting our tools, processes, and ways of working. The target model may look a bit different a year from now.

The same applies to other areas like QA, project management, DevOps, and other roles. We’re not setting one start or end date for their transformation yet. Each area will move at its own pace as we find the right use cases and prove their value.

Last but not least

AI is already giving software businesses an advantage in delivery speed, efficiency, cost, and margins. So companies are no longer competing just on whether they use AI, but on how effectively they use it.

This is also changing what we expect from developers: knowing how to work with AI is becoming part of the job.

At the same time, it’s not about using AI for every task. What matters is knowing where it actually adds value and how to use it without losing quality.

Previous: learn how we engaged the team and built in-house AI expertise before the rollout in Part 2: From AI strategy to adoption.

Start from the beginning: explore where AI creates value and the principles guiding our approach in Part 1: Vision and principles.

Article Authors