The Hard Part of an AI Software Factory Is Teaching It How Your Systems Work

Writing code is becoming the easy part. The hard part is defining a system well enough that an agent can change it without quietly violating assumptions somewhere else.

I see plenty of demonstrations where an agent gets a ticket, edits a repository, opens a pull request, and passes a test suite. That is useful. It is also the cleanest possible version of the problem.

At work, we have hundreds of repositories. They do not all represent the same kind of thing. Some build deployable services. Some produce packages consumed by other teams. Some describe infrastructure. Some are old enough to contain decisions whose explanations have outlived the people who made them. They use different languages, release processes, deployment targets, and definitions of risk.

A generic coding agent can operate in all of them. That does not mean it understands any of them.

A template is not an architecture

Templates help. A common repository layout, test command, lint configuration, pull-request checklist, and deployment convention remove needless variation. They give agents a predictable front door.

But a template cannot explain why one service must remain compatible with a client two releases behind, why a library cannot add a dependency, why a schema change requires a staged rollout, or why a harmless-looking retry can multiply load on the system behind it. Those constraints belong to the architecture and to the operational history of the software.

That is the first real job in an AI-assisted software factory: turn the important parts of that knowledge into something the tools can use.

Some rules apply everywhere. Do not commit credentials. Run the tests. Preserve public interfaces unless the change explicitly includes a migration. Others only make sense for one repository or even one directory. A workable model is a shared baseline with progressively more specific overlays, not one enormous company prompt that every agent is expected to interpret correctly.

The rules also need owners. A stale instruction can be worse than no instruction because it gives a bad decision the appearance of policy. Repository guidance has to change when the architecture, deployment process, or business requirement changes. If nobody is responsible for that maintenance, the agent harness becomes another source of configuration drift.

The architect is building a product for engineers

Our software architect is constantly dogfooding new agent harnesses and building the toolchains that will eventually be handed to software engineers. That ordering matters.

The first useful question is not, “Which model writes the best code?” It is, “What does a safe unit of delegated work look like here?” The answer includes how context is assembled, which commands can run, where changes are made, what evidence must accompany a result, which checks are deterministic, and where a human must make the decision.

A good harness makes the expected path easy and the dangerous path conspicuous. It should not rely on a paragraph in a prompt to protect production credentials or stop an agent from modifying an unrelated repository. Those limits belong in the environment, permissions, and workflow.

This is platform engineering, whether or not a team calls it that. The platform is no longer only a deployment pipeline and a paved road for developers. It is also the operating environment in which agents receive work, acquire context, create changes, and return evidence.

Google’s 2025 DORA research describes AI as an amplifier of the organization around it, not a repair for weak delivery practices. It also identifies a high-quality internal platform as a key enabler for using AI at scale. That matches what I see: a capable agent makes a coherent engineering system faster, and an incoherent one noisier.

Every agent leaves a footprint

The code change is only the visible artifact. Agents also create worktrees, branches, temporary environments, package caches, logs, model traffic, test data, credentials, and compute load. Run enough of them concurrently and those side effects become an infrastructure problem.

Our infrastructure engineer is frequently involved in corralling that footprint. Isolation has to be real. Resource limits have to exist before one repository’s test suite consumes the host. Credentials need the narrowest practical scope. Temporary resources need names, ownership, expiration, and cleanup. Logs need enough detail to reconstruct what happened without becoming a second uncontrolled copy of sensitive data.

Concurrency changes assumptions too. A script that is harmless when one developer runs it twice a week may behave very differently when twenty agents discover it at once. Shared development databases, API rate limits, artifact repositories, CI runners, and staging environments all become contention points.

This is why “the agent passed CI” is not the end of the discussion. CI proves the things the pipeline knows how to test. It does not prove that the new behavior belongs in the larger system or that the process used to produce it was operationally safe.

Review can enforce a rule only after somebody defines it

We have been experimenting with Kodus as one part of the review side. Its Kody rules can express repository and path-specific expectations in plain-language files, and its reviewer can bring those rules into a pull request. That is closer to the problem than a generic review bot because the useful question is rarely, “Is this valid code?” It is, “Is this change valid here?”

Even then, the difficult work has only moved. Somebody must decide which architectural decisions should become enforceable rules, distinguish requirements from preferences, tune noisy checks, and retire guidance that no longer matches reality.

Cross-repository effects are harder. A pull request can be correct within its own diff and still break a consumer, duplicate a rule already implemented elsewhere, or change the timing and load seen by a neighboring service. Kodus is actively working on cross-repository context, which is a useful acknowledgment of the problem. No reviewer can infer an organization’s entire dependency graph from good intentions.

I do not expect one tool to solve that. I expect several layers: deterministic tests, static analysis, repository-specific review rules, dependency information, deployment checks, observability, and human judgment. The AI reviewer is useful when it directs attention toward the places where judgment is needed. It is less useful when it produces enough plausible commentary that engineers learn to ignore it.

DevOps knowledge moves closer to every engineer

As agents absorb more implementation work, software engineers will need a wider view of how their changes reach production. That means understanding build systems, deployment sequencing, feature flags, data migrations, observability, capacity, rollback behavior, and the services on either side of an interface.

This does not mean every developer becomes the on-call infrastructure specialist. It means the line between “my code” and “the system” becomes less defensible. If an engineer is supervising an agent that can modify several components in an afternoon, that engineer needs enough operational context to recognize when the proposed solution is locally elegant and globally wrong.

AI makes DevOps literacy more important because it increases the rate at which architectural misunderstandings can become working code.

The factory is the maintained knowledge around the agents

I would not measure progress by the number of pull requests an agent fleet can open or merge. That number is easy to inflate by changing the size of a task.

I care whether lead time improves without increasing failed changes, whether review attention moves toward meaningful decisions, whether rollbacks remain routine, whether incidents become easier to understand, and whether engineers can safely delegate more work without losing track of the system.

The model will improve. Code generation will get cheaper. Neither development changes the central problem.

A useful software factory is the maintained body of architecture, constraints, tooling, permissions, tests, and operational feedback that lets agents work without inventing the organization around them. Building that control system is the engineering work.

The broader shift behind this is why I think AI is making syntax less valuable without making software engineering less important.

Sources

• Google Cloud DORA, State of AI-assisted Software Development 2025.
• Kodus, open-source AI code review platform.
• Kodus, repository and path-scoped policy-as-code review.