Building Kodi Tax Agent: What If Tax Worked More Like a Modern Coding Agent?

Tax is something almost everyone eventually has to deal with, but the process still assumes that the person dealing with it understands far more about tax than they probably want to.

The documents may be in one place. The tax rules are somewhere else. Previous information may live in another system. Calculations have to be performed, evidence has to be checked, deadlines have to be understood, and missing information has to be resolved before anything useful can happen.

For an accountant or tax agent, this means constantly moving between documents, rules, calculations, systems, and conversations with the taxpayer.

For the taxpayer, it can be even more confusing.

You may know what you want to achieve, but not necessarily every step required to get there.

That is the problem I wanted Kodi Tax Agent to work through for the All Things Agentic Hackathon.

One of the most interesting changes happening in software development is the way people interact with coding agents.

With a capable coding agent, I do not always need to tell it exactly which file to open, which function to inspect, which command to run, and what to do immediately after that.

I can give it an objective.

The agent can inspect the environment, understand what information is available, use the tools it has been given, observe the results, and determine what to do next.

That interaction is very different from traditional software where the human has to drive almost every step.

I wanted to explore what that same idea could look like in tax.

Instead of giving Kodi a long sequence of instructions, the interaction begins with a tax objective.

Kodi then works through the case.

The goal is not to build an AI that simply knows a lot about tax.

The goal is to build an AI agent that can work on tax.

The central concept behind Kodi is something I call the Tax Case.

A coding agent has a repository. Kodi has a Tax Case.

The Tax Case is the durable working state of the tax problem. It contains the facts currently known about the taxpayer, the evidence supporting those facts, unresolved questions, tasks that need to be completed, computations, validations, generated artifacts, and the current version of the case.

That distinction is important.

The model's conversation context is not the source of truth.

The Tax Case is.

Every time Kodi needs to continue working, it can reconstruct the current state from durable data rather than depending on an AI model to remember what happened several turns ago.

This makes the system much more useful for workflows that may take time, require clarification, involve document processing, or need to recover after interruption.

If the browser is refreshed, the case does not disappear.

If the user answers a question, that answer becomes part of the case.

If evidence is discovered, it is persisted before the agent plans again.

The system therefore follows a pattern closer to:

act, observe, persist, reload, reassess, continue.

Kodi is entered in the Collaborative Partner category because the system is not designed around either extreme.

It is not a chatbot where the human has to manually tell it every next action.

It is also not an agent that assumes it should make every decision without involving the taxpayer.

Kodi works autonomously where it has enough evidence and an approved capability.

When it reaches information that cannot safely be inferred, it asks.

Consider tax residency.

Whether someone was resident in Kenya during a particular tax year can materially affect the tax case. A document may provide clues, but clues are not always enough to make that determination safely.

Kodi should not simply guess because a language model is capable of producing an answer.

Instead, the system can identify that the fact is unresolved and create a focused clarification question.

The taxpayer answers.

That answer is stored as a durable fact with its source recorded as a user statement.

The case version advances.

Kodi reloads the new state and continues automatically.

The taxpayer becomes part of the workflow when human knowledge is actually required, not because the system needs someone to approve every small technical action.

That is the collaboration model I wanted.

Kodi does the work, while knowing when it needs you.

Tax is an area where a confident answer is not enough.

A number can look perfectly reasonable and still be wrong.

For Kodi, evidence therefore has to be part of the architecture rather than something added at the end.

When a tax document is uploaded, it does not simply disappear into a language model and return as tax truth.

The document moves through a controlled processing pipeline.

The source document is stored privately. A Document AI worker processes it. Gemini performs document understanding and produces structured observations. Kodi validates those observations and converts them into its canonical document representation. Evidence derived from that representation can then be connected to the Tax Case.

Retrieval embeddings are created separately to support semantic search over the document content.

The important part is that the original source and the evidence Kodi later uses remain connected.

That allows the system to preserve a chain such as:

source document → structured observation → canonical evidence → case evidence → case fact → computation → validation → output

That lineage makes it possible to ask a simple but important question:

Where did this tax result come from?

For a tax system, I think that question should always have an answer.

A major architectural decision in Kodi is that the model is not allowed to become the authority for everything.

Gemini provides semantic intelligence.

Kodi provides control.

The autonomous Agent Runtime uses Google ADK together with Gemini 3.7 Flash through Vertex AI to reason over a bounded representation of the Tax Case and propose the next action.

But proposing an action and being allowed to execute it are two different things.

Kodi validates the proposed action against the current case state, available evidence, case version, allowed tool contract, identity information, and autonomous capability policy.

Only then can the action move through the governed tool gateway and into the wider system.

The same separation appears elsewhere.

Gemini can interpret a tax document, but Kodi decides how the output becomes canonical evidence.

Gemini can extract semantic claims, but Kodi validates the resulting structure.

Gemini can help determine the relationship between two ambiguous claims, but deterministic comparison runs first.

Gemini can synthesize a user-facing response, but Kodi still performs citation and integrity verification before returning it.

Tax calculations remain deterministic.

Tax validation remains deterministic.

Identity, authorization, durable state, retries, tool execution, evidence lineage, and auditability remain controlled by the application.

In simple terms:

Gemini provides intelligence. Kodi provides authority.

Another decision I made was not to use one model for every task.

Different workloads have different levels of complexity, and using the strongest available model for a bounded classification problem does not automatically make the architecture better.

Kodi therefore uses different Gemini 3.5+ models depending on the job.

Gemini 3.7 Flash handles autonomous Tax Case planning. This is the most reasoning-heavy part of the current system because the model has to inspect bounded case state and determine the next permitted action.

Gemini 3.6 Flash handles document understanding and grounded response synthesis. These workloads require more semantic depth and, in the case of document understanding, multimodal processing.

Gemini 3.5 Flash handles ambiguous claim-relationship judgment where deterministic logic cannot safely determine whether two pieces of evidence agree, contradict each other, qualify each other, or relate in another way.

Gemini 3.5 Flash-Lite handles bounded workloads such as conversation-turn resolution and canonical claim extraction.

The principle is simple:

Use the stronger model where the reasoning genuinely requires it.

Use a lighter model where the task is constrained.

All of those generation paths run through Vertex AI using Google Cloud authentication.

OpenAI remains in the system only for embeddings used in retrieval.

Kodi contains more capabilities than the current autonomous demo is allowed to use.

The broader platform includes tax knowledge retrieval, deterministic tax computation, validation, form generation, report generation, and document capabilities.

But having a tool in the repository does not automatically mean the agent should be free to use it.

For the current Tax Case implementation, the autonomous policy is deliberately narrow.

The active autonomous tool family is Document AI.

Kodi can inspect document processing state, ground conversation documents, search document evidence, retrieve evidence, and derive evidence through the governed execution path.

Other capabilities are implemented or registered but are not automatically enabled for autonomous use in the current demo.

I think this matters.

A useful agent architecture is not just a long list of tools connected to an LLM.

The important question is what the agent is allowed to do, under which conditions, with which evidence, and how the system behaves when something goes wrong.

One thing I deliberately did not want to do for the hackathon demo was create a fake perfect ending.

The current autonomous Tax Case slice does not pretend that every uploaded document contains everything needed to complete a tax return.

The demo can create or resume a durable Tax Case, work with document evidence, identify missing requirements, ask a focused residency question, persist the taxpayer's answer, resume automatically, and reassess the case.

If the next required financial fact, such as employment income, is not actually supported by the available evidence, Kodi stops.

It does not invent the figure.

It does not silently fill the gap because the demo would look more impressive with a completed return.

It creates a safe blocker.

For me, that is not a failure of the agent.

That is part of what makes the agent useful.

An AI system working in tax should know the difference between "I can continue" and "I do not have enough evidence."

The autonomous runtime is built in Python using FastAPI and Pydantic.

Google ADK provides the agent framework, while the Google GenAI SDK provides the direct Gemini integration for bounded semantic workloads.

Gemini models run through Vertex AI.

The deployed hackathon services run on Google Cloud Run.

PostgreSQL 15 on Cloud SQL provides durable state for the Tax Case and other backend data.

The Document AI worker runs as a Cloud Run Job.

Cloudflare R2 stores private source document objects.

The frontend is built with React, TypeScript, and Vite.

An Event Store provides append-only audit support alongside the durable PostgreSQL state.

The important architectural point is not simply the list of technologies.

It is how their responsibilities are separated.

The AI model should not become the database.

The conversation should not become the workflow engine.

The workflow engine should not become the tax authority.

The document model should not become the evidence store.

Each part of the system has a boundary.

One of the strongest lessons from working on Kodi is that making an application "agentic" is not mainly about allowing a model to call tools.

The difficult part is everything around that decision.

What state is the model allowed to see?

Which actions are currently permitted?

What evidence supports the action?

How is the action validated?

What happens if the tool fails?

What gets persisted before the next planning step?

What happens if the user refreshes the browser?

What happens if the information is ambiguous?

When should the system ask a human?

When should it stop?

Those questions become even more important in tax because the consequences of inventing a fact can be much greater than the consequences of giving an imperfect general answer.

The architecture therefore became less about giving Gemini more authority and more about giving Gemini useful intelligence inside strong boundaries.

The current hackathon implementation is intentionally only one slice of a larger direction.

The broader Kodi platform already contains tax computation, validation, tax knowledge, forms, reports, storage, and other capabilities.

The next challenge is not simply switching all of those tools on.

The challenge is expanding autonomy carefully.

A future Kodi should be able to understand a taxpayer's objective, gather the necessary evidence, determine the applicable tax requirements, calculate using deterministic tax logic, validate the result, prepare the appropriate artifacts, identify unresolved issues, and involve the taxpayer or professional only where required.

And every meaningful step should remain traceable.

That is a much more interesting direction than building another chatbot that can answer tax questions.

Tax compliance is already complicated enough.

People should not have to understand the entire tax system before they can begin dealing with their own tax obligations.

The interaction should be able to start much closer to how people naturally think:

Tell Kodi what you need to achieve. Give it access to the information it is authorised to use. Let it work through the tax case with you.

That is the direction behind Kodi Tax Agent.

I created this piece of content for the purposes of entering the All Things Agentic Hackathon.