What I learned converting an Amazon Bedrock prototype into a governed financial workflow

A finance team could connect reporting data to an LLM and produce management commentary within days. Getting that commentary into a management pack is a different problem.

Before that happens, the AI must address three fundamental questions. 

1. Where did these numbers come from? 

2. What did the model add to itself? 

3. Has an analyst reviewed and approved the conclusion?

I came across this issue while building an AI-assisted FP&A prototype using Python, PostgreSQL, Streamlit, and Amazon Bedrock.

The prototype worked. An analyst could select a reporting date range, calculate the KPIs, and then ask the application to explain what was happening in the numbers.

What surprised me wasn’t simply that the model could produce commentary. The output was actually quite close to the way I already think about management reporting.

From an IT perspective, this may appear to be a successful proof of concept. For finance, however, this is where the more complex questions begin.

A Useful Answer Is Not Necessarily a Controlled Answer

The prototype is based on synthetic insurance-finance data and has separate workflows which calculate each of the individual KPIs, including product mix, new business written, gross written premium, customer volume, average premium, business mix, and performance against plan and forecast.

The application then takes the relevant financial context and sends it to Amazon Bedrock to generate management commentary.

In one of the demonstration runs, the system worked through the premium data and arrived at a total of about $177.4 million, roughly 22% below plan. New business was around $52 million, while renewals accounted for approximately $122.6 million.

The useful part wasn’t only the calculation. The system also produced a variance summary, organised the supporting information, and highlighted several questions that still needed an analyst to review before anything could be signed off.

The commentary made sense, but could I actually control it?

A few questions came up:

1. Did the LLM calculate the 22% variance, or was that calculation already done elsewhere?

2. Does the model really understand what gross written premium means, or is it simply generating plausible language around the numbers it was given?

3. Exactly what information was sent to the model?

4. Can an analyst trace the numbers in the commentary back to their source without having to reconstruct the whole process?

Where the First Version Broke

The first version broke down at the point of verification.

In one early run, the application produced a convincing explanation of a roughly 22% unfavorable premium variance. The narrative itself looked reasonable; the output didn’t really make sense and did not make it clear whether that percentage came from a deterministic calculation upstream or from the model’s interpretation of the figures it received.

Tracking the number back to its source also required manually reconstructing the workflow.

The problem was not that the answer was obviously wrong. It was that I could not prove, quickly and consistently, why it was right.

That was the point at which the architecture had to change.

The challenge was not generating commentary, but determining the appropriate scope of the model’s responsibilities.

The model explains. It does not calculate.

The numbers used in the commentary need to come from a source that an analyst can independently check. The model should not be responsible for deciding what a KPI means, calculating its value, or approving the result.

If that boundary isn’t clear, you can quickly end up with a black box that happens to sound like a finance analyst.

That led me to redesign the prototype around five control areas.

The Five-Control FP&A AI Framework

1. Source Control

The first question is a security question: what data is the model actually allowed to see?

The model shouldn’t have unrestricted access to the data warehouse or to the underlying financial records.

Instead, the analyst selects the reporting period, scenario, and workflow. The application then retrieves only the structured, approved financial data that the specific task requires.

Take a case where the task is to explain KPI performance against the plan. The model should not draw on its parametric memory to introduce another forecast figure, external information, or an alternative basis for comparison. What it explains has to come from the data the workflow handed it.

That changes the question a CFO needs to ask. “Which LLM are we using?” turns out to be the narrow question. The one that matters is what underlying data the model can access.

2. Calculation Control

I wanted the financial calculations to stay outside the LLM.

Python and SQL handle the KPI calculations, actual-versus-plan variance, prior-period trends, reporting-period filters, materiality checks, and the price-volume decomposition. All of it is computed before the model receives anything.

The model never touches the underlying data. It works only from figures that have already been calculated, so it is never in a position to originate a number.

That matters because a language model is probabilistic. It can produce a convincing explanation with no guarantee that it would return the same figure twice from the same filters. Financial reporting needs the opposite property: the same period, the same scenario, and the same filters should produce the same number every time.

For financial planning, I want the sequence fixed. Calculate the number, review it, then explain it.

3. Context Control

Getting the calculation right isn’t always enough.

The model also needs enough context to summarise what the number means for the business.

The Python orchestration layer builds that controlled context. It holds the top-line KPI values, plan and forecast comparisons, drivers, trends, and metric definitions, along with vendor information, polarity, and the relevant business rules.

The point is not to give the model everything. It is to give it the context that the specific reporting task needs.

In a future scope of work, the model would draw on a larger federated library containing official finance documents, the KPI glossary, reporting templates, and previously approved management commentary. Vector retrieval would help locate the right material, but the retrieval step would stay inside the governed workflow, so the model never reaches a document that sits outside the approved library.

4. Generation Control

Once the context reaches Amazon Bedrock, I want the model to have a precise role.

Its task is to explain the financial results, organise the insights, and turn approved numbers into management commentary.

The guardrails are equally clear. It should not create its own KPI definitions, modify the data feed, authorise any number, or present a fictitious figure confidently.

Take a case where the model shows a KPI below plan, but the data does not establish the reason for the decline. Here, the model should not offer an explanation. It should hand the question back to the analyst.

For me, going into a meeting with that question still open is better than going in with a confident explanation.

5. Approval Control

The finance professional has the final call.

The data layer supplies the records; Python and SQL calculate the KPIs; the analyst selects the context and reporting period; and Amazon Bedrock generates the management commentary. Even then, that commentary is not final.

This is where the human step comes in. The analyst reads what the model produced, confirms that the underlying numbers are approved, and decides whether the commentary is ready for management.

That gives the analyst real authority: to accept the commentary, or to reject the interpretation before it becomes part of the management pack.

From Prototype to Regulated Enterprise Use

This is a governed MVP, not a production-level enterprise platform.

A broader deployment would require additional controls around access, monitoring, and change management: multi-factor authentication, role-based access, segregation of duties, QA testing, persistent audit logs, incident procedures, model versioning, and formal control testing.

Those controls can support a SOX-relevant environment. They do not, on their own, make an application SOX compliant. The system still has to operate inside the organisation’s internal control framework for financial reporting.

Those controls can support a SOX-relevant environment. On their own, though, they do not make an application SOX-compliant. The system still has to operate inside the organisation’s internal control framework for financial reporting.

My view on healthcare is the same. The prototype uses synthetic data. It does not process protected health information, and I am not claiming HIPAA compliance. In healthcare, a platform of this kind would need stricter controls around auditability, data integrity and transmission security, plus stronger authentication, documented risk management and the necessary business associate arrangements.

Amazon Bedrock is HIPAA eligible, and its assurance reports can support an architecture of that kind. But compliance depends on the access controls, the architecture around the platform, and how the organisation actually uses.

Where Finance Must Stay in Control

Generating the management commentary was the easiest part of the prototype. Which model to use was an important question, but knowing exactly what the model was fed was equally important.

What I was after was calculations that were deterministic and could be verified independently, alongside a controlled data environment, a context that had a purpose, and boundaries that were clear to the model. Above all, the final call stayed with the analyst.

The model can take information and turn it into useful language for financial reporting, but audit, review, and judgment remain with the analyst.
 

Implementation note: Readers interested in exploring the prototype in more detail can view Adinath Kadam’s public GitHub repository. The repository uses synthetic data only.