A boutique bank should expand an AI pilot only when evidence shows that the bounded task works in the actual mandate context, a named deal-team reviewer can detect and correct material errors, the source records and permissions are understood, and review capacity remains adequate as volume grows. Compare the pilot with a defined baseline using the same task definition, record error types and overrides, test difficult and mandate-specific examples, and review results at an agreed cadence. Set thresholds before expansion: a quality floor, a maximum unresolved-error level, a reviewer-capacity limit, and stop triggers for confidentiality, unsupported claims, or unsafe output. If a threshold is missed, narrow, redesign, pause, or stop the workflow rather than treating a promising sample as proof of universal readiness.
Start with one task, not an AI ambition
Expansion decisions are clearer when the pilot has one observable job. Examples include extracting selected fields from a buyer questionnaire, comparing a draft company description with approved source records, or producing a first-pass list of missing diligence items. The task should state what the system may do, what it must not do, and what a human must decide. A broad goal such as use AI across more mandates cannot produce a meaningful pass or fail test.
Write down the mandate context before testing. A workflow that performs acceptably on one sector, document set, or preparation stage may behave differently on another. Record the intended users, inputs, output format, downstream use, and risk if the output is wrong. NIST describes this lifecycle framing through Govern, Map, Measure, and Manage, with the framework intended to improve how organizations incorporate trustworthiness into AI design, use, and evaluation. [1]
Define the evidence packet before the pilot starts
Every pilot should produce an evidence packet rather than a collection of favorable anecdotes. Include the task specification, tool version if known, prompt or instruction set, test cases, source records used, reviewer identity by role, reviewer decisions, corrections, unresolved questions, and dates. Distinguish an output accepted as-is from one edited or rejected. That distinction helps reveal whether the tool is producing useful assistance or simply shifting work into checking.
Use a representative set of actual workflow patterns, subject to confidentiality and permission rules. Include ordinary and deliberately difficult cases, such as incomplete records, conflicting figures, unusual terminology, stale materials, and documents containing information that should not enter the output. NIST’s Playbook calls for documented test sets and tools, performance measures in conditions similar to deployment, and written limitations on generalization beyond those conditions. [2]
Measure quality at the level of the task
Do not use a single AI accuracy score as the expansion decision. Define task-level checks. For extraction, check whether each field is present, supported by the source, and transferred without changing units or meaning. For drafting, check factual faithfulness, completeness against the approved source pack, clarity about uncertainty, and whether prohibited claims or unapproved disclosures appear. For classification or triage, check both missed items and unnecessary flags, then examine the cases behind each result.
Log error severity, not just error count. A typo in a non-material description is different from an unsupported financial statement, a wrong date, or a disclosure sent to the wrong audience. The reviewer should record the correction and its cause, such as missing source material, ambiguous instructions, retrieval failure, or generated content absent from the record. NIST recommends selecting metrics for significant risks, reassessing their effectiveness, and tracking unanticipated risks rather than measuring only convenient technical signals. [2]
Test reviewer capacity and independence
Human-in-the-loop is a control only if the reviewer has enough time, context, and authority to challenge the output. Assign the accountable reviewer by role, not by an unnamed deal team. State what must be checked before an output can enter a client-facing draft, buyer communication, valuation analysis, or diligence response. The reviewer should be able to reject the output, request a source check, and escalate a recurring issue without being measured only on speed.
Measure the review burden during the pilot. Track how many outputs require substantive edits, how long review takes in the chosen workflow, which errors reviewers miss on first pass, and whether a second reviewer is needed for higher-risk work. These are observations about the pilot, not promises of time savings. The FCA identifies capability reviews for human-in-the-loop functions, including training on system limitations, automation bias, independent judgment, and when to override or escalate. [4]
Verify source-of-truth and information boundaries
Before expansion, identify the records that control each output. A mandate may have an approved company profile, financial workbook, diligence tracker, management-approved facts, and an internal instruction set. The pilot should show whether the workflow uses those records, distinguishes current from superseded material, and surfaces a missing or conflicting source instead of filling the gap with plausible language.
Permissions must be explicit. Decide which documents may be processed, where outputs may be stored, which roles may view them, and whether a third-party tool is permitted for the information involved. A pilot should not be expanded because a tool handled public material well if the next mandate includes restricted or personal information. FINRA states that firms exploring AI should conduct their own due diligence and legal analysis, assess regulatory implications and risks, and establish mitigation measures. It also says AI use does not remove applicable securities-law obligations. [3]
Set expansion, narrowing, and stop thresholds
Thresholds make the decision repeatable. Before reviewing results, set a minimum acceptable task-quality level, a maximum level of unresolved material errors, a maximum reviewer-capacity load, and a required source-verification rate. Add qualitative stop triggers for an unsupported assertion in a client-facing draft, an apparent confidentiality or permission breach, an inability to reconstruct the source of a material statement, or a reviewer who cannot meaningfully challenge the output.
Use three outcomes rather than a binary launch decision. Expand only the same bounded task to a defined set of mandates when every required threshold is met. Narrow or redesign when the task works only for certain document types, sectors, or stages. Pause or stop when a material control fails, the reviewer cannot keep up, or the bank cannot explain what information entered the workflow. The FCA describes thresholds as pre-agreed trigger points for monitoring, escalation, intervention, or stopping, and warns against vague escalation criteria and long gaps between reviews. [4]
Run a controlled extension, not a blanket rollout
If the pilot passes, extend one variable at a time. The bank might add a second mandate type while keeping the task, reviewer role, source pack, and approval gate unchanged. Or it might extend the task within the same mandate pattern while retaining the same evidence checks. Record what changed so a later result can be attributed to scope, data, instructions, or operating conditions rather than treated as proof that AI works.
Keep a rollback path. A reviewer should know how to return to the prior manual process, preserve underlying source records, and quarantine questionable outputs. Continue sampling after expansion, including difficult cases and outputs that reviewers edited heavily. NIST calls for production monitoring, regular evaluation for safety and reliability, and approaches to identify existing, unanticipated, and emergent risks. [2]
Make the decision an accountable operating record
The final decision memo can be short, but it should answer five questions: what exact task was tested, what evidence was observed, which risks remain, who owns human approval, and what will cause the bank to narrow or stop. Attach the test-set summary, error log, reviewer-capacity observations, permission decision, and next review date. If the evidence is mixed, say so. A useful decision may be to keep a pilot limited to internal preparation, exclude certain mandates, or require a second review for a defined output class.
Acquivo is a software platform that delivers the entire sell-side M&A process, valuation, marketing materials, buyer matching, and deal management, on one platform for business owners, M&A attorneys, and boutique investment banks. It is user-operated software, so the bank remains responsible for selecting the workflow, controlling permissions, checking sources, and approving outputs. The objective is not to promise automatic staffing reduction or a universal result. It is to give the deal team a disciplined way to decide whether a bounded workflow has enough evidence to continue.
Key Points
- Approve a specific task and mandate context, not a general plan to use more AI.
- Compare outputs with a defined baseline and record accepted, edited, rejected, and unresolved results.
- Assign a human reviewer by role and test whether that reviewer has capacity and authority to challenge outputs.
- Use approved source records and explicit permission boundaries before sensitive mandate material enters the workflow.
- Predefine quality, unresolved-error, reviewer-capacity, and confidentiality stop thresholds.
- After a pass, extend one variable at a time and keep a rollback path with continued sampling.
Sources and references
Source retrieval dates record research access, not article publication. Read our editorial and date policy.
- [1] AI Risk Management Framework · Accessed
- [2] NIST AI Risk Management Framework Playbook · Accessed
- [3] AI Applications in the Securities Industry · Accessed
- [4] AI in financial services: good and poor practice · Accessed
Choose your next step
Acquivo is user-operated software. Owners and their professionals remain responsible for decisions, communications, and professional review.
Educational information, not individual legal, tax, or investment advice. Consult qualified professionals for your circumstances. Acquivo provides software, not representation or negotiation on your behalf.