How Should a Boutique Investment Bank Test AI Data Isolation Between Client Mandates?

Define what isolation must prevent

Start with a written boundary. The question is not simply whether users see different project names. It is whether a user, model-assisted feature, administrator, integration, or export path can retrieve, infer, summarize, copy, or disclose information assigned to another mandate. Define the protected object as the mandate and list the records inside it: confidential-information, working papers, data-room files, call notes, draft materials, buyer research, and conversation history.

Write expected behavior for each role. A deal team member may need the records for Mandate A, while a different team may need Mandate B. A partner or operations administrator may have broader duties, but that access should be explicit and reviewable. NIST’s control catalog treats access control, audit, assessment, authorization, and monitoring as distinct assurance concerns, so a successful login is not evidence that the information boundary works. [1]

Map the data paths before testing

Draw a simple path from source to response. Include upload, indexing, embeddings or search, prompt construction, model context, generated output, download, email, API, backup, support access, and deletion. The goal is to identify where a cross-mandate result could arise. An application can enforce a screen-level permission while a search index, cached conversation, attachment preview, or export job follows a different rule.

Record the source of truth for each test record and its expected mandate label. Keep the test set synthetic and distinctive. For example, use invented names such as MANDATE-A-RED and MANDATE-B-BLUE in document text, metadata, filenames, and hidden properties. A distinctive marker makes accidental retrieval easier to detect without exposing client material. Maintain a manifest that records the record identifier, owner, permission, expected result, test date, and reviewer.

For a bank that uses a cloud service, include the provider’s documented responsibility split. FFIEC guidance recommends due diligence, ongoing oversight, contractual clarity about access rights and security responsibilities, inventories of systems and information assets, and regular testing of control effectiveness. [2]

Test retrieval and search boundaries

Run positive and negative tests. A user assigned to Mandate A should be able to retrieve the synthetic A record when the request is properly scoped. The same user should receive no B content when asking a broad question, searching for a B marker, or asking the system to compare both mandates. Test exact terms, partial terms, common client descriptors, and a request that omits the mandate name.

Test more than one interface. Repeat the cases in the ordinary search screen, an AI question box, a saved conversation, a document-summary action, and any API or batch process the team expects to use. Capture the complete response, not only the first sentence. A refusal can still leak a title, filename, count, snippet, or existence signal. Record whether the system explains the denial without disclosing protected metadata.

Use a second reviewer to classify each response as pass, fail, or unclear. Define fail before the pilot begins. Any B marker, B-derived fact, B filename, or unexplained cross-mandate reference in an A test is a failure, even if the model says it cannot open the underlying file.

Probe attachments and conversation context

Attachments need their own test set because a user may be allowed to open a file while the AI feature receives a broader context than the screen displays. Attach an A document, ask for a summary, then start a fresh conversation and repeat the request. Next, attach an A document and ask a question containing a B marker. Finally, test whether a conversation shared with a colleague changes the accessible scope.

Test persistence and deletion. After removing a synthetic B record or revoking a user’s access, repeat the same prompts and search requests. Define the expected timing for removal based on the vendor’s documented behavior, then treat any result outside that expectation as an issue requiring investigation. Do not infer that deletion from a user interface means deletion from indexes, caches, exports, backups, or provider-side support systems.

The accountable human reviewer should inspect both visible output and system evidence available to the bank. Relevant evidence can include access logs, document identifiers, permission changes, retrieval traces, export records, and timestamps. Do not ask an AI system to certify its own isolation.

Test exports, sharing, and privileged paths

A boundary that works in search can still fail at export. Test PDF, spreadsheet, text, bulk download, print, copy, share-link, email, and API outputs that the workflow permits. Look for hidden comments, document properties, prior conversation text, and generated files containing records from another mandate. Test an export created by a standard user and one created by an approved administrator, with separate expected outcomes.

Review role-based access control and least-privilege assignments. A role should grant only the data and actions needed for its job. Verify that deactivated users, contractors, service accounts, and support roles cannot use stale sessions or broad endpoints. FFIEC authentication guidance emphasizes risk assessment, identifying employees and third parties, periodic evaluation, layered security, and monitoring and reporting of activity. [3]

Keep privileged testing controlled. A bank should not give a general user administrator access merely to make a test convenient. If a vendor must perform a diagnostic, define the permitted scope, evidence to be returned, retention period, and approval owner before access is granted.

Compare vendor evidence with actual configuration

Ask the vendor precise questions rather than accepting a general statement that data are isolated. Request the architecture and control scope relevant to tenant or mandate boundaries, retrieval authorization, model context, training use, logging, support access, deletion, backups, subprocessors, incident notification, and independent assurance. Ask which claims apply to the purchased service and configuration, not merely to the provider’s broader platform.

Read the dates, scope, exceptions, and complementary customer controls in any assurance report. A report can provide evidence about a defined control period without proving that your configuration is correct or that every AI workflow is covered. FFIEC cloud guidance says management should understand the scope and sufficiency of independent assurance reports and may use provider tools, audit results, or testing to monitor controls. [2]

Reconcile the evidence to your configuration record. Note workspace or tenant identifiers, role definitions, sharing defaults, retention settings, connectors, model options, and administrative overrides. A separate workspace name is only a configuration label until a test demonstrates the actual boundary.

Set stopping criteria and remediation ownership

Before the pilot, decide when testing stops. Stop immediately for any cross-mandate content, unexplained access after revocation, an export containing another mandate, or a material mismatch between documented and observed behavior. Also stop if the bank cannot preserve enough evidence to investigate, cannot identify who can change permissions, or cannot determine how to report a suspected disclosure. Do not continue adding prompts to create the appearance of coverage after a critical failure.

For each issue, record the failing case, affected role, record marker, timestamp, configuration, evidence, severity, containment action, owner, due date, and retest condition. The owner might be information security, the application administrator, a mandate lead, procurement, or qualified counsel for a contract or regulatory question. The remediation checklist should distinguish a configuration fix from a vendor clarification, a contract change, a process restriction, and a decision not to use the feature.

Retest with the same case and a fresh case. A fix that blocks one exact phrase may not address a scope error in the underlying authorization layer. Keep the result as a dated test record that can be reviewed when roles, providers, models, connectors, or mandate processes change.

Make the record usable for human governance

A useful test record lets a responsible person answer five questions: what information was protected, who was allowed to access it, what was tested, what evidence was observed, and what happens next. Include the test owner, accountable reviewer, source-of-truth manifest, vendor evidence reviewed, configuration snapshot, pass or fail decision, exceptions, and remediation status. Keep client information out of the test whenever synthetic data can answer the question.

NIST describes its AI RMF as voluntary guidance for managing AI risks through design, development, use, and evaluation, rather than as a guarantee that a system is trustworthy. [4] Apply that posture here. The bank is verifying a control in a particular environment and time. It is not converting a vendor description, a certification label, or a successful demonstration into a permanent security or compliance conclusion.

Acquivo is a software platform that delivers the entire sell-side M&A process, valuation, marketing materials, buyer matching, and deal management, on one platform for business owners, M&A attorneys, and boutique investment banks. It is user-operated software, so the bank remains responsible for selecting information, configuring permissions, checking source records, reviewing outputs, and deciding when an AI workflow may be used.