How Should M&A Attorneys Evaluate an AI Vendor Before a Deal Pilot?

Define the task and build an evidence-based scorecard

Write a pilot brief before requesting a demonstration. An illustrative task is extracting assignment and change-of-control language from a small set of synthetic contracts into a draft issue log. Identify the accountable deal attorney, authorized users, permitted input categories and intended output. Exclude legal conclusions, external disclosures, negotiations and approvals. The original contract versions remain the source of truth; the AI output is a working aid, not a replacement record.

Create scorecard entries for processing terms, training or reuse, subprocessors, access controls, retention, export, incident handling and testing. For each entry, record the requirement, dated evidence, relevant clause or document section, applicable product tier, reviewer, unresolved gap and decision. Use statuses such as verified for this pilot, conditional, unresolved or unacceptable. Do not let an aggregate score conceal a confidentiality gate that has not been met.

Ask the responsible attorney and qualified professional-responsibility or privacy counsel to assess applicable rules, engagement terms and disclosure permissions before client information enters the tool. ABA Formal Opinion 512, issued July 29, 2024, interprets the ABA Model Rules rather than every jurisdiction's binding rules. It treats tool understanding, confidentiality and supervision as professional responsibilities. [1]

Separate processing permission from training and secondary use

Request the proposed agreement, processing addendum, privacy policy and product-specific AI terms. Ask the vendor to trace what happens to uploaded documents, prompts and outputs, including extracted text, search indexes, support copies and diagnostic logs. Record why each category is collected, who can use it, where it is processed and which contractual language governs that use. A sales answer is useful for identifying questions, but it is not the same evidence as the applicable contract.

Ask separately whether content can be used for model training, fine-tuning, evaluation, human review or improvement of shared services. Clarify defaults, optional settings, exceptions and upstream provider treatment. A statement that data is not used for training leaves these other purposes unanswered. Verify that any required restriction covers the selected trial configuration, not just a different enterprise offering.

The ABA opinion recommends understanding terms of use, privacy policies and contractual terms. Its confidentiality discussion warns of disclosure risks from self-learning tools and explains that, when informed consent is required, boilerplate authorization is insufficient. The responsible attorney must determine the applicable jurisdiction's requirements for the particular use; consent should not be treated as a cure for every unresolved processing risk. [1]

Identify subprocessors and changes in the processing chain

Request a current inventory of third parties that handle pilot content. Subprocessors are downstream providers involved in processing that content. Ask about the model provider, hosting, document conversion, monitoring and support services where relevant. For each provider, seek its role, data categories, processing locations and retention arrangements. Confirm whether an optional feature introduces a different provider or changes the route that documents take.

Ask what notice accompanies a subprocessor or model change, what choices the firm has, and how downstream restrictions are maintained. Qualified privacy counsel should assess any cross-border processing issues under the laws and contractual obligations actually applicable to the matter, rather than assuming a location alone settles suitability.

NIST's July 2024 Generative AI Profile recommends inventorying third parties with access to organizational content and incorporating embedded AI, privacy and security into procurement diligence. It is voluntary risk-management guidance, not a certification or proof that a vendor has satisfied those controls. [2]

Test access restrictions in the actual trial environment

Have the firm's IT or security reviewer examine identity controls, role permissions, matter separation and vendor support access. Request documentation and a demonstration of user removal, administrator privileges and relevant access records. Ask how access is restricted across search, generated answers and exports, not merely the document list. Establish who can authorize support personnel to inspect content and what record of that access is available.

Use synthetic materials to test an ordinary user's attempt to retrieve a matter they are not permitted to access. Repeat the check after permission changes and user removal. Record actual behavior and limitations instead of assuming a feature label proves effective separation. If access evidence is unavailable, keep confidential matter data out of the trial while the gap is assessed.

The ABA opinion addresses disclosure risks both outside the firm and among internal users who should not receive particular client information. Its supervision analysis also calls for investigating vendor security measures and enforceable confidentiality arrangements. These are fact-dependent inquiries, not an assurance that a legal-branded tool preserves confidentiality or privilege. [1]

Resolve retention, deletion and export before uploading

Request a retention schedule covering uploads, prompts, outputs, derived indexes, logs and backups. Clarify what deletion removes immediately, what persists, the reason for any exception and how long retained copies remain. Ask whether the same treatment applies to model providers and support systems. Distinguish deletion through the interface from deletion across the processing chain. The ABA opinion specifically highlights retention during and after service discontinuation and possible vendor proprietary claims over submitted information. [1]

Test export using synthetic material before a pilot begins. Can the attorney recover the draft issue log, source identifiers and reviewed output in a usable format? Keep original documents and approved work product in the firm's designated records system. Define who closes the trial, revokes access, requests deletion and retains evidence of the closure steps.

Qualified counsel should reconcile deletion choices with applicable preservation, engagement and records obligations. A recommended pilot restriction is to avoid uploading material that the firm cannot retrieve or manage appropriately on exit. Export capability alone does not resolve backup retention or downstream reuse.

Require workable incident and service-change commitments

Ask for the incident procedure and proposed contractual notification terms. Determine what events trigger notice, who receives it, what information will be provided and how the firm can obtain support. Cover unauthorized access and disclosure, but also material processing changes that undermine the approved pilot assumptions. Have contracting counsel examine responsibility allocation and liability limitations rather than treating the existence of an incident policy as sufficient.

Give the firm's security lead responsibility for technical escalation and the deal attorney responsibility for assessing affected work and client obligations with qualified counsel. Applicable notification duties depend on the facts and governing rules. The pilot plan should identify who may suspend uploads, restrict access and preserve relevant evidence. Keep a manual route for continuing contract review if the service becomes unavailable.

NIST recommends assigned incident-response responsibilities, third-party incident notice and support commitments, and fallback arrangements that may include manual processing. [2] These recommendations help structure questions; they do not establish that the vendor's proposed response terms are adequate for this matter.

Demand testing evidence that matches the proposed task

Request the evaluation method, test materials description, model or application version, settings, error categories and known limitations. Ask whether the testing covers the same document formats and extraction task as the pilot. If the vendor offers only a broad accuracy statement, request its definition and supporting evidence. A polished demonstration should not substitute for inspecting omissions and unsupported statements. NIST advises against extrapolating performance from narrow, anecdotal assessments and recommends checking generated sources and citations. [2]

Build an illustrative test set with synthetic agreements containing amendments, missing pages, scanned text and clauses whose meaning changes with an exception. The reviewing attorney should prepare expected clause locations and issue-log entries from the source documents independently of the AI. Compare quotations, party references, amendment precedence and statements that a clause was not found. Distinguish a missing answer from a confidently incorrect one.

Require each substantive extraction to identify its document version and location. The attorney should open the source and check whether the quoted text supports the entry, including relevant definitions and exceptions. Log corrections and unresolved ambiguities. The ABA opinion makes the appropriate degree of independent verification task-dependent while retaining lawyer responsibility for the work. [1] This pilot framework calls for checking every proposed issue-log entry before reliance.

Set trial permissions and stopping conditions in writing

Use synthetic documents first. A move to real deal material requires a separate documented decision that the specific inputs are permitted under the firm's legal, client and confidentiality assessment. Redaction is a choice to evaluate, not a guarantee: contextual information may still disclose the matter. Keep the user group narrow and external sharing disabled where the environment supports that restriction. Do not connect the pilot to automated outreach, negotiation or approval actions.

Set hard stops before testing: unexpected access to another matter, an undisclosed processing route, inability to retrieve sources, or a material fabricated clause supporting a proposed conclusion. Also pause if the firm cannot verify the agreed data-use settings. Record who receives each escalation and who can authorize a restart after remediation and retesting.

Keep the lawyer's source-based review workflow available throughout the trial. NIST recommends defined deactivation criteria, assigned authority and escalation procedures for systems inconsistent with intended use. [2] A stopped pilot is a documented evaluation result, not something to bypass because a transaction deadline is approaching.

Document a bounded suitability decision

Prepare a vendor evidence request and a documented suitability decision for the proposed pilot. The decision should identify the exact service configuration, approved task, allowed inputs, reviewers, supporting documents, open conditions and stopping criteria. Record whether the firm will reject the pilot, continue with synthetic material only, or permit a limited use subject to stated conditions. Reassess before expanding tasks, adding users or accepting material vendor changes.

Approval for one extraction exercise is not approval for autonomous drafting, every client matter or unrestricted data-room access. The deal attorney retains responsibility for professional judgment and reviewed work product. This is a general procurement framework; qualified counsel must assess the particular legal and professional obligations. No suitability memo can promise security, confidentiality, privilege or accurate outcomes.

Acquivo is a software platform that delivers the entire sell-side M&A process, valuation, marketing materials, buyer matching, and deal management, on one platform for business owners, M&A attorneys, and boutique investment banks.

Acquivo is user-operated software with no success fee, not a representative of clients or a provider of legal advice. The evaluation framework above is a recommended process, not a claim that Acquivo or another vendor provides particular controls, integrations or certifications.