What Defense Attorneys Should Ask Before Using Any AI Tool

The Current Landscape: Excitement Meets Caution

The legal profession's relationship with AI is defined by a tension between genuine promise and well-founded caution. On the promise side, AI tools can transcribe hours of audio in minutes, surface relevant passages across thousands of documents, and identify patterns in evidence that human reviewers might miss under time pressure. For overloaded public defenders handling 200 or more cases simultaneously, these capabilities are not luxuries — they are potential lifelines.

On the caution side, the profession has already seen high-profile failures. In Mata v. Avianca (2023), attorneys submitted a brief containing AI-fabricated case citations, leading to sanctions. Several state bars have since issued guidance requiring attorneys to verify any AI-generated content before submission to a court. The Florida Bar's Advisory Opinion 24-1, the California State Bar's Practical Guidance for the Use of Generative AI, and the New York City Bar Association's Report on Generative AI all emphasize the same core point: AI is a tool, and the attorney remains responsible for everything that tool produces.

This tension is healthy. The solution is not to avoid AI entirely — that would itself be a competence failure as these tools become standard practice. The solution is to adopt AI deliberately, with clear-eyed assessment of what each tool does, how it handles your data, and where its limitations lie. That assessment starts with asking the right questions.

Security Questions: Where Is Your Client's Data Going?

Before any AI tool touches client evidence, you need precise answers to these questions. Not marketing language. Specific, verifiable answers.

1. Where is client data stored, and in what jurisdiction? Cloud-based AI tools store data on servers somewhere. You need to know where. Data stored outside the United States may be subject to foreign government access laws. Data stored in a U.S. jurisdiction but on shared infrastructure with inadequate tenant isolation creates a different set of risks. The vendor should be able to tell you the specific cloud provider (AWS, Azure, GCP) and the specific region(s) where your data resides. If they cannot answer this question precisely, stop the evaluation.

2. Who can access your data within the vendor's organization? In many SaaS companies, engineers, support staff, and data scientists have broad access to customer data for debugging, product improvement, and analytics. This is incompatible with attorney-client privilege. Ask for specifics: How many people at the company can access customer data? Under what circumstances? Is access logged and auditable? Is there a formal access control policy, and will they share it with you under NDA?

3. Is client data used to train or improve the AI model? This is the single most important question, and the answer must be an unequivocal "no" with contractual backing. If the vendor uses your client's interrogation transcript, body camera footage, or case documents to improve its AI model, that data could influence outputs generated for other users. This is functionally a disclosure of privileged information. Many consumer-grade AI products do use input data for training. Legal-specific tools should not, and that commitment should appear in the terms of service or a data processing agreement — not just in a blog post or FAQ.

4. What happens to data after processing? When you upload a video for transcription or a document for analysis, the AI tool creates intermediate data: transcripts, embeddings, analysis results, logs. How long is this intermediate data retained? Is it cached anywhere? Can you trigger deletion of all data associated with a case? The vendor should have a clear data lifecycle policy with defined retention periods and verified deletion procedures.

5. Does the tool use third-party AI providers, and what are their terms? Many legal AI tools are wrappers around third-party AI models. The tool you interact with may send your data to a separate company for actual processing. You need to know: Which AI providers are involved? What are their data retention and training policies? Are they using enterprise-grade agreements with zero-retention guarantees, or consumer-tier API access with different terms? The chain of custody for your client's data extends through every subprocessor.

Accuracy Questions: Can You Trust the Output?

AI systems produce output that looks authoritative regardless of whether it is correct. This is the fundamental accuracy challenge in legal AI, and it demands a different set of questions.

6. What is the tool's error rate, and how was it measured? Every AI system has an error rate. The question is whether the vendor knows what it is and can document how they measured it. For transcription tools, ask for word error rate (WER) benchmarks on audio similar to what you will process — body camera footage with background noise, cross-talk, and poor microphone quality, not clean studio recordings. For analysis tools that identify legal issues in evidence, ask what their false positive and false negative rates are. A tool that flags Miranda violations should be evaluated not just on how many it catches, but on how many it misses and how many false alarms it generates.

7. Does the tool hallucinate, and how does it handle uncertainty? Large language models can generate plausible-sounding but entirely fabricated content. This is not a bug that will be fixed in the next release — it is an inherent characteristic of how these models work. For legal AI tools, the critical question is how the tool handles uncertainty. Does it flag low-confidence outputs? Does it distinguish between what it found in the source material and what it inferred? Does it provide citations or timestamps that let you verify every claim against the original evidence? A tool that presents AI-generated summaries as fact without linking back to source material is dangerous in a legal context.

8. Can you verify every output against the source material? This is the practical test of accuracy. If the tool says "the officer administered Miranda warnings at 14:23," can you click on that claim and be taken to the exact moment in the video? If it identifies an inconsistency between an officer's report and the video footage, does it show you both sources? AI outputs in legal work should always be verifiable, not just plausible. If the tool does not provide a clear path from output to source, you cannot meet your duty to verify the work.

9. How does the tool perform on edge cases common in criminal defense? Criminal defense evidence is messy. Audio is muffled. People talk over each other. Officers mumble through Miranda warnings. Suspects speak English as a second language. Interrogation rooms have poor acoustics. Ask the vendor how their tool performs in these real-world conditions, not just on clean test data. Request permission to run a pilot with actual case materials (properly anonymized if necessary) and evaluate the results yourself before committing.

Ethical Questions: What Do the Rules Require?

10. Does your use of this tool comply with ABA Model Rules 1.1, 1.6, and 5.3? Rule 1.1 (competence) requires that you understand the technology well enough to use it effectively and to recognize its limitations. Rule 1.6 (confidentiality) requires that the tool's data handling meets the "reasonable efforts" standard for protecting client information. Rule 5.3 (supervision of nonlawyer assistants) has been interpreted by several bar associations to extend to AI tools — meaning you have a duty to supervise the AI's work product with the same diligence you would apply to a paralegal's work.

The practical implication of Rule 5.3 is significant: you cannot treat AI output as final work product. Every analysis, summary, or identification the AI produces must be reviewed by a licensed attorney who takes responsibility for its accuracy and completeness. If your workflow does not include this review step, your use of the tool may itself be an ethical violation, regardless of how good the tool is.

11. Do you have a duty to disclose your use of AI to the court or opposing counsel? This varies by jurisdiction. Several courts have adopted standing orders requiring parties to disclose when AI was used in preparing filings, with requirements that attorneys certify AI-generated text was verified by a human. Check the standing orders in every court where you practice. Even in courts without formal requirements, consider whether non-disclosure of AI use in evidence analysis could later become a basis for challenging your work product.

12. What are the liability implications if the AI produces an error that harms your client? If an AI transcription tool misses a key statement in a body camera recording, and you rely on that transcription without watching the footage yourself, who bears the liability? The answer is you. The vendor's terms of service almost certainly disclaim liability for the accuracy of AI outputs. Malpractice insurance may or may not cover AI-related errors depending on your policy and how the error occurred. Understanding these liability contours before adoption is not optional.

The Practical Evaluation Checklist

Distill the questions above into a structured evaluation that you apply to every AI tool before it touches client data:

• Data security: Encryption at rest (AES-256) and in transit (TLS 1.2+), U.S. data residency, role-based access controls, audit logging, SOC 2 Type II certification or equivalent.
• Data usage: No training on client data (contractually committed), defined retention periods, verified deletion capability, published subprocessor list.
• Accuracy: Documented error rates on realistic data, uncertainty flagging, source-linked outputs that enable verification, demonstrated performance on edge cases.
• Ethics compliance: Compatible with Rules 1.1, 1.6, and 5.3; workflow supports mandatory human review; disclosure requirements identified for your jurisdictions.
• Vendor stability: Data export capability, business continuity protections, track record in the legal market.
• Liability: Understood limits of vendor liability, malpractice coverage confirmed, risk allocation acceptable.

Score each category and set a threshold below which you will not proceed. Document the evaluation. This documentation serves a dual purpose: it demonstrates your competence under Rule 1.1, and it creates a record of your reasonable efforts under Rule 1.6.

Red Flags That Should Stop Adoption

Some findings should halt your evaluation immediately. If you encounter any of the following, the tool is not ready for criminal defense work:

• Terms of service that grant the vendor rights to use uploaded content for training, improvement, or any purpose beyond providing the service to you.
• No data processing agreement available and no willingness to execute one.
• Outputs that cannot be traced to source material. If the AI makes a factual claim about your evidence and there is no way to verify it against the original, the tool is unsuitable for legal work.
• Marketing claims of accuracy without supporting methodology. "99% accurate" means nothing without a description of what was tested, under what conditions, and how accuracy was measured.
• No clear answer on third-party AI providers. If the vendor will not tell you which AI models process your data and under what terms, your data could be flowing anywhere.
• No access controls or audit logging. If every user at your firm can see every case, and there is no record of who accessed what, the tool fails the most basic confidentiality requirements.
• The vendor has not thought about legal-specific use cases. A general-purpose AI tool repurposed for legal work without specific attention to privilege, confidentiality, and accuracy requirements is a liability waiting to materialize.

How to Pilot Responsibly

Once a tool passes your evaluation framework, pilot it before full adoption. A responsible pilot program looks like this:

Start with non-sensitive materials. Use publicly available body camera footage, court recordings, or sample data provided by the vendor. Evaluate the tool's accuracy and usability before introducing any client data.

Run a parallel workflow. For your first several real cases, use the AI tool alongside your existing manual process. Compare the results. Did the AI capture everything the manual review found? Did it miss anything significant? Did it flag issues that manual review missed? This parallel process is your validation period.

Maintain full human review. During the pilot, every AI output should be verified against the source material by an attorney. This is non-negotiable. The purpose of the pilot is to calibrate your trust in the tool, and trust is earned through verification, not assumed through marketing materials.

Document everything. Keep records of the pilot — what you tested, what the results were, where the tool performed well and where it fell short. This documentation supports your competence determination under Rule 1.1 and informs your decision about whether to adopt the tool for regular use.

Establish ongoing review. AI tools are updated regularly. A tool that was accurate in your pilot may behave differently after a model update. Build periodic accuracy checks into your workflow — perhaps reviewing one in every ten AI outputs in detail even after the pilot period ends.

The Bottom Line

AI is coming to criminal defense whether individual attorneys welcome it or not. Prosecutors are already using AI tools for case preparation, evidence analysis, and risk assessment. Defense attorneys who refuse to engage with these tools risk falling behind. But defense attorneys who adopt AI tools without rigorous evaluation risk something worse: compromising their clients' cases and their own professional standing.

The path between these extremes is straightforward. Ask the questions in this article. Demand specific answers. Verify those answers independently where possible. Pilot before committing. Maintain human oversight after adopting. Document your process throughout. In criminal defense, where the stakes are measured in years of a person's life, that level of care is not excessive. It is the minimum.