Prompts, Privilege, and Penalties: The New Rules of AI in Litigation
As artificial intelligence tools become ubiquitous in legal practice, courts are grappling with novel questions about the discoverability of AI-generated materials and the ethical obligations of attorneys and litigants who rely on these tools. A wave of recent decisions in both state and federal courts has begun to clarify the legal landscape, but significant uncertainty—and risk—remains. This alert surveys the key developments.
I. DISCOVERABILITY OF AI-GENERATED MATERIALS
A threshold question in any litigation involving AI is whether a party’s interactions with an AI tool—including prompts, inputs, uploaded documents, and AI-generated outputs—are discoverable. Courts are reaching different conclusions depending on: (1) who created the prompts, (2) the purpose for which they were used, and (3) whether the user can claim work-product or attorney-client privilege.
A. State Court Developments
In Assini v. Hayward, 2026 WL 1677232 (Sup. Ct. Nassau County June 4, 2026), the New York Supreme Court quashed a subpoena directed at OpenAI seeking a pro se defendant’s ChatGPT account data—including prompts, inputs, uploaded materials, and corresponding outputs used in connection with the litigation. The court held that a pro se litigant could assert work-product protections over his AI communications where the materials were prepared in anticipation of litigation, relying on the conditional privilege under CPLR § 3101(d). The court distinguished United States v. Heppner, 820 F. Supp. 3d 292 (S.D.N.Y. 2026), a federal criminal case that had held AI communications were not privileged, noting that Heppner involved a criminal matter in which the defendant’s AI interactions were conducted “on his own volition” and did not reflect counsel’s mental impressions or litigation strategy. Critically, the Heppner court also emphasized that the defendant had used a consumer-grade AI platform whose terms of service permitted the provider to review prompts for training purposes and disclose data to government authorities—leaving no reasonable expectation of confidentiality. The Assini court instead adopted the reasoning of Morgan v. V2X, Inc., 2026 WL 864223 (D. Colo. Mar. 30, 2026), holding that “in the context of a pro se litigant’s use of AI to assist with their litigation preparation, the use of AI closely resembles the kind of confidential, strategy-laden iterative work product that Rule 26(b)(3) was designed to protect.”
The decision signals that New York state courts may afford significant protection to AI-generated litigation materials under the work-product doctrine, at least where those materials reflect a pro se litigant’s litigation strategy. The decision leaves open whether AI communications created outside the litigation context, or those that do not reflect mental impressions or strategy, would receive the same treatment.
B. Federal Court Developments
Federal courts have been more willing to order disclosure of AI interactions, particularly in the expert-witness context.
Expert AI Prompts Are Discoverable. In Conservation Law Foundation v. Shell Oil Co., Case No. 3:21-cv-00933, ECF 970 (D. Conn. May 18, 2026), Magistrate Judge Farrish ordered a plaintiff to produce the AI prompts its expert used to analyze documents, holding that the prompts are part of the expert’s methodology and fully discoverable under Federal Rule of Civil Procedure 26. The expert, Dr. Naomi Oreskes, had used AI tools to review the defendant’s document production and identify a subset of documents for closer analysis. The court rejected the plaintiff’s three arguments against disclosure: (1) that AI prompts fall outside the scope of Rule 26(b) discovery—the court held that expert methodology is “fair game”; (2) that a Rule 29 agreement between the parties protected the prompts as “notes”—the court declined to read the agreement that broadly, holding that before a court will deny otherwise-relevant discovery based on a Rule 29 agreement, that agreement “must be quite clear”; and (3) that no additional prompts existed—the court found that a prior declaration referencing “prompt[s]” gave the defendants an evidence-based reason to doubt this representation.
This ruling has significant practical implications. Parties retaining testifying experts who will use AI should treat AI prompts as part of the expert’s methodology from the outset—preserving them, understanding how they are used, and assuming they may need to be disclosed. Discovery protocols and agreements should expressly address AI prompts and outputs; an agreement that is silent on AI tools may not protect them from discovery. Of note, a court’s decision may be different where it is a consulting expert, not a testifying expert, at issue.
AI Communications by Pro Se Litigants: A Split Emerges. As noted above, Heppner held that AI communications with Claude were not privileged or subject to work-product protection because the defendant created them “on his own volition,” not at counsel’s direction and not reflecting counsel’s mental impressions or litigation strategy. However, the V2X, Inc., came to the contrary conclusion, holding that a pro se plaintiff could assert work-product protections over AI communications and that the use of AI platforms did not automatically waive all expectations of privacy, despite the fact that AI companies collect user data. The Morgan court also cited Warner v. Gilbarco, Inc., 820 F. Supp. 3d 629 (E.D. Mich. 2026), which similarly allowed work-product protections for a party’s AI use. In Warner, the court denied a motion to compel the production of documents regarding the plaintiff’s use of third-party AI tools, holding that “ChatGPT (and other generative AI programs) are tools, not persons,” and that using generative AI tools does not waive work-product protection because waiver requires disclosure “to an adversary or in a way likely to get in an adversary’s hand.” The court characterized the opposing party’s demand for AI-usage data as an irrelevant and disproportionate “fishing expedition” that distracted from the merits of the case.
This emerging split means that the discoverability of AI communications will depend heavily on jurisdiction, the nature of the case (civil versus criminal), and the specific circumstances of the AI use.
Relevance Remains the Threshold. Even where AI materials are not privileged, they must still satisfy the relevance requirements of Rule 26(b)(1) to be discoverable. In New York Times Co. v. Microsoft Corp., 757 F. Supp. 3d 594 (S.D.N.Y. 2024), the court denied a motion to compel production of evidence related to the newspaper’s use of generative AI tools, holding that the discovery sought was not relevant to the defendant’s fair-use defense or to the claims in the complaint. The court noted that the fair-use defense required scrutiny of the AI company’s conduct, not the newspaper’s. The decision is a reminder that the novelty of AI does not override traditional relevance limitations on discovery.
II. ETHICAL CONSIDERATIONS
The rapid adoption of AI tools has also brought a cascade of ethical issues—and sanctions—for attorneys and pro se litigants alike.
A. AI-Generated “Hallucinations” and the Duty to Verify
The most prominent ethical risk of generative AI in litigation remains “hallucinations”—instances where AI tools fabricate cases, quotations, or legal propositions. Courts have imposed sanctions for the submission of AI-generated filings containing fabricated authorities with increasing regularity and severity.
In Cassata v. Michael Macrina Architect, P.C., 89 Misc. 3d 865, 250 N.Y.S.3d 778 (Sup. Ct. Suffolk County Jan. 27, 2026), the court imposed sanctions on an associate attorney, her supervising attorney, and their law firm after the associate’s opposition papers contained non-existent case citations, fabricated quotations, and propositions of law unsupported by the cited authorities—all apparently lifted from another attorney’s AI-generated brief in an unrelated case. The court found violations of multiple New York Rules of Professional Conduct: Rule 1.1 (competence), Rule 1.3 (diligence), Rule 3.1 (non-meritorious claims), Rule 3.3 (candor toward the tribunal), and Rule 5.1 (supervisory responsibility). The court struck the offending opposition papers, imposed $1,000 fines on both the associate and the supervising attorney, and ordered the firm to pay $8,000 for the opposing party’s attorney’s fees.
The Cassata decision is notable for its extended analysis of a supervising attorney’s obligations in the AI era. The court found it “incompatible” that a supervising attorney could properly supervise an associate’s use of AI technology when the supervisor concededly did not “really know how to use any AI stuff.” Citing the technology-competence amendment to Rule 1.1, Comment [8], the court stated that “[w]e all need to learn this technology. It is not a train that is coming, it is here and speeding fast and we are on it whether we like it or not.”
Similarly, in Gully v. Varghese, 89 Misc. 3d 1208(A) (Sup. Ct. Albany County May 12, 2026), the court struck a pro se plaintiff’s summary judgment motion and related filings after determining that all three cases cited in her legal memorandum were AI-generated fabrications. Taking the plaintiff’s pro se status into account, the court declined to impose additional monetary penalties. The court emphasized that proceeding pro se does not excuse a party from ensuring that cited legal authorities are genuine.
In a related development, the Surrogate’s Court in Matter of Weber as Trustee of Michael S. Weber Trust, 85 Misc. 3d 727, 220 N.Y.S.3d 620 (Surr. Ct. Saratoga County 2024), held that counsel has an affirmative duty to disclose the use of AI prior to introducing AI-generated evidence and that such evidence should be subject to a Frye hearing before admission. The court noted that the expert witness in the case could not recall what input or prompt he used when relying on Microsoft Copilot and could not state what sources the AI relied upon. The decision underscores the importance of documenting and preserving AI prompts and inputs whenever AI tools are used in connection with the preparation of evidence.
B. New York’s Regulatory Framework: Part 161 and the Advisory Committee Report
The New York State Unified Court System’s Advisory Committee on Artificial Intelligence and the Courts issued its inaugural Annual Report in December 2025, proposing a statewide policy (new Part 161 of the Rules of the Chief Administrator) that takes a measured approach:
- The use of generative AI in preparing court papers should not be prohibited, provided such use complies with existing duties and responsibilities.
- Disclosure of AI use should not be required at the time of submission, because an attorney’s or party’s existing obligation to verify the accuracy of filed papers renders such disclosure unnecessary.
- A model rule is available for individual judges to adopt, which reminds attorneys and parties that by signing a paper, they certify it contains no fabricated or fictitious content, consistent with 22 NYCRR §§ 130-1.1 and 130-1.1a.
The Advisory Committee’s report also flagged that AI-enhanced legal research platforms, including Westlaw and Lexis, “fall well short of being fully trustworthy.” Citing a Stanford University study, the report noted that the Lexis AI product hallucinations in 17% of its responses, while Westlaw’s AI product hallucinated in 33% of its responses. The Committee cautioned that “[e]ven when using the AI-enhanced features that have been incorporated into established legal research platforms, any content generated by AI should be independently verified for accuracy.”
C. Duty of Technological Competence
New York’s Rules of Professional Conduct, as amended, require that attorneys “keep abreast of the benefits and risks associated with technology the lawyer uses to provide services to clients.” Rule 1.1, Comment [8]. The Cassata court made clear that this duty is not aspirational: law firm supervisors must understand AI tools provided to their attorneys to properly supervise their use, and the failure to do so can independently support sanctions.
The New York Advisory Committee’s Interim Policy on the Use of AI Within the UCS, which took effect in October 2025, further reinforces these principles by mandating initial and ongoing AI training for all judges and nonjudicial employees with computer access, and requiring that all AI-generated content be thoroughly reviewed for accuracy and bias before use.
III. KEY TAKEAWAYS
Discuss AI tools with clients and experts. At the outset, explain the risks of using AI (absent express instruction from counsel) with both clients and experts.
Preserve AI interactions. Treat all AI prompts, inputs, and outputs as potentially discoverable materials. Implement litigation-hold procedures that encompass AI platform data.
Tailor discovery protocols. Expressly address AI-generated materials in discovery and ESI agreements, protective orders, and discovery protocols. Silence on AI may leave critical materials unprotected. When negotiating protective orders, consider defining “Authorized AI Tools” for permitted use, prohibiting the uploading of protected materials into consumer or public AI tools, and addressing whether vetting or private AI use is deemed a waiver.
Verify everything. Never submit AI-generated content—whether drafted text, legal citations, or expert analysis—without independent human verification using authoritative primary sources. This obligation applies equally to attorneys and pro se litigants.
Train on AI competence. Law firm supervisors have an affirmative obligation under Rule 1.1 to understand the AI tools provided to attorneys and to ensure adequate supervision. Supervisors who cannot use the technology cannot supervise its use.
Expect expert AI prompts to be disclosed. Expert witnesses who use AI should be prepared for their prompts and methodology to be scrutinized and potentially produced in discovery. Well-documented, disciplined AI use may strengthen an expert’s credibility, while poorly defined prompts may invite Daubert or Frye challenges.
Distinguish enterprise from consumer AI tools. The emerging case law draws sharp lines based on platform configuration. Consumer-grade AI tools whose terms of service permit the provider to review prompts for training or disclose data to authorities may destroy any reasonable expectation of confidentiality. Enterprise-grade tools that contractually guarantee data privacy and prohibit model training on user inputs provide a stronger basis for asserting privilege. Evaluate the terms of every AI platform your organization uses and ensure that confidential or privileged information is entered only into tools with appropriate contractual safeguards.
Consider AI use as a discovery tool. Lawyers questioning adverse witnesses should consider probing the witness’s use of AI in connection with the litigation. Unprotected AI interactions—including prompts, outputs, and chatlogs—may be discoverable and could reveal litigation strategy, witness preparation, or factual admissions that provide a significant advantage in prosecuting or defending an action.
Monitor evolving rules. New York’s proposed Part 161, the federal Advisory Committee on Evidence Rules proposed Rule 707 on machine-generated evidence, and the growing body of case law continue to shape this rapidly developing area. Stay current on jurisdictional requirements.
For additional information on the topics discussed in this alert, please contact Julia M. Hilliker at jhilliker@hodgsonruss.com or 716.848.1547.
This alert is intended as a general overview and does not constitute legal advice. Specific questions should be addressed to qualified counsel.