Built to Inspire

You Can’t Trust ChatGPT, Esq. Why Access to Information & AI has not Democratized Expertise.

Unprecedented access to online information—and now AI-generated answers—has led many people to believe they are capable of answering their own complicated legal questions without professional guidance. While search engines and AI tools can surface general knowledge, they lack the ability to interpret issues in context, to weigh factors, or account for particular fact patterns. 

Not All Information is Equal 

Relying on experts involves trust –trust in an institution, trust in the information, and trust in the experts themselves. However, when opinion, individual experience, and even lies are presented online and in the media with the same weight and prevalence as facts, scientific knowledge may no longer bear, by default, a higher truth value than opinions or the beliefs. 1

Research shows that exposure to a concept alone, regardless of whether it is factual, makes people believe it is accurate. This is known as the illusory truth effect where repeated exposure to information increases its perceived accuracy regardless of whether it is actually true. A large meta-analysis of more than 180 studies involving over 30,000 participants found that repetition consistently makes statements feel more believable. 2 These effects persist even among people who have relevant knowledge or are motivated to be accurate. 

This means that individuals may feel increasingly confident in their online “research” not because it is correct, but because it is familiar. 

Garbage In – Garbage Out 

The term “garbage in, garbage out” has been used by professionals working in many fields even prior to AI. The term has to do with data quality—if we base assumptions on bad data, we’ll get bad outcomes.

Large language models (LLMs) work by accessing mass amounts of data. If an LLM is fed misinformation (“garbage in”) it does not have the ability to critically distinguish it from fact and therefore its input directly impacts its output (“garbage out”). 

Reliability is a core problem with current AI tools especially in the legal field. Research shows that legal AI tools can generate incorrect or fabricated information at troubling rates, particularly when dealing with complex legal reasoning. 3

In Patent Law, this is not a minor flaw, where precision in claim language and citations is paramount. A single hallucinated reference or misinterpreted disclosure can undermine an entire application or litigation strategy.

Even when AI performs well, its strengths are uneven. A randomized controlled study published in the Minnesota Law Review found that AI assistance increased the speed of legal work but only “slightly and inconsistently” improved the quality of legal analysis. 4 In other words, AI is an efficiency tool—not a substitute for expertise. This distinction is critical in patent law, where the value lies not in generating text quickly, but in crafting claims that withstand scrutiny years later.

More fundamentally, patent practice depends on forms of judgment that AI does not replicate. Legal judgment is a composite of experience, ethics, and contextual awareness—capabilities grounded in human reasoning. Patent attorneys routinely make strategic decisions about claim scope, prosecution history, and disclosure tradeoffs that require exactly this kind of nuanced judgment. These are not pattern-recognition tasks; they are decisions under uncertainty with long-term consequences.

In addition to information access, the availability of sophisticated AI tools creates an illusion that expertise has been democratized. However, patent law depends on trained judgment developed through years of education and experience—judgment that cannot be reduced to pattern recognition or information retrieval. 

None of this is to say AI is unimportant. AI excels at scale, speed, and pattern recognition. Good patent attorneys excel at judgment, creativity, and strategy. The future of patent practice lies in combining these strengths—not choosing between them.

With the above in mind it is critical that when you have patent law questions and/or are seeking to build a patent portfolio, you: 

  • Seek out qualified expertise; 
  • Be wary of firms that claim to rely heavily on AI to reduce costs and seem to replace judgment-based skills (e.g., claim writing) with AI tools; 
  • Be critical of online information or AI-generated legal advice; 
  • And most importantly: remember, ChatGPT, esq., is not a substitute for good patent counsel.

**Please note that this article is not legal advice; it is not a legal opinion; nor should you rely on it as legal advice or as a legal opinion. This article merely expresses the author’s general thoughts on a topic regarding the business of patents. Nothing in this article establishes any form of an attorney-client relationship between you, the reader, and the author of this article.

1 Toni G.L.A van der Meer & Michael Hameleers, Science and the Crisis of Trust, 67 Current Opinion in Psychology 102202 (2026).  

2 Jessica Undry & Sarah J, Barber, The illusory truth effect: A review of how repetition increases belief in misinformation, 56 Current Opinion in Psychology 101736 (2024).

3 Faiz Surani & Daniel E. Ho, AI on Trial: Legal Models Hallucinate in 1 out of 6 (or More) Benchmarking Queries, Stanford University Human-Centered Artificial Intelligence, News, (May 23, 2024).

4 Jonathan H. Choi, et al., Lawyering in the Age of Artificial Intelligence, 109 Minn. L. Rev. 147 (2024).