Why I Stopped Trusting AI to Write My Contracts (And What I Do Instead)

Using AI to write contracts is risky because these tools can hallucinate legal language, miss jurisdiction-specific requirements, and omit protections that only show up when something actually goes wrong. I learned this the slow, uncomfortable way — by comparing what a chatbot handed me against what a real attorney flagged in the same document. Now I use AI for speed, not for judgment, and I want to walk you through why that shift matters.

What Made Me Stop Trusting AI With Contracts

For a while, I did what a lot of small business owners do. I typed my situation into a chatbot, asked for a services agreement or an NDA, and treated whatever came back as good enough. It read like a contract. It had the right headings, the right stiff phrasing, the word “indemnify” in it somewhere. That was enough to make me feel covered.

Then I started paying attention to what was happening in courtrooms, not just in my own inbox. Lawyers — actual licensed attorneys with malpractice insurance and law degrees — were getting caught submitting briefs full of fake case citations generated by AI. In one Alabama case, attorneys used ChatGPT to prepare motions, and the court ended up disqualifying defendants’ attorneys from the case and referring the matter to the state bar. In South Florida, a judge sanctioned a lawyer for including “false, fake, non-existent, AI-generated legal authorities” in eight related cases.

These are professionals trained to catch exactly this kind of error, and they still missed it. If they can get fooled, so can I.

The research backs this up. A widely cited Stanford study, “Large Legal Fictions,” tested general-purpose AI models against hundreds of thousands of verifiable legal questions and found hallucination rates between 58% and 88%, with GPT-4 hallucinating 58% of the time compared to GPT-3.5 at 69% and Llama 2 at 88%. Even the paid, purpose-built legal research tools weren’t clean. A follow-up Stanford study published in the Journal of Empirical Legal Studies found hallucination rates of 17% for Lexis+ AI, 33% for Westlaw AI-Assisted Research, and 43% for GPT-4. Those are tools lawyers pay real money for, and they’re still wrong roughly one out of every three to five times on legal questions.

The Real Risk Isn’t Just Made-Up Case Law

Fake citations are the headline-grabbing problem, but for someone like me — not filing court briefs, just trying to protect my business with solid agreements — the bigger risk is quieter. It’s the contract that looks completely fine and isn’t.

Clauses That Sound Right But Don’t Fit Your Situation

One law firm walked through an example that stuck with me: a company used AI to draft a software licensing agreement, and the tool inserted a broad indemnity clause requiring the company to cover the client for any third-party claim tied to the software. Legally, the clause looks sound and enforceable. However, from a commercial perspective, this is problematic: the company operates in a sector where intellectual property disputes are common, and accepting unlimited indemnity exposes the business to potentially huge financial risk that its risk appetite and insurance coverage weren’t built for. Nothing about that clause was “wrong” in a hallucination sense. It just didn’t fit the business signing it, and AI has no way of knowing that.

Jurisdiction Problems You Won’t Notice Until It’s Too Late

Contracts aren’t universal. One attorney who reviews AI-drafted agreements for clients pointed out that what’s enforceable in Delaware might not be in Michigan, and she’s seen AI-generated agreements that copy boilerplate language from another state’s laws, completely undermining enforceability in the client’s home jurisdiction. You’d never catch that by reading the contract closely, because it still sounds fine. It just doesn’t hold up where you actually operate.

Missing Protections You Don’t Know You Need

Another common gap: independent contractor and IP ownership language. Attorneys who deal with AI-drafted agreements regularly note that the technical requirements for securing IP rights from independent contractors differ from those for employees and vary by the type of work, and getting this wrong is one of the most expensive mistakes a business can make — one that AI-generated contracts get wrong regularly. Supplier agreements have a similar blind spot. AI tools tend to draft for the “good path” — but when a shipment arrives late, a supplier raises prices mid-contract, or a vendor gets acquired, the supplier agreement is what determines your remedies, and AI-generated versions routinely fail to address these scenarios in a meaningful way.

Why This Matters Even If You’re Not Signing Million-Dollar Deals

You don’t need to run a corporation for any of this to bite you. A freelance services contract, a client NDA, a vendor agreement for your online shop — these all carry the same structural risk. As one firm put it plainly, without proper oversight, AI-generated contracts can expose businesses to unclear terms, enforceability issues, and unintended liabilities that may only surface when a dispute arises. That’s the part that gets me. The mistake doesn’t announce itself. It sits quietly in your files until the exact moment you need the contract to protect you, and it doesn’t.

What I Do Instead

I haven’t ditched AI entirely. I’ve just changed where it sits in the process.

• I use AI for the first rough pass, never the final version. It’s genuinely good at generating a starting structure — sections, headings, a general sense of what a services agreement usually includes. I treat that output as a rough draft written by someone who’s never met me or my business, because that’s essentially what it is.
• I get a flat-fee human review before anything gets signed. Contract review doesn’t have to mean a massive retainer. Many attorneys now offer flat-fee pricing specifically for this, and on average, drafting a business contract costs around $750, while a contract review runs approximately $490 — figures that shift with complexity, but give you a real ballpark. Some firms even offer issue-specific reviews, where a lawyer just looks over the specific issue you have questions on, which is the cheapest form of review and a good way to feel more confident before signing.
• I build a library of contracts that have actually been vetted once, and reuse those. Instead of generating a new contract from scratch every time, I keep templates that a lawyer has already checked for my state, my industry, and my typical deal structure. AI can help me fill in the specific details later — names, dates, scope — but the bones came from someone who understood my actual risk, not just legal-sounding phrasing.
• I read every clause and ask “does this actually apply to me?” Not “does this sound professional,” but does this fit my state, my industry, my relationship with this particular client or vendor. If I can’t answer that, it goes back for review instead of getting signed.

The Bottom Line

AI didn’t get worse at writing contracts. I got better at understanding what it’s actually good at, and what it isn’t. It’s a decent first draft generator and a terrible substitute for someone who understands your business, your state’s laws, and what happens when a deal goes sideways. A one-time flat fee to have a real person check the document before you sign is cheap insurance compared to what it costs to unwind a bad clause after a dispute has already started.

Hi! I use AI to help research and write posts on this site. I do my best to keep things accurate, but please double-check anything important — and nothing here replaces advice from a licensed or certified professional.