A fifty-million-dollar Manhattan deal reportedly hit a snag recently because both sides ran the number past the same AI chatbot for a gut check, and it agreed with each of them. Ask if the price is too high, the answer is yes. Ask if it is too low, also yes.
That is not a coincidence, it is the whole problem with letting AI weigh in on a number this important. Coldwell Banker’s own CEO has pointed out that these tools are built to be agreeable, echoing back whatever answer a person hopes to hear rather than delivering an honest, independent read on value. Below is what is actually happening behind the scenes when ChatGPT prices a home, how badly that process can miss, and a single follow-up question that can catch it when it does.
What Is ChatGPT Actually Doing When It Prices a Home?
Ask ChatGPT to value a home and it goes to work scraping publicly available listing sites, most often Zillow and Redfin, then runs a rough price-per-square-foot calculation against nearby homes. What comes back reads like a confident, well-reasoned answer.
The problem is everything that calculation cannot see. A chatbot has no way to walk through a house and judge its actual condition, and it has no idea that the home two doors down backs onto a busy road and sells for noticeably less as a result. In more than one documented case, it could not even reliably tell a home that had genuinely sold apart from one that was simply still listed.
If AI Could Really Price a Home, Why Do Lenders Still Pay for an Appraiser?
Here is a question I like to pose whenever this topic comes up: mortgage lenders require a licensed appraiser to physically inspect a property before approving financing, an inspection that typically costs a homeowner somewhere between three and six hundred dollars per loan. If a chatbot could genuinely replace that process, banks would have dropped the requirement and pocketed the savings years ago. They have not, and that tells you something.
How Far Off Can an AI Home Valuation Actually Be?
Real estate educator Chris Scott ran an experiment to find out. He fed ChatGPT nothing but a property address, no photos, no condition notes, nothing else. The chatbot came back with a range of one million to one million fifty thousand dollars. A licensed agent then pulled an actual comparative market analysis on that same house, landing at eight hundred twenty three thousand to nine hundred forty thousand, which put the AI’s guess roughly sixteen percent too high.
A second experiment flipped the error entirely. Neuhaus Real Estate ran a similar address-only test on a property near Austin, and this time ChatGPT lowballed the home by roughly eleven percent instead. One tool, two properties, two opposite ways of getting it wrong, and that inconsistency is exactly what makes it so hard to trust.
What Happens When You Give AI Real Data Instead of a Guess?
There is a third test worth knowing about, because it points to the actual fix. Working from nothing but an address, ChatGPT guessed a range of six hundred sixty thousand to six hundred eighty five thousand dollars for a separate property. The tester then handed it the property’s real MLS sheet along with a completed CMA, and with that information added, the estimate tightened to one specific figure: six hundred forty thousand dollars. Nothing about the process changed except the quality of what went into it.
What's the One Question That Can Expose a Bad AI Valuation?
Before trusting any AI-generated number, ask it this: for every comparable sale it used, request the closing date, the closed price, and the source, and ask it to flag anything that is actually a list price rather than a completed sale.
This exact follow-up has caused the chatbot to walk back its own numbers in testing. One comparable it had confidently presented as a closed sale at nine hundred seventy five thousand dollars turned out, once questioned, to be nothing more than an asking price. It had never actually sold.
Why Do Even Zillow's Own Zestimates Miss the Mark?
Part of the reason these tools struggle is that most of them lean on the same handful of public databases underneath, whether that’s Redfin, Realtor.com, or Zillow itself. The Zestimate in particular is built almost entirely from county tax assessor records, which means any updates you’ve made since your last permit filing, a new kitchen, a finished basement, an added bedroom, simply don’t exist as far as the algorithm knows. No one from Zillow has ever walked through your front door.
Frequently Asked Questions
Get a Zestimate That Actually Reflects Your Home
If Zillow has never set foot in your house, its estimate is only guessing. My free guide walks you through claiming your home on Zillow and correcting the details it is missing, so your number is based on your real home.
Ken Mandich is a Realtor® and Listing Expert with Complete Realty Team, serving Metro Atlanta with a focus on Cobb and Cherokee County. You can reach him at 404-410-6465 or [email protected].
