Gary N. Smith

Gary N. Smith is the Fletcher Jones Professor of Economics at Pomona College. His research on stock market anomalies, statistical fallacies, the misuse of data, and the limitations of AI has been widely cited. He is the author of more than 100 research papers and 18 books, most recently, Standard Deviations: The truth about flawed statistics, AI and big data, Duckworth, 2024.

Archives

Yes, the AI Stock Bubble Is a Bubble

_{It's unfolding the way a financial bubble typically does}_{Gary Smith
March 28, 2025}

Unlike the Internet, LLMs are not useful enough to prompt customers to pay prices that reflect the hundreds of billions of dollars needed to develop them.

Why LLMs Are Not Boosting Productivity

_{If LLMs were as reliably useful as economist Tyler Cowen alleges, businesses would be using them to generate profits faster than LLMs generate text. They aren’t.}_{Jeffrey Funk and Gary Smith
March 6, 2025}

So far, AI is dragging down economic growth by diverting so much human talent and natural resources away from more productive uses.

Intelligence Requires More Than Following Instructions

_{Post-training improves the accuracy and usefulness of LLMs but does not make them intelligent in any meaningful sense — as the Monty Hall problem shows}_{Gary Smith
March 3, 2025}

The danger is not that computers are smarter than us but that we think they are and thus trust them to make decisions they should not be trusted to make.

The Large Language Model (LLM) “Superpower” Illusion Dies Hard

_{Historic confirmation bias around ESP and spirit cabinets makes for an interesting comparison with the current need to believe in the abilities of LLMs}_{Gary Smith
February 24, 2025}

Someone blinded by confirmation bias might believe that robots and Go victories are evidence that ChatGPT would be a reliable lawyer.

Why LLMs (chatbots) Won’t Lead to Artificial General Intelligence

_{The biggest obstacle is seldom discussed: Most consequential real-world decisions involve uncertainty}_{Gary Smith
February 20, 2025}

LLMs can’t help when key decisions don’t feature objectively correct probabilities but rather subjective probabilities that need interpretation.

Some Lessons From DeepSeek, Compared With Other Chatbots

_{I tested OpenAI o1, Copilot, and Gemini Flash, along with DeepSeek, on a question about Tic-Tac-Toe}_{Gary Smith
January 30, 2025}

Here’s the problem with the use case for LLMs: If you know the answer, you don’t need to ask an LLM; if you don’t know the answer, you can’t trust one.

Sloppy Science is a Statistical Sin

_{Evidence of sloppy science encourages readers to wonder if the entire research project is compromised}_{Gary Smith
January 17, 2025}

Journals should require authors to make their data available online before publication and encourage others to try to replicate the research they publish.

The Hype and Limitations of Generative AI

_{Robert J. Marks and Gary Smith
January 16, 2025}

On this episode, host Robert J. Marks concludes his conversation with economics professor and author Gary Smith about the hype and limitations of generative AI. Smith is the Fletcher Jones Professor of Economics at Ponoma College and a frequent contributor to Mind Matters News. In this portion of the conversation, Smith and Marks explore the hype around artificial general intelligence (AGI) and explain how current large language models lack true reasoning and creative capabilities, despite regular claims of impending AGI from people like OpenAI’s Sam Altman. Smith provides examples demonstrating how these models make nonsensical or incorrect responses to logical problems and financial questions, highlighting their inability to understand context and perform meaningful reasoning. …

AGI Is Not Already Here. LLMs Are Still Not Even Intelligent

_{Recent tests continue to show huge failures in comprehending common sense issues}_{Gary Smith
January 16, 2025}

In one test, for example — not understanding the situation — Open AI launched an extremely verbose and confused description of the situation.

Large Language Models (LLMs) Flunk Word Game Connections

_{Despite hype, ChatGPT and its competitors, in all their iterations, are still just text-generators based on statistical patterns in the text databases they train on}_{Gary Smith
January 6, 2025}

Identification of statistical patterns in text that Large Language Models (LLMs) do not understand is not going to give us AGI, let alone superintelligence.

The Promise of Artificial General Intelligence is Evaporating

_{Revenue from corporate adoption of AI continues to disappoint and, so far, pales in comparison to the revenue that sustained the dot-com bubble — until it didn’t}_{Jeffrey Funk and Gary Smith
December 5, 2024}

Recognition is growing that fundamental challenges make LLMs unreliable. Increasingly expensive scaling will likely hasten the popping of the AI bubble.

Do Fantasy Sports Tell Us Something About Artificial Intelligence?

_{My biggest takeaway from my own involvement is how well fantasy football illuminates some weaknesses of artificial intelligence (AI)}_{Gary Smith
November 15, 2024}

AI models are really good at finding patterns in data but they are bad at deciding whether these spotted patterns can be used to make reliable predictions.

The World Series of Coin Flips

_{Here we go again with the annual coin-flipping ritual known as the World Series}_{Gary Smith
October 22, 2024}

With the Yankees and Dodgers so evenly matched, this World Series will be somewhat like a coin-flipping contest. That's the paradox of luck and skill.

P-Hacking: The Perils of Presidential Election Models

_{History professor Alan Lichtman’s model uses 13 true/false questions reflecting likely voter interests. But some of them seem rather subjective}_{Gary Smith
October 8, 2024}

Lichtman’s Thirteen Keys to the White House prediction model for the presidency shows some evidence of p-hacking, as does Helmut Norpoth’s Primary Model.

Presidential Pundits—a P-Hacking Parable

_{In politics, as elsewhere, too many studies flop when other researchers attempt to replicate them with fresh data}_{Gary Smith
October 7, 2024}

Some prediction models were developed by well-intentioned researchers before the perils of p-hacking were clearly understood, hence the failures.

A Sloppy “AI Scientist” Could Make the Science Crisis Much Worse

_{A research team claims to have developed the AI Scientist that “generates novel research ideas, writes code, executes experiments ...” Really?}_{Jeffrey Funk and Gary Smith
August 29, 2024}

It continues to astonish us how easily people can be persuaded that LLMs (chatbots) can do our thinking for us

Bad Luck Seldom Persists — But it Never Guarantees Good Luck

_{Many people embrace the fallacious law of averages in their daily lives when "regression toward the mean" is a more realistic picture}_{Gary Smith
August 21, 2024}

For example, the baseball player with the highest batting average in any season generally does not do as well the season before or the season after.

What the Luck? How Luck Matters to Olympic and Major League Wins

_{One way to think about the relative importance of skill and luck is to consider the consistency of the outcomes}_{Gary Smith
August 16, 2024}

Statistical analysis shows that in major sports, the players or teams that won most of their games benefitted to a surprising degree from good luck.

The Government-Debt Tipping Point Is Nonsense

_{There are serious problems with the economics paper by Reinhart and Rogoff, whose recommendations were widely followed}_{Gary Smith
August 9, 2024}

Reinhart/Rogoff’s dismissal of criticism of their deeply flawed study as “academic kerfuffle” is unconscionably cavalier.

How can someone who fails be faulted when everyone else is failing?

_{Market turbulence can cause endowment fund managers to travel with the herd — sacrifice returns in order to reduce annual volatility}_{Gary Smith
July 22, 2024}

The current, widely-favored 60/40 strategy has little or nothing to recommend it beyond the fact that it is what everyone else is doing.