Generative AI: What It Does Well, and What It Doesn’t

Image of a person holding out their hand with the letters AI
Listen

Generative AI has taken the technological world by storm, offering immense potential in content creation, design, and even decision-making processes. Babson College Associate Professor of Practice in Management Peter Cohan investigated the impact of generative AI in his new book, Brain Rush: How to Invest and Compete in the Real World of Generative AI. His research seeks to cut through portrayals of AI as either a utopian enabler or a dystopian disruptor, instead analyzing its current impact while looking for tangible evidence of value creation or potential pitfalls.

“I’m focused on what is happening right now in terms of generative AI,” Cohan said. “Is there anything that’s working, or is it all hype? Is there a return on investment in this? Is there any value being created or is this just another tech fad like Blockchain that fades away after people hype it for a few years?”

Contrary to fears of AI replacing jobs, Cohan encourages students to use and understand generative AI, emphasizing the importance of developing skills that complement AI capabilities. The limitations of AI in areas such as numerical and survey analysis highlight some of the skills that will remain indispensable.

“I start off all my classes by saying, ‘I want to future proof you. I don’t want you to be replaced by generative AI,’ ” Cohan said.

Cohan recently was named to RETHINK Retail’s inaugural list of Top AI Leaders in Retail for 2024. RETHINK Retail, a global leader in retail content and insights, named 150 leaders across 10 categories on the list for driving AI innovation. Cohan was named in the research and academia category.

Here, Cohan discusses what generative AI does well and what it doesn’t:

Generative AI Hits

Text-based generative AI, such as ChatGPT and Google’s Gemini, can help spark new ideas and streamline content quickly, Cohan said.

Peter Cohan, associate professor of practice in the Management Division, says generative AI struggles with analysis.
Peter Cohan, associate professor of practice in the Management Division, says generative AI struggles with analysis.

“It can come up with a list of ideas, and even though five of the ideas are obvious, there might be one that makes you think a little differently,” Cohan said. “It’s good for brainstorming in that way.”

The tool also excels at creating customer surveys, Cohan said. The management professor sometimes asks students to interview customers about a potential product they’re selling, then analyze those responses to find out if their startup creates enough value that customers will buy in.

With a few prompts, generative AI can easily create such a survey that will ensure students cover all the necessary marketing angles with consumers.

“It’s really good if you want to do a customer survey so you can decide whether a customer will buy from you or if they’ll buy from a competitor,” Cohan said. “It can come up with all the right questions in the right order.”

Generative AI Misses

Strangely enough, generative AI starts to reveal its weaknesses when it’s asked to analyze the answers to the customer survey it created, Cohan said.

“Generative AI doesn’t have a massive data storage of all these interviews, and it doesn’t know what customers want. It can’t even guess what customers want,” Cohan said.

Surprisingly, the analytical blind spot extends to numbers. More specifically, generative AI has issues analyzing, for example, a company’s annual report over the past five years to interpret what those numbers meant for the venture.

“I asked it to do things like produce a chart or go look at an annual report for a company and show me their revenues, their costs, and their profits. I’d ask it to detail how their financial statements have changed over time and explain why and where that company is going in the future,” Cohan explained.

“Basically, I was asking it to analyze a company. It can’t do any of that. It will maybe reprint part of the first page of a letter that the CEO wrote to shareholders, but it will not be able to penetrate any of the numbers or do any kind of quantitative analysis.”

While generative AI is constantly evolving, Cohan’s research offers a practical guide to navigating the complex landscape of one of the most significant technological advancements of our era.

Posted in Insights

More from Insights »

Latest Stories

Warren Buffett
Buffett Retires: Babson Professors Highlight Lessons from a Historic Transition Warren Buffett’s retirement is more than a milestone, Babson thought leaders say. It’s also a masterclass in thoughtful succession and cultural continuity.
By
Hillary Chabot
Writer
Hillary Chabot
Hillary Chabot is a writer for Babson Thought & Action and Babson Magazine. An award-winning journalist, she is known for her insightful reporting and dedication to detailed storytelling. With a career spanning over two decades, she has covered a wide range of topics, from presidential campaigns and government policy to neighborhood issues and investigative series. As a reporter for The Boston Herald, Hillary earned a reputation for tenacity and integrity. Her work at Babson College fuels her passions—to learn something new every day and conduct thoughtful, empathic interviews. She’s thrilled to be at Babson College, where students, faculty, staff members and classes provide compelling copy daily.
May 30, 2025

Posted in Community, Insights

Zainabou Thiam ’28 sits at a table selling products to waiting customers
Paying Homage to One’s Ancestors Through Entrepreneurship Zainabou Thiam ’28 was raised in the values of Senegal. With the help of Babson, she’s growing her venture, Sunu Body, an African-inspired skin care and wellness brand.
By
John Crawford
Senior Journalist
John Crawford
A writer for Babson Thought & Action and the Babson Magazine, John Crawford has been telling the College’s entrepreneurial story for more than 15 years. Assignments for Babson have taken him from Rwanda to El Salvador, from the sweet-smelling factory of a Pennsylvania candy maker, to the stately Atlanta headquarters of an NFL owner, to the bustling office of a New York City fashion designer. Beyond his work for Babson, he has written articles and essays for The Philadelphia Inquirer, Notre Dame Magazine, The Good Men Project, and other publications. He can be found on Twitter, @crawfordwriter, where he tweets about climate change.
May 29, 2025

Posted in Entrepreneurial Leadership

The champion tennis players pose for a photo with their trophies alongside their coaches
Repeat Champs: Babson’s Doubles Team Nets Second Tennis National Title Olivia Soffer ’25 and Matia Cristiani ’26 captured their second consecutive NCAA Division III doubles national championship in women’s tennis.
By
May 28, 2025

Posted in Community