Generative AI: What It Does Well, and What It Doesn’t

Image of a person holding out their hand with the letters AI
Listen

Generative AI has taken the technological world by storm, offering immense potential in content creation, design, and even decision-making processes. Babson College Associate Professor of Practice in Management Peter Cohan investigated the impact of generative AI in his new book, Brain Rush: How to Invest and Compete in the Real World of Generative AI. His research seeks to cut through portrayals of AI as either a utopian enabler or a dystopian disruptor, instead analyzing its current impact while looking for tangible evidence of value creation or potential pitfalls.

“I’m focused on what is happening right now in terms of generative AI,” Cohan said. “Is there anything that’s working, or is it all hype? Is there a return on investment in this? Is there any value being created or is this just another tech fad like Blockchain that fades away after people hype it for a few years?”

Contrary to fears of AI replacing jobs, Cohan encourages students to use and understand generative AI, emphasizing the importance of developing skills that complement AI capabilities. The limitations of AI in areas such as numerical and survey analysis highlight some of the skills that will remain indispensable.

“I start off all my classes by saying, ‘I want to future proof you. I don’t want you to be replaced by generative AI,’ ” Cohan said.

Cohan recently was named to RETHINK Retail’s inaugural list of Top AI Leaders in Retail for 2024. RETHINK Retail, a global leader in retail content and insights, named 150 leaders across 10 categories on the list for driving AI innovation. Cohan was named in the research and academia category.

Here, Cohan discusses what generative AI does well and what it doesn’t:

Generative AI Hits

Text-based generative AI, such as ChatGPT and Google’s Gemini, can help spark new ideas and streamline content quickly, Cohan said.

Peter Cohan, associate professor of practice in the Management Division, says generative AI struggles with analysis.
Peter Cohan, associate professor of practice in the Management Division, says generative AI struggles with analysis.

“It can come up with a list of ideas, and even though five of the ideas are obvious, there might be one that makes you think a little differently,” Cohan said. “It’s good for brainstorming in that way.”

The tool also excels at creating customer surveys, Cohan said. The management professor sometimes asks students to interview customers about a potential product they’re selling, then analyze those responses to find out if their startup creates enough value that customers will buy in.

With a few prompts, generative AI can easily create such a survey that will ensure students cover all the necessary marketing angles with consumers.

“It’s really good if you want to do a customer survey so you can decide whether a customer will buy from you or if they’ll buy from a competitor,” Cohan said. “It can come up with all the right questions in the right order.”

Generative AI Misses

Strangely enough, generative AI starts to reveal its weaknesses when it’s asked to analyze the answers to the customer survey it created, Cohan said.

“Generative AI doesn’t have a massive data storage of all these interviews, and it doesn’t know what customers want. It can’t even guess what customers want,” Cohan said.

Surprisingly, the analytical blind spot extends to numbers. More specifically, generative AI has issues analyzing, for example, a company’s annual report over the past five years to interpret what those numbers meant for the venture.

“I asked it to do things like produce a chart or go look at an annual report for a company and show me their revenues, their costs, and their profits. I’d ask it to detail how their financial statements have changed over time and explain why and where that company is going in the future,” Cohan explained.

“Basically, I was asking it to analyze a company. It can’t do any of that. It will maybe reprint part of the first page of a letter that the CEO wrote to shareholders, but it will not be able to penetrate any of the numbers or do any kind of quantitative analysis.”

While generative AI is constantly evolving, Cohan’s research offers a practical guide to navigating the complex landscape of one of the most significant technological advancements of our era.

Posted in Insights

More from Insights »

Latest Stories

First row (left to right): Laura Bautista ’29, Ezel Bhatty ’29, Sydney Fojas ’29; Second row (left to right): Lucas Lebrija ’29, Tia Malhotra ’29, Remy Witt ’29
Class of 2029 Blank Leadership Scholars Bring Excitement to Campus The Class of 2029 Blank Leadership Scholars arrive at Babson equipped with entrepreneurial spirit and social impact experience, from launching nonprofits and tech solutions to championing environmental education.
By
Melissa Savignano
Writer
Melissa Savignano
Melissa Savignano, a content marketing manager at Babson College, has worked in higher education for almost a decade, where she tells authentic, compelling campus and community stories. Before Babson, she managed communications for Boston University’s largest college, the College and Graduate School of Arts & Sciences. She previously worked in client relations, helping brands of various sizes launch content marketing strategies and storytelling initiatives. When not at work, you will find her in the city of Boston, probably at the movie theater.
September 25, 2025

Posted in Community

An overview of the fair with students in a blur as the move around
Unscripted Moment: Club Fair Blur In our occasional photo series, staff photographer Nic Czarnecki shares a unique, long-exposure image to illustrate the movement of students at the Club Fair.
By
September 24, 2025

Posted in Community

Two students cheer with pompoms during move-in day
No. 1 Again: Babson Tops Entrepreneurship Ranking for 29th Consecutive Time For the 29th consecutive time, U.S. News & World Report has named Babson College the No. 1 undergraduate school for entrepreneurship.
By
September 23, 2025

Posted in Community