Generative AI: What It Does Well, and What It Doesn’t

Image of a person holding out their hand with the letters AI
Listen

Generative AI has taken the technological world by storm, offering immense potential in content creation, design, and even decision-making processes. Babson College Associate Professor of Practice in Management Peter Cohan investigated the impact of generative AI in his new book, Brain Rush: How to Invest and Compete in the Real World of Generative AI. His research seeks to cut through portrayals of AI as either a utopian enabler or a dystopian disruptor, instead analyzing its current impact while looking for tangible evidence of value creation or potential pitfalls.

“I’m focused on what is happening right now in terms of generative AI,” Cohan said. “Is there anything that’s working, or is it all hype? Is there a return on investment in this? Is there any value being created or is this just another tech fad like Blockchain that fades away after people hype it for a few years?”

Contrary to fears of AI replacing jobs, Cohan encourages students to use and understand generative AI, emphasizing the importance of developing skills that complement AI capabilities. The limitations of AI in areas such as numerical and survey analysis highlight some of the skills that will remain indispensable.

“I start off all my classes by saying, ‘I want to future proof you. I don’t want you to be replaced by generative AI,’ ” Cohan said.

Cohan recently was named to RETHINK Retail’s inaugural list of Top AI Leaders in Retail for 2024. RETHINK Retail, a global leader in retail content and insights, named 150 leaders across 10 categories on the list for driving AI innovation. Cohan was named in the research and academia category.

Here, Cohan discusses what generative AI does well and what it doesn’t:

Generative AI Hits

Text-based generative AI, such as ChatGPT and Google’s Gemini, can help spark new ideas and streamline content quickly, Cohan said.

Peter Cohan, associate professor of practice in the Management Division, says generative AI struggles with analysis.
Peter Cohan, associate professor of practice in the Management Division, says generative AI struggles with analysis.

“It can come up with a list of ideas, and even though five of the ideas are obvious, there might be one that makes you think a little differently,” Cohan said. “It’s good for brainstorming in that way.”

The tool also excels at creating customer surveys, Cohan said. The management professor sometimes asks students to interview customers about a potential product they’re selling, then analyze those responses to find out if their startup creates enough value that customers will buy in.

With a few prompts, generative AI can easily create such a survey that will ensure students cover all the necessary marketing angles with consumers.

“It’s really good if you want to do a customer survey so you can decide whether a customer will buy from you or if they’ll buy from a competitor,” Cohan said. “It can come up with all the right questions in the right order.”

Generative AI Misses

Strangely enough, generative AI starts to reveal its weaknesses when it’s asked to analyze the answers to the customer survey it created, Cohan said.

“Generative AI doesn’t have a massive data storage of all these interviews, and it doesn’t know what customers want. It can’t even guess what customers want,” Cohan said.

Surprisingly, the analytical blind spot extends to numbers. More specifically, generative AI has issues analyzing, for example, a company’s annual report over the past five years to interpret what those numbers meant for the venture.

“I asked it to do things like produce a chart or go look at an annual report for a company and show me their revenues, their costs, and their profits. I’d ask it to detail how their financial statements have changed over time and explain why and where that company is going in the future,” Cohan explained.

“Basically, I was asking it to analyze a company. It can’t do any of that. It will maybe reprint part of the first page of a letter that the CEO wrote to shareholders, but it will not be able to penetrate any of the numbers or do any kind of quantitative analysis.”

While generative AI is constantly evolving, Cohan’s research offers a practical guide to navigating the complex landscape of one of the most significant technological advancements of our era.

Posted in Insights

More from Insights »

Latest Stories

Three photos in a side-by-side collage depict Michael Kopelman coaching, Brady Anderson in action, and the doubles team celebrating
Courting Success: How Babson Tennis Teams Transformed into National Powerhouses With championships and top-tier recruiting classes, Babson Director of Tennis Michael Kopelman has overseen the transformation of the men’s and women’s programs into national powerhouses.
By
Scott Dietz
Writer
Scott Dietz
Scott Dietz is Babson College's Associate Athletic Director for Strategic Communications. Dietz is responsible for the department's internal and external communications, including branding, campus engagement, marketing, social media, sponsorships and website management. Additional duties consist of event coverage, facilitation of interview requests, media pitching, video content, writing and editing. Before Babson, Dietz spent 13-plus years at fellow NEWMAC institution Wheaton College, worked for the NFL, New England Patriots, and in the media relations department with the NBA's Cleveland Cavaliers. A native of South Park, PA, Dietz began his sports information career at Westminster College.
September 2, 2025

Posted in Community

Babson Build students from HBCUs participate in an exercise in the Weissman Foundry
An Entrepreneurial Summer: How Babson Impacts Communities Year-Round The school year may be over, but Babson’s work of educating entrepreneurial leaders doesn’t stop. In the summer, many entrepreneurs, educators, and leaders descend on campus. They come from around the globe to connect and to learn. What they take away from the College can impact their companies, their classrooms, and their communities.
By
John Crawford
Senior Journalist
John Crawford
A writer for Babson Thought & Action and the Babson Magazine, John Crawford has been telling the College’s entrepreneurial story for more than 15 years. Assignments for Babson have taken him from Rwanda to El Salvador, from the sweet-smelling factory of a Pennsylvania candy maker, to the stately Atlanta headquarters of an NFL owner, to the bustling office of a New York City fashion designer. Beyond his work for Babson, he has written articles and essays for The Philadelphia Inquirer, Notre Dame Magazine, The Good Men Project, and other publications. He can be found on Twitter, @crawfordwriter, where he tweets about climate change.
August 29, 2025

Posted in Community, Entrepreneurial Leadership

Juan “JC” Grullon ’27 and Ty Bradford ’29 shake hands outside Publishers Hall
Publishers Hall Welcomes First-Year Students to Its Cozy Confines During move-in day, a new crop of first-year students made Publishers Hall their home. The small residence hall, built over a century ago, is known for the tight-knit communities that form there.
By
John Crawford
Senior Journalist
John Crawford
A writer for Babson Thought & Action and the Babson Magazine, John Crawford has been telling the College’s entrepreneurial story for more than 15 years. Assignments for Babson have taken him from Rwanda to El Salvador, from the sweet-smelling factory of a Pennsylvania candy maker, to the stately Atlanta headquarters of an NFL owner, to the bustling office of a New York City fashion designer. Beyond his work for Babson, he has written articles and essays for The Philadelphia Inquirer, Notre Dame Magazine, The Good Men Project, and other publications. He can be found on Twitter, @crawfordwriter, where he tweets about climate change.
August 28, 2025

Posted in Community