Harvard, MIT, and Wharton research reveals pitfalls of relying on junior staff for AI training

As companies race to adopt artificial intelligence systems, conventional wisdom suggests that younger, more tech-savvy employees will take the lead in teaching their managers how to effectively use the powerful new tools.

But a new study casts doubt on that assumption when it comes to the rapidly-evolving technology of generative AI.

The research, conducted by academics from Harvard Business School, MIT, Wharton, and other institutions in collaboration with Boston Consulting Group, found that junior employees who experimented with a generative AI system made recommendations for mitigating risks that ran counter to expert advice. The findings suggest that companies cannot rely solely on reverse mentoring to ensure the responsible use of AI.

“Our interviews revealed two findings that run counter to the existing literature,” wrote the authors. “First, the tactics that the juniors recommended to mitigate their seniors’ concerns ran counter to those recommended by experts in GenAI technology at the time, and so revealed that the junior professionals might not be the best source of expertise in the effective use of this emerging technology for more senior members.”

Junior consultants struggle with AI risk mitigation in GPT-4 experiment

The researchers interviewed 78 junior consultants in mid-2023 who had recently participated in an experiment giving them access to GPT-4, a powerful generative AI system, for a business problem-solving task. The consultants, who lacked technical AI expertise, shared tactics they would recommend to alleviate managers’ concerns about risks.

But the study found the junior employees’ risk mitigation tactics were often grounded in “a lack of deep understanding of the emerging technology’s capabilities,” focused on changing human behavior rather than AI system design, and centered on project-level interventions rather than organization or industry-wide solutions.

Navigating the challenges of generative AI adoption in business

“To explain how and when junior professionals may fail to be a source of expertise in the use of an emerging technology for more senior members, we must take into account not only status threat, but also risks to valued outcomes,” the researchers wrote, noting AI’s exponential rate of change, superhuman capabilities, and reliance on vast amounts of data.

The findings come as companies grapple with the opportunities and challenges presented by generative AI systems, which can engage in open-ended dialogue, answer follow-up questions, and assist with writing, analysis, and coding tasks. By highlighting the limitations of relying on digital native employees to guide AI implementation from the bottom-up, the study underscores the need for top-down AI governance, expert input, and upskilling across all levels of the organization.

“Senior professionals face the dual responsibility of quickly implementing emerging technologies today, and anticipating future versions of technologies and their implications for both their clients and their own organizations,” the authors noted. “To lead their teams and organizations in grappling with a rapidly expanding technological frontier, seniors need to develop a deep understanding of new technologies and their associated capabilities.”