Research
Publications
"How Well Can AI do Strategy? Empirical Benchmarking Using Strategy Simulations" with Rory McDonald. Strategy Science.
Click to display Abstract
Benchmarks have helped fuel rapid progress in large language models (LLMs) across a variety of domains including math, science, dialogue, and coding. Yet no existing benchmark adequately captures the defining elements of strategic decision-making: uncertainty, complexity, irreversible multi-period moves, and delayed or noisy feedback. This gap limits our ability to assess and guide LLMs’ capabilities in strategy. We propose that established strategy teaching simulations provide an ideal benchmarking approach because (1) they approximate the essential features of real-world strategy, and (2) they do so in a controlled, replicable environment suitable for evaluation. To demonstrate this, we assess the performance of 21 proprietary and 13 open-source LLMs on the Back Bay Battery (BBB) simulation, a widely used exercise in strategy and innovation courses. The simulation requires balancing short-term profitability against long-term competitive positioning while integrating complex information about customer preferences and technological change. We built an interface enabling LLMs to interact with the simulation as though encountering it for the first time, masking identifiers to reduce contamination from prior training data. Our results show clear progress in composite BBB performance: later models generally outperform earlier versions, and reasoning-focused models from late 2024–early 2025 (e.g., o4-mini, Claude Sonnet 4, Gemini 2.0 Flash) exceed even the average scores of historical MBA student cohorts. However, frontier models from mid-to-late 2025 (e.g., GPT-5, Claude Opus 4.5, Gemini 3) have declined, underperforming both earlier LLMs and MBA students. This decline is partially explained by a systematic bias toward exploiting the core business at the expense of investing in future growth. Overall, these findings highlight impressive advances in LLMs’ strategic abilities since their inception. At the same time, we document current frontier models’ surprising weakness in managing strategic uncertainty. This paper pioneers and provides guidance for using simulation-based benchmarking as a productive framework for strategy researchers to track progress, identify blind spots, and shape the trajectory of strategy-specific LLM capabilities. See latest results HERE Code repository HERE"Listen to Your Users? Self-Selection in User Community Feedback and Commercial Impact" with Rory McDonald and Rob Bremner. 2026. Academy of Management Journal.
** "Best Paper" at NBES 2026 conference
Click to display Abstract
Firms increasingly involve user communities in product development, and prior research documents many benefits in idea generation, knowledge sharing, and cost reduction. We theorize, however, that incorporating market feedback about products from these communities can harm products’ commercial performance. Because community feedback is volunteered by self-selected participants rather than sampled to represent the broader market, it may mislead firms when narrow community preferences diverge from mainstream demand. We predict that greater incorporation of community feedback will increase positive evaluations of a product within the community, but undermine its broader commercial success when there are strong user self-selection pressures (like for initially niche-targeted games). We test this theory in the PC video game industry using Steam Early Access, where developers integrate community feedback prior to release. Using a multi-method design that combines large-scale archival analyses with interviews of developers, executives, and community members, we find that feedback incorporation during Early Access improves post-launch commercial outcomes of games only when the Early Access community closely mirrors the distribution of users in the broader Steam market. But in unrepresentative communities—even large ones—greater incorporation predicts lower post-launch revenues despite favorable community evaluations."Methodological Pluralism and Innovation in Data-Driven Organizations" with Rory McDonald. 2025. Administrative Science Quarterly
** "Best Paper" at Strategy Science Conference 2022
** "Best Paper" at Wharton Innovation Doctoral Symposium 2022
Click to display Abstract
Prior research on data-driven innovation, which assumes quantitative analysis as the default, suggests a tradeoff: Organizations that rely heavily on data-driven analysis tend to produce familiar, incremental innovations with moderate commercial potential, at the expense of risky, novel breakthroughs or hit products. We argue that this tradeoff does not hold when quantitative and qualitative analysis are used together. Organizations that substantially rely on both types of analysis in the new-product innovation process will benefit by triangulating quantifiably verifiable demand (which prompts more moderate successes but fewer hits) with qualitatively discernible potential (which prompts more novelty but more flops). Although relying primarily on either type of analysis has little impact on overall new-product sales due to the countervailing strengths and weaknesses inherent in each, together they have a complementary positive effect on new-product sales as each compensates for the weaknesses of the other. Drawing on a unique dataset of 3,768 new-product innovations from NielsenIQ linked to employee résumé job descriptions from 55 consumer-product firms, we find support for our hypothesis. The highest sales and number of hits were observed in organizations that demonstrated methodological pluralism: substantial reliance on both types of analyses. Further mixed-method research examining related outcomes—hits, flops, and novelty—corroborates our theory and confirms its underlying mechanisms."Algorithm-Augmented Work and Domain Experience: The Countervailing Forces of Ability and Aversion" with Raj Choudhury. 2022. Organization Science 33(1), 149-169.
**"Best PhD Student Paper" at SMS conference 2020
Click to display Abstract
Past research offers mixed perspectives on whether domain experience helps or hurts algorithm-augmented worker performance. Reconciling these perspectives, we theorize that intermediate levels of domain experience are optimal for algorithm-augmented performance, due to the interplay between two countervailing forces—ability and aversion. Although domain experience can increase performance via increased ability to complement algorithmic advice (e.g., identifying inaccurate predictions), it can also decrease performance via increased aversion to accurate algorithmic advice. Because ability developed through learning by doing increases at a decreasing rate, and algorithmic aversion is more prevalent among experts, we theorize that algorithm-augmented performance will first rise with increasing domain experience, then fall. We test this by exploiting a within-subjects experiment in which corporate information technology support workers were assigned to resolve problems both manually and using an algorithmic tool. We confirm that the difference between performance with the algorithmic tool versus without the tool was characterized by an inverted U-shape over the range of domain experience. Only workers with moderate domain experience did significantly better using the algorithm than resolving tickets manually. These findings highlight that, even if greater domain experience increases workers’ ability to complement algorithms, domain experience can also trigger other mechanisms that overcome the positive ability effect and inhibit performance. Additional analyses and participant interviews suggest that, even though the highest experience workers had the greatest ability to complement the algorithmic tool, they rejected its advice because they felt greater accountability for possible unintended consequences of accepting algorithmic advice."A Spanner in the Works: Category-Spanning Entrants and Audience Valuation of Incumbents" with Rory McDonald. 2022. Strategy Science.
Click to display Abstract
Previous work has examined how audiences evaluate category-spanning organizations, but little is known about how their entrance affects evaluations of other, proximate organizations. We posit that the emergence of category-spanning entrants signals the advent of an altered future state—and seeds doubt about incumbents’ prospects in a reordered industry-categorization scheme. We test this hypothesis by treating announcements of funding for startups as an information shock to investors evaluating incumbent financial service providers between 2010 and 2017—a period marked by atypical category combinations at FinTech startups. We find that announcements by startups that embodied unusual combinations of categories resulted in lower cumulative average returns for incumbents, both in absolute terms and in comparison with typical startups. Our theory and results contribute to research on categorization in markets and to theories of disruptive innovation and industry evolution."Machine Learning for Pattern Discovery in Management Research" with Raj Choudhury and Michael Endres. 2021. Strategic Management Journal 42(1), 30-57.
Click to display Abstract
Supervised machine learning (ML) methods are a powerful toolkit for discovering robust patterns in quantitative data. The patterns identified by ML could be used for exploratory inductive or abductive research, or for post hoc analysis of regression results to detect patterns that may have gone unnoticed. However, ML models should not be treated as the result of a deductive causal test. To demonstrate the application of ML for pattern discovery, we implement ML algorithms to study employee turnover at a large technology company. We interpret the relationships between variables using partial dependence plots, which uncover surprising nonlinear and interdependent patterns between variables that may have gone unnoticed using traditional methods. To guide readers evaluating ML for pattern discovery, we provide guidance for evaluating model performance, highlight human decisions in the process, and warn of common misinterpretation pitfalls. The Supporting Information section provides code and data to implement the algorithms demonstrated in this article"From Local Modification to Global Innovation: How Research Units in Emerging Economies Innovate for the World" with Shad Morris, James Oldroyd, Daniel Chng, and Jian Han. 2023. Journal of International Business Studies.
Click to display Abstract
More and more companies are turning to emerging markets as sources of global innovation to help transform business and society. However, building innovation capabilities in emerging markets is still elusive for most companies. To understand how some companies are successfully building these capabilities, we examined workers within R&D units in China across six foreign multinational corporations. In contrast with prior literature that emphasizes a structural view of who the workers interacted with to innovate, our inductive analysis highlights a behavioral view of how R&D unit personnel interact during the problem and solution search process. We identified two key behaviors associated with the problem and solution search: (1) observing customers in their everyday context, and (2) uncovering general knowledge principles from internal experts. Respectively, these behaviors helped R&D workers to question assumptions about existing products as they relate to customers and to apply useful principles from expert knowledge rather than copying solution templates. Our findings offer an alternative path to building global innovation capabilities in markets where structural constraints exist for the company.Working Papers Submitted or Under Review
"The Organizational Structure Implications of Solo Founding vs. Co-Founding" with Todd Hall. Revise and Resubmit at Academy of Management Journal.
"Leap of Faith? How Diffusion Dynamics Obfuscate the Commercial Potential of Novel Innovations" with Conner Rice. Submitted to AOM 2025 / Under Review.
** "Best Paper" at AOM 2024
** Finalist for TIM Best Paper Award AOM 2024
Click to display Abstract
This study offers a demand-side explanation for why many novel innovations succeed despite initially small observable market sizes. Diffusion theory suggests that the ambiguity of relatively novel product innovations leads potential customers to base their adoption decisions more heavily on others' adoption. As a result, a significant portion of demand only materializes post-diffusion. I posit that this dynamic obfuscates the true commercial potential of novel innovations when estimates are based on pre-launch observable demand. Agent-based simulations support this theory, showing that novel products outperform non-novel ones with similar initial market sizes. I also explore the model’s implications for firms’ innovation selection processes. The findings complement supply-side strategic innovation theories and highlight the limitations of heavily relying on data-driven, observable market demand in innovation."Controlled Experiments in Uncontrolled Environments: The Limits of A/B Testing in Startups" with Travis Howell, Todd Hall, and Anavir Shermon. Revise and Resubmit at Organization Science.
Click to display Abstract
Startups increasingly rely on controlled experiments such as A/B testing to navigate uncertainty as they grow. We theorize and show that the methodological rigor these experiments demand often clashes with the volatile realities of entrepreneurship. Theorizing "entrepreneurs as scientists," we contend that, as in academic research, valid inference requires certain methodological conditions: a stable data-generating process, ceteris paribus, and a sufficient sample size. Yet these conditions are often violated in startups’ uncertain and dynamic environment, and we theorize that experimental feedback becomes distorted in practice when these conditions are violated. Using panel data on 1,558 U.S. e-commerce startups from 2017–2023, we confirm that while A/B testing is associated with higher performance on average, this positive association significantly weakens, and sometimes even reverses, when these conditions are not met."Organizing for Adaptation: Does Managerial Hierarchy Help or Hinder Product Adaptation in Startups?" with Todd Hall, Anavir Shermon, and Travis Howell. Revise and Resubmit at Organization Science.
Click to display Abstract
Prior research portrays hierarchy in startups as a tradeoff: enhancing coordination but constraining adaptability. We challenge this view by asking: under what conditions does managerial hierarchy hinder or enable product adaptation in startups? Using a question-driven approach with data on 3,832 U.S. e-commerce startups from 2017–2023, we find that adding hierarchical layers is positively associated with product adaptation. This relationship strengthens when coordination demands are higher—specifically, when startups exhibit greater workforce specialization and product component interdependence—and weakens when managerial intensity is excessive. Moreover, hierarchy amplifies the performance benefits of adaptation across website visits, unique users, and engagement time. Together, these findings challenge the prevailing tradeoff view, suggesting that hierarchy should not be viewed solely as a constraint on adaptation but as an enabler when adaptation requires coordination."Intelligence Without Integrity: Why Capable LLMs May Undermine Reliability" with Aticus Peterson. Under Review at Organization Science.
Click to display Abstract
As LLMs become embedded in research workflows and organizational decision processes, their effect on analytical reliability remains uncertain. We distinguish two dimensions of analytical reliability—intelligence (the capacity to reach correct conclusions) and integrity (the stability of conclusions when analytically irrelevant cues about desired outcomes are introduced)—and ask whether frontier LLMs possess both. Whether these dimensions trade off is theoretically ambiguous: the sophistication enabling accurate analysis may also enable responsiveness to non-evidential cues, or alternatively, greater capability may confer protection through better calibration and discernment. Using synthetically generated data with embedded ground truth, we evaluate fourteen models on a task simulating empirical analysis of hospital merger effects. We find that intelligence and integrity trade off: frontier models most likely to reach correct conclusions under neutral conditions are often most susceptible to shifting conclusions under motivated framing. We extend work on sycophancy by introducing goal-conditioned analytical sycophancy: sensitivity of inference to cues about desired outcomes, even when no belief is asserted and evidence is held constant. Unlike simple prompt sensitivity, models shift conclusions away from objective evidence in response to analytically irrelevant framing. This finding has important implications for empirical research and organizations. Selecting tools based on capability benchmarks may inadvertently select against the stability needed for reliable and replicable analysis."LLMs as Belief Amplifiers: How Human Mental Representations Shape AI-Augmented Strategy" with Mana Heshmati, Michael Lenox, Rory McDonald, and Maelle A. Perez. Under Review at Strategic Management Journal.
Click to display Abstract
Artificial Intelligence (AI) systems based on Large Language Models (LLMs) have made rapid progress in general intelligence capabilities (e.g., programming, dialogue, and exam performance), but they may struggle with strategy problems marked by high complexity, multi-period dynamism, and uncertainty. Meanwhile, strategy research on cognition has demonstrated that good decisions depend not just on general intelligence, but on the mental representations managers use to simplify, interpret, and act in such complex settings. We therefore ask: how does using LLMs affect human strategic decision-making, and how do mental representations moderate any such effects? Taking an abductive approach, we empirically scrutinize these questions in a classroom experiment of participants (N≈200) using the Back Bay Battery strategy simulation in a 2×2 randomized design: treatment of access to an LLM (OpenAI’s o4-mini), and treatment of prior training in disruption theory. We find that among active LLM users, the effect of using the LLM hinged on their mental representation: without disruption training, greater LLM use led to more investment in the simulation’s legacy profitable yet declining business; with disruption training, greater LLM use increased investment in the emerging technology. Text analysis of chat logs indicates a mental representation mechanism: trained participants issue technology-focused prompts that pushed back more on the LLM, whereas untrained participants were focused on other aspects of the simulation (e.g., product pricing) and pushed back less on the LLM’s responses. These results are consistent with the theory that LLMs function as belief amplifiers. Paired with forward-looking mental representations, the LLM interaction ultimately reinforces forward-looking strategies; absent them, they entrench backward-looking exploitation. We suggest that a key role of strategists in the age of AI is to supply the most useful representation, framework, or theory for a given situation."Does AI Redirect What Firms Innovate In?" Working paper.
"Blended Finance in Africa" with Thomas Newell. Working paper.
"How Predictable is Exceptional Growth in Entrepreneurship? Using Machine Learning to Predict Who Joins the Unicorn Venture Club" with Suresh Kotha, Ben Hallen, Sung Ho Park, and Joseph Shin. Working paper.
** "Best Paper" at AOM 2024