Ryan T. Allen
Research
Publications
"Algorithm-Augmented Work and Domain Experience: The Countervailing Forces of Ability and Aversion" with Raj Choudhury. 2022. Organization Science 33(1), 149-169. PDF
**"Best PhD Student Paper" at SMS conference 2020
Click to display Abstract
Past research offers mixed perspectives on whether domain experience helps or hurts algorithm-augmented worker performance. Reconciling these perspectives, we theorize that intermediate levels of domain experience are optimal for algorithm-augmented performance, due to the interplay between two countervailing forces—ability and aversion. Although domain experience can increase performance via increased ability to complement algorithmic advice (e.g., identifying inaccurate predictions), it can also decrease performance via increased aversion to accurate algorithmic advice. Because ability developed through learning by doing increases at a decreasing rate, and algorithmic aversion is more prevalent among experts, we theorize that algorithm-augmented performance will first rise with increasing domain experience, then fall. We test this by exploiting a within-subjects experiment in which corporate information technology support workers were assigned to resolve problems both manually and using an algorithmic tool. We confirm that the difference between performance with the algorithmic tool versus without the tool was characterized by an inverted U-shape over the range of domain experience. Only workers with moderate domain experience did significantly better using the algorithm than resolving tickets manually. These findings highlight that, even if greater domain experience increases workers’ ability to complement algorithms, domain experience can also trigger other mechanisms that overcome the positive ability effect and inhibit performance. Additional analyses and participant interviews suggest that, even though the highest experience workers had the greatest ability to complement the algorithmic tool, they rejected its advice because they felt greater accountability for possible unintended consequences of accepting algorithmic advice.
"A Spanner in the Works: Category-Spanning Entrants and Audience Valuation of Incumbents" with Rory McDonald. 2021. Strategy Science. PDF
Click to display Abstract
Previous work has examined how audiences evaluate category-spanning organizations, but little is known about how their entrance affects evaluations of other, proximate organizations. We posit that the emergence of category-spanning entrants signals the advent of an altered future state—and seeds doubt about incumbents’ prospects in a reordered industry-categorization scheme. We test this hypothesis by treating announcements of funding for startups as an information shock to investors evaluating incumbent financial service providers between 2010 and 2017—a period marked by atypical category combinations at FinTech startups. We find that announcements by startups that embodied unusual combinations of categories resulted in lower cumulative average returns for incumbents, both in absolute terms and in comparison with typical startups. Our theory and results contribute to research on categorization in markets and to theories of disruptive innovation and industry evolution.
"Machine Learning for Pattern Discovery in Management Research" with Raj Choudhury and Michael Endres. 2021. Strategic Management Journal 42(1), 30-57. PDF
Click to display Abstract
Supervised machine learning (ML) methods are a powerful toolkit for discovering robust patterns in quantitative data. The patterns identified by ML could be used for exploratory inductive or abductive research, or for post hoc analysis of regression results to detect patterns that may have gone unnoticed. However, ML models should not be treated as the result of a deductive causal test. To demonstrate the application of ML for pattern discovery, we implement ML algorithms to study employee turnover at a large technology company. We interpret the relationships between variables using partial dependence plots, which uncover surprising nonlinear and interdependent patterns between variables that may have gone unnoticed using traditional methods. To guide readers evaluating ML for pattern discovery, we provide guidance for evaluating model performance, highlight human decisions in the process, and warn of common misinterpretation pitfalls. The Supporting Information section provides code and data to implement the algorithms demonstrated in this article"From Local Modification to Global Innovation: How Foreign Research Units in Emerging Economies Innovate for the World" with Shad Morris, James Oldroyd, Daniel Chng, and Jian Han. Forthcoming at Journal of International Business Studies
Click to display Abstract
The role of research units in emerging economies is shifting from one focused on local modification (modifying existing products for local markets) to one focused on global innovation (learning from local markets to develop novel products for the world). We examine individual behaviors within six foreign R&D units in China that were actively trying to make this shift. In contrast with prior literature that emphasizes a structural view of who the innovators interacted with in problem and solution search, our inductive analysis highlights a behavioral view of how R&D unit personnel interact during the problem and solution search process. We identified two key behaviors associated with global innovation: (1) observing customers in their everyday context and (2) uncovering general knowledge principles from internal experts. Respectively, these behaviors helped R&D personnel to question assumptions about existing products as they relate to customers and to apply useful principles from expert knowledge rather than copying solution templates. Our findings offer an alternative path to building global innovation capabilities in emerging economies.Dissertation Working Papers
"Methodological Pluralism and Innovation in Data-Driven Organizational Cultures" Revise and Resubmit at Administrative Science Quarterly (Job Market Paper; Dissertation Chapter 1)
** "Best Paper" at Strategy Science Conference 2022
** "Best Paper" at Wharton Innovation Doctoral Symposium 2022
Click to display Abstract
A long tradition in innovation research suggests that data-driven organizational cultures excel at commercializing incremental innovations, but allocate resources away from less measurable breakthrough innovations. Questioning this premise, I distinguish the magnitude of an organization’s use of quantitative analysis from the methodological pluralism of its organizational culture (the extent to which its members value different kinds of analyses). I argue that organizations using more quantitative analysis will actually produce more breakthrough innovations—provided that their cultures are pluralistic enough to use qualitative analysis liberally as well. To test my theory, I measure innovation performance using product-level sales and attribute data for over 3,500 consumer product launches from 61 organizations between 2010 and 2016; I measure the use of qualitative and quantitative analysis using natural language processing on employee résumés. I confirm that qualitative analysis positively moderates quantitative analysis: increased quantitative analysis negatively impacts innovation performance when qualitative analysis is low, but positively impacts innovation performance when qualitative analysis is high. Additional analyses test how data-driven cultures impact product novelty, explore the effect of methodological pluralism within (rather than between) organizational members, establish market uncertainty as an important boundary condition, and investigate the antecedents of data-driven culture. The paper contributes to organizational theories of innovation, strategic and entrepreneurial decision-making, and to research linking organizational culture to strategic performance."The Market Size Paradox of Niche Product Innovations" (Dissertation Chapter 2)
Click to display Abstract
When do niche product innovations—new products that initially appeal to narrow market segments—achieve widespread commercial success? To answer this question, I develop an agent-based model of innovation diffusion in a competitive product market. The model varies two important parameters of the market: demand heterogeneity (the degree to which customers have heterogeneous preferences), and demand interdependence (the degree to which customers’ demand is influenced by other customers’ demand). The model indicates that niche product innovations are more likely to be commercially successful than broadly appealing products when demand heterogeneity and interdependence are high, but less likely when both are low. I empirically validate the model using a consumer panel dataset of over 2,000 new consumer product launches. The results imply a “market size paradox”: under certain conditions, innovations that initially appeal to relatively narrow market niches (as opposed to broad appeal) actually tend to achieve greater widespread commercial success. Thus prior to launch, niche innovations, which actually have more potential for commercial success, may systematically appear to have small potential market sizes according to traditional market-sizing techniques."Experimentation-driven Product Innovation in User-Communities: The Engagement Dilemma" with Rory McDonald and Rob Bremner. Under review at Organization Science. (Dissertation Chapter 3)
Click to display Abstract
This study investigates experimentation-driven product innovation in user communities. Prior research has largely focused on the innovation benefits of leveraging users and user communities as an experimentation resource. In this paper, we posit that reaping the innovation-related benefits of experimentation is contingent upon the degree to which the community provides an unbiased representation of the broader market. Using longitudinal and interview data on experimental PC game development, we find that adapting games in response to community feedback increases games’ commercial performance on average, unless the game’s user community is concentrated in a narrow market segment. Such narrowly concentrated niche communities give signals of market demand that, when incorporated into the game, can decrease the game’s appeal to broader audiences. However, responding to the feedback from such concentrated communities is necessary to maintain and grow the user community. Jointly, our theory and results suggest that firms involved in experimentation-driven product innovation with user communities face an engagement dilemma: incorporating feedback from engaged niche communities has the potential to mislead innovation efforts, but ignoring that feedback may stunt the development of the community resource.Other Working Papers
"Data-driven Decision-making and Organizational Hierarchy" with Kramer Quist.
Click to display Abstract
This study develops and empirically tests a formal model for how organizational hierarchy affects demand for data-driven decision-making. The model shows that although data can substitute for hierarchy by establishing a framework for consensus, hierarchy also increases demand for data because hierarchies require legible and commensurable results. We empirically validate the model using data from employee profiles on a career networking website. We use job titles to measure the span of control across levels of hierarchy in 61 consumer product organizations, and job descriptions to measure the prevalence of data-driven decision-making."The Sequencing of Scaling: Resolving Technological and Market Uncertainty in Entrepreneurial Ventures" with Aticus Peterson