What happens to Reflexivity? Centering the traditional qualitative research process in a world of AI tools

Watching the curve bend toward AI.

I am seeing qualitative work drift toward computation across my field of managment research. Certainly a lot of it at the academy in Copenhagen this year.

This is not a surprise. As organizational digital traces become increasingly abundant and in some ways essential to inquiry what kinds of questions become thinkable that were previously out of reach, and what kinds of questions become harder to keep going? Is there even a future where we can get away with not using AI?

In this blog post I want to noodle on some thoughts on using AI while keeping the traditional qualitative research process at the center. If one thinks of  research with AI as a spectrum that starts on the one hand from “computationally entangled research” meaning a more-or-less traditional scholar using off-the-shelf tools to enagage with a corpus, to the more invovled prompt-based structuring of qual reserach (see Matt Grimes or Henri Schildt's efforts), all the way to “interpretive data science” where a coder writes and curates an algorithm like LDA or text2vec to engage with a corpus, what role does the "algorithm" play? How, within inductive and abductive traditions (as well as hypothetico-deductive designs) does this play out? 

The many “ings” of the research process and which tools belong where.

I'm a process guy at heart, and my personal preference is for qualitative research. And so for me, qualitative research can be thought of as patterned progression (not necessarily linear) of doings. The list of "ings" includes problematizing literatures, formulating research questions, choosing settings and data, sampling, summarizing materials, representing what we see, testing ideas against counterexamples, validating patterns, abstracting, connecting abstractions to theory, theorizing, articulating a contribution, writing, reviewing, and more. I'm sure I'm missing steps, but you get the gist. 

In this view of research as processual and variable, I'm intrested in examining which tool(s) belongs where, and how should/can they be used meaningfully? For each "ing" what is/are the right tool(s) for the job and how, what are the specific affordance given an "ing"/tool relation? What are the risks? And what belongs in the methods section so readers can see the mix of human judgment, agency, and what role computational plays?

What does reflexivity mean in this context?

All of this raises a reflexivity problem. 

Regardles of where you sit in the spectrum above, using these tools to do research creates a new form of knowing in which tools shape what we notice and what we know, and what the algorithm spits back at us. Also- these tools vary in opacity depending on the tool itself, the researcher’s skill, and the conscious and intentional effort to unbox. 

In this context, what does it mean to be reflexive, as a philosophical stance, and reflexing, as specific doings in the various part of the qual research process, along the spectrum of tools and usage behaviors alluded to above? 

I dont have the answers, but I'm excited to explore. 

Note: These questions emerged in the context of a series of conversations I've been having with Prof. Charlotte Cloutier, who's working on a book on qualitative research with Ann Langley and Kevin Corley (all notable qual scholars who are interested in methods), and with whom I've organized a few workshops at McGill/HEC Montreal over the last few months where we attempt to use a variety of LLM-based tools to "do" qual. 

Case in point: Google's NotebookLM can create a "timeline" from an unstructured corpus of texts- here, a set of depositions and statements from a 2022-2023 inquiry into the UK Horizon IT scandal