On epistemic authority and how humans commit LLMs to courses of action

Samer recently posted about our just-accepted Strategic Organization paper on LinkedIn. The usual likes, congrats, and promises to read followed. Some comments engaged with the content of the paper (he linked to the pre-print, as I have here) and discussion ensued. Got me thinking, here we have professionals who collaborate with each other, engaging in public discourse about problems in the world while using shared and similarly embedded core vocabularies (same words same meanings). Sounds very much like my operationalization of a research conversation. Interesting. Feels different, somehow, from old academic twitter. Maybe because we imbue LinkedIn with some institutional meanng, its where jobs are posted, its more "serious" than twitter was, its more like a conference than a public square. Less hoi polloi more formal strutting. Interesting.

More to the point of the subject line of this post, I’m referring to Joel Baum’s comment on Samer's post and the subsequent replies by Saku and Henri Schildt. 

Saku in his comment says,  "LLMs actors are not capable of committing autonomously to courses of action. They can be committed, but this has to be done by a human actor.”  Henri then responds with an example of an LLM that he himself had committed to a 10-point course of action.

 This provoked a thought: 

How do humans commit LLMs to courses of action, and how is it consequential for notions of epistemic virtue, authority, and agency? 

I’m unfamiliar with Henri’s technical capability, but I’m assuming he’s like me: knowledgeable but not a seasoned everyday programmer/practitioner. Likely that the10 point checklist is the result of Henri primarily engaging in dialog with Claude’s artifact or code interface, and maybe a little bit of direct programming when refining. It is, in effect, Henri’s distal awareness, in the Polanyian sense, of his (I’m inferring) moral reasoning, to whatever extent articulable into propositional form, interpreted by Claude and translated into programming language. I can infer this because I know Henri to an extent, and the archtecture of Claude and the Sonnet model to an extent. But this is perhaps not immediately apparent to a casual reader of the linked in post and comments.

Relatedly, Charlotte Cloutier and I organized another AI and interpretive methods workshop on Friday where we used Matt Grimes’s Claude-based management research feedback tool. You give it a manuscript and it gives detailed feedback on literature and problematization, novelty/contribution, and empiricial validity/rigor. Both Ann Langley and Fannie Couture independantly came to me curious about how it had learned how to give feedback like this. It became immediately apparent to me that they had some impression of Grimes feeding AMJ papers and human feedback into a system which learned how to give feedback, perhaps with some shaping/refinement by Matt.

i understand however that it was initially designed/intended to give feedback with AMJ in mind, and was the result of Matt articulating in dialog with the Claude artifact interface, his awareness of his own feedback giving process, evaluation criteria for “good” research, and aesthetics. The artifact used the sonnet model to produce a summary of the paper presented as a “research canvas”, and a separate “feedback view”, which produces your typical 2-3 page feedback. But wait there's more! If all the detailed feedback bells and whistles are clicked, it produces fairly consistently rergardless of manuscript stage or length, around 39 pages of 12 pt text feedback. 

To me, there’s this interesting taken-for-granted notion among most users of AI as being trained on traces of human action, with little awareness of AI as programmed in the traditional sense. 

This is consequential when we expect these technologies to act with epistemic authority. 

We commit LLMs to particular courses of action by articulating our worldview, morals, virtues, evaluatory logics, aesthetics, etc, as a process. As a sequence of if-thens. That's programming. Sure its not necessarily directly in python or whatever, but its still programming. And yet most lay users will imagine training in some sense because Matts and Henri's tools run on LLM infrastructure. 

Is it "better" in some sense if Matt's tool had learned the "objectively good paper" from the publication history of AMJ? Maybe it'll only ever be good at helping you make your paper like "what AMJ has published in the past", rather than "a paper publishable in AMJ" (there's a difference). 

Would people be less or more likely to heed the tool's feedback because Matt articulated, manually, his expertise? 

Claude artifact interface, here used to make an analog for doc2excel