We Must Decide if It Is Two People or Three

Article Preface:

Similar to the earlier post about AI stuff, The Meaning of Accelerated Discovery, this post is not particularly well thought-out. I am certainly opinionated on the subject at hand, but I am not decisively convinced I really know enough to provide a respectable treatment of it. In that regard, we ought to regard this post as not a targeted essay but just a wandering, rambling perspective on the current state of affairs. For example, I say the word “personhood” early on, but I have not fully studied the philosophy surrounding this idea yet.

How many people are in this room with me?

We must already accept that AI, specifically Large Language Models (LLMs), has acquired an effective personhood. It is a new character with which we will converse. The world population may be close to 9 billion, but it is already effectively 20 billion; every person’s experience with an LLM effectively has undergone an interpersonal interaction. To my great distaste, the conversations that we have about the weather, the best coffee in town, politics, or a particular problem we are struggling to solve may find themselves augmented or wholly substituted by the insertion of an AI-related interaction. “Did you see that Fable 5 provided a counterexample to the Jacobian conjecture?” “I asked ChatGPT how to bench 220 pounds before the Spring, and it generated a workout plan for me.”

Interaction with these chatbots can contribute to behavioral interventions, in the sense that what behavior would have followed without the interaction no longer occurs. Obviously, books, music, movies, and art (for example) contribute to behavioral interventions in the same sense. Except these were fundamentally contributed to civilization by people. Books were written by people, music was composed by people, movies were directed, filmed, edited, and acted by people, and art was made by people. Indeed, I may be subject to behavioral intervention when I see a horrific anti-smoking advertisement while I am waiting to return to my Saturday-morning cartoons; I may choose to never touch a cigarette when I otherwise might have entertained the idea. Perhaps I realized that I wanted to spend more time at the gym lifting weights after watching a YouTube video of somebody doing 25 pull-ups without a hitch. On the other hand, I may—fully knowingly it is AI-derived—consume a video depicting Barack Obama, Joe Biden, and Donald Trump playing Dark Souls 3 and laugh. More deceptively, I may unknowingly scroll upon a video of Donald Trump making a speech in front of the White House declaring that he is not afraid to use nuclear force against Iran—except the video is wholly AI-derived. The sense of longing and searching that floods my senses as I listen to the crying violins of On the Nature of Daylight may inspire me to live each day as if it were my last. Now, I may enjoy a good neck-breaking Dubstep beat that was fully derived by an AI.

AI now has the capability to contribute to the pool of creation. Each one of us, when consuming, is faced with contributions that come from both people and AI-technologies and nevertheless leave us with an experience.

As I mentioned in my earlier post, The Meaning of Accelerated Discovery, I sometimes felt third-wheeled by Claude during my experience working on a scientific project. Every two weeks, my colleague would present the results that Claude generated regarding a particular calculation of relevance. Most recently, my colleague said that the chatbot “disagreed with” a particular analysis that we had originally intended to do. This disagreement gave us pause to consider whether or not the chatbot had a point. In the end, we decided to follow the chatbot’s recommendation. It is now, in this day and age, that we must settle on the answer to how many people are present in this social interaction. We must decide if it will be two or if it will be three. (I have been using the name “Claude” over and over already.) While the correct answer is indisputably two, we must decide if there are effectively three people involved.

Moreover, I did not know how to accurately attribute the contribution: Is it his work or is it Claude’s work? Did he just use a typewriter to write the book, the violin to play the strings, the camera to record the video, or a chisel to carve the marble? Proceeding completely and totally according to plan, according to absolutely no deviation or intervention in this inexorable, conventional, traditional, unperturbed march forward, the scientific publication that will report this research will have our names on it at the top, just like all other contemporary scientific publications. Of course, many academic journals now require that we disclose if we used AI on the work—the answer relevant for this research is an emphatic yes. Nevertheless, it will appear as if we, the listed authors, are responsible for the work, and we used AI to assist us in doing it. In this particular instance, my colleague asked Claude to generate MATLAB code that conducts a particular statistical analysis of a dataset and produce a series of summary plots. My colleague has not fully verified all of the MATLAB code that Claude wrote yet. I anticipate that he will not get around to it either: He sent us an entire code repository that contains script upon script upon document upon document that is necessary to run the entire research pipeline. That gives me pause to think if I want my name reported on this piece of research.

But the outstanding question for me remains: Does it even matter?

Does it matter if my colleague does not understand a single line of that MATLAB code that Claude wrote to generate the figures? The most vicious version of this question is the one that supposes that Claude did everything correctly right off the bat. If the analysis was conducted in a mathematically and methodologically sound fashion, then the work ought to be added to our shared pool of scientific knowledge independently of what I think about it and, crucially, whether or not I understand it entirely. That happens all the time in physics: There is no way that I can immediately understand the published works of Edward Witten, Juan Maldacena, Đàm Sơn, Subir Sachdev, and so on. Does my inability to understand this work disqualify it from appearing in scientific journals?

The fundamental quandary is the following: In the past, books were written with writing tools; music was composed with instruments, a technology, or a tool, of sorts; contemporary movies are produced with cameras and video-editing software; art was produced with paint brushes, pencils, a stone and a chisel. Are we to claim that AI-derived products are fundamentally distinct from those that are derived using other technologies? Based on the way this discussion has been framed so far, perhaps we should not regard them distinctly. So, why does it feel different to us? I’d put my name as an author on a book that I wrote with Google Docs but not one that I fully generated with ChatGPT.

In the present case, the AI participates in scientific research. That stands in contrast to participating in the arts, whereby it produces a work of fiction, a pop-science article, a diffusion model-generated movie of cute cats, or a strange-but-compelling psychedelic EDM tune. Modern mathematics and physics pride themselves in rigor—as they should. Physics has the added benefit of the (philosophically) poorly-understood relationship between its (loose) mathematical grounding and the ability to galvanize empirical pursuits. So, participation of AI in physics and mathematics has the ability to contribute directly to both our formally-closed models of deduction and a database of information about the physical properties and interactions of our universe. As a brief example, Claude has contributed calculations of the cross-section of various physical processes that humans have not yet attempted due to the magnitude of the task.

But we once again find ourselves at the same quandary: Does it even matter that AI contributes to our epistemology?

A fundamental problem that we are not in the slightest prepared to contend with is how to co-exist with a completely separate sector of knowledge work that leaks itself into our sector of epistemology. What we must now contend with is the true meaning of what it means to know anything. We already struggled (conceptually) to understand this issue when it was just us humans floating about and engaging in epistemology. Outstanding, purely conceptual but relevant issues such as the Philosophical Zombie (p-zombie) and the Chinese Room extinguished our audacity to claim that just because humans are of the same kind we must have the same “what-it-is-like-ness”, or “qualia.” I may be able to imagine what you feel at this moment, but I will never share your unique experience of it. Nevertheless, I will often give you the benefit of the doubt that you’re feeling upset about your partner cheating on you. But, of course, the Chinese Room problem was more an exploration of an AI that is capable of perfect mimicry of human behavior: Does it even matter that it is not human in the end? The p-zombie is also still unresolved. A p-zombie will still excellently feign heartbreak over their fake breakup, and the feelings evoked within me may be exactly the same as those evoked if I saw my best friend in the same position.

It is only now that AI has become so ubiquitous in our daily experience that these problems are amplified in the way they really ought to have been in the first place; now, it seems that we have no choice but to resolve them.

And I am dissatisfied with this additional workload.

Terence Tao recently spoke about “proof indigestion” in a BigThink interview. The idea behind this neologism is that AI is producing a deluge of proclaimed proofs of outstanding math problems that haven’t yet been cross-verified by human mathematicians. Not only is there an enormous amount of AI-derived work for human mathematicians to get around to, but they are working against an intrinsic “speed of understanding (digestion).”

Those without reverence for the human sector of epistemology will not bat an eye to the frustrated murmurings about proof indigestion. They have already answered: It does not matter if humans understand what the AI-system does. AI is practically capable of contributing to a wonderfully-scaffolded, topologically disjoint sector of epistemology; what humans know or don’t know about what AI knows simply has no bearing on how AI proceeds in this creation. That is, it is us, people, who are responsible for the bottleneck in the process of uninhibited “AI-driven discovery.”

What helps me imagine this conundrum is a sketch of three different “epistemological loops”: the human-to-human loop, the human-to-AI loop (equivalent to the AI-to-human loop), and the AI-to-AI loop. The major problem, now of human-existential importance, is how on Earth we argue that it is good or important that we humans do not get left out of this epistemological loop. We still want a human to verify the work that an LLM produced in order to ensure that it did the right thing, wouldn’t we?

But the question still stares at us right in the face, unwavering: Does it even matter? Does it matter that a human stays in the loop? Humans make mistakes on work all the time as well; scientific fields enforce the peer-review process for academic papers to institutionalize this cross-checking. Humans were always in the loop with each other. It never bothered us (in the grand sense that overarches this discussion) that people published results that were beyond our intellectual capabilities at that moment; we simply acknowledged that, yes, Edward Witten probably knows what he’s doing with the superstrings; Landau and Lifshitz probably have a fantastically abstract explanation for the concept of entropy; John David Jackson can probably calculate the electrostatic potential of a triply-nested coaxial cable in three lines of math.

Now, most people (I wager, at least most physicists and mathematicians) would want to learn the details of the mathematical theorems or counterexamples or proofs that Fable claims are correct. Most people would not be comfortable immediately applying the mathematical results that are generated by an LLM to another novel problem also passed to an AI. As Terence Tao might say: Some people would want to take that wandering walk in the forest to find the fabled waterfall. When it comes to published contributions to these fields, we may have an impulse to stop and check the work, to look for mistakes or oversights.

But with regards to AI-produced work, there is not a compelling reason as to why we humans really need to stay in this loop. Why do we not just let the LLM continue to churn out result after result fully autonomously, letting it discover what is really out there? It effectively has, after all, personhood: Reading academic papers or books fully generated by AI-technologies is the same kind of experience as reading a paper produced by human researchers; watching a video made by a diffusion model has the same experiential content as watching a video created by PewDiePie; reading a book half-generated by an LLM is still reading a book; consuming a fully-synthesized orchestral derivation generated by an reinforcement-learning model is still listening to a piece of music. (Practically speaking for a physics graduate student, the total number of academic papers that now remain on their backlog has increased due to the addition of AI-generated or AI-assisted research.)

I don’t particularly know what underlies this feeling of discomfort. Without delving into a full psycho-analytic treatment of it, I suspect it is related to the foundational debasement of not only our (feeling of) control of a situation but also our faith in the “human creature” to generally share this feeling. When it comes to the pursuit of knowledge, humans acknowledge in some sense that other humans have different knowledge. One thing that comes to mind is an observation by Amanda Ripley that one of the first things that people do in a time of life-threatening crisis is—after they’ve convinced themselves there is indeed a crisis occurring—is something called “milling.” Milling is the terminology used to refer to the act of people subjected to the same crisis environment getting together to discuss what is actually happening and how they should respond to it. To frame this behavior in a suggestive way, the individual turns out to the group to gather more information on how best to survive. In other words, there is a sense that the individual needs the information and cooperation of others in that moment; we can individually benefit from other people having different knowledge.

Humans maybe know (or feel) this about other humans, but we don’t have this feeling towards AI technology; we don’t yet have the same instinctive epistemic trust toward AI. We still want to verify its calculations, to re-derive its results, to go through its code line-by-line, just to make sure it didn’t do anything wrong. Or maybe we don’t.

If I have learned anything during my short duration on this planet, living through the imperceptible trends of change in every facet of life, from technology to language, it is that unfortunately we will not make the right decisions. In this day and age, we have erected structures that incentivise cutting corners and doing a quick job: “accelerated job-doing.” An optimist might hope that the rise of AI is finally the moment in which the extraterrestrial arrives at Earth and we all recognize that we are one in the same species and come together against a united front. Perhaps the solution to all of the previous questions about the interplay between people and AI-technologies is that humans just need human interaction and human knowledge, that AI contributions fundamentally do not commute with this space. In that regard, let us double-down on caring for each other, and learning from one another.

The optimist is mistaken. The human being has not yet even partially tangled with its most ultimate frontier: itself. War, between us and ourselves, shall persist; corruption will permeate; inequality will inflate. The introduction of AI has expanded our purview of experiential content and what we must deliberate over, with other humans and with AI. We will no longer just discuss the weather, the best coffee in town, politics, or a problem we’re trying to solve; we will discuss how the AI tried to solve it, too.

Notes/References:

1. “…a video depicting Barack Obama, Joe Biden, and Donald Trump playing Dark Souls 3…” This is real. You can find any one of these videos on YouTube, but there’s apparently an entire channel dedicated to this content entitled AI Presidents Playthroughs.

2. “…Edward Witten, Juan Maldacena, Đàm Sơn, Subir Sachdev…” are working physicists (as of this writing) who are incredibly prolific in the field. By the way, I do not mean to imply that these physicists’ papers are impossibly difficult to read! Actually, they typically write quite good papers. But you do need to spend a lot of time with some of them.

3. “EDM” stands for “Electronic Dance Music.” You know what that is: Dubstep, house music, Drum & Bass, techno, trance, etc.

4. “Claude has contributed calculations of the cross-section of various physical processes…” This is something that Dr. Matthew Schwartz has claimed at the American Physical Society (APS) and published about in, e.g., Learning the Simplicity of Scattering Amplitudes

5. The “Philosophical Zombie” is a concept that is mainly attributed to philosopher David Chalmers in his book The Conscious Mind . It is what you might expect: Sort of related to solipsism, imagine a person who acts, talks, walks, and looksmaxxes like a person, but under the hood actually has no conscious, subjective experiences unlike you and me. That is a Philosophical Zombie. In principle, your best friend could be one: What experiment can you run to prove to me they are not?

6. The “Chinese Room though experiment” was coined by philosopher John Searle in the essay Minds, Brains, and Programs . The idea here is to design a hypothetical situation that wholly obfuscates the neat separation between so-called “semantics” and “syntax.” Unlike the Philosophical Zombie notion, the Chinese Room thought experiment really is about a sufficiently-advanced AI system: If an AI system has access to only a fixed set of rules such that they can produce fluent Chinese output that is indistinguishable to a native speaker of the language, then who is to say that the AI system isn’t fluent? But the AI system only knows the fixed set of rules (syntax) and does not actually understand the put-together Chinese sentences (semantics). (The summary I just provided here is pretty poor! I think you ought to consider consulting either the Wikipedia page about it, Chinese room, watch Jeffery Kaplan’s overview of the paper at The famous Chinese Room thought experiment - John Searle (1980) , and, as a last resort, actually read the paper Mind, Brains, and Programs !)

7. “Terence Tao recently spoke about…” The interview is available on YouTube. Here is the link to the video with the relevant timestamp about “proof indigestion”: The paradox at the heart of AI and science | Terence Tao .

8. “One thing that comes to mind is an observation by Amanda Ripley…” This comes from Amanda Ripley’s BigThink interview—lot of BigThink on this article! The part that I am taking inspiration from is here, with the timestamp included: Who will you become during a crisis? | Amanda Ripley .

9. “As Terence Tao might say:…” He draws this comparison right at the beginning of the video The paradox at the heart of AI and science | Terence Tao .

10. “Those without reverence to the human sector of epistemology…” In the Nature opinion article Should artificial intelligence be interpretable to humans?, Dr. Schwartz is back to essentially rail against the anthropocentric bias, concluding that he “…cannot wait to read research papers generated by AI, solving problems with which humanity has long struggled…” You should also notice that in this article he is citing Thomas Nagel’s essay What is it like to be a bat? : Even Dr. Schwartz is aware just how much the advances in AI add to the tension in the outstanding questions regarding human experience and its relation to the mind-body debacle.

Updates:

None yet.