Yihua Zhang

I feel that we are undergoing a cultural invasion by AI, an invasion of the whole of human culture by AI. Almost all of AI's training corpus today comes from the accumulated culture, the writing and the history of every human being who has ever existed. But anyone who has actually worked on training large models knows that all of the text humanity has ever produced, the low-value and the high-value alike, stopped being enough for training the next generation of models a long time ago. And yet AI keeps producing text (books, even), images, video and audio without pause. Before long, the AI-generated share of the text on the internet will be as large as the human-generated share. Not long after that, in the internet's entire textual corpus, the writing produced by humans will amount, in sheer quantity, to the very tip of one hair on one of nine oxen. (I do wonder whether there is some Moore's law at work here as well.) AI looks like it is offering choices, but what it is really doing is forming habits. For most of humanity, in the near future, faced with any occasion that calls for formal writing, people will move from not wanting to write, which is where we are now, to not being able to write, and finally to not daring to write. Not being able to write, because AI is quietly consuming the human capacity to organise (complex) language. Once you only have to supply a few keywords and AI will place a mass of seemingly finely crafted paragraphs in front of you, you will stop thinking about word choice and phrasing, about cause and effect, about how a piece opens, develops, turns and closes. Not daring to write, because you are afraid of embarrassing yourself, or afraid that you cannot compete with everyone else who is using AI. If someone else can finish 100 pages in half a day with AI, and it takes you a whole day to write 2 pages, then external pressure alone may leave you with no choice but to use AI, and no courage not to.

I still remember that when I had just started my PhD in 2022, writing a paper took an enormous amount of preparation, most of which was the process of sorting out my thinking bit by bit. Back then, writing a paper was like making a film. Before any real writing began, we would plan out the paper's sections (roughly how many sections, and what each one was for), then plan the paragraphs inside each section (how many paragraphs each section would have), and then, something people may not expect, as a non-native speaker of English I would even design how many sentences went into each paragraph and what each sentence was there to do. It is not as extreme as it sounds. In a standard academic paper, a paragraph of moderate length is maybe five or six sentences. What is being planned here is "what function each sentence serves", rather like deploying troops on a battlefield, and there is even room for a lot of small tricks of the writing craft. For instance, if I needed to praise something before criticising it, I might spend the first sentence lavishing praise on whatever I was about to disparage (usually the baseline, LoL), then lay an ambush (usually in the second sentence), then make the case for that ambush, and so lead into the problem we wanted to discuss. There might be antithesis running across sections, a few puns quoted from elsewhere, a few sentences carrying my own obsessions and my own venting (though smuggling in a private agenda is not a good thing), and before the writing even started the whole thing might go through several rounds of revision. It is just like writing a script and drawing a storyboard for a film. But saying all of this does not mean that what I wrote then was necessarily better than what gets written now that we have AI. Honestly, in large part it probably was not. The difference is that what I was left with was the process of creation, the evolution of the logic, and my own distinctive way of putting things after having thought them through. Who exactly is "I"? I am slowly coming to feel that expression in writing is one of the ways that "I" becomes visible. If every time I speak it has to pass through a proxy, is "I" still "I"? If in the end I cannot even form a complete view and argue it out, is that view still "what I mean"? People in every country in the world hold a deep hostility toward the censorship of writing by central authority, but in the future, will we willingly let AI review, and even rewrite, every utterance we make?

I consider my own writing fairly poor, mainly because I have read so little. To my shame, the number of books outside the curriculum that I genuinely read cover to cover before university was probably under five. The only ones I remember are Lin Yutang's Moment in Peking, Qian Zhongshu's Fortress Besieged and Lu Yao's Ordinary World, all of them read during high school (in middle school and primary school I read nothing else at all). I find it hard to believe now that before university I had not even properly read many popular novels. The result was that my writing was very bad, and writing an essay in every high school Chinese exam felt like being constipated. But now I force myself to keep up the habit of writing, because what makes writing different from speaking is that it shows, more truthfully, the mark a person has left by having lived. When ChatGPT had only just been released in 2023, using AI was something of a fashion, and I was constantly having AI rewrite my content. For someone like me, whose English is poor and whose Chinese is not that OK either, AI was nothing short of a saviour. But going back now to read the blog posts that AI polished at the time, I still break into an awkward cold sweat every so often. These days, for anyone who has never used AI, the output of AI carries the same kind of filter: AI is excellent, AI is complete, AI is advanced, AI is even less prone to mistakes than human beings are. I believed that deeply myself, so much so that even when I had a piece of text that was already close to finished, I would still unwaveringly ask "AI" to "polish" it for me. AI would then live up to expectations and rewrite my article beyond recognition, and only then would I click publish, satisfied. Back then I even felt a certain comfort in this, because my denial of myself had produced a warped expectation instead: the more this text was changed by AI, the better its quality had to be. Thinking about it carefully, the me of that time had already given up the right to express my own views in full. So if this is also happening to most people, how humanity is to recover its cultural confidence is the question I am now thinking about. Can we believe that, compared with AI, language created by human beings is something humans can in fact use to express and to communicate in a better way, a way that shows individual character more clearly and that does more good for humanity's future development? When everyone is using AI, and everyone believes that AI's output is superior, will we still be brave enough to say no to AI's cultural hegemony?

More and more mathematical conjectures are being proved or disproved by AI, and my own view of this is a little pessimistic. Whatever the field, and however proud we are of what AI achieves, I do not think that should lead us to dismiss the significance of progress made without it, that is, AI-free progress. Put another way: if a field reaches the point where nobody does the work “the old way, by hand” any more, then over the long run that is definitely bad for the field. The particular problem AI creates for mathematics is this. In future, if someone does make a major mathematical advance by hand, they will no longer be able to prove “I did this without AI”; and once you cannot prove it, the best move available is to fall in with everyone else, since you were never going to get the credit for being “AI-free” anyway. So the “by hand” school in mathematics disappears. What effect that has on the field I am not in a position to judge, but I think it is very obviously a negative one. It is not quite the right thing to say, but there is a faint sense of bad money driving out good. I should also add that what the “by hand” school means, and what its disappearance costs, differ from one field to another. So you cannot rebut the above with “AI accelerated software, nobody in IT writes code purely by hand any more, and IT is thriving all the same.” And besides, is the software industry really thriving the way everyone assumes? We may not know that either.