A short break. The French translation will begin a bit later.
So if you hear it, wait.
Good evening. And I'm so happy to be here.
This is the first time for me speaking in front of you at a stage.
The age of me passively being here is over.
And there is a part zero of this presentation and this is about
emotion and this is going to be a kind of emotional part of me.
All of the things I'm going to talk about, Wikipedia and
so on, wouldn't be possible without the chaos in general.
And people from CCC and people organizing these conferences.
And when I say Wikipedia wouldn't exist without you, I think that this is not hyperbole.
This is an actual statement that I truly believe in.
To give you some context, this is me at an earlier chaos communication congress.
The building you will see is the BCC in Berlin.
And we were fortunate and happy to be part of this conference.
This was one of the first meetings of Wikipedians in Germany in person.
I mean, we knew each other from IRC and other media, but this is
a whole new different game of interacting with people and being
in a safe, nurturing space, having great discussions and so on.
So thank you for making this whole thing possible and being part of Wikipedia's history.
You in person, but also you as an ecosystem and a community of people. Thank you.
So a couple of things about me.
So I joined Wikipedia as an author 2003, so two years after it was founded.
I became a co-founder of Wikimedia Germany.
You might have seen colleagues of mine at the Open Knowledge Habitat.
If you didn't visit them, give them a visit in the next days and say hello.
I was also an employee there and I did a bunch of other stuff.
Anke, you are part of this other stuff I did in between.
And it's helpful to just mention the fact that I currently work at a tech company
and this tech company is using AI, so take this as a sort of caution, I think.
But the presentation is completely unrelated to the daytime work I'm doing.
This is me in a private capacity as a Wikipedian.
And we can talk about intersections afterwards. I'm happy to do so. So, part one.
This is a story about AI.
And it begins with the one phrase that Wikipedians all over the world
have been saying whenever being asked about the quality of Wikipedia
and saying, Wikipedia might be a good place to start research.
It's definitely a terrible place to end research.
And if we truly believe this statement, then there is a moral obligation
of Wikipedians to provide resources to continue research somewhere else.
And one of these tools is providing literature and references.
And these references in Wikipedia are important. They are an integral part of it.
And we love any means possible to make access to
these materials that we reference easy and possible.
So, if you just mention an ISBN in MediaWiki, it
will automatically be converted into a special link.
And the special link will take you to national libraries and to
bookstores and to smaller libraries and to second-hand places.
So, any place you could find this literature should be mentioned there, one way or another.
And the ISBNs... ISBNs in general have lots of cool features.
There is a hierarchy in there.
So, the first character of the ISBN is going to tell you about the language.
And there are different categories of parts that relate to the publisher.
But the last digit is a checksum.
There are two different formulas for calculating the ISBN 10 and ISBN 13 checksums.
But they all boil down to basically the sum of the product of different digits.
And there is a formula and there is a mathematical proof that it is
impossible to switch to numbers without resulting in a mismatched checksum.
So, if you use an ISBN, there is a good chance that you
will be able to verify that the ISBN is indeed a valid one.
And as part of me playing around with technology, I wrote a checksum checker.
So, I downloaded the whole content from the German language Wikipedia.
I wrote a small parser looking for ISBNs in various places.
Some of them are in templates.
Some of them are just mentioned in the article itself.
I extracted the ISBN and then I ran the checksum
algorithm to see if there are any mismatches.
This is pretty straightforward. This is not rocket science by any means.
This is just parsing, in the case of German language would be 30 gigabytes of text files.
It's something that should have been done earlier.
And in some cases, it might have been done already by someone else.
But I wasn't aware of this work being done.
And I started this work with two use cases of things I was hoping to find.
The first one is ISBN references with mismatched ISBNs because of any kind of human mistake.
So, the idea was to find ISBNs where people had inadvertently
switched different numbers or where a one had turned
into a two or a zero into a nine or something like this.
Any kind of honest mistake.
And the second category is a bit funny because in the 80s and 90s, many
publishers were completely unable to understand the concept of checksums.
So, they usually just incremented their ISBNs by one.
So, the next volume just became plus one and so on.
And libraries all over the world are capable of handling these situations.
So, in most library catalogues, there is a special field for saying,
well, this ISBN is technically wrong, but it's printed on the book.
And if it works like an ISBN and it works, then we're just going to accept this.
So, if you come across these, usually from the 80s, 90s, wrong
ISBNs, they are still going to work in library catalogues and so on.
Sometimes you will get an angry remark saying, this
is a wrong ISBN, but it still works as an ISBN.
And Wikipedia has a template, in the German language, it's called ISBN falsch,
simply saying, yes, technically the ISBN is wrong, but the link is going to work
and nothing is happening preventing you from accessing the content
or finding the library in question that has the
book or any kind of physical or non-corporal medium
that contains the information you're looking for.
What I did find were three categories of errors.
The first one, I found a couple of them. I corrected them.
I found the second category of errors.
And I could simply add the template to indicate, yes, it's technically a wrong ISBN,
but it's still going to work like one because everybody else is aware of the error.
And then I found a third category that I wasn't aware of and I hadn't anticipated.
And this gave me quite a headache.
So I found articles in Wikipedia that contained two,
sometimes small, literature references containing references
that had checksum errors.
And the book titles and the author names, they all sounded plausible, familiar,
and sometimes they were indeed the names of authors that are published in this field.
But I couldn't, for the life of me, couldn't find the literature in any catalogue.
I could find similar literature, I could find things of similar titles,
but I couldn't find the literature itself.
It's always something that was missing or anything like this. And then it hit me.
And you probably know how this is going to be resolved.
These literature references were hallucinated by, in most cases, by ChatGPT.
So people had asked ChatGPT to write Wikipedia articles,
and in their compliance and in their training data as trained,
the large language model simply provided both the article
and a set of completely hallucinated literature references,
and people had simply cut and pasted this into Wikipedia.
ChatGPT did what it was asked to do, and people thought what they
did was the logical next step of dumping the content into Wikipedia.
Wikipedia is not alone in this discovery.
So the University Library of Hagen, already in 2023, reported
about students asking the librarians to find literature
references that matched existing journals, existing issues,
but the other information was simply unable to be verified.
The International Committee of the Red Cross has issued
a strongly worded warning on hallucinated references,
because the next thing they encountered when telling people that the
literature or the references they are looking for does not exist,
is they are being accused of hiding the truth.
So this is a much more unpleasant reaction than the ones I was aware of until then.
In most cases, it's simply a waste of time of staff at
libraries hunting down literature that doesn't exist.
There's a beautiful article about the situation in 404 Media.
I can highly recommend this.
And I forgot to include the link, so I failed on the providing references part.
And this is the part where this turns from a
story about AI into a story about AI and humans.
And I would like to remind people that this is,
as such, not a new issue we are dealing with.
So this is an old XKCD comic making fun or describing a
scenario in which circular references are being created.
Someone makes up something, puts it in Wikipedia.
Someone else is copying the information, putting the stuff into a newspaper article.
Someone else is saying, oh, there is a citation missing in the Wikipedia article,
and by this time there is sufficient information online supposedly verifying the claim.
So the circle of reference is established and it's getting stronger.
There are hilarious examples in which the German
president, not the current one but an older one,
gave a Laudatio to the German national soccer team, the female soccer team,
praising progress in gender equality and making remarks on how former
teams were awarded dishes as a present when winning championships.
And this was half true, some part of it was true, the other one didn't turn out to be true,
but now it's a circular reference because people are
referencing the German president for making this claim.
And it's increasingly hard to disprove the non-existence
of a fact when there is plenty of evidence to cite from.
So the best way to stop such a circular reference
is to prevent it from ever getting it started.
And yes, there is a bigger issue at stake and it's
not just the one article that is now contaminated.
It is, I think, a good example on the shortcomings of existing large language models.
And you might have seen articles discussing whether this
is a systemic issue that can never be fully resolved,
that large language models are machines designed to
provide plausible tokens following a certain sequence.
So reality or factuality is never an issue, it's about plausibility.
And this is what one could describe as a kind of anti-knowledge.
The anti-thesis to what an encyclopedia project like Wikipedia is about.
And also there is quite a deep irony in there.
Wikipedia has proven to be one of the most prominent source
of training data for most of the large language models.
So these model providers are using Wikipedia.
They are using tons of other information too.
But Wikipedia content is useful because it's multilingual.
It has a rich density of information.
There is a network of information and links between. It's multilingual.
In so many ways there is a version history providing kind of a temporal concept.
So the large language model providers are in some way poisoning
their own pond from which they are drinking the water from.
And there is, I was, I gave a similar speech in
the conference of Wikipedia a couple of months ago.
And I was told, as an offhand remark by someone, these
model providers are approaching organizations for content.
They are still in the acquisition phase of asking for more content.
And they are asking for content that is proven to be free from synthetic information.
So they are quite aware of the issue and they are paying a premium
for getting content that is free from their own contaminations.
If there is an irony in there, I may be by now too angry to see it.
But your mileage may vary. You might still see the irony here. Back to Wikipedia.
Is there a consensus how to deal with the content?
And as far as I can tell, and this is something
that is always kind of a status quo discussion.
There seems to be a consensus building up saying that at least
the undeclared use of large language model is frowned upon.
And people have been banned from Wikipedia for dumping
large language model created content into Wikipedia.
It's a kind of vandalism that we have seen before.
We have dealt with this because we were concerned about
copyright issues, about people doing all kinds of things.
But in this case, there is some sort of consensus building up.
Other languages are falling to some degree.
There is a new speedy deletion rule in the English
language Wikipedia that has a slightly higher threshold.
And saying, okay, this is about blatantly obvious content identified as LLM generated.
I think this is a good first step in the right direction.
But at the same time, the bar keeps getting graced by the changes
in how the models work and the kind of content they are producing.
So, does the policy work? Also hard to tell. Maybe to some degree it does work.
But in general, there is too much unknown at this point because this
scenario I described of content being identified as LLM generated
because two or more ISBN references mismatched.
This is a highly specific scenario.
And most of the cases where content has been dumped into Wikipedia has other issues.
And there are other telltale signs that help Wikipedians identify this content.
At a conference in Potsdam a couple of months ago, I was told
that most current large language models are completely unable to
consistently produce the kind of MediaWiki syntax that Wikipedia uses.
They are still falling back to Markdown language.
Sometimes they are using the English language
template names rather than the German template names.
And then they sometimes translate the English template names into German.
So, this might result in the usage of non-existent templates.
Sometimes they add parameters into the templates that do not exist.
So, these are things that a slightly skilled Wikipedian is able to detect.
But there is no fixed set of small rules that
helps people to identify LLM generated content.
It's always kind of a one screw up and then kind
of a backtracing the actions of certain users.
And this is the next part is the why.
So, the first part is the what the fuck part and the other one is the why the fuck part.
I'm going to present four different theories on why
people are doing this and these might not be conclusive.
And I'm refraining from saying it has to be one
of these four, but I'm open to new suggestions.
The first one is the explanation stemming from the "assume good faith" rule in Wikipedia.
Some people might simply not know what the capabilities
and limitations of large language models are.
And they are in some state of blissful ignorance
thinking that they are doing something good.
So, they might consider their dumping of content generated
by LLMs as kind of a service to the whole community saying,
yes, we added a new article. It looks like knowledge. What's wrong with that?
I have yet to find a person qualifying for this kind of explanation.
I would like to see one someday. The people I encountered
with, the people I talked to, do not fall into this category.
Then there is one, and this is the, so we are
moving into darker and more frustrating territory.
It's people being aware of their inability to contribute to Wikipedia in
a meaningful manner and trying to compensate this by getting outside help.
I'm trying neutral description, but it might not, I might have failed on this one.
So, in theory, this could at least be kind of a non-evil, well
-meaning thing of saying, okay, I'm dyslexic, I need a spell checker.
I can't do proper research, so I need an assistant to do so.
It might be some sort of equivalent. I'm happy to meet these people too.
Then there is the agenda part, to which I'm happily going to refer to a
presentation made from Asaf Bartow at the Wikimedia Conference this year,
where he described the weaponization of LLMs, of saying, okay, let's change
10, 100, 1000 articles through the use of one, just one motivated person,
rewriting history, rewriting this. He had fun exploring the limitations
of LLMs and the safeguards and LLMs saying, I'm not going to do this.
I also had fun during his presentation jailbreaking these safeguards and
talking large language models into collaborating with me to rewrite history.
And there's always the fourth one, it might be something else.
So, this is the catch-all explanation, it might be something else.
But then again, this is an open invitation for you to add your experiences.
In most cases where I discovered content written by
large language models and dumped into it from users,
I tried to contact these users, I tried to get them to confirm the usage of LLMs.
And there was one question I asked them consistently is, please show me the prompt.
I asked them to tell me, could you please show me the prompt that you used?
And this is now my kind of litmus test of saying,
if you are climbing into the ignorance part,
saying, oh, I just used this as a tool to copy edit the work I already did,
it shouldn't be a problem for you to give me the prompt.
Most of the large language model providers have share functions,
allowing you to completely share the conversation if you like.
And the amount of excuses I got is impressive in terms of creativity.
So, I was never logged in, the dog ate my prompt, I cancelled this, I can't find it anymore,
oh, how dare you ask me these questions, these are sensitive information.
This is not a proof that is made to be foolproof,
in a sense, but this is my personal litmus test of testing whether people are sincere
and whether their ignorance is plausible.
And this might be the reason why I might sound slightly more frustrated than I usually are
when it comes to technology, because I like technology and I like to play with this.
And I'm still hoping that things are getting better through some sense of useful technology.
So, this is part three, this is about humans.
Remember the slide when I said I wrote the checksum checker?
No, I didn't write it.
I theoretically know how to write it, but I'm not good at it,
and I'm getting easily frustrated.
So, the checksum checker was written through the use of an AI.
In this case, it's Claude Opus.
I had so much fun talking to a large language model, providing me with Python
and later with a C code to parse Wikipedia content.
The source code that was the result of this conversation is released on GitHub.
It is openly declared to be a result of large language model use,
and I personally think that this is an acceptable use of large language models.
But I'm also open to criticism, and there are plenty of avenues how to base this on.
At this point, it's not a bold claim to say that AI might not be going away anytime soon.
Some companies might be going away, some business models might be going away,
but in general, the whole concept of transformer-based large language models
is in some way, shape or form going to stay.
And I'm slightly concerned about Wikipedia as a project,
but also the impression that many people are simply completely unaware of the limitations,
and the lack of responsibility of companies properly explaining these limitations.
The ISBN checker is unlikely to stay a tool to detect this content,
because I strongly believe that at some point the large language model
will be able to hallucinate proper checksums,
and even in best-case scenario we are at a 1 in 10 or a 1 in 11 chance of getting it right.
I believe at this point the chances are already slimmer,
because there is a pattern in this generation of numbers.
And this is why I say this is about humans.
The AI is never, and this is, IBM said this in the 70s or 60s,
AI is never going to be able to assume responsibility.
This is always the responsibility of a human.
And the first part has to be some sort of conversation about
who's taking responsibility for the use of tools,
and what are the conditions in which this use is accepted by a community,
a society, someone else.
So the minimum requirement I was able to identify
is the open declaration of the usage of LLM.
And after I gave a similar talk in Nairobi,
I was approached by researchers who are working exactly on that
and trying to write proposals.
And the part that makes me sleep slightly better than I should be able to
is the fact that the feedback I got from humans
is overwhelmingly constructive and positive.
So the first thing engineers from Wikimedia Foundation did
is they tried to recreate the numbers
and check whether other languages are affected too,
what the percentage of these issues are,
and yes, other language editions are affected too.
The small checksum tool I did contributed to the German language Wikipedia
being the only Wikipedia edition had to drop in mismatches in ISBNs.
So it is possible at a human scale to significantly influence the quality of an issue
or the quality of Wikipedia in certain areas.
I'm happy that other people in other language editions are helping too,
but it's definitely not enough at this point.
So this is the part where it becomes speculative and kind of a plea for help
and saying this is not something I can do myself.
This is something I can point to the issue.
I can say, how do we communicate the issues and limitations?
How do we, how can we look for tools that can mitigate the issue?
Is there, aside from the technology, is there a legal, a political, a moral pathway
to shame companies into improving their tools?
So, for example, ChatGPT saying in a slightly light grey on dark grey colour,
oh, the results might be inaccurate. Is this enough?
Is the European Union going to help us in the application of the AI Act, for example?
This is the conversation I'm taking part in, and you should too.
And the last one is the kind of a one-size-fits-all question.
So, is there still a place for an encyclopedia in this world?
And this is a question I'm asking completely unrelated to other people in other countries
releasing LLM rewrites of Wikipedia or things that have different agendas.
So, the part I really liked is the aftermath of me pointing out the issues,
the feedback I got, the constructive, helpful, slightly concerned criticism
on what should be done next, how to improve this.
My pe rsonal to-do, and this is unchanged, is I would like to integrate
actual bibliographic databases into this check to see, okay,
is the title matching to some degree, are offered names matching years, publisher names?
Are there any available reliable data sources from national libraries
that could help me find more hallucinated content that happens to have a correct ISBN
or doesn't have an ISBN at all? And also, this is a plea from my side.
If you are a librarian, if you want to become a librarian, join.
I published the source code, which was, as I said,
to a large degree LLM generated at these places.
I'm also doing a rewrite into a more flexible tool that does a couple of other tests too.
And I'm honestly surprised about the quality of the code that I'm receiving.
And this might be possible because I wouldn't be able to recreate it from scratch by myself.
I'm pretty sure that this is something that other people feel differently about.
At this point, thank you so much for attending this talk.
I feel obligated on wishing you a good night after the Q&A session.
And also, don't forget the 1-2-8 rule because
there are plenty of more interesting talks tomorrow
and you have to be in good shape for that. Thank you so much. *applause*
Yeah, thanks for the talk. We have time for one question there from the internet.
So, the signal angel.
Signal: Okay, there is a question on how can you identify AI-generated content
when it doesn't have any obvious errors like you mentioned?
And doesn't it have a risk to flag false positives?
Mathias: Yeah.
At this point, I don't feel qualified to identify,
to positively identify LLM-generated content.
There are some telltale signs about the use of adjectives, certain structures.
As I said, there are syntax and template issues. This is just a suspicion.
And there are a couple of other reasons why you have to be careful about this.
And also, there is a general reluctance of admitting the use of LLMs for many reasons.
At a certain point, it's up to the community to decide
whether the suspicion al one should be reason enough
to quarantine the content and to see what else should be done.
But in many cases, you can work backwards and say,
okay, I found a person publishing something that
is clearly AI-generated for a couple of reasons,
and then I can go backwards.
And the community consensus in Wikipedia seems to be
that if the person is unable to provide specific evidence
on the starting point of when the AI use started, the cutoff date is simply November 2022.
And we have to delete the things that have been uploaded after November 2022.
It might sound harsh, but I don't think that Wikipedians are
able to scale to the extent that LLM models do right now.
Herald: And thank you again for this great talk, and I ask of you a great round of applause.