Meta used a pirated library of millions of books and papers to train its AI because they thought everybody was doing it
In January, we learned from internal Meta communications revealed in a copyright lawsuit that the company downloaded LibGen, a massive collection of pirated, copyrighted works including millions of books and academic papers, to train its Llama AI model. This legally dubious move was approved by “MZ.”
More details are emerging surrounding this consequential decision as the lawsuit plays out. New court filings detail the internal deliberations within Meta involving researchers who knew using pirated works was a big no-no, but they did it anyway, as they suspected their competitors were using the archive, too. Meta employees wrote:
“everyone is using lib-gen (startups, but also google, openAI)”
“And I’m pretty sure other folks have no issues taking all of libgen 😊”
The Atlantic took a deeper look at what exactly is in this dataset. Using a “snapshot” of the archive (just a list of what is in there, not the works themselves), they created a search tool you can use to find exactly what works were in the archive. Authors who found that their works were in the dataset have taken to social media to express their outrage.
More details are emerging surrounding this consequential decision as the lawsuit plays out. New court filings detail the internal deliberations within Meta involving researchers who knew using pirated works was a big no-no, but they did it anyway, as they suspected their competitors were using the archive, too. Meta employees wrote:
“everyone is using lib-gen (startups, but also google, openAI)”
“And I’m pretty sure other folks have no issues taking all of libgen 😊”
The Atlantic took a deeper look at what exactly is in this dataset. Using a “snapshot” of the archive (just a list of what is in there, not the works themselves), they created a search tool you can use to find exactly what works were in the archive. Authors who found that their works were in the dataset have taken to social media to express their outrage.
