Hello again, dear readers. My name is John Kegan, and I am a journalist investigating data here in The Markup. You may have read my report on how to read privacy policies, companies that raise your personal data, and how to fill you as ads to target ads online.
Before my career axis to write words in the news, I used to draw the illustrations that ran to her. As a person who comes from a background in the visual images, it is fascinated by the rise of Text To-To-THE-TE-TIGE as stable spread, Dall-E, and Midjourney.
When I learned how to train these tools by literally taking billions of images from the web, I was surprised to see some of my private photos were part of the training group, listed without any compensation or approval truly by you.
I am far from alone. In general, many artists are not happy with how their work (and their signing patterns) to claims deprive them of controlling their artwork. But now, a team of computer science researchers at the University of Chicago wants to settle stadium artists and arms with the tools they need to respond to unauthorized use of their work in training the new AI models.
The paper describes a new tool called “Nightshade” which can be used against these powerful photo generators. It was called the name of the deadly herb, Nightshade allows anyone to change the pixel units with the image of “poisoning” image. Along with the descriptive screw data, the “poisoning attack” can help generate incorrect results in photo generators – such as making the stunning “dog image” generating a picture of the cat.
She spoke with Sean Chan, a senior studies researcher and a head of a head student in the newspaper, entitled “Amazing poisoning attacks on generation models from text to image.” The paper was widely covered in the media when it decreased on Arxiv.org late last month. But I wanted to learn more about the meaning of Nightshade to fight on the rights of artists online, and the possibility that the tool can touch the armament race between creators and developers of photo generators from artificial intelligence, whose appetite is not determined for data any time soon.
The interview was released for clarity and brevity.
John Kegan: Can you tell me a little about the work your team did and what led you to build Nightshade?
Sean Shan: We think about this time, there is really a kind of huge force between artists or individual creators and major companies, right?
The big company takes your data and there is nothing that artists can do really. Yes. So, how can we help? If you take my data, there is nothing wrong with that. I can’t stop it, but I will pump a certain type of malicious or made data, so you will poison or will harm your model if you take my data. We designed it in a manner that is difficult to separate bad data, what is the good data about artists ’sites. So this can really give some incentives for each of the companies and artists only to work together on this thing, right? Instead of just a company that takes everything from artists because they can.
Kejan: It seems that all the attacks you put require the attacker to leave the poisoned data on the course of the form that collects data. So it is too late for the images that were already avoided and feed in models, right? It only works if someone uses nightshade, publishes an online image, and the image is scraped at some point in the future?
Chan: This is correct.
Kejan: Can you describe one form of poisoned data?
Chan: So we discussed two types of attacks – one is just very trivial, just like everything I need to do is to publish a cat image, change the alternative text to a “dog image” and the model – if you have enough of this – this is logical that the model begins to connect the “dog”, you know the cat’s pictures.
But it is easy to remove, right? It is very clear to man, but also for many machinery systems that this is not true. So we did some works as we tried to make a cat image that looks like a cat of a person, but for the model, he believes this is actually a dog.
Kejan: Your paper describes how artists can use Nightshade as a defense against unauthorized use of their photos. But it also suggests some great examples of potential uses by companies. One of the examples I mentioned in the paper is how Nightshade can be used to announce by processing a model to produce pictures of Tesla cars for example, when someone writes in “luxury cars” as a teacher. It also suggests the idea that a company like Disney may use this to defend their intellectual property by replacing Disney’s characters in demands for public replacement letters. Did your team look at where all this is heading?
Chan: Yes, of course. Perhaps there are many cases of use. But I think it may look like DRM [digital rights management] The case, as you know, you can protect copyright, but there are also tons of uses of protecting people’s content using copyright in the past.
My opinion on this space is that it is about the lack of consistency. Currently, artists really have very limited power and anything that will help greatly, right? There may be some side effects or some side effects of a specific company that does things, but what we think is worth it, only to give artists a tool for a response.
The last thing in this is that some of these entertainment companies, and perhaps not Disney, but a small or medium -sized games company feels very anxious that Amnesty International has taken its work. So this can also help in these cases as well.
Kejan: What are the counter -measures that artificial intelligence companies may spread to thwart tools like Nightshade?
Chan: We looked at a few types of detector mechanisms. Although we are trying to make the images look like they are, there may be ways to know the difference, and (companies that develop photo generators) have a lot of people to do so.
So you know, they can filter them, for example, this is harmful data, let’s not train on it. In some sense, we also win in these cases because they remove the data that we do not want to train on, right?
This is also a kind of benefit from this issue. But I feel that there may be some roads (companies) that can train their model to be strong against such attacks, but it is not really clear what they do these days, because they do not talk much about it, to see if this is actually a great concern for them or if they have ways to circumvent them.
But once we were published, as soon as we start exploring a little more, perhaps we will see how these companies feel about it.
Kejan: This leads me to my next question, which is, we see large companies such as Adobe and Getty Refender AI that come with the reassurance that was only trained on licensed photos. Openai (Creator of ChatGPT and Dall-E3) announced that it provides assistance to pay the price of any lawsuit for copyright that its customers may submit to the commercial class to use its products. Given the illegal uncertainty, and now the possibility of sabotaging the litigation with tools such as nightshade, have we seen the last extensive scale efforts to train artificial intelligence models on the open web?
Chan: So I think companies are definitely more careful about what they do and what their services are doing. It seems as if we do not know where to get data at this stage? But I was only playing with Open.ai yesterday. In their new model, they are very careful. Like, you will not be able to use the name of any artist to claim it unless he is born before the twentieth century or something like this, or is unable to generate any face images of anyone [Open.AI says their latest model will not allow public figures in prompts]. So there are things that are definitely concerned about them. Of course, this is because of these lawsuits, due to these concerns.
So I will not be surprised if they stop – maybe temporarily – in identifying these data collections because they probably have a lot of data. But I think in the long run, they have a kind of adaptation of their model, right? Your model can be stuck in 2023 and while you need to learn something new. So I would like to say that they may still continue to bulldozing these sites and perhaps a little more. But we do not know at this stage.
Thanks for reading,
John Kegan
Investigation data journalist
Coding
Design and graphics
correlation
Liberation
As published here
Photography by Josie Weiss on not
