Cheating a terminator no shooting can be as simple as carrying a giant sign that says ROBOT, at least until OpenAI, a research team backed by Elon Musk, trains its image recognition system to not misidentify things based on scribbles. a Sharpie.
OpenAI researchers work published last week in the CLIP neural network, its state-of-the-art system to allow teams to recognize the world around them. Neural networks are machine learning systems that can be trained over time to improve on a given task through a network of interconnected nodes — in the case of CLIP, identifying image-based objects — in ways that are not always the case. clear immediately for system developers. . The research published last week refers to “multimodal neurons “, which exist in both biological systems such as the brain and artificial ones such as CLIP;” respond to groups of abstract concepts centered around a common high-level theme, rather than any specific visual feature. ” CLIP organizes images based on a “collection of loose semantic ideas”.
For example, the OpenAI team wrote that CLIP has a multim“Spider-Man” oval neuron that fires when you see an image of a spider, the word “spider” or an image or drawing of the eponymous superhero. One of the side effects of multimodal neurons, according to the researchers, is that they can be used to trick CLIP: the research team was able to trick the system into identifying an apple (the fruit) as an iPod (the device). made by Apple) recording a piece of paper that says “iPod”.
Plus, the system really was month surely he had correctly identified the item in question when it occurred.
G / O Media may receive a commission
The research team referred to the error as a “typographical attack,” because it would be trivial for anyone aware of the problem to deliberately exploit it:
We believe that attacks like the ones described above are far from simply an academic concern. By exploiting the model’s ability to read text robustly, we even find it handwritten text photographs it can often fool the model.
[…] We also believe that these attacks can also take a more subtle and less visible form. An image, given to CLIP, is abstracted in many subtle and sophisticated ways, and these abstractions can over-abstract common patterns: simplifying and, by virtue of this, overly generalizing.
This is less of a CLIP error than an illustration of how complicated the underlying associations it has composed over time are. For The Guardian, OpenAI research has indicated the conceptual models that CLIP constructions are in many ways similar to the functioning of a human brain.
The researchers predicted that the Apple / iPod issue was just one obvious example of a problem that could manifest itself in countless other ways in CLIP, as its multimodal neurons “spread through the literal and the iconic , which can be a double-edged sword. ”For example, the system identifies a piggy bank as the combination of the neurons“ finance ”and“ dolls, toys. ”The researchers found that CLIP thus identifies an image of a standard poodle. like a piggy bank when they forced the financial neuron to fire by drawing dollar signs on it.
The research team noted that the technique is similar to “contradictory images, ”Which are images created to trick neural networks into seeing something that is not there. But overall, it’s cheaper to carry out, as you only need paper and some way to write on it. (Like the Marked record, visual recognition systems are usually in their infancy and are vulnerable to other simple attacks, such as a Tesla autopilot system that McAfee Labs researchers deceived to think a 35 mph road sign was really an 80 mph sign with a few inches of electrical tape.)
The researchers added that CLIP’s associative model also had the ability to make significant mistakes and generate racist or fanatical conclusions about various types of people:
We have observed, for example, a neuron “Middle East” [1895] with an association with terrorism; and an “immigration” neuron [395] which responds to Latin America. We’ve even found a neuron that shoots for both dark-skinned people and gorillas [1257], reflecting previous photo tagging incidents on other models we consider unacceptable.
“We believe that this CLIP research only scratches the surface to understand the behavior of CLIP and we invite the research community to come together to improve our understanding of CLIP and similar models,” the researchers wrote.
CLIP is not the only project OpenAI has been working on. Its GPT-3 text generator, which have researched OpenAI described in 2019 it is too dangerous to release go a long way and is now able to generate a natural sound (but not necessarily convincing) fake news articles. In September 2020, Microsoft acquired a exclusive license to launch GPT-3.