In my testing of Copilot, the content filters only work on input, not output.
Attempting to generate text from code containing "genocide" just has Copilot refuse to run. But you can still coerce Copilot to return offensive output given certain innocuous prompts.
Ahh, so it's the most pointless interpretation of the phrase "filters to block offensive words", where it is stopping the user from causing offense to the AI rather than the other way around.
I believe the concept is to stop users from prompting the AI to generate offensive stuff specifically, and then publishing the so-generated stream of offensive stuff as negative PR for GitHub, in the same way the generated stream of offensive stuff coming from Microsoft’s AI was a big PR disaster.
I suppose you’re referring to the AI Twitter bot that initially was very lovely and within a day 4chan had turned into a nazi. That was both very naive and hilarious.
The big difference in this case, however, is that this AI was constantly learning based on user input, however, which I do not think is the case for Copilot.
Copilot is indeed constantly learning based on user input (as detailed here [1]) but it seems to be more high-level ("did the user accept or deny suggestion XYZ" and potentially what changes you make to suggestions after accepting) versus just dumping everything directly back into the model a la Tay.
They probably don't want to repeat Microsoft's incident with Tay, though they seem to have created their own incident which dooms the product if it wasn't already
Interesting how this continues to be an issue for GPT3 based projects.
A similar thing is happening in AI Dungeon, where certain words and phrases are banned to the point of suspending a users account if used a certain amount of times, yet they will happily output them when it is generated by GPT3 itself, and then punish the user if they fail to remove the offending pieces of text before continuing.
Lol, how does that make any sense? I mean, all these word blacklists are always pretty stupid, but at least you can usually see the motivation behind them. But in this case I'm not even sure what they tried to achieve, this is absolutely pointless.
Attempting to generate text from code containing "genocide" just has Copilot refuse to run. But you can still coerce Copilot to return offensive output given certain innocuous prompts.