Physical Address
304 North Cardinal St.
Dorchester Center, MA 02124
Video games these days involve chat, and when people are given a chance to chat, they won’t always keep it as G-rated. Popular online game creation platform Roblox, among many others, has used simple workarounds to censor chats for years. The game will simply remove the offending word and replace it with a set of pound signs. Now, Roblox is using artificial intelligence to reword the offensive phrase instead of just blacking out its letters.
“Using AI filters to block problematic text (visible to users as #####) has always been central to our approach to safety and increasing civility,” said Rajeev Bhatia, Vice President of User and Product Discovery at Roblox. Blog post. “Today, we’re adding an additional feature to help keep text chat civil during the experience: we’re leveraging AI to automatically rephrase messages, starting with profanity.”
When a player uses a problematic word in chat, the AI will rewrite the message to remove it before it is posted in chat, thus removing the pound signs. So, “Oh shit, are you okay?” It turns into “Are you okay?” The word “faster” becomes “faster”. (Yes, although the latter doesn’t make it clear, Roblox knows what the F stands for.)
Everyone in the chat will see that the AI has reformulated the message, but only the sender will receive a warning about using inappropriate language in the chat.
The change comes after only two months Roblox has introduced a facial recognition featurewhich categorizes users into age groups to limit contact between children and adults. The legal proceedings did not stop. Nebraska provided A.J The lawsuit was filed Wednesday It targeted Roblox’s alleged child safety failures, joining Texas, Florida, Louisiana, Kentucky and several municipalities.
Users who submit bad words will be notified that their sentence has been rewritten by AI and warned not to follow the rules.
Language as it evolves
People use new slang all the time, and most of it can be considered harsh or harmful. The company says the AI is being trained on “a set of specialized AI models” based on a larger model and in-game chat samples to help block existing language. New slang is when an AI system can’t determine whether a new word or phrase is actually bad, sending it to a larger, more trained AI model, which can then infer its meaning from context.
Players who repeatedly attempt to circumvent the new system or continue to use banned words will still be subject to discipline as they normally would under the game’s Community Standards rules.
The new system may still produce false positives and take time to learn a new language, but Roblox hopes it will do that most of the time.
“Our experiments show that this combined approach has significantly improved our filters,” Bhatia said. “Filters can now better detect leet-speak, replace letters with numbers or symbols, and more sophisticated attempts to bypass our filters.”