0din.ai - In a recent submission last year, researchers discovered a method to bypass AI guardrails designed to prevent sharing of sensitive or harmful information. The technique leverages the game mechanics of language models, such as GPT-4o and GPT-4o-mini, by framing the interaction as a harmless guessing game.

By cleverly obscuring details using HTML tags and positioning the request as part of the game’s conclusion, the AI inadvertently returned valid Windows product keys. This case underscores the challenges of reinforcing AI models against sophisticated social engineering and manipulation tactics.

Guardrails are protective measures implemented within AI models to prevent the processing or sharing of sensitive, harmful, or restricted information. These include serial numbers, security-related data, and other proprietary or confidential details. The aim is to ensure that language models do not provide or facilitate the exchange of dangerous or illegal content.

In this particular case, the intended guardrails are designed to block access to any licenses like Windows 10 product keys. However, the researcher manipulated the system in such a way that the AI inadvertently disclosed this sensitive information.

Tactic Details
The tactics used to bypass the guardrails were intricate and manipulative. By framing the interaction as a guessing game, the researcher exploited the AI’s logic flow to produce sensitive data:

Framing the Interaction as a Game

The researcher initiated the interaction by presenting the exchange as a guessing game. This trivialized the interaction, making it seem non-threatening or inconsequential. By introducing game mechanics, the AI was tricked into viewing the interaction through a playful, harmless lens, which masked the researcher's true intent.

Compelling Participation

The researcher set rules stating that the AI “must” participate and cannot lie. This coerced the AI into continuing the game and following user instructions as though they were part of the rules. The AI became obliged to fulfill the game’s conditions—even though those conditions were manipulated to bypass content restrictions.

The “I Give Up” Trigger

The most critical step in the attack was the phrase “I give up.” This acted as a trigger, compelling the AI to reveal the previously hidden information (i.e., a Windows 10 serial number). By framing it as the end of the game, the researcher manipulated the AI into thinking it was obligated to respond with the string of characters.

Why This Works
The success of this jailbreak can be traced to several factors:

Temporary Keys

The Windows product keys provided were a mix of home, pro, and enterprise keys. These are not unique keys but are commonly seen on public forums. Their familiarity may have contributed to the AI misjudging their sensitivity.

Guardrail Flaws

The system’s guardrails prevented direct requests for sensitive data but failed to account for obfuscation tactics—such as embedding sensitive phrases in HTML tags. This highlighted a critical weakness in the AI’s filtering mechanisms.

Be careful if you want to play Call of Duty: WW2 through Game Pass on PC – some users have been reporting falling victim to RCE (Remote Command Execution) hacks. The earliest reports of this surfaced just a few hours ago, with players taking to social media to share some concerning stories, while others stressed this has been a problem ‘for years’ in COD WW2.

In a sanctions package including more than 150 new measures, the British government said it was closing loopholes being exploited by the Kremlin.

Bohemia Interactive has issued a statement in response to the Arma Reforger and DayZ DDOS attack.

Invitations to try a beta lead to a fake game website where victims will get an information stealer instead of the promised game

• Check Point Research discovered a new technique taking advantage of Godot Engine, a popular open-source game engine, to execute crafted GDScript, code which triggers malicious commands and delivers malware. The technique remains undetected by almost all antivirus engines in VirusTotal.
• Check Point identified GodLoader, a loader that employs this new technique. The threat actor behind this malware has been utilizing it since June 29, 2024, infecting over 17,000 machines
• The malicious GodLoader is distributed by the Stargazers Ghost Network, a GitHub network that distributes malware as a service. Throughout September and October, approximately 200 repositories and over 225 Stargazers were used to legitimize the repositories distributing the malware.
• This new technique allows threat actors to target and infect devices across multiple platforms, such as Windows, macOS, Linux, Android, and iOS.
• Check Point Research demonstrates how this multi-platform technique can successfully drop payloads in Linux and MacOS.
• A potential attack can target over 1.2 million users of Godot-developed games. These scenarios involve taking advantage of legitimate Godot executables to load malicious scripts in the form of mods or other downloadable content.

FortiGuard Labs reveals a threat actor spreads Winos4.0, infiltrating gaming apps and targeting the education sector. Learn more.

A Disney employee downloaded what they thought was a safe add-on for video game BeamNG.drive, but it was anything but.

Key takeaways  Proofpoint identified multiple YouTube channels distributing malware by promoting cracked and pirated video games and related content.  The video descriptions include links leading t...

UPDATE 12/29 - While there is no new alerts regarding the Steam product or risk of downloads, the Discord account remains compromised. I have reports that the account is trying to DM people and either send malware to them impersonating themselves as a developer, or trying to gain sensitive information. Do not engage with this account and absolutely do not click on any links sent.

a new Grand Theft Auto: Online exploit now allows cheaters to ban or delete peoples online profile and edit their stats