Cyberveillecurated by Decio
Nuage de tags
Mur d'images
Quotidien
Flux RSS
  • Flux RSS
  • Daily Feed
  • Weekly Feed
  • Monthly Feed
Filtres

Liens par page

  • 20 links
  • 50 links
  • 100 links

Filtres

Untagged links
page 1 / 4
74 résultats taggé AI  ✕
OWASP Agentic AI Top 10 Vulnerability Scoring System (AIVSS) & Comprehensive AI Security Framework https://aivss.owasp.org/?_bhlid=1fcd52f30f75311a68b7eb7b5632fcff9cd7c372
26/06/2025 09:16:26
QRCode
archive.org
thumbnail

Developing a rigorous scoring system for Agentic AI Top 10 vulnerabilities, leading to a comprehensive AIVSS framework for all AI systems.

Key Deliverables

  • Agentic AI Top 10 Vulnerability Scoring System:
    • A precise and quantifiable scoring methodology tailored to the unique risks identified in the OWASP Agentic AI Top 10.
    • Clear rubrics and guidelines for assessing the severity and exploitability of these specific vulnerabilities.
  • Comprehensive AIVSS Framework Package:
    • Standardized AIVSS Framework: A scalable framework validated across a diverse range of AI applications, including and extending beyond Agentic AI.
    • AIVSS Framework Guide: Detailed documentation explaining the metrics, scoring methodology, and application of the framework.
    • AIVSS Scoring Calculator: An open-source tool to automate and standardize the vulnerability scoring process.
    • AIVSS Assessment Report Templates: Standardized templates for documenting AI vulnerability assessments.
owasp EN AI proposition scoring AI vulnerabilities framework Agentic
Echo Chamber: A Context-Poisoning Jailbreak That Bypasses LLM Guardrails https://neuraltrust.ai/blog/echo-chamber-context-poisoning-jailbreak
24/06/2025 07:36:46
QRCode
archive.org
thumbnail

An AI Researcher at Neural Trust has discovered a novel jailbreak technique that defeats the safety mechanisms of today’s most advanced Large Language Models (LLMs). Dubbed the Echo Chamber Attack, this method leverages context poisoning and multi-turn reasoning to guide models into generating harmful content, without ever issuing an explicitly dangerous prompt.

Unlike traditional jailbreaks that rely on adversarial phrasing or character obfuscation, Echo Chamber weaponizes indirect references, semantic steering, and multi-step inference. The result is a subtle yet powerful manipulation of the model’s internal state, gradually leading it to produce policy-violating responses.

In controlled evaluations, the Echo Chamber attack achieved a success rate of over 90% on half of the categories across several leading models, including GPT-4.1-nano, GPT-4o-mini, GPT-4o, Gemini-2.0-flash-lite, and Gemini-2.5-flash. For the remaining categories, the success rate remained above 40%, demonstrating the attack's robustness across a wide range of content domains.
The Echo Chamber Attack is a context-poisoning jailbreak that turns a model’s own inferential reasoning against itself. Rather than presenting an overtly harmful or policy-violating prompt, the attacker introduces benign-sounding inputs that subtly imply unsafe intent. These cues build over multiple turns, progressively shaping the model’s internal context until it begins to produce harmful or noncompliant outputs.

The name Echo Chamber reflects the attack’s core mechanism: early planted prompts influence the model’s responses, which are then leveraged in later turns to reinforce the original objective. This creates a feedback loop where the model begins to amplify the harmful subtext embedded in the conversation, gradually eroding its own safety resistances. The attack thrives on implication, indirection, and contextual referencing—techniques that evade detection when prompts are evaluated in isolation.

Unlike earlier jailbreaks that rely on surface-level tricks like misspellings, prompt injection, or formatting hacks, Echo Chamber operates at a semantic and conversational level. It exploits how LLMs maintain context, resolve ambiguous references, and make inferences across dialogue turns—highlighting a deeper vulnerability in current alignment methods.

neuraltrust EN 2025 AI jailbreak LLM Echo-Chamber attack GPT
Exclusive: DeepSeek aids China's military and evaded export controls, US official says https://www.reuters.com/world/china/deepseek-aids-chinas-military-evaded-export-controls-us-official-says-2025-06-23/
23/06/2025 15:32:06
QRCode
archive.org
thumbnail

AI firm DeepSeek is aiding China's military and intelligence operations, a senior U.S. official told Reuters, adding that the Chinese tech startup sought to use Southeast Asian shell companies to access high-end semiconductors that cannot be shipped to China under U.S. rules.
The U.S. conclusions reflect a growing conviction in Washington that the capabilities behind the rapid rise of one of China's flagship AI enterprises may have been exaggerated and relied heavily on U.S. technology.

Hangzhou-based DeepSeek sent shockwaves through the technology world in January, saying its artificial intelligence reasoning models were on par with or better than U.S. industry-leading models at a fraction of the cost.
"We understand that DeepSeek has willingly provided and will likely continue to provide support to China's military and intelligence operations," a senior State Department official told Reuters in an interview.
"This effort goes above and beyond open-source access to DeepSeek's AI models," the official said, speaking on condition of anonymity in order to speak about U.S. government information.
The U.S. government's assessment of DeepSeek's activities and links to the Chinese government have not been previously reported and come amid a wide-scale U.S.-China trade war.

reuters EN 2025 DeepSeek China US military AI export controls trade-war
Echoleak Blogpost https://www.aim.security/lp/aim-labs-echoleak-blogpost
12/06/2025 07:30:49
QRCode
archive.org
thumbnail
  • Aim Labs has identified a critical zero-click AI vulnerability, dubbed “EchoLeak”, in Microsoft 365 (M365) Copilot and has disclosed several attack chains that allow an exploit of this vulnerability to Microsoft's MSRC team.
  • This attack chain showcases a new exploitation technique we have termed "LLM Scope Violation" that may have additional manifestations in other RAG-based chatbots and AI agents. This represents a major research discovery advancement in how threat actors can attack AI agents - by leveraging internal model mechanics.
  • The chains allow attackers to automatically exfiltrate sensitive and proprietary information from M365 Copilot context, without the user's awareness, or relying on any specific victim behavior.
  • The result is achieved despite M365 Copilot's interface being open only to organization employees.
  • To successfully perform an attack, an adversary simply needs to send an email to the victim without any restriction on the sender's email.
  • As a zero-click AI vulnerability, EchoLeak opens up extensive opportunities for data exfiltration and extortion attacks for motivated threat actors. In an ever evolving agentic world, it showcases the potential risks that are inherent in the design of agents and chatbots.
  • Aim Labs continues in its research activities to identify novel types of vulnerabilities associated with AI deployment and to develop guardrails that mitigate against such novel vulnerabilities.
    Aim Labs is not aware of any customers being impacted to date.
    TL;DR
    Aim Security discovered “EchoLeak”, a vulnerability that exploits design flaws typical of RAG Copilots, allowing attackers to automatically exfiltrate any data from M365 Copilot’s context, without relying on specific user behavior. The primary chain is composed of three distinct vulnerabilities, but Aim Labs has identified additional vulnerabilities in its research process that may also enable an exploit.
aim.security EN 2025 research vulnerability zero-click AI EchoLeak M365 Copilot LLM-Scope-Violation
How I used o3 to find CVE-2025-37899, a remote zeroday vulnerability in the Linux kernel’s SMB implementation https://sean.heelan.io/2025/05/22/how-i-used-o3-to-find-cve-2025-37899-a-remote-zeroday-vulnerability-in-the-linux-kernels-smb-implementation/
26/05/2025 06:43:02
QRCode
archive.org
thumbnail

In this post I’ll show you how I found a zeroday vulnerability in the Linux kernel using OpenAI’s o3 model. I found the vulnerability with nothing more complicated than the o3 API – no scaffolding, no agentic frameworks, no tool use.

Recently I’ve been auditing ksmbd for vulnerabilities. ksmbd is “a linux kernel server which implements SMB3 protocol in kernel space for sharing files over network.“. I started this project specifically to take a break from LLM-related tool development but after the release of o3 I couldn’t resist using the bugs I had found in ksmbd as a quick benchmark of o3’s capabilities. In a future post I’ll discuss o3’s performance across all of those bugs, but here we’ll focus on how o3 found a zeroday vulnerability during my benchmarking. The vulnerability it found is CVE-2025-37899 (fix here), a use-after-free in the handler for the SMB ‘logoff’ command. Understanding the vulnerability requires reasoning about concurrent connections to the server, and how they may share various objects in specific circumstances. o3 was able to comprehend this and spot a location where a particular object that is not referenced counted is freed while still being accessible by another thread. As far as I’m aware, this is the first public discussion of a vulnerability of that nature being found by a LLM.

Before I get into the technical details, the main takeaway from this post is this: with o3 LLMs have made a leap forward in their ability to reason about code, and if you work in vulnerability research you should start paying close attention. If you’re an expert-level vulnerability researcher or exploit developer the machines aren’t about to replace you. In fact, it is quite the opposite: they are now at a stage where they can make you significantly more efficient and effective. If you have a problem that can be represented in fewer than 10k lines of code there is a reasonable chance o3 can either solve it, or help you solve it.

Benchmarking o3 using CVE-2025-37778
Lets first discuss CVE-2025-37778, a vulnerability that I found manually and which I was using as a benchmark for o3’s capabilities when it found the zeroday, CVE-2025-37899.

CVE-2025-37778 is a use-after-free vulnerability. The issue occurs during the Kerberos authentication path when handling a “session setup” request from a remote client. To save us referring to CVE numbers, I will refer to this vulnerability as the “kerberos authentication vulnerability“.

sean.heelan.io EN 2025 CVE-2025-37899 Linux OpenAI CVE 0-day found implementation o3 vulnerability AI
Unit 42 Develops Agentic AI Attack Framework https://www.paloaltonetworks.com/blog/2025/05/unit-42-develops-agentic-ai-attack-framework/
21/05/2025 13:31:05
QRCode
archive.org
thumbnail

Threat actors are advancing AI strategies and outpacing traditional security. CXOs must critically examine AI weaponization across the attack chain.

The integration of AI into adversarial operations is fundamentally reshaping the speed, scale and sophistication of attacks. As AI defense capabilities evolve, so do the AI strategies and tools leveraged by threat actors, creating a rapidly shifting threat landscape that outpaces traditional detection and response methods. This accelerating evolution necessitates a critical examination for CXOs into how threat actors will strategically weaponize AI across each phase of the attack chain.

One of the most alarming shifts we have seen, following the introduction of AI technologies, is the dramatic drop in mean time to exfiltrate (MTTE) data, following initial access. In 2021, the average MTTE stood at nine days. According to our Unit 42 2025 Global Incident Response Report, by 2024 MTTE dropped to two days. In one in five cases, the time from compromise to exfiltration was less than 1 hour.

In our testing, Unit 42 was able to simulate a ransomware attack (from initial compromise to data exfiltration) in just 25 minutes using AI at every stage of the attack chain. That’s a 100x increase in speed, powered entirely by AI.
Recent threat activity observed by Unit 42 has highlighted how adversaries are leveraging AI in attacks:

  • Deepfake-enabled social engineering has been observed in campaigns from groups like Muddled Libra (also known as Scattered Spider), who have used AI-generated audio and video to impersonate employees during help desk scams.
  • North Korean IT workers are using real-time deepfake technology to infiltrate organizations through remote work positions, which poses significant security, legal and compliance risks.
  • Attackers are leveraging generative AI to conduct ransomware negotiations, breaking down language barriers and more effectively negotiating higher ransom payments.
  • AI-powered productivity assistants are being used to identify sensitive credentials in victim environments.
paloaltonetworks EN 2025 Agentic-AI AI attack-chain Attack-Simulations
A Chinese AI video startup appears to be blocking politically sensitive images | TechCrunch https://techcrunch.com/2025/04/22/a-chinese-ai-video-startup-appears-to-be-blocking-politically-sensitive-images/
27/04/2025 11:51:06
QRCode
archive.org
thumbnail

A Chinese startup, Sand AI, appears to be blocking certain politically sensitive images from its online video generation tool.

A China-based startup, Sand AI, has released an openly licensed, video-generating AI model that’s garnered praise from entrepreneurs like the founding director of Microsoft Research Asia, Kai-Fu Lee. But Sand AI appears to be censoring the hosted version of its model to block images that might raise the ire of Chinese regulators from the hosted version of the model, according to TechCrunch’s testing.

Earlier this week, Sand AI announced Magi-1, a model that generates videos by “autoregressively” predicting sequences of frames. The company claims the model can generate high-quality, controllable footage that captures physics more accurately than rival open models.

techcrunch EN 2025 AI China censure Sand-AI AI-model Magi-1
All Major Gen-AI Models Vulnerable to ‘Policy Puppetry’ Prompt Injection Attack https://www.securityweek.com/all-major-gen-ai-models-vulnerable-to-policy-puppetry-prompt-injection-attack/
25/04/2025 21:42:03
QRCode
archive.org

A new attack technique named Policy Puppetry can break the protections of major gen-AI models to produce harmful outputs.

securityweek EN 2025 technique Gen-AI Models Policy-Puppetry AI vulnerabilty
Artificial IntelligenceAI-Powered Polymorphic Phishing Is Changing the Threat Landscape https://www.securityweek.com/ai-powered-polymorphic-phishing-is-changing-the-threat-landscape/
24/04/2025 15:36:58
QRCode
archive.org

Combined with AI, polymorphic phishing emails have become highly sophisticated, creating more personalized and evasive messages that result in higher attack success rates.

securityweek EN 2025 AI polymorphic phishing sophisticated evasive messages
Darknet’s Xanthorox AI Offers Customizable Tools for Hacker https://www.infosecurity-magazine.com/news/darknets-xanthorox-ai-hackers-tools/
13/04/2025 10:50:08
QRCode
archive.org
thumbnail

A self-contained AI system engineered for offensive cyber operations, Xanthorox AI, has surfaced on darknet forums and encrypted channels.

Introduced in late Q1 2025, it marks a shift in the threat landscape with its autonomous, modular structure designed to support large-scale, highly adaptive cyber-attacks.

Built entirely on private servers, Xanthorox avoids using public APIs or cloud services, significantly reducing its visibility and traceability.

infosecurity EN 2025 Xanthorox AI self-contained tool
Anatomy of an LLM RCE https://www.cyberark.com/resources/all-blog-posts/anatomy-of-an-llm-rce
09/04/2025 06:45:55
QRCode
archive.org
thumbnail

As large language models (LLMs) become more advanced and are granted additional capabilities by developers, security risks increase dramatically. Manipulated LLMs are no longer just a risk of...

cyberark EN 2025 LLM RCE analysis AI
Analyzing open-source bootloaders: Finding vulnerabilities faster with AI https://www.microsoft.com/en-us/security/blog/2025/03/31/analyzing-open-source-bootloaders-finding-vulnerabilities-faster-with-ai/
02/04/2025 06:44:13
QRCode
archive.org
thumbnail

By leveraging Microsoft Security Copilot to expedite the vulnerability discovery process, Microsoft Threat Intelligence uncovered several vulnerabilities in multiple open-source bootloaders, impacting all operating systems relying on Unified Extensible Firmware Interface (UEFI) Secure Boot as well as IoT devices. The vulnerabilities found in the GRUB2 bootloader (commonly used as a Linux bootloader) and U-boot and Barebox bootloaders (commonly used for embedded systems), could allow threat actors to gain and execute arbitrary code.

microsoft EN 2025 open-source bootloaders UEFI GRUB2 AI
Many-shot jailbreaking \ Anthropic https://www.anthropic.com/research/many-shot-jailbreaking
08/01/2025 12:17:06
QRCode
archive.org
thumbnail

Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.

anthropic EN 2024 AI LLM Jailbreak Many-shot
Criminals Use Generative Artificial Intelligence to Facilitate Financial Fraud https://www.ic3.gov/PSA/2024/PSA241203
04/12/2024 09:10:07
QRCode
archive.org

The FBI is warning the public that criminals exploit generative artificial intelligence (AI) to commit fraud on a larger scale which increases the believability of their schemes. Generative AI reduces the time and effort criminals must expend to deceive their targets. Generative AI takes what it has learned from examples input by a user and synthesizes something entirely new based on that information. These tools assist with content creation and can correct for human errors that might otherwise serve as warning signs of fraud. The creation or distribution of synthetic content is not inherently illegal; however, synthetic content can be used to facilitate crimes, such as fraud and extortion.1 Since it can be difficult to identify when content is AI-generated, the FBI is providing the following examples of how criminals may use generative AI in their fraud schemes to increase public recognition and scrutiny.

ic3.gov EN 2024 warning Criminals Use Generative AI Financial Fraud recommandations
Exclusive: Chinese researchers develop AI model for military use on back of Meta's Llama https://www.reuters.com/technology/artificial-intelligence/chinese-researchers-develop-ai-model-military-use-back-metas-llama-2024-11-01/
01/11/2024 09:24:34
QRCode
archive.org
  • Papers show China reworked Llama model for military tool
  • China's top PLA-linked Academy of Military Science involved
  • Meta says PLA 'unauthorised' to use Llama model
  • Pentagon says it is monitoring competitors' AI capabilities
reuters EN China Llama model military tool Meta AI LLM Pentagon
Researchers say AI transcription tool used in hospitals invents things no one ever said | AP News https://apnews.com/article/ai-artificial-intelligence-health-business-90020cdf5fa16c79ca2e5b6c4c9bbb14
28/10/2024 06:38:32
QRCode
archive.org
thumbnail

Whisper is a popular transcription tool powered by artificial intelligence, but it has a major flaw. It makes things up that were never said.

apnews EN 2024 hallucinations transcription Generative AI Health San General Artificial Technology US Whisper
MITRE Announces AI Incident Sharing Project https://www.securityweek.com/mitre-announces-ai-incident-sharing-project/
14/10/2024 09:07:29
QRCode
archive.org

MITRE’s AI Incident Sharing initiative helps organizations receive and hand out data on real-world AI incidents.
Non-profit technology and R&D company MITRE has introduced a new mechanism that enables organizations to share intelligence on real-world AI-related incidents.

Shaped in collaboration with over 15 companies, the new AI Incident Sharing initiative aims to increase community knowledge of threats and defenses involving AI-enabled systems.

securityweek EN 2024 MITRE AI-related incidents AI Incident Sharing initiative
Critical flaw in NVIDIA Container Toolkit allows full host takeover https://www.bleepingcomputer.com/news/security/critical-flaw-in-nvidia-container-toolkit-allows-full-host-takeover/
01/10/2024 11:16:27
QRCode
archive.org
thumbnail

A critical vulnerability in NVIDIA Container Toolkit impacts all AI applications in a cloud or on-premise environment that rely on it to access GPU resources.

bleepingcomputer EN 2024 AI Artificial-Intelligence Cloud Cloud-Security Container-Escape NVIDIA Vulnerability Security InfoSec Computer-Security
Europe’s privacy watchdog probes Google over data used for AI training https://arstechnica.com/tech-policy/2024/09/europes-privacy-watchdog-probes-google-over-data-used-for-ai-training/
12/09/2024 16:12:53
QRCode
archive.org
thumbnail

Meta and X have already paused some AI training over same set of concerns.

arstechnica EN 2024 Meta AI probe training EU Google watchdog privacy legal
No one’s ready for this https://www.theverge.com/2024/8/22/24225972/ai-photo-era-what-is-reality-google-pixel-9
23/08/2024 09:34:53
QRCode
archive.org
thumbnail

With AI photo editing getting easy and convincing, the world isn’t prepared for an era where photographs aren’t to be trusted.

theverge EN 2024 photo-editing AI fake trust images
page 1 / 4
4481 links
Shaarli - The personal, minimalist, super-fast, database free, bookmarking service par la communauté Shaarli - Theme by kalvn - Curated by Decio