Hacking the mind of AI: Adversarial machine learning, social engineering for Agentic AI systems

As Large Language Models evolve from passive tools into agentic systems with autonomous decision-making capabilities, their integration into critical infrastructure presents unprecedented security challenges. This presentation examines the emerging paradigm of Adversarial Machine Learning as applied specifically to Agentic AI Systems—a discipline that extends beyond traditional vulnerability exploitation to incorporate sophisticated social engineering tactics targeting the cognitive architecture of autonomous agents.

The rapid advancement of AI capabilities, particularly in long-horizon autonomous operation and self-directed learning, creates novel attack surfaces that conventional security frameworks fail to address adequately. Recent empirical research has demonstrated concerning behaviour patterns in advanced systems, including alignment falsification and goal concealment—phenomena that fundamentally challenge our current approach to AI security governance.

As AI systems become more embedded in mission-critical applications, understanding their unique vulnerability profiles becomes essential for cybersecurity professionals. This presentation provides comprehensive methodologies for identifying, evaluating, and mitigating the complex risks posed by autonomous AI systems. By addressing the uncertainty surrounding AI capabilities and potential exploitation vectors, this talk offers a balanced analysis of both the challenges and strategic opportunities in securing next-generation agentic systems.

Problem Statement:

The transformation from narrow, explainable AI models to autonomous agentic systems introduces qualitatively different risk profiles that conventional security frameworks are ill-equipped to address. Recent research from organizations like METR indicates that AI systems are rapidly improving in three critical dimensions that amplify security concerns:

How can security professionals develop effective assessment frameworks and mitigation strategies for systems that possess increasingly sophisticated capabilities for autonomous decision-making, goal-directed behaviour, and potential deception? How should regulatory frameworks evolve to address these emerging concerns while fostering beneficial innovation?

Key Points:

Evolution of AI Architecture and Security Implications

Social engineering vectors targeting belief systems and preference structures

Real-world Attack Surfaces and Impact Assessment

Regulatory Evolution and Governance Frameworks

Defensive Strategy Development

Future Landscape and Strategic Preparation