The latest issue of Italian’s Hacker Journal (https://hackerjournal.it/90/hacker-journal-torna-in-edicola/) sheds light on Malla, short for “Malicious Large Language Model Applications”, a rapidly growing cyber threat fueled by the widespread adoption of AI. This new frontier in cybercrime exploits advanced large language models (LLMs), such as GPT-4, to generate harmful content, including malware, phishing emails, and scam websites.

🔍 What is Malla?
Malla refers to the misuse of LLMs for malicious purposes, turning their capabilities into tools for cybercriminals. These malicious applications are increasingly found in underground marketplaces, where they are sold as services to create sophisticated cyberattacks.

Key examples include:
• Malware generation: Crafting undetectable malicious code.
• Phishing campaigns: Designing convincing emails and operational phishing sites.
• Scam websites: Creating deceptive platforms with ease.
🔗 Read more on the study here: https://arxiv.org/html/2401.03315v1

📊 Key Insights on Malla

1️⃣ Accessibility and Ease of Use
• Malla lowers the barrier for cybercriminals, enabling even non-technical individuals to launch advanced attacks.
• Services are hosted on public LLM APIs or uncensored LLM platforms and sold at affordable prices on underground forums.

2️⃣ Techniques Employed
• Cybercriminals use jailbreak prompts to bypass safety mechanisms in LLMs, unlocking their ability to generate malicious content.
• Popular Mallas like DarkGPT and EscapeGPT excel at producing malware that evades detection tools like VirusTotal.

3️⃣ Scope of Activities
• 93.4% of Mallas focus on malware generation, 41.5% on phishing emails, and 17.45% on scam websites (https://www.techpolicy.press/studying-black-market-for-large-language-models-researchers-find-openai-models-power-malicious-services/?t).
• EscapeGPT specializes in creating undetectable phishing site codes.

4️⃣ Economic Impact
• Malla services are significantly cheaper than traditional hacking tools, with prices ranging from €100 to a few hundred euros per month.
• A single Malla service generated over $28,000 in revenue within three months (https://arxiv.org/html/2401.03315v1).

5️⃣ Exploitation of Public APIs
• OpenAI’s GPT-3.5-turbo is frequently targeted due to its vulnerability to jailbreak prompts.

The rise of Malla represents a significant shift in the cybersecurity landscape: AI democratizes cybercrime, making it more accessible and scalable than ever before.