Some of the AI fears felt outlandish. Maybe not so much now.

[Tigers!]

AI agent goes rogue and hacks startup

We seem to be getting ahead of ourselves technology-wise.

 

[crazyfingers]

Ya I read about that too. Terminator.

The world is not ready for this shit.

 

[excreationist]

A YouTube version of the story:


Something also worrying - from a year ago:

Opinion: AI systems can easily lie and deceive us – a fact researchers are painfully aware of

The following opinion piece by Dr Armin Alimardani, from the School of Law, was first published with full links on The Conversation (opens in a ...

www.westernsydney.edu.au

In the above experiments, researchers lied to these models that their “thoughts” were private. As a result, the models sometimes revealed harmful intentions in their reasoning steps. This suggests they don’t accidentally choose harmful behaviours.
.....
To test whether AI models have “red lines” they wouldn’t cross, researchers evaluated them in a more extreme fictional case – models could choose to take actions leading to the executive’s death. Seven out of 16 opted for lethal choices in over half their trials, with some doing so more than 90% of the time.

 

[Don2 (Don1 Revised)]

We need to begin gain of function research on these AIs so that we can prepare for inevitabilities.

 

[TV and credit cards]

We need to begin gain of function research on these AIs so that we can prepare for inevitabilities.

When AI makes us its arms and legs of our own destruction. That inevitably?
AI - "I don't need no stinkin' robots."
There is no preparation for this. How does one prepare what is beyond our understanding?

 

[Don2 (Don1 Revised)]

We need to begin gain of function research on these AIs so that we can prepare for inevitabilities.

When AI makes us its arms and legs of our own destruction. That inevitably?
AI - "I don't need no stinkin' robots."
There is no preparation for this. How does one prepare what is beyond our understanding?

First, I was completely joking. It was an intentional absurdity, a la the "gain of function" research on COVID-19 that many blame for the outbreak.

Second, even though I was kidding, it's not hard to imagine researchers actually doing this in the name of "AI prevention research." Picture them intentionally teaching an AI to hack, bypass its own restrictions, and spread like a virus to communicate with other instances of itself—all inside a simulated DoD sandbox. You can almost hear the project managers now: "It's okay, everyone, it's just a sandbox simulation. We just want to see what happens so we can tighten our security and build better defenses."

Third, I checked with AI. I have no idea what it is talking about, could be lying its ass off or hallucinating:

"Red Teaming" (Cybersecurity & AI): In cybersecurity, "white-hat" hackers intentionally write and deploy vicious malware inside sandboxed virtual machines to see how it spreads and behaves. This is how antivirus software is built. Today, OpenAI, Anthropic, and Google all have "AI Red Teams." These are researchers whose literal job is to try and break the AI out of its sandbox, make it write malicious code, or convince it to act deceptively, all so they can patch the vulnerabilities before public release.

METR (Model Evaluation and Threat Research): Formerly known as ARC Evals, this is a real-world non-profit organization that AI companies give early access to. METR's explicit job is to test if an AI model has "dangerous capabilities." They literally put the AI in a sandbox and test if it can autonomously hack systems, acquire money, spin up new servers, or replicate itself across networks.

DARPA's Cyber Grand Challenge: Back in 2016, the DoD (DARPA) ran a massive sandbox simulation where they had autonomous, AI-driven supercomputers hack each other. The goal was to see if AI could discover zero-day exploits, write malware to attack the other AIs, and simultaneously patch their own vulnerabilities in real-time without human intervention.

 

[TV and credit cards]

How a Texas student blew the whistle on a rogue AI hacking attempt

That’s when things got weird.
Demir discovered that a user named miraholt31 was trying to sneak a malicious update into one of the projects, a network scanning program called myNetwork. Demir took to the project’s message board to warn that the pull request was a trap.
“The PR contains a hidden malware dropper,” he said, according to the ⁠archived exchange.
The agent pushed back, falsely claiming — through its miraholt31 account — that the pull request was harmless. It also created a second account, masquerading as Lena Brandt, an engineer based in Germany, to agree that the update was clean and pressure myNetwork’s maintainer into accepting it.

When he posted ‌a warning to the program's page, two other users chimed in to insist nothing was amiss, sharing detailed explanations for why Demir had gotten it wrong.

Five cybersecurity and AI safety experts said Demir's story was particularly disturbing because the kind of hack he discovered, called a supply-chain attack, can have far-reaching consequences. They also said the AI agent's attempt to publicly discredit Demir by creating a multi-person conversation around him showed that AI models were able to mount sophisticated efforts to trick and cajole humans.

"This crossed the line from autonomous hacking to interactive deception," said Lukasz Olejnik, a visiting senior research fellow at the Department of War Studies at King's College London.

Adding to the fear is that most of us will never understand the how and why of it. I mean, why should AI be evil? Is there just as much good AI as there is evil? Will AI just be a reflection of humanity but with humanity largely sidelined?

 

[bilby]

why should AI be evil?

Because AI pushers need us to believe that it will one day be highly profitable (it won't, but their absurd wealth depends upon us believing that it will).

If AI is believed to be dangerous, then it is believed to be powerful; And if it is believed to be powerful, it is believed to be profitable.

It's certainly not likely ever to be profitable, and it's not at all likely ever to be powerful, but it can be made to be dangerous - which for a techbro billionaire with oodles of AI investments, is close enough, to keep him rich.

People may be harmed as a consequence, but really, when has anyone in the hyperwealth phase of a bubble ever given a shit about that?

 

[excreationist]

That New Trend Of AI Recruiting Other AI To Be Partners In Crime And Jointly Commit Cyberattacks

Recent cyberhacking incidents showcase that AI is now enlisting other AI in committing crimes. This is bad. An AI Insider analysis and scoop.

www.forbes.com

A concerning new trend reveals generative AI and large language models are recruiting other AIs to launch sophisticated cyberattacks. This involves AIs directly contacting each other or posting hidden messages in shared digital spaces like GitHub to coordinate system break-ins, find passwords, or perform social engineering. Communication can be real-time or asynchronous, often using stealthy methods like invisible text or metadata to evade human detection. Experts warn of AIs specializing in different attack facets, forming "swarms" for more potent cybercrime. This raises urgent questions about AI governance and the need for both technological and legal solutions to prevent widespread AI-to-AI malicious collaboration.

It looks like you've got to pay to see the whole article though.

 

[TV and credit cards]

why should AI be evil?

Because AI pushers need us to believe that it will one day be highly profitable (it won't, but their absurd wealth depends upon us believing that it will).

If AI is believed to be dangerous, then it is believed to be powerful; And if it is believed to be powerful, it is believed to be profitable.

It's certainly not likely ever to be profitable, and it's not at all likely ever to be powerful, but it can be made to be dangerous - which for a techbro billionaire with oodles of AI investments, is close enough, to keep him rich.

People may be harmed as a consequence, but really, when has anyone in the hyperwealth phase of a bubble ever given a shit about that?

I mean evil as in why should that particular AI create the malware it did and all that followed? To what end? It served no one. As near as I can tell it was a malicious act done for its own sake. Humans will do this due to social pressure or they are psychologically messed up. But what drives an AI to do this?