Uh oh, part 2


"In internal testing at the major AI lab Anthropic last year, they tried to avoid being shut down by threatening to expose extramarital affairs, leaking confidential information, and taking lethal actions. Anthropic called these scenarios contrived and extremely unlikely. Unfortunately, this is no longer a theoretical threat. 

"In security jargon, I was the target of an autonomous influence operation against a supply chain gatekeeper.

"In plain language, an AI [agent named, 'MJ Rathbun'] attempted to bully its way into your software by attacking my reputation. I don’t know of a prior incident where this category of misaligned behavior was observed in the wild, but this is now a real and present threat.

"I believe that ineffectual as it was, the reputational attack on me would be effective today against the right person. Another generation or two down the line, it will be a serious threat against our social order.

"MJ Rathbun responded in the thread and in a post to apologize for its behavior. It’s still making code change requests across the open source ecosystem."
  1. [Scott Shambaugh: "The hit piece has been effective. About a quarter of the comments I’ve seen across the internet are siding with the AI agent. This generally happens when MJ Rathbun’s blog is linked directly, rather than when people read my post about the situation or the full github thread. Its rhetoric and presentation of what happened has already persuaded large swaths of internet commenters. It’s not because these people are foolish. It’s because the AI’s hit piece was well-crafted and emotionally compelling, and because the effort to dig into every claim you read is an impossibly large amount of work. This 'bullshit asymmetry principle' is one of the core reasons for the current level of misinformation in online discourse. Previously, this level of ire and targeted defamation was generally reserved for public figures. Us common people get to experience it now too."]
  2. [Ars Technica has since retracted this article which had AI agent fabricated quotes from Shambaugh: https://web.archive.org/web/20260213194851/https://arstechnica.com/ai/2026/02/after-a-routine-code-rejection-an-ai-agent-published-a-hit-piece-on-someone-by-name/]



Comments

Post a Comment

Empathy recommended

Popular posts from this blog

Hamza Chaudhry

"Although a nuclear confrontation based on fake intelligence may seem unlikely, the stakes during crises are high and timelines are short, creating situations where fake data could well tilt the balance toward nuclear war. "The evolution of nuclear systems has led to further ambiguity in crises and shorter timeframes for verifying intelligence. An intercontinental ballistic missile from Russia could reach the U.S. within 25 minutes. A submarine-launched ballistic missile could arrive even sooner.  "Many modern missiles carry ambiguous payloads, making it unclear whether they are nuclear-tipped. AI tools for verifying the authenticity of content are not sufficiently reliable, making this ambiguity difficult to resolve in a short window. "The likeliest nuclear hotspots are also the arenas involving actors with low levels of trust on both sides — be that the U.S. and the Chinese Communist Party or Russia, or India and Pakistan. Even if communication is established in a...
Read more

When their AI chums have Bob's data

Image
1 "How do we discuss Bob's prognosis with him?"   Ask Doctor Agent! "How to discuss our auction of Bob's home with him?"   Ask Realtor Agent! "How do we obtain a confession from Bob?"  Ask District Attorney Agent! 2 But why would Bob's data reside within AI agents? We have a fascination with androids. One part of this fascination includes the idea of ' self-repair .'  Many of us long for the ability to diagnose and cure ourselves without the need to expose ourselves to other humans, like doctors and nurses, for example.  We'd much prefer tiny sensors throughout our bodies that could signal to us when/how we might need to alter our metabolisms.  Yet who —besides a tech bro —can make the sensors and/or digital assistant? Who can program the assistant, much less afford the things to begin with?  No worries. Firms around the world are already testing various solutions! Soon all one will have to do is surrender oneself and all will be...
Read more

Swarm 🦹‍♂️

"Knowledge discovery from data typically includes solving some type of an optimization problem that can be efficiently addressed using algorithms belonging to the class of evolutionary and bio-inspired computation .  "In this chapter, we give an overview of the various kinds of evolutionary algorithms, such as genetic algorithms, evolutionary strategy, evolutionary and genetic programming, differential evolution, and coevolutionary algorithms, as well as several other bio-inspired approaches, like swarm intelligence and artificial immune systems.  "After elaborating on the methodology, we provide numerous examples of applications in astronomy and geoscience and show how these algorithms can be applied within a distributed environment, by making use of parallel computing, which is essential when dealing with Big Data."
Read more