Feed

“Our findings establish frontier AI as a more capable conversational persuader than the most prepared, incentivized, and expert humans we could recruit. Training humans does not appear to close that gap,” they write. “As access to these systems continues to grow, the question is no longer whether AI can out-persuade humans but how, where, and on whose behalf this capability will be exercised.”

That era of building a viable SaaS business in a few months is gone. I mean, it technically still exists today but only in the arbitrage sense that the rest of the world hasn’t yet caught on to how quickly and easily software can be built. It’ll be gone soon, I promise.

In atproto, every app is a projection of the whole Atmosphere, just like Feedly and Google Reader are projections of the entire Blogosphere. You mostly “decentralize” by swapping your hosting, and/or by making and trying new apps. Running many full copies of the Bluesky database server is possible, but it’s not any more useful than running many copies of Google Reader.

Here’s a partial list of scam-relevant capabilities that LLMs have that would previously require significant skilled human effort per target:

Researching a mark to find out the best way to go after them. Personally tailoring all communication with a mark in mind, dynamically adjusting based on how they respond to various approaches. Cloning the voice of a person the mark trusts, like a relative. Plausible, real-time deepfaking of a video call. Building a plausible-looking corroborating fake web presence 6. Realtime monitoring of compromised resources, and dynamically building up the scam based on this monitoring. Better triage and discovery of marks. Avoiding signature-detection based spam filters (shown by Heiding et al). Scanning for and chaining known exploits in unpatched deployed software. Mass scanning isn’t new, but cheaply building tooling that can keep tabs on the latest CVEs and learn new tricks is7.

These are capabilities that exist today, and they’ll only improve from here. We should look at these skills as a floor, not a ceiling.

The user opens the URL on whatever device they want, not necessarily the box running the CLI. They log in, see the requested scopes and the client name, confirm the short code matches what the CLI printed, and approve.

The most frustrating failure mode right now is that people submit issues that are not in their own voice. They contain an observed problem somewhere, but it has been thrown into a clanker and the clanker reworded it and made a huge mess of it. Typically, it was prompted so badly that the conclusions produced are more often than not inaccurate but always full of confidence. The result is complete guesswork on root causes, fake-minimal repros, suggested implementation strategies, analogies to adjacent but often the wrong code, and long lists of error classes that might or might not matter.

That is worse than no diagnosis.

This change maps to a brewing theory among economists: that one consequence of automation via AI is that humans move to figuring out how to validate the outputs and price the operational risks of AI systems. That increasingly seems to me to be what we’re doing inside the company. The more we add AI automation, the more humans move to some “verification layer” that sits atop it. The verification layer sits atop of a much larger “virtual organization” which consists of increasingly large quantities of AI systems working on behalf of humans.

A compiler does not feel humiliated when I swear at it, a car does not suffer when I call it a shitbox and a power drill is not oppressed by being handled roughly. An LLM is more complicated than those things, and the interactions you can have with them can be truly uncanny, but a moral status does not appear just because the machine can emit text in the first person.

In this sense, prompts are a worse form of technical debt than code. When technical debt blows up, it usually causes errors or a tangible slowdown as you try to understand the code. Prompts will decay silently. Also, even janky code tends to be relatively stable when untouched, but every single model upgrade could turn a functional prompt into a non-functional one.

Earth is not metaphysically different from the heavens, living beings are not metaphysically different from inanimate matter, humans are not metaphysically different from other animals. The soul is not metaphysically different from the body. We are all parts of nature, like anything else in this sweet world.