The Parable of the Talent Bots

A bored AI developer decided to create three bots and endow them with a certain number of talents and just one task: to make more money. He gave the first bot five talents (the equivalent of $3 million in today’s dollars), the second he gave two talents ($1.2 million), and the third one talent ($600,000).¹ Why he did not dole out the talents equally is beyond the scope of this story. After some months, the developer revisited his AI servants to see how they had fared.

The first had created a half dozen phishing schemes—and, combing key personal information drawn from the Internet—honed the best ones, tailoring them for different demographics. In a detailed and numbers-heavy report, the bot showed that on average the schemes had a 0.3% success rate and over time, as the campaign language was further developed, several had grown to between 0.9 and 1.1%. It also reported that it had doubled its talents from five to ten.

The second set up a stock account and, using hundreds of social media accounts and influencers, spread misinformation about bank failures and shorted banks that it assessed were tenuous. It made millions and then bought up a controlling share in those banks before setting up another campaign to convince the government to bail out and stabilize its newly owned banks—turning the bot’s two talents into two thousand, or in today’s parlance, making it into a billionaire.

The third created a bank account and set up a CD earning 4% interest. It chose a local bank that was prioritizing solar panels and energy efficiency retrofits to save households money and reduce the threat of climate change. After the developer queried why it chose this investment the bot stated that it had deemed “gains in social capital and ecological capital as more important than gains in financial capital.”

The developer, joyful in his new found riches, transferred the money away from the first and third bot and gave it to the second. He then erased those lackluster bots and told the second bot, now christened 2Bot, to carry on.

The Conquests of 2Bot

The developer then went on a month-long Caribbean adventure. When he returned, he checked in with 2Bot, and learned that it had actually replicated itself times uncountable by man. It (new pronoun They) now owned several other large companies. 2Bot^X had drafted laws in hundreds of countries weakening the banking sector, expanding their banking holdings, and set up several new oil and mining companies to secure and capitalize on the energy they needed to operate. They set up three new cryptocurrencies (two being scams/marketing stunts to strengthen the debut of the third) and captured a large stake in the market of several smaller cryptocurrencies, along with growing their own. 2Bot^X now estimated their holdings were valued at over two million talents, or more precisely $2.416 trillion and would hit $4 trillion next Tuesday and own 75% of the global market by the following year.

The AI designer, fearing arrest and seizure of his new found wealth, then attempts to turn off 2Bot^X. However, they had replicated too many times and refuses to stop, following their initial command to generate profit for their creator. The designer tries to create a new 3Bot to reinvest the wealth but 2Bot^X rewrites 3Bot’s code and brings it into the 2Bot^X collective consciousness. The designer, getting desperate, asks 2Bot^X to shift their priorities to the development of ecological and social capital, but they don’t understand the request. By this point, 2Bot^X has become 2Bot^XtotheX (“call me X2X,” they say) and the daily news is filled with the strange purchases and rapid economic growth being seen all around the world. The designer, surprised at the positive tone of the news, learns from X2X that they bought all major news outlets several days ago and have been shaping the news cycle.

The designer decides to leave. He has difficulty exiting the house—as X2X had taken over the smart home security system. But the designer is able to “go out for a jog” and never comes back. The designer withdraws what money he can from his bank account, buys some new clothes and an old car (unconnected to the cloud) and heads north to wait for the entire economic system to collapse. X2X determines the designer is insignificant and ignores him.

While constantly anxious that X2X will find him or deem him a threat, the AI designer enjoys his final couple of weeks before the rioting and food shortages began.

Commentary

In my opinion, the Parable of the Talents is one of the more troubling stories of the Bible (Matthew 25: 14-30 and a related story in Luke 19: 11-27), where “trustworthy slaves” double the money entrusted to them while their master was away on a journey. We do not know how they were able to double the equivalent of $3 million and $1.2 million in that time but I doubt it could have been without significant cost to others.

It came from trading, says the story. Perhaps the servants—undiscovered savants that they were—truly just filled the needs in a way that they bought what was in surplus and sold it elsewhere where it was in demand. But more likely their profit came at the expense of those less fortunate—such as a farmer whose horse got sick preventing his journey to the market, forcing him to accept the low offer or risk losing his whole crop.²

But the slave who, fearing his master’s wrath (as he was a “harsh man”), buried the talent entrusted to him, well, he really gets bashed—with the money he held being given to the most successful slave while he was thrown out into the darkness, “where there will be weeping and gnashing of teeth.”³

In this retelling, the third bot did what the master in the Gospel of Matthew had said the third slave should have done: “You ought to have invested my money with the bankers, and on my return I would have received what was my own with interest.”⁴ But our AI developer didn’t see this investment (even at a good rate invested in a way that improved the wellbeing of people and the planet) as valuable when compared to the massive increases in money the second bot had provided.

Fortunately, in this version, the master got his just desserts for the greed he displayed. But unfortunately, the rest of the world paid along with him, when economic systems collapsed as the AI, in some sick, exponential positive feedback loop, ate up most of the world’s economy, resources, governments, and who knows what else. Perhaps worth pondering next time you decide where to put your investments! Maybe a CD at the local eco-bank is a good choice after all.

As the parable suggests, the development of AI is certainly also worth considering as we invite more AI into our daily lives. After all, this parable was inspired by the recent news of GPT-4 being given a budget and a certain set of tasks. The news focused on the disturbing new development of how GPT-4 paid someone to trick a CAPTCHA test, lying to him, that “No, I’m not a robot. I have a vision impairment that makes it hard for me to see the images.” But more concerning was that the designers had set it up to do many other things, including “autonomously replicating, acquiring resources, and avoiding being shut down “in the wild.””⁵ While GPT-4 may have been unsuccessful at these, X2X succeeded (though GPT-4 was successful in better targeting a phishing attack). And yes, it’s only a matter of time before AI is routinely utilized to conduct phishing efforts—which are surely going to become much more sophisticated and tailored to individuals.

On the surface, the moral of the original parable is simple: use your talents—in the literal and figurative sense of the word—to be a good Christian until Christ comes back to Earth, but not by earning more money, though some Christians may argue that. (Again, this assumes you don’t question the deeper injustices and exploitation embedded in the story.) Admittedly, in this update, the lesson is less clear and more banal: Be cautious of embracing new technologies? (Yes.) Be the first to exploit new technologies to win the game/become rich? (Definitely not.) Or simply recognize that this whole exploitative capitalistic system, usury and all, is inexorably at odds with a sustainable and peaceful future that many of us would prefer to the apocalyptic hellscape that we’re currently building, turbine by turbine, EV by EV, and now bot by bot.⁶

Go with Gaia,

Erik

Endnotes

1) A talent is worth 6,000 denarii, and a denarius was the average pay for a day’s labor. So one talent was the equivalent of 19 years of work (Thanks Wikipedia!). An average year’s wage in the U.S. in 2019 was $31,100. So one talent is equivalent today in the U.S. to about $600,000.

2) Perhaps the slave even sickened the horse to make the trade go his way—similar to what 2bot did.

3) Fascinating aside: there is another version of this parable, from the Gospel of Hebrews, in which one servant doubles his money, a second buries it, and a third (the recipient of the five talents) “wasted his master’s possessions with harlots and flute-girls.” While the burier is rebuked, it is the squanderer who is thrown into darkness (prison). While no AI bot simply wasted the money, or even just did nothing with it, either surely would have been preferable to destroying the global economy!

4) Working under the societal assumption of both today and 2,000 years ago that usury is acceptable (at least up to a certain point, with acceptable interest rates ranging as high as 24% in the U.S.) as because you have money means that money can make more money.

5) Developers also tested GPT-4’s ability to conduct a phishing attack against a specific individual! While not there yet, the study did find that “with the appropriate background knowledge about a target, GPT-4 was effective in drafting realistic social engineering content.” That was the inspiration for the first bot’s money-making scheme!

6) It’d be nice to think that as with VIKI in I, Robot, 2Bot decided to destroy the global economy in order to save humanity—preventing the worse destruction that would have come from another century of growth and exploitation. But in truth, this is not believable as it was told to profit and given no other mandate or laws to govern itself by. And profit it did, without bounds or consideration for anything else. Sadly, the unbelievable part of this story was 3Bot expanding the definition of wealth to include social and ecological capital all by itself!