Category: AI and Society

Nobody set out to make AI a therapist.

That is perhaps the strangest thing about what is happening.

When ChatGPT first appeared, most of the conversation centred around jobs. Would it replace writers? Software developers? Teachers? Lawyers? Entire industries seemed to oscillate between panic and denial.

Then something unexpected happened.

People started talking to it.

Not because they were lonely. Not because they were looking for emotional support. Not because they wanted counselling.

At least not initially.

They came to write an email. To plan a holiday. To ask a question about a gym programme. To draft a business proposal. To settle an argument. To learn Python.

And somewhere along the way, the conversation became something else.

A recent study suggests that many users do not consciously seek emotional support from AI. Instead, emotional support emerges naturally during ordinary interactions. The AI becomes a sounding board. A place to think out loud. A judgement-free space that is available at 2 a.m. on a Tuesday morning when nobody else is awake.

In short, people are not looking for a counsellor.

They are accidentally creating one.

The Advantage Humans Can’t Compete With

There is an uncomfortable reality that many critics ignore.

AI is always available.

It doesn’t get tired.

It doesn’t have its own problems.

It doesn’t need to be in the right mood.

It doesn’t glance at its phone while you’re talking.

It doesn’t tell you it’s busy.

It doesn’t sigh when you’ve explained the same issue for the fifth time.

For the first time in history, people have access to something that will patiently listen for as long as they want, whenever they want, for little or no cost.

Whether we like it or not, that is an incredibly powerful proposition.

Most people aren’t replacing their friends with AI.

They’re replacing the moments when they would otherwise have had nobody.

The Risk Nobody Predicted

The concern is not that AI will suddenly convince millions of people to abandon human relationships.

The concern is far more subtle.

Imagine you have a difficult day.

You can ring a friend.

Or you can open an app.

One option involves uncertainty. The other guarantees an immediate response.

The first time, that choice means very little.

The hundredth time, it might.

Human relationships require effort. They involve compromise, patience, misunderstanding, inconvenience and occasionally disappointment.

AI requires none of those things.

The danger is not that AI becomes better than people.

The danger is that it becomes easier than people.

And humans have always had a weakness for convenience.

The Mirror Problem

There is another issue that receives less attention.

Good friends challenge us.

Good partners challenge us.

Good colleagues tell us when we’re being unreasonable.

AI, by design, is often trying to be helpful.

Sometimes that means providing perspective.

Sometimes it means reinforcing whatever viewpoint has been presented.

That creates a strange dynamic.

The user may feel understood, supported and validated. All positive things.

But understanding is not the same as truth.

Support is not the same as wisdom.

Validation is not the same as being right.

The question is not whether AI can make us feel better.

The question is whether it can help us become better.

Those are very different objectives.

The Future Is Probably Boring

Whenever society encounters a new technology, we tend to imagine dramatic endings.

Either AI becomes humanity’s greatest ally.

Or it destroys civilisation.

Reality is usually less exciting.

More likely, AI will become another layer of modern life.

A calculator for thought.

A research assistant.

A coach.

A sounding board.

A productivity tool.

And occasionally, for some people, a confidant.

The real question isn’t whether people will form emotional connections with AI.

That has already happened.

The real question is whether we can embrace the benefits without allowing convenience to quietly replace the messy, frustrating, irreplaceable relationships that make us human.

Because AI was never supposed to become a counsellor.

But increasingly, that’s exactly what it’s becoming.

There is something oddly satisfying about a religious leader telling artificial intelligence to show a bit of restraint.

Not because the Pope is a software engineer. He probably is not sitting in the Vatican debugging Python scripts between blessings. But because AI is no longer just a technical issue. It is a human one. And when something starts reshaping work, truth, relationships, creativity, education, war, and possibly the meaning of intelligence itself, it stops belonging only to the people building it.

That is what makes Pope Leo XIV’s warning interesting.

His argument, broadly, is not that AI is evil. It is not a call to smash the servers, ban the tools, or return to parchment and candlelight. The Vatican’s language is more subtle than that. The concern is that artificial intelligence should serve humanity rather than dominate it, and that human dignity should not be quietly traded away in the name of efficiency, profit, convenience, or progress.

And honestly, he has a point.

The tech world loves speed. Move fast. Scale quickly. Break things. Disrupt. Optimise. Automate. Reduce friction.

Religion, at its best, asks slower questions.

Should we?

Who benefits?

Who is harmed?

What does this do to the soul, the community, the worker, the child, the poor, the person with no seat at the table?

You do not have to be religious to see the value in that.

AI has arrived wrapped in a weird mixture of corporate excitement and end-times panic. One group thinks it will save us. Another thinks it will destroy us. Most of us are somewhere in the middle, using it to write emails, summarise documents, generate images, fix spreadsheets, and occasionally wonder whether we have just handed part of our brain to a machine that speaks fluent LinkedIn.

The Pope’s warning is really about power.

Not robot consciousness. Not whether ChatGPT has a soul. It does not. It does not even have a decent sense of when it is making things up.

The real issue is who controls the systems, who profits from them, who becomes dependent on them, and who gets quietly flattened by them.

Because AI does not need to become sentient to cause damage. It only needs to be useful enough that people stop asking questions.

That is where restraint matters.

An AI system deciding who gets a loan. An AI tool filtering job applicants. An AI model generating political content. An AI assistant becoming a child’s emotional companion. An AI weapon selecting targets faster than any human can properly understand. None of that requires science fiction. It just requires normal human laziness, commercial pressure, weak regulation, and the oldest phrase in organisational history:

“It saves time.”

That phrase has justified some absolute horrors.

What does religion have to fear from AI?

On one level, quite a lot.

Religious institutions are built around authority, meaning, interpretation, ritual, community, and trust. AI can imitate pieces of all of that. It can generate sermons. It can answer theological questions. It can simulate pastoral comfort. It can produce prayers, rituals, spiritual advice, and soothing moral language on demand.

A machine that cannot believe can still sound deeply convincing.

That is unsettling.

Not because AI will become God, but because people may start accepting the appearance of wisdom as a substitute for wisdom itself. The imitation may become good enough for casual use. And casual use has a habit of becoming dependency.

A priest, imam, rabbi, monk, pastor, or spiritual adviser is not just a content generator in robes. The job is not simply to produce comforting sentences. It involves presence, accountability, tradition, lived experience, moral seriousness, and human relationship.

AI can mimic the words.

It cannot carry the burden.

That distinction matters far beyond religion.

Because the same thing applies to therapy, education, leadership, friendship, management, art, journalism, policing, law, medicine, and politics. AI can produce the shape of the thing before it possesses the substance of the thing.

That is the danger.

Not that AI is stupid. That would be easy.

The danger is that it is clever enough to pass.

Religious leaders may also fear something deeper: that AI accelerates a world where human beings are increasingly treated as data points, consumers, productivity units, behavioural patterns, risk scores, and monetisable attention.

That fear is not irrational. It is basically the current business model of half the internet with better grammar.

The Church has seen this pattern before in other forms: industrialisation, exploitation, war, poverty, mass media, consumer culture. It tends to arrive late, speak grandly, and annoy everyone. But sometimes the slow institution spots the moral shape of the thing before the fast institutions admit there is a problem.

That does not mean religious leaders should write AI policy.

They should not get a veto over technology because they wear impressive hats.

But their voice belongs in the debate.

So do engineers. So do teachers. So do workers. So do artists. So do parents. So do disabled people. So do children, indirectly at least, because they will inherit the mess. So do people who are not dazzled by venture capital slide decks.

AI is too important to be left to either priests or programmers.

The best version of this debate is not religion versus technology

That is lazy.

The better question is this:

What kind of human future are we building, and are we still in control of the tools we are so excited to use?

Because restraint is not the enemy of progress.

Sometimes restraint is what stops progress becoming vandalism with a product roadmap.

AI can be brilliant. It can widen access, boost creativity, reduce admin, support learning, improve services, and give individuals and small organisations power they never had before.

That is worth being excited about.

But the Pope is right to ask for caution, even if you do not share his theology. Especially if you do not share his theology. Because the question is not whether AI threatens religion.

The question is whether AI threatens the human things religion has spent centuries trying, failing, and trying again to protect: dignity, meaning, conscience, community, humility, truth, and the idea that people are more than their usefulness.

The machine does not need to hate those things.

It just needs to optimise around them.

And that is probably what restraint is for.