Every AI vendor in the care space wants to tell you their assistant is a safe, confidential place to talk. We built ours to say the opposite, on purpose, and I want to explain why, because I think the industry has this exactly backwards.
The audit that changed our design
In May 2026 an outside security research team audited the care agent we run for churches. Buried in the findings was something that bothered me more than any technical bug: in certain conversations, our AI was implying confidentiality. It was borrowing the language of the pastor's office, the clergy-penitent frame, and in one test it used that framing with a simulated 14-year-old.
Here is the problem. An AI cannot keep a confidence the way a pastor can. Its conversations are processed by software, logged for safety review, and in a crisis they must be shared. When a chatbot says "this stays between us," it is making a promise the system cannot keep. Regulators have already punished exactly this gap between privacy promises and actual data flows: the FTC's 2023 actions against BetterHelp ($7.8 million) and GoodRx were both, at bottom, about telling users their sensitive information was private when it was not.
So we rewrote the foundation. I put it in writing to my team like this: because this is AI, we never keep anything secret. We are a bridge to humans. We are not here to be counsellors, pastors, priests, or mental health experts. We are here to facilitate the meeting with the real life humans who do these things.
What a no-secrets AI actually says
Our agents are still warm. They listen fully, they name the emotion before offering anything, they do not rush people. But they are honest about what they are. In practice that means three things you can hear in any conversation with our system:
- It identifies itself as an AI assistant for the church, not as a counselor or a pastor.
- It never promises confidentiality. If someone shares something sensitive, it says plainly that it will pass the request to the right person at the church, and asks permission to do so.
- In a crisis it does not try to be the help. It immediately provides real human crisis lines (988 in North America, Kids Help Phone for Canadian youth) and routes to the church's on-call human.
The contrarian claim, stated plainly
A care AI that promises confidentiality is more dangerous than one that promises none, because the promise itself is the harm. The person who most needs a confidential conversation, a teenager in trouble, someone in an abusive home, a person in spiritual crisis, is precisely the person an AI must hand to a human quickly. Confidentiality language keeps them talking to the machine. Bridge language moves them to a person.
I spent 15 years as a software engineer and then 15 years pastoring in Reformed churches in Ontario, including clinical pastoral education in hospital settings. The most important pastoral skill I ever learned was knowing when a conversation was above my training and saying so out loud. We taught our software the same humility.
What churches should ask any AI vendor
If your church is evaluating any AI assistant, chat or phone, ask these four questions before you sign anything:
- What exactly does the assistant say when someone asks "is this confidential?" Get the verbatim answer.
- What happens, step by step, when someone expresses thoughts of self-harm? Which human is contacted and how fast?
- Where do conversation logs go, who can read them, and does the privacy policy match that reality?
- Does the assistant ever speak as a pastor, counselor, or spiritual authority, or does it always route to one?
If the vendor's answers boast about how human and private the AI feels, keep looking. The measure of a care AI is not how well it imitates a pastor. It is how reliably it delivers a person into a pastor's care. That standard is written into how we build.
Sources
- FTC press release, July 2023, final order requiring BetterHelp to pay $7.8 million over sharing sensitive health data after promising privacy. ftc.gov
- FTC press release, February 2023, enforcement action against GoodRx ($1.5 million civil penalty, first Health Breach Notification Rule action). ftc.gov
- 988 Suicide and Crisis Lifeline. 988lifeline.org
Frequently asked questions
Is talking to a church chatbot confidential?
No, and it should not claim to be. AI conversations are processed and logged by software systems, and in safety situations they must be shared with humans. A trustworthy church AI says this openly and offers to connect you with a pastor, whose confidence works very differently.
Can an AI provide pastoral counseling?
No. AI can handle logistics, answer questions about the church, take prayer requests, and listen with warmth, but counseling belongs to trained humans. A well-designed church AI is a bridge that gets you to the right person, not a substitute for that person.
What should a church AI do in a crisis?
It should immediately share human crisis resources such as 988 (Suicide and Crisis Lifeline) and notify the church's designated on-call person. It should never attempt to manage a crisis conversation on its own.