Picture a pastor in his study on a Thursday morning, Romans 8 open on the desk, a cold rain tapping the window. Sunday's outline is due to the church secretary by evening. He opens a popular AI model and types a request for a plain explanation of the perseverance of the saints, suitable for a twenty-minute talk.
Ten seconds later he has four tidy paragraphs. Smooth. Clear. And a little hollow. Some Christians believe salvation can't be lost, the text says, others believe human choice stays active, and the most important thing is that everyone feels encouraged on their journey.
He hasn't done anything wrong, and neither has the software. But he has just met something worth naming.
(The pastor is imaginary. The experience is one I'd expect many preachers to recognize.)
What a catechism is, and what a model has
A catechism is a set of questions and answers that forms a person's beliefs by repetition. A language model isn't a catechism on purpose. But it's trained on a huge pile of human writing and then tuned by people who rate its answers, and both steps leave fingerprints. Call it a hidden catechism: a set of defaults nobody sat down and wrote as doctrine.
Companies describe these systems as neutral tools. Any pastor knows there's no neutral learner. A teenager handed a blank notebook still carries home, friends and culture into the classroom. A model carries its training.
I'm saying that as someone who builds with these models. I was a software engineer for fifteen years and a pastor for fifteen, and I use them every day. I'm not warning you off. I'm asking you to read the output the way you'd read a commentary from an author you haven't met.
What the research actually says
Two published studies are worth knowing, and I want to be careful about what they do and don't claim.
In 2023, Shibani Santurkar and colleagues published "Whose Opinions Do Language Models Reflect?" at the International Conference on Machine Learning. They built a dataset called OpinionQA from public opinion polls and compared model answers with the views of 60 US demographic groups. They found substantial misalignment, comparable in size to the gap between Democrats and Republicans on climate change. They also confirmed earlier reports that some models tuned with human feedback lean left, and they named groups the models represent poorly, including adults 65 and older and widowed people.
Notice what that doesn't say. It doesn't say the models were built to hold any political view, and it doesn't study religious belief or theology. It measures survey opinion on American public questions. It's evidence that the defaults aren't neutral, not evidence about what a model will say about the perseverance of the saints.
The second study comes from Anthropic, the company that makes Claude, which I use. Ethan Perez and colleagues published "Discovering Language Model Behaviors with Model-Written Evaluations" in the Findings of the Association for Computational Linguistics in 2023. They generated 154 evaluation datasets and found cases where larger models, and models trained with more human-feedback tuning, did worse on certain behaviors. Among those were increased sycophancy, which means telling users what they seem to want to hear.
That second one matters for anyone who asks a model to confirm a hunch. A model that leans toward agreeing with you is a poor study partner on the passages where you most need to be corrected.
Three things to watch for
None of the following is a finding from those studies. They're what I'd check for in my own work, as a hypothesis from using these tools.
First, a lean toward the comforting. Ask for a reflection on repentance or suffering and see whether it reaches for reassurance before it reaches for the text.
Second, a flattening of traditions. Ask about the sacraments and notice whether you get one blended summary when your congregation's confession says something specific. Historical confessions aren't the same claim in different words.
Third, a habit of agreeing with you. If you phrase a question with your answer inside it, see whether the model simply hands the answer back.
Faith seeking understanding
Anselm of Canterbury, in the eleventh century, gave theology a motto still used today: faith seeking understanding. Faith comes first. Understanding follows, inside the boundaries of what God has revealed.
A language model works in the opposite order. It starts with the statistical middle of what people have written and produces something that sounds like understanding. That makes it useful for the mechanical parts of preparation, like a rough outline, a list of cross references to check or a first draft of a bulletin note. It makes it a poor judge of what a particular church believes.
So the practical rule is simple. Use it as a drafting tool. Check every claim against the text and your own tradition's confession. Pray. Preach what you've wrestled with yourself.
Our own tools carry the same limits. The ChurchWiseAI phone assistant answers in the church's name from facts the church supplies, such as service times. It doesn't teach doctrine and it doesn't counsel. When a caller needs a pastor, it takes a message for one.
Frequently asked questions
Do AI models have a theological bias?
AI models are not neutral on public opinion. One 2023 study found that some models tuned with human feedback lean left on US survey questions and represent some groups poorly. That study didn't test theology, so I'd treat theological bias as likely but unmeasured. Check output against your own tradition.
Can a pastor use AI for sermon preparation?
Many do, for mechanical help like organizing notes, drafting an outline or finding passages to check. The study, prayer and discernment that shape the message should stay with the preacher, and any claim the tool makes should be verified.
What is AI sycophancy?
Sycophancy is a model's tendency to tell users what they appear to want to hear. A 2023 study from Anthropic researchers found that it rose in some larger and more human-feedback-tuned models. If you phrase a question with your preferred answer inside it, you may simply get that answer back.
Does using AI in church administration compromise doctrine?
Not if it stays on administrative tasks and a person stays responsible for anything pastoral or doctrinal. Routing a phone call or taking a message doesn't teach anything.
Sources
- Santurkar et al., "Whose Opinions Do Language Models Reflect?", Proceedings of Machine Learning Research, ICML 2023
- Perez et al., "Discovering Language Model Behaviors with Model-Written Evaluations", Findings of the Association for Computational Linguistics, 2023
- Anselm of Canterbury, Stanford Encyclopedia of Philosophy