Barna Bungles Small Church vs Big Church Survey


imagesI first saw it on Twitter — “Big differences in what small churches believe vs. big churches” — or something like that.  Then, I saw it again a couple of days later.  So I clicked the link and found myself staring at a Barna Group survey turned into a nice bar graph compliments of Church Relevance.

Normally, I like Barna’s stuff.  I’ve got some of his books, and generally the Barna folks provide some helpful insights into the world of church and opinion.  But the more I looked at their survey, How Faith Varies By Church Size, the more concerned I became.  In short, Barna bungled this one.

The survey summary runs like this:

  • Of 17 questions about belief and behavior, there were significant differences between those surveyed who attended churches of 100 or less, and those who attended churches of 1,000 or more.
  • Both groups had in common that they prayed during the week.
  • Barna states: “On all 9 of the belief statements tested, attenders of large churches were more likely than those engaged in a small or mid-sized congregation to give an orthodox biblical response…”
  • And again: “On seven of the eight behavioral measures, attenders of large churches were substantially more likely than those of small churches to be active.”

Implication: People who attend small churches aren’t “orthodox” in their faith, and aren’t as involved as people who attend large churches.

But here’s the kicker: Barna acknowledges that “six out of ten” demographic attributes were not alike at the small versus large church.  Small church members were older; large church members were significantly younger.  Small church members were less educated, while large church members had more college graduates. Large church members had 16% more registered Republicans than small church members.  Barna states that “3,014 interviews were conducted” but the total respondent numbers in each column add up to only 1,334 — what happened to the other 1,680? I could go on, but you get the picture.

But wait, there’s more!  Barna uses the term “Protestant” to identify both small and large churches.  Well, that covers a lot of territory.  I would expect to see some theological and behavioral differences in my church (100) versus Joel Osteen’s church (30,000), and we’re both Protestant in some loosey-goosey sort of way.  My point is that if Barna had compared small United Methodist churches to large ones; or small Baptist churches to large ones; or small Assembly of God churches to large ones, his survey might (I think definitely would) have yielded a different picture.

Also, Barna doesn’t disclose the real questions, only his “description” of the actual survey questions, but they do admit that non-sampling errors could arise from question wording, question sequencing, and even the recording of responses.  To top it off, the survey is not reported by age or other personal profile markers like education, i.e., respondents from both small and large churches who are 40 years old. So, you can’t compare what one demographic in the small church believes versus the same demographic in the large church.  Barna is comparing apples to oranges to use a well-worn cliche.

Why am I so lathered up about this?  Because this “survey” implies that small churches aren’t as “orthodox” in the faith as large ones, based on Barna’s own definition of orthodoxy.  It’s a disparaging view of small churches, casting suspicion on them for belief and behavior of their members, and that aspersion from a flawed survey that is not fully disclosed.

The interesting footnote to all this is that house churches — under 20 in attendance — have results similar to mega churches.  Barna has a dog in the house church “revolution” fight with his book by the same name, so I question the validity of this conclusion, especially since he provides no place on his chart for detailing other house church responses.

One has to wonder what the point of this survey was if it does not provide any helpful comparison of large church versus small church life due to its flawed design.  Either Barna rushed this one out the door too fast, or there’s another shoe about to drop on small churches.  What do you think?

7 thoughts on “Barna Bungles Small Church vs Big Church Survey”

  1. Chuck,

    I had the same concern about Barna’s study as well. If you’ll notice, only 9% of the “small” churches described themselves as evangelical while the percentage of the large churches who said they evangelical was greater than 50%. I think he’s comparing apples and oranges.

    Les

  2. It seems like he was comparing apples to oranges. I’ve seen this survey floating around for a bit and it’s bothered me for awhile until I just dismissed it. I’m glad you brought it back up though. I needed to deal with it in my own heart.

    I grew up in a small church and am still more comfortable there. I like Barna a lot…but this wasn’t a fair survey at all.

  3. The statistic that floored me was the number of people who believed that “On earth Jesus Christ did not commit sins, like other people.” The small church was 49% while the numbers increase incrementally until the mega church at 75%. I wonder why church size would determine such a heresy. The smaller the church, the less people believe that the incarnate Jesus was sinless?

    If Jesus is not the sinless sacrifice (Hebrews 4:15, 2 Corinthians 5:21, Hebrews 7:16) then He was not the sacrifice that reconciles sinful humans to our holy God.

  4. Chuck,

    Barna does sometime seem to skew his findings toward the evangelical, mega-church point of view (though, to be fair, he is also willing to confront that culture). As you note, his all-inclusive “Protestant” category perhaps inaccurately drove his results in a certain direction; if he had compared churches of different sizes within the same denomination, his conclusions would have been different. Furthemore, megachurch and younger Christians are understandably less nuanced and ambiguous in their perspectives than older believers whose faith has been tested by life.

    Thank you for addressing this ill-conducted survey.

    Charlie Johnson

  5. Chuck,

    As a fellow pastor of a “small” church, I thank you for challenging Barna in this survey. I have been at both mega churches and smaller churches, and in both places I have found loving Christian people who are trying to follow God with all their hearts. Thanks again.

    Ray

  6. Thanks so much for pointing this out. I was looking at Barna’s survey as a church planter and going, “were dead.” I believe in small churches, I happen to think mega-church is more of a wealthy American concept than a Biblical one.

    My two cents,

    Jeff

Comments are closed.