[Note: this post was co-authored by J. Max Wilson and LDS Philosopher]
“If the answer to any of these questions is not 100% yes, then come with me right now for a free introductory seminar on power through positive real-estate.”
Regarding the “Mormon Gender Issues Survey” that you may have seen being passed around social media:
The survey was created by a group calling itself the “Mormon Gender Survey Group”, which includes in its membership well-known LDS dissenter and agitator, John Dehlin, as well as other progressive Mormon activists who have pushed for the church to ordain women and to change its doctrine regarding homosexuality.
The survey questions are worded in ways that subtly push their typical agenda.
This isn’t the first time Dehlin has used stilted academic surveys to support his agitation. In 2012 his “Mormon Research Foundation” created a survey that was used to over-estimate the number of unhappy women in the LDS church, even though by its own admission the survey results made “no claim of representativeness or statistical significance in the sample“. It was propaganda. But that didn’t stop it from being cited widely to support agitating for change in the church.
By all means, participate in the survey. Make your faithful voices heard. But be aware that your answers will likely be used to promote agitation in the church, regardless of your answers.
When you do participate, make good use of the free form response sections to express dissatisfaction with the wording of the questions and the oversimplification, and to push back against the agenda they represent.
The church does its own internal surveys to gauge member feelings on various topics, including inviting individual women and men to meet with them and discuss potential problems and propose changes. While the results are not public, there is no doubt that the church has pretty good information about the feelings of its members on a variety of subjects.
To demonstrate the bias of the survey, let’s a take a look a couple of specific questions. (You can jump to the bottom of this post to view screenshots of all of the questions.)
Example #1
One of the questions on the survey asks:
Which statement comes closer to your own view, even if neither is exactly right?
A. A good Latter-day Saint should obey the counsel of priesthood leaders without necessarily knowing why.
B. A good Latter-day Saint should first seek his or her own personal revelation as the motivation to obey.
The problem with this question is that the doctrinal truth combines the best of both positions. We ought to seek personal revelation to know how and when we should apply prophetic counsel. We must seek personal revelation to know if prophets are indeed God’s servants. Following the prophet must be a prayerful activity. Simultaneously, we should prayerfully follow the prophet even if we do not always understand why. God does not always give us reasons — either through the prophets or through prayer — for His instructions. He requires us to study our way to those reasons, and often times we might never even get to that point before our obedience is tested.
Thus, neither statement is exactly right, or tells the whole picture. And the question admits that. But that’s the crucial point: because neither statement is right (by itself), forcing people to pick and choose is going to be un-interpretable, and will not represent people’s true perspectives on the issue. Other questions on the survey included an “other” option, allowing participants to write in their personal perspective if it does not line up with the available answers. But this question did not — the authors of the survey wanted to make sure that people choose between these two options. Why would they do that? Because that way, they can use this question to paint the very picture they wish to paint.
The survey asks elsewhere for people to self-identify themselves on a scale of conservative to liberal. I don’t know how people will answer this question (since both statements are incomplete). But if self-reported conservatives tend to choose response A, the study authors will in all likelihood conclude that conservative members are more likely to be in the “blind sheep” category, while liberal members are more likely to affirm an individuals right to seek personal revelation. Note: this would be an untrue conclusion. The question did not permit the nuances that the question itself admits are likely there. But they’ll openly trumpet it as verified by survey anyways.
This is all just a prediction. Who knows if it will come true. But what good can come of a question that forces participants to choose between a false dichotomy, between two statements that are, at best, only partly true? Surely, this can do nothing more than entrench an already troublesome false dichotomy.
Example #2
Differences between the roles of women and men in the Church are:
A. Cultural
B. A mix between culture and doctrine
C. Doctrinal
D. Don’t know
E. Prefer not to respond
This question also entrenches a false dichotomy between culture and doctrine. Certainly, not all of our cultural practices are grounded in doctrine; certainly, not all of our practices are always best, or even good. Things can improve for the better (and, fortunately, the survey gives participants a chance to list those later on). However, there is an implicit, unstated assumption in this dichotomy: if a practice is part of our particular place and time (and not universal to all dispensations), it must therefore be cultural and for that reason uninspired. The assumption is that God does not inspire cultural practices.
This question can make those surveyed feel like they cannot answer correctly without also conveying something that is wholly untrue. If they answer that many of the differences between the roles of women and men are cultural, they would be implying that they are therefore uninspired. But the fact is, they can be both. Something can be cultural, and also inspired of God. God is often the cultivator of cultures; He frequently invites us to arrange our lives, customs, mannerisms, etc. in ways that prioritize what He prioritizes, but doesn’t always dictate precisely how. So customs and practices might differ from dispensation to dispensation, but still be inspired by divine priorities.
To conclude, it is true that we value empirical evidence. But we cannot and must not mistake every survey, or every social science study, as valid empirical evidence. The authors of studies and surveys have ideological biases that inform their question writing, data analysis, sampling, and study design. This survey was, for example, pushed onto Facebook first through particularly liberal leaning groups. The theory they are operating under is that, after 3-4 shares, they will have a representative sample instead of a convenience sample. This is highly debatable.
Given that we know that several people involved in creating and analyzing this survey are well know dissenter and agitators, we should take it and receive its future results with eyes wide open.
Here are screenshots of the Mormon Gender Issues Survey questions:
TheMillennialStar: What You Should Know About That Mormon Gender Issues Survey http://t.co/nOIEnWDoKE #lds #mormon
Ah, yes — “How to Lie With Statistics”.
Pingback: About the Mormon Gender Issues Survey - Mormon Women Stand
So has anybody contacted those institutions mentioned to see if they really did give approval, and what this study is associated with?
John hasn’t been involved much to this point (but he may be at some point, this is a long term project with different phases). We welcome feedback. In fact we just set up a Google form: http://goo.gl/forms/xy2HExFTYa. We also have an FAQ set up on our website.
To respond quickly, yes, we really are who we say we are, and yes, we really do have IRB approval from the institutions listed. We have been criticized as being too “Mormon,” too “religious,” not religious enough, of having various nefarious hidden agendas. In reality, we’re just a group of academics and other interested parties that wanted better information on this issue than what was currently publicly available and decided to go out and get it (and raised some money on Kickstarter to do it).
RT @Millennialstar: New Post: What You Should Know About That Mormon Gender Issues Survey: [Note: this post was co-au… http://t.co/Jy0V3i…
I’m pretty sure this survey was developed by John Dehlin and greatly informed by the viewpoints of those who advocated for the ordination of women.
I reposted something on Facebook about the two secrets to successful relationships based on an actually scientific study. Turns out the secrets are kindness and generosity. The poison that destroys relationships is contempt. And these keys are true also for the relationship between individuals and their faith.
This survey is composed from a schema that fundamentally despises the idea that God could be inspiring the imperfect women and men in leadership positions in the Church of Jesus Christ of Latter-day Saints. If you don’t understand why I might say that, I’ll invite you to say shibboleth five times really quickly.
A tiny part of my soul is miffed that this post got published less than the traditional several hours after the previous post, but as I said, that part of my soul is tiny. And I’m sure there are people in Asia who really needed to see this information before Monday morning.
Your screenshots are different from the survey I saw. I didn’t get asked the Jackson County question, and instead got a question asking whether I thought it was likely that there would be a revelation that would open up missionary work in mainland China. The difference seems odd.
Hi Brent,
Looked at the Kickstarter and saw the list of many people involved in this survey.
A key question would be how you intend to get a statistically representative sampling of respondents. You mention this difficulty on the the Kickstarter, but I didn’t see how you plan to address this issue.
Have individuals in your group performed similar studies of other faith groups? If so, what methodologies did you use? In what manner did you report the results, and were these results ultimately used to strengthen the faith group being studied, or did the faith group find that the survey was used “against them,” even if that was not your intent?
The questions about options other than being ordained appear to be plumbed from the discussion on the internet. I think I’ve even made some of these suggestions myself (though I think others have come up with these ideas independently).
And yet as I review the survey images, I don’t feel confident that I, as a member of the studied population, would be properly represented. As I understand ethnographic research, the studied population must be respected, there should be some benefit for the participants in exchange. And I do not see that this survey ensures that.
Perhaps you are suffering from the phenomenon of approaching a wary population after others have mistrodden the ground before you. In which case I am sorry.
Brent, or anyone else willing to listen.
A few other problems with your survey: giving an option that a change of doctrine came as a “revelation from God” vs. “A response from God due to an inquiry” is to make one option a subset of another and destroys almost all nuance. At least those you offered an “other” on.
Probably a bigger problem is your questions surrounding if members would accept a revelation ordaining women to the priesthood. Whether they would or wouldn’t would almost assuredly (on a statistical level at least) depend on the perception of the source or cause for the change. This is due to the LDS Church having as part of its “reason for existence” the idea that the Church is directly led by revelation. A revelation ordaining women to the priesthood that smacked of pressure from the outside would be devastating to the Church probably, though not at all because of women receiving the priesthood per se. A revelation out of the blue and unexpected stating the same would not.
I realize you go on to ask such questions, but of course someone taking the survey doesn’t know that is the case and has to blindly answer the question without any context at all.
This is definitely a survey that could be easily improved further. But it doesn’t look all bad to me either. I hope you’ll consider refining it to be less leading.
Meg, feel free to swap the order of this post and yours.
I went back and redid the survey and got the China question a second time, and then the third time the Jackson County question appeared instead.
The survey writers don’t appear to have written the survey in such a way as to understand Mormons. If they had, they would have written it in the language Mormons speak. We don’t talk about our faith this way. We don’t place our worldview inside the box of popular ideology. We actually believe God exists. We believe he speaks to prophets. We’re not interested in treating our beloved church as another human institution.
They’ll get the usual crowd giving the usual responses, but they may as well survey Muslims on the extent to which they resent Muhammad for forbidding bacon for all the use it will be in identifying the way a religious group actually thinks about the world.
I absolutely love research and am working on some as we speak. With public opinion research, it is always difficult to please everyone with wording. I took the survey and was actually quite pleased with the way in which many of the questions were worded. Perhaps I can look at surveys a little more objectively because that is what I do. Furthermore, the wording on the questions cannot be changed at this point. It has been my experience that IRBs do not take kindly to researchers changing the questions that were pre-approved. Additional approval would be necessary, and that takes time and effort. If you do not like the wording of the questions, simply refrain from finishing the survey.
And for the love of Pete, please do not take the survey several times. There is nothing more frustrating for a researcher than to try to obtain somewhat generalizable data only to have people take and re-take the surveys. Different questions for different people happen all the time in surveys. Let the researchers research. If we, as LDS members, have nothing to hide then let them research our opinions. Stop trying to mess up the data and inflate the numbers. Research participation is never mandatory as is clearly explained in the informed consent. I think it is great that this topic is being researched in a quantifiable manner and look forward to the results.
With all due respect, this article just makes me roll my eyes. Thought it may seem at first glance full of substantive objections, it really just boils down to guilt by association.
What do I mean?
Just take a look at the objections:
1. ” This survey was, for example, pushed onto Facebook first through particularly liberal leaning groups. The theory they are operating under is that, after 3-4 shares, they will have a representative sample instead of a convenience sample. This is highly debatable. ”
Except that if you actually looked into the research you’ll notice that the online component is only one half of the work. The other half is fully randomized national survey (which is why they actually needed money from Kickstarter). This would allow them to evaluate how representative the online study was using tried and tested statistical methods.
2. “If they answer [one of the questions] that many of the differences between the roles of women and men are cultural, they would be implying that they are therefore uninspired. BUT THE FACT IS, THEY CAN BE BOTH Something can be cultural, and also inspired of God. ” (emphasis mine)
Right… Which is why one of the answers to that question was explicitly A MIX OF BOTH. The question was shared verbatim and somehow that was missed?
3. ” The problem with this question is that the doctrinal truth combines the best of both positions. We ought to seek personal revelation to know how and when we should apply prophetic counsel. ”
Using personal revelation to determine when and how to apply prophetic counsel sounds an awful lot like seeking “personal revelation as the motivation to obey… the counsel of priesthood leaders.”
Which was again the EXACT WORDING OF THE QUESTION that was explicitly shared!
sigh…
So in reality it just comes down to “John Dehlin was involved. Beware of agitators!”
Which is funny because he’s not even really involved at the moment.
But I do agree with the article on one point: We should really keep our eyes open for bias in things we read on the internet.
😉
From a person who has taken multiple statistics classes and even survey design courses while getting my MBA, this survey stinks of bias. The wording is designed to elicit specific responses. Whether their attempt in this survey was noble and innocent or not, the survey is clearly designed to draw a specific conclusion.
Why would a survey ask some people a question about mainland China and other people about returning to Jackson County? If you’re trying to get a representative sample to answer a questionnaire, splitting the sample with different questions for different people would seem to me to defeat what the surveyors are trying to achieve. If you don’t want people to take a survey multiple times, don’t ask different sets of questions which only stir curiosity. It’s hard to resist an impulse that says, “Oooo, I didn’t get that question! Let me take it again and see if I do.” That’s simple human nature. A “scientific” study should be scientific enough to factor in human nature and keep the questions consistent across all surveys. In my unprofessional but very human natured opinion.
How many more people do we expect to come here to defend the survey and downplay Dehlin’s role in it?
You took issue with Example One, saying that a faithful Mormon would combine both a) and b) and then went on to explain how – but ended up only supporting a). In other words, you said they must (prayerfully) obey without knowing why. You suggest they pray only to know when and how, but not WHETHER or WHY to obey. question b) is asking, should you seek personal revelation for motivation to obey. If you need motivation, you’re not obeying God’s servants, you’re seeking your own answer. You’re obeying your own conviction. In that scenario God’s servants do not hold any weight as an authority to obey. b) implies you do NOT obey without getting your own personal revelation on the matter. You can’t combine the two without implying you’d reject the counsel if you didn’t get your own revelation to obey it.
The examples of questions seem to be looking for criticism of the LDS Church, rather than personal experience with gender roles in context of Church participation. My spedculation is that the questions are written so that no matter where the majority of responses come out, the people doing the survey will be able to use option A to say that Mormons are critical of the Church, while option B will let them criticize Mormons themselves as “sheep” who don’t think for themselves.
I took the survey, as I have a survey John Dehlin pushed out a few years ago. I treated the survey as though it was a good-faith attempt.
I didn’t like the fact that that one question was worded as an either/or (the question of obeying even without understanding versus seeking first one’s own revelation before deciding whether to obey). Other than that, I suppose the survey might be OK.
I liked my essay answers. And when the other boxes were there, I used them to put in the nuanced answer I might have wished was one of the provided options.
Given that the Kickstarter took place around the time Kate was in the news, it shouldn’t be surprising that the questions are what they are. I do, however, wish that the question asking the perceived likelihood of a revelation extending the priesthood to women had been asked before the question of whether I would want the offered priesthood and whether I thought others would approve women having the priesthood (if revealed).
To conflate the two news-worthy issues of this week, I think opening up to doors on polygamy, even if we might not have chosen to volunteer to have it a lead story all over the world, will allow us to access the thousands of inspiring female examples from our past who have remained buried because their stories were irrevocably entwined with polygamy. And that can’t help but enrich our collective understanding of the cast range of influence a righteous woman can have.
“I think opening up *the* doors on polygamy…[will] enrich our collective understanding of the *vast* range of influence a righteous woman can have.”
::I hate not noticing typos until after I’ve irrevocably hit the “Post Comment” button::
A. A good Latter-day Saint should obey the counsel of priesthood leaders without necessarily knowing why.
B. A good Latter-day Saint should first seek his or her own personal revelation as the motivation to obey.
^^^These questions actually came from what I think you would view as a much more reputable statistical analysis conduced by two LDS political scientists. The story and background info here: http://radiowest.kuer.org/post/mormons-and-american-politics
What You Should Know About That Mormon Gender Issues Survey http://t.co/eNnTdKMhyo #LDS #twitterstake
@LDSNana http://t.co/yQAvDF3AyK
http://t.co/OPCOwaYCb1 The first time this type of survey was fielded, they made no claim to knowing anything. Propaganda?
When we are told that John Dehlin “hasn’t been involved much up to this point, but may be in the future,” it raises alarm bells. Why in the world would reputable researchers without an agenda need to involve John Dehlin in such a project at all, much less indicate that they did openly plan to consider involving him heavily in the future? John is in no sense an unbiased party.
The list of individuals participating in this survey seem to be an odd collection, scattered widely across geography and levels of educational stature. I read a lot of proposals, and I am generally merely told about the principal investigators, not the students who will be helping out. And the e-mail addresses aren’t predominantly issued by gmail. Information that one of the researchers is a board member for Dialogue is also less than comforting, even though my husband has written a book review for Dialogue and we have attended Sunstone Symposia (even spoken at one – perhaps that is part of why I have the concern).
The survey link is also a bit weird, in that it contains lots of stuff implying that it legitimate, but it didn’t seem to contain the standard “edu” termination. The initiation of the Kickstarter was during a period of time when the leader of those advocating for female ordination in the Church were not yet under official sanction, but by virtue of the letter Kate Kelly herself published, had been placed on notice by her local authorities that her activities were considered problematic and potentially worthy of discipline.
And the tagline for the group? Eternal questions, empirical answers. Seriously? No wonder serious researchers and the independent review board (which is what I presume IRB is supposed to mean) thought the group was too invested in things Mormon.
The group appears to have come together due to lack of satisfaction with the “90% of Mormon women don’t want the priesthood” response to a question in the Pew Research study. Sure, it would be nice if the study had also clarified whether that was resistance to partaking of a fruit before it is offered under the right circumstances. But a study performed by invested academics that doesn’t even promise to inform the LDS leadership in a privileged manner (a promise John Dehlin’s survey had made) could only serve to “study the natives” on behalf of those who either members of the tribe but unwilling to accede to the authority of the leaders of the tribe, or those who actually consider themselves outside the tribe but are masquerading as members of the tribe. This group therefore resembles in some ways the group that formed in the late 20th century to address issues of ministerial abuse in the LDS faith, a group consisting of members of the faith but considering themselves outside the ecclesiastical authority recognized by the faith.
Anyway, it is disingenuous for the Mormon Gender Survey Group to assert that they have no agenda that might be of concern to members who consider themselves faithful. If we accept that our manner of presentation online is a sort of shibboleth, I assert that the Mormon Gender Survey Group has clearly indicated that they are neither firmly within the academic tradition nor firmly within the LDS faith tradition.
Tom S., all due respect, but it sounds as if you’re putting your beloved institution in the place of God. A faith that must put down other faiths to validate itself isn’t a healthy kind of faith, IMO.
You say this…..”We welcome feedback. In fact we just set up a Google form: http://goo.gl/forms/xy2HExFTYa. We also have an FAQ set up on our website.”
But this is what it says….”This feedback is NOT part of the survey. Comments and other information provided here WILL NOT be connected in any way with responses to The Mormon Gender Issues Survey.”
It does not sound like you really welcome feedback, for it wont help anyone who is unhappy with the way your questions are written and one wanting to truly express their views.
What You Should Know About That Mormon Gender Issues Survey http://t.co/CvMnQ1q8jV
Ted F. – I think you need to reread Tom’s comment again if you think he was putting down another faith.
When it asks if I’m in favor or opposed to women getting the priesthood, I feel like they are asking the wrong question. My answer is: I’m for whatever the prophet/apostles are saying. If they all of the sudden came out with a statement saying women could be ordained, then I’d be all for it. But they haven’t, so I’m not for female ordination.
A more accurate question might be: Are you opposed to the Lord/prophet revealing that women can have the priesthood? That would probably much more accurately reflect answers. That question I could answer with a clear “No, I’m not against doing what the Lord says through his prophet, even if that includes women being ordained.”
I don’t support doctrines based on the doctrine itself. I support doctrines based on the source of the doctrine. This question I mentioned above never mentioned the source, which would be my determining factor in whether or not I supported the doctrine.
Their question I mentioned should really be broken into three more accurate questions:
1. Would you support the Brethren if they said they had received revelation that women were not to get the priesthood?
2. Would you support the Brethren if they said they had received revelation that women were to get the priesthood?
3. Would you support the Brethren maintaining the current policy if no further revelation was given?
Everyone has a right to do a survey. I hadn’t seen it and I have a lot of LDS friends in social media. I don’t go to all their direct pages much but haven’t seen it in my news feed. It is good to know the source of a survey so I appreciate the education. I see from the post here that some questions would not show the whole story of a situation. I took interviewing at my University and know that one should not have loaded questions and also needs to have a clear middle in most instances where you can not agree or disagree. I know who taken out of context it can really change the meaning.
The point of research is to test a hypothesis or answer a question. On face value, the questions will be leading in a survey because they are trying to find an answer. Some of the questions are obviously designed to be forced response. Leaving little room for a range. The problem is we do not have full disclosure of what the survey is trying to measure. IRB approval on topic is fine, but where is the disclosure and concent information? This needs to be added. We need to be informed of the study piramiters to have gained truly informed concent to avoid harm. The survey could be tweaked a bit but all studies are subjected to bias.
As one of the people involved in this project, I appreciate the interest that has been expressed. To speak to the question of whether the universities involved have approved the research, yes, most definitely. Where I work, at Georgia Southern University, the Institutional Review Board approved the study as indicated on the first page of the study. So did the University of Tampa’s IRB, where another of the researchers is based. You may contact them if you’d like.
For what its worth, I attend church at my local ward house each week. My goal in the study is not to attack the church; it is to better understand how people think about these issues, and to learn what their experiences have been like. It is a topic that has been in the news a great deal – not just relating to the LDS Church, but to many Christian denominations and in other religions as well. We hope to add to the social scientific literature on women in religion, as we think that LDS women’s perspectives are important for other social scientists to know about. That is my goal.
I’ve done research for nearly 30 years and recognize that there is no perfect study. We invite people to share their concerns and to offer suggestions for questions that we could use in the future, as we hope to do at least one follow-up study at some point down the road. There is a place for you to leave your suggestions on this webpage form:
https://docs.google.com/a/georgiasouthern.edu/forms/d/19r63Y0UHfgBiDKvHVaZB-b5YXxhD9cDsbs2dr0rpQ9M/viewform?c=0&w=1
or, if that link does not work, on our FAQ page
http://mormongendersurvey.org/faq/
I am very happy that people are taking the survey seriously. It means that we will get a better understanding of these important issues.
Thank you for your help,
Michael Nielsen, Ph.D.
Professor and Chair of Psychology
Georgia Southern University
Bro. Michael Nielsen, I would really encourage you to re-think how you do these surveys. The question mentioned above leaves no good options for latter-day Saints. When you ask–
“Which statement comes closer to your own view, even if neither is exactly right?
A. A good Latter-day Saint should obey the counsel of priesthood leaders without necessarily knowing why.
B. A good Latter-day Saint should first seek his or her own personal revelation as the motivation to obey.”
–you are showing a lack of understanding on how latter-day Saints actually make decisions. You are framing the question as A)you are a sheep or B)you know better than the prophets. It is obvious that in reality most latter-day Saints consider personal revelation before making decisions but will consider priesthood counsel as well. Your either-or framing of this question makes it nonsensical.
Michael Nielsen,
“I am very happy that people are taking the survey seriously.”
What a joke. Read the OP and the responses again. Most of them are NOT taking the survey seriously. In fact the opposite. You have failed before getting out of the gate because your ivory tower methods are silly and wrong.
Kind of reminds me of how Lucifer offered the apple to Adam and Eve. He told a lot of true, laced with a little lie and thus sin was introduced into the world. Remember, rat poison is 98% oatmeal, and we all know how good oatmeal is for you. Its that other 2% that kills the rat.
Mormons, being less than 6 million active participants, need to realize that you do not control the dialogue and get to define how people perceive your religion. Just because outsiders choose to not use your culturally scripted lingo to soften various bizarre and mystical practices, does not mitigate the fact that these people are right.
For example: Garments vs. Magic Underwear. The reality is both are correct. Just as the author is explaining false dichotomies, this is exactly one of those. (For any of you to deny it is intellectually dishonest and being willful and purposefully ignorant) That is how they have been presented time and time again culturally, having metaphysical properties and protections put into them, regardless the current PR video about them.
Joseph Smith ordained his wives into the priesthood. The precedent has been set. “I haven’t heard about that!? Where is your source!?” Just because you have chosen to not read your own history or have willfully chosen to ignore it, again, does not mitigate the fact it happened. Just as your religion finally coming clean about Joseph Smith’s polygamy and polyandry on its website. “Oh, well that information was ALWAYS available.”
Exactly, Joseph Smith ordaining his wives to the Priesthood information has always been available. Read your own Joseph Smith Papers. It will happen again.
But feel free to keep kicking against the pricks. Those of us in the ‘know’ and ahead of the curve will keep watching above the LDS maze in sadness as you scurry about in that limited thought pattern.
Dr. Nielsen,
You say, “For what its worth, I attend church at my local ward house each week.” That is not worth anything to me. I can think of several people in my own stake who show up every Sunday but are nothing more than wolves in sheep’s clothing, including one couple who several have suggested ought to be disciplined for the things they teach in private study groups in their home.
I took the survey and one question I had a problem with was the one which started “Women and men are treated differently in the church.” This is false. Men and women have different roles in the Church, but this does not equate to being treated differently. Some members of the Church treat men differently than women, and this is something that needs to be corrected. But the Church treats all women and men with the same high level of respect that any child of God deserves. It was clear to me before I even saw this blog that this is just an effort to give support to apostates. I personally choose to turn my back on Satan and his workers. I suggest you do the same, no matter how much you have been paid by these apostates. Your 30 pieces of silver will not go far in the eternities.
“We hope to add to the social scientific literature on women in religion, as we think that LDS women’s perspectives are important for other social scientists to know about.”
I’m an LDS woman and it was very challenging to answer the questions in a way that reflected my perspective accurately. You have my results, but they aren’t very useful. Hopefully the longer answer section will be utilized and published as much as the rest of the survey.
D Crockman: “You say, “For what its worth, I attend church at my local ward house each week.” That is not worth anything to me.”
Obviously the church feels differently, and is happy to count as members believers and non-believers, as well as attenders and non-attenders. Why should your standard be higher than the church’s?
““Women and men are treated differently in the church.” This is false. Men and women have different roles in the Church, but this does not equate to being treated differently.”
If they have different roles, it follows that they are treated differently.
Well, I don’t have time to look at everything but in the fist example, I think it’s a bit of a trick. Anyone who chooses A is operating on a culture they don’t even understand themselves.
You can’t choose A, without first having already chosen B. That is, no one would choose A unless they’ve already had personal revelation that gives them motivation to obey a priesthood leader’s counsel. Saying we should follow the prophet, etc. is already based on revelation (testimony) where someone says I know Pres. Monson is a true prophet. How does one know? Personal revelation. So that revelation is the motivating factor in difficult cases for putting someone elses will ahead of your own.
Put it another way, for someone who is a convert, would you imagine they would “obey” President Monson before they had a testimony/revelation telling them the church is true? Nope, don’t think so.
Is it likely there are some members who really aren’t using their agency, aren’t truly converted and just go with the flow because they’ve done so their whole lives? Sure. Would it be better that they were going with the flow in the church or going with the flow in the world at large? Remember, this hypothetical is people who follow the path of least resistance… I’d rather have that person in the church, hope that person catches the vision at some point — and the very heavy demands of discipleship (callings, meetings, service, study, etc.) practically necessitate it as person at some point asking, “why am I doing all this?” In a sense, the “to do” list itself forces the person going through life on autopilot at some point to wake up and prove the Lord and become converted, or bug out.
We know which way Dehlin went.
As an active Mormon who also happens to be a PhD holding faculty member in the social sciences at a respected university, these questions look like good social science to me.
In general, respondents should remember that you can seldom tell what’s going on when you’re taking a survey. You’ve got to see the actual analysis before making a judgment. Just because a question looks biased to the survey taker does not mean it is in fact biased. Nor is it a red flag for different people to see different questions. These are standard research techniques–you randomize which questions appear to which people, or you expose respondents to wording that you know is unusual or provocative merely to see how widely it is condemned.
Take the survey and look for the results later. But nothing in here raised a red flag to me.
“Mormons, being less than 6 million active participants, need to realize that you do not control the dialogue and get to define how people perceive your religion. Just because outsiders choose to not use your culturally scripted lingo to soften various bizarre and mystical practices, does not mitigate the fact that these people are right.”
Kessee, was writing this survey supposed to be about being “right” about someone else’s religion? Or was it supposed to be about understanding? It’s simply good anthropology to attempt to understand a people on its own terms, and not to naively impose your own worldview upon theirs. If I were surveying another group, I hope I would have the humility to let its members answer in the way they like and not to shape their responses to fit a convenient narrative.
“As an active Mormon who also happens to be a PhD holding faculty member in the social sciences at a respected university, these questions look like good social science to me.”
As an active Mormon, this question didn’t raise your eyebrows?
“Which statement comes closer to your own view, even if neither is exactly right?
A. A good Latter-day Saint should obey the counsel of priesthood leaders without necessarily knowing why.
B. A good Latter-day Saint should first seek his or her own personal revelation as the motivation to obey.”
This question is terrible, on several levels. It might be ok to you as a “PhD holding faculty member in the social sciences at a respected university” but as an “active Mormon” it should have come across as extremely biased.
I saw it with a plea from another friend to take the survey and make our voices heard. I started into it, but then was so disgusted that they are trying to turn The Church of Jesus Christ into a popular opinion poll I shut it down and fumed about it all night. The Church is true. People are human and fallen and weak. I’ve only ever felt empowered by my faith and nothing in His Church makes me feel less than a beloved child of God.
adano – as a Mormon with a PhD in Rhetoric at a respected university, the fact a social scientist sees no problems with the questions means little. Cognitive bias is a huge problem in the social sciences, and I find that social scientists often can’t (or refuse to) see how their framing of the issues is quite often biased towards conservative/traditional view points.
See these links for more on this:
http://reason.com/blog/2014/11/04/is-social-science-biased-against-conserv
http://www.newyorker.com/science/maria-konnikova/social-psychology-biased-republicans
“Haidt believes that the problems start with the selection and formulation of research topics to begin with. In his paper, he and his co-authors review how liberal values can influence the choice of topics and method of research. What questions, for instance, do researchers choose to tackle? Those likely to get better traction are those that most resonate with the researchers—after all, psychologists often jest that research is little more than “me-search.” Given a homogeneity of views, topics can fall into ruts because conflicting approaches won’t be considered.”
@adano, my primary issue is with sampling and the level of transparency these researchers will provide when stating their personal biases. I have no problem with the fact that many of these researchers represent a more activist approach to Mormonism. I do take issue with them not being transparent about their views on these issues. We all bring biases into our research and analysis. But we should make others aware of those biases so readers can properly evaluate our conclusions.
As I read through the survey, I felt like the presentation had already ghost-written with charts and all. They were just looking for the data to include.
And certainly, the questions could have been worded in different ways (with both progressive and traditional Mormon perspectives) to more objectively (nothing is perfectly objective) test the acceptance of certain viewpoints.
Adano – I guess I can wait for the results to come out to find out what’s going on, but I’m confused how that changes the experience of people who found that there were questions that had no good options for them to choose. However the results are used, how can that not affect accuracy if the questions themselves aren’t able to glean people’s real thoughts and opinions? Even if the survey authors don’t ‘think’ the same way as many Mormons and therefore couldn’t word the questions with enough nuance, wouldn’t they want to know that? What would their response be if they were getting similar feedback from another faith tradition?
Kessee says: “Joseph Smith ordained his wives into the priesthood”
Kessee, this is the wrong audience to use that line with. We are all knowledgeable about church history and know that you’re using that totally out of the true context. Joseph Smith most certainly did not ordain his wives to the priesthood in the sense that you are trying to use it.
This is a well researched subject and you’re the one behind the curve on it.
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1754069
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1664187
I think the problem is that this questions come across too one-sided.
I would love to see a survey like this ask a much better blend of questions. I like what one friend suggested:
1. Would you support the Brethren if they said they had received revelation that women were not to get the priesthood?
2. Would you support the Brethren if they said they had received revelation that women were to get the priesthood?
3. Would you support the Brethren maintaining the current policy if no further revelation was given?
I mean this is sort of obvious isn’t it? This would be a lot better data if you asked these questions instead. We could quickly, for example, match up the answers against liberal and conservative. Liberals face difficult questions too, not just conservatives, etc. More to the point, it captures things more how Mormons see their own beliefs (i.e. based around support of the church leaders) rather than a series of answers that can all be interpreted in a negative way.
This really could have been a good study. But as it is, the results will be not very useful.
I took the survey and I think people are making a whole lot out of nothing. Beyond this one person’s supposed involvement (which has been acknowledged, by the way), there is nothing to indicate that the people making the survey have nefarious intentions.
Surveys are notorious for being difficult to do correctly. If they haven’t managed to make the questions and options perfect, that does not equate “They are trying to attack the Church!” It might just mean that they are struggling with the details and if this study is meant to produce anything of substance, they will have other parts that will help fit.
Obviously, this survey is not going to be scientifically accurate, given that they aren’t attempting to restrict multiple takes via IP address, but again, if there are other segments to this study, that could end up being okay. I think that people need to calm down and not jump to conclusions. If I’m wrong about them, then, I’m wrong. I’ve been wrong before, but I prefer to give people the benefit of the doubt.
Ivan: You have training in rhetoric, I have training in survey research. Your conclusions are fine if we’re talking about how the question is presented as part of an argument. But that’s not the context; rather, the question is part of a lengthy survey.
This specific question isn’t really a problem. Where problems will arise is if the researchers cherry pick the questions they report results for. It’s fine to come at the topic that interests you in a variety of ways, including with questions that seem provocative like this one. Asking a variety of questions is much better than asking just one, because a variety of questions will produce a variety of results. The problem comes when researchers ask a variety of questions and get a variety of results, but then they publicize only the small set of questions that produce results that point in a specific direction.
So by all means, people should screen capture the questions. And when the results come out, if results are made available for only a small subset of questions, pressure the authors to produce a full topline report.
Teri, for what it is worth, I agree with you that there was not likely any nefarious intent. I suspect we’re dealing with a “intellectual/academia” (I do not mean that in a negative way) view of religion that blinded the making of the questions such that they couldn’t really be seen my a lot of members as valid questions. Its hard to really ‘get it’ for another group of people. But since the whole point is to ‘get it’ about another group of people, there is a legitimate problem here.
adano –
the mere selection of the survey questions indicates the survey has a rhetorical end, so I don’t really buy your attempt to claim we have non-overlapping magisteria here.
I think it is important to recognize that John Dehlin is not the only known activist listed among the researchers involved in the survey. Some of the others have also been intimately involved in other agitation campaigns.
The bias is clearly shown in the selection and phrasing of the questions, which is a huge part of the problem with sociology today – as indicated in the linked articles (articles which adano ignored). It isn’t just how the results are correlated – it’s how the questions were phrased and the fact the researchers and those friendly to them have such strong cognitive bias, they can’t even see the problems with the questions.
“Non-overlapping magisteria” — Coolest thing I’ve read online this week.
All I know is that I clicked “Prefer not to respond” on one question, and I got kicked off the quiz. After 2-3 questions, it was clear that the survey had an agenda, and they were interested in a key demographic (active members of the church) and building a cache of answers that supports discord in the church.
In sum, the survey is not interested in getting an even or fair sample size. It’s interested in finding one specific demographic and building a case with intentionally ambiguous and gray-area examples to push a premise that church-membership is split.
I am a retired professor of Education. (Queen’s University, Kingston ON, Canada) If a graduate student in my Faculty presented this survey form for
approval for a master’s level thesis, I don’t think it would be
approved. (I was on the Faculty’s Research Ethics Review Committee for a couple of years and read many research proposals from both faculty members and
students, with regard to ethics in research using human subjects.) It
would not pass the university’s ethics test, specifically in that the researchers did not identify themselves by name as required, nor the name of a specific academic supervisor or faculty officer to whom questions or complaints might be forwarded. (An unidentified phone number would not be enough.)The researchers did not identify the nature of their
affiliation with the two, presumably post-secondary education
institutions. Are they students, professors, or full-time researchers?
Further, the survey
post did not give an adequate, though brief,
rationale for seeking this information. The survey program did
not give me an opportunity to review my answers and make a final
decision about whether or not to submit my responses. That is, there was
no opportunity to abort because there was no opportunity to decide
whether or not to click on a send button. (The program sent automatically upon the final question being answered, in spite of promising that the
responder would be able to move back and forth among the answers.) There
was no description of how the data is to be stored, for how long,
and what means of securing the data will be used. Also, the researcher
must provide a commitment that the responder can retrieve his or her
data at any time, thus deleting it from the data base, and must specify a process by which this may be done. For me, this
survey does not pass the “smell” test. I would describe the quality of this
survey and its questions as the first draft attempt of a master’s degree
student, before the draft goes to the student’s thesis committee for comment,
review, and revision.
Mormon Feud is a dissident game show in which two factions compete against each other in a contest to name the most popular responses to a survey question posed to 100 people. The show was created byJohn Dehlin and is now an integral part of the so-called Bloggernacle.
I enjoyed the survey, granted my lowly JD does not give me the academic insight claimed by others. Dr Magleby taught me in my survey class many moons ago that all surveys have some bias. There is NO single survey that will answer THE QUESTION but surveys with good intentions that represent a well intended and designed effort to glean meaningful information can lend to the general knowledge of the subject at hand and when combined with other relevant information may actually lead to some intellectual benefit. Of course demonstrable nefariousness would render any conclusions suspect.
Regardless of whether this survey is biased or not, won’t it be invalid, now that most people taking it have already been told what’s on it, and/or how to respond? Can’t imagine these methods getting any respect in a peer-reviewed journal.
I think the survey questions were generally done well. Yes, a few could have been reworded to sound better or more accurate for conservative members (which is the majority of the American Church). That said, I will withhold my overall judgment until the results are shared. It may be that even an activist like John Dehlin can occasionally control his bias. I like the fact that several questions included “other” and/or essay answers, so that each individual could share their nuanced views. I liked being able to write in that I’m Libertarian, rather than conservative or liberal, which I feel are somewhat problematic and loaded terms. Many conservative members are very liberal in how they help the poor, for example. Some Liberals like Al Gore give almost nothing from their own pocket to the poor. So it can be meaningless in some ways, yet it still gives a basis on which to make some determinations.
So, I did the survey. Not sure how China fits in with the rest of the survey, unless it (and the Jackson County question) were intentionally placed in as distractors or “toss away” questions, which are a valid method in surveys to keep questions a bit more honest.
“By all means, participate in the survey. Make your faithful voices heard. But be aware that your answers will… http://t.co/JBDCpcc2YX
First rule of social science research: establish trust/credibility with the population you’re studying.
@Meg Stout on November 16, 2014 at 11:29 pm
This isn’t ethnography. That is a completely different type of data collection. If you do not know the difference then you clearly do not have an in-depth understanding of social science methodology. This is a survey, and the goal is likely NOT to try and say the results are representative of the church population as a whole. Clearly that is the limitation of not using simple random sampling as part of the data collection. Furthermore, the IRB contact information is clearly listed. As having received an MS in sociology I can attest to the thoroughness of the IRB process as I also conducted research on a LDS religious topic. The methods are pretty sound. I may not agree with the exhaustiveness of some of the answers for some of the questions but for the most part I see no major problems with the study. I also would not assume some sort of inherent nefarious purpose behind the study just because you don’t like the topic. Questions regarding gender roles in a religious in institution such as the LDS church are valid and cogent to current social changes within and outside of the church.
I haven’t read all of the comments, and at this point I realize there’s a very small chance anyone will ever read mine, but I didn’t notice anyone (including the article) point out the really obvious problem with this survey.
Its results do not matter.
The LDS Church is not a democracy. Even if the survey comes back claiming that 90% of it’s takers believe that women should be ordained, that won’t change anything. All it can do is hurt outside public perception of the LDS Church. Therefore, the only purpose of this survey is to spread Anti-Mormon opinions/feelings.
You’re misunderstanding the purpose of the survey. I can’t fathom anyone expecting this to exact any change. And it sounds like you haven’t taken it either, as there are other topics addressed other than women in the priesthood.
Important w/all the news and opinions going around. Important Things To Know About the #Mormon Gender Issues Survey: http://t.co/ZNxTcPeBIi
A previous person commented saying that “they can’t change the wording” and its too late, because they already got it approved… well apparently they can change the wording because when I took the survey last night, it didn’t say “a good Latter-day Saint” does XYZ it said a “good Mormon” which is a term often used in an offensive or degrading way. This minor change can make a huge difference, and its also proof that, obviously they can change the wording. I’m not sure how much good it will do, but at least leave your feedback on the survey. ( https://docs.google.com/forms/d/19r63Y0UHfgBiDKvHVaZB-b5YXxhD9cDsbs2dr0rpQ9M/viewform?c=0&w=1 ) The survey I took was extremely subjective and leading. It was pretty clear that the survey was written by a group of people not happy with how the church is currently being run.
John,
I did take the survey, and you misunderstand my point: It isn’t the answers to the survey that matters, it’s the response of the LDS Church to those answers, which will be non-existent.
The purpose of surveys is to get a feel for general opinions on issues to inform leaders what needs to change. The problem with THIS survey is that the LDS Church isn’t going to change any practices based on its results. So any results published about the answers to these questions will have a negative impact on non-LDS public perception of the LDS Church leadership, as they won’t be seen to do anything about the answers. Should all the answers come back as fully supportive of the way things are run in the LDS Church, then it will hurt non-LDS public perception of the LDS membership as a whole, as the LDS Church’s ways are not the world’s ways. Should the results show any sort of mix (which is most likley) it will be seen and interpreted as LDS Members not being unified in their beliefs, weakening the public perception of the LDS Church as an organization. There is no potential positive outcome for the LDS Church from such a survey. That’s what we call an “agenda”. In my opinion you can call the people running this survey “Anti-Mormon” for all intents and purposes.
RT @Stan_Way: Important w/all the news and opinions going around. Important Things To Know About the #Mormon Gender Issues Survey: http://t…
I KNEW something felt off with the wordings of the questions in the Mormon gender issue survey; can’t trust it. http://t.co/SghbS2HuMM
Knowing who created the survey & what their agenda is, perhaps it’s extra important for faithful members to take it. http://t.co/SghbS2HuMM”
I have a PhD and am trained in survey methodology. I understand the intent of the survey, and it’s for academic research, not to sway or influence anyone. So, in that regard, I can’t really agree with you. Surveys that are intended to sway public opinion are not scientific. All surveys will have a bias; there is no doubt about that. But that bias alone is no reason to not try and see what the sentiments of respondents are. We all have opinions.
I’m sorry that you feel that people with a desire to conduct scientific inquiry (many, if not most of whom are members of the church) are anti-Mormon.
C212 – Perhaps as part of the IRB they explained that the terms LDS, Latter-Day Saint, and Mormon are interchangeable. Perhaps that is part of the study: framing. It is not normally a practice to change questions in the middle of a study without having prior approval to do so.
To others – far too many to list:
I would personally love to see people act like the latter-day saints we want portrayed and not deride people for wanting to research this issue. Someone had a question and wanted to answer it. Whether the church does something with it has no bearing to the benefits of looking at quantifiable data. There is something wonderful about any attempt to better understand our world. We do not know what the researchers will find. Perhaps they will find that there is no significant relationship between membership and the church and gender inequality, which would be great for everyone complaining, now wouldn’t it?
Before you berate the researchers saying that they are wolves in sheep clothing and many of the other nasty descriptives I’ve read, take a step back and realize that this research will likely have no bearing in your life. You are not accountable for what the researchers do with the data; however, you are responsible for the way in which you deal with others. The “love one another” and “kindness begins with me” mantras still apply to online exchanges. If you do not wish to participate, don’t. There are a myriad of opinions among faithful, dedicated members of the church regarding this issue that are worthy of research. Stop letting fear rule the day.
Clarisse,
This isn’t about researchers and research in general. The fact is that these particular researchers are already well known for their participation in media-oriented agitation and dissent to try to bring public pressure on the church to make changes related to the specific subject of this survey. Their agenda isn’t hypothetical, it is well established by their previous actions; including the previous survey conducted by a John Dehlin sponsored group, which although admittedly statistically inconclusive, was still used as propaganda to push their agenda.
From what I understand, the church has extensive internal data on this and all kinds of other issues that they use in consultation with the Apostles.
Some would like that internal data to be made public. I could be wrong, but I don’t think the church can make the data available to academics or the public unless the participants were advised of that possibility when they agreed to participate and consented to having the resulting data shared. Even if the data is anonymized first, knowing that it will be released publicly could affect how participants respond. So it is probably in the church’s interests to keep the data confidential in order to let the participants share their views with confidence of confidentiality and privacy.
So the Church doesn’t need this survey. It is useful only to outside groups who want it for their own reasons. And the concern isn’t about knowledge. It’s about how the data will be used by parties already known to have an agenda to manipulate perceptions and create additional media pressure, as they have in the past.
Wouldn’t it have been wise of John Dehlin to step back and not be part of this survey group? He has become the poster boy for agitation in the Church and it’s a stretch to believe that this group doesn’t have an agenda when he’s part of it. Whether his role is big or small (Brent’s comment above), his name appears on the list. Maybe it would be wise if Dehlin stuck to producing podcasts and updating his facebook page. Those techniques have helped him gain a following, but I wonder if participating in generating a survey of his pet topics was overreach.
This survey suffers from the same fundamental problem that all of the surveys connected with John Dehlin do: self-selection bias.
A college-level statistics class, usually required to obtain a degree in sociology and related fields, discusses this subject. Ignorance of the problem is inexcusable, and therefore must be deliberate on the part of those conducting this survey.
I wonder if anyone can identify a single proponent of this study that is neutral on the question of female ordination or the various “proposals” articulated in the survey. I couldn’t find evidence that any are neutral. Is there any reason to believe that the results of this survey will be published in any manner supportive of current Church policy given who is involved? The list of the proponents is available on the Facebook page for the group behind the survey, as well as the crowd-funding website they used.
I went to Google and Facebook and found the following regarding the proponents. I could find nothing that suggests that any of these folks are even motivated by anything other than animosity towards the current doctrine and policy of the Church. Can anyone show anything contradicting this thesis?
In looking specifically at the individuals listed, I found the following. I did not use a scientific representative sample, but have looked at the entire population of identified proponents of the study, though I have anonymized my findings slightly:
A professor of pyschology of religion at a university in the south-east United States. He is facebook friends with Nadine Hansen and other OW luminaries.
A business professor at a small college in Texas and a permablogger at a liberal/progressive blog for folks interested in LDS issues. His writings on that site do not suggest he is sympathetic to current Church doctrine or practice.
A social sciences professor who is in a traditional heterosexual marriage with the previous subject and also blogs at the same site. She is a driving force behind the “equality is not a feeling” drive which has been trumpeted by OW.
I could find nothing whatsoever about the next subject. Who has zero presence these days when they are googled?
A “senior research scientist” at the University of Wyoming. While I assume this means he teaches something or another, I find no evidence of this. His facebook profile indicates that he “likes” Exponent II and a number of gay rights causes.
A female graduate student at Wisconsin with interests in feminist issues. She has an OW profile.
A male PhD candidate in political science at Wisconsin. Not much relevent information was available on him but dollars to donuts he has a crush on the previous subject.
A professor of sociology of religion at the University of Tampa. He is on record as a former Mormon, claiming he left the Church in 2002. In a letter to Dialogue this summer he was very critical of the Church and those who choose to remain in it.
The next subject is John Dehlin.
A female academic from the UK who (according to linkedin) is a Coder/Scaler and a current student at Kings College seeking a masters degree in contemporary religious studies. She is facebook friends with Kate Kelly, Nancy Ross and other OW luminaries.
A graduate student at Penn State in sociology. Her page at faithstreet.com identifies her as an LGBT ally, a progressive and a feminist. She did not vote for Mitt Romney.
A professor of sociology at a small college in Texas. He has a beard. He is pictured wearing a rugby jersey with a Budweiser advertisement across his chest. There is nothing on his public facebook profile to indicate he is LDS, but nothing to indicate that he isn’t.
A female academic with a PhD in political science with a fellowship of Univ of San Diego.
An art historian from Dixie State who apparently has a history in researching feminist issues. One of her research interests is “Gothic Apocalypse manuscripts and their readers.” She is on the OW board.
A graduate student studying sociology at Ohio State. Apparently he has published an article in the Journal of Peace Research, and a chapter in a book titled Sex and World Peace.
A former doctoral candidate at the Univ. of Utah. Her facebook profile identifies her as a feminist and shows likes for various LGBT causes and Heavenly Mother groups.
An undergrad student at Weber State who studies history.
I agree with assessment in this post. The gender survey is unscientific with bad sampling & bias http://t.co/8uE2ruoLke #lds #sharegoodness
RT @MyMormonOpinion: Knowing who created the survey & what their agenda is, perhaps it’s extra important for faithful members to take it. h…
That said I encourage everyone to take it, at least 10 or 12 times as I did
@Traci Burnett on November 17, 2014 at 12:18 pm
Hi Traci,
Got it that this isn’t enthographic research. However when I dabbled in strategic planning and organizational behavior on an academic level, I was advised that the survey I planned to deploy should be based on an indepth understanding of the population, which required (in my case) that I perform ethnographic research to ensure that I had properly decoded the language and meanings the members of the community ascribed to those meanings.
I presume that these researchers felt they were sufficiently knowledgable about the studied population that they could skip the ethnographic step. And yet, there needs to be an ethical approach to the studied population.
Pat Chiu on my post about racial divisiveness shared the story of her engagement to a Chinese man in the early 1960s, when it was still illegal for an Asian man to marry a white woman in Utah. Apparently her mother (a staunch Democrat) strongly suggested that all non-whites should have the priesthood taken from them, to prevent miscegenous marriages. What benefit would have come from surveying that woman’s opinion about race and the priesthood?
As it turns out, the Mormon Church had conducted its own internal study and research in the 1940s, determining that the ban was not doctrinal and simultaneously that there would be widespread resistance were the policy to be changed. Thus we see the masterful statement about the priesthood ban that comes from George Albert Smith in 1949, covering the negative comments Brigham Young had made, but then transitioning to the messages of hope and even promise that all mankind would someday have access to all the blessings of the priesthood. The 1949 statement also countered the idea that a person’s mortal state (if underprivileged) was due to a lack of valor in the pre-mortal world. The 1949 statement was intended for a people entrenched in a wrong direction, to help them change by just a few degrees at a time, but to continue changing over the course of the years. And thus we see how pronouncements by the brethren have in the past guided the people to an expectation of a promised gift, a desire and even a yearning for a state that at some earlier time had been thought unthinkable and even wrong. With this approach, the arrival of the promised time is celebrated, like Christmas. A change that is yearned for and celebrated will be received with joy, reducing the likelihood that the change will be rejected or opposed by the body of the organization.
@Michael Davidson on November 17, 2014 at 2:39 pm
Delightful – I had suspected what you found out. Thank you for doing that digging.
Shibboleth.
First and foremost, I place my own faith and confidence implicitly in God. His inspired leaders administer the affairs of the Church. Popular trends and surveys are not generally relevant and do not drive the direction of Church policy. The survey questions seemed to be informed by very different assumptions. Nothing solicited by the survey seemed to elicit any very concerned responses from me, nor did I believe any of the “issues” addressed were of particular interest. Perhaps others will find it different.
Michael Davidson, the undergrad you mentioned in your list has said in another forum that he is neutral on women’s ordination. So, there is at least one person who is neutral. FWIW.
I found the political allegiance area amusing. The survey puts Conservative and Liberal at opposite ends of the scale. The church is a worldwide church. As such I chose “Other” and put socialist (socialist does not mean communist!). There is a joke here in the UK that says you have 2 parties in the US, the right and the far right. The question was very poorly thought out and failed to encompass much of the church membership. Incidently, I have a friend who was a Stake President and activist for the Labour party 9socialist) so socialist does not mean apostate!
Debra, just FYI: there are many people in the U.S., including myself, who think we have two parties in the U.S., the left and the far left. And when you study the growth of government spending over time, this perspective is difficult to dispute. This, by the way, is a position that was shared by several former apostles and prophets.
I agree with you that socialist does not mean apostate, and I have stated it on the blog many, many times, but I make this comment just to give you another perspective.
Meg, those are great points and it certainly seems that the church has functioned after that pattern–at least in the 20th and (so far) in the 21st centuries. But it seems to suggest that God is a respecter of persons, or at least his authorized representatives are, and this seems to contradict the widely held notion that God does what he wants no matter what the people think and that His prophets are sent to deliver His will/word/message regardless of how it will be received.
We look back on the example you shared, Meg, which occurred in the 1940’s, and frame it in a manner that depicts the wonderful mercy the Lord showed for His children that weren’t at the spiritual level to receive the revelation. But in this discussion we see these people as the tares that ought to be sifted.
Michael,
Whoa! No bias there.
Clarisse,
Out of curiosity, is there any way you know of that the authors could have expressed concern about a survey that you would NOT have interpreted as a failure to “act like Latter day Saints”?
I suppose I would want to make a division here between comments and OP. Perhaps you’re right to berate some of the commenters. (Though I have yet to see a site discuss anywhere where you don’t get some overly harsh comments.)
But the tone of the OP seems pretty mild to me and seems to truly drive to the heart of a real potential problem with the survey.
Ya…. it would seem that the authors of this survey don’t believe — or even understand — the depth of the beliefs held by most members that those who lead the Church are literally prophets, seers, and revelators. The questions never allows for options that really reflected that in it’s proper nuance.
@Observer on November 17, 2014 at 3:40 pm
No one says these folks are tares that ought to be sifted. We’re just commenting that there’s something fishy about this survey, which a frankly bizarre seeming set of individuals have collaborated to deploy on Mormons. Not that any of the individuals themselves are bizarre or unworthy of being part of the body of Christ, merely that they don’t appear as a group to be dispassionate researchers approaching an important topic in a uniformly professional manner.
I suspect the consensus from the 90% will be:
Yes, if the leaders told me female ordination was appropriate, I would accept it and (as a female) seek to be ordained. I don’t think members of the Church would have a problem with that. Except the 90% will guess that the likelihood of females getting ordained any time soon is well-less than 10%.
Yes, there are some matters other than female ordination that I would be happy to see added to the acceptable duties for women (e.g., healing blessings).
Yes, I am either a Liahona Mormon or an Iron Rod Mormon (the single question that has evoked the most objection, making that an either/or question).
Yes, I am male or female (with perhaps a few who identify as hermaphrodite or some other unusual gender).
Yes, I have a political viewpoint. (Though I suspect many will decline to indicate what it is).
RT @Millennialstar: New Post: What You Should Know About That Mormon Gender Issues Survey: [Note: this post was co-au… http://t.co/Jy0V3i…
“Thus the unfacts, did we possess them, are too imprecisely few to warrant our certitude…”
― James Joyce, Finnegans Wake
After reading through the comments I think that there is one more thing that needs to explicitly stated: that this survey has been approved by the IRBs of The University of Tampa and Georgia Southern University does not mean that the methodology being employed (e.g., wording of questions, medium of data collection) is scientifically sound. All IRB certification means is that the rights and privacy of any human subjects involved (read: survey participants) is protected. For instance, this would mean that The Mormon Gender Survey Group will not publicly release responses associated, say, with the IP addresses of the recipients. This is also the reason why they are required to actively mark that you agree to participate in the survey.
Aside from that, I agree with many of the other comments made: this survey is biased. To be scientifically sound, the questions and answers do need to be rewritten in a bias-neutral format. Furthermore, The Mormon Gender Survey Group will have a hard time publishing their results in a peer-reviewed scientific journal simply due to their data collection methodology. To name a few of the many issues: there is no way to guarantee that the responses represent a accurate sample of general Church membership (What percentage of the Church even speaks English and has an internet connection? What percentage will be found on the social media outlets through which this survey is being promoted?), there is no way to prevent the same individuals from taking the survey multiple times, and there is no way to guarantee that those surveyed are accurately stating their background as members of the Church or not.
However, as a scientist one of the most *alarming* statements made by The Mormon Gender Survey Group in regards to this survey is given on their FAQ page (http://mormongendersurvey.org/faq/, and linked to at the start of the survey). To quote:
——————————————————————————-
Q: Will you share the data?
A: Maybe… Why do you ask? More seriously, if you want access to the data you’ll need to contact one of the members of The Mormon Gender Survey Group and explain how you plan to use it. The group will then discuss it and make a decision.
——————————————————————————-
That should set off all sorts of red flags to any researcher. That they would be unwilling to share their data openly—allowing it to be scrutinized by other academics—says everything. That they would even ask one’s motivations for wanting to look at the data collected (“Maybe… Why do you ask?”) speaks openly to the biases of those conducting the survey (correct or not, I cannot but infer that they would be unwilling to share data with someone whose opinion may differ from their own and who might go about discrediting their methodology and results). I would suggest that The Mormon Gender Survey Group take a few pointers from Pew. Or, better yet, let’s see if Pew would be willing to conduct research along these lines.
And lastly, can someone please explain to me why The Mormon Gender Survey Group has gone to kickstarter for funding? Why is this “research” not being funded by research grants?
@mormongeologist #genderroles #LDS http://t.co/eNnTdKMhyo
What You Should Know About That Mormon Gender Issues Survey: http://t.co/uVeOQyalk0 #manipulationofstatistics
You know that Mormon Gender Survey? http://t.co/OXES92o4Pf
@Geoff, I did not locate the reference you are referring to with respect to the Weber State undergrad in question. In attempting to be more comprehensive with respect to this gentlemen I did, however, find a guest post authored by him at FHM, the he has posted several left-leaning guest posts on Rational Faiths, has been featured on a FMH podcast (I didn’t listen to all two hours, but he was interviewed with Richard Dutcher regarding the Mountain Meadows Massacre), he was a moderator at a “new order mormon” website, was a research fellow with the Community of Christ in Nauvoo and was a presenter at the 2014 Sunstone Symposium along with D. Michael Quinn on the history of Church discipline. I didn’t have the time to listen to all of that as well. Having looked at all that, I still didn’t find anything to suggest that he leans one way or the other, though I accept that your report is accurate, assuming we are talking about the same Weber State undergrad.
I am the King of the Echo People . . .
Andrew, thank you for explaining that. You’re right, that is very concerning that they won’t let people openly review their data.
The best part of the survey? They’ve got an open-ended “other” AND a “prefer not to respond” option for sex. No forced choice between male and female. Just between possible ideologies. Because one’s political opinions and religious beliefs are much less nuanced than one’s sex.
@Andrew Auman on November 17, 2014 at 4:54 pm
As for the Kickstarter, I suppose they had a second methodology they wanted to employ that cost money, thus asking the Kickstarter world to give them $3K. Plus they could justify limiting results and data to that group, because it was one of the promised perks of donating. Given the timing of the Kickstarter campaign, it seems unlikely that the pool of those donating is representative of general Church membership (e.g., it was running during the weekly discussions/conversations being posted by those seeking female ordination, and the survey certainly appears to have been crafted to bolster the position held by those who had sympathized or authored those discussions/conversations, from my point of view having made a point to read and critique each one).
As for the approval of the IRBs, It’s like ISO 5000 – it’s an international standard for processes, and compliance means that your process is repeatable. But if you repeatable process turns out, for example, violin shaped objects without quality tone, then all ISO 5000 certification proves is that you will consistently get VSOs without quality tone.
I knew there was a problem with the survey when it gave the option of not being male or female. What??????????????? And many of the questions forced an answer when actually a combination of answers was really correct. Obviously, the survey is by liberals, non-members, “progressive” Mormons, those who want change in the Church. There are plenty of things that could be improved in the Church, not just the human factor. Programs, policies, curriculum are often changed because inspired leaders are led to make changes and improvements.
There were two questions that I did not answer: Both asked the percentage likelihood of a new revelation on subjects. They might as well have asked “What is the likelihood you will see God, as did Moses?” They are, quite simply, silly questions, the answers to which have no statistical significance, nor relevance. This puts the bona fides of the entire survey in serious doubt.
I decided to take the survey before this particular M* post came out. I gave up after probably a dozen or fewer questions because I could absolutely not answer in a way that represented my actual viewpoints. For example, with the “good Mormon” question:
My first thought was “what kind of counsel?” Did it mean counsel within the line authority of the priesthood leader? (If a bishop asked me to do something as a leader/teacher in the ward, and it falls within his line authority, I would do it even if I disagreed with it–it’s not my call. The bishop gets to decide how the metaphorical chairs are arranged in the ward under his direction.) Did it mean personal counsel of a bishop regarding advice on a difficult situation in my life or family? (in which case, I wouldn’t necessarily feel bound, because I am the “line authority” for that decision, not the bishop, and the only one with all the information)
More so than the questions worded in a “Yes or No — Have you stopped beating your spouse yet?” way, my greatest frustration in answering the questions was that my answers almost always hinged on what the surveyors *meant* when they used certain words in their questions (as in the example I gave above). To the extent that the context or nuance of a word or phrase would have changed my answer, I was completely unable to give one that I felt was informed (or informative).
As a consumer researcher of 15 years I usually do not qualify for most market research surveys. Thus, most of the surveys I qualify for are academic in nature and all seem to have one thing in common – written by researchers whose research training is limited to a couple doctoral level research classes. The surveys are fundamentally flawed and this one was no exception. I do not know if the researchers who wrote this survey had an agenda but I do know the survey is flawed and the researchers have limited understanding of practices of the LDS church.
Here are a few examples:
1. A person can not recommend someone for church discipline. If someone learns of the indiscretions of a member of the LDS church they can certainly let the person’s bishop or stake president know but only a bishop or stake president can recommend someone for church discipline. To this end it would be no different than a customer at McDonalds going to the manager and recommending that someone be fired for some violation of rules. The manager should know of the indiscretion but a decision to fire an employee rests with the person who has that authority.
2. Several instances where “none of these” or “don’t know” should have been included but were not. For example, on the “check all that apply” list of things that the respondent has witnessed in the LDS church (being left out of a baby blessing, not being consulted for baby blessings, etc), there was not a “none of these” option. Yes I was able to skip that question, but sound research would have forced a response and included an “out” for those to whom it does not apply.
3. Ambiguity of terms such as what it means to be a “good Mormon” and choices unrelated to whether someone is a good Mormon.
There are others, but those stood out to me. My purpose here is not to prove or disprove that the researchers have an anti-Mormon agenda. I am just pointing out that the survey instrument itself is flawed.
When creating research it is important for the researcher to begin with the end in mind and keep in mind how the responses will be reported and how the story might come together. As I envision this research being presented, I can predict several objections to the methodology and limitations to the interpretation of the results.
I have heard some they are taking the survey so their voice can be heard. Whether someone is “pro-Mormon”, “anti-Mormon”, or “Mormon neutral” – their voice may be heard but it will only be partially heard and heard with a lot of noise and ambiguity.
To “The Obivous Problem”
I did read your comment, and hope you read my response. You are correct, the Church is not a democracy, it is the Kingdom of God. However, the results clearly are going to be used as propaganda to sway people to follow strange paths that lead away from salvation. This is the real impact of the survey.
(waves hello to Meg Stout!)
Ah, the sly ones. If only I had $50 for every time someone tried to show me the “truth” about my church, my culture, and may faith, by using deceptive tactics and outright lies. Y’all gotta try harder, agitators. Every time you take a swing at me, my testimony moves an inch closer to Christ, and our living Prophet. Our church is not a club. It’s the church. You either buy it (warts and all) or you don’t. To include this: we affirm the leaders as being chosen by the Lord for His purposes, not ours. If you think the church ought to “evolve” on certain things, and you want to help shove and push it along, I think you either don’t have an actual testimony, or you’ve allowed yourself to care more about the ways and opinions of the World, vs. the ways and opinions of Heavenly Father. Therefore it’s not the church’s job to repent and get closer to where you stand, it’s your job to repent and get closer to where the church stands.
…be aware that your answers will likely be used to promote agitation in the church, regardless of your answers. http://t.co/nt8IRFaPBW
… be aware that your answers will likely be used to promote agitation in the church, regardless of your answers. http://t.co/QKIz6hI93v
RT @scripturecookie: …be aware that your answers will likely be used to promote agitation in the church, regardless of your answers. http…
One problem is Dehlin is saying disaffected members to participate. That would mess up the results.
Jello,
Take the survey. Your question is on there.
@Brad R. Torgersen on November 17, 2014 at 9:50 pm
::hugs:: Lovely to see you comment here!
I suppose the only thing I would amend about your fabulous comments is the “warts and all” comment. Because sometimes a “wart” is a growth that can or should be removed (for example, the stage 3 cancer in my mother’s gut). And sometimes the “wart” is actually a fleck of mud flung by an external entity that has been allowed to remain on the body through ignorance.
When an unhealthy growth needs to be removed, however, I would look to those with the stewardship to do so, and the overarching responsibility for the body which makes them accountable for the impact of the “fix” on all of that body. I wouldn’t allow any random shmoe to cut the growth from my body so it looks “right” to them!
However when the wart is a fleck of mud or a trick of the light, I think it is entirely appropriate for folks to take a sponge to the thing, or point out the trickery that has produced the illusion.
Pingback: The Cultural Hall Podcast – Mormon News Report, 18-November-2014
The church does it’s own surveys. Well, inspriration isn’t enough, I guess.
I did not think that the questionnaire was subtle at all and the available answers were embarrassingly manipulative. One could have written their report on the results by merely reading the questions and I would be astonished if any reputable poll firm had anything at all to do with it. I filled it in , almost impossible given the restrictive nature of available responses, because why not . Keeps them busy, made my feelings felt in extra answers job done . As for possibly being used to agitate against the church whatever our answers might be …..last days pal we will not be snoozing through them in the armchair so might as well suit up and get on with it . Brigham Young said at the start that the biggest threat to the church in the last days would come from within and these are they, the last days . Lots will be picked off from within only thing you can do is make sure you are not one of them cos the church is going nowhere this time around , none of this is a threat to the church . As for women and the priesthood they would have to catch me first ! Let’s not and say we did .
“The church does it’s [sic} own surveys. Well, inspriration [sic] isn’t enough, I guess.”
What part of “study it out in your mind” did you not understand? 😉
Pingback: What You Should Know About That Mormon Gender Issues Survey | Meridian Magazine
Pingback: *That* Mormon Gender Survey – the FAQ | Religious Reason
@ Michael Davidson,
thanks for ferreting out who the supposedly neutral folks are who put this survey together. They sound about as neutral as the jury of Jabez Stone.
@Andrew Auman,
it stinks to high heaven that they aren’t going to make their data public and even try to attack the motives of people who’d ask for it. That is the behavior of advocates and partisans. It speaks volumes. Thanks for pointing that out.
Pingback: To the Fence Sitters Regarding the LDS Gender Issues Survey | Life Outside The Book of Mormon Belt
In order to get IRB approval you must agree to keep certain protections on the data in order to protect those who take the survey. Some IRBs require more protections, some less. Usually they can be shared for legitimate research purposes but there are strict controls on that, too. This is standard IRB procedure, not some vast liberal conspiracy.
Just a few points of clarification. Our motivation is primarily academic. Existing data on this topic (from Pew, for example) isn’t reliable for all sorts of reasons (and it’s not very detailed). For example, Pew reports that 90% (or so) of women in the LDS church don’t see gender issues as a problem. At the same time, on any given Sunday, 2/3rds of LDS women (2/3rds of the women that are part of the 15+ million membership number) are absent, and we know that the majority of these women are inactive and/or no longer identify as LDS. We also know that gender issues are one of the primary reasons why women disaffiliate from the church. Putting all this together leads pretty quickly to a contradiction (and it’s a contradiction that can’t be easily resolved without more–and better–data). That was (and continues to be) our primary motivation. Our data will be peer-reviewed (and it will, eventually, be made public). John Dehlin has not been involved to this point in the project, but there is still a lot of work to be done, and John may participate in that process (that’s largely up to him). We paid for a national representative sample (that’s what the Kickstarter campaign was for), and we intend to use that sample to make sure that our larger “snowball’ sample is representative. Several of the questions that seem to bother folks the most actually aren’t ours. We pulled them from other surveys (from Pew, for example), so that we could compare results. We do not have an activist agenda. What we want to know, basically, is how big of a problem gender issues are in the LDS church, and that requires data. If asking the church membership–all 15+ million members–what they think about these issues is threatening to some members, I think the natural question is why such an approach is threatening? Are active members afraid of what the 2/3rds of other members are going to say? We’re interested in answering the question. What the members or the institution chooses to do (or not to do) with our answers isn’t something that we’re concerned about right now (we’ll leave that part up to the activists on both sides).
Overheard on the internet:
“Just a reminder that the survey was reviewed and approved by the Institutional Review Board (IRB). They are the gold academic standard when it comes to social and behavioral science research and it’s their number one job to make sure research surveys like this contain no bias. It’s far more likely that the writer’s bias has been exposed – not the survey’s.”
Comments like this are difficult to understand. If someone ‘feels the survey is biased’ (and we’ve seen a large number of people here that felt the same way) it’s unclear how having it reviewed by the IRB for approval — or any third party — could somehow make it non-biased. I mean isn’t the fact that the very people meant to take the survey (to find out their feelings on a subject) feel like its biased sort of make it tautologically biased?
Or is there some sense in which the survey could be unbiased and, as this commenter above states, that somehow the bias of the people being studied is somehow causing the feelings of bias?
Put more simply, if you make a survey to find out what a group of people think and those people feel the survey is biased, doesn’t that by definition make it biased? If not, how so?
IRBs are hardly “gold standards” – see Meg’s comments above. And AH’s comment is somewhat incorrect, see Andrew Auman’s comments above.
Mr. Beal,
Perhaps you would be willing to clarify your clarification a bit.
You say, “We also know that gender issues are one of the primary reasons why women disaffiliate from the church.” How do you know this?
You say, “Our data will be peer-reviewed (and it will, eventually, be made public).” Your data will be peer reviewed, or the publications involving your extrapolations of this data will be peer reviewed?
You say, “John Dehlin has not been involved to this point in the project, but there is still a lot of work to be done, and John may participate in that process (that’s largely up to him).” Why is he listed as a participant? Why is everyone trying to distance the study from him?
You say, “We paid for a national representative sample (that’s what the Kickstarter campaign was for), and we intend to use that sample to make sure that our larger “snowball’ sample is representative.” What good is the “snowball sample” in the first place if you are already investing on a truly representative sample? Why waste time on a “snowball” sample in the first place?
You say, “We do not have an activist agenda.” Then why is there not a single person involved in the study who is on the record, aside from a single undergraduate, as opposing female ordination. What assurances can you give, provided the list of individuals working on this and the bias perceived by so many that this data isn’t being gathered to further an activist agenda?
Regarding 2/3rd inactivity rate among women. That sounds about right, on a worldwide basis. And I believe the inactivity rate among men is the same or higher. In the US wards that Ive been in, it has seemed that a higher percentage of men are inactive than women.
The main dividing line of inactivity rates is US versus non-US. The non-US inactivity rate is much higher. My understanding is that the US rate is something like 50% and non-US is about 75%. I forget where I picked that up, maybe at cumorah.org.
So given that more women than men are present in almost all representative samples of active Mormon life, should we be more concerned about the 2/3rds of baptized women who no longer attend, or the presumable 3/4th of baptized men who no longer attend?
Last I heard, it isn’t the Mormon men who are having a hard time finding suitable spouses within the faith.
@Bookslinger on November 18, 2014 at 8:46 pm
So it would appear that rather than focus on the 35-45% of US Mormon women who leave the faith, we should be focusing on the 55-65% of US Mormon men who leave the faith. But only after we figure out what we can do to retain the 60-70% of non-US women and 80-90% of non-US men who are leaving the faith.
I wonder how many women are freaking out because they’re being told they are among a majority that are sloughing off of the Church, rather that part of a minority of malcontents?
Can’t help hearing the tune “The world has need of willing men…”
Brother Brent Beal,
Thank you for chiming in again. I appreciate what you are saying and your willingness to try to clarify your intent.
Obviously I can’t speak for everyone who has agreed with this blog post. However, I think you’ll find that the skepticism with which your study has been approached by many here is not because of ignorance about academics. Many of us have quite a lot of experience with academia, some directly, and others indirectly through close friends and family who are also scientists and academics themselves.
And our skepticism is born of that familiarity. In other words, we know how academic work functions well enough to not take this kind of thing at face value. Objectivity is a myth. The academy is rife with camps, and agendas, and politics, and even personal vendettas. At least in my own experience, it is those who try to mask their agendas in a false claim of objectivity that I have learned to distrust more than those who are open about their bias.
In my estimation, modern social science often tends toward “applied” research, which is research that is intentionally oriented toward causing social change (in contrast to to merely “evaluative” research).
Given that,
1. the vast majority of the researchers involved with your project have very strong progressive views that run counter to the majority of the members of the church as well as the teachings of the leaders of the church,and
2. that some of the primary researchers, including yourself, have been publicly outspoken about your desire to change the church and are recognized as activists, and
3. other primary researchers and people funding the project have actively participated in targeted, public actions meant to bring pressure on the church to change,
It seems clear to many of us that this study is “applied” social science research that is intended to bring about change and not merely evaluative, as your comment above suggests.
Do you deny that it is “applied” research?
It seems that if you didn’t want this study to be construed as an act of agenda driven activism, you should have made a better effort to include a diversity of researchers and people supplying funds. The lopsidedness of your team cannot avoid integrating their largely shared worldview into the study itself.
So rather than continue to claim objectivity and dispassion, perhaps it would be best to own up to the “applied” agenda of your group. Because at this point, you aren’t really changing any minds by continuing to claim otherwise.
Mr. Beal, if you are ever interested in re-writing this survey so that Mormons don’t see it as biased but will still obtain for you the information you are looking for (actually, much better information than you are looking for) several of us have been discussing how to rewrite the questions so that it would really allow people to express their real views and test people’s views. Let us know if interested.
If the LDS Church has a 33% activity rate worldwide, that would make us top tier as far as church’s and activity rate.
The 1990s study Why Strict Churches are Strong showed the LDS Church with the single highest activity rate of any church. And that was at 40%. Our hey day.
LeviBarnes – The question was definitely not on the survey when I first took it, so it must have been added.
Jello,
It isn’t really a problem or surprising that different participants get slightly different questions. It isn’t uncommon for surveys to be designed to vary the wording of questions for certain percentages of the participants, or to send participants down a slightly different set of questions based on their previous answers. It all depends on what kinds of things the researchers are testing for. Often what is being asked can be testing for something different than you would expect based on the language of the question.
For example, in a survey designed to test career interest, there might be a question that asks you to rate how much you would enjoy cleaning test tubes. You might think that if you answer that you would dislike washing test tubes that it would obviously lead to results that would exclude “lab tech” as a career suggestion. But, contrary to your supposition, they would be likely correlating you answers with the answers given by people who are already in certain careers and have reported enjoying what they do. So it could be that lab techs who report that they love their jobs also report disliking cleaning test tubes to a degree similar to your answer, even though it is an unavoidable part of a job they love. So saying you would dislike washing test tubes might in fact increase the likelihood of having the test suggest you might like being a lab tech. It all depends on what they are really measuring and what they are comparing it to.
In my opinion, it is likely that the variations people have reported in their experience taking this survey were built into it from the beginning and not introduced in response to the attention it has received.
Dr. Beal,
Your motivation shouldn’t be “primarily academic”, it should be academic. And that’s part of the issue here—that there appears to be an agenda motivating your research. Given, we are all human. We all have opinions. And sometimes our opinions can, naturally, influence our scientific research. That’s something all of us scientists need to recognize. And I think that the concern you see being expressed here and elsewhere is that your group is letting its opinion get in the way of the science; your group is coming across as setting out to prove your hypothesis to be true instead of setting out to test whether your hypothesis is true.
If your group seeks the truth, the first thing you need to do is remove any perceived bias on the issues being discussed from both the questions and answers of the survey. This is absolutely critical in the social sciences because any time a respondent can perceive a bias in a survey it is going to influence how they respond, tainting the results. For example, many of those expressing their opinions on this forum disagree with the perceived agenda of The Mormon Gender Survey Group. The sense I get from the comments being made is that these individuals feel put on the defensive by the wording of this survey and the intent of those behind it. Many of those I know have stopped midway through the survey—or have chosen not to take the survey—because they feel that your group’s bias will lead you to misrepresent their opinions on the matter. Indeed, many I know feel that how you have worded both the questions and the answers does not allow respondents to adequately express opinions that differ from those of The Mormon Gender Survey Group. Hence, already you are alienating—and missing—what I suggest constitutes a significant portion of Church membership. Clearly, those who agree with the agenda that comes across in your survey will be more willing to respond, and this will bias the mean of the distribution of the results towards your position. This will lead to unrepresentative data and erroneous results.
Acknowledging that people—scientists included—do have opinions which can influence their research, I strongly recommend you follow the standard that Pew has set. I find it extremely disappointing that you would call into question Pew’s research without providing any scientific analysis to back up your claims, and I feel that in denigrating Pew you do yourself and your group a disservice. Consider what Pew does in order mitigate any biases its personnel may have from the research they perform. To quote from their website (http://www.pewresearch.org/about/mission-and-history/ under “Code of Ethics”):
//————————————————
Independence, impartiality, open-mindedness and professional integrity are indispensable to the mission and success of the Pew Research Center. To promote and preserve these values, the center’s Code of Ethics includes the following policies:
Conflicts of Interest
Pew Research employees must avoid conflicts of interest or the appearance of conflicts of interest. They should never engage in any activity that might compromise or appear to compromise the center’s credibility or its reputation for independence or impartiality. All employees are required to seek prior approval from a supervisor before engaging in any activity that may be deemed a potential conflict of interest, including membership in groups, boards and associations that may call into question the center’s credibility or its reputation for impartiality.
Prohibitions on Electioneering
As a nonprofit 501(c)3 organization, Pew Research and all of its employees, when acting in their professional capacity, are prohibited from participating, directly or indirectly, in any political campaign activities on behalf of, or in opposition to, any candidate for public office. In addition, the center has a strict prohibition against partisan political activity by senior staff, even when they are acting in their individual capacity and on their personal time.
Integrity of Research
To ensure that the information we generate is of the greatest value to citizens and policymakers, the center is committed to conducting research in a manner that is impartial, open-minded and meets the highest standards of methodological integrity. We employ only those tools and methods of analysis that, in our professional judgment, are well suited to the research question at hand. We describe our findings and methods accurately and in sufficient detail to permit outsiders to evaluate the credibility of our results. We encourage inquiries about our research methods and practices, and attempt to answer requests for information promptly.
————————————————//
Can you and the members of your group lay claim to such a code of ethics? I think not. That’s something worth considering.
Now, you claim that Pew’s data “isn’t reliable for all sorts of reasons”. Yet, the only sort-of-reason you provide in an attempt to substantiate this claim is that:
“Pew reports that 90% (or so) of women in the LDS church don’t see gender issues as a problem. At the same time, on any given Sunday, 2/3rds of LDS women (2/3rds of the women that are part of the 15+ million membership number) are absent, and we know that the majority of these women are inactive and/or no longer identify as LDS.”
So, let’s analyze this. The first sentence above is a statement from Pew’s 2011 research “Mormons in America” (http://www.pewforum.org/files/2012/01/Mormons-in-America.pdf). I’ll accept that, it is what they reported. Next, you purport that the second sentence quoted above substantiates your claim that Pew’s research is unreliable. Question: how does LDS sacrament meeting attendance speak for or against the validity of Pew’s research? The answer is that, it doesn’t—not unless Pew chose to only accept answers from those surveyed who state that they actively attend LDS church meetings on Sunday. Fortunately, Pew is epitome of sound research, and they very, very clearly state their methodologies. Read: we don’t have to guess whether or not they did this, they tell us exactly what they did. Quoting from “Identification of Mormons” on page 70 of “Section 5: Survey Methodology” in their report, they state:
//————————————————
For both freshly sampled households and those in the recontact sample, the survey began with a screening interview. Respondents reached by landline were selected by randomly asking for the youngest adult male or female currently at home. Interviews on cell phones were conducted with the person who answered the phone, if that person was an adult 18 years of age or older. The screening interview consisted of a few short warm-up questions (about the respondent’s level of satisfaction with their community and their life), followed by a question about the respondent’s religious affiliation: “What is your present religion, if any? Are you Protestant, Roman Catholic, Mormon, Orthodox such as Greek or Russian Orthodox, Jewish, Muslim, Buddhist, Hindu, atheist, agnostic, something else, or nothing in particular?” Those who described themselves as Mormons in response to this question were then administered the main survey, while the interview was discontinued for non-Mormons.
After identifying themselves as Mormons, qualified respondents were asked a separate question, “And is that The Church of Jesus Christ of Latter-day Saints, the Community of Christ, or some other Mormon church?” All self-identified Mormons were eligible for the survey, regardless of whether they identified themselves as part of the LDS Church. In practice, nearly all self-identified Mormons (99% in the current sample) describe themselves as part of the LDS Church.
————————————————//
Note, rather importantly, that the level of Church activity was not a factor in whether or not a respondent was administered the main survey. All that was asked, was whether they identify as being Mormon. Accordingly, the only reason you give to support your claim that Pew’s research is unreliable is utterly impertinent. At best, you could question the reliability of Pew’s research by stating that it included responses from Mormons who are not members of the Church of Jesus Christ of Latter-Day Saints. But, as Pew identifies, this only equates to 1% of those surveyed—well within the error margins they give for the statistic you quote from their report. Or you could state that this survey was only conducted in the United States, but Pew is completely open that this is the case and that their results might not reflect worldwide membership sentiments.
Consider, on the other hand, the lack of reliability demonstrated by your group’s approach. Anyone can take your survey. Anyone can take your survey as many times as they want. And everyone already knows what your survey is about—you don’t even have any pre-survey survey to determine whether your respondents should be given the main survey, as Pew did—so if I have an agenda on this issue I can take your survey as many times as I want to try to sway the results to my opinion. And don’t try to tell me you can filter out multiple responses from the same individual. You can’t. And if someone who isn’t a member of the Church feels strongly enough about this issue that they choose to respond, how do you prevent them from identifying themselves as active, faithful members of the Church? Because you claim to need something that accurately reflects Church members’ views, which—in other words—means only Church members’ views. What do you think will be the error margins on your group’s research? Is it even possible to quantify the error in the approach you’re taking?
Now, even though I’ve shown that your 2/3 argument has no bearing whatsoever on the credibility of Pew’s research, let’s talk about this a bit. First off, you give no references for your data. At all. Can you provide verifiable data to support your attendance statistics? And how accurate is that data? Sure, you can look at the Church’s statistical report, take the number of congregations, make some guess as to the average size of a congregation, multiply the two, and then see what percentage the result is of the total listed membership of the Church. But, again, that involves some guessing. And although you can try to make an educated guess, it’s still a guess and it will be very hard to measure the error associated with it. You see, as per Church Handbook of Instructions, Book 2, Section 21.1.35 “Research Studies in the Church” (https://www.lds.org/handbook/handbook-2-administering-the-church/selected-church-policies), “The only authorized research agency of the Church is the Research Information Division of the Correlation Department.” So if you’re going to make claims about worldwide Church attendance, you’re going to have to get your information from them. And they don’t give that information out. But whether or not you agree with their policy, their non-disclosure of those numbers doesn’t give you the right to guess what those numbers are and then claim that your guess is accurate. Furthermore, upon what scientific basis would do you claim to know that “gender issues are one of the primary reasons why women disaffiliate from the church”.
What I do infer from these statements you made is that you want to try to discover the viewpoint on this topic which accurately reflects the worldwide membership of the Church (those whose names are on the records of the Church). What makes you believe that your survey accomplishes this? Aside from the very important points made above about the problems with your very public and open internet survey, what are you doing to reach the general membership of the Church? Are you translating your survey into all the languages of the members of the Church? What are you doing about those who don’t have internet access? What are you doing to address the disparities between age and social media usage (as I’ve only seen your survey being promoted through social media)?
Also, you claim that Pew’s research is unreliable because it doesn’t account for the views of those who “no longer identify as LDS”. Again, I’ve already shown that this statement doesn’t support your claim that Pew’s research is unreliable. However, I think that your statement begs an important question: if you are seeking the viewpoints of Mormons on gender issues, why would you seek to include the viewpoints of those who do not self-identify as being Mormon and then present their views as being those of Mormons? If you want to study the viewpoints on gender issues of those who do not identify as being Mormon but whose names are on the records of the Church, by all means go ahead. But if, in order to get the conclusion you want, you have to present the views of those who do not self-identify as being Mormon as the views of Mormons—irrespective of whether or not their names are on the records of the Church—then you have gone very far afield of what we call science. And as a side note: if you do correlate individuals whose names are on the records of the Church but who no longer self-identify as being Mormon with opposing views to the Church on gender issues, you’re still a long, long way from showing that those differing views were what caused these individuals to no longer self-identify as being Mormon.
Are you beginning to see why your group is being called out on what it is trying to pass as research? Can you not see the stark disparities in the application of the scientific method between your group and serious, unbiased, agenda-free research groups such as Pew? Can you not see how poorly it makes you appear to call into question Pew’s research given your own methodologies? You claim that you “do not have an activist agenda”, but I feel that your comments and methodologies do not support that claim. Not even close thereto. And as much as you are IRB certified, what I am really starting to wonder is whether the research ethics committees at The University of Tampa and Georgia Southern University are aware of, and approve of, your methodologies with this research.
It is well and fine that you don’t consider yourselves as having an activist agenda, but if you want to convince me—and many others—that you don’t have an agenda you are going to need to go back to the drawing board.
I could read the deception from the first question. It’s the good ole’ “bait and switch.”
Since when do the members of the Gospel of Jesus Christ decide what is best for ourselves? Only in this generation are some so bold and arrogant as to assume that RIGHT. The last time I checked, the Savior is in able control of his Gospel. He has it covered through an order called the Priesthood. The Prophet, the First Presidency, and the Twelve Apostles, hold the keys, and lead the way. There is no quicker way to truth and true understanding than obeying the Commandments of God, which by the way, are available to all religions. Those ten laws are gifts to all mankind, not omens. They teach us how to attain happiness. They bring freedom, truth and light to ALL who are wise enough to put them to the test. They GUARANTEE blessings. They do not bind, or tolerate darkness. They prepare each soul for deeper knowledge and understanding, that is the Gospel of Jesus Christ. They are the road map back home to our Creator/Father and the Savior of all mankind. A person who abides by the Commandments of God will never have to hide, shrink from the masses, feel shame or even worry about the ever-changing issues on the table. That painful price has been paid by those who came before us in this Gospel. Those brave and valiant who sacrificed and died, sealed our blessings with their very blood, sweat, tears, and lives, giving all. Our part is to be obedient to truth, no matter how enticing the calls of the adversary. Our part is to endure to the end.
Andrew, feel free to go ahead and drop the mic. Bravo.
From Brent Beal:
” At the same time, on any given Sunday, 2/3rds of LDS women (2/3rds of the women that are part of the 15+ million membership number) are absent, and we know that the majority of these women are inactive and/or no longer identify as LDS. We also know that gender issues are one of the primary reasons why women disaffiliate from the church. ”
I find this statement astonishing. First of all, it tells us clearly that the purpose of this survey is to push an agenda. Second, it comes up with a result that completely contradicts the anecdotal experience of most active Latter-day Saints that I know.
I was a member of a bishopric in Brazil. I interviewed perhaps a dozen women who went inactive. Not one — NOT ONE — ever said she was concerned about “gender issues.” The primary reason for women not going to church on Sunday in Brazil was — far and away — the fact that they would rather be at the beach. There were also concerns about the Word of Wisdom and the law of chastity. Brazil has more than 1 million latter-day Saints, and I would venture to say that the number of women in Brazil who are concerned about “gender issues” is close to zero.(Keep in mind that the LDS church is considered much more progressive in Brazil than the Catholic church and various evangelical churches, which are the main competition).
I lived in Miami for many years. I was on the High Council and was involved in ward councils as the HPGL. I never heard of a single case — NOT ONE — where a woman became inactive because of “gender issues.” The primary reasons were, again, the Word of Wisdom and the law of chastity. Wanting to go to the beach or do something else on Sundays also played a role.
I am not saying that “gender issues” are never a concern for women. But Bro. Beal’s statement shows a disconnect from the reality of how Latter-day Saints truly live that is, frankly, difficult to process. And the fact that he can make this statement on a faithful Mormon blog and expect no pushback shows a huge disconnect from reality. Talk about your ivory towers.
I’ve been grateful for many of the comments on this thread, but I want to echo Michael – Andrew, you’re awesome.
Pingback: *That* Mormon Gender Survey – A Researcher Responds! | Religious Reason
Thank you to Andrew for finding the Pew’s guidelines. In particular – “Pew Research employees must avoid conflicts of interest or the appearance of conflicts of interest. They should never engage in any activity that might compromise or appear to compromise the center’s credibility or its reputation for independence or impartiality. All employees are required to seek prior approval from a supervisor before engaging in any activity that may be deemed a potential conflict of interest, including membership in groups, boards and associations that may call into question the center’s credibility or its reputation for impartiality.”
How would John Dehlin’s facebook status from this morning, in which he is advertising a carol singing event on Temple Square, work for him if he was a contributor to this study through the standards of the Pew Forum? This status is hardly surprising, but the timing is sort of amusing after Andrew’s comments.
John Dehlin
1 hr · Logan, UT, United States · Edited ·
I am John Dehlin, and I support this event with Ordain Women, Feminist Mormon Housewives, and Young Mormon Feminists.
Sure wish I would have seen this BEFORE I filled out the survey this morning! Oh well, I was honest, but I did think some of the questions were not worded well. For some reason, I thought it was sponsored by some university students locally, and just wanted to help.
All, this was/is a fantastic discussion. I love the fact that we can always point to this post now for any time data from this survey gets used and point out that the concerns were expressed in advanced that it was biased.
@Andrew Auman on November 19, 2014 at 12:08 am
Bravo!
@Andrew Auman
Very impressive response, i read all of it. You write with such logic and organization, i feel converted to your opinion.
As Meg said, Brave!
oops. “Bravo”.
The presence of Brent Beal and and Heather Beal, who have quite progressive views in their “Doves and Serpents” blog and where they are anything but shy about their views about gender, etc., should make it clear where the survey is going. It seems they’ve drunk the Kool-Aid (as many others have) that how you “feel” is a good substitute for good thinking, rationality, common sense, etc. Their axe to grind, and pointed lack of attention to anything the women leaders of the church have said in the past two years should be illustrative.
A liberal in the Church hasn’t a testimony. Unfortunately, some are among us who claim to be Church members but are somewhat like the scoffers in Lehi’s vision—standing aloof and seemingly inclined to hold in derision the faithful who choose to accept Church authorities as God’s special witnesses of the gospel and His agents in directing the affairs of the Church.
There are those in the Church who speak of themselves as liberals who, as one of our former Presidents has said, “read by the lamp of their own conceit” (Joseph F. Smith, Gospel Doctrine [Salt Lake City: Deseret Book Co., 1939], p. 373). One time I asked one of our Church educational leaders how he would define a liberal in the Church. He answered in one sentence: “A liberal in the Church is merely one who does not have a testimony.”
Dr. John A. Widtsoe, former member of the Quorum of the Twelve and an eminent educator, made a statement relative to this word liberal as it applied to those in the Church. This is what he said: “The self-called liberal [in the Church] is usually one who has broken with the fundamental principles or guiding philosophy of the group to which he belongs. He claims membership in an organization but does not believe in its basic concepts; and sets out to reform it by changing its foundations. It is folly to speak of a liberal religion, if that religion claims that it rests upon unchanging truth.”
And then Dr. Widtsoe concludes his statement with this: “It is well to beware of people who go about proclaiming that they or their churches are liberal. The probabilities are that the structure of their faith is built on sand and will not withstand the storms of truth.” (“Evidences and Reconciliations,” Improvement Era 44 [October 1941]: 609.)
(Harold B. Lee, The Teachings of Harold B. Lee, 394.)
It seems pretty clear to me that the purpose of this survey is agitation, not understanding. I find the mindset behind it offensive, totally foreign to my experience in the Church, and will not participate, even if contacted randomly by the sponsors.
If you want your voice to be heard about whether or not the Mormon Gender Issues Survey was biased, take this survey.
https://docs.google.com/forms/d/1_CwyMANSw5NBd4A90E-Uj6WNSbEcJo21pTvvVS9j504/viewform
Then share the link on social media.
Brent, I have trouble buying that you just want information for the sake of having information. And if you wanted better information than what is already out there then you would have a better survey than what was previously used and not rely on such a biased means of collecting the data too. There is no way that a social media survey will be a representative random sample and no way you will get better information using questions like this even when you do have a good random sample.
It is designed to produce a certain results and you want the results of this survey for a reason. It looks very obvious to me that the reason you want that information is so that you can pressure the church to conform to your will or try to shame the church in the media.
@Kessee “Mormons, being less than 6 million active participants, need to realize that you do not control the dialogue and get to define how people perceive your religion. Just because outsiders choose to not use your culturally scripted lingo to soften various bizarre and mystical practices, does not mitigate the fact that these people are right.
“For example: Garments vs. Magic Underwear. The reality is both are correct. Just as the author is explaining false dichotomies, this is exactly one of those. (For any of you to deny it is intellectually dishonest and being willful and purposefully ignorant) That is how they have been presented time and time again culturally, having metaphysical properties and protections put into them, regardless the current PR video about them.”
It is YOU who is being intellectually dishonest here. By your pseudo-reasoning, the militant atheists’ bigoted taunt of “Invisible Sky Wizard” is an acceptable term to use for God. Or does that not apply since people who believe in God have a population higher than 6 million? There’s no intellectual honesty in THAT approach, either.
In short, if you insist on using that language, you are a bigot. Period.
Kind of an ironic approach, using another survey about the first survey. I started it but I noticed that the second page had nothing to do with the stated topic, and just seemed like a short version of the first survey.
In the end, it doesn’t matter how many people think it is biased, what matters is the FACT that it is biased. You have no clue how many of the survey results are duplicates, or from anti-mormons or bored kids who just want to mess around. You have no possible way of getting a representative random sample or anything that can lay any claim to being a statistically reliable result.
If what you really wanted was good data, take down the web pages for the survey and announce that the results will be discarded as unreliable and do it right (which will likely mean hiring it out to Pew or a similar organization). Otherwise the only reason I can see for it is to use it as a propaganda tool against the church.