This is a guest post by Dr. Andrew Auman, who holds B.Sc. and M.Sc. degrees in physics from Utah State University, and a Ph.D. in astrodynamics from the University of Surrey. His research interests include geometric integration, geometric estimation, and attitude and orbital mechanics. He is also a semi-regular contributor to the blog Just An Average Mormon
I.
Recently, a survey by The Mormon Gender Issues Survey Group (TMGISG) has been floating around social media, and I have accepted an invitation to write this guest post as I wanted to weigh in on the discussion surrounding this survey and TMGISG’s approach to their research.
To those unfamiliar with the online dialogue surrounding the research being performed by TMGISG, many individuals are calling into question TMGISG’s research methodologies. The concern is that the wording used in the TMGISG’s survey shows a bias in support of the ordination of women to the priesthood in The Church of Jesus Christ of Latter-Day Saints—whose members are commonly referred to as Mormons. Individuals have also expressed concern in the ambiguity of some questions; e.g., what constitutes a “good Mormon”? And there are further concerns that not only do the answers provided on the survey not reflect the most commonly held views on the topics in question, but that at times the only answers provided contain views with which respondents cannot fully agree mixed in with those views to which they do ascribe. That this is the case is acknowledged in the survey. But as these biases, ambiguities, and false dichotomies could easily be removed by the inclusion of additional choices and/or the rewording of current answers, why was the effort not made?
The purpose of this post is to discuss research ethics, and apophasis is not my intent in the above expression of concerns being brought up in the dialogue elsewhere regarding TMGISG’s research practices. May the interested reader peruse the survey and what has been said on the matter for themselves, thoughtfully reflect, and then draw their own conclusions about its phraseology. I simply mention these issues as they are pertinent to the matter of ethics in research, and will be referred to herein without rehashing them for the sake of brevity—brevity being an admittedly relative term.
Before going any further, in the spirit of open, honest communication let me clear the air about who I am and am not. I am not, for instance, a social scientist. Nor do I claim to be fully apprised of the minutia of research methodologies in the social sciences. However, as a physicist I do perform research and am certainly acquainted with the scientific method. For those who resort to ad hominem in debate as in an ill-founded attempt to bolster their own arguments or attack the arguments of those of differing views, I preemptively state that I do have a Ph.D. I am also a Mormon, and I fully ascribe to the tenets of The Church of Jesus Christ of Latter-Day Saints. Among these tenets is the belief that God calls prophets in our day, that He reveals His will and eternal truths to His prophets, and that when His prophets speak in the official capacity of their calling what they speak represents His will and teachings (if you struggle to understand how a scientist can believe in such things, I suggest reading the post I wrote here). Among the truths revealed by prophets is that “The Lord has directed that only men will be ordained to offices in the priesthood.” Full disclosure: this view stands in contrast to the expressed views of members of TMGISG, who appear to support ordination of women to the priesthood.
II.
This post on research ethics is motivated by a comment made by Dr. Brent D. Beal, a TMGISG member, to J. Max Wilson’s post “What You Should Know About That Mormon Gender Issues Survey”. Pew’s 2011 research Mormons in America found that 90% of self-identifying Mormon women in America do not feel that dedicated women members of the LDS Church should be ordained to the priesthood, and in his comment Dr. Beal claims that Pew’s research “isn’t reliable for all sorts of reasons”. Dr. Beal states:
For example, Pew reports that 90% (or so) of women in the LDS church don’t see gender issues as a problem. At the same time, on any given Sunday, 2/3rds of LDS women (2/3rds of the women that are part of the 15+ million membership number) are absent, and we know that the majority of these women are inactive and/or no longer identify as LDS. We also know that gender issues are one of the primary reasons why women disaffiliate from the church. Putting all this together leads pretty quickly to a contradiction (and it’s a contradiction that can’t be easily resolved without more–and better–data).
Before addressing Dr. Beal’s comment and the reliability of TMGISG’s research methodologies, I first turn to the reliability of Pew’s research. On their website Pew states their code of ethics:
Independence, impartiality, open-mindedness and professional integrity are indispensable to the mission and success of the Pew Research Center. To promote and preserve these values, the center’s Code of Ethics includes the following policies:
Conflicts of Interest
Pew Research employees must avoid conflicts of interest or the appearance of conflicts of interest. They should never engage in any activity that might compromise or appear to compromise the center’s credibility or its reputation for independence or impartiality. All employees are required to seek prior approval from a supervisor before engaging in any activity that may be deemed a potential conflict of interest, including membership in groups, boards and associations that may call into question the center’s credibility or its reputation for impartiality.
Prohibitions on Electioneering
As a nonprofit 501(c)3 organization, Pew Research and all of its employees, when acting in their professional capacity, are prohibited from participating, directly or indirectly, in any political campaign activities on behalf of, or in opposition to, any candidate for public office. In addition, the center has a strict prohibition against partisan political activity by senior staff, even when they are acting in their individual capacity and on their personal time.
Integrity of Research
To ensure that the information we generate is of the greatest value to citizens and policymakers, the center is committed to conducting research in a manner that is impartial, open-minded and meets the highest standards of methodological integrity. We employ only those tools and methods of analysis that, in our professional judgment, are well suited to the research question at hand. We describe our findings and methods accurately and in sufficient detail to permit outsiders to evaluate the credibility of our results. We encourage inquiries about our research methods and practices, and attempt to answer requests for information promptly.
In harmony with the claims that Pew makes under the section “Integrity of Research” quoted above, Pew makes all of its research datasets available to interested researchers. These datasets can be found here generally, and here in regards to the Mormons in America study. Those who wish to download datasets are required to agree to a “Dataset Use Agreement”, but aside from the legal talk the purpose of this agreement is mainly to protect the privacy of research participants and assure that those using the data will never present the Pew Research Center as supporting any side of any debate. See here for more information.
Indeed, Pew is so concerned about maintaining the integrity of the research it performs that it doesn’t even refer to itself as a “think tank”, but rather as a “fact tank”.
To summarize, Pew demonstrates the epitome of ethical research by:
- Having no conflicts of interest. In fact, all Pew employees must even avoid the appearance of a conflict of interest, and seek approval from their supervisor before establishing personal affiliations which may taint Pew’s perceived impartiality.
- Having no participation in partisan politics or political campaigns. For senior staff, this even includes when they are acting in their individual capacity and on their personal time.
- Assuring that all research meets the highest standards of methodological integrity. It must be impartial.
- Assuring that only the most well suited tools and methods of analysis will be used. These tools and methods are evaluated for each and every study performed.
- Assuring that their findings and methods are presented in sufficient detail to permit outsiders to evaluate the credibility of their results. In fact, they encourage outside evaluation of their research.
- Making all datasets available to outsiders for analysis. These datasets are typically available either immediately after or six months after all reporting is completed, depending on the Pew center performing the research.
III.
In regards to their Mormons in America study called into question by Dr. Beal, Pew’s specific research methodologies for this study are described in “Section 5: Survey Methodology” of their report. In part, they state on pg. 70 of their report:
For both freshly sampled households and those in the recontact sample, the survey began with a screening interview. Respondents reached by landline were selected by randomly asking for the youngest adult male or female currently at home. Interviews on cell phones were conducted with the person who answered the phone, if that person was an adult 18 years of age or older. The screening interview consisted of a few short warm-up questions (about the respondent’s level of satisfaction with their community and their life), followed by a question about the respondent’s religious affiliation: “What is your present religion, if any? Are you Protestant, Roman Catholic, Mormon, Orthodox such as Greek or Russian Orthodox, Jewish, Muslim, Buddhist, Hindu, atheist, agnostic, something else, or nothing in particular?&rldquo; Those who described themselves as Mormons in response to this question were then administered the main survey, while the interview was discontinued for non-Mormons.
After identifying themselves as Mormons, qualified respondents were asked a separate question, “And is that The Church of Jesus Christ of Latter-day Saints, the Community of Christ, or some other Mormon church?” All self-identified Mormons were eligible for the survey, regardless of whether they identified themselves as part of the LDS Church. In practice, nearly all self-identified Mormons (99% in the current sample) describe themselves as part of the LDS Church.
To clarify the terms “freshly sampled” and “recontact sample”, understand that when Pew finds an individual who is willing to take a survey they keep that person’s contact information in a pool to draw from for future surveys. Having been contacted before for other surveys this group is known as the “recontact sample”, whereas those who have not previously participated in a Pew research survey make up the “freshly sampled” group. Respondents from both groups were included in the Mormons in America survey.
In these two paragraphs from Pew’s report there are three key elements to their research methodologies which ought to be explicitly highlighted. First, they began with a “screening interview”. If in the screening interview it was discovered that the respondent qualified for the “main” survey—in this case, qualification being self-identifying as Mormon—they were then given the actual survey used for this study. The interview was discontinued for all others. This step is critical; it acts as a filter which only permits involvement of those whose views are pertinent to the study at hand. Note that during this screening interview no indication of the study being performed was given. They even used an open-ended question as the main interview qualifier; they did not, for example, ask “Do you self-identify as being Mormon?” Hence, although a respondent may have been able ascertain that the study in question had to do with religion, they would not have been able to know how they needed to respond in order to be involved. Such an approach prevents those whose strong opinions may lead them to knowingly participate in a study which doesn’t pertain to them—thereby tainting the results. Consider the caution and foresight demonstrated by Pew in this approach.
The second key element I want to highlight is that the qualification for participation was specifically whether or not the individual described themselves as being Mormon. Why is this important? The answer is twofold: i. it doesn’t put any weight on the respondent’s faithfulness to their religion or its tenets, e.g., a respondent’s church attendance was not considered relevant; and ii. it prevents the views of those who do not self-identify as being Mormon from being presented as the views of those who do self-identify as being Mormon—whether or not their names are on the records of any given church.
Note that there may be some individuals who do self-identify as being Mormon, but who may not be members of The Church of Jesus Christ of Latter-Day Saints. This brings me to the third key element: qualifying respondents were asked to which denomination they belonged, but they still had no way of knowing whether this study pertained to any single church or group within the set of those who self-identify as Mormon. Those who did self-identify as Mormon but who were not members of The Church of Jesus Christ of Latter-Day Saints were still included in the survey for the report—recall that the title of the study is Mormons in America not Members of The Church of Jesus Christ of Latter-Day Saints in America—but Pew reports that these respondents only comprised 1% of those given the main survey.
IV.
Having discussed, in sections II and III respectively, Pew’s code of ethics and research methodologies generally and in regards to their Mormons in America study, I would like to now address Dr. Beal’s claim that Pew’s research is unreliable. As quoted in section II, Dr. Beal attempts to substantiate his claim by arguing:
- Pew reports that 90% of self-identifying Mormon women in America do not feel that dedicated women members of the LDS Church should be ordained to the priesthood .
- Two-thirds of women whose names are on the records of the Church do not regularly attend Sunday meetings of the Church.
- Women whose names are on the records of the Church but who do not regularly attend the Church’s Sunday meetings: are inactive, do not identify as being LDS, or are inactive and do not identify as being LDS.
- Gender issues are one of the primary reasons women disaffiliated from the Church.
- These statements constitute a contradiction.
- Hence, Pew’s research is unreliable.
I acknowledge that I have taken some poetic license in this expression of Dr. Beal’s argument in order to clarify it, but did so without changing or misrepresenting that argument. Please see the exact excerpt of his quote in section II, or his full quote in the comments section of the aforementioned J. Max Wilson post.
Now, if Pew’s screening interview had only allowed for those who both self-identified as being Mormon and who attended The Church of Jesus Christ of Latter-Day Saints regularly to be eligible to take the main survey; and if one assumed the following statements as premises: “two-thirds of women whose names are on the records of the Church do not regularly attend church”, “women whose names are on the records of the Church and who do not regularly attend church have disaffiliated from the Church”, and “women who have disaffiliated from the Church have done so because they feel that dedicated women members of the Church should be ordained to the priesthood”; then one could logically draw the conclusion “two-thirds of women whose names are on the records of the Church feel that dedicated women members of the Church should be ordained to the priesthood“. If this statement were true, it would contradict the results found in Pew’s study if Pew’s study had also been worldwide—one would have to define a premise giving percentages of women whose names are on the records of the Church and who live in America in order to correlate this conclusion to Pew’s actual survey results.
But that’s not Dr. Beal’s argument, nor is that Pew’s study.
Regarding Dr. Beal’s actual argument, a simple question: with the qualifier for the Pew study only being self-identification as Mormon—having nothing whatsoever to do with Church attendance—then how do statements concerning Church attendance statistics have any logical bearing on the reliability of the findings of Pew’s research?
Answer: they don’t. At all.
And that, in and of itself, completely refutes Dr. Beal’s argument against the reliability of Pew’s research.
But while I’m at it, I would like to address Dr. Beal’s premises. Setting the validity of his original premises aside for a moment, these premises are still too ambiguous to allow for the development of any pertinent logically sound conclusion. To shore up his premises, Dr. Beal would have to give percentages for his classifications of the three groups of women whose names are on the records of the Church and who do not regularly attend, as well as the percentage of women who disaffiliate from the Church because of gender issues. At this point one could logically make conclusions about the percentage of women whose names are on the records of the Church and who have disaffiliated from the Church due to gender issues. However, this logical framework could still not be used to quantify the validity of Pew’s research. Furthermore, even if such a framework is logically sound, it is not guaranteed to be accurate. To be accurate, the validity of Dr. Beal’s premises would need to be established.
So, even though this has no bearing on the reliability of Pew’s research, are Dr. Beal’s stated premises even accurate? Well, since he doesn’t substantiate the claims he made in his premises how can we assume they are valid? For the statement that two-thirds of women whose names are on the records of the Church do not regularly attend Sunday meetings of the Church, one could look at the Church’s statistical report, take the number of congregations, make some guesses as to the average number in a congregation attending weekly, multiply the two, and then see what percentage the result is of the total listed membership of the Church. But, that involves some guessing. And although one could try to make an educated guess, it would still be a guess and it would be very hard to measure the error associated with it. The Church Handbook of Instructions, Book 2, Section 21.1.35 “Research Studies in the Church” states, “The only authorized research agency of the Church is the Research Information Division of the Correlation Department. ” So, if one is going to make claims about worldwide Church attendance, they are going to have to get that information from Research Information Division. To the best of my knowledge they don’t give that information out. Yet, whether or not one agrees with the Research Information Division policies, their non-disclosure of those numbers doesn’t grant an individual or group the right to guess what those numbers are and then claim that guess as being accurate.
What about Dr. Beal’s other premises? I would not doubt that the majority of the women: i. with whom Dr. Beal associates, ii. and whose names are on the records of the Church, iii. and who have disaffiliated from the Church, iv. have done so primarily because of gender issues. However, if this is true I cannot see how Dr. Beal could make the extrapolation to the general set of women whose names are on the records of the Church and who have disaffiliated from the Church—at least, not without substantiating such an extrapolation with uncompromised data. I further posit that, world-wide, gender issues are a relatively small contribution to the disaffiliation of women from the Church. I have not performed a study to verify this hypothesis, but the statement from Pew’s Mormons in America study referenced earlier leads me to place more confidence in my hypothesis than in unsubstantiated claims to the contrary.
V.
Moving on to TMGISG as a whole, I question the integrity of this group’s research generally.
Consider the lack of reliability in the web survey floating around. First off, in simply making survey respondents feel that there is a bias—even if that was unintentional—TMGISG has tainted their results. Why? Well, because it’s hard to prevent a respondent from reacting one way or another to a perceived bias, especially on such an emotionally driven topic. For example, many of my Mormon acquaintances feel put on the defensive by the phraseology used in the survey. This has led to individuals choosing to stop midway through the survey, or not even take it at all, because they feel that TMGISG’s perceived bias will lead them to misrepresent these respondents’ opinions on the matter. How can a survey that alienates—to the point of non-participation—a non-negligible portion of the population being studied ever accurately reflect the views of that population? Furthermore, with those Mormons who do feel that women should be ordained to the priesthood not likely taking offence to the phraseology used in the survey, how can the survey results not be biased towards their view?
One may try to argue that some respondents are reading too much into the phraseology of the survey because of the socio-political views of those behind the survey. However, I don’t think such a claim is founded. Furthermore, even if that claim was justified, the aforementioned consequences on the reliability of the results of a survey seeking to sample the entire body of the Church would remain. This also speaks to the importance of a social science research group not giving any indication of subscribing to any socio-political agenda.
Also, how can a survey whose respondents do not feel they have a response option to a posed question which reflects their viewpoint ever accurately portray the viewpoints of the respondents?
How can a survey which employs no screening method prevent those with strong opinions from intentionally misleadingly presenting their views as those of a group to which they don’t belong?
How can a survey which allows anyone to take it as many times as they want not produce biased results? Don’t tell me responses from the same individual can be identified and filtered out. They can’t.
If TMGISG feels that Pew’s restriction of their study to just Mormons in America is insufficient, what is TMGISG doing to reach the world-wide membership? Has TMGISG translated their survey into the languages of all those who do not speak English?
How does this web survey reflect the viewpoints of those without internet access?
How does this web survey, which is being promoted through social media outlets, account for the discrepancies between age and social media usage? I.e., how do they prevent the results from being biased towards the viewpoints of those groups who tend to be more predominantly active on social media?
If, judging from Mr. Beal’s comment, TMGISG is seeking the viewpoints of Mormons on gender issues, why would they seek to include the viewpoints of those who do not self-identify as being Mormon and then present their views as being those of Mormons? If TMGISG wants to study the viewpoints on gender issues of those who do not identify as being Mormon but whose names are on the records of the Church, by all means they should go ahead. But if, in order to get the conclusion they want from their current survey, they have to present the views of those who do not self-identify as being Mormon as the views of Mormons—irrespective of whether or not their names are on the records of the Church—then they have gone very far afield of what we call science. (As a side note: if one were to correlate individuals whose names are on the records of the Church but who no longer self-identify as being Mormon with opposing views to the Church on gender issues, one would still be a long, long way from showing that those differing views were what caused these individuals to no longer self-identify as being Mormon.)
Taking these things into consideration, what will be the magnitude of error in the results produced from TMGISG’s web survey? Is it even possible for this error to be quantified?
Yes, I know that TMGISG’s web survey has been approved by the Institutional Review Boards (IRBs) at The University of Tampa and Georgia Southern University. But please understand, dear internet, that the only thing IRBs concern themselves with is the ethical treatment of human subjects in a research project (even though I’m a physicist, I’ve gone through training on how to meet IRB requirements because—as a scientist—I was curious about the research ethics surrounding the use of human subjects). That this survey has been approved by these IRBs gives us confidence that the privacy of participants will be maintained, but not that the research methodologies are scientifically sound.
VI.
Given all of the issues with their web survey, I’m going to give TMGISG the benefit of the doubt and assume that they recognize the utter lack of its scientific value. So why then, did they create it? The only hypothesis I can come up with is that they intended this survey to become a social media campaign as a means of drawing attention to their cause. That would explain why they are pushing the survey through social media outlets. It is also in harmony with the general tactics of those who seek for women’s ordination: to encourage any form of dialogue in order to draw attention to their cause, inflate its perceived importance to the average Mormon, and pressure the Church to change divinely decreed doctrines.
Now, TMGISG states that aside from the web survey, a second survey will be conducted using a “random, nationally representative sample of members of The Church of Jesus Christ of Latter-day Saints. ” Unfortunately, no more details concerning this survey are provided. Given the lack of merit of the web survey, I imagine that TMGISG will be spending its Kickstarter funds on a database of phone numbers of individuals to survey. If so, this sounds a lot like Pew’s research—only without the open, impartial objectivity they demonstrated. But by exhausting their research integrity capital in their purchase of a social media campaign, TMGISG have left themselves with nothing more than their Kickstarter revenue for their second survey.
And that’s too bad, because if TMGISG’s intent with their web-survey social media campaign is to help instigate and contribute to a conversation on gender issues in the Church, they are going about it the wrong way. For an excellent example on how to constructively contribute to this dialogue, see Sharon Eubank’s 2014 FairMormon address “This is a Women’s Church”.
Furthermore, regarding whether TMGISG will make the data from both of their surveys in their study available for external evaluation, they state:
Maybe… Why do you ask? More seriously, if you want access to the data you’ll need to contact one of the members of the Mormon Gender Survey Group and explain how you plan to use it. The group will then discuss it and make a decision.
This raises all sorts of red flags. One of the most important steps of the scientific method is repeatability. If one’s research cannot withstand the scrutiny of other scientists, then it is not scientifically sound. But even more alarming than whether or not TMGISG shares their data, is their statement “Maybe… Why do you ask? ” and the case-by-case need for a group discussion determining whether they will share their data. I cannot but infer from this that their willingness to share their data is dependent upon the motives of those requesting it. An unwillingness to share scientific data with those who disagree with one’s hypothesis—and who may discredit one’s analysis—speaks not of a desire to seek the truth but rather a desire to protect an agenda.
To TMGISG: can you not see why you are being called out on your research ethics, methodologies, and policies? Your motivation shouldn’t be “primarily academic” as Dr. Beal states; it should just be academic. And that’s the main issue here—that there appears to be a non-academic agenda motivating your research. Given, we are all human. We all have opinions. And sometimes our opinions can, naturally, influence our scientific research. That’s something all scientists need to both recognize and keep in mind. But I think that the concern you see being expressed here and elsewhere is that your group is letting its opinion get in the way of the science; your group is coming across as setting out to prove your hypothesis to be true, instead of setting out to test whether your hypothesis is true.
It is for these reasons that the codes of ethics, rigorous research methodologies, and dataset policies espoused by respectable research groups, such as Pew, are so important. And it is because of the well-established research integrity demonstrated by Pew, that you only do yourselves a disservice by calling it into question.
Therefore, I encourage and caution you to mark your path forward according to the following five cairns placed by Pew:
- Completely remove yourselves from any actual or perceived conflicts of interest. This includes your individual participation in and affiliation with any group which may call into question your impartiality.
- Employ only those tools and methods of analysis which are best suited for testing the validity of your hypothesis. Read: ditch the web survey; it is of no scientific worth. And be very, very, very careful about how you set up and conduct your second survey.
- Describe your findings and methods accurately and in sufficient detail to permit outsiders to evaluate the credibility of your results. Repeatability, repeatability, repeatability .
- Make your raw, unaltered datasets publicly available to any researcher desiring to evaluate the credibility of your results. You may have spent much time and money collecting your data, but unless you are willing to share it we have no way of knowing whether you are being honest and accurate in the findings you report.
- Explicitly establish a code of ethics and make it publicly available. This code of ethics should not only pertain to the treatment of those participating in your surveys, but should outline how you plan on handling conflicts of interest of researchers involved, maintaining your impartiality, and assuring only sound research tools and methodologies are employed.
Otherwise, TMGISG, you might consider yourselves to be on the right side of history, but you will be on the wrong side of science.
New Post: Research Integrity in the Social Sciences: 5 Things The Mormon Gender Issues Survey Gro… http://t.co/BG6LGuqiFD #LDS #Mormon
TheMillennialStar: Research Integrity in the Social Sciences: 5 Things The Mormon Gender Issues Survey http://t.co/q93W85zm21 #lds #mormon
Very,very well stated.
I do have one kickback, though. In your second cairn you state that TMGISG should ditch the web survey. While I agree that the web survey as it exists now is a poorly worded, there is nothing inherently wrong with the methodology that requires it to be entirely discarded. All survey methodologies (e.g., phone, web, door-to-door) will have some demographic bias. This is not a sufficient reason to reject those methodologies. Instead, what is important is that when results of these studies are published that there is also an accurate description of the biases and limitations imposed by the selection methodology.
For example, I use Amazon Mechanical Turk to conduct social science experiments. Amazon Mechanical Turk contains respondents that are largely American and skew female. This does not mean I can’t use Amazon Mechanical Turk to conduct my research, it only means that I must report this populations nature in my limitations section. Even Pew has a biased demographic, i.e., participants who are willing to take surveys. True, no one is quite sure what this bias entails, but it does exist.
I’m impressed that someone (Dr. Andrew Auman) has spent this much time analyzing a quick response I typed in 5 minutes. I’m going to spend another 10 minutes responding (because that’s about all this post deserves, honestly).
First, a few points (to summarize):
1) The church claims 15+ members (this claim in made often enough by leadership and member alike that this shouldn’t be controversial),
2) The Pew number that gets thrown around suggests that 90% of church “members” don’t see any gender problems,
3) There are “problems” with this number (or at least I think so, and so do most of the other social science researchers involved in this project), and one of those problems is how they define “member,”
4) I suspect (and so many of the researchers involved in the project), that if “member” is defined differently (say, for example, in the same way the church ITSELF defines it, in a way that includes 15+ million people), the picture is going to look pretty different. I don’t know that this is the case, however, thus the survey; that’s why we need more data on this issue.
5) Although I didn’t say this explicitly in my email response, the Pew data is based on a small number of questions that didn’t take into account a number of nuances that we think are important (we’ll talk more about that in some of the academic publications that result from the survey).
Here’s a few comments and questions:
1) When someone says upfront that they don’t know what they’re talking about, we should believe them (e.g. “Before going any further, in the spirit of open, honest communication let me clear the air about who I am and am not. I am not, for instance, a social scientist. Nor do I claim to be fully apprised of the minutia of research methodologies in the social sciences.”).
2) A survey is not a poll. The argument that folks didn’t get to tell us what they wanted to tell us on our survey (and that it is therefore biased) is silly (and is only made by folks that don’t understand survey research).
3) As a group we spent months worrying about things like sampling and representativeness. Our results will be peer reviewed. We’ll see how it goes. I’ll make a prediction right now, though. Not only is our data valuable, based on the overwhelming (and unprecedented) response to our survey, I’ll make this claim: Right now, our group has more detailed and more nuanced scientifically-rigorous data about what Mormons think about gender issues and gender roles in the LDS church than anyone else. I sincerely thank all those that took our survey in good faith. Our methods and findings will be open to public review, and our data will eventually be made public.
4) What on earth are church members so afraid of here? We’re talking about a church in which 2/3rds of the members it claims aren’t around on a given Sunday. At minimum, wouldn’t it be nice to know something about what these people think (in addition to what active members think)? Wouldn’t it be useful to know if gender issues play a role here?
5) We got together, we raised some money, and we set out to ask some interesting questions. Dr. Auman is welcome to do the same. If he does, I won’t question his integrity. I won’t behave like a 6-year-old on the internet. Dr. Auman should know better, even if he isn’t a social scientist.
For those that would like to leave feedback on the survey, please use this link: http://goo.gl/forms/xy2HExFTYa.
Brent
Brent, it says a lot about you that you are incapable of taking criticism of your work without lashing out with personal attacks. You should be ashamed.
Brent, could you comment on the first “cairn”? It was:
“Completely remove yourselves from any actual or perceived conflicts of interest. This includes your individual participation in and affiliation with any group which may call into question your impartiality.”
A lot of us see your group as having conflicted interests. When this survey was announced months ago I was hoping, even assuming, that most of the researchers wouldn’t be Mormons, and that none would be associated with activist groups. So the reality looks pretty bad, especially when compared to Pew. Do you plan on doing something to fix this?
“What on earth are church members so afraid of here?”
One answer to this is in your quote below.
“Our methods and findings will be open to public review, and our data will eventually be made public.”
What we expect — based upon past experience — is that your methods and findings will be made available, generate news headlines, and be promoted to damage the Church. Then the data will eventually be made public, and when the examined data is shown to have flaws or not support the findings the narrative in the media will already be set.
For me at least, and I think many others, you could buy yourself a lot of good will by agreeing to release methods, findings, and all your data simultaneously.
“A survey is not a poll.” This one needs a bit of explanation from someone. My attempts at finding out the difference between a poll and a survey have so far described the difference as being one of degree rather than kind, a poll being explained as a simple, quick, single-question survey.
“and our data will eventually be made public.” How kind of you to change your mind…certainly much better than “Maybe, why do you ask?”…but seriously, did no one learn anything from the quagmire climate science got itself into by being so darn reticent over data?
4) I suspect (and so many of the researchers involved in the project), that if “member” is defined differently (say, for example, in the same way the church ITSELF defines it, in a way that includes 15+ million people), the picture is going to look pretty different. I don’t know that this is the case, however, thus the survey; that’s why we need more data on this issue.
If it’s misleading for the Church to claim all 15 million members-of-record as actual members when describing the size of the Church, isn’t it also misleading for you to include all such people as members when you’re trying to describe the opinion of the Church–particularly when addressing an audience that doesn’t understand the vast difference in terms of commitment and lifestyle between practicing and non-practicing Mormons?
And, by the way–are you planning to also publish the fact that (as of today) at least ninety-three of your survey’s participants reported that your survey was “manipulative”, ninety-six reported that the survey did not accurately reflect their points of view, and one hundred four of your survey’s participants disagreed with the suggestion that your survey was “unbiased”?
2) A survey is not a poll. The argument that folks didn’t get to tell us what they wanted to tell us on our survey (and that it is therefore biased) is silly (and is only made by folks that don’t understand survey research).
Silly people, thinking that academics running a survey actually wanted to know what those people thought.
Do you know Jonathan Gruber, by any chance?
At minimum, wouldn’t it be nice to know something about what these people think (in addition to what active members think)?
I’m going to go out on a limb and suggest that people who tend not to go to a church, tend not to believe some or part of that Church’s theology.
Now, where’s my Nobel Prize?
Brent,
This is a very unprofessional response to this post. I am very disappointed. This throws your research into question for me personally. A lot of your comments were great and helpful, but a few things you said really worried me.
1) “I’m going to spend another 10 minutes responding (because that’s about all this post deserves, honestly.)”
Not the response I would expect from a professional researcher. This is a very condescending response to a well written and honest post from Dr. Auman. Are the counter-opinions of your peers not worth your time? The peer-reviewing process will be a long one with this attitude! Not worth your time, because it’s not in favor of your methodology. This is a disappointing way to begin your response, and not the way to begin your path to scientific credibility.
2) “When someone says upfront that they don’t know what they’re talking about, we should believe them.”
Ouch! Is that what Dr. Auman said? I think the rest of his article says otherwise! Dr. Auman didn’t say that he doesn’t know what he’s talking about. Read his next line after your quote. He’s talking about research and the scientific method. He has immense research experience and appears to be well-read on the subject of social science, even for one who is not a professional or a graduate in the field. He is definitely qualified for discussion on this topic. Questioning Dr. Auman’s intelligence is again, really unprofessional. Clearly, he is not ignorant about what is important in research.
-“The argument that folks didn’t get to tell us what they wanted to tell us on our survey (and that it is therefore biased) is silly (and is only made by folks that don’t understand survey research)”
Why is that silly? Is this survey being fielded to social scientists who do understand survey research? Guess what? They don’t understand survey research. Unforunately almost EVERYONE taking your survey doesn’t understand survey research. Is that a prerequisite? They shouldn’t HAVE to understand survey research. If they bail on your survey because they feel misrepresented, that’s a problem! We (the average human being who don’t have doctorates in social science or research methods) take surveys because we want our opinion to be made known, and we want OUR personal opinon represented in the final data. No one wants words put in their mouth, especially on REALLY strong social issues like this! If there is any time to have an impartial, unbiased survey, it’s now! You take that group of people who don’t feel adequately represented (which is very real and affects your results) and throw them out the window because their feelings are “silly” to you. Do you even take the “silly” people into account who quit halfway through? Do you ask them why they quit taking the survey? That’s a legimate concern and I hope you don’t toss it aside like you did here. It’s a concern that’s been voiced by active Mormons taking the survey (a.k.a. your demographic). (Just hop on a few other blog posts regarding your survey and you’ll see that it’s not a negligible group). Does your survey allow for ALL opinons and viewpoints of the demographic to be represented? Some don’t feel so.
-“What on earth are church members so afraid of here?”
After being so picky on your semantics you haven’t defined members here, but I think you’re labeling the active church members who are opposing your methods. They are afraid of being misrepresented. Even bad research gets media attention. Mormon’s are open to gathering facts, the problem is that they are worried that you aren’t actually gathering facts. It’s the methodology and the intent that worries Mormons. Being in the minority has never bothered the faithful, active members of Church of Jesus Christ of Latter-day Saints. Maybe your perception of fear is a moral victory for you, I don’t know.
-“I won’t behave like a 6-year-old on the internet. Dr. Auman should know better, even if he isn’t a social scientist.”
Wait, you won’t behave like a six-year old? What do six-year olds do? This is just flat out name-calling! This is a textbook example of ad hominem debate that Dr. Auman mentioned (and wanted to avoid.) Six-year olds will put each other down, to increase their own credibility.
In the future, please take more than 10 minutes to respond to a well-constructed scientific critique of your research. I was on the fence with this issue, but your response gained you at least one opponent to your methods and your intent. I am not against what you are doing, but I just want your responses to be clear and professional if you want to gain scientific credibility! Take all responses to your work seriously. Word them thoughtfully, and don’t attack others intelligence, experience or beliefs. If someone writes an article that is negative, poorly worded, or even flat out incorrect, response tactfully and with dignity. Turn the other cheek, as a Christian would say. Be the bigger man.
I work with many different academic researchers on a daily basis. I cannot imagine any of them responding to a critic as you have done here.
Setting science aside for me personally… The church has long stated that votes, surveys, polls, etc do not define the doctrine of the church. They cannot define truth. Only God can. Even the majority can be wrong. I know you have proponents of ordination and gender issues such as John Dehlin and others on your side. Proponents of ordination feel if they can get a majority or a large percentage on their side, the church will change. It’s true that in the world, a majority or a vocal minority wins. You can fight and scrap to show that large percentages of people hold certain beliefs. Even if that is true, it doesn’t change what truth really is. I have a testimony of the prophets and the gospel of Jesus Christ. This is a battle for truth, but no matter what poll or vote says, the truth is known to me and to many other Latter-day Saints. We’re not afraid, but we will defend what we know as truth.
I took more than 10 minutes here. I felt it was worth my time.
I’m with Fraggle – I’m confused, Brent, about what you said about polls and surveys. This is representative of the definition I found online, “Polls and surveys are two different methods that can be used to obtain information. They are usually utilized by researchers to gauge opinions and receive feedback. Essentially, the main difference between the two is the fact that a poll is small, simple and quick. Whereas, a survey can be slightly long, open-ended and time-consuming”.
Are you saying that the people who were frustrated by the questions would have preferred to have taken a poll which is smaller, simpler, and quicker? I’m confused.
Also, please address the bias issue that has been brought up several times. Bias can be found by doing something as easy as looking on Facebook. I’ve noticed several comments made by your wife (who is also in the TMGISG) on John Dehlin’s page. This includes comments about an event on Temple Square that is sponsored by FMH, OW, Young Mormon Feminists and Queer Mormons. I’m trying to give you some credit by wondering if you are so immersed in gender issues that perhaps you aren’t aware of your bias and advocacy?
I find interesting that the Pew survey puts forth that on any given Sunday, 2/3 of LDS females are absent from their Sunday meetings, & hence are “inactive”. As a mother who raised 4 children, there were many Sundays in a row when I was absent from my Sunday meetings, because I was home with a sick child. My children seldom had the foresight or courtesy to get sick all together, so I would only have to miss one Sunday to care for all of them. As a result, even tho I was fully “active”, I missed a lot of Sundays because I was a mother, caring for my children. I suspect many of the rest of the “2/3” of LDS women who are absent each Sunday from their meetings are doing the same.
Dear Dr.Brent Beal
Thanks for taking 10 minutes of your time to address some of the perceived issues with your research. I am sure this will be to be a good investment of your time.
As an experienced social researcher, I am sure that you would also agree that one should never consider our research perfect or complete and that there is always room for improvement. Some of the most important lessons are sometimes learned from our critics, even those in different areas than ours as they can provide never before considered perspectives. This is for me the true open spirit of research.
Historically, I think we would also agree that the Social Sciences would still be in the Dark Ages without the Scientific method. Therefore, the Social Sciences are naturally indebted to the Natural Sciences (pardon the pun).In addition, scientific criticism from someone with a strong Natural Sciences background (like Dr.Auman) can only add more robustness to your research. Openness to scrutiny is always a sign of good research. I am also sure that you are doing the best you can and know with the resources available to you.
However, I think you are missing the entire point of this discussion around your research. The main issue being discussed is the CONSTRUCT VALIDITY within the research instrument (survey). And as you know this is one of the most difficult aspects to get right in Social Research. My only question for you is – what have you done to address this issue?
Most of the time this is the question that a lot of researchers (in Social or Natural Sciences) seem to forget or completely ignore. And from my experience, this happens mostly for one of two reasons: they do not have a good understanding on how to conduct research or they want to steer their research towards a pre-determined conclusion.
So, my question again is – what measures have you put in place to address the CONSTRUCT VALIDITY issue? I am also sure that this would be an easy question to answer as your group spent months putting this study together and as you say the data is “scientifically-rigorous”.
Just to follow up my comment about bias. Brent, would you think that Mormon Women Stand would have any conflict of interest, or be seen to have a conflict of interest, in setting up a study to get more accurate results than the Pew Forum on gender issues? Would it be fear that would keep you from embracing such a study? I admit my bias – I agree with most of the goals of MWS. I would not, however, feel comfortable with them assembling a group of scholars, who mostly all support their views and are affiliated with them in some way, and then have them claim that they are a serious, world class group of scholars conducting a survey. No matter their professional credentials, how could they not be seen to have a conflict of interest? Would the Pew Forum, with their standards, ever assemble a group of MWS supporters to conduct such a survey? I’m hoping that by turning the tables that you will be able to recognize bias and conflict of interest.
The Pew Forum is a respected group. No survey is perfect, obviously. I think it was unwise, though, to compare your group to them if you want to be taken seriously.
Research Ethics & the Mormon Gender Issues Survey Group’s research practices:
http://t.co/dq4qrkpRB8
#lds #Mormon #survey
marivene, the “2/3rds women are inactive” is a statistic that was both used out of context and misapplied by someone in this topic (actually on another pos, tI think.)
As Meg pointed out on that other post, that is a pretty good estimate of the world-wide lds inactivity rate. However, the proper context is that that number is actually better (ie, less inactives) than most other churches/religions, -and- our male inactivity rate is higher than our female inactivity rate.
Whoever used it was apparently trying to imply that women like the lds church less because of sexism. However, the lower inactivity rate of women compared to men would seem to imply the opposite, that women like the church more than men do.
More women join the church as converts than men, and after joining, women stay in the church at a higher rate than men.
I think it’s obvious that those behind this survey have an agenda. Because… The stats are already available as to what happens when churches open their ecclesiastical leadership roles to women and otherwise feminize or de-masculinize their churches: their membership dwindles.
One female-led congregation (i posted the link somewhere, but lost it, i think it was a youtube video), did an experiment of shifting the selection of hymns to more masculine ones, and found that even that helped re-activate and retain male participation in their congregation.
Whether they admit it or not, the key people behind the kate kelly movement are trying to destroy the church. Most of those key people have actually admitted they don’t believe the founding truth claims of the chirch. i think many of their followers are merely duped, as they are PC-brainwashed. But actual real churches have had real life experience in feminizing their churches, and it has always been followed by decline.
Hi Andrew, I appreciate your post. It’s thoughtful and raises some good points. As someone whose bias is admittedly more toward the side of increased inclusion of women in pastoral, decision-making, and administrative roles in the church, I didn’t have any trouble with the survey when I took it, but I understand that is likely my own perspective talking.
My main concern is that you might be putting the cart before the horse here. I feel like you’re criticizing the survey before you’ve seen the conclusions or even have full information about the purpose/research question the researchers are asking. It’s pretty common practice for survey research NOT to disclose exactly what they’re looking for in order to not bias the results. So my thought is that rigorous critique of the survey should probably wait until after the results and analysis are in.
Having said that, a couple of thoughts in response to your post…
Regarding Dr. Beal’s actual argument, a simple question: with the qualifier for the Pew study only being self-identification as Mormon—having nothing whatsoever to do with Church attendance—then how do statements concerning Church attendance statistics have any logical bearing on the reliability of the findings of Pew’s research?
Answer: they don’t. At all.
But what if this one of the questions that the survey is trying to uncover? There are many members of record who no longer self-identify as Mormon and thus would not have been included in Pew’s sample. One of the questions the survey is trying to uncover is, of those who don’t attend church regularly, who has concerns about gender inequality? The researchers hypothesize that that number is high (I’m not so sure it’s all that high, myself), but without data it’s all speculation. It’s an interesting question and one that isn’t answerable by Pew’s data.
Now, TMGISG states that aside from the web survey, a second survey will be conducted using a “random, nationally representative sample of members of The Church of Jesus Christ of Latter-day Saints. ” Unfortunately, no more details concerning this survey are provided. Given the lack of merit of the web survey, I imagine that TMGISG will be spending its Kickstarter funds on a database of phone numbers of individuals to survey. If so, this sounds a lot like Pew’s research—only without the open, impartial objectivity they demonstrated. But by exhausting their research integrity capital in their purchase of a social media campaign, TMGISG have left themselves with nothing more than their Kickstarter revenue for their second survey.
My understanding is that they’ve actually already done this. They did this first, and the snowball sample from the web survey is going to be compared to the results to see if it is, in fact, a representative sample.
There’s a post here that argues that the survey’s methodology is fairly standard and not at all alarming. I’d be interested in your thoughts.
For context on the 33% claim for active participation, it is instructive to look at the 50 year statistics for the Church of England, which opened its clergy to women 50 years ago. I believe their current statistics on attendance are roughly 1%.
And as Marivene indicated, there are circumstances that could be imagined to disproportionately affect women (e.g., sick kids). Yet I have rarely been in a congregation where there were more men than women, and the one time my congregation fit that description, it was a university congregation where two male apartment complexes were combined with only one female apartment complex.
Those advocating for ordination of women are presuming that this single change (ordaining women) would allow reversal of the problems malcontents see in the religion.
As for conducting surveys, my research in this area was published in my Master’s Thesis (republished on Amazon – search for Margaret C. Stout and strategic planning). I also regularly evaluate the Defense Equal Opportunity Climate Survey (DEOCS) conducted annually, to see what initiatives are indicated for our unit. So I have some familiarity with surveys dealing with “soft” matters. While I did participate in the MGIS, I did find that both the tool and team conducting it fail to reflect professional standards.
Katie – how would you address the bias and conflict of interest problem? I’m anxious for anyone who doesn’t have a problem with this survey to address this.
I think conflict of interest and bias are being conflated here. Every researcher comes to the table with bias. There’s no getting around it. It’s up to researchers to take precautions to be as vigilant as they can against their own biases, but of course they can’t ever fully get away from it. The peer review process helps with this, as does a thoughtful and intentional study design–which I believe this has undergone. I haven’t been involved in the development of this study (and I’m not an academic anyway), but I don’t see any reason to conclude that the researchers haven’t taken steps to be vigilant about their acknowledged biases. We don’t have the results of their work yet, so it’s premature to cast judgment about it
Conflict of interest is different and usually refers to issues of financial remuneration or other intangible benefits, such as promotions at work. For example, there is serious concern about the commercialization of clinical research, so if you are conducting a drug trial that is being paid for by the drug company that manufactures the product, for example, you must disclose that. By the way, such a thing doesn’t render your research moot, it doesn’t mean that your findings are necessarily wrong, nor does it mean that you shouldn’t/can’t do the study, but it’s something that is watched closely. I don’t see any evidence of conflict of interest here. No one’s getting rich or taking intangible spiffs under the table for doing a study on gender roles in the Mormon church.
tl;dr: conflict of interest and bias aren’t the same things. There is no evidence that there is any conflict of interest involved in the study. Bias exists, but bias exists in every study, because human beings conduct studies and you can’t get around it. Professionals are trained to manage their biases, and those conducting the study are professionals. The results aren’t here yet, so there’s no reason to assume that the researchers were unable to check their own biases to a satisfactory degree.
Katie – The Pew Forum standards are to, “Completely remove yourselves from any actual or perceived conflicts of interest. This includes your individual participation in and affiliation with any group which may call into question your impartiality.”
I agree with you that bias and conflict of interest aren’t the same. I’ve acknowledged my own bias in a previous comment. It’s challenging to not see a conflict of interest in the members of this group, though. No one commenting on conflict of interest has mentioned concerns about money. Impartiality is the concern. Through simple social media investigation, affiliation with groups that would call into question impartiality is abundantly evident. This group is trying to put their results in the same caliber as the Pew findings. It’s very difficult to imagine the Pew Forum putting together such a non-diverse group who had such strong views on the subject being studied. Take a look at some Facebook pages to see some of the views and groups to which the individuals are associated. Of course everyone can have their views, but a reputable survey would have a diverse, impartial group who isn’t known for advocacy.
Also, people with different biases than yours did not find that the survey answer options were varied enough for them to accurately respond. However the findings are used, this problem takes away from the accuracy of any use.
Again, you’re conflating conflict of interest and bias. And Pew, while an excellent organization, doesn’t set the only standard of research ethics. You can have an ethical and effective study while coming into the question with a definite perspective or set of assumptions, as long as you are aware of your bias and take steps to mitigate it. We actually don’t know the research question(s) they’ve posed, so we don’t know if the way the questions were written was biased or not. All we know is that some participants of the study didn’t feel that a few of the questions could be answered in ways that perfectly described their perspective, *something that is actually not all that uncommon in survey research, nor is it necessarily a red flag.* We’ll know more when the results and analysis are published.
I’m still looking for someone who found this study unbiased to explain how this group can be taken seriously with such an obvious conflict of interest.
You’re only going to hear what you want to hear, eh, Lisa? Why did you even engage with me if you were going to ask a question and then completely ignore my response? Sheesh.
Hi Katie,
On the splash page after one completed the Gender Issues survey, there was an assertion that surveys that have been forwarded multiple times achieve a representative cross-section of the targeted population. There was no reference so folks could research this intriguing hypothesis on their own.
Given that there was no mechanism to prevent individuals from taking the survey multiple times (no participant key issued, no flag that a prior survey had been submitted from your IP address, no reason the survey couldn’t have been completed from multiple different devices) or even screening for actual membership in the target survey population, how can it possibly be asserted that the results of the survey will be representative of the target population?
Would you be OK with such a method being used, say, for electing public officials?
If yes, why yes? If not, why not?
“And Pew, while an excellent organization, doesn’t set the only standard of research ethics. You can have an ethical and effective study while coming into the question with a definite perspective or set of assumptions, as long as you are aware of your bias and take steps to mitigate it.”
This is helpful–you’ve clarified that the group isn’t holding itself to Pew’s standard on removing real or perceived conflicts of interest. My understanding, however, was that this new group was trying to improve upon the Pew survey. How can it claim to do that if some of its standards are apparently inferior?
Katie – to put the concern about bias/conflict of interest another way, it’s like what would happen if the Democratic National Committee leadership put together a poll for republicans and former republicans on how the Republican National Platform should be changed. The sample would been a poor representation because of the perception from said Republicans that there is an ulterior motive behind the survey. We understand that every survey has some bias; it’s the amount of bias apparent that is the trouble.
Meg – The post Katie linked at 10:48 explains the other point of view pretty well. They’re not looking for no bias. There’s just the assumption that whatever bias there is is acceptable. The argument is that every survey is self selecting, so no survey is free of bias. I’m not terribly hopeful for the results of this survey, but I’m curious to see what turns out, and how the media is used to exploit it.
And our elections aren’t representative of the entire population; they’re representative of the percentage of the population that cares enough to vote. No, it doesn’t resolve the issue of multiple votes, but I don’t believe, with the amount of work it takes to do the survey, that repeated survey taking should be that big a concern.
This is helpful–you’ve clarified that the group isn’t holding itself to Pew’s standard on removing real or perceived conflicts of interest.
Ugh, Tom, I have no inside knowledge about what steps the group has or hasn’t taken, and I don’t appreciate you twisting my words like that. My greater point was that we have no reason to believe that there is a conflict of interest–and that bias and conflict of interest are two separate things. Bias you have to work with and can never fully eliminate. But I don’t believe there is any risk for conflict of interest.
Frank, the thing that makes me super sad about this whole thing is the division in the Body of Christ over this. “It’s like the Republicans doing a survey on the Democrats.” Except that it’s not at all like that–we’re all members of the church, right?
But that’s naive, isn’t it. I took a FB break so that I wouldn’t see all the arguments over this survey because they genuinely hurt my soul, but my husband told me about this post from Andrew, who is an old friend of mine, and I wanted to dialogue with him more about it. I think it’s probably best that I take a step back here, because I’m getting kind of upset again. Andrew, if you happen to read this, maybe Facebook me your feedback at some point–when I get back on there, I can chat with you more about it.
Wishing you well,
Katie
There are few things that my Bachelor’s degree in Political Science taught me that can be applied to the real world. One of the few things I learned that have been beneficial was the importance of being critical of statistics. What do they say? “Lies, damned lies and statistics”. Anyone with an interest can put a “study” together to make pretty much any point they want. My question is really simple, if the creators of this “study” really wanted to gather information, why didn’t they just use proven data collection methods that yield accurate results? There is NO way any useful information can be extrapolated from this survey. This isn’t my opinion, this is a fact. The only people I saw that were passing around this survey were my feminist friends. My mind is blown that anyone out there that has taken ANY class in statistics and proper data collection methods can take this survey seriously. How can anyone defend this as anything close to accurate? It’s a facebook survey! This means that people can choose whether they participate in a facebook survey that is represented by a feminist group. Hmm… I wonder who’s going to take the time to complete the survey? The methodology used for data collection is so bad.. Anyone who doubts what I’m say, take this “study” to any Statistician, and ask them whether the data collected from this facebook survey could possibly be accurate or even provide any useful information.
Katie – You are defending a study that asks questions about “good” Mormons, “Liahona” Mormons, and “Iron Rod” Mormons. Please don’t take offence to a comparison to political parties that you say divides the body of Christ. I’m not sure if you’re familiar with the groups that the TMGISG are affiliated with. There are many comments on those sites that could be construed as dividing members….Liberal Mormons, TBMs, Conservative Mormons, etc.
Here is a definition of conflict of interest. “A situation in which a person is in a position to derive personal benefit from actions or decisions made in their official capacity.” The individuals who are part of the TMGISG are affiliated with groups who are open about their goals to ordain women and to change the church’s teachings on marriage. That is their driving purpose in existing. They would derive personal benefit from the survey showing certain results, ie. effective media exposure to reflect negatively on church teachings they don’t agree with. This survey is a conflict of interest. I may be conflating things…but, I don’t know how this group isn’t both biased and involved in conflicting interests.
It may have been wise to private message Andrew initially. If you were already tired of the discussion on Facebook about this study, it may have been good to avoid commenting on a post.
Again, how can this group of individuals be taken seriously as a group of impartial people conducting a survey?
Conflict of interest. I don’t think people using that word know what it means, if they claim members of the group deploying this survey do not have a conflict of interest.
Would it have been useful had Pew clarified the reason 90% of self-identified Mormon women (the youngest adult in the household or the one answering the cell phone) had explored why they responded as they did? Yes. I, for one would have been intrigued by a follow-up questions, such as:
“Do you think there will come a future time when women will exercise priesthood power?”
If yes, “Do you think that time might come in this life, or do you think it might come only in an afterlife?
If no, “Do you think women might exercise priesthood power in the afterlife?”
“If the Mormon Church were to open priesthood up to women, as they opened priesthood to all men in 1978, would you, as a woman, be interested in being ordained to the priesthood?”
I think that the answer to the last question would be 100% affirmative.
If they asked one more question, I think we would have had another interesting data point:
“Do you feel it is appropriate to lobby the Church for change?”
I suspect the answer to the lobbying question would be 95% in the negative.
Katie Langston, check out this post here: https://www.millennialstar.org/on-social-science-and-bias/
From a scientist friend of mine back in Logan. This is what’s wrong with the Mormon Gender Issues Survey. On a s… http://t.co/jhKcChV3hp
A Pew, a Pew! We have a Pew and we do not need any more Pew!
Take heed of Gamaliel: Refrain from these researchers, and let them alone: for if this counsel or this work be of humans, it will come to nought: but if it be of God, ye cannot overthrow it; lest haply ye be found even to fight against God.