As previously noted, the US Air Force Academy responded to Freedom of Information Act requests by holding a press conference and releasing the previously withheld results of the 2009-2010 climate survey.

The release was part of an official Air Force announcement; the Academy announcement contains a link to the presentation made by USAFA Superintendent Lt Gen Michael Gould at the press conference (available here, 5.6MB pdf).

The Academy made a concerted effort to explain the results (or, more accurately, describe how the results couldn’t be explained).  It appears the Academy survey was “imperfect” in several respects: 

The caveats include ambiguous and vague questions that left room for multiple interpretations; changes in questions from previous surveys as well as new questions that had no trending data; and the timing of the survey, which was conducted during finals and holiday breaks.

The survey was also completely voluntary.  While admirable in most cases, in a closed society those with “issues” on their mind — whether they are shoes to shine or axes to grind — are far more likely to spend the extra effort to participate in an otherwise optional feedback session.

The Academy also acknowledged the survey was perception-based (a “feelings” survey, as others have said).  This manifested itself in at least one demonstrable instance:  Cadets indicated females and minorities were less likely to have leadership opportunities at USAFA.  However, a factual analysis of cadet leadership positions showed “fair representation across the leadership spectrum.”  Thus, the survey showed the cadets felt one way, when in fact reality was “better.”  This “perception does not equal reality” fallacy may have occurred in other areas of the survey as well.

The result receiving the most attention was the headline that “41% of non-Christian AF cadets cite proselytizing.”

An Air Force Academy survey found that 41 percent of cadets who identified themselves as non-Christian said they were subjected to unwanted proselytizing at least once or twice last year.

It is worth noting that while “41%” sounds dramatic, it accounts for 32 cadets of the 4400-member cadet wing (or 0.7%).  Even then, the data is difficult to analyze because of the weakness of the survey itself.  For example, the

survey asked cadets how often during the previous 12 months they had been in situations when someone subjected them to unwanted proselytizing and gave five choices: very often, often, sometimes, once or twice and never.

With 80% of the answer choices in the affirmative, the question appears to presume some level of proselytizing.  In addition, the survey did not specify whether the “someone” was a superior, subordinate, peer, military, civilian, or even associated with the Academy at all.

More importantly, the survey did not define “unwanted proselytizing,” meaning cadets were free to interpret it as they saw fit.  For example, a non-Christian cadet invited to church by his roommate may have said he experienced “unwanted proselytizing,” despite the fact such conduct is not unreasonable and is actually perfectly permissible — and is therefore not an actionable “problem.”

The problem with a survey of this nature — a problem Gould likely foresaw — is that numbers can be made to say anything.

For example, based on the survey data:

• More Christians than non-Christians fear for their physical safety because of their faith;
• Independent of faith, there is a reported decrease in “religious freedom” at USAFA;
• More Caucasians than minorities fear for their physical safety because of their race;
• Females make up a disproportionate number of the instigators of sexual discrimination and harassment.

While those statements are factually correct based on the survey results, it would be irresponsible to assert that white Christians are in a physically threatened position at USAFA, or that the Academy has a female-led sexual harassment problem.  The survey does not yield sufficient data from which to draw any larger conclusions.  It makes for interesting headlines and controversy, but little else.  That inefficacy is likely behind Gould’s original decision not to release the results — doing so helps nothing.

Regardless, the Academy staff has taken several steps to address even the perception of problems within the wing.  In addition, it has indicated it will fix its own problem: creating a survey that is actionable:

The [headquarters] staff will review survey questions to improve the next survey’s accuracy, make future surveys more actionable and maintain trending questions where possible.

As a sidebar to the press conference, Michael Weinstein was again denied the “special” attention he so often seeks.  The briefing was only open to credentialed press, despite Weinstein’s juvenile “demand” that he be invited.  (The previously effusive Weinstein said Gould, whom Weinstein now calls the “lord of lies”, was trying to “muffle the opposition.”)

In being rebuffed at the press conference, Weinstein may still be smarting from the loss of his personal “bat-signal” with the Academy.  With the demise of his ‘favored status,’ Weinstein and his MRFF now appear to be relegated to the sidelines, little more than an irrelevant fringe group espousing wild conspiracy theories.