The front page of The Washington Times recently blared: “Navy sailors distrust commanders, fear crippling political correctness.” Quoting a “survey of the fleet,” the report labeled the distrust as “widespread.” Surely, a headline like this should send shockwaves through the senior leadership in the Navy. But, the fact is, this so-called “survey of the fleet” was nothing more than an effort by the survey’s organizers and respondents to lobby Navy leadership for policy change, resulting in questionable recommendations based on unsupported extrapolations.
The story of this survey dates back to the March publication of CDR Guy Snodgrass’s widely read and discussed paper on officer retention, “Keep a Weather Eye on the Horizon.” What happened next is a bit more interesting. CDR Snodgrass organized a group of almost 20 officers and civilians to develop a web-based survey with the stated goal of understanding retention in the Navy. The report that emerged from this survey was quite broadly titled the “2014 Navy Retention Study.” This was an unofficial venture, but one could easily think otherwise given that title. If believed, the summary of the survey’s findings are nothing short of stunning: low Navy-wide morale, poor work-life balance, waning desire for senior leadership positions, and widespread distrust of senior leaders.
However, there are two problems with the survey: The methodology is riddled with problems and the survey report’s most conspicuous recommendation comes out of nowhere.
The methodology used for this survey does not deserve the label. It was more akin to a signature petition for change submitted to Navy leadership. According to the reported findings, the survey was conducted by distributing a polling form online through “military social media channels.” Participants learned of the survey by word of mouth, apparently spread by a number of blogs and websites. But, while the report places confidence in its sample size and margin of error, it fails to consider the issue of selection or sample bias. Not only were those who knew of and participated in the survey limited to those active in social media, they also represent a demographic that sought out the survey in order to participate and – it seems – complain. Coupling the informal crowdsourcing approach (as described by CDR Snodgrass himself) with the probability that those who participate in voluntary surveys may harbor strong opinions on the survey subject, there is a very real possibility that the sampling method suffers from a volunteer bias that renders the findings unreliable.
While I recognize the obvious limits in constructing an unofficial survey that is representative, the survey’s authors do not. Instead, the survey’s report provides seemingly limitless extrapolations in speaking for the Fleet’s, not the sample’s, dissatisfaction with leadership. CDR Snodgrass’ report offers a series of recommendations to Navy leaders to help them overcome that gulf in trust.
The report’s very first recommendation is to “stop highlighting Commanding Officer and Command Master Chief firings.” The report labels this a “significant breach of trust with our Sailors.” Initially, it is worth noting that this recommendation appears unsupported by the data. Over 50% of survey participants indicated that they believed senior leaders would not hold themselves accountable. Analyzed against the report’s conclusion that highlighting misconduct has a negative influence on officer retention and morale, the overwhelming belief among respondents that senior officer’s are above the law either means that participants believe that there is both a lack of accountability in the Navy and an over-emphasis on accountability at the same time, or the report’s authors are expressing a pre-existing opinion disconnected from the data.
More importantly, though, the recommendation to stop publicizing the misconduct and shortfalls of commanding officers demonstrates a puzzling disconnect on the part of the authors with the current challenges Navy leadership faces. The misconduct of military leaders is a subject in which the civilian leaders in Washington consistently express increased interest. For instance, in May, the U.S. House of Representatives voted to include a provision in the FY15 National Defense Authorization Act mandating that the Department of Defense publicly release all DoD Inspector General investigations into senior officials. These would seemingly include both substantiated and unsubstantiated investigations. Similarly, DoD has expressed concern in the last year with the extent of misconduct among senior officers.
So, assuming for the moment that there is merit in this recommendation, the notion that the Navy realistically has room to back-peddle in publicly acknowledging senior officer misconduct fails to appreciate the current context. Reversing course now would appear self-serving at best, and complicit at worst.
Turning to the merits of the recommendation to stop publicizing commanding officer firings, public acknowledgement of misconduct serves both a practical and philosophical purpose. Practically, public firings allow the Navy to “get ahead” of the release of the information via gossip and social media and avoid the inevitable damage control that results when leadership appears to be caught on its heels. In his 2012 article in the Naval War College Review, “The Navy’s Moral Compass”, Captain Mark Light observed that holding commanding officers publicly accountable for their actions “is vastly preferable to hiding them until a disgruntled subordinate posts a video online for the world to see.” There is also a valid general deterrence argument in support of the publication of misconduct by superior officers. Corporate ethics scholarship speaks to the value of publicizing the nature of employee misconduct and the resulting disciplinary action in the communication of expectations throughout an organization…
Philosophically, public firings are consistent with the unique position that a commanding officer in the naval service holds. In the naval service, commanding officers have an express statutory duty, laid out in 10 U.S.C. 5947, to demonstrate “exemplary conduct” which goes beyond the required conduct of all uniformed members under the Uniform Code of Military Justice. Moreover, those in positions of authority are directed to “suppress all dissolute and immoral practices.” These expectations are reiterated in Admiral Roughead 2011 Charge of Command, in which he emphasizes the importance of accountability in maintaining trust with one’s subordinates.
The retention survey’s recommendation misjudges to whom that trust is owed in the context of misconduct by a commanding officer – it is first and foremost owed to the subordinates of the commanding officer, not to the commanding officer. Publicizing the firing of a naval commanding officer keeps faith with the his or her subordinates by demonstrating that we all must abide by the same standards of conduct, while also reinforcing the commanding officer’s historic stature in the U.S. Navy. Just as a commanding officer’s successes are an example to others, so to are his or her failings. Perhaps, as the report suggests, there are those officers in the Fleet who hope against taking command because they would have a problem being publicly relieved. That ultimately sounds like a convenient way of describing a fear of one’s own self-discipline.
The retention survey was done in the name of retaining good sailors and in the interests of keeping faith with them – a goal for which the organizers should be commended. Unfortunately, the product of this “study” was most effective at publicly shaming Navy leadership with misleading data and trivializing serious challenges that they face, ironically the very sort of thing the report so zealously argued against. Perhaps the public messaging of this report was not well thought out, but what appeared to be a constructive endeavor in March hardly seems so now.
Ryan Santicola is a judge advocate in the U.S. Navy. The views expressed in this article are his own and do not reflect those of the U.S. Government, the Department of Defense, or the U.S. Navy.
Photo credit: Official U.S Navy Imagery
Ryan Santicola 
MCPO David Mattingly USN Ret 
Rellag 
W2 
Michael Miller 
I would hardly qualify the most “conspicuous recommendation” as the one calling for an end to the public flogging of COs. There were many others that I found far more compelling — and had the data to back them up.
As for the methodology, it’s important to note that two recent OFFICIAL retention surveys used similar acquisition methodologies as the most recent, unofficial one. Last year’s SWO survey was done on Survey Monkey and anybody could access it via Facebook or from an email forward (ask me know I know…). The recent official aviation retention survey was assigned to individuals by email address, but again, there was nothing random about it. Same potential selection bias — only those most motivated may have answered the survey.
Finally, while the author may disagree with the results and the methodology, what other statistically significant and relevant data can he point to, aside from anecdote, that analytically convinces him of the Retention Study’s inaccuracy? If Mr. Santicola is aware of a Gallup type poll of Sailors that refutes the findings, many of us would love to be enlightened!
Ben – I appreciate your comment. I agree that some of the recommendations could have value when it comes to job satisfaction and retention. I do have a hard time accepting, however, that there is a realistic connection with retention for things like public firings and mandatory training (both of which are things that the Navy does not necessarily dictate).
My critique of your methodology is not relative to other surveys. The official surveys you reference may suffer from the same selection bias and offer no more reliable data, But, whether official or unofficial, surveys that seek to be relied on should acknowledge possible biases in their methodology and take a measured approach in drawing conclusions.
I can’t point to a Gallup-type poll, but it is funny you bring up Gallup. According to Gallup’s Global Workplace study, only 30% of US workers are “engaged” in their jobs. Compared to the over 50% approval by respondents on your Quality of Work questions (which resemble the Q12), the respondents are substantially more satisfied with their work and “engaged” than their civilian counterparts in the US. There is obviously always room for improvement, but these stats suggest that the Navy is doing something right on issues that impact retention.
As you reference, there was actually quite a bit of good news in the survey, which was published and referenced in the final report. Attention often goes to the controversial, bad news while ignoring good things, namely Sailors feel they are making a difference, and generally have the right tools to do their job. Furthermore, they actually rate the superiors they personally know relatively well (which hits on the Congress paradox — people hate Congress, but always overwhelmingly reelect their personal representative.) It’s not all bad news, as we point out.
As for the Gallup poll comparison, you draw a favorable conclusion for our Navy based on the results of a survey you yourself call methodologically flawed and biased. I’m curious how you’re able to square that given what you argued previously.
Retention of qualified Sailors both officer and enlisted dates back to my entry in the Navy.
The idea that the Navy should “hide” senior leadership firings is abhorrent and fails the test of the Navy’s Core Values of Honor, Courage, and Commitment. What it should be is a lesson to those who accept the challenge of command to put Sailors First! Not their personal interests.
The Navy and Department of Defense can be its own worst enemy as the wars in Iraq and Afghanistan began to draw down the “scuttlebutt” of a reduction in force became reality and Sailors looked at options to protect their families.
I met a young PhD candidate this weekend that left the Navy as a 11 year SWO. His reason was he joined the Navy but after three tours with the Army in Iraq he opted out essentially deciding he didn’t join the Navy to be a soldier.
The role of the military to respond to events like the current Ebola crisis makes all servicemen and women to ask, “is this what I singed up for?”
Ryan,
Thank you for taking the time to write your blog piece. While I don’t agree with your argument, I can certainly appreciate your opinion and applaud you for taking the time to put your thoughts out for discussion and scrutiny. That takes guts.
Like you, I’m not a statistician — though I recruited a couple PhD-weilding statisticians for the project: One from the Naval War College (Cambridge PhD) and another from the Rady School of Management. We discussed this very issue early on, however both felt the sample size received for many communities was enough to provide statistical accuracy.
I believe your assertion is correct: Accuracy is affected by the randomness of the sample. By that metric, I’m not sure I’ve ever participated in an official Navy survey with a guaranteed random sample. Even those conducted by the Navy Personnel Research, Studies, and Technology (NPRST) division on OPNAV N1 aren’t “random”, as participation is voluntary, leading one back to the distracting argument that participants must be strongly biased since they took the time to participate. This would also hold true to random phone sampling, such as popularly used during an election cycle. Although random sampling is used to reach respondents, the participation rate is largely affected by a person’s interest in the particular topic, therefore bias is injected. The only way to “guarantee accuracy” of results is through a 100% participation rate of a randomly selected sample size, thereby removing participation bias.
The survey provides data to help inform a thoughtful and deliberate discussion about what motivates and influences a Sailor’s stay in/get out decision making. Some here may feel strongly the survey was flawed. Others may believe the responses received are reflective of the current state of affairs at the waterfront. Personally, I’m not given to attempt to prove a negative, especially since representatives from the Chief of Naval Personnel’s office have said the unofficial survey “roughly mirrors the command’s own polling” in official statements.
More important is the sense of ownership for the Navy shared by many who are reading this post and beyond. I am truly heartened by those willing to read widely, think deeply, write passionately, and act decisively on behalf of this institution. Remember – nothing worthwhile is ever easy, but it becomes more so when putting the needs of the service before self — something many readers here are already intimately familiar with.
Best,
Bus
XO,
I appreciate the comment and respect very much your call for critical thinking and debate in the Navy, regardless of the topic. While we may take different views on some of the conclusions drawn from this survey as to what the Navy should do to retain good Sailors, I think we both agree that the Navy should strive to retain good people and craft good policy in doing so. I look forward to discussing this topic more with you in the future sir.
V/R
-Ryan
You wrote:
“the overwhelming belief among respondents that senior officer’s [sic] are above the law either means that participants believe that there is both a lack of accountability in the Navy and an over-emphasis on accountability at the same time”
Pure sophistry, my friend.
Wishing to see an end to the public humiliation and reputational impairment of Commanding Officers who fail to meet an amorphous, non-legal, and entirely subjective standard (typically, the “loss of confidence” of their ISIC) is not the same as a collective request for application of a lesser standard. Whatever that would mean in such a a context)
The fact the author of this piece has a beef with the survey, that it’s, in his opinion, statistically flawed or that it’s conclusion is predetermined is missing the point. It’s a FREE data point, done by someone who appears to care deeply about the organization and the direction it is headed. The survey should be reviewed with an eye for finding potential issues, not nitpicking. As one of the author’s of the study points out above, there was both goodness and badness that came out of the survey. Throwing the baby out with the bath water is a sure sign of supporting the status quo. Surely the author of this piece doesn’t have a predetermined conclusion about the survey results, right?
I am deeply disturbed by the strongly stated assertions made in this piece. As a participant in the survey, and one who encouraged my fellow officers to participate to gather data on the issue, it is frankly hurtful to read someone so vehemently criticize a project which received such a high level of effort from quality officers who I know and respect.
But, the question remains, is Ryan right? Is the survey flawed? In terms of characterizing the survey as not worthy of its name because it was crowd-sourced, nothing could be farther from the truth. I’m currently reading “Cybersecurity and Cyberwarfare: What Everyone Needs to Know” from the official Navy Reading List. While the title obviously doesn’t jump out at one as a thrilling read, it’s actually quite readable and informative. Pertinent to this discussion is the statement in the introduction that “the centrality of computers to our lives is almost impossible to comprehend…the Internet is no longer just about sending mail or compiling information: it now also handles nearly everything . Indeed, Cisco estimated that 8.7 Billion devices were connected to the Internet by the end of 2012, a figure it believes will rise to 40 Billion by 2020.” To claim that distributing the survey via “military social media channels” and Facebook will constrain and limit a sample size is outrageous. On the contrary, this medium is likely to be the most effective and expedient way to reach a large and diverse group of people (particularly a younger demographic of junior officers and enlisted personnel) and gain more participation which effectively minimizes sampling error. This assertion is supported by expert advice from respected statisticians sought out by CDR Snodgrass which he referenced in a previous post, and further confirmed by correlating with the findings from different survey sampling methods utilized by the Chief of Naval Personnel office. While Ryan makes a fair case for the “possibility” of volunteer bias, he provides absolutely no valid reasoning to support such strong rhetoric that it is a fatal flaw to the 2014 Navy Retention Study.
Switching gears from the frontal attack on the validity of the study itself, the final seven paragraphs of Ryan’s work are dedicated to refuting the specific recommendation to “stop highlighting the firings of Commanding Officers and Command Master Chief’s.” The main argument centers around the statistic that 50% of respondents feel that senior leaders are not held accountable for their actions. He labels the comparison of this statistic with the recommendation to cease highlighting firings as a “puzzling disconnect,” on the part of the survey’s authors. In reality, there is no such disconnect, Ryan has just misunderstood the basis for the recommendation which, quite frankly, is quite clear in the full report and will be addressed below. It seems implicitly clear that he is interpreting this to mean that the Navy should “cover-up” misconduct cases. If that were the case, then his argument for an accountability disconnect and back-peddling on current policy would hold true. Unfortunately, that’s not the point being made and no one in the Navy Retention Study is advocating that course of action.
Ryan’s two paragraphs on the merits of public firings replete with several references to US Navy policy documents and a recitation of Title 10 law do not change the fact that “What was originally intended to demonstrate accountability to the public has, instead, resulted in a significant breach of trust with our Sailors and resulted in an almost ‘reality TV’ mentality.” The recommendation of the Retention Study is not to avoid accountability and clearly states that it is a “cornerstone of leadership and must be preserved.” What it does recommend is finding a way to “hold leadership accountable without appearing to throw them to the wolves.”
The fact is that there has to be a balanced approach to achieve both desired end states. The way forward will likely involve the contributions of judge advocates like Ryan who are able to research and define the Navy’s legal and moral requirement to release certain information while at the same time protecting the rights to due process and privacy that should be afforded to the CO or CMC whose conduct is in question. Concurrently, US Navy public affairs officers with experience in media relations will likely have insights in how best to time and frame the release of such information to ensure accountability without allowing the situation to deteriorate into a soap opera on the national stage dragging the reputation of an entire organization of committed, hard-working Sailors through the mud in the process. That is the “breach of trust” that is being referred to and for those who have been involved with it, it is a painful and demoralizing process that inevitably causes them to question the worth of continued naval service.
Like CDR Snodgrass, I applaud Ryan Santicola for writing passionately about the issues but I wholeheartedly disagree with his argument and conclusions. I encourage him and others like him who are clearly intelligent and passionate and write well to join in the process of fixing the problem, instead of just criticizing the hard work of others who are trying to fix flawed policies which are a very real threat to the continued health and viability of the US Navy.