Part 9 -- Waffle Words And Intel-Speak (The Revolution Begins On Page Five: The Changing Nature Of The NIE And Its Implications For Intelligence)

Part 1 -- Welcome To The Revolution
Part 2 -- Some History
Part 3 -- The Revolution Begins
Part 4 -- Page Five In Detail
Part 5 -- Enough Exposition, Let's Get Down To It...
Part 6 -- Digging Deeper
Part 7 -- Looking At The Fine Print
Part 8 -- Confidence Is Not the Only Issue

Part 9 -- Waffle Words And Intel-Speak

The Words Of Estimative Probability (WEPs) outlined in Part 8 are not the only WEPs that analysts can use to express probabilistic judgments. The recent National Intelligence Estimates (NIEs) also specifically refer to a series of words such as "might" or "may" and phrases such as “we cannot dismiss” or “we cannot rule out” that are meant to signify events of undetermined probability or events that are remote but significant if they do occur. Analysts often perceive the use of some of these type words as unavoidable if they wish to convey the full range of possibilities inherent in an estimate. Decisionmakers have another attitude about these words. They call them "waffle words" or "intel-speak" and often believe that the primary reason for their inclusion is to cover the analyst’s backside.

The six public estimates reviewed in this sample contain a number of these type words. Again, I counted only examples where the analyst was making an estimate (excluding, for example, those times where “might” was used as a noun). I included a broader range of waffle words than some might agree to on first blush. I hope to outline the problems with each but there seem to be three conclusions that jump out from the chart above.

First, there appear to be roughly the same number of waffle words as there are statements containing WEPS (In fact, in the Iran NIE, the number was exactly the same) across the entire set of NIEs examined. I think this is a bad thing and will argue (hopefully convincingly) later on that, using the system the Intelligence Community (IC) has already laid out, there is not only a good reason not to use waffle words but also a simple way to keep from ever having to use intel-speak again.

Second, it seems likely that the ratio of waffle word sentences to WEP sentences is a good indicator of overall confidence in the estimate. I find it fascinating, for example, that the first Iraq Stability NIE contained over 3 times as many sentences using waffle words words such as "could" and "might" as sentences containing more more meaningful WEPs such as "likely". Such a strong preference for one type of formulation over another sends a strong signal that the analysts involved were (perhaps unconsciously) hedging their bets in a very real way.

The third conclusion is that there is a moderately strong preference for the waffle-words, “could” and “if”. It is easy to see what is wrong with “could”. Anything "could" happen. To tell a decisionmaker that something could happen is to increase his or her uncertainty, not reduce it. Certainly, with no WEP and confidence statement to even roughly assign a probability to the described event, decisionmakers are left on their own to figure out an appropriate level of time or other resources to devote to thwarting this nebulous threat or to take advantage of this ephemeral opportunity. This is tantamount to asking the decisionmaker to be the analyst! “All I can figure out is that something could happen, boss,” the IC seems to be saying, “You need to figure out how likely it is so you can assign the appropriate resources to deal with it. Oh, and by the way, if you guess wrong, I will still be able to say that I warned you.” It is easy to see why decisionmakers don’t like “could”.

Take a look at this statement from the August 2007 Prospects for Iraq’s Stability NIE:

• A multi-stage process involving the Iraqi Government providing support and legitimacy for such initiatives could foster over the longer term political reconciliation between the participating Sunni Arabs and the national government.

Or it could not. The use of the word “could” here is really not very helpful. I suppose it is possible that the policymakers that were involved with the Iraq situation at the time were not aware of this possibility but I kinda doubt it. Now the policymakers themselves have to figure out if they should pursue a policy that supports the Iraqi Government along these lines or not. The IC has pointed out the obvious (to the decisionmakers, anyway) and has not given them any sense of which way it will go.

How might it have been better phrased? What about:

• We assess with low confidence that a multi-stage process involving the Iraqi Government providing support and legitimacy for such initiatives will likely foster over the longer term political reconciliation between the participating Sunni Arabs and the national government.

While the specific terms I have used here ("low confidence' and "likely") are clearly notional (I don't know what words the authors would have used had they used this formulation), this statement is entirely consistent with the guidelines laid out in the Explanation of Estimative Language (EEL) page. It does what the intel community should do, make the call, while still being properly caveated. It does not ask the decisionmaker to be the analyst as well. Finally, the analyst, by using consistent terminology and providing a more useful estimate, is less open to unjustified criticism in any sort of after action review.

The point is that appropriately using confidence combined with a WEP allows an analyst to make the best call with the facts he or she has at the time yet still send a signal to the decisionmaker about the firmness of the estimate without having to resort to useless waffle-words. It is a better system. It is better because it is clearer. It is better because it is consistent. It is better because it stands up to after-action scrutiny. It seems to be what the IC has in mind on page 5 but it is clearly not what the IC is doing (at least not with its most recent public NIEs).

Tomorrow: Part 10 -- The Problem With “If”