“Structured Analytic Techniques for Intelligence Analysis” Notes
Structured Analytic Techniques for Intelligence Analysis by Richards J. Heuer Jr. and Randolph H. Pherson is a useful guide to 55 structured analytic techniques (SATs) for improving your intelligence analysis by removing cognitive biases and identifying alternatives to consider. For each technique, the book describes when to use it, the value added, the method (steps), potential pitfalls, the relationship to other techniques, and the technique’s origin. The book is well-written and easy to follow.
I read the 2nd Edition (2014), but if you can find it, read the 3rd Edition (2020).
I read this because I’ve seen it recommended for cyber threat intelligence analysts, including by Katie Nickels, Scott J. Roberts, and Andy Piazza.
My notes follow.
This page contains one or more affiliate links. As an Amazon Associate, I earn from qualifying purchases.
Introduction and Overview
SATs form a methodology for qualitative analysis of the kinds of uncertainties that analysts deal with.
This book builds on Hueur’s Psychology of Intelligence Analysis.
SATs don’t always give the correct answer, but they identify alternatives that deserve consideration.
Building a System 2 Taxonomy
System 1 thinking: intuitive, fast, efficient, often unconscious. Often accurate, but common source of cognitive biases and intuitive mistakes. System 2 thinking: analytic, slow, deliberate, conscious. Includes SATs, critical thinking, empirical and quantitative analysis.
Structured analysis externalizes analyst’s thinking so it can be reviewed, discussed, critiqued piece by piece.
Choosing the Right Technique
Core SATs
- Structured Brainstorming
- Cross-Impact Matrix
- Key Assumptions Check
- Indicators
- Analysis of Competing Hypotheses (ACH)
- Premortem Analysis and Structured Self-Critique
- What If? Analysis
5 Habits of a Master Thinker
- Know when to challenge key assumptions.
- Consider alternative explanations/hypotheses for all events, including deception hypothesis and null hypothesis (that what’s being hypothesized isn’t true).
- Look for inconsistent data that provides sufficient justification to quickly discard a hypothesis. ACH is best way.
- Focus on key drivers that explain what has occurred or may occur.
- Anticipate customers’ needs and understand context of analysis.
SATs can take as little as 1–2 hrs, and save time in long run by avoiding wrong tracks and reducing time for editing and coordination. They also produce higher-quality and more compelling analysis.
Selecting right technique
- Define project: Decomposition and Visualization (Getting Started Checklist, Customer Checklist, Issue Redefinition, Venn Analysis), Idea Generation
- Get started (generate list of driving forces, variables, indicators, players, precedents, info sources, questions); organize, rank, score, prioritize list: Decomposition and Visualization, Idea Generation.
- Examine and make sense of data: Decomposition and Visualization (Chronologies and Timelines, Sorting, Network Analysis, Mind Maps, Concept Maps), Idea Generation (Cross-Impact Matrix).
- Explain recent event; assess most likely outcome of developing situation: Hypothesis Generation and Testing, Assessment of Cause and Effect, Challenge Analysis.
- Monitor situation for early warning; avoid surprise: Scenarios and Indicators; Challenge Analysis.
- Generate and test hypotheses: Hypothesis Generation and Testing, Assessment of Cause and Effect (Key Assumptions Check)
- Assess possibility of deception: Hypothesis Generation and Testing (Analysis of Competing Hypotheses, Deception Detection), Assessment of Cause and Effect (Key Assumptions Check, Role Playing, Red Hat Analysis).
- Foresee future: Scenarios and Indicators, Hypothesis Generation and Testing (Analysis of Competing Hypotheses), Assessment of Cause and Effect (Key Assumptions Check, Structured Analogies), Challenge Analysis, Decision Support (Complexity Manager).
- Challenge your mental model: Challenge Analysis, Idea Generation, Hypothesis Generation and Testing (Diagnostic Reasoning, Analysis of Competing Hypotheses), Assessment of Cause and Effect (Key Assumptions Check).
- See events from perspective of adversary or others: Assessment of Cause and Effect (Key Assumptions Check, Role Playing, Red Hat Analysis), Challenge Analysis (Red Team Analysis, Delphi Method), Conflict Management, Decision Support (Impact Matrix).
- Manage conflicting mental models or opinions: Conflict Management, Hypothesis Generation and Testing (Analysis of Competing Hypotheses, Argument Mapping), Assessment of Cause and Effect (Key Assumptions Check).
- Support manager, planner, policymaker in deciding; draw actionable conclusions: Decision Support, Conflict Management, Hypothesis Generation and Testing (Analysis of Competing Hypotheses).
Decomposition and Visualization
AIMS: Before starting a paper, think through Audience, Issue or intel question, Message, Storyline.
Network Analysis: Review, compile, and interpret data to determine associations between entities, meaning of associations to entities, and degrees and ways in which associations can be strengthened or weakened.
Idea Generation
Structured Brainstorming: Systematic, multistep process with silent brainstorming and sticky notes or wiki, led by facilitator, to identify variables, driving forces, hypotheses, key players/stakeholders, evidence, potential solutions, potential outcomes.
Starbursting: Brainstorm, focusing on generating questions, not ideas or answers. Use who, what, where, when, why, how?
Scenarios and Indicators
Scenarios Analysis: Postulate different scenarios to identify ways situation might develop, to help decide how to exploit opportunities or avoid risks.
Cone of Plausibility: Use key drivers and assumptions to generate range of plausible alternative scenarios, to help imagine various futures and their effects.
Alternative Futures Analysis: Create a 2x2 matrix analyzing 2 driving forces/factors/events, each with 2 extremes, to describe 4 possible outcomes.
Multiple Scenarios Generation: Similar to Alternative Futures Analysis, but use multiple 2x2 matrices to pair every combination of multiple driving forces.
Indicators: Monitor observable actions/conditions/facts/events to track events, spot trends, avoid surprise.
Indicators Validator: Assess whether a given indicator would appear in only 1 scenario/hypothesis, to assess its diagnostic strength (how strongly it points to only 1 scenario/hypothesis and suggests others are unlikely).
Hypothesis Generation and Testing
Hypothesis Generation: Create comprehensive list of mutually exclusive potential explanations/conclusions, to be supported or refuted by observation or experimentation.
Characteristics of good hypotheses
- Written as definite statement, not question.
- Based on observations, knowledge.
- Testable, can be proven wrong.
- Predicts anticipated results clearly.
- Contains a dependent variable (phenomenon being explained) and independent variable (how phenomenon is explained).
Diagnostic Reasoning: Apply hypothesis testing to new development, single new item of info/intel, or reliability of source.
Analysis of Competing Hypotheses
Analysis of Competing Hypotheses (ACH): Identify mutually exclusive hypotheses, systematically evaluate data that are consistent or inconsistent with hypotheses, and reject hypotheses with most data against them until you discover the most likely one.
- Identify mutually exclusive hypotheses.
- List relevant info (evidence, assumptions, absence of things one would expect).
- Create matrix with hypotheses across top and relevant info down left side. Mark intersecting cells as Consistent, Inconsistent, Not Applicable. Mark compelling cells.
- Review where analysts differ in assessments, and decide if adjustments are needed in ratings.
- Refine matrix by reconsidering hypotheses (combine, add new).
- Draw tentative conclusions and relative likelihood of each hypothesis.
- Consider how dependent conclusions are on a few critical items of info, and consider reinterpreting.
- Report conclusions and relative likelihood of each hypothesis. State items of info that have most diagnostic value, and how compelling a case they make.
- Identify milestones or indicators for future observation. Create list of future events or discoveries that would prove validity of your judgment. Create list of indicators that would suggest your judgment is less likely to be right, or that situation has changed. Monitor both lists.
Argument Mapping: Take single conclusion/hypothesis, graphically branch out its reasons, evidence, assumptions to see logical relationship between them.
Deception Detection: Use checklists to determine when to look for deception, discover whether deception is present, and figure out how to avoid being deceived. Checklists: Motive, Opportunity, and Means (MOM), Post Opposition Practices (POP), Manipulability of Sources (MOSES), Evaluation of Evidence (EVE).
Assessment of Cause and Effect
Key Assumptions Check: Systematically identify and question assumptions (preconceptions).
Role Playing: Analysts assume roles of leaders who are subject of analysis, act out their responses to developments.
Red Hat Analysis: Try to perceive threats and opportunities as adversary/competitor sees them. Best done by those with cultural understanding of adversary/competitor.
Outside-In Thinking: Identify global, political, environmental, technological, economic, social, legal forces and trends outside your area of expertise, which could profoundly affect situation.
Challenge Analysis
Premortem Analysis: Imagine yourself looking back from future in which your analysis has proven wrong, and figure out why. This can help identify potential failure before it occurs.
What If? Analysis: Imagine that an event has occurred, consider how events could have led to it, and its consequences.
High Impact/Low Probability Analysis: Imagine a high-impact, low-probability event has occurred, figure out how, and plan ahead for it.
Devil’s Advocacy: Make the best possible case against a proposed judgment/plan/decision.
Red Team Analysis: Create team with substantive, cultural, or analytical skills to challenge conventional wisdom about how adversary/competitor thinks.
Decision Support
Decision Trees: Chart a range of options, estimate probability of each, show likely outcomes.
Decision Matrix: List options, criteria for judging them, weights assigned to criteria, and evaluation of extent to which each option satisfies criteria.
SWOT Analysis: List strengths, weaknesses, opportunities, threats.
Impact Matrix: Identify key actors involved in a decision, their level of interest, impact of decision on them.
Practitioner’s Guide to Collaboration
Even a single dissenting opinion makes a group’s decisions more nuanced and its decision-making process more rigorous, regardless of whether dissenter is correct.
