If Intelligence Analysts Disagree, How Reliable Are Organized Crime Threat Assessments?

AI Summary: Research question: How do criminal intelligence analysts decide what makes an organized crime group successful, and do those perceptions differ across countries and levels of government?

Bottom line: They differ substantially. Surveying more than 150 police professionals from local, state/provincial, federal, and international agencies revealed that analysts do not share a common understanding of the characteristics that make organized crime groups successful. Their perceptions varied by country and organizational level, raising important questions about the consistency of organized crime threat assessments and the strategic decisions built upon them. Read this AI generated summary for details.

Modern policing increasingly depends on intelligence. National threat assessments influence everything from funding priorities to investigative targets and operational strategies. But there is a fundamental assumption behind every intelligence-led policing model that is rarely questioned: that intelligence analysts largely agree about what makes criminal organizations dangerous.

Our research suggests that assumption deserves much closer scrutiny.

In this study, we examined whether intelligence professionals from different countries and different levels of government actually view organized crime groups in the same way. The answer was surprisingly clear—they often do not. Those differences matter because intelligence assessments are not simply academic exercises. They help determine which criminal organizations receive investigative attention and how scarce law enforcement resources are allocated.

Intelligence-Led Policing Depends on Shared Understanding

Over the past two decades, intelligence-led policing has become the dominant strategic framework in many countries. National intelligence models encourage police agencies to identify priority threats, assess criminal organizations, and coordinate responses across local, regional, and national jurisdictions.

For this model to function effectively, analysts at different organizational levels should ideally have similar perceptions of criminal threats. If local analysts see organized crime differently from federal analysts, or if analysts in different countries emphasize different characteristics of criminal organizations, coordinated strategies become far more difficult to implement. Poor communication and conflicting priorities are well-known barriers to successful strategy implementation, and intelligence-led policing is no exception.

Measuring Criminal Success

The study built on the Royal Canadian Mounted Police's Sleipnir framework, a structured methodology designed to evaluate organized crime groups using twelve characteristics associated with criminal capability and resilience.

These characteristics included factors such as organizational cohesion, corruption of public officials, use of violence, money laundering capability, collaboration with other criminal groups, intelligence use, geographic scope, infiltration of legitimate businesses, organizational discipline, and market monopoly. Rather than asking analysts to score each characteristic independently, we used a Q-sort methodology that required participants to rank the characteristics relative to one another. This forced respondents to identify what they considered genuinely important rather than simply rating everything as highly significant.

The survey was administered to more than 150 intelligence professionals attending the 2011 International Association of Law Enforcement Intelligence Analysts (IALEIA) and Law Enforcement Intelligence Units (LEIU) conference, representing local, state, federal, Canadian, American, and international agencies.

Analysts Don't See Organized Crime the Same Way

The results showed substantial variation.

American respondents viewed organizational cohesion, discipline, and violence as the most important ingredients of organized crime success. Canadian respondents, however, ranked corruption of public officials as the single most important characteristic, followed by cohesion and violence. Analysts from countries outside North America emphasized cohesion and corruption but placed considerably less importance on violence.

Differences also emerged across levels of government.

Local analysts tended to emphasize organizational discipline and violence, while state and regional analysts placed considerably greater importance on money laundering. Federal and national analysts viewed corruption as more important than did local respondents and assigned less importance to organizational discipline. Statistical analysis confirmed that many of these differences remained significant even after accounting for experience and geographic location.

These findings suggest that analysts working within different organizations are not simply prioritizing different threats—they may actually be conceptualizing organized crime differently.

Why Do These Differences Exist?

There are several possible explanations.

One possibility is that organized crime genuinely differs across countries and jurisdictions. Criminal organizations operating in one country may rely more heavily on corruption, while those elsewhere depend more on violence or organizational cohesion. If so, analysts are accurately reflecting different criminal environments.

A second explanation is organizational perspective.

Federal agencies often investigate large, sophisticated criminal enterprises that operate nationally or internationally. Local agencies are more likely to encounter violent street-level organizations that create immediate public safety concerns. Analysts working within these environments naturally develop different understandings of what criminal success looks like because they observe different parts of the organized crime landscape.

A third possibility is that training, organizational culture, and analytical methodology shape professional judgment. Analysts may reach different conclusions not because the criminal organizations differ, but because the institutions employing them encourage different ways of thinking about criminal threats.

The study could not definitively determine which explanation was correct, and the reality is likely some combination of all three.

Why This Matters

The implications extend well beyond organized crime research.

Threat assessments increasingly influence policing strategy, intelligence priorities, and resource allocation. If analysts working in different organizations consistently interpret criminal threats differently, then intelligence products may reflect organizational perspective as much as objective criminal reality.

That does not mean threat assessments are unreliable. It does suggest they should not be viewed as purely objective measurements. Every assessment is shaped by the experiences, organizational context, and analytical frameworks of the people producing it.

Recognizing those influences allows intelligence organizations to strengthen decision-making by incorporating multiple perspectives rather than assuming there is a single universally correct assessment.

The Bigger Lesson

One of the central goals of intelligence-led policing is to improve strategic decision-making through better analysis.

That goal requires not only improving analytical techniques, but also examining how analysts themselves interpret criminal problems. Intelligence products are created by professionals working within organizations, and those organizations influence what analysts notice, what they prioritize, and how they define criminal threats.

Understanding those influences is therefore not a distraction from intelligence analysis—it is an essential part of making intelligence better.

Citation

Ratcliffe, J. H., Strang, S. J., & Taylor, R. B. (2014). Assessing the success factors of organized crime groups: Intelligence challenges for strategic thinking. Policing: An International Journal of Police Strategies & Management, 37(1), 206–227.