I've been deep in content analysis for my research — the discipline of turning language into data you can count and test — and it's given me a healthy respect for both how powerful and how treacherous that process is. When you take thousands of tweets, articles, or comments and reduce them to numbers, you gain the ability to see patterns no human could read by hand. You also, if you're not careful, launder a great deal of subjective judgement into something that looks objective because it's now a number. This post is about doing it honestly, with the two tools I've been living in: Python for the heavy lifting, SPSS for the analysis.

The core move: from words to variables

Content analysis is, at heart, one idea: take unstructured text and systematically turn it into structured variables you can analyse. Is this message positive, negative, or neutral? Does it mention a particular theme? What tone does it take? Each of those becomes a variable, each piece of text becomes a row, and suddenly a pile of language is a dataset you can run statistics on.

That transformation is genuinely powerful. It lets you ask questions across a scale no close reading could manage — "has the sentiment around this topic shifted over two years across ten thousand posts?" is answerable as data and unanswerable by hand. But every bit of that power rests on the quality of the coding — the rules by which text becomes numbers — and that's where it gets hard.

Where the treachery lives: coding is a human judgement

Here's the uncomfortable truth at the centre of it. Deciding whether a message is "positive" or "critical," whether it touches a theme or not, is a human interpretive judgement dressed up, at the end, as a clean number. And humans disagree. Show the same hundred tweets to two careful people and they will code some of them differently, because language is ambiguous and people read it through different eyes.

If you ignore that, you get a dataset that looks rigorous and is quietly built on sand. So the discipline of serious content analysis is largely the discipline of managing that subjectivity honestly:

  • Write the codebook first, and precisely. Before coding anything, define exactly what each category means, with rules and examples, so that "positive" isn't a vibe but a definition. The more precise the rules, the less room for two coders to drift apart.
  • Measure your agreement. Have more than one person code the same subset, and actually compute how often they agree (inter-coder reliability). If two trained coders can't agree using your codebook, the codebook is too vague — and no amount of downstream statistics will fix data that unreliable at the source.
  • Only then, scale up. Once the rules are tight enough that humans agree, you can trust the coding — and then automate the grind.

Where Python earns its place

This is where Python comes in, and it's a genuine force multiplier. Hand-coding ten thousand items is soul-destroying and slow. So you use Python to do the mechanical work: collecting the text, cleaning it (stripping out the noise, normalising the format), and applying the more rule-based parts of the coding at scale — counting theme keywords, applying a sentiment dictionary, flagging patterns.

But — and this is the line I hold — automation handles the parts that are genuinely rule-based, and humans stay in the loop for the parts that need judgement. A sentiment dictionary is fast and consistent and also cheerfully misreads sarcasm, negation, and context, because it doesn't understand meaning — it matches words. So I use it to handle scale, and I validate it against human-coded samples to know exactly how much to trust it. Automation that's never checked against human judgement isn't efficiency; it's confident error at scale.

Where SPSS comes in

Once the text is reliably coded into variables, the analysis itself — the testing of whether the patterns you're seeing are real or just noise — happens in SPSS. This is the more familiar statistical ground: relationships between variables, significance, the actual questions the whole exercise was for. It's almost the easy part by comparison, precisely because all the hard, honest work happened upstream in the coding.

The statistics are the easy bit. The integrity of a content analysis is decided long before, in how honestly you turned messy human language into tidy numbers.

A quick example of the trap

To make the dictionary problem concrete: take the sentence "Oh, brilliant, another reorganisation." A dictionary sees "brilliant," scores it positive, and files it as a contented employee. A human reads the sarcasm instantly and knows it means the opposite. Negation trips tools the same way — "not a great experience" contains "great," and a naïve counter cheerfully marks it positive.

These aren't rare edge cases. Sarcasm, negation, and context are everywhere in real human writing, and especially in how people talk about work. Which is exactly why the automated pass has to be validated against human-coded samples rather than trusted on faith — so you know, with an actual number, how often it's quietly getting things backwards. Speed you haven't checked isn't speed; it's confident error, delivered faster.

Why this stays with me

What content analysis has drummed into me is a lesson I suspect reaches well beyond research: a number is only ever as trustworthy as the judgement that created it. When you see a clean figure — "68% positive sentiment" — the cleanliness hides a chain of human decisions about what "positive" means and how consistently it was applied. Forget that chain and you'll trust the number more than it deserves.

That's a habit of suspicion I don't think I'll ever lose now: whenever I meet a tidy dataset, I want to know how the tidiness was manufactured — because the making of the data is where its honesty is decided, long before anyone runs a test on it.