What is Unfalsifiability?

The typology is often accused of being unfalsifiable. This is of course quite a significant claim, and it’s worth considering. But first, we need to consider what unfalsifiability is and why it matters.

The main way to move forward epistemologically is to test various hypotheses. For this to work, we need experiments to distinguish the true hypotheses from the false ones. This requires us to be able to test the hypotheses and possibly end up with the result that they are false – that is, the theories must be falsifiable.

What does an unfalsifiable hypothesis look like, then? The obvious guess is that it’s a theory that permits everything. However, that’s not quite true. For example, suppose we were trying to understand some sequence of bits. In that case, a theory that permits everything might assign equal probability to each possible sequence. An opposing theory might say that the sequence is alternating between 0 and 1. How do we test which one is true?

Well, we observe the sequence. If it goes “0101010101…”, the opposing theory consistently makes correct predictions, whereas the theory that permits everything… well, it doesn’t get it wrong, but it assigns much lower probability to the correct options than the opposing theory, so it rapidly loses credibility.

Of course, it might have been right. Perhaps the sequence is highly unpredictable. In that case, the opposing theory would quickly get disproven, whereas the one that permits everything would survive.

In practice, we can never predict anything perfectly, so the opposing theory would probably assign some moderate probability to non-perfect flips, so that “010110101…” or “0101011101…” are also quite likely. However, this would still not make it unfalsifiable, as it would still have to compete with the other theories.

But if this is not what an unfalsifiable hypothesis looks like, then what is? I’m inclined to say that unfalsifiability is not a property of specific hypotheses, but instead of families of hypotheses. Generally, you don’t work with a specific hypothesis, because there are parameters that you are still uncertain about; instead you work with a parameterized hypothesis which depends on some variable. For example, you might have a family which says that 1’s occur stochastically at a rate of X. This depends on X and so is not a specific hypothesis yet.

Usually, this is no problem. The family I mentioned above can often be falsified by finding a better predictor than this, but of course it is more general than the original one. The problem arises when you allow the parameter to fit the hypothesis to arbitrary data.

As a toy example, you might say that the sequence begins with some specific sequence of bits, followed by stochastic values. This family can fit to any data that you see, making it impossible to test. Of course, real-world examples are hardly as transparent as this, but the basic pattern is the same: they fundamentally change their predictions to exactly match the data, regardless of how justified this is. For this to work, they often stay vague about their mechanism, so it becomes difficult to truly call them out on it.

So, there’s assertions which work based on noise, which are fully falsifiable, and assertions which work based on overfitting, which are not. How can we recognize the difference? I have some thoughts:

First of all, I think the ones which invoke noise still tend to have consistent macroscopic patterns. Noise is local, it only affects the individual that it’s relevant for, so a focus on general patterns instead of specific instances is important. Of course, the noise may be a result of specific repeating phenomena that it’s useful to study, but a failure to mention specific big patterns that clearly arise from the model is a red flag.

Secondly, as a followup to the above, the anecdotes must be thought about in the context of the greater patterns, so a big focus on specific contradictory anecdotes is dangerous. This is especially true if the anecdotes focus a lot on self-reports, as these can be unreliable. Data is more important than cherry-picked exceptions.

Third, it’s much easier to be unfalsifiable if you’re vague about your mechanism, because it makes things much harder to call out.

Fourth, those who have unfalsifiable theories will be hostile to empirical research, since it ends up requiring a lot of work and propaganda to refit things.

I would argue that the trans typology does quite well at avoiding these pitfalls; the explanation for exceptions to the typology are usually more like noise than unfalsifiability.

On the other hand, the competitor, which I’ve started calling Magical Innate Gender Identity (coined by my friend Trent), doesn’t do very well. There is hardly any focus on the macroscopic patterns, and the patches that get applied to MIGI are to a large degree about fixing its prediction failures there, rather than being relevant to specific individuals. The use of questionable anecdotes with unknown statistical support is heavy, and there’s very little consideration for the accuracy of them. The mechanism is incredibly vague, and it has little empirical basis. That is just not a workable starting point!

Of course, there is a very important counterargument here. It goes like this: If we don’t know the true mechanism of transness, then of course it looks like we’re working from an unfalsifiable starting point, but really we’re just looking for enough data to justify the conclusions. I take that argument seriously, and I’ve made it myself before I became convinced of the typology, but I think the effectiveness of the theory makes it dubious. If this was true, then the typology would not be as powerful as it is.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s