When Standard Errors Should(n’t) Be Zero

Why do we report statistical significance if we analyse the effect of X on Y across all states in US? In theory, one does not sample 50 states from the pool of states available. Rather, one uses all states — i.e. the whole `population’ of states. If standard errors inform about uncertainty surrounding the point estimate in population samples, what uncertainty is left when we have `sampled’ the whole population? Why standard errors shouldn’t be 0?

The straightforward answer is that we assume there is a superpopulation. We assume that although we have `sampled’  the whole population, there is, for example, a future population that we have not sampled and as such our initial population becomes a sample of this increasingly infinite population. Hence, the standard errors represent the uncertainty of the point estimate, by accounting for this hypothetical alternative population, which may be different from our initial population, say, because of time-variant factors.

A similar problem can be stated in the potential outcomes framework, where we assume these are independently and identically drawn (iid) from an infinite population.