1. Exquisite Tweets from @AndrewJDBell

    Woody_WongECollected by Woody_WongE

    New paper with @danjholman and Kelvyn Jones out now in Methodology journal – and #openaccess as well! The paper focuses on #intersectionality, but the message of it actually applies to all multilevel models… so here’s a nerdy methods thread econtent.hogrefe.com/doi/abs/10.102… (1/16)

    Reply Retweet Like

    AndrewJDBell

    Andrew Bell

    In a MLM, we are often interest in what higher-level units are doing. Eg are schools different from each other? Or neighbourhoods? Or countries? Or different intersections of society? If so, which schools? Which nhoods? (2/16)

    Reply Retweet Like

    AndrewJDBell

    Andrew Bell

    To do this, we might look at a model’s random effects – but these might be unreliable, eg if there aren’t many observations in a given unit. So a neighbourhood might appear high performing in some way, but that’s just because there aren’t many measurements of people in it. (3/16)

    Reply Retweet Like

    AndrewJDBell

    Andrew Bell

    Fortunately, multilevel models / random effects use Bayesian shrinkage – if higher level units are small, these will be shrunk closer to the mean – ie the random effect will be closer to zero than its raw effect. (4/16)

    Reply Retweet Like

    AndrewJDBell

    Andrew Bell

    This shrinkage is determined by two other things too: (1) how much variance there is between higher level units, and (2) how much variance there is within those units. (5/16)

    Reply Retweet Like

    AndrewJDBell

    Andrew Bell

    If there’s lots of variance *between* higher level units, that implies there are generally real differences between places. So we don’t want as much shrinkage. (6/16)

    Reply Retweet Like

    AndrewJDBell

    Andrew Bell

    Conversely, if there’s lots of variance *within* higher level units, that suggests actually differences between units might be more to do with chance differences of composition. So we want more shrinkage. (7/16)

    Reply Retweet Like

    AndrewJDBell

    Andrew Bell

    The problem arises if there is some omitted variable not considered. So let’s say our higher level units are schools, and we have state-funded and private-funded schools in our sample. (8/16)

    Reply Retweet Like

    AndrewJDBell

    Andrew Bell

    There will be lots of between-school variance because of the difference between private and state schools, meaning less shrinkage. But if we were to control for school type, the variance of those random effects would reduce, and so the shrinkage would increase. (9/16)

    Reply Retweet Like

    AndrewJDBell

    Andrew Bell

    Which is correct? Well, probably the second one controlling for schools – because there are real differences between the two groups. The sample of schools isn’t random – it’s split in two. (10/16)

    Reply Retweet Like

    AndrewJDBell

    Andrew Bell

    If we didn’t control for that variable, we are saying we are actually more certain about the differences between the schools than we actually are. So we might end up saying schools are different from each other when actually they aren’t. (11/16)

    Reply Retweet Like

    AndrewJDBell

    Andrew Bell

    Even 2 state schools might appear different in the model without controlling for school type, when that difference is actually down to chance, because not enough shrinkage will happen because of the state school/private school differences. (12/16)

    Reply Retweet Like

    AndrewJDBell

    Andrew Bell

    This applies to intersectionality studies (eg sciencedirect.com/science/articl…) as shown in the paper, where the higher level is social intersection. These intersections aren’t a random sample – they are divided by the vars making up the intersections (gender, age, ethnicity etc) (13/16)

    Reply Retweet Like

    AndrewJDBell

    Andrew Bell

    A fixed effects approach (dummy variables for each intersection) is even worse as it has no shrinkage at all. So what's the solution?, think about what might be causing differences between your higher-level units, and control for them. (14/16)

    Reply Retweet Like

    AndrewJDBell

    Andrew Bell

    So w/intersectionality, control for the main effect of gender etc, and (if there are still diffs between intersections) 1st order interactions, etc Or in education, control for the variables making the big differences between schools – state/private, or whatever else. (15/16)

    Reply Retweet Like

    AndrewJDBell

    Andrew Bell

    Without doing so, the residuals might show bigger differences than there really are. Shrinkage is useful, and some is better than none (as you’d get if using dummy variables / fixed effects in a regression). But it relies on assumptions that we know are often imperfect (16/16)

    Reply Retweet Like

    AndrewJDBell

    Andrew Bell

    Ps, if you want to know more about how shrinkage works, this is a good explainer bristol.ac.uk/cmm/learning/v…

    Reply Retweet Like

    AndrewJDBell

    Andrew Bell