Aylin's claim that Meacock conclusions are "deeply flawed" is deeply flawed

Aylin and colleagues have written a public google docs document to challenge the conclusions of a peer-reviewed study by Meacock and colleagues.
The latter, titled “Higher mortality rates amongst emergency patients admitted
to hospital at weekends reflect a lower probability of admission”, seemed to
show that the “week-end effect” was an illusion. Although a slightly higher
proportion of patients admitted at week-ends die within 30 days, this is not
the case if one considers all patients attending A&E. Looking at
attendances, the proportion of people who die does not vary between week-end
and weekday. The obvious interpretation is that the patients who are admitted
at week-ends are the ones who are more likely to die, presumably because they are more unwell to start with.

Aylin’s grounds for saying that the conclusions are “deeply
flawed” rest on a rather subtle statistical point which is that if one looks at
the “risk adjusted relative risk” for direct admissions then it is higher for
the week-end admissions than for the non-adjusted risk. What the adjustments
try to do is to correct for known risk factors. So if the adjusted risk
(relative to these known risks) goes up relative to the non-adjusted risk then
it implies that the correcting factors would have suggested that people should
have a lower risk than they actually do. Now, if the risk adjusted relative
risk increases for week-end admissions then this implies that those admissions
had less in the way of risk factors. According to Aylin, it shows they were
less sick. This goes against the inference of the Meacock paper that people who
get admitted at week-ends are more sick and hence more likely to die.

I have a couple of things to say about this. Firstly, the
quantitative difference is very small. The crude relative risk is 1.15 and the adjusted
relative risk is 1.21. Secondly, all that one may be seeing is that the
adjustments don’t work very well. If the wrong factors are being included with
the wrong weights then these figures are meaningless. So although these figures
don’t provide independent support to the notion that week-end admissions are
more unwell I certainly don’t accept that they prove the opposite.

In fact, these figures are only for “direct admissions”.
When one does a similar exercise for A&E attendances one finds the adjusted
and non-adjusted relative risks to be identical, at 1.05. Again, no independent
evidence that week-end attenders are sicker, but not proof that they are not.

Aylin points to other factors in the appendix suggesting
that week-end patients might be less unwell, in particular that fewer are over
65 and that they have fewer comorbid conditions. However I would counter that
there are glaring differences that he does not draw attention to. During the
week, 7.32% of A&E attenders are referred by their GP and 60.3% arrive by
ambulance. At week-ends the corresponding figures are 2.34% and 64.03%. These
are big differences. A more than three-fold difference in GP referrals, 5% in
absolute terms. GP referrals are when somebody attends their GP and their GP
wants a second opinion. These are not the acutely unwell patients who have to
attend A&E by ambulance because they are too unwell to get there any other way.
For whom there is a 4% excess among week-end attendances.

So what do we conclude? Certainly Aylin’s critique does not
in fact amount to a comprehensive demolition of the Meacock paper. The Meacock
conclusions are not “deeply flawed”. All it tells me is that logistic
regression does not work very well.

My opinion is that there is no evidence that a patient
admitted at the week-end is more or less likely to die because of anything to
do with the care they receive. In fact, everything I have looked at suggests the
opposite. I don’t think that the "week-end effect" has been shown to be real.