This could be solved by simply reordering the fields, but it
is more robust to move the atomics into their own structure,
and the extra indirection should not cost much.