Realize that *_empty_fifo() and *_fill_fifo() are always called
under the card lock, so no additional protection when reading/writing
fifos is needed.