Instead of always using the largest blocksize the card supports, check
if it can support smaller block sizes and use 512 bytes if possible.
Most cards do support this, and other parts of u-boot seem to have
trouble with block sizes different from 512 bytes.