When running with OPTEE, the MMU table in u-boot does not remove the OPTEE
memory from its settings. So ARM speculative prefetch in u-boot may access
that OPTEE memory. Due to trust zone is enabled by OPTEE and that memory
is set to secure access, then the speculative prefetch will fail and cause
various memory issue in u-boot.
The fail address register and int_status register in trustzone has logged
that speculative access from u-boot.