
Apparently, reading from dr3 is slower than reading from memory with cache hit. Also, depending on hypervisor configuration, accessing dr3 may cause a VM exit (and, at least on kvm, it does), what makes it much slower than a memory access even when there is a cache miss.