drm/xe: keep pulling mem_access_get further back