Enhance device selection and logging for inference; add end-to-end tests

1386847

zyf0717 commited on Apr 21