what makes bytes so difficult to do?

[buran]

Have a look at this related issue and especially at the link to mailing list discussion in the last post.

Guido's opinion on whether int() and float() should accept bytes:

Yeah, practicalibty beat purity on that one. I'd say let it beat purity on int() and float() as well.

In other words int() and float() implementation in this respect should be considered the exception, not the norm.