I don't know about anyone else, but this would not be permitted by me, to my home. There is no way that I will authourize a random person to enter my home while I am not there, not even for 'five minutes'. Delivering a package is NOT rocket science, and their drivers do not get vetted or background checks before working for Amazon. Also, delivery of packages can happen like they normally do - left in the doorway, or left according to my instructions, and the likelihood of someone stealing it is far less than an unethical driver stealing something from the inside of the house - like a purse or other target of opportunity.
I was thinking there should be a mail box on the outside like in the porch where it can be deposited there. I don't want anyone to have a 'key' either but there must be people going for it if its getting popular.