If you lived in the UK with no TV would you allow a licence inspector in to check?
They have no legal power to come in but you can easily prove that you don't have a TV by letting them in. On the other hand it's a bit like being guilty til proven innocent.