‘It’s a disgrace that housing workers live in worse conditions than those they are trying to house’