A former taxi booking office in Blackburn's Cathedral Quarter Conservation area can continue to be used as a boxing gym.