The Foreign Secretary was approached by the activists while he was inside the restaurant on the fringes of Labour's conference.