Bringing migration down will disarm those who have weaponised one statistic against the government, but it's not clear that will happen, experts say.