"The PM and Foreign Secretary aren't keen"