"A smart man would've known I was on to him."