"It would have been totally within my rights"