You can even place a backdoor on a circuit without any noticeable physical changes (other than chemical differences which are extremely hard to detect). You basically change which chemicals are used to "dope" the transistors, which changes their polarity.
See: https://news.ycombinator.com/item?id=26860715