I would only worry about contamination of your equipment, since its hard to find almond butter that isn't contaminated with peanut and other tree nuts.
When dough climbs to the top of the beater so high it touches the mixer, you'd need to wipe that with a Clorox wipe (proven to get rid of peanut protein) for it to not be contaminated.
Our family des enjoy almonds very much.