Stash extra pacifiers in crib corners
Kurt's hack brought back memories -- we did just this when our son was a baby:
If your child sleeps with a pacifier she probably also wakes up from time to time crying because the pacifier is no longer in her mouth and she can’t find it (or isn’t looking). Out of bed you get to check on her. No matter how good your night vision is it can be difficult to locate the missing pacifier. Is it under the blanket? Behind Elmo?
Try placing your extra pacifiers in the corners of the crib before bedtime. If you can’t immediately locate the one she fell to sleep with, just reach for a corner and grab a replacement. Unless your child is really attached to a particular pacifier she won’t care that it’s not the same one she fell asleep with.