Hey
I had the same problem with my daughter at about the same age.
I was getting up six to seven times a night to put the pacifier back in her mouth. It was getting to the point where I was sleeping less and less. So my husband and I decided to take away the pacifier from her cold turkey.
It took one night of alot of crying, but that was the worst of it. She 'forgot' her pacifier rather quickly and began sleeping much better.
They become dependant on the pacifier and once it falls out they wake up and cry for it....so this way they learn to self sooth sooner than later...which is good for long term sleeping.
Hope that helps. Good luck