Fiona, when I was training my dog Cody, I ended up tossing the rug outside. He kept going on a small area rug and I just tossed it outside. In doing so, it helped put the smell out there. I read someplace that dogs are attracted to the smell. They really don't remember the area visually. They use their sense of smell, and when they smell it they say "Oh! That's me! So this must be my potty spot". When, on the rare occassion he went potty someplace else in the house, I would toss the paper towels I used to clean it up out in the back yard. Eventually he formed the habit to start going out there rather than inside the house.
The cat for some reason was much harder to train. I had a hell of a time teaching him to use his cat pan.