Is it the same person coming back? Or new people? The new people thing wouldnt make sense unless they have double territories checked out or something. How it works worldwide (at least in places where we arent banned) is one person checks out a territory card and goes through the territory. If people are not home, they come back to the "not at home" houses. There is a little map and it shows where to go so places arent "double done" unless its a mistake (I've done a wrong street on accident before, when people were saying "you guys were just here yesterday" I figured something was wrong and I was on the wrong street. Oops. I said "I'm sorry" and went and re-read the card.)
Anyways, if its the same person I'm guessing you are a "return visit" which means she (he?) thought you showed interest or were otherwise friendly nice/liked to talk about the bible, etc. If you continue taking literature, they will think you still want it. I know some people who never want to read the bible, change religions, etc but they like reading the magazines and have asked me to bring those back (the Awake magazine has articles about health, animals, islands, misc stuff.) Or some elderly people without many visitors, sometimes they just like company...or have you read them a scripture and then help with their mail or something. If its the same person I think you just need to be honest with them and say "I really truly do not want to come back."
When the next person checks out the territory card (every few months they are worked in areas where there are a lot of us, like here in Southern Cali but in rural areas more like 1 x a year) a new person may come. We come back because circumstances change, people move, new ones move in, etc but to me what you are describing is a "return visit" (repeated visits from the same person.)
Ultimately we do it because we believe in following Jesus footsteps and Jesus preached. We believe that the Bible tells us to do so (Matthew 24:14, Matthew 28: 19, 20, Isaiah 61:1) I know its not always comfortable. I get pest control people here a few times a week and I honestly sometimes think "AGAIN?!" I said "no thank you" 50 times...but last week I found a bunch of carpenter ants..so when he came this last time, we chatted and I asked for more info. HAHA Not really the same thing but having been on both sides of the door, that's my closest relatable story. LOL!!!
When you called the hall did you give them your address? You might need to so they can know what congregation even covers your territory. Our halls are pretty small so we usually "share" the hall with 3+ congregations. In my area we have 3 english and 3 spanish speaking. Your territory will only be "covered" by one of those though, it's all divided up.