From the sites I have read, it appears the reference is a taunt responding to another taunt. The enemy he was trying to conquer stated even their lame and blind could defend against David and his men.

This site has a good explanation about it:

“Remember, this is what David is saying this to his men. They all knew that they had been taunted by the people of the city. They all knew that the people of the city had said that the lame and blind could keep the armies of Israel away. David is basically said, “Okay, let’s see them (their blind and lame) try it.” Basically, he was using their misplaced confidence as a motivational tactic for his men. He hated the men that taunted him, and in this case it was the men who defended the city. The men who defended this city said that the blind and lame could do it. Therefore, David was saying that if this is actually the case and the blind and the lame are really defending that wall, let them try to hold us off.

David hated them because they taunted the children of Israel and God. He did not hate them because they had disabilities, and from what I can tell, we really do not even know if there were any defenders with disabilities. It seems as if this was largely a taunt thrown down by the inhabitants of Jerusalem which was then turned into a motivational technique by David.”

Another good site addressing this question: