Thats because when they get shot they tend to fall of the wall or sometimes they just fall off because they cant climb that good
Have you ever tried straight up climbing up a wall? Its not easy. Not all of the Japanese are ninjas.
man you gotta move east or else you're screwed the eastern shogunate clans get so powerful and it's almost impossible to take them on