A naked rider on a horse can breach most defenses, be it wooden palisades or stone walls. When he land he quickly equips his armor again.
This is made possible by the combination of speed + jump height.
I suggest you cut the jumping height of horses to half.
Whats the point of building palisades and walls if everyone can jump over them as simple as that?
P.S Also remove the ability to use Pavise as a ladder that you can lean on the walls.