This is actually currently my biggest problem with combat. More so then clunkiness, wonky parrying and blocking, and weapon imbalance. I personally don't do this hit and run tactic anymore because I realized how cheap it is. If you time it right, you can swing, unequip, and have your ghost swing hit while you are out of combat and running.
I think one solution would be, like you said, is slowing down unequipping (and maybe the re-equip speed), as well as tweaking "coward" so it only effects people out of combat/ fleeing, while forcing those who are in combat or who go into combat mode to stay so for the duration of the effect. Also, coward should make those fleeing not only twitch but also fall down in order to encourage actually standing and fighting.