Actually that's not true, LiF is offering a lot of freedom in skilling your character in any way. You can make a hybrid of sort, you can level up War horse handling, then Chainmail armor skills and forget about other cavalry skills. So you will probably wanna spend max 180 (90 + 90) points in these and you can spend rest in whatever skills you want. So basicly you can have that swordsman infantry in chain armor, or for that matter an archer in plate armor.
There are lots of combinations you can try, and remember that you can always downgrade your skills and learn different ones, the only limit here is your statcap, which can be slightly increased with your base stats rising.