Okay, so I won't take credit for this, I found it via the steam community forums. It may or may not have been posted here already but I can't find it, and google struggles to find this answer too.
The draw distance is set to draw pretty much the entire island in one go, thus reducing any un-needed loading but reducing FPS down to a mere crawl.
I was averaging about 10-15fps maximum whilst looking inland with every other setting on it's lowest. When I applied this fix, I could raise everything to maximum and be on 40-50fps whilst looking inland.
Now onto the part you've been waiting for.
You need to go to:
Steam\SteamApps\common\Life is Feudal Your Own\scripts\client
and open cm_environment.cs with word/notepad.
Find visibleDistance = "10000";
Set it to 200-500, depends on how you want it, The lower you set it the more you limiting your vision, 500 should be more than enough for good vision.
It's as simple as that. Just remember, the lower you set it shorter the distance you can see but the more of an FPS boost you'll get. I haven't messed around with it too much but at 200 you cannot see the top of the big mountain if you stand at the bottom of it, however it is suitable whilst roaming around the forest and such.