I think the farming yields should slowly degrade after years of farming.
This could be countered by composting byproducts.
Nearly every product could have a byproduct (fish guts, animal guts, egg shells)= your typical nitrogen or "green" byproducts. These can be collected from houses and taken to the compost pile, which also can take your farms or lumber yards byproducts = phosphorus and potassium or "brown" byproducts. The compost pile could be mixed, at your desired amount, to initiate a timer from which it would become active fertilizer.
This would require a dedicated job. Someone who picks up byproducts with wheel barrel and drops off at pile. Once mixed, the farmers would pick up and spread as necessary.
What ya think?