The threshold for where the line is drawn in whether a guardian is harming or neglecting the incompetent is arbitrary.
For example, I would at least draw the line at whether an action causes "immediate permanant damage". But then even in certain cultures immediate permanant damage is considered acceptable... such as male genital mutilation, or the injection of aluminum causing immune response against oneself: random autoimmune diseases.
Personally I'd consider feeding a child a standard american diet full of sugar and grains neglect. That offering milk & historically natural paleo diet meals be the standard. But that seems pertty draconian, especially since my diet is pretty expensive, and most people haven't come to the same conclusions to me on what is good for the human digestive system & body.
To solve this I'd propose highly distributed governance on this particular issue. For example, a large multi-cultural civilization may allow for multiple guardian-incompetent contracts & arbitration companies. A civilization may then require its residents to have a contract with one of these arbitration companies in order to be eligable to be a guardian.