Iterative roots are a valuable tool for modeling and analyzingdynamical systems. They provide a natural way toconstruct a continuous time model from discrete time data.However, they are in most cases extremely difficult tocompute analytically. Previously we have demonstratedhow to use neural networks to calculate the iterativeroots and fractional iterations of functions. We used aspecial topology of MLPs together with weight sharing.This paper shows how adding a regularization term to theerror function can direct any backpropagation basedtraining method to the same result but in a fraction of epochswhen using advanced 2-nd order learning rules.