Cost-Aware Iteration Allocation for Sequential Multi-Fidelity First-Order Optimization

Optimization can be expensive when evaluating the objective and its gradient requires a large dataset or a costly simulation. A cheaper approximation (i.e., a surrogate) can provide a useful initial point, but the work spent on that surrogate must be justified by the saving on the original problem.