I suppose I don't follow. Strong Wolfe conditions do not require convexity in order to guarantee convergence. If I recall correctly, the requirement for convergence is that the gradient be Lipschitz continuous, not convexity, which should be Theorem 3.2 in Nocedal and Wright's book that you referenced.
As far as expense, yes, there is a cost for a line-search. However, machine learning algorithms typically use automatic differentiation (back-propogation) in order to calculus the derivative. This is far more expensive than a function evaluation. As such, in optimization, we generally try to use cheaper computations such as a function evaluation to better capitalize on expensive calculations like the gradient. Essentially, it seems like using a learning rate rather than doing a line search means we're wasting a huge amount of information and doing something that's not guaranteed to converge even if the underlying functions were convex, which they're not.
As far as expense, yes, there is a cost for a line-search. However, machine learning algorithms typically use automatic differentiation (back-propogation) in order to calculus the derivative. This is far more expensive than a function evaluation. As such, in optimization, we generally try to use cheaper computations such as a function evaluation to better capitalize on expensive calculations like the gradient. Essentially, it seems like using a learning rate rather than doing a line search means we're wasting a huge amount of information and doing something that's not guaranteed to converge even if the underlying functions were convex, which they're not.