COAP 2021 Best Paper Prize.

Since the proximal gradient method reduces to the standard steepest descent method for the particular case where HT <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mi> </mi><mo> </mo><mn>0</mn></mrow></math> ht , the rate-of-convergence is typically slow (sublinear). In principle, the proximal gradient method has the major advantage that the subproblems (1) can be solved very efficiently (even analytically) for some practically important proximal functions HT <math xmlns="http://www.w3.org/1998/Math/MathML"><mi> </mi></math> ht . The classical proximal gradient method corresponds to the choice HT <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>H</mi><mi>k</mi></msub><mo>=</mo><msub><mi> </mi><mi>k</mi></msub><mi>I</mi></mrow></math> ht for all I k i with some (penalty or line search parameter) HT <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi> </mi><mi>k</mi></msub><mo> </mo><mn>0</mn></mrow></math> ht , whereas HT <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>H</mi><mi>k</mi></msub><mo>=</mo><msup><mi mathvariant="normal"> </mi><mn>2</mn></msup><mi>f</mi><mrow><mo stretchy="false">(</mo><msup><mi>x</mi><mi>k</mi></msup><mo stretchy="false">)</mo></mrow></mrow></math> ht leads to the proximal Newton method, and HT <math xmlns="http://www.w3.org/1998/Math/MathML"><mrow><msub><mi>H</mi><mi>k</mi></msub><mo> </mo><msup><mi mathvariant="normal"> </mi><mn>2</mn></msup><mi>f</mi><mrow><mo stretchy="false">(</mo><msup><mi>x</mi><mi>k</mi></msup><mo stretchy="false">)</mo></mrow></mrow></math> ht with a (limited memory) quasi-Newton approximation of the Hessian is naturally called a (limited memory) proximal quasi-Newton method. [Extracted from the article]