(phGSrSSKrSSKJr SSKJrJr SSKJ r /SQr "SS5r "S S \ 5r "S S \ 5r "S S\ 5rg)z@Hessian update strategies for quasi-Newton optimization methods.N)norm)get_blas_funcs issymmetric)warn)HessianUpdateStrategyBFGSSR1c6\rSrSrSrSrSrSrSrSr Sr g ) r afInterface for implementing Hessian update strategies. Many optimization methods make use of Hessian (or inverse Hessian) approximations, such as the quasi-Newton methods BFGS, SR1, L-BFGS. Some of these approximations, however, do not actually need to store the entire matrix or can compute the internal matrix product with a given vector in a very efficiently manner. This class serves as an abstract interface between the optimization algorithm and the quasi-Newton update strategies, giving freedom of implementation to store and update the internal matrix as efficiently as possible. Different choices of initialization and update procedure will result in different quasi-Newton strategies. Four methods should be implemented in derived classes: ``initialize``, ``update``, ``dot`` and ``get_matrix``. The matrix multiplication operator ``@`` is also defined to call the ``dot`` method. Notes ----- Any instance of a class that implements this interface, can be accepted by the method ``minimize`` and used by the compatible solvers to approximate the Hessian (or inverse Hessian) used by the optimization algorithms. c[S5e)wInitialize internal matrix. Allocate internal memory for storing and updating the Hessian or its inverse. Parameters ---------- n : int Problem dimension. approx_type : {'hess', 'inv_hess'} Selects either the Hessian or the inverse Hessian. When set to 'hess' the Hessian will be stored and updated. When set to 'inv_hess' its inverse will be used instead. z=The method ``initialize(n, approx_type)`` is not implemented.NotImplementedErrorselfn approx_types Z/var/www/html/venv/lib/python3.13/site-packages/scipy/optimize/_hessian_update_strategy.py initialize HessianUpdateStrategy.initialize%"#9: :c[S5e)Update internal matrix. Update Hessian matrix or its inverse (depending on how 'approx_type' is defined) using information about the last evaluated points. Parameters ---------- delta_x : ndarray The difference between two points the gradient function have been evaluated at: ``delta_x = x2 - x1``. delta_grad : ndarray The difference between the gradients: ``delta_grad = grad(x2) - grad(x1)``. z>The method ``update(delta_x, delta_grad)`` is not implemented.rrdelta_x delta_grads rupdateHessianUpdateStrategy.update7rrc[S5e)Compute the product of the internal matrix with the given vector. Parameters ---------- p : array_like 1-D array representing a vector. Returns ------- Hp : array 1-D represents the result of multiplying the approximation matrix by vector p. z)The method ``dot(p)`` is not implemented.rrps rdotHessianUpdateStrategy.dotIs"#9: :rc[S5e)zReturn current internal matrix. Returns ------- H : ndarray, shape (n, n) Dense matrix containing either the Hessian or its inverse (depending on how 'approx_type' is defined). z0The method ``get_matrix(p)`` is not implemented.r)rs r get_matrix HessianUpdateStrategy.get_matrixZs"#9: :rc$URU5$N)r$r"s r __matmul__ HessianUpdateStrategy.__matmul__gsxx{rN) __name__ __module__ __qualname____firstlineno____doc__rrr$r'r+__static_attributes__r-rrrr s 2:$:$:" :rrcp\rSrSrSr\"SSS9r\"SSS9r\"SSS9rSSjr S r S r S r S r S rSrSrg)FullHessianUpdateStrategykzKHessian update strategy with full dimensional internal representation. syrddtypesyr2symvcHXlSUlSUlSUlSUlgr*) init_scalefirst_iterationrBH)rr>s r__init__"FullHessianUpdateStrategy.__init__ss'$ $rcSUlXlX lUS;a [S5eURS:Xa[R "U[ S9Ulg[R "U[ S9Ulg)r T)hessinv_hessz+`approx_type` must be 'hess' or 'inv_hess'.rEr9N) r?rr ValueErrornpeyefloatr@rArs rr$FullHessianUpdateStrategy.initialize|s^ $& 2 2JK K   v %VVAU+DFVVAU+DFrc[R"X5n[R"X"5n[R"[R"X!55nUS:Xd US:XdUS:XagURS:XaXE- $XT- $)NrrE)rHr$absr)rrrs_norm2y_norm2yss r _auto_scale%FullHessianUpdateStrategy._auto_scalesk&&*&&0 VVBFF:/ 0 91 1    v %< < rc[S5e)Nz9The method ``_update_implementation`` is not implemented.rrs r_update_implementation0FullHessianUpdateStrategy._update_implementations!#9: :rc[R"US:H5(ag[R"US:H5(a[S[SS9 gUR(Ga[ UR [5(a"UR S:XaURX5nO UR nSn[R"U5S:Xa [U5nGO[R"U5(a [S 5eS nURS :Xa7[R"UR5nURR nO6[R"UR"5nUR"R n[R$"X6S S 9nU[R"U5=n:wa['S USUS35e[)U5(d ['S5eURS :Xa$U(aX0lO9U=RU-slO#U(aX0lOU=R"U-slSUlUR+X5 g)rrMNzdelta_grad == 0.0. Check if the approximated function is linear. If the function is linear better results can be obtained by defining the Hessian as zero instead of using quasi-Newton approximations.) stacklevelautoFrNz3init_scale contains complex elements, must be real.TrE)r:copyzMIf init_scale is an array, it must have the dimensions of the hess/inv_hess: z. Got .z}If init_scale is an array, it must be symmetric (passing scipy.linalg.issymmetric) to be an approximation of a hess/inv_hess.)rHallr UserWarningr? isinstancer>strrSsizerJ iscomplexobj TypeErrorrshaper@r:rAarrayrGrrV)rrrscalereplacerer: init_shapes rr FullHessianUpdateStrategy.updates 66'S. ! !  66*# $ $ #   ,     $//3//DOOv4M((= Gwwu~"e ''!011##v-HHTVV,E FFLLEHHTVV,E FFLLE$?288E?:Z;$&IINP--7L&;<<#5))$&STT6)"FFFeOF"FFFeOF#(D  ##G8rcURS:XaURSURU5$URSURU5$)r!rErN)r_symvr@rAr"s rr$FullHessianUpdateStrategy.dots@   v %::a+ +::a+ +rcURS:Xa![R"UR5nO [R"UR5n[R "USS9nUR UX'U$)zReturn the current internal matrix. Returns ------- M : ndarray, shape (n, n) Dense matrix containing either the Hessian or its inverse (depending on how `approx_type` was defined). rE)k)rrHr\r@rAtril_indices_fromT)rMlis rr'$FullHessianUpdateStrategy.get_matrix sZ   v %AA  ! !!r *Br)r@rArr?r>rN)r[)r.r/r0r1r2r_syr_syr2rlrBrrSrVrr$r'r3r-rrr5r5ksP %s +D 6 -E 6 -E,4  :N9`,&rr5cF^\rSrSrSrSU4SjjrSrSrSrSr U=r $) riaBroyden-Fletcher-Goldfarb-Shanno (BFGS) Hessian update strategy. Parameters ---------- exception_strategy : {'skip_update', 'damp_update'}, optional Define how to proceed when the curvature condition is violated. Set it to 'skip_update' to just skip the update. Or, alternatively, set it to 'damp_update' to interpolate between the actual BFGS result and the unmodified matrix. Both exceptions strategies are explained in [1]_, p.536-537. min_curvature : float This number, scaled by a normalization factor, defines the minimum curvature ``dot(delta_grad, delta_x)`` allowed to go unaffected by the exception strategy. By default is equal to 1e-8 when ``exception_strategy = 'skip_update'`` and equal to 0.2 when ``exception_strategy = 'damp_update'``. init_scale : {float, np.array, 'auto'} This parameter can be used to initialize the Hessian or its inverse. When a float is given, the relevant array is initialized to ``np.eye(n) * init_scale``, where ``n`` is the problem dimension. Alternatively, if a precisely ``(n, n)`` shaped, symmetric array is given, this array will be used. Otherwise an error is generated. Set it to 'auto' in order to use an automatic heuristic for choosing the initial scale. The heuristic is described in [1]_, p.143. The default is 'auto'. Notes ----- The update is based on the description in [1]_, p.140. References ---------- .. [1] Nocedal, Jorge, and Stephen J. Wright. "Numerical optimization" Second Edition (2006). c>US:XaUbX lO+SUlO#US:XaUbX lOSUlO [S5e[TU] U5 Xlg)N skip_update:0yE> damp_updateg?z<`exception_strategy` must be 'skip_update' or 'damp_update'.) min_curvaturerGsuperrBexception_strategy)rrr}r> __class__s rrB BFGS.__init__Bsb  .(%2"%)" = 0(%2"%("12 2 $"4rcURSU- XBURS9UlURX-US-- X@RS9Ulg)aUpdate the inverse Hessian matrix. BFGS update using the formula: ``H <- H + ((H*y).T*y + s.T*y)/(s.T*y)^2 * (s*s.T) - 1/(s.T*y) * ((H*y)*s.T + s*(H*y).T)`` where ``s = delta_x`` and ``y = delta_grad``. This formula is equivalent to (6.17) in [1]_ written in a more efficient way for implementation. References ---------- .. [1] Nocedal, Jorge, and Stephen J. Wright. "Numerical optimization" Second Edition (2006). arYN)rwrArv)rrRHyyHyss r_update_inverse_hessianBFGS._update_inverse_hessianUsE"D2Iq7BHa/ff=rcURSU- X@RS9UlURSU- X RS9Ulg)aTUpdate the Hessian matrix. BFGS update using the formula: ``B <- B - (B*s)*(B*s).T/s.T*(B*s) + y*y^T/s.T*y`` where ``s`` is short for ``delta_x`` and ``y`` is short for ``delta_grad``. Formula (6.19) in [1]_. References ---------- .. [1] Nocedal, Jorge, and Stephen J. Wright. "Numerical optimization" Second Edition (2006). g?rrN)rvr@)rrRBssBsys r_update_hessianBFGS._update_hessianis<38Q&&14#:rVV4rcURS:XaUnUnOUnUn[R"X45nX-nURU5nUS::aURX5nURS:Xa,U[R"UR [ S9-UlO+U[R"UR [ S9-UlX-nURU5nXPRU-::a[URS:XagURS:Xa:SUR- SXW- - - n X-SU - U--n[R"X45nURS:XaURXVXt5 gURXVXt5 g)NrErMr9rzr|rN) rrHr$rSrIrrJr@rAr}rrr) rrrwzwzMwwMwrg update_factors rrVBFGS._update_implementation{sU   v %AAAA VVA\ XffQi #:$$W9E6)e!<<e!<<B&&)C ##c) )&&-7((M9!"4#5#5!5!bf* E !Oq&::VVA\   v %   0  ( ( 8r)r@rArr})rzNr[) r.r/r0r1r2rBrrrVr3 __classcell__rs@rrrs,"HHL"5&>(5$+9+9rrc6^\rSrSrSrSU4SjjrSrSrU=r$)r iaSymmetric-rank-1 Hessian update strategy. Parameters ---------- min_denominator : float This number, scaled by a normalization factor, defines the minimum denominator magnitude allowed in the update. When the condition is violated we skip the update. By default uses ``1e-8``. init_scale : {float, np.array, 'auto'}, optional This parameter can be used to initialize the Hessian or its inverse. When a float is given, the relevant array is initialized to ``np.eye(n) * init_scale``, where ``n`` is the problem dimension. Alternatively, if a precisely ``(n, n)`` shaped, symmetric array is given, this array will be used. Otherwise an error is generated. Set it to 'auto' in order to use an automatic heuristic for choosing the initial scale. The heuristic is described in [1]_, p.143. The default is 'auto'. Notes ----- The update is based on the description in [1]_, p.144-146. References ---------- .. [1] Nocedal, Jorge, and Stephen J. Wright. "Numerical optimization" Second Edition (2006). c0>Xl[TU] U5 gr*)min_denominatorr~rB)rrr>rs rrB SR1.__init__s. $rcURS:XaUnUnOUnUnX-nXE- n[R"X65n[R"U5UR[ U5-[ U5-::agURS:Xa#UR SU- X`RS9UlgUR SU- X`RS9Ulg)NrErNr) rrHr$rOrrrvr@rA)rrrrrr z_minus_Mw denominators rrVSR1._update_implementations   v %AAAA XV ffQ+  66+ $"6"6tAw">tJ?O"O O    v %YYq}jFFYCDFYYq}jFFYCDFr)r@rAr)r{r[) r.r/r0r1r2rBrVr3rrs@rr r s:%DDrr )r2numpyrH numpy.linalgr scipy.linalgrrwarningsr__all__rr5rr r-rrrsWF4 3]]@o 5odI9 $I9X6D #6Dr