(ph@/SQrSrSSKrSSKrSSKrSSKrSSKJrJrJ r J r J r J r SSKr SSKJrJrJr SSKJr SSKJrJrJrJr SS KJr SS KJr SS KJrJrJ r J!r!J"r" SS K#J$r$J%r% SS K&J'r' SSK(J)r* SSSSSSS.r+"SS5r,S]Sjr-"SS\ 5r."SS\/5r0Sr1Sr2Sr3\ "\ Rh"\55Rl5r7S^S jr8S_S"jr9S#r:S$r;S%rS(r?S)r@"S*S+\A5rBS,rCS`S.jrDSaS0jrE\74S1jrF\""S2S3S49\7S5SS6.S7j5rGS8rH"S9S:\A5rIS;rJSS!S<\ R\7SSSSSSS-S=S4S>jrLS!SSS<\ R\7SS/S/SSS-S=S4S?jrMS]S@jrNSS!S<\ R\7SSSSSS-SA4 SBjrOS!SSS<\ R\7SS/S/SS-SA4 SCjrPSSS!S<\7SSSSSS-S=4 SDjrQS!SSSSS<\7SS/S/S-S=4 SEjrRSbSFjrSScSGjrT"SHSI5rUSdSJjrVSeSKjrWS!S\7SSL4SMjrXSS!\7SLS4SNjrYSfSOjrZ"SPSQ\A5r[SRr\SSr]SgSTjr^S`SUjr_ShSVjr`SWraS!SXS\DS/S4SYjrb"SZS[5rcSiS\jrdg)j)fmin fmin_powell fmin_bfgsfmin_ncgfmin_cg fminboundbrentgoldenbracketrosen rosen_der rosen_hessrosen_hess_prodbrute approx_fprime line_search check_gradOptimizeResult show_optionsOptimizeWarningzrestructuredtext enN)eyeargminzerosshapeasarraysqrt)cholesky issymmetric LinAlgError)LinearOperator)line_search_wolfe1line_search_wolfe2r#LineSearchWarning)approx_derivative)getfullargspec_no_self) MapWrappercheck_random_state _RichResult_call_callback_maybe_halt_transition_to_rng)ScalarFunction FD_METHODS)array_namespace)array_api_extraz%Optimization terminated successfully.z9Maximum number of function evaluations has been exceeded.z/Maximum number of iterations has been exceeded.z=Desired error not necessarily achieved due to precision loss.zNaN result encountered.z-The result is outside of the provided bounds.)successmaxfevmaxiterpr_lossnan out_of_boundsc0\rSrSrSrSrSrSrSrSr g) MemoizeJac=zgDecorator that caches the return values of a function returning ``(fun, grad)`` each time it is called.c:XlSUlSUlSUlgN)funjac_valuex)selfr;s K/var/www/html/venv/lib/python3.13/site-packages/scipy/optimize/_optimize.py__init__MemoizeJac.__init__As c([R"XR:H5(aURb URcQ[R "U5R 5UlUR"U/UQ76nUSUlUSUlgg)Nr!r)npallr>r=r<rcopyr;)r?r>argsfgs r@_compute_if_neededMemoizeJac._compute_if_neededGspvva66k""dkk&9TXX=MZZ]'')DF!#d#B!uDHQ%DK >NrCc@UR"U/UQ76 UR$)zreturns the function value )rJr=r?r>rHs r@__call__MemoizeJac.__call__Ns )D){{rCc@UR"U/UQ76 UR$r:)rJr<rMs r@ derivativeMemoizeJac.derivativeSs )D)xxrC)r=r;r<r>N) __name__ __module__ __qualname____firstlineno____doc__rArJrNrQ__static_attributes__rCr@r7r7=s   rCr7c^TbUS;aT$[R"T5n[UR5S1:XaU4SjnO US:XaU4SjnOUS:XaU4SjnOU4SjnS UlU$) zAWrap a user-provided callback so that attributes can be attached.>tncslsqpcobylacobyqaintermediate_resultc>T"US9$)N)r_rYrescallbacks r@wrapped_callback(_wrap_callback..wrapped_callback`s4 4rC trust-constrcR>T"[R"UR5U5$r:rErGr>ras r@rdrecsBGGCEENC0 0rCdifferential_evolutioncf>T"[R"UR5UR5$r:)rErGr> convergenceras r@rdrefs BGGCEENCOO< T"[R"UR55$r:rhras r@rdreisBGGCEEN+ +rCF)inspect signatureset parametersstop_iteration)rcmethodsigrds` r@_wrap_callbackrtXsq6%II   H %C 3>>455 5 > ! 1 + + = ,',# rCc\rSrSrSrSrg)rpa Represents the optimization result. Attributes ---------- x : ndarray The solution of the optimization. success : bool Whether or not the optimizer exited successfully. status : int Termination status of the optimizer. Its value depends on the underlying solver. Refer to `message` for details. message : str Description of the cause of the termination. fun : float Value of objective function at `x`. jac, hess : ndarray Values of objective function's Jacobian and its Hessian at `x` (if available). The Hessian may be an approximation, see the documentation of the function in question. hess_inv : object Inverse of the objective function's Hessian; may be an approximation. Not available for all solvers. The type of this attribute may be either np.ndarray or scipy.sparse.linalg.LinearOperator. nfev, njev, nhev : int Number of evaluations of the objective functions and of its Jacobian and Hessian. nit : int Number of iterations performed by the optimizer. maxcv : float The maximum constraint violation. Notes ----- Depending on the specific solver being used, `OptimizeResult` may not have all attributes listed here, and they may have additional attributes not listed here. Since this class is essentially a subclass of dict with attribute accessors, one can see which attributes are available using the `OptimizeResult.keys` method. rYN)rSrTrUrVrWrXrYrCr@rrps (R rCrc\rSrSrSrg)rrYNrSrTrUrVrXrYrCr@rrrCrcBSnUbU"U5(d [S5egg)Nc^[U5(a [U5 gg![a gf=f)NTF)rrr)As r@ is_pos_def,_check_positive_definite..is_pos_defs3 q>>    s  ,,z+'hess_inv0' matrix isn't positive definite.) ValueError)Hkr~s r@_check_positive_definiters- ~"~~JK KrCcU(aKSR[[UR555n[R "SU3[ SS9 gg)Nz, zUnknown solver options:  stacklevel)joinmapstrkeyswarningswarnr)unknown_optionsmsgs r@_check_unknown_optionsrsCiiC!5!5!789  06TUV rCcn[R"U5S:H=(a [R"U5$)zJTest whether `x` is either a finite scalar or a finite array scalar. r!)rEsizeisfinite)r>s r@is_finite_scalarrs# 771:? -r{{1~-rCc`U[R:Xa*[R"[R"U55$U[R*:Xa*[R"[R"U55$[R "[R"U5U-SS9SU- -$)Nraxis?)rEinfamaxabsaminsum)r>ords r@vecnormrsn bff}wwrvvay!! wwrvvay!!vvbffQin1-c ::rCrYc [U5(aUnOU[;aSnUnOSnUnUcSnUc![R*[R4n[ XX8UXdUS9n U $)a$ Creates a ScalarFunction object for use with scalar minimizers (BFGS/LBFGSB/SLSQP/TNC/CG/etc). Parameters ---------- fun : callable The objective function to be minimized. ``fun(x, *args) -> float`` where ``x`` is an 1-D array with shape (n,) and ``args`` is a tuple of the fixed parameters needed to completely specify the function. x0 : ndarray, shape (n,) Initial guess. Array of real elements of size (n,), where 'n' is the number of independent variables. jac : {callable, '2-point', '3-point', 'cs', None}, optional Method for computing the gradient vector. If it is a callable, it should be a function that returns the gradient vector: ``jac(x, *args) -> array_like, shape (n,)`` If one of `{'2-point', '3-point', 'cs'}` is selected then the gradient is calculated with a relative step for finite differences. If `None`, then two-point finite differences with an absolute step is used. args : tuple, optional Extra arguments passed to the objective function and its derivatives (`fun`, `jac` functions). bounds : sequence, optional Bounds on variables. 'new-style' bounds are required. eps : float or ndarray If ``jac is None`` the absolute step size used for numerical approximation of the jacobian via forward differences. finite_diff_rel_step : None or array_like, optional If ``jac in ['2-point', '3-point', 'cs']`` the relative step size to use for numerical approximation of the jacobian. The absolute step size is computed as ``h = rel_step * sign(x0) * max(1, abs(x0))``, possibly adjusted to fit into the bounds. For ``jac='3-point'`` the sign of `h` is ignored. If None (default) then step is selected automatically. hess : {callable, '2-point', '3-point', 'cs', None} Computes the Hessian matrix. If it is callable, it should return the Hessian matrix: ``hess(x, *args) -> {LinearOperator, spmatrix, array}, (n, n)`` Alternatively, the keywords {'2-point', '3-point', 'cs'} select a finite difference scheme for numerical estimation. Whenever the gradient is estimated via finite-differences, the Hessian cannot be estimated with options {'2-point', '3-point', 'cs'} and needs to be estimated using one of the quasi-Newton strategies. Returns ------- sf : ScalarFunction N2-pointcgr:rY)r>rHs r@hess&_prepare_scalar_function..hesssrC)epsilon)callabler-rErr,) r;x0r<rHboundsrfinite_diff_rel_steprgradsfs r@_prepare_scalar_functionrswx}}   |  ~66'266" T,g GB IrCc^^UU4SjnU$)Nc,>[UT5nT"U5$r:) _check_clip_x)r>rfuncs r@eval_clip_x_for_func..eval/s !V $AwrCrY)rrrs`` r@_clip_x_for_funcr)s  KrCcXS:R5(dXS:R5(a9[R"S[SS9 [R "XSUS5nU$U$)Nrr!zJValues in x were outside bounds during a minimize step, clipping to boundsr)anyrrRuntimeWarningrEclip)r>rs r@rr6se 1I AY 3 3 5 5 :$ 4 GGAay&) , HrCcF[U5nURU5nURURS5(a*UR XRS5R5nUR SUSSUSSS-- S--SUSS- S--SURS 9nU$) a The Rosenbrock function. The function computed is:: sum(100.0*(x[1:] - x[:-1]**2.0)**2.0 + (1 - x[:-1])**2.0) Parameters ---------- x : array_like 1-D array of points at which the Rosenbrock function is to be computed. Returns ------- f : float The value of the Rosenbrock function. See Also -------- rosen_der, rosen_hess, rosen_hess_prod Examples -------- >>> import numpy as np >>> from scipy.optimize import rosen >>> X = 0.1 * np.arange(10) >>> rosen(X) 76.56 For higher-dimensional input ``rosen`` broadcasts. In the following example, we use this to plot a 2D landscape. Note that ``rosen_hess`` does not broadcast in this manner. >>> import matplotlib.pyplot as plt >>> from mpl_toolkits.mplot3d import Axes3D >>> x = np.linspace(-1, 1, 50) >>> X, Y = np.meshgrid(x, x) >>> ax = plt.subplot(111, projection='3d') >>> ax.plot_surface(X, Y, rosen([X, Y])) >>> plt.show() integralrgY@r!N@r)rdtype)r.risdtyperastyper)r>xprs r@r r AsT  B 1 A zz!'':&& IIaB-- . u!"#2 +c11Q3BZ#4EEQWW  &A HrCc[U5nURU5nURURS5(a*UR XRS5R5nUSSnUSSnUSSnUR U5nSX#S-- -S XBS-- -U-- SSU- -- USS&S US -USUS S-- -SSUS - -- US 'SUSUSS-- -US'U$) a The derivative (i.e. gradient) of the Rosenbrock function. Parameters ---------- x : array_like 1-D array of points at which the derivative is to be computed. Returns ------- rosen_der : (N,) ndarray The gradient of the Rosenbrock function at `x`. See Also -------- rosen, rosen_hess, rosen_hess_prod Examples -------- >>> import numpy as np >>> from scipy.optimize import rosen_der >>> X = 0.1 * np.arange(9) >>> rosen_der(X) array([ -2. , 10.6, 15.6, 13.4, 6.4, -3. , -12.4, -19.4, 62. ]) rrr!rNpr)r.rrrr zeros_like)r>rxmxm_m1xm_p1ders r@r r ts6  B 1 A zz!'':&& IIaB-- . 1RB crFE abEE -- CAX &A &+,./1r6l;C"I AaD[AaD1Q47N +a1qt8n >> import numpy as np >>> from scipy.optimize import rosen_hess >>> X = 0.1 * np.arange(4) >>> rosen_hess(X) array([[-38., 0., 0., 0.], [ 0., 134., -40., 0.], [ 0., -40., 130., -80.], [ 0., 0., -80., 200.]]) r!ndimrrrrNr)offsetrrrrrr)r) r.xpx atleast_ndrrrrcreate_diagonalrr)r>rHdiagonals r@r r s"<  B qqR(A zz!'':&& IIaB-- .  TAcrF]1 <   sQsV|B2 > ?Axx !''x2H1q.31:-1HQKHRL4!Ab'1*,,sQqrU{:HQrN s""83 33rCcl[X5n[R"USUS9nURURS5(a*UR XR S5R5nUR XRS9nURURSURS9nSUSS--S US-- S-US-S US-US-- US'S US S -US S -S SUSSS---S USS -- USS--S USS-USS -- USS&S US -US -SUS--US'U$)a Product of the Hessian matrix of the Rosenbrock function with a vector. Parameters ---------- x : array_like 1-D array of points at which the Hessian matrix is to be computed. p : array_like 1-D array, the vector to be multiplied by the Hessian matrix. Returns ------- rosen_hess_prod : ndarray The Hessian matrix of the Rosenbrock function at `x` multiplied by the vector `p`. See Also -------- rosen, rosen_der, rosen_hess Examples -------- >>> import numpy as np >>> from scipy.optimize import rosen_hess_prod >>> X = 0.1 * np.arange(9) >>> p = 0.5 * np.arange(9) >>> rosen_hess_prod(X, p) array([ -0., 27., -10., -95., -192., -265., -278., -195., -180.]) r!rrrrrrrrrNrrrr) r.rrrrrrrr)r>prHps r@rrsk>  B qqR(A zz!'':&& IIaB-- . 1GG $A !''!*AGG ,B AaD!G^cAaDj (1 ,! 4sQqTzAaD7H HBqEq"v #2&ta"gqj((312;6!Ab'ABa"g !"%&BqHAbE\AbE !C"I -BrF IrCc4^^^S/mTcTS4$UUU4SjnTU4$)Nrc.>TS==S- ss'T"[R"U5/UT-Q76n[R"U5(d'[R"U5R 5nU$U$![ [ 4an[ S5UeSnAff=f)Nrr!@The user-provided objective function must return a scalar value.)rErGisscalarritem TypeErrorr)r> wrapper_argsfxerHfunctionncallss r@function_wrapper/_wrap_scalar_function..function_wrappersq Q bggaj 9L4$7 9{{2 GZZ^((* r z* G "?@EFG Gs $A33B BBrY)rrHrrs`` @r@_wrap_scalar_functionrs/SFt|  # ##rCc\rSrSrSrg)_MaxFuncCallErrorirYNryrYrCr@rrrzrCrc8^^^^S/mTcTS4$UUUU4SjnTU4$)NrcV>TST:a [S5eTS==S- ss'T"[R"U5/UT-Q76n[R"U5(d'[R"U5R 5nU$U$![ [4an[S5UeSnAff=f)NrzToo many function callsr!r)rrErGrrrrr)r>rrrrHrmaxfunrs r@rA_wrap_scalar_function_maxfun_validation..function_wrappers !9 #$=> >q Q bggaj 9L4$7 9{{2 GZZ^((* r z* G "?@EFG Gs$BB( B##B(rY)rrHrrrs``` @r@'_wrap_scalar_function_maxfun_validationrs4SFt|" # ##rC-C6?c UUUUUU U S.n [U 5n [XU4SU 0U D6n U(a'U SU SU SU SU S4nU (aXS4- nU$U (a U SU S4$U S$) a Minimize a function using the downhill simplex algorithm. This algorithm only uses function values, not derivatives or second derivatives. Parameters ---------- func : callable func(x,*args) The objective function to be minimized. x0 : ndarray Initial guess. args : tuple, optional Extra arguments passed to func, i.e., ``f(x,*args)``. xtol : float, optional Absolute error in xopt between iterations that is acceptable for convergence. ftol : number, optional Absolute error in func(xopt) between iterations that is acceptable for convergence. maxiter : int, optional Maximum number of iterations to perform. maxfun : number, optional Maximum number of function evaluations to make. full_output : bool, optional Set to True if fopt and warnflag outputs are desired. disp : bool, optional Set to True to print convergence messages. retall : bool, optional Set to True to return list of solutions at each iteration. callback : callable, optional Called after each iteration, as callback(xk), where xk is the current parameter vector. initial_simplex : array_like of shape (N + 1, N), optional Initial simplex. If given, overrides `x0`. ``initial_simplex[j,:]`` should contain the coordinates of the jth vertex of the ``N+1`` vertices in the simplex, where ``N`` is the dimension. Returns ------- xopt : ndarray Parameter that minimizes function. fopt : float Value of function at minimum: ``fopt = func(xopt)``. iter : int Number of iterations performed. funcalls : int Number of function calls made. warnflag : int 1 : Maximum number of function evaluations made. 2 : Maximum number of iterations reached. allvecs : list Solution at each iteration. See also -------- minimize: Interface to minimization algorithms for multivariate functions. See the 'Nelder-Mead' `method` in particular. Notes ----- Uses a Nelder-Mead simplex algorithm to find the minimum of function of one or more variables. This algorithm has a long history of successful use in applications. But it will usually be slower than an algorithm that uses first or second derivative information. In practice, it can have poor performance in high-dimensional problems and is not robust to minimizing complicated functions. Additionally, there currently is no complete theory describing when the algorithm will successfully converge to the minimum, or how fast it will if it does. Both the ftol and xtol criteria must be met for convergence. Examples -------- >>> def f(x): ... return x**2 >>> from scipy import optimize >>> minimum = optimize.fmin(f, 1) Optimization terminated successfully. Current function value: 0.000000 Iterations: 17 Function evaluations: 34 >>> minimum[0] -8.8817841970012523e-16 References ---------- .. [1] Nelder, J.A. and Mead, R. (1965), "A simplex method for function minimization", The Computer Journal, 7, pp. 308-313 .. [2] Wright, M.H. (1996), "Direct Search Methods: Once Scorned, Now Respectable", in Numerical Analysis 1995, Proceedings of the 1995 Dundee Biennial Conference in Numerical Analysis, D.F. Griffiths and G.A. Watson (Eds.), Addison Wesley Longman, Harlow, UK, pp. 191-208. )xatolfatolr2r1disp return_allinitial_simplexrcr>r;nitnfevstatusallvecs)rt_minimize_neldermead)rrrHxtolftolr2r full_outputrretallrcroptsrbretlists r@rr-sN .  0Dh'H t I ID ICc(CJE CKXN  I) )G s8S^+ +s8OrCFc j[U 5 UnUn[R"U5R5n[R"UR [R 5(a UR O[Rn[R"UUS9nU (a2[[U55nSnSSU- -nSSSU-- - nSSU- - nOSnSnSnSnSnSnU bU RU RnnUU:R5(a [S 5e[R"UU:5(d[R"UU:5(a[R "S ["S S 9 U b[R$"UWW5nUc[U5n[R&"US-U4UR S9nUUS '[)U5H=n[R*"USS9nUUS :waSU-UU-UU'OUUU'UUUS-'M? O[R,"U5R/5n[R"UR [R 5(a UR O[Rn[R"UUS9nUR0S:wd#UR2S UR2SS-:wa [S5e[U5UR2S:wa [S5eUR2SnU(aUS /nUcUc US-nUS-nO[Uc+U[R4:XaUS-nO>[R4nO-Uc*U[R4:XaUS-nO[R4nU b;UW:n[R6"USU-U- U5n[R$"UWU5n[9[)SUS-55n [R:"US-4[R4[S9n![=XU5un"n[)US-5HnU"UU5U!U'M [R@"U!5n#[RB"UU#S 5n[RB"U!U#S 5n![R@"U!5n#[RB"U!U#S 5n![RB"UU#S 5nSn$U"S U:GasU$U:Gal[RD"[RF"[RH"USSUS - 555U ::a[RD"[RH"U!S U!SS- 55U ::a[R@"U!5n#[RB"UU#S 5n[RB"U!U#S 5n!U(aWRKUS 5 [MUS U!S S9n%[OUU%5(GaeGOc[RPRSUSSS 5U- n&SU-U&-UUS-- n'U b[R$"U'WW5n'U"U'5n(S n)U(U!S :aXSUU--U&-UU-US-- n*U b[R$"U*WW5n*U"U*5n+U+U(:a U*US'U+U!S'GOU'US'U(U!S'GO U(U!S:a U'US'U(U!S'OU(U!S:aNSUU--U&-UU-US-- n,U b[R$"U,WW5n,U"U,5n-U-U(::a U,US'U-U!S'OMSn)OJSU- U&-UUS--n.U b[R$"U.WW5n.U"U.5n/U/U!S:a U.US'U/U!S'OSn)U)(aOU HIn0US UUU0US - --UU0'U b[R$"UU0WW5UU0'U"UU05U!U0'MK U$S- n$[R@"U!5n#[RB"UU#S 5n[RB"U!U#S 5n!U(aWRKUS 5 [MUS U!S S9n%[OUU%5(aOU"S U:a U$U:aGMlUS n1[RT"U!5n2S n3U"S U:a-Sn3[VSn4U(a[R "U4[XS S 9 O|U$U:a-Sn3[VSn4U(a[R "U4[XS S 9 OI[VSn4U(a9[[U45 [[SU2S35 [[SU$-5 [[SU"S -5 [MU2U$U"S U3U3S :HU4U1UU!4S9n5U(aWU5S'U5$![>a GN f=f![R@"U!5n#[RB"UU#S 5n[RB"U!U#S 5n!f=f![>a GNf=f![R@"U!5n#[RB"UU#S 5n[RB"U!U#S 5n!U(aWRKUS 5 [MUS U!S S9n%[OUU%5(a GMf=f)a Minimization of scalar function of one or more variables using the Nelder-Mead algorithm. Options ------- disp : bool Set to True to print convergence messages. maxiter, maxfev : int Maximum allowed number of iterations and function evaluations. Will default to ``N*200``, where ``N`` is the number of variables, if neither `maxiter` or `maxfev` is set. If both `maxiter` and `maxfev` are set, minimization will stop at the first reached. return_all : bool, optional Set to True to return a list of the best solution at each of the iterations. initial_simplex : array_like of shape (N + 1, N) Initial simplex. If given, overrides `x0`. ``initial_simplex[j,:]`` should contain the coordinates of the jth vertex of the ``N+1`` vertices in the simplex, where ``N`` is the dimension. xatol : float, optional Absolute error in xopt between iterations that is acceptable for convergence. fatol : number, optional Absolute error in func(xopt) between iterations that is acceptable for convergence. adaptive : bool, optional Adapt algorithm parameters to dimensionality of problem. Useful for high-dimensional minimization [1]_. bounds : sequence or `Bounds`, optional Bounds on variables. There are two ways to specify the bounds: 1. Instance of `Bounds` class. 2. Sequence of ``(min, max)`` pairs for each element in `x`. None is used to specify no bound. Note that this just clips all vertices in simplex based on the bounds. References ---------- .. [1] Gao, F. and Han, L. Implementing the Nelder-Mead simplex algorithm with adaptive parameters. 2012. Computational Optimization and Applications. 51:1, pp. 259-277 rr!rg??g?gMb0?NzENelder Mead - one of the lower bounds is greater than an upper bound.0Initial guess is not within the specified boundsrrrT)rGz5`initial_simplex` should be an array of shape (N+1,N)z5Size of `initial_simplex` is not consistent with `x0`rr>r;rrr1r2r0! Current function value: f Iterations: %d! Function evaluations: %d)r;rrrr0messager> final_simplexr).rrE atleast_1dflatten issubdtyperinexactfloat64rfloatlenlbubrrrrrremptyrangearray atleast_2drGrrrwherelistfullrrargsorttakemaxravelrappendrr*addreducemin_status_messagerprint)6rrrHrcr2r1rrrrradaptiverrrrrdimrhochipsisigmanonzdeltzdelt lower_bound upper_boundNsimkyrmskone2np1fsimfcallsind iterationsr_xbarxrfxrdoshrinkxefxexcfxcxccfxccjr>fvalwarnflagrresults6 r@rrs l?+ F F r  " " $B bhh ;;BHHE Be $BCGn!C%iQ#YAcE H E #)99fii[ + % * * , ,?@ @ 66+" # #rvvb;.>'?'? MML)a 9 WWRk 2 GhhAqz2AqA$'AtqyH ad*!!CAJ mmO,113]]399bjjAA rzzjjE* 88q=CIIaLCIIaL1,<<TU U r7ciil "TU U IIaL q6(6>c'S  RVV #gGffG  bff WFVVF  KhhsAkMC/5ggc; 45AE?#G 77AE8RVV5 1D:4vNLFD%q1uA3q6lDG jjggc3"wwtS!$ **T C 774a D ''#sA CJ !9v *w"6H rxxs12wQ'7 89:eCFF266$q'DH"456%?|**T"C''#sA&C774a(Ds1v&"03q6tAw"G (3FGGG66==Sb1-1Dc'T!C#b'M1B!WWRk:r(CHT!W}#)mt+cCi#b'.AA%[+>B2h9 CG"DH CG"DHb> CG"DHT"X~#)mt3cCi#b'6II!-!#[+!FB"2h#:&(CG'*DH'(H !3w$.s2w>!-"$''#{K"HC#Cy$r(?&)CG'+DH'(H!(A%(Ves1vA.G%GCF%1)+$'FK*FA&*3q6lDG ") !OJ**T"C''#sA&C774a(Ds1v&"03q6tAw"G (3FGGHQ !9v *w"6V AA 66$ ? +j8 9 5q A B *6!9#+h!m$'1S$KIF#y Mi    jjggc3"wwtS!$P!   **T"C''#sA&C774a(Ds1v&"03q6tAw"G (3FGGHsX#d<Bf$G f< e e e  e Af f&"f)%f&&f))Bh21h2c b[R"U[5nU"U/UQ76n[XSUX4S9$)a- Finite difference approximation of the derivatives of a scalar or vector-valued function. If a function maps from :math:`R^n` to :math:`R^m`, its derivatives form an m-by-n matrix called the Jacobian, where an element :math:`(i, j)` is a partial derivative of f[i] with respect to ``xk[j]``. Parameters ---------- xk : array_like The coordinate vector at which to determine the gradient of `f`. f : callable Function of which to estimate the derivatives of. Has the signature ``f(xk, *args)`` where `xk` is the argument in the form of a 1-D array and `args` is a tuple of any additional fixed parameters needed to completely specify the function. The argument `xk` passed to this function is an ndarray of shape (n,) (never a scalar even if n=1). It must return a 1-D array_like of shape (m,) or a scalar. Suppose the callable has signature ``f0(x, *my_args, **my_kwargs)``, where ``my_args`` and ``my_kwargs`` are required positional and keyword arguments. Rather than passing ``f0`` as the callable, wrap it to accept only ``x``; e.g., pass ``fun=lambda x: f0(x, *my_args, **my_kwargs)`` as the callable, where ``my_args`` (tuple) and ``my_kwargs`` (dict) have been gathered before invoking this function. .. versionchanged:: 1.9.0 `f` is now able to return a 1-D array-like, with the :math:`(m, n)` Jacobian being estimated. epsilon : {float, array_like}, optional Increment to `xk` to use for determining the function gradient. If a scalar, uses the same finite difference delta for all partial derivatives. If an array, should contain one value per element of `xk`. Defaults to ``sqrt(np.finfo(float).eps)``, which is approximately 1.49e-08. \*args : args, optional Any other arguments that are to be passed to `f`. Returns ------- jac : ndarray The partial derivatives of `f` to `xk`. See Also -------- check_grad : Check correctness of gradient function against approx_fprime. Notes ----- The function gradient is determined by the forward finite difference formula:: f(xk[i] + epsilon[i]) - f(xk[i]) f'[i] = --------------------------------- epsilon[i] Examples -------- >>> import numpy as np >>> from scipy import optimize >>> def func(x, c0, c1): ... "Coordinate vector `x` should be an array of size two." ... return c0 * x[0]**2 + c1*x[1]**2 >>> x = np.ones(2) >>> c0, c1 = (1, 200) >>> eps = np.sqrt(np.finfo(float).eps) >>> optimize.approx_fprime(x, func, [eps, np.sqrt(200) * eps], c0, c1) array([ 2. , 400.00004208]) r)rrabs_steprHf0)rErrr%)xkrrrHrJs r@rrs8T B B 2B Q9w"& //rCseed) position_numrF)r directionrngc NUn[R"U5nSnUS:Xa[R"U"U/UQ765n U RS:a [ S5e[ U5n U R URS9n XU 4U-n Un [R"S5n[R"X5nO$US:XaUn Un UnU"U/UQ76nO[ US35e[R"[R"[R"U[XU/U Q76- S -555$) aCheck the correctness of a gradient function by comparing it against a (forward) finite-difference approximation of the gradient. Parameters ---------- func : callable ``func(x0, *args)`` Function whose derivative is to be checked. grad : callable ``grad(x0, *args)`` Jacobian of `func`. x0 : ndarray Points to check `grad` against forward difference approximation of grad using `func`. args : \\*args, optional Extra arguments passed to `func` and `grad`. epsilon : float, optional Step size used for the finite difference approximation. It defaults to ``sqrt(np.finfo(float).eps)``, which is approximately 1.49e-08. direction : str, optional If set to ``'random'``, then gradients along a random vector are used to check `grad` against forward difference approximation using `func`. By default it is ``'all'``, in which case, all the one hot direction vectors are considered to check `grad`. If `func` is a vector valued function then only ``'all'`` can be used. rng : `numpy.random.Generator`, optional Pseudorandom number generator state. When `rng` is None, a new `numpy.random.Generator` is created using entropy from the operating system. Types other than `numpy.random.Generator` are passed to `numpy.random.default_rng` to instantiate a ``Generator``. The random numbers generated affect the random vector along which gradients are computed to check ``grad``. Note that `rng` is only used when `direction` argument is set to `'random'`. Returns ------- err : float The square root of the sum of squares (i.e., the 2-norm) of the difference between ``grad(x0, *args)`` and the finite difference approximation of `grad` using func at the points `x0`. See Also -------- approx_fprime Examples -------- >>> import numpy as np >>> def func(x): ... return x[0]**2 - 0.5 * x[1]**3 >>> def grad(x): ... return [2 * x[0], -1.5 * x[1]**2] >>> from scipy.optimize import check_grad >>> check_grad(func, grad, [1.5, -1.5]) 2.9802322387695312e-08 # may vary >>> rng = np.random.default_rng() >>> check_grad(func, grad, [1.5, -1.5], ... direction='random', seed=rng) 2.9802322387695312e-08 cU"X U--/UQ76$r:rY)wrrvrHs r@gcheck_grad..gLsB1H$t$$rCrandomr!z1'random' can only be used with scalar valued func)rr!rFz1 is not a valid string for ``direction`` argumentr)rEr asanyarrayrrr(standard_normalrrdotrrrr)rrrrrOrPrHsteprU_gradrng_genrT_args_funcvarsanalytical_grads r@rr s)~ D BB%H d2oo. ::>%& &$S)  # #"(( # 41 $xx~&&* e r/D/I;'223 3 77266"&& =dCUC CaG rCc>U"U4U-6nU"XU--4U-6nXe- U- $r:rY)rrfprimerrHf1f2s r@approx_fhess_prghs7 2%$, B 2 >#d* ,B Gw rCc\rSrSrSrg)_LineSearchErroriorYNryrYrCr@ririorzrCric URSS5n[XX#UXV40UD6n U Sb'Ub$X)SU--n U"U SXSU S5(dSn U Scb[R"5 [R"S[ 5 0n SHn X;dM X|X'M [ XX#UXV4SU0U D6n SSS5 U Sc [5eU $!,(df  N =f) z Same as line_search_wolfe1, but fall back to line_search_wolfe2 if suitable step length is not found, and raise an exception if a suitable step length is not found. Raises ------ _LineSearchError If no suitable step size is found extra_conditionNrrr:ignore)c1c2r)popr"rcatch_warnings simplefilterr$r#ri) rrdrKpkgfkold_fval old_old_fvalkwargsrkretxp1kwargs2keys r@_line_search_wolfe12r|ssjj!2D9O Q% '% 'C 1vo9q6B;s1vsFCF;;C 1v~  $ $ &  ! !(,= >G+=#);GL,%Q%-05D0(/0C ' 1v~  J' &s*&C C  Ch㈵>g?c UUUU UU U U UUS. n[U 5n [XX24SU 0UD6nU(a0USUSUSUSUSUSUS 4nU (a UUS 4- nU$U (a USUS 4$US$) a Minimize a function using the BFGS algorithm. Parameters ---------- f : callable ``f(x,*args)`` Objective function to be minimized. x0 : ndarray Initial guess, shape (n,) fprime : callable ``f'(x,*args)``, optional Gradient of f. args : tuple, optional Extra arguments passed to f and fprime. gtol : float, optional Terminate successfully if gradient norm is less than `gtol` norm : float, optional Order of norm (Inf is max, -Inf is min) epsilon : int or ndarray, optional If `fprime` is approximated, use this value for the step size. callback : callable, optional An optional user-supplied function to call after each iteration. Called as ``callback(xk)``, where ``xk`` is the current parameter vector. maxiter : int, optional Maximum number of iterations to perform. full_output : bool, optional If True, return ``fopt``, ``func_calls``, ``grad_calls``, and ``warnflag`` in addition to ``xopt``. disp : bool, optional Print convergence message if True. retall : bool, optional Return a list of results at each iteration if True. xrtol : float, default: 0 Relative tolerance for `x`. Terminate successfully if step size is less than ``xk * xrtol`` where ``xk`` is the current parameter vector. c1 : float, default: 1e-4 Parameter for Armijo condition rule. c2 : float, default: 0.9 Parameter for curvature condition rule. hess_inv0 : None or ndarray, optional`` Initial inverse hessian estimate, shape (n, n). If None (default) then the identity matrix is used. Returns ------- xopt : ndarray Parameters which minimize f, i.e., ``f(xopt) == fopt``. fopt : float Minimum value. gopt : ndarray Value of gradient at minimum, f'(xopt), which should be near 0. Bopt : ndarray Value of 1/f''(xopt), i.e., the inverse Hessian matrix. func_calls : int Number of function_calls made. grad_calls : int Number of gradient calls made. warnflag : integer 1 : Maximum number of iterations exceeded. 2 : Gradient and/or function calls not changing. 3 : NaN result encountered. allvecs : list The value of `xopt` at each iteration. Only returned if `retall` is True. Notes ----- Optimize the function, `f`, whose gradient is given by `fprime` using the quasi-Newton method of Broyden, Fletcher, Goldfarb, and Shanno (BFGS). Parameters `c1` and `c2` must satisfy ``0 < c1 < c2 < 1``. See Also -------- minimize: Interface to minimization algorithms for multivariate functions. See ``method='BFGS'`` in particular. References ---------- Wright, and Nocedal 'Numerical Optimization', 1999, p. 198. Examples -------- >>> import numpy as np >>> from scipy.optimize import fmin_bfgs >>> def quadratic_cost(x, Q): ... return x @ Q @ x ... >>> x0 = np.array([-3, -4]) >>> cost_weight = np.diag([1., 10.]) >>> # Note that a trailing comma is necessary for a tuple with single element >>> fmin_bfgs(quadratic_cost, x0, args=(cost_weight,)) Optimization terminated successfully. Current function value: 0.000000 Iterations: 7 # may vary Function evaluations: 24 # may vary Gradient evaluations: 8 # may vary array([ 2.85169950e-06, -4.61820139e-07]) >>> def quadratic_cost_grad(x, Q): ... return 2 * Q @ x ... >>> fmin_bfgs(quadratic_cost, x0, quadratic_cost_grad, args=(cost_weight,)) Optimization terminated successfully. Current function value: 0.000000 Iterations: 7 Function evaluations: 8 Gradient evaluations: 8 array([ 2.85916637e-06, -4.54371951e-07]) ) gtolnormepsrr2rxrtolrnro hess_inv0rcr>r;r<hess_invrnjevrr)rt_minimize_bfgs)rrrdrHrrrr2rrrrcrrnrorrrbrs r@rrsj " $Dh'H  Hx H4 HCs8SZUS_v;F S]<  I) )G s8S^+ +s8OrCc p[U5 [U5 U n[U5R5nURS:XaSUlUc[ U5S-n[XX2UU S9nURnURnU"U5nU"U5nSn[ U5n[R"U[S9nUcUOUnU[RRU5S- -nUnU(aU/nSn[UUS9nUU:GaUU:Ga[R "UU5*n [#UUUU UUUS S XS 9 un!n"n#nnn$U!U -n%UU%-n&U(aWR'U&5 U&nU$cU"U&5n$U$U- n'U$nUS - n[)UUS 9n([+UU(5(aGOe[UUS9nUU::aGOSU![U 5-X[U5--::aGO1[R,"U5(dSnGO[R "U'U%5n)U)S:XaSn*U (aSn+[/SU+5 OSU)- n*UU%SS2[R04U'[R0SS24-U*-- n,UU'SS2[R04U%[R0SS24-U*-- n-[R "U,[R "UU-55U*U%SS2[R04-U%[R0SS24--nUU:a UU:aGMUn.US:Xa [2Sn+OUU:a S n[2Sn+Ot[R4"U5(dD[R4"U.5(d)[R4"U5R75(a Sn[2Sn+O [2Sn+U (aY[/UU+5 [9SU.S35 [9SU-5 [9SUR:-5 [9SUR<-5 [)U.UUUR:UR<UUS:HU+UUS9 n/U(aWU/S'U/$![$a SnGM?f=f)a Minimization of scalar function of one or more variables using the BFGS algorithm. Options ------- disp : bool Set to True to print convergence messages. maxiter : int Maximum number of iterations to perform. gtol : float Terminate successfully if gradient norm is less than `gtol`. norm : float Order of norm (Inf is max, -Inf is min). eps : float or ndarray If `jac is None` the absolute step size used for numerical approximation of the jacobian via forward differences. return_all : bool, optional Set to True to return a list of the best solution at each of the iterations. finite_diff_rel_step : None or array_like, optional If ``jac in ['2-point', '3-point', 'cs']`` the relative step size to use for numerical approximation of the jacobian. The absolute step size is computed as ``h = rel_step * sign(x) * max(1, abs(x))``, possibly adjusted to fit into the bounds. For ``jac='3-point'`` the sign of `h` is ignored. If None (default) then step is selected automatically. xrtol : float, default: 0 Relative tolerance for `x`. Terminate successfully if step size is less than ``xk * xrtol`` where ``xk`` is the current parameter vector. c1 : float, default: 1e-4 Parameter for Armijo condition rule. c2 : float, default: 0.9 Parameter for curvature condition rule. hess_inv0 : None or ndarray, optional Initial inverse hessian estimate, shape (n, n). If None (default) then the identity matrix is used. Notes ----- Parameters `c1` and `c2` must satisfy ``0 < c1 < c2 < 1``. If minimization doesn't complete successfully, with an error message of ``Desired error not necessarily achieved due to precision loss``, then consider setting `gtol` to a higher value. This precision loss typically occurs when the (finite difference) numerical differentiation cannot provide sufficient precision to satisfy the `gtol` termination criterion. This can happen when working in single precision and a callable jac is not provided. For single precision problems a `gtol` of 1e-3 seems to work. rrXNr)rHrrrrr0.++}Ô%IT)rrrnror!rg@@z.Divide-by-zero encountered: rhok assumed largeTrr3r2rr4r0rrrr ! Gradient evaluations: %d) r;r<rrrrr0r r>rr)rrrr rrrrr;rrErintlinalgrrr[r|rir rr*r_print_success_message_or_warnnewaxisr$isnanrr%rngev)0r;rrHr<rcrrrr2rrrrrnrorrrrrmyfprimerurtr2r0IrrvrKrrFgnormrsalpha_kfcgcgfkp1skxkp1ykr_rhok_invrhokrA1A2rErGs0 r@rr0sn?+Y' F    B ww!|b'C- !#337K MB AwwHuH 2,C A BA qAYBbiinnS1A55L B $H CT "E 4 > {{8$$H 66"b> r>DF.tS9=D ArzzM"R A %66= = ArzzM"R A %66= = VVBr2 '4"Q ]2C+C13BJJM1B,CDu 4.>.@.@e$i( &x5 1$q:; '!+, 1BGG;< 1BGG;< #!#%-]SB !#F#y Ma  H  sP$$ P54P5cnU(d [U5 g[R"X=(d [SS9 g)Nrr)r%rrr)rFr warntypes r@rrs  g g:?qIrCg?cUUUU UU S.n[U 5n [XX24XU S.UD6nU(a(USUSUSUSUS4nU (a UUS4- nU$U (a USUS4$US$) a Minimize a function using a nonlinear conjugate gradient algorithm. Parameters ---------- f : callable, ``f(x, *args)`` Objective function to be minimized. Here `x` must be a 1-D array of the variables that are to be changed in the search for a minimum, and `args` are the other (fixed) parameters of `f`. x0 : ndarray A user-supplied initial estimate of `xopt`, the optimal value of `x`. It must be a 1-D array of values. fprime : callable, ``fprime(x, *args)``, optional A function that returns the gradient of `f` at `x`. Here `x` and `args` are as described above for `f`. The returned value must be a 1-D array. Defaults to None, in which case the gradient is approximated numerically (see `epsilon`, below). args : tuple, optional Parameter values passed to `f` and `fprime`. Must be supplied whenever additional fixed parameters are needed to completely specify the functions `f` and `fprime`. gtol : float, optional Stop when the norm of the gradient is less than `gtol`. norm : float, optional Order to use for the norm of the gradient (``-np.inf`` is min, ``np.inf`` is max). epsilon : float or ndarray, optional Step size(s) to use when `fprime` is approximated numerically. Can be a scalar or a 1-D array. Defaults to ``sqrt(eps)``, with eps the floating point machine precision. Usually ``sqrt(eps)`` is about 1.5e-8. maxiter : int, optional Maximum number of iterations to perform. Default is ``200 * len(x0)``. full_output : bool, optional If True, return `fopt`, `func_calls`, `grad_calls`, and `warnflag` in addition to `xopt`. See the Returns section below for additional information on optional return values. disp : bool, optional If True, return a convergence message, followed by `xopt`. retall : bool, optional If True, add to the returned values the results of each iteration. callback : callable, optional An optional user-supplied function, called after each iteration. Called as ``callback(xk)``, where ``xk`` is the current value of `x0`. c1 : float, default: 1e-4 Parameter for Armijo condition rule. c2 : float, default: 0.4 Parameter for curvature condition rule. Returns ------- xopt : ndarray Parameters which minimize f, i.e., ``f(xopt) == fopt``. fopt : float, optional Minimum value found, f(xopt). Only returned if `full_output` is True. func_calls : int, optional The number of function_calls made. Only returned if `full_output` is True. grad_calls : int, optional The number of gradient calls made. Only returned if `full_output` is True. warnflag : int, optional Integer value with warning status, only returned if `full_output` is True. 0 : Success. 1 : The maximum number of iterations was exceeded. 2 : Gradient and/or function calls were not changing. May indicate that precision was lost, i.e., the routine did not converge. 3 : NaN result encountered. allvecs : list of ndarray, optional List of arrays, containing the results at each iteration. Only returned if `retall` is True. See Also -------- minimize : common interface to all `scipy.optimize` algorithms for unconstrained and constrained minimization of multivariate functions. It provides an alternative way to call ``fmin_cg``, by specifying ``method='CG'``. Notes ----- This conjugate gradient algorithm is based on that of Polak and Ribiere [1]_. Conjugate gradient methods tend to work better when: 1. `f` has a unique global minimizing point, and no local minima or other stationary points, 2. `f` is, at least locally, reasonably well approximated by a quadratic function of the variables, 3. `f` is continuous and has a continuous gradient, 4. `fprime` is not too large, e.g., has a norm less than 1000, 5. The initial guess, `x0`, is reasonably close to `f` 's global minimizing point, `xopt`. Parameters `c1` and `c2` must satisfy ``0 < c1 < c2 < 1``. References ---------- .. [1] Wright & Nocedal, "Numerical Optimization", 1999, pp. 120-122. Examples -------- Example 1: seek the minimum value of the expression ``a*u**2 + b*u*v + c*v**2 + d*u + e*v + f`` for given values of the parameters and an initial guess ``(u, v) = (0, 0)``. >>> import numpy as np >>> args = (2, 3, 7, 8, 9, 10) # parameter values >>> def f(x, *args): ... u, v = x ... a, b, c, d, e, f = args ... return a*u**2 + b*u*v + c*v**2 + d*u + e*v + f >>> def gradf(x, *args): ... u, v = x ... a, b, c, d, e, f = args ... gu = 2*a*u + b*v + d # u-component of the gradient ... gv = b*u + 2*c*v + e # v-component of the gradient ... return np.asarray((gu, gv)) >>> x0 = np.asarray((0, 0)) # Initial guess. >>> from scipy import optimize >>> res1 = optimize.fmin_cg(f, x0, fprime=gradf, args=args) Optimization terminated successfully. Current function value: 1.617021 Iterations: 4 Function evaluations: 8 Gradient evaluations: 8 >>> res1 array([-1.80851064, -0.25531915]) Example 2: solve the same problem using the `minimize` function. (This `myopts` dictionary shows all of the available options, although in practice only non-default values would be needed. The returned value will be a dictionary.) >>> opts = {'maxiter' : None, # default value. ... 'disp' : True, # non-default value. ... 'gtol' : 1e-5, # default value. ... 'norm' : np.inf, # default value. ... 'eps' : 1.4901161193847656e-08} # default value. >>> res2 = optimize.minimize(f, x0, jac=gradf, args=args, ... method='CG', options=opts) Optimization terminated successfully. Current function value: 1.617021 Iterations: 4 Function evaluations: 8 Gradient evaluations: 8 >>> res2.x # minimum found array([-1.80851064, -0.25531915]) )rrrrr2rrcrnror>r;rrrr)rt _minimize_cg)rrrdrHrrrr2rrrrcrnrorrbrs r@rrs@  "Dh'H qd X  Cc(CJF S[#h-O  I) )G s8S^+ +s8OrCc v^^^!^"^#^$^%^&^'^([U5 U n[U5R5nUc[U5S-n[ XX2UU S9nUR nUR m$U"U5nT$"U5m#SnUm(U[RRT#5S- -nU(aT(/nSnT#*m%[T#TS9nSm'UT:aUU:a[R"T#T#5m"S/m!SU"U#U$UU%U(4Sjjm&U!UU&U'4S jn[UT$T(T%T#UUXS S US 9 unnnnnnUT!S:Xa T!unm(m%m#nOT&"UU5unm(m%m#nU(aWRT(5 US - n[T(US9n[!UU5(aOUT:aUU:aMUnUS:Xa ["SnOUU:a S n["SnOt[R$"U5(dD[R$"U5(d)[R$"T(5R'5(a Sn["SnO ["SnU (aY[)UU5 [+SUS35 [+SU-5 [+SUR,-5 [+SUR.-5 [UT#UR,UR.UUS:HUT(US9 n U(aWU S'U $![a SnGM>f=f)a Minimization of scalar function of one or more variables using the conjugate gradient algorithm. Options ------- disp : bool Set to True to print convergence messages. maxiter : int Maximum number of iterations to perform. gtol : float Gradient norm must be less than `gtol` before successful termination. norm : float Order of norm (Inf is max, -Inf is min). eps : float or ndarray If `jac is None` the absolute step size used for numerical approximation of the jacobian via forward differences. return_all : bool, optional Set to True to return a list of the best solution at each of the iterations. finite_diff_rel_step : None or array_like, optional If ``jac in ['2-point', '3-point', 'cs']`` the relative step size to use for numerical approximation of the jacobian. The absolute step size is computed as ``h = rel_step * sign(x) * max(1, abs(x))``, possibly adjusted to fit into the bounds. For ``jac='3-point'`` the sign of `h` is ignored. If None (default) then step is selected automatically. c1 : float, default: 1e-4 Parameter for Armijo condition rule. c2 : float, default: 0.4 Parameter for curvature condition rule. Notes ----- Parameters `c1` and `c2` must satisfy ``0 < c1 < c2 < 1``. Nr)r<rHrrrrrg{Gz?c>T UT --nUcT "U5nUT- n[S[R"X15T- 5nU*UT --n[UT S9nXXQU4$)Nrr)rrEr[r) alpharrrbeta_kpkp1rdeltakrtrrrsrKs r@polak_ribiere_powell_step/_minimize_cg..polak_ribiere_powell_stepsj ?D} BBFF2-67F6FRK'DEt,Ee4 4rCc>T "X5TSS&TuppVnUT ::ag[R"XV5T *[R"Xf5-:*$)NT)rEr[) rrfp1rrKrsrtr cached_steprrsigma_3s r@descent_condition'_minimize_cg..descent_conditionsP7uDKN(3 %Er}66"?wh1A&AA ArCrr)rnrorrrkr!rr3r2rr4r0rrrr r) r;r<rrrr0r r>rrr:)rrr rrr;rrErrrr[r|rir rr*r$rrrr%rr))r;rrHr<rcrrrr2rrrrnrorrrrrur2rvrrFrrrrrrr_rErrGrrrtrrsrrrKs) `` @@@@@@@@r@rrsR?+ F    Bb'C- !#ss7K MB AwwHuH 2,C A BbiinnS1A55L $H B CT "EG 4.>.@.@e$i( &x5 1$q:; '!+, 1BGG;< 1BGG;< #BGG!#%-]SB !#F#y MW  H  s7J'' J87J8cUUUU U S.n[U 5n [XXRXC4XUS.UD6nU (a,USUSUSUSUSUS4nU (a UUS 4- nU$U (a USUS 4$US$) a Unconstrained minimization of a function using the Newton-CG method. Parameters ---------- f : callable ``f(x, *args)`` Objective function to be minimized. x0 : ndarray Initial guess. fprime : callable ``f'(x, *args)`` Gradient of f. fhess_p : callable ``fhess_p(x, p, *args)``, optional Function which computes the Hessian of f times an arbitrary vector, p. fhess : callable ``fhess(x, *args)``, optional Function to compute the Hessian matrix of f. args : tuple, optional Extra arguments passed to f, fprime, fhess_p, and fhess (the same set of extra arguments is supplied to all of these functions). epsilon : float or ndarray, optional If fhess is approximated, use this value for the step size. callback : callable, optional An optional user-supplied function which is called after each iteration. Called as callback(xk), where xk is the current parameter vector. avextol : float, optional Convergence is assumed when the average relative error in the minimizer falls below this amount. maxiter : int, optional Maximum number of iterations to perform. full_output : bool, optional If True, return the optional outputs. disp : bool, optional If True, print convergence message. retall : bool, optional If True, return a list of results at each iteration. c1 : float, default: 1e-4 Parameter for Armijo condition rule. c2 : float, default: 0.9 Parameter for curvature condition rule Returns ------- xopt : ndarray Parameters which minimize f, i.e., ``f(xopt) == fopt``. fopt : float Value of the function at xopt, i.e., ``fopt = f(xopt)``. fcalls : int Number of function calls made. gcalls : int Number of gradient calls made. hcalls : int Number of Hessian calls made. warnflag : int Warnings generated by the algorithm. 1 : Maximum number of iterations exceeded. 2 : Line search failure (precision loss). 3 : NaN result encountered. allvecs : list The result at each iteration, if retall is True (see below). See also -------- minimize: Interface to minimization algorithms for multivariate functions. See the 'Newton-CG' `method` in particular. Notes ----- Only one of `fhess_p` or `fhess` need to be given. If `fhess` is provided, then `fhess_p` will be ignored. If neither `fhess` nor `fhess_p` is provided, then the hessian product will be approximated using finite differences on `fprime`. `fhess_p` must compute the hessian times an arbitrary vector. If it is not given, finite-differences on `fprime` are used to compute it. Newton-CG methods are also called truncated Newton methods. This function differs from scipy.optimize.fmin_tnc because 1. scipy.optimize.fmin_ncg is written purely in Python using NumPy and scipy while scipy.optimize.fmin_tnc calls a C function. 2. scipy.optimize.fmin_ncg is only for unconstrained minimization while scipy.optimize.fmin_tnc is for unconstrained minimization or box constrained minimization. (Box constraints give lower and upper bounds for each variable separately.) Parameters `c1` and `c2` must satisfy ``0 < c1 < c2 < 1``. References ---------- Wright & Nocedal, 'Numerical Optimization', 1999, p. 140. )rrr2rrrr>r;rrnhevrr)rt_minimize_newtoncg)rrrdfhess_pfhessrHavextolrr2rrrrcrnrorrbrs r@rr9sB  "D h'H QD% F&." F@D FCs8SZVc&kv;H /  I) )G s8S^+ +s8OrCc ^ ^4^5^6^7^8^9^:^;[U5 Uc [S5eUnUnUnUnU m9[U5R5n[ XX2XS9m:T:R nT:R nT:RU5nU[;d[U[5(a SnU:4SjnUnU4U U5U6U7U8U9U:U;4 SjnSm6U c[U5S-n S[U5-n[U5U-n[R"[5Rn[R "U5m;T9(aT;/m4Sm7Sm5U"U5m8Sn[R"[R"5R$nUU:GaxT7U :aS [&S -nU"S U5$U"T;5*n[R(R+US S 9n[-S [.R0"U55nUU-n [3[U5UR4S9n!U*n"U"*n#Sn$[R6"U"U"5n%UbT:RT;5n&T6S - m6[9U5GHn'[R:R=[R>"U"55U ::a OUc"Uc[AT;U#UU5n(O!U"T;U#/UQ76n(T6S - m6OW&R7U#5n([U(5RC5n([R6"U#U(5n)SU)s=::a SU-::aO O OjU)S:aU$S:a O\U%U)*- U-n! OQU%U)- n*U!U*U#-- n!U"U*U(-- n"[R6"U"U"5n+U+U%- n,U"*U,U#--n#U$S - n$U+n%GM SnU"SU5$U!n-U*m5[EUUT;U-T5T8UXS9 un.n/n0m8nn1U.U--n2T;U2- m;T9(aT4RIT;5 T7S - m7[KT;T8S9n3[MUU35(a U"SS5$[R(R+U2S S 9nUU:aGMx[RN"T85(d[RN"U5(aU"S[&S5$[&SnU"SU5$![Fa S [&S-nU"SU5s$f=f)a  Minimization of scalar function of one or more variables using the Newton-CG algorithm. Note that the `jac` parameter (Jacobian) is required. Options ------- disp : bool Set to True to print convergence messages. xtol : float Average relative error in solution `xopt` acceptable for convergence. maxiter : int Maximum number of iterations to perform. eps : float or ndarray If `hessp` is approximated, use this value for the step size. return_all : bool, optional Set to True to return a list of the best solution at each of the iterations. c1 : float, default: 1e-4 Parameter for Armijo condition rule. c2 : float, default: 0.9 Parameter for curvature condition rule. Notes ----- Parameters `c1` and `c2` must satisfy ``0 < c1 < c2 < 1``. Nz)Jacobian is required for Newton-CG method)rHrrcD>TRU5RU5$r:)rr[)r>rrHrs r@_hessp"_minimize_newtoncg.._hessps771:>>!$ $rCc P> T(af[X5 [ST S35 [ST-5 [ST R-5 [ST R-5 [ST-5 T n[ UTT RT RTUUS:HUT TS9 nT (aTUS 'U$) Nrrrr rz Hessian evaluations: %dr) r;r<rrrrr0r r>rr)rr%rrr) rFrrErGrrrthcallsr2rurrrKs r@ terminate%_minimize_newtoncg..terminates  *8 9 5hq\B C +a/ 0 5? @ 5? @ 4v= >Dc%'WW6()1Q$%'  'F9  rCrrz Warning: r2r!rrrrzMWarning: CG iterations didn't converge. The Hessian is not positive definite.)rnror3rrrlr4r0)(rrrr rr;rrr- isinstancer rrEfinforrrGrrr$rrr#mathrrrr[rr!r"rrgsqueezer|rir rr*r)D4KEC%%-'E FADA$F(CQ$ $ b %)!VRS*2LRP :FBHlE" f   NN2  Q,rx@ $X/B C CQ# # v15 _ $ d 88H  -!8!8Q 67 7i(C  )  % ::CQ$ $ %sP77QQchUUUS.n[XU4U40UD6n U(aU SU SU SU S4$U S$)aBounded minimization for scalar functions. Parameters ---------- func : callable f(x,*args) Objective function to be minimized (must accept and return scalars). x1, x2 : float or array scalar Finite optimization bounds. args : tuple, optional Extra arguments passed to function. xtol : float, optional The convergence tolerance. maxfun : int, optional Maximum number of function evaluations allowed. full_output : bool, optional If True, return optional outputs. disp: int, optional If non-zero, print messages. ``0`` : no message printing. ``1`` : non-convergence notification messages only. ``2`` : print a message on convergence too. ``3`` : print iteration results. Returns ------- xopt : ndarray Parameters (over given interval) which minimize the objective function. fval : number (Optional output) The function value evaluated at the minimizer. ierr : int (Optional output) An error flag (0 if converged, 1 if maximum number of function calls reached). numfunc : int (Optional output) The number of function calls made. See also -------- minimize_scalar: Interface to minimization algorithms for scalar univariate functions. See the 'Bounded' `method` in particular. Notes ----- Finds a local minimizer of the scalar function `func` in the interval x1 < xopt < x2 using Brent's method. (See `brent` for auto-bracketing.) References ---------- .. [1] Forsythe, G.E., M. A. Malcolm, and C. B. Moler. "Computer Methods for Mathematical Computations." Prentice-Hall Series in Automatic Computation 259 (1977). .. [2] Brent, Richard P. Algorithms for Minimization Without Derivatives. Courier Corporation, 2013. Examples -------- `fminbound` finds the minimizer of the function in the given range. The following examples illustrate this. >>> from scipy import optimize >>> def f(x): ... return (x-1)**2 >>> minimizer = optimize.fminbound(f, -4, 4) >>> minimizer 1.0 >>> minimum = f(minimizer) >>> minimum 0.0 >>> res = optimize.fminbound(f, 3, 4, full_output=True) >>> minimizer, fval, ierr, numfunc = res >>> minimizer 3.000005960860986 >>> minimum = f(minimizer) >>> minimum, fval (4.000023843479476, 4.000023843479476) )rr2rr>r;rr)_minimize_scalar_bounded) rx1x2rHrrrroptionsrbs r@rrmsWf G #4b4 C7 CC3xUS]CK??3xrCc Z[U5 Un[U5S:wa [S5eUup[U5(a[U 5(d [S5eX:a [S5eSn Sn Sn [ S5n S S [ S 5- -nXnnXUU- --nUUnnS =nnUnU"U/UQ76nS nS UU4n[ R nU=nnS UU--nU [ R"U5-US - -nSU-nUS:a([S5 [U 5 [SUU 4--5 [ R"UU- 5US UU- -- :GaPS n [ R"U5U:aSn UU- UU- -n!UU- UU- -n"UU- U"-UU- U!-- n#SU"U!- -n"U"S :aU#*n#[ R"U"5n"Un!Un[ R"U#5[ R"S U"-U!-5:aaU#U"UU- -:aUU#U"UU- -:aIU#S -U"- nUU-nSn UU- U:d UU- U:a'[ R"UU- 5UU- S:H-n$UU$-nOS n U (aUU:aUU- nOUU- nUU-nSn [ R"U5US:H-n$UU$[ R"[ R"U5U5--nU"U/UQ76nUS - nUUU4nUS:a[SUU 4--5 UU::aUU:aUnOUnUUnnUUnnUUnnO6UU:aUnOUnUU::dUU:Xa UUnnUUnnOUU::d UU:XdUU:XaUUnnS UU--nU [ R"U5-US - -nSU-nUU:aS n O)[ R"UU- 5US UU- -- :aGMP[ R"U5(d6[ R"U5(d[ R"U5(aSn Un%US:a[UU U%XsU5 [U%XS:HSS[SS.RU S5UUUS9n&U&$)a Options ------- maxiter : int Maximum number of iterations to perform. disp: int, optional If non-zero, print messages. ``0`` : no message printing. ``1`` : non-convergence notification messages only. ``2`` : print a message on convergence too. ``3`` : print iteration results. xatol : float Absolute error in solution `xopt` acceptable for convergence. rzbounds must have two elements.z+Optimization bounds must be finite scalars.z(The lower bound exceeds the upper bound.rz2 Func-count x f(x) Procedurez initialgOd@rr)rrrrrrErrr%signmaximumr _endprintrr$get)'rrrHrr2rrrrrflagheaderr\sqrt_eps golden_meanarfulcnfcxfratrr>rnum fmin_datafuffulcfnfcrtol1tol2r rqrsirErGs' r@rrs.?+ F 6{a9:: FB R %5b%9%9FGG wCDD D AF DG}HtCy)K qA a!e$ $DDCMC! A a$B CB I BED AB bffRj 53; .D :D ax c  f  (I,?@A 66"r'?dSAE]2 3 66!9t Fcb5j)AdrDy)Ada28q.0Aq1u A3wBq AAARVVCE!G_,1q!b&z>a"f%3w!mH)UdNQ$b)b2g!^%@ADD"  s 5F MrCcB\rSrSrS SjrS SjrSrSrS SjrSr g) Brentia cXlX lX0lX@lSUlSUlSUlSUlSUlSUl X`l g)Ngdy=gŋ!r?r) rrHtolr2_mintol_cgxminrEiterfuncallsr)r?rrHrr2rrs r@rABrent.__init__c sE         rCNcXlgr:)brack)r?rs r@ set_bracketBrent.set_bracketr s rCcURnURnURnUc[XS9upEpgpn O[ U5S:Xa[XSUSUS9upEpgpn On[ U5S:XaTUupEnXF:aXFpFXE:aXV:d [ S5eU"U4U-6nU"U4U-6nU"U4U-6n X:aX:d [ S5eSn O [ S 5eXEXgXU 4$) NrHrrr!xaxbrHrWBracketing values (xa, xb, xc) do not fulfill this requirement: (xa < xb) and (xb < xc)cBracketing values (xa, xb, xc) do not fulfill this requirement: (f(xb) < f(xa)) and (f(xb) < f(xc))3Bracketing interval must be length 2 or 3 sequence.)rrHrr rr) r?rrHrr r r@fafbrrs r@get_bracket_infoBrent.get_bracket_infou syyyy  =/6t/G ,BBBH Z1_/6ta:?(0O ,BBBH Z1_JBBBW27 I &B &B &BW27 M H78 8rrx//rCcURnUR5up#pEpgnURn URn U=n =pU=n=nnX$:aUnUnOUnUnSnSnURS:a7[ S5 [ SSSSSSSS 35 [ US SU S SUS 35 UUR :GaCUR[R"U 5-U -nS U-nS UU--n[R"U U- 5US UU- -- :aGO[R"U5U::aU U:aUU - nOUU - nU U-nOX- UU- -nX- UU- -nX- U-X- U-- nS UU- -nUS:aU*n[R"U5nUnWnUUUU - -:aoUUUU - -:ac[R"U5[R"S U-U-5:a/US-U- nU U-nUU- U:d UU- U:aUU - S:aUnOU*nOU U:aUU - nOUU - nU U-n[R"U5U:aUS:aU U-nO U U- nOU U-nU"U4UR-6nUS- nUU:a4UU :aUnOUnUU::dX:Xa U n Un UnUnO,UU::d X:XdX:XaUn UnOUU :aU nOU nU n U n Un UnUnUnURS:a[ US SU S SUS 35 US- nUUR :aGMCXl UUl UUl Xlg)Nrrrr Func-count^12r>f(x) ^12^12g^12.6grrrr!)rrrrrr%r2rrErrHrrErr)r?rr r r@rrrrrrr>rSrTfwfvrrrdeltaxrrrxmidrtmp1tmp2rdx_tempurs r@optimizeBrent.optimize syy+/+@+@+B(,,hhAR" GAAAA 99q= #J \#&aCy&? @ XdO1QvJa6{; <dll"88bffQi''1D:D!a%=Dvva$h4#Q-#78v$&IUFUFFl"r'*"r'*UdNaet^3dTk*3JAvvd| Q'a$!a%..@RVVC$J,@%AAc'D.CCAQ$1q5D.!8q="&C#'%CT !"Q!"Q,Cs d"!8DADAGtyy(*B MHREAA"H!&AABBBhAFABFAAyy1}$q6 !Bv;?@ AIDcdll"l    rCcU(a.URURURUR4$UR$r:)rrErr)r?rs r@ get_resultBrent.get_result s. 99diiDMMA A99 rC) rrrHrrrrrErr2rr)rY`sbO>rrr:)F) rSrTrUrVrArrr$r'rXrYrCr@rra s$;>%& $0Lr!hrCrcbUUS.n[XU40UD6nU(aUSUSUSUS4$US$)a Given a function of one variable and a possible bracket, return a local minimizer of the function isolated to a fractional precision of tol. Parameters ---------- func : callable f(x,*args) Objective function. args : tuple, optional Additional arguments (if present). brack : tuple, optional Either a triple ``(xa, xb, xc)`` satisfying ``xa < xb < xc`` and ``func(xb) < func(xa) and func(xb) < func(xc)``, or a pair ``(xa, xb)`` to be used as initial points for a downhill bracket search (see `scipy.optimize.bracket`). The minimizer ``x`` will not necessarily satisfy ``xa <= x <= xb``. tol : float, optional Relative error in solution `xopt` acceptable for convergence. full_output : bool, optional If True, return all output args (xmin, fval, iter, funcalls). maxiter : int, optional Maximum number of iterations in solution. Returns ------- xmin : ndarray Optimum point. fval : float (Optional output) Optimum function value. iter : int (Optional output) Number of iterations. funcalls : int (Optional output) Number of objective function evaluations made. See also -------- minimize_scalar: Interface to minimization algorithms for scalar univariate functions. See the 'Brent' `method` in particular. Notes ----- Uses inverse parabolic interpolation when possible to speed up convergence of golden section method. Does not ensure that the minimum lies in the range specified by `brack`. See `scipy.optimize.fminbound`. Examples -------- We illustrate the behaviour of the function when `brack` is of size 2 and 3 respectively. In the case where `brack` is of the form ``(xa, xb)``, we can see for the given values, the output does not necessarily lie in the range ``(xa, xb)``. >>> def f(x): ... return (x-1)**2 >>> from scipy import optimize >>> minimizer = optimize.brent(f, brack=(1, 2)) >>> minimizer 1 >>> res = optimize.brent(f, brack=(-1, 0.5, 2), full_output=True) >>> xmin, fval, iter, funcalls = res >>> f(xmin), fval (0.0, 0.0) rr2r>r;rr)_minimize_scalar_brentrrHrrrr2rrbs r@rr sNN!#G d >g >C3xUSZV<<3xrCc <[U5 UnUS:a[SU<35e[XUSXES9nURU5 UR 5 UR SS9uppX:=(a8 [ R"U 5=(d [ R"U 5(+n U (aSUS3nOGX:aSn[ R"U 5(d[ R"U 5(a [S nU(a[U (+W5 [XXU WS 9$) a Options ------- maxiter : int Maximum number of iterations to perform. xtol : float Relative error in solution `xopt` acceptable for convergence. disp : int, optional If non-zero, print messages. ``0`` : no message printing. ``1`` : non-convergence notification messages only. ``2`` : print a message on convergence too. ``3`` : print iteration results. Notes ----- Uses inverse parabolic interpolation when possible to speed up convergence of golden section method. rztolerance should be >= 0, got T)rrHrrr2r)rk Optimization terminated successfully; The returned value satisfies the termination criteria (using xtol =  )& Maximum number of iterations exceededr4)r;r>rrr0r ) rrrrr$r'rErr$rr)rrrHrr2rrrrr>rErrr0r s r@r-r-f s6?+ C Qw9#ABB tC"G @E e NN))d);ASmCRXXa[%BBHHTN CG$$(6- >?G 88A;;"((4..(/0G &7{G< dS")7 <>> def f(x): ... return (x-1)**2 >>> from scipy import optimize >>> minimizer = optimize.golden(f, brack=(1, 2)) >>> minimizer 1 >>> res = optimize.golden(f, brack=(-1, 0.5, 2), full_output=True) >>> xmin, fval, funcalls = res >>> f(xmin), fval (9.925165290385052e-18, 9.925165290385052e-18) r,r>r;r)_minimize_scalar_goldenr.s r@r r sCJ/G !$t ?w ?C3xUS[003xrCc [U5 UnUc[XS9upppnO[U5S:Xa[XSUSUS9upppnOn[U5S:XaTUupn X:aXpX:aX:d [S5eU"U4U-6n U"U 4U-6n U"U 4U-6n X:aX:d [S5eSnO [S 5eS nS U- nU nUn[R "X- 5[R "X- 5:a U nU UX- --nO U nU UX- -- nU"U4U-6nU"U4U-6nUS- nSnUS:a![ S 5 [ S SS SSS SS35 [U5Hn[R "UU- 5U[R "U5[R "U5--::a OtUU:aUnUnUU-UU--nUnU"U4U-6nOUnUnUU-UU--nUnU"U4U-6nUS- nUS:a%UU:aUUnnOUUnn[ USS USS US35 US- nM UU:aUnUnOUnUnUU:=(a8 [R"U5=(d [R"U5(+nU(aSUS3nOHUU:aSn[R"U5(d[R"U5(a [SnU(a[U(+W5 [UUUUUWS9$)a Options ------- xtol : float Relative error in solution `xopt` acceptable for convergence. maxiter : int Maximum number of iterations to perform. disp: int, optional If non-zero, print messages. ``0`` : no message printing. ``1`` : non-convergence notification messages only. ``2`` : print a message on convergence too. ``3`` : print iteration results. r rrr!r rrrrgz7?rrrrr>rrrrr0r1r2r4r;rr>rr0r ) rr rrrErr%rrr$rr)rrrHrr2rrrr r r@rrrr_gR_gCx3rrrrerfrrrrEr0r s r@r4r4 s]*?+ C }+24+C( Uq+24!H6;AhT,K( Uq  GrwE REDL " REDL " REDL "rwI NOO C )C B B rw"&&/)  #/ !  #/ !  B  B MH C ax c  c"!C9AfT];< 7^ 66"r'?cRVVBZ"&&*%<= =  GBBrC"H$BB &BBBrC"H$BB &BA  !8Rddd XdO1T&M4-@ A q/4 RGmFRXXd^%Erxx~ FG$$(6- '>?G 88D>>RXXd^^(/0G &7{G< dTs")7 <>> import numpy as np >>> import matplotlib.pyplot as plt >>> from scipy.optimize import bracket >>> def f(x): ... return 10*x**2 + 3*x + 5 >>> x = np.linspace(-2, 2) >>> y = f(x) >>> init_xa, init_xb = 0.1, 1 >>> xa, xb, xc, fa, fb, fc, funcalls = bracket(f, xa=init_xa, xb=init_xb) >>> plt.axvline(x=init_xa, color="k", linestyle="--") >>> plt.axvline(x=init_xb, color="k", linestyle="--") >>> plt.plot(x, y, "-k") >>> plt.plot(xa, fa, "bx") >>> plt.plot(xb, fb, "rx") >>> plt.plot(xc, fc, "bx") >>> plt.show() Note that both initial points were to the right of the minimum, and the third point was found in the "downhill" direction: the direction in which the function appeared to be decreasing (to the left). The final points are strictly ordered, and the function value at the middle point is less than the function values at the endpoints; it follows that a minimum must lie within the bracket. gw?gO ;rrrzNo valid bracket was found before the iteration limit was reached. Consider trying different initial points or increasing `maxiter`.r!rzcThe algorithm terminated without finding a valid bracket. Consider trying different initial points.)rErr RuntimeErrorr BracketErrordata)rr r rH grow_limitr2_gold_verysmall_numrrr@rrrr r!valdenomrSwlimrrcond1cond2cond3rs r@r r d s.T EN ZZ !FB ut| B ut| B BB rw B  BH D 7BG$BG$k 66#; '.(E#IE 7d"bg%55> >"'**' >s# #   FrAv  $t %B MHRr'bg&&At %B MH$h #s *At %B MH$ha 3 &t %B MHR"'**QD4K)A bg&&At %B MH      q 7vW ! imins r@_recover_from_bracket_errorrM s$ AS4373 J  A!f+,66("|B 66"((B8$ % %VVRVVsAs99R=DXr$xsA#"#UCA A As  CB"C:CCcUR5unX$X4p2XXpX - U- nX0- U- nUS:n[R"XuS5n[R"USU5n [R"XvS5n [R"USU5n [R"X-5n [R"X-5n X:aX4$S$)aW Given a parameter vector ``x0`` with length ``n`` and a direction vector ``alpha`` with length ``n``, and lower and upper bounds on each of the ``n`` parameters, what are the bounds on a scalar ``l`` such that ``lower_bound <= x0 + alpha * l <= upper_bound``. Parameters ---------- x0 : np.array. The vector representing the current location. Note ``np.shape(x0) == (n,)``. alpha : np.array. The vector representing the direction. Note ``np.shape(alpha) == (n,)``. lower_bound : np.array. The lower bounds for each parameter in ``x0``. If the ``i``th parameter in ``x0`` is unbounded below, then ``lower_bound[i]`` should be ``-np.inf``. Note ``np.shape(lower_bound) == (n,)``. upper_bound : np.array. The upper bounds for each parameter in ``x0``. If the ``i``th parameter in ``x0`` is unbounded above, then ``upper_bound[i]`` should be ``np.inf``. Note ``np.shape(upper_bound) == (n,)``. Returns ------- res : tuple ``(lmin, lmax)`` The bounds for ``l`` such that ``lower_bound[i] <= x0[i] + alpha[i] * l <= upper_bound[i]`` for all ``i``. r)rr)nonzerorErrr#)rrr.r/rOlowhighposlmin_poslmin_neglmax_poslmax_neglminlmaxs r@_line_for_searchrY) sL}}HG*3[5I U^  u $C   %D !)Cxx!$HxxQ%Hxx1%HxxQ$H 66(% &D 66(% &D T"TUT--5$r:rY)rrrxis r@myfunc"_linesearch_powell..myfunc} sAbL!!rCN)rrr!)rErd)rc<>T"[R"U55$r:)rEtan)r>r]s r@$_linesearch_powell.. s&+rC)rErrMr-tupler>r;rYisneginfisposinf_linesearch_powellrarctanra) rrr\rr.r/rErb alpha_minfretboundr]s ``` @r@rgrgh s*" 66"::!%!1q" HQB7GH  !4)*@*0$cK%%4 ^QVR B A ;;uQx R[[q%:%:%dArF FU1X&&r{{58/D/D*65c JCB77AFB& & IIeAh'58)<>> def f(x): ... return x**2 >>> from scipy import optimize >>> minimum = optimize.fmin_powell(f, -1) Optimization terminated successfully. Current function value: 0.000000 Iterations: 2 Function evaluations: 16 >>> minimum array(0.0) )rrr2r1rdirecrrcr>r;rmrrrr)rt_minimize_powell)rrrHrrr2rrrrrcrmrrbrs r@rr sl  "Dh'H 4T EH E ECs8SZWs5zv;H /  I) )G s8S^+ +s8OrCc [U 5 Un U n[U5R5nU(aU/n[U5nUcU c US-nUS-n O[Uc+U [R :XaUS-nO>[R nO-U c*U[R :XaUS-n O[R n [ XU 5unnU c[U[S9n OX[U [S9n [RRU 5U RS:wa[R"S[SS9 UcSunnOnURUR nn[R""UU:5(d[R""UU:5(a[R"S[SS9 U"U5nUR%5nSnUnSnS n['U5H1nU UnUn[)XUUS -UUUS 9unnnUU- U:dM*UU- nUnM3 US - nU(aWR+U5 [-UUS 9n[/UU5(aGOMU[R0"U5[R0"U5--S-nSUU- -U::aGO USU :aGOUU:aO[R2"U5(a[R2"U5(aOUU- nUR%5nUcUcS n O[5UUUU5un!n U[7U S 5U--n"U"U"5nUU:aqSUU-SU-- -n#UU- U- n$U#U$U$--n#UU- n$U#UU$-U$--n#U#S :a=[)XUUS -UUUS 9unnn[R""U5(a U SU U'UU S'GMSn%[:Sn&U(aH[R""UU:5(d[R""UU:5(a Sn%[:Sn&OvUSU :a S n%[:Sn&OaUU:a Sn%[:Sn&OO[R2"U5(d)[R2"U5R#5(a Sn%[:Sn&U (a?[=U%U&[>5 [ASUS35 [ASU-5 [ASUS-5 [-UU UUSU%U%S:HU&US9n'U(aWU'S'U'$![8a GMHf=f)aU Minimization of scalar function of one or more variables using the modified Powell algorithm. Parameters ---------- fun : callable The objective function to be minimized:: fun(x, *args) -> float where ``x`` is a 1-D array with shape (n,) and ``args`` is a tuple of the fixed parameters needed to completely specify the function. x0 : ndarray, shape (n,) Initial guess. Array of real elements of size (n,), where ``n`` is the number of independent variables. args : tuple, optional Extra arguments passed to the objective function and its derivatives (`fun`, `jac` and `hess` functions). method : str or callable, optional The present documentation is specific to ``method='powell'``, but other options are available. See documentation for `scipy.optimize.minimize`. bounds : sequence or `Bounds`, optional Bounds on decision variables. There are two ways to specify the bounds: 1. Instance of `Bounds` class. 2. Sequence of ``(min, max)`` pairs for each element in `x`. None is used to specify no bound. If bounds are not provided, then an unbounded line search will be used. If bounds are provided and the initial guess is within the bounds, then every function evaluation throughout the minimization procedure will be within the bounds. If bounds are provided, the initial guess is outside the bounds, and `direc` is full rank (or left to default), then some function evaluations during the first iteration may be outside the bounds, but every function evaluation after the first iteration will be within the bounds. If `direc` is not full rank, then some parameters may not be optimized and the solution is not guaranteed to be within the bounds. options : dict, optional A dictionary of solver options. All methods accept the following generic options: maxiter : int Maximum number of iterations to perform. Depending on the method each iteration may use several function evaluations. disp : bool Set to True to print convergence messages. See method-specific options for ``method='powell'`` below. callback : callable, optional Called after each iteration. The signature is:: callback(xk) where ``xk`` is the current parameter vector. Returns ------- res : OptimizeResult The optimization result represented as a ``OptimizeResult`` object. Important attributes are: ``x`` the solution array, ``success`` a Boolean flag indicating if the optimizer exited successfully and ``message`` which describes the cause of the termination. See `OptimizeResult` for a description of other attributes. Options ------- disp : bool Set to True to print convergence messages. xtol : float Relative error in solution `xopt` acceptable for convergence. ftol : float Relative error in ``fun(xopt)`` acceptable for convergence. maxiter, maxfev : int Maximum allowed number of iterations and function evaluations. Will default to ``N*1000``, where ``N`` is the number of variables, if neither `maxiter` or `maxfev` is set. If both `maxiter` and `maxfev` are set, minimization will stop at the first reached. direc : ndarray Initial set of direction vectors for the Powell method. return_all : bool, optional Set to True to return a list of the best solution at each of the iterations. rrzBdirec input is not full rank, some parameters may not be optimizedrr)NNrrr_)rr.r/rEr!rg#B ;rrr0rr5r1rr2r4rrrr )r;rmrrrr0r r>r)!rrr rrErrrrr matrix_rankrrrrrrrrGrrgr rr*rrrYr#rr$rrr%)(rrrHrcrrrr2r1rrmrrrrr>rr0r7r.r/rErrrbiginddeltardirec1fx2r_bndrX_rttemprFrrGs( r@rnrn/ sx?+ F F A # AA6>d(T  RVV $hGffG  bff XFVVF;4vNLFD }AU#U+ 99  '5;;q> 9 MM-)a 9~ $. [$*99fii[ 66+" # #rvvb;.>'?'? MML)a 9 7D B D = BFE1Xq"4Tf9=ALAL:> #@a $J%'$JEF AIDq!"01$"? (3FGG"&&*rvvd|34u Et&)#+h!m$'1.F#y MA!   s867R=1AR=6?R=7 R=R= 6R=CR== S  S cUS:XaUS:a [SUS5 gUS:XaSnOUS:Xa S[S3n[UW5 g) Nrr!r0)zO Maximum number of function evaluations exceeded --- increase maxfun argument. r r4)r%r$r)r>rrErrrrs r@rr!s[ qy !8 #$(# /  qy- ?5)*+"4- rCrc [U5nUS:a [S5e[U5n [U5HUn [ X[ 5(aM[X5S:a[ X5[U54-X'[ X6X'MW US:XaU Sn [RU n U Rn US:a<[R"XS[R"U SS545Rn [R"U5(dU4n[X5n [!US9n[R""[U"X555nUS:XaU 4n [R$"U5nO@US:a:[R"XSS5n[R"U RU 5n SSS5 [W5n['UR)5SS 9n[R*"U[,5n[R*"U[.5n[US- SS5Hn UU nUUU -UU 'UU-nM [U5Hn X[ U5UU 'M U[ U5nUS:Xa U Sn USn[1U5(a[3U5R4n[75nS U;aSUS 'S U;aUUS 'O S U;aS U0US 'U"UU4S U0UD6n[ U[85(a%UR:nUR<nUR>nOUSnUSnUSS:HnU(d!U(a[@RB"S[DSS9 U(aUUX4$U$!,(df  GN=f)aMinimize a function over a given range by brute force. Uses the "brute force" method, i.e., computes the function's value at each point of a multidimensional grid of points, to find the global minimum of the function. The function is evaluated everywhere in the range with the datatype of the first call to the function, as enforced by the ``vectorize`` NumPy function. The value and type of the function evaluation returned when ``full_output=True`` are affected in addition by the ``finish`` argument (see Notes). The brute force approach is inefficient because the number of grid points increases exponentially - the number of grid points to evaluate is ``Ns ** len(x)``. Consequently, even with coarse grid spacing, even moderately sized problems can take a long time to run, and/or run into memory limitations. Parameters ---------- func : callable The objective function to be minimized. Must be in the form ``f(x, *args)``, where ``x`` is the argument in the form of a 1-D array and ``args`` is a tuple of any additional fixed parameters needed to completely specify the function. ranges : tuple Each component of the `ranges` tuple must be either a "slice object" or a range tuple of the form ``(low, high)``. The program uses these to create the grid of points on which the objective function will be computed. See `Note 2` for more detail. args : tuple, optional Any additional fixed parameters needed to completely specify the function. Ns : int, optional Number of grid points along the axes, if not otherwise specified. See `Note2`. full_output : bool, optional If True, return the evaluation grid and the objective function's values on it. finish : callable, optional An optimization function that is called with the result of brute force minimization as initial guess. `finish` should take `func` and the initial guess as positional arguments, and take `args` as keyword arguments. It may additionally take `full_output` and/or `disp` as keyword arguments. Use None if no "polishing" function is to be used. See Notes for more details. disp : bool, optional Set to True to print convergence messages from the `finish` callable. workers : int or map-like callable, optional If `workers` is an int the grid is subdivided into `workers` sections and evaluated in parallel (uses `multiprocessing.Pool `). Supply `-1` to use all cores available to the Process. Alternatively supply a map-like callable, such as `multiprocessing.Pool.map` for evaluating the grid in parallel. This evaluation is carried out as ``workers(func, iterable)``. Requires that `func` be pickleable. .. versionadded:: 1.3.0 Returns ------- x0 : ndarray A 1-D array containing the coordinates of a point at which the objective function had its minimum value. (See `Note 1` for which point is returned.) fval : float Function value at the point `x0`. (Returned when `full_output` is True.) grid : tuple Representation of the evaluation grid. It has the same length as `x0`. (Returned when `full_output` is True.) Jout : ndarray Function values at each point of the evaluation grid, i.e., ``Jout = func(*grid)``. (Returned when `full_output` is True.) See Also -------- basinhopping, differential_evolution Notes ----- *Note 1*: The program finds the gridpoint at which the lowest value of the objective function occurs. If `finish` is None, that is the point returned. When the global minimum occurs within (or not very far outside) the grid's boundaries, and the grid is fine enough, that point will be in the neighborhood of the global minimum. However, users often employ some other optimization program to "polish" the gridpoint values, i.e., to seek a more precise (local) minimum near `brute's` best gridpoint. The `brute` function's `finish` option provides a convenient way to do that. Any polishing program used must take `brute's` output as its initial guess as a positional argument, and take `brute's` input values for `args` as keyword arguments, otherwise an error will be raised. It may additionally take `full_output` and/or `disp` as keyword arguments. `brute` assumes that the `finish` function returns either an `OptimizeResult` object or a tuple in the form: ``(xmin, Jmin, ... , statuscode)``, where ``xmin`` is the minimizing value of the argument, ``Jmin`` is the minimum value of the objective function, "..." may be some other returned values (which are not used by `brute`), and ``statuscode`` is the status code of the `finish` program. Note that when `finish` is not None, the values returned are those of the `finish` program, *not* the gridpoint ones. Consequently, while `brute` confines its search to the input grid points, the `finish` program's results usually will not coincide with any gridpoint, and may fall outside the grid's boundary. Thus, if a minimum only needs to be found over the provided grid points, make sure to pass in ``finish=None``. *Note 2*: The grid of points is a `numpy.mgrid` object. For `brute` the `ranges` and `Ns` inputs have the following effect. Each component of the `ranges` tuple can be either a slice object or a two-tuple giving a range of values, such as (0, 5). If the component is a slice object, `brute` uses it directly. If the component is a two-tuple range, `brute` internally converts it to a slice object that interpolates `Ns` points from its low-value to its high-value, inclusive. Examples -------- We illustrate the use of `brute` to seek the global minimum of a function of two variables that is given as the sum of a positive-definite quadratic and two deep "Gaussian-shaped" craters. Specifically, define the objective function `f` as the sum of three other functions, ``f = f1 + f2 + f3``. We suppose each of these has a signature ``(z, *params)``, where ``z = (x, y)``, and ``params`` and the functions are as defined below. >>> import numpy as np >>> params = (2, 3, 7, 8, 9, 10, 44, -1, 2, 26, 1, -2, 0.5) >>> def f1(z, *params): ... x, y = z ... a, b, c, d, e, f, g, h, i, j, k, l, scale = params ... return (a * x**2 + b * x * y + c * y**2 + d*x + e*y + f) >>> def f2(z, *params): ... x, y = z ... a, b, c, d, e, f, g, h, i, j, k, l, scale = params ... return (-g*np.exp(-((x-h)**2 + (y-i)**2) / scale)) >>> def f3(z, *params): ... x, y = z ... a, b, c, d, e, f, g, h, i, j, k, l, scale = params ... return (-j*np.exp(-((x-k)**2 + (y-l)**2) / scale)) >>> def f(z, *params): ... return f1(z, *params) + f2(z, *params) + f3(z, *params) Thus, the objective function may have local minima near the minimum of each of the three functions of which it is composed. To use `fmin` to polish its gridpoint result, we may then continue as follows: >>> rranges = (slice(-4, 4, 0.25), slice(-4, 4, 0.25)) >>> from scipy import optimize >>> resbrute = optimize.brute(f, rranges, args=params, full_output=True, ... finish=optimize.fmin) >>> resbrute[0] # global minimum array([-1.05665192, 1.80834843]) >>> resbrute[1] # function value at global minimum -3.4085818767 Note that if `finish` had been set to None, we would have gotten the gridpoint [-1.0 1.75] where the rounded function value is -2.892. (z5Brute Force not possible with more than 40 variables.rr!rN)poolrrrrrrHzhEither final optimization did not succeed or `finish` does not return `statuscode` as its last argument.rr)#rrrrrslicerdcomplexrEmgridrreshapeprodTiterable_Brute_Wrapperr'rrrrrrrr_getfullargspecrHdictrr>r;r0rrr)rrangesrHNsrfinishrworkersr0lranger2grid inpt_shape wrapped_funcmapperJoutNshapeindxNindxrthisNJmin finish_args finish_kwargsrbr0s r@rr3s4Z F A2v./ / &\F 1X&)U++69~!!&), ~= vy)FI  Q 88F DJ Azz$A 120G HIKK ;;t  w!$-L  !VxxVL789 F7D::d#D!e::dqrN3D::dffj1D "4[F $**,R (D HHQ E 88Au D 1q5"b !q &)#au}"1X'%,'Q e D QAwAw%f-22  K '+,M- ( [ $(M& ! + %)/~M) $T4 * *55D77DkkGq6Dq6D"glG S,<T4%% w " !s *BM!! M0c$\rSrSrSrSrSrSrg)ri8zM Object to wrap user cost function for optimize.brute, allowing picklability c4XlUc/UlgUUlgr:)rrH)r?rrHs r@rA_Brute_Wrapper.__init__=s,B D rCcUR"[R"U5R5/URQ76$r:)rrErr rH)r?r>s r@rN_Brute_Wrapper.__call__As+vvbjjm++-: ::rC)rHrN)rSrTrUrVrWrArNrXrYrCr@rr8s1;rCrc 2SSKnSSSSSSS .nUc/S QnUR[S S S 95 UR/SQ5 UR[SS S 95 UR/SQ5 UR[SS S 95 UR/SQ5 UR[SS S 95 SR U5nGO?UR 5nX;a[ SU<35eUcb/nX@HFupgURSU-SS[U5--S-/5 UR[XS S 95 MH SR U5nOUR 5n[X@5nX;a[ SU<35eXnURS5n SR U SS5n [U 5 [[RU U S5n U Rn U b URU 5R!5nOSnU(a [#U5 gU$)a Show documentation for additional options of optimization solvers. These are method-specific options that can be supplied through the ``options`` dict. Parameters ---------- solver : str Type of optimization solver. One of 'minimize', 'minimize_scalar', 'root', 'root_scalar', 'linprog', or 'quadratic_assignment'. method : str, optional If not given, shows all methods of the specified solver. Otherwise, show only the options for the specified method. Valid values corresponds to methods' names of respective solver (e.g., 'BFGS' for 'minimize'). disp : bool, optional Whether to print the result rather than returning it. Returns ------- text Either None (for disp=True) or the text string (disp=False) Notes ----- The solver-specific methods are: `scipy.optimize.minimize` - :ref:`Nelder-Mead ` - :ref:`Powell ` - :ref:`CG ` - :ref:`BFGS ` - :ref:`Newton-CG ` - :ref:`L-BFGS-B ` - :ref:`TNC ` - :ref:`COBYLA ` - :ref:`COBYQA ` - :ref:`SLSQP ` - :ref:`dogleg ` - :ref:`trust-ncg ` `scipy.optimize.root` - :ref:`hybr ` - :ref:`lm ` - :ref:`broyden1 ` - :ref:`broyden2 ` - :ref:`anderson ` - :ref:`linearmixing ` - :ref:`diagbroyden ` - :ref:`excitingmixing ` - :ref:`krylov ` - :ref:`df-sane ` `scipy.optimize.minimize_scalar` - :ref:`brent ` - :ref:`golden ` - :ref:`bounded ` `scipy.optimize.root_scalar` - :ref:`bisect ` - :ref:`brentq ` - :ref:`brenth ` - :ref:`ridder ` - :ref:`toms748 ` - :ref:`newton ` - :ref:`secant ` - :ref:`halley ` `scipy.optimize.linprog` - :ref:`simplex ` - :ref:`interior-point ` - :ref:`revised simplex ` - :ref:`highs ` - :ref:`highs-ds ` - :ref:`highs-ipm ` `scipy.optimize.quadratic_assignment` - :ref:`faq ` - :ref:`2opt ` Examples -------- We can print documentations of a solver in stdout: >>> from scipy.optimize import show_options >>> show_options(solver="minimize") ... Specifying a method is possible: >>> show_options(solver="minimize", method="Nelder-Mead") ... We can also get the documentations as a string: >>> show_options(solver="minimize", method="Nelder-Mead", disp=False) Minimization of scalar function of one or more variables using the ... rN))bfgsz'scipy.optimize._optimize._minimize_bfgs)cgz%scipy.optimize._optimize._minimize_cg)r]z*scipy.optimize._cobyla_py._minimize_cobyla)r^z*scipy.optimize._cobyqa_py._minimize_cobyqa)doglegz3scipy.optimize._trustregion_dogleg._minimize_dogleg)zl-bfgs-bz*scipy.optimize._lbfgsb_py._minimize_lbfgsb)z nelder-meadz-scipy.optimize._optimize._minimize_neldermead)z newton-cgz+scipy.optimize._optimize._minimize_newtoncg)powellz)scipy.optimize._optimize._minimize_powell)r\z(scipy.optimize._slsqp_py._minimize_slsqp)r[z!scipy.optimize._tnc._minimize_tnc)z trust-ncgz3scipy.optimize._trustregion_ncg._minimize_trust_ncg)rfz?scipy.optimize._trustregion_constr._minimize_trustregion_constr)z trust-exactz=scipy.optimize._trustregion_exact._minimize_trustregion_exact)z trust-krylovz9scipy.optimize._trustregion_krylov._minimize_trust_krylov) )hybrz%scipy.optimize._minpack_py._root_hybr)lmz"scipy.optimize._root._root_leastsq)broyden1z'scipy.optimize._root._root_broyden1_doc)broyden2z'scipy.optimize._root._root_broyden2_doc)andersonz'scipy.optimize._root._root_anderson_doc) diagbroydenz*scipy.optimize._root._root_diagbroyden_doc)excitingmixingz-scipy.optimize._root._root_excitingmixing_doc) linearmixingz+scipy.optimize._root._root_linearmixing_doc)krylovz%scipy.optimize._root._root_krylov_doc)zdf-sanez&scipy.optimize._spectral._root_df_sane))bisectz3scipy.optimize._root_scalar._root_scalar_bisect_doc)brentqz3scipy.optimize._root_scalar._root_scalar_brentq_doc)brenthz3scipy.optimize._root_scalar._root_scalar_brenth_doc)ridderz3scipy.optimize._root_scalar._root_scalar_ridder_doc)toms748z4scipy.optimize._root_scalar._root_scalar_toms748_doc)secantz3scipy.optimize._root_scalar._root_scalar_secant_doc)newtonz3scipy.optimize._root_scalar._root_scalar_newton_doc)halleyz3scipy.optimize._root_scalar._root_scalar_halley_doc))simplexz,scipy.optimize._linprog._linprog_simplex_doc)zinterior-pointz'scipy.optimize._linprog._linprog_ip_doc)zrevised simplexz'scipy.optimize._linprog._linprog_rs_doc)z highs-ipmz.scipy.optimize._linprog._linprog_highs_ipm_doc)zhighs-dsz-scipy.optimize._linprog._linprog_highs_ds_doc)highsz*scipy.optimize._linprog._linprog_highs_doc))faqz-scipy.optimize._qap._quadratic_assignment_faq)2optz.scipy.optimize._qap._quadratic_assignment_2opt))rz/scipy.optimize._optimize._minimize_scalar_brent)boundedz1scipy.optimize._optimize._minimize_scalar_bounded)r z0scipy.optimize._optimize._minimize_scalar_golden)minimizeroot root_scalarlinprogquadratic_assignmentminimize_scalar)z ======== z minimize z ======== rF)r)z =============== zminimize_scalar z=============== r)z ==== zroot z==== r)z ======= zlinprog z======= rrzUnknown solver z r|=zUnknown method .r)textwrapr rextendrlowerrrrsplit __import__getattrsysmodulesrWdedentstripr%) rIrrrr doc_routinestextnamerwmethodspartsmod_nameobjdocs r@rrFsV ,     !  s>L@~? L%89 * + L!2?@   Le45 " # L78wwt}  %vj9: : >D'/ Vd]D3s4y=,@6,IJK LEBC0774=D\\^F</0G$ ?6*!=>>?DJJsOExxcr +H x #++h/r;C++Cs+113  d  rCr:)r)NrYNNNN) rYrrNNrr!rNN) rYNNNFFNrrFN)rYr}r*rr!)rYr}r*r)rYNr)rr*)NrYr)r*r)rrrYg[@rp)gMbP?NNN) rYNNrrNNFNF)NNT)e__all__ __docformat__rrrrmnumpyrrrrrrrE scipy.linalgrrrscipy.sparse.linalgr _linesearchr"r#rr$_numdiffr%scipy._lib._utilr&rr'r(r)r*r+(scipy.optimize._differentiable_functionsr,r-scipy._lib._array_apir. scipy._libr/rr$r7rtr UserWarningrrrrrrr_epsilonrrrrr r r rrr;rrrrrrrgrir|rrrrrrrrrrrrr-r r4r r<rMrYrgrrnrrrrrYrCr@rst(  &  ::;;.--(FMMO1-F/*33%. /60* [* Z k  LW. ## $;AE@D"&[|  0 f'T(4V*Z$0  $6HLIMzz6:9>;?HLL^"*N/bF+.6TZ,Zz | *Z!r266!!Q4CM`"$266x%dt! nbJRdd!dstn "tdXtDSWt%)2tt1tup&(TD $4Xt!ey!x8;"#[|13;<SlrrjM`