J-jǠNdZddlmZmZmZ[[[dZdZddlZddlZddl m Z ddl Z ddl mZd d lmZd d lmZd d lmZd ZdZGddeZGddeZGddeZGddeZGddeZdZ dZ!GddeZ"GddeZ#y#e$rdZYvwxYw)zIFitness surrogate model classes and handler for incremental evaluations. )absolute_importdivisionprint_functionzNikolaus Hansenz BSD 3-clauseN) defaultdict kendalltauct||dfSN) _kendall_tau)xys `/Users/jameslopez/projects/TradingBot25/.venv/lib/python3.12/site-packages/cma/fitness_models.py _kendalltaursAq!4'')utils) LoggerDummy)DefaultSettingsc d}t|t|k(sJtj|tj|}}d}tt|D]}ddkrktj|||d|z }tj|||d|z }|t ||zz }|sX||t |dk(|dk(zzz }st|D]b}|tj||||z tj||||z zz }|sF||||||k(z||||k(zz }d|dzt|t|dz zz }ddkriddlm} | ||d} tj| rB| |dt|z z ks| |dt|z zkDrtjd || fz|S) aKreturn Kendall tau rank correlation coefficient. Implemented only to potentially remove dependency on `scipy.stats`. This >>> import numpy as np >>> from cma.fitness_models import _kendall_tau >>> kendalltau = lambda x, y: (_kendall_tau(x, y), 0) >>> # from scipy.stats import kendalltau # incomment if not available >>> for dim in np.random.randint(3, 22, 5): ... x, y = np.random.randn(dim), np.random.randn(dim) ... t1, t2 = _kendall_tau(x, y), kendalltau(x, y)[0] ... # print(t1, t2) ... assert np.isclose(t1, t2) ?rrNg@ rz tau=%f not close to stats.tau=%f) lennpasarrayrangesignsum scipy.statsrisfinitewarningswarn) r r equal_value_contributionsidxdyjtaurts rr r s$ % q6SV   ::a="**Q-qA A 3q6] q51"1&B1"1&B R"W A'-R1Wq4I0JJJ1XRWWQqTAaD[)BGGAaD1Q4K,@@@+1QqTQqT\BadaPQdlSSA b&CFc!fqj) *C Av* q! Q  ;;q>3SV##q3SV+;'; @C8KL Jrcddkrt||}|S t||d}tj|sd}|S#t$rd}Y&wxYw)zAreturn rank correlation coefficient between data `x` and `y` rrr)r r TypeErrorrr )r r r)s r kendall_taur-Csb Av1a  J  a#A&C{{3C J  C s= A  A c2eZdZdZdZdZdZdZdZdZ dZ dZ y ) SurrogatePopulationSettingsrc Pttdtd|zd|zffS)Ng333333??)intmaxmin)popsi nevaluateds rz$SurrogatePopulationSettings.Ss'#c2sC*=3 evaluations (compared to LS-CMA and lmm-CMA) - the model is linear at first, then diagonal, then full - the model uses the fitness ranks as weights for weighted regression. >>> import cma >>> import cma.fitness_models as fm >>> from cma.fitness_transformations import Function as FFun # adds evaluations attribute >>> # fm.Logger, Logger = fm.LoggerDummy, fm.Logger >>> surrogate = fm.SurrogatePopulation(cma.ff.elli) Example using the ask-and-tell interface: >>> for ifun, fun in enumerate([cma.ff.elli, cma.ff.sectorsphere]): ... for irun in range(7): ... assert irun < 4 # measure best of four ... es = cma.CMAEvolutionStrategy(5 * [1], 2.2, ... {'CMA_injections_threshold_keep_len': 1, ... 'ftarget':1e-9, 'verbose': -9, 'seed':3 + irun}) ... fitfun = FFun(fun) ... surrogate = fm.SurrogatePopulation(fitfun) ... while not es.stop(): ... X = es.ask() ... es.tell(X, surrogate(X)) # surrogate evaluation ... es.inject([surrogate.model.xopt]) ... # es.disp(); es.logger.add() # ineffective with verbose=-9 ... # print(irun, fitfun.evaluations) # was: (sig=2.2) 12 161, 18 131, 18 150, 18 82, 15 59, 15 87, 15 132, 18 83, 18 55, 18 68, 19 231 75, ... assert 'ftarget' in es.stop(), es.result.asdict ... if ifun == 0: ... if fitfun.evaluations < 20: ... break ... if ifun == 1: ... if fitfun.evaluations < 90: ... break Example using the ``parallel_objective`` interface to `cma.fmin`: >>> for fitfun, evals in [[FFun(cma.ff.elli), 22], [FFun(cma.ff.ellirot), 40]]: ... surrogate = fm.SurrogatePopulation(fitfun) ... inject_xopt = fm.ModelInjectionCallback(surrogate.model) # must use the same model ... xopt, es = cma.fmin2(None, 5 * [1], 2.2, ... {'CMA_injections_threshold_keep_len': 1, ... 'ftarget':1e-12, 'verbose': -9}, ... parallel_objective=surrogate, ... callback=inject_xopt) ... # print(fitfun.evaluations) ... assert fitfun.evaluations == es.result.evaluations ... assert es.result[1] < 1e-12 ... assert es.result[2] < evals >>> # fm.Logger = Logger Nc||_|r|n t|_tt d||_d|_d|_t|gd|_ t|jdg|_ y)z If ``model is None``, a default `LQModel` instance is used. By setting `self.model` to `None`, only the `fitness` for each population member is evaluated in each call. r9r)tau0tau1zevaluated ratiolabels eigenvaluesN) fitnessLQModelmodelr/localssettingscount evaluationsLoggerloggerlogger_eigenvalues)selfrOrQr?r@s r__init__zSurrogatePopulation.__init__s] #U 3FHaF  T*MN "(m_"ErcleZdZdZdZdZdZd dZ d dZdZ e d Z e d Z e d Z y)%SurrogatePopulation.EvaluationManageraManage incremental evaluation of a population of solutions. Evaluate solutions, add them to the model and keep track of which solutions were evaluated. Uses `model.add_data_row` and `model.eval`. Details: for simplicity and avoiding the copy construction, we do not inherit from `list`. Hence we use ``self.X`` instead of ``self``. c~||_t|dgz|_t|tjgz|_y)z6all is based on the population (list of solutions) `X`FN)Xr evaluatedrnanfvalues)rYr^s rrZz.SurrogatePopulation.EvaluationManager.__init__s1DF Vug-DNq6RVVH,DLrc@||j|<d|j|<y)z"add fitness(self.X[i]), not in useTN)rar_)rYr%fvals radd_evalz.SurrogatePopulation.EvaluationManager.add_evals"DLLO $DNN1 rc|j|r$td||jt|fz||j||j |<d|j|<||j||j |y)z=add fitness(self.X[i]) to model data, mainly for internal usezi=%d, evals=%d, len=%dTN)r_ ValueErrorrUrr^ra)rYr%rO model_adds revalz*SurrogatePopulation.EvaluationManager.evalsu~~a  !9Q@P@PRUVZR[<\!\]]%dffQi0DLLO $DNN1  dffQia 1rNc|tt|j}t|jt|jk(sJ|j|ks8t j d|j|t|jfz||jz |_|D]6}|j|k\rn|j|r$|j|||8|j|krZ|jt|jkr8t j d|j|t|jfzy|j|k(s.|jt|jcxk(r|ksJJyy)aAevaluate unevaluated entries of X[idx] until `number` entries are evaluated *overall*. Assumes that ``sorted(idx) == list(range(len(self.X)))``. ``idx`` defines the evaluation sequence. The post condition is ``self.evaluations == min(number, len(self.X))``. Nz/Expected evaluations=%d < number=%d, popsize=%dz2After eval: evaluations=%d < number=%d, popsize=%d) rrr^r_rUr!r"last_evaluationsrh)rYnumberrOrgidxr%s r eval_sequencez3SurrogatePopulation.EvaluationManager.eval_sequencesT{CK(t~~&#dff+5 55##f, O!%!1!163tvv; GHI$*T-=-=$=D !##v-~~a(IIa)4  ##f,1A1ACK1OMM"V%)%5%5vs466{$K#LM##v-1A1AS[1YSY1Y YY1Y YY1Y-rcz|r.|jt|jk(r |jS|jDcgc] }|| }}|t j |j}t j |r+t j |}|Dcgc] }||z |z c}S|Scc}wcc}w)zGreturn surrogate values of `model_eval` with smart offset. )rUrr^rarnanminr )rY model_evaltrue_values_if_all_availablef_offsetr F_modelm_offsetfs rsurrogate_valuesz6SurrogatePopulation.EvaluationManager.surrogate_valuess,0@0@CK0O||#.2ff5fz!}fG599T\\2{{8$99W-9@AAH x/AA6 Bs B3B8c,t|jS)z8should replace ``len(self.X)`` etc, not fully in use yet)rrarYs r__len__z-SurrogatePopulation.EvaluationManager.__len__st||$ $rcF|jt|jz Sr )rUrrarxs revaluation_fractionz9SurrogatePopulation.EvaluationManager.evaluation_fractions##c$,,&77 7rc,t|jSr )rr_rxs rrUz1SurrogatePopulation.EvaluationManager.evaluationsst~~& &rcXt|jt|jz S)z%number of not yet evaluated solutions)rr^rr_rxs r remainingz/SurrogatePopulation.EvaluationManager.remainings tvv;T^^!44 4rr )TN)r:r;r<__doc__rZrdrhrmrvrypropertyr{rUr~rFrrEvaluationManagerr\sh  -  % 2 Z8MQ&*  %  8  8  '  '  5  5rrc |xjdz c_|j}|jjt |z|jj kDrW|jj r|j |jjt |z|j_tj|}||_ ddkr|j|jjkr|j|jj|jz |j|j|xj|jz c_|j!|jj"xs |j|j%|j&|jj(SddkrP|jd|jj*zzr*|j%|j&|j,dSt/dt1t ||jj2zdz d|jj4z |jz fz}|j6rU|jdkDr2t9j:|Dcgc]}|j'|c}nd}|j||j|j||j!||j=|jj?t ||j}|j@|k(r|jBjE|||jjF|jjH|jJzz k\rn3|t/t9jL|dz z } |j6rU|j!|jj"xs |j|jO|jBjE||xj|jz c_|jdk(rd |_|jBjE|jt |z |jjPr|jBjS|j%|j&|jj(Scc}w) a;return population f-values. Also update the underlying model. Evaluate at least one solution on the true fitness. The smallest returned value is never smaller than the smallest truly evaluated value. Uses (this may not be a complete list): `model.settings.max_absolute_size`, `model.settings.truncation_ratio`, `model.size`, `model.sort`, `model.eval`, `model.reset`, `model.add_data_row`, `model.kendall`, `model.adapt_max_relative_size`, relies on default value zero for ``max_absolute_size``. rrrr)rrdNr9g{Gz?)*rTrQrSr?rmax_absolute_sizeresetrHrevalssizer=rmrO add_data_rowrUsortrCrvrhrD crazy_sloppyFr3r4rBtruncation_ratior~rargsortkendallr>rjrWaddr@rAr{ceiladapt_max_relative_sizerEpush)rYr^rQrnumber_evaluatedr rlr)s r__call__zSurrogatePopulation.__call__s a   == . .Q 7%..:Z:Z Z~~// /3}}/R/RUXYZU[/[ENN ,#55a8 6zzDMM<<<##DMM$D$Duzz$Q$(LL%2D2DF  E$5$55  4==<<Q@Q@QR--ejj$--:]:]^^ 6djjA (B(B$BC))%**uwwqz)J Jq3A1P1P(PSV(V()ENN,K,K(Kejj(X(Z$[ [\oo  DDrttSyPSUSXSXS`S`SfSffg 4==??"$$)KbeemmNaNaa a HH 22 2H KKOOE " ' ' ) KKOOUF # ( ( *rN)r:r;r<rrZrrFrrrrKs J +rrceZdZdZy)Tauz/placeholder to store Kendall tau related thingsN)r:r;r<rrFrrrrgs5rrctt|jdz|jj|j dzzf}t |j |fS)z+truncate worst solutions for model buildingr9r)r3r4current_complexityrSrrr5)mns r_n_for_model_building_defaultrjsR C%%) ++qvvz:< = >A { rc@tj|jSr )rrY)rs r_sorted_index_defaultrps ::acc?rc^eZdZdZdZdZeddez dz fZdZdZ dZ e Z d Z d Zd ZeZd Zy) LQModelSettingsNr9g?r2rrg?rrFFcd|jcxkrdksntd|jz|jxs |j}d|j|jz cxkr|cxkr|jksEntd|jt |j|j|jfz|S)Nrrz9need: 0 < truncation_ratio <= 1, was: truncation_ratio=%fzpneed max_relative_size_end=%f >= max_relative_size_init=%s >= min_relative_size/self.truncation_ratio=%f/%f >= 0)rrfmax_relative_size_initmax_relative_size_endmin_relative_sizestr)rYmax_inits r _checkingzLQModelSettings._checkings4((-A-K%%&' '..L$2L2L4))D,A,AAkXkQUQkQkkC++S1L1L-MtOeOegkg|g|}~  r)r:r;r<rrmax_relative_size_factorr4r tau_threshold_for_model_increaserrrn_for_model_building max_weightdisallowed_typesf_transformationr sorted_indexrrFrrrrssb!#C!&;";q!@AB'*$8J(L rrceZdZdZddgddggZeDcgc]}|d c}}ZeeZedZ d/d Z d Z d0d Z ed Z ed ZedZedZedZdZdZdZdZd1dZd2dZdej2fdZdej2fdZdZdZdZdZdZ dZ!dZ"d Z#d0d!Z$d"Z%d#Z&d$Z'd%Z(d&Z)d'Z*ed(Z+ed)Z,ed*Z-ed+Z.ed,Z/ed-Z0ed.Z1ycc}}w)3rPaG Up to a full quadratic model using the pseudo inverse to compute the model coefficients. The full model has 1 + 2n + n(n-1)/2 = n(n+3) + 1 parameters. Model building "works" with any number of data. Model size 1.0 doesn't work well on bbob-f10, 1.1 however works fine. TODO: change self.types: List[str] to self.type: str with only one entry >>> import numpy as np >>> import cma >>> import cma.fitness_models as fm >>> # fm.Logger, Logger = fm.LoggerDummy, fm.Logger >>> m = fm.LQModel() >>> for i in range(30): ... x = np.random.randn(3) ... y = cma.ff.elli(x - 1.2) ... _ = m.add_data_row(x, y) >>> assert np.allclose(m.coefficients, [ ... 1.44144144e+06, ... -2.40000000e+00, -2.40000000e+03, -2.40000000e+06, ... 1.00000000e+00, 1.00000000e+03, 1.00000000e+06, ... -4.65661287e-10, -6.98491931e-10, 1.97906047e-09, ... ], atol=1e-5) >>> assert np.allclose(m.xopt, [ 1.2, 1.2, 1.2]) >>> assert np.allclose(m.xopt, [ 1.2, 1.2, 1.2]) Check the same before the full model is build: >>> m = fm.LQModel() >>> m.settings.min_relative_size = 3 * m.settings.truncation_ratio >>> for i in range(30): ... x = np.random.randn(4) ... y = cma.ff.elli(x - 1.2) ... _ = m.add_data_row(x, y) >>> print(m.types) ['quadratic'] >>> assert np.allclose(m.coefficients, [ ... 1.45454544e+06, ... -2.40000000e+00, -2.40000000e+02, -2.40000000e+04, -2.40000000e+06, ... 1.00000000e+00, 1.00000000e+02, 1.00000000e+04, 1.00000000e+06, ... ]) >>> assert np.allclose(m.xopt, [ 1.2, 1.2, 1.2, 1.2]) >>> assert np.allclose(m.xopt, [ 1.2, 1.2, 1.2, 1.2]) Check the Hessian in the rotated case: >>> fitness = cma.fitness_transformations.Rotated(cma.ff.elli) >>> m = fm.LQModel(2, 2) >>> for i in range(30): ... x = np.random.randn(4) - 5 ... y = fitness(x - 2.2) ... _ = m.add_data_row(x, y) >>> R = fitness[1].dicMatrices[4] >>> H = np.dot(np.dot(R.T, np.diag([1, 1e2, 1e4, 1e6])), R) >>> assert np.all(np.isclose(H, m.hessian)) >>> assert np.allclose(m.xopt, 4 * [2.2]) >>> m.set_xoffset([2.335, 1.2, 2, 4]) >>> assert np.all(np.isclose(H, m.hessian)) >>> assert np.allclose(m.xopt, 4 * [2.2]) Check a simple linear case, the optimum is not necessarily at the expected position (the Hessian matrix is chosen somewhat arbitrarily) >>> m = fm.LQModel() >>> m.settings.min_relative_size = 4 >>> _ = m.add_data_row([1, 1, 1], 220 + 10) >>> _ = m.add_data_row([2, 1, 1], 220) >>> print(m.types) [] >>> assert np.allclose(m.coefficients, [80, -10, 80, 80]) >>> assert np.allclose(m.xopt, [22, -159, -159]) # [ 50, -400, -400]) # depends on Hessian >>> # fm.Logger = Logger For results see: Hansen (2019). A Global Surrogate Model for CMA-ES. In Genetic and Evolutionary Computation Conference (GECCO 2019), Proceedings, ACM. lq-CMA-ES at http://lq-cma.gforge.inria.fr/ppdata-archives/pap-gecco2019/figure5/ quadraticcd|zdzS)Nr9rrFds rr8zLQModel.s A rfullc0t||dzzdz dzS)Nrr9r)r3rs rr8zLQModel.s3qAE{Q/!3rrcvjrtfdjDSjdzS)z:degrees of freedom (nb of parameters) of the current modelc3\K|]#}j|j%ywr ) complexitydim).0r*rYs r z-LQModel.current_complexity..s'HZ)tq)$((3Zs),r)rr4rrxs`rrzLQModel.current_complexitys/ ::HTZZHH Hxx!|rNc0ttd|j|_gd|_dggz\|_|_|_|_|_ |_ t|dgddg|_ t|dgd |_ |jy ) a Increase model complexity if the number of data exceeds ``max(min_relative_size * df_biggest_model_type, self.min_absolute_size)``. Limit the number of kept data ``max(max_absolute_size, max_relative_size * max_df)``. )r^rrZcountshashes logging_tracez||X[0]-X[1]||^2z||X[0]-Xopt||^2rLrNModeleigenvalues)nameN)rrRrrS _fieldnamesr^rrrrrrVrWlog_eigenvaluesr)rYrrrrrs rrZzLQModel.__init__s (!T:DDF CCDt8@ T[TO#4%68I%KL  &d]O+= ? rc|jD]}t||gg|_tt|_d|_ d|_|jjxs|jj|_ d|_ d|_ d|_d|_t!|_d\|j"_|j"_y)Nlinearrrr)rsetattrrrlist type_updates_typerTrSrrmax_relative_size_coefficients_count _xopt_count_xoffsetnumber_of_data_last_addedrr)rrYrs rrz LQModel.reset s$$D D$ #% '- > !%!E!E"= ;; #%  )*&5#'  dhhjrctj|jjd|||jkDr|jS|S)z®ression weights in decreasing orderr)rlinspacerSrr)rYrks rsorted_weightszLQModel.sorted_weightssD{{4==33Q(.&499:L499( ( &( (rcBt|jdkrddgSg}tj|jd|jdz }tj|jd|jz }|t |dzt |dzgz }|S)zBsome data of the current state which may be interesting to displayr9rr)rr^rrrr)rYtraced1d2s rrzLQModel.logging_trace s tvv;?q6M ZZq "TVVAY . ZZq "TYY . #b!e*c"a%j)) rct|j|jd|j|jz|j j fSNr)r4r known_typesrrrSrrxs rmax_sizezLQModel.max_size-sQ9DOOD$4$4R$89$((C../MM3356 6rcZ|j|jd|jSr)rrrrxs rmax_dfzLQModel.max_df3s'4tt//34TXX>>rc,t|jS)z+number of data available to build the modelrr^rxs rrz LQModel.size7466{rc`t|jrt|jdSdS)Nrrrxs rrz LQModel.dim<s#!$TVVs466!9~6$6rct|js*ddkr$|jdk(stjdyt|jt|jd}}|j dddD]}||j jz|j|||j jzk\sJ||jvsY||j jvsr|jj|||_|j|j|jxx|gz cc<y)z?model type/size depends on the number of observed data rrrzempty model is not linearNrr)rr^ _current_typer!r"rrSrrrrrappendreset_ZrrT)rYrrtypes r update_typezLQModel.update_type@s466{Avd00H< 9: 466{Cq N1$$TrT*DDMM2226Kdood6KA6NQUQ^Q^QpQp6pp * > >> !!$'%)" !!$**-$7-+rc`t|jd}|jdddD]m}|j|jj z|j |||jjzk\sT||jjvsmnd}||_ |jS)zzone of the model `known_types`, depending on self.size. This may replace `types`, but is not in use yet. rNrr) rr^rrrSrrrrr)rYrrs rrz LQModel.typeSs q N$$TrT*D DMM:::>Sdood>STU>VY]YfYfYxYx>xx > >>+ D!!!!rc6t|j|jkDrt|j|jjzdz |j |jj zk\r|jD]}t||jt|j|jkDrUt|j|jjzdz |j |jj zk\ryyyy) deprecatedrN) rr^rrSrrrrgetattrpoprs r_prunezLQModel._prunebs466{T]]*466{T]];;;a?4;;QUQ^Q^QpQpCpp((d#''))466{T]]*466{T]];;;a?4;;QUQ^Q^QpQpCpp+p+rc t|jt|j|j|j j z|j jz fz }|dkry|jD]} t||t||d|  y#t$r/t||}t|D]}|jYYwxYw)z'prune data depending on size parametersrN)r3rr4rrrSrrrrr r,rr )rYremoverfield_s rprunez LQModel.pruneisTYYdmm&*kkDMM4S4S&SVZVcVcVtVt&t&v"wwx Q; $$D dGD$$7&$AB% d+vAIIK' s<B5CCc|j|}||jvrtjd|rdndz|syddkr|jj d||j j d|j||jj d||jj d|ntj|g|jdkDr |jgngz|_tj|j|g|jdkDr |j gngz|_tj|g|jdkDr |jgngz|_tj|g|jdkDr |jgngz|_ |xjdz c_ |jj d|j|jj d|d|_|j!|r|j#|j$j&r*|j$j'|j|_ |S) z1add `x` to `self` ``if `force` or x not in self``zx value already in Model, z!use `force` argument to force addz nothing addedNrrrr)_hashrr!r"r^insertrexpand_xrrrvstackrThstackrrrrrSr)rYr rurforcehashs rrzLQModel.add_data_rowwszz!} 4;;  MM67<3/S T 6 FFMM!Q  FFMM!T]]1- . FFMM!Q  FFMM!Q YYs$**q.tvvhbIJDFYY a 01adffXUWXYDFYYs$**q.tvvhbIJDFYYs$**q.tvvhbIJDF a  1djj) 1d#)*&   JJL == ) )]]33DFF;DF rct|t|k7r"tdt|t|fztj|ddd}|D]}|j |||||t||_|S)zFadd a sequence of x- and y-data, sorted by y-data (best last) z+input X and Y have different lengths %d!=%dNr)r)rrfrrrr)rYr^rrrlr%s rrzLQModel.add_datas q6SV JcRSfVYZ[V\M]]^ ^jjmDbD!A   adAaD  6),Q& rc:|jt|jk(sJ|t|j}|dkr|St|t|jf}|t |Dcgc]}|j|c}}|j D]A}t||}|Dcgc]}|| }}t t|D] }||||< Ct|jd|t|jd|k(sJycc}wcc}w)zXold? sort last `number` entries TODO: for some reason this seems not to pass the doctestNr) rrr^r5rrrr rsorted)rYrkrr%rlrrtmps r_sortz LQModel._sortsyyCK''' >[F Q;Kfc$&&k*+%-8-Qtvvay-89$$DD$'E%()S58SC)3s8_q6a% % DFF7FO$tvvgv(????9*s 2D. Dc||dur |j}t||jf}|dkr|S||jkr||jd|}n||j}|jD]}t ||} |||d|ddkrtt|jd|t|jd|k(sJt|j}|j||t|jk(sJyy#t $r3|Dcgc]}|| ncc}w}}t |D] }||||< YwxYw)zsort last `number` entriesNTrrr) rr5rrr r,rrrr) rYrkrrlrrr%r$rs rrz LQModel.sortsD >Vt^YYFfdii() Q;K DII $&&&/*C$&&/C$$DD$'E $!&sgv% 6w(F466'6?,CC CCTVV A JJv TVV $ $$   $'*+s!U1Xs++vA tE!H' $sD E D"! EEct|jt|jk(rb||jjkrHt |jj |jj|jzf|_yyyr ) rrrrSrr5rrr)rYr)s rrzLQModel.adapt_max_relative_sizeso tzz?c$"2"23 3dmm>l>l8l%($--*M*M $ F FI_I_ _*a&bD "9m 3rcDtj|jdS)Nr)axis)rrr^rxs rxmeanz LQModel.xmeanswwtvvA&&rcXtj||_|jyr )rrrr)rYoffsets r set_xoffsetzLQModel.set_xoffsets 6*  rctj|jDcgc]}|j|c}|_d|_d|_ycc}w)zset x-values Z attributerN)rrr^rrrrrYr s rrzLQModel.reset_ZsCtvv>v!T]]1-v>?#% ?sAc tj||jz}tjd|g}d|jvrtj|tj |g}d|jvratj|t t|Dcgc],}t t|D]}||ks ||||z.c}}g}|Scc}}w)Nrrr)rrrrrsquarerr)rYr zr%r(s rrzLQModel.expand_xs JJqMDMM ) IIq!f  $** $ 1biil+,A#IIqc!f "f 1uUXYZU[}!`ade`e1Q4!A$;}; "fgh#gs '!C# C# c|jr]t|t|jdk7r9tdt |t|jdt|fz|jdkry |j |j |}tj|j|S#t$r'|j|rJ|j|}YOwxYw)zreturn Model value of `x`rzx = %s must be of len %d != %d) rTrr^rfrrindexisinrrdot coefficients)rYr r*s rrhz LQModel.evals ::#a&Cq N2= #ADFF1IA?@A A ::? !tzz!}%Avvd''++ !yy| ## a A !s;B99-C)(C)cl|Dcgc]$}|jdk(rdn|j|&c}Scc}w)z3never used, return Model values of ``x for x in X``r)rTrh)rYr^r s revalpopzLQModel.evalpops@AZZ1_$))A,6 s)1ch|j||||j|kr|jdd}|jt|j||jt ||jz dzdt t j |jd|jdz dzzkrMt j |jd|jdz dz }|j||||j|kr|j|fS)zthis works very poorly e.g. on Rosenbrock :: x, m = Model().optimize(cma.ff.rosen, [0.1, -0.1], 13) TODO (implemented, next: test): account for xopt not changing. Nr9gMbP?rr)rrTrrrrrr^)rYrOx0rxopt_oldx_news roptimizezLQModel.optimizes "gbk*jj5 yy|H   d499owtyy/A BHtyy(1,-sBJJtvvay"DHHO,0 )DHHL$((/"466'6?BG-#P-QDIIdffQi$8-#PR xx||$Qs?%D c|j|}||jvr|jj|dzSdS)z9return False if `x` is not (anymore) in the model archiverF)rrr,)rYr rs rr-z LQModel.isins9zz!}.2dkk.At{{  &*LuLrcV|jj|j|Sr )rr,rr's rr,z LQModel.index$s{{  A//rc2t|tr|St|tjr t |j S t |S#t $rt t|cYSwxYw#t$rt t|cYSwxYwr ) isinstancer3rndarrayrtobytesAttributeErrorbytesr,tupler's rrz LQModel._hash's a H 2:: & &AIIK(( &Aw " &E!H~% &  &E!H~% &s#A A6A32A36BBcjtj|tj|j|S)z^caveat: this can be negative because hessian is not guarantied to be pos def. )rr.hessian)rYr&s rmahalanobis_norm_squaredz LQModel.mahalanobis_norm_squared8s$vvb"&&r233rcDtj|j}td|jj |f}|r%|d| }|j |j|z }n|j }|tj||jzS)jreturn weighted Z, worst entries are clipped if possible. Z can be a vector or a matrix. rN) rrrr4rS remove_worserrrT)rYrrlclipws rold_weighted_arrayzLQModel.old_weighted_array>s jj At}}11$789 fu+C##DII$45A##%A2::a=%''''rc|jj|}|jj|}||jkr|d|}|j |t j ||jzS)rHN)rSrrrrrrrJ)rYrrrls rweighted_arrayzLQModel.weighted_arrayLsl }}11$7mm((. $)) et*C""4(2::a=+=+?+???rc  tjj|j|jj |_|j S#tjj$r}tjd|jrt|jdndt|jt|jt|fzYd}~|j Sd}~wwxYw)zreturn Pseudoinverse, computed unconditionally (not lazy). `pinv` is usually not used directly but via the `coefficients` property. Should this depend on something and/or become lazy? zAModel.pinv(d=%d,m=%d,n=%d): np.linalg.pinv raised an exception %srrN)rlinalgpinvrOrrJ_pinv LinAlgErrorr!r"rrr^ _coefficientsrrYlaerrors rrRz LQModel.pinvWs '(;(;DFF(CDFFDJzzyy$$ ' MM4*.))DFF1ID../DFF G 7&& ' ' zz 'sAAC62A/C11C6cR|j|jkr|j|_tj|j|j |j |_|jj|jj|jS)z4model coefficients that are linear in self.expand(.)) rrTrr.rRrOrrUrWrrrxs rr/zLQModel.coefficientsksv  # #djj 0'+zzD $!# 43F3Ftvv3N!OD  KK     % % '!!!rc t|jd}t|j}||dzg|jDcgc]}|j||c}zvsJt j ||f}d|zdz}t|D]}||dzkDr+d|jvsJ|j||zdz|||f<nW||dzk(sJ|j|dzdz |||f<tt j|jdddz |||f<|d|zkDsd|jvsJ||jd|k(sJt|dz|D]%}|j|dz x|||f<|||f<|dz }'|Scc}w)Nrrr9rrrr) rr^r/rrrzerosrrr5abs)rYrrrHkr%r(s rrEzLQModel.hessianus q N !! "QUGDDTDTUDTD4tt4Q7DTUUUUU HHaV  EAIqA1q5y"djj000++AEAI6!Q$AEz!z++AE2S8!Q$bffT%6%6qr%:;A#566rc |j|jkr|j|_|js3|jdd|jddzz |_|j S t jt jj|j|jdz |jz |_|j S|j S#t jj$r}tjd|j xsdt#|j$|j&t)|fzt+|ds^|j rRt j,|j |_t j0j3|j |_Yd}~|j Sd}~wwxYw)Nrr9rgzAModel.xopt(d=%d,m=%d,n=%d): np.linalg.pinv raised an exception %sr_xopt)rrTrr^r/rarr.rQrRrEr_rrTr!r"rrrUrrhasattrrZ_optrandomrandnrVs rrz LQModel.xoptsQ   djj (#zzD ::!VVAYT->->qr-B)BB zz >!# t||(Ddffsl!SVZVcVc!cDJzztzzyy,, >MM#< $B #D$6$6 7 $ #G ?.#./ #41dhh$&HHTXX$6 $&IIOODHH$= zz >s+ACG rvs@@X~  #(5)?*X  / gW&gWR_+V+86&6 o6b7fb7 (((sBB$#B$