+ &jY ^RIHt^RIt^RIHt^RIHtHtHtHtH t H t H t H t H t HtHtHtHtHtRR.t!RR]4tRR ] R ] R ] R ] R 2 ,]nRR RlltRtRRltRRltRRltR#))castN)Tensor)_default_to_fused_or_foreach_device_dtype_check_for_fused_differentiable_doc _foreach_doc_get_scalar_dtype _get_value _maximize_doc _params_doc _to_scalar_use_grad_for_differentiable _view_as_real DeviceDict OptimizerParamsTAdagradadagradcaa]tRt^toRRRRRRR/V3RlV3RlllltV3RltV3R lR ltR t]RR l4t R t Vt V;t #)rNmaximizeFdifferentiablefusedc<V^8dQhRS[RS[S[,RS[RS[RS[RS[RS[R,R S[R S[R S[R,R R/ #) paramslrlr_decay weight_decayinitial_accumulator_valueepsforeachNrrrreturn)rfloatrbool)format __classdict__s"k/Users/jameslopez/projects/CWCArchive/cwc-podcast/.venv/lib/python3.14/site-packages/torch/optim/adagrad.py __annotate__Adagrad.__annotate__sEEE FNE E  E $) EEEEEd{E Ec<\V\4'd!VP4^8wd \R4hRV8:g\RV 24hRV8:g\RV 24hRV8:g\RV 24hRV8:g\RV 24hRV8:g\RV 24hRVR VR VR VR VR VRVRV RV / p \SV`W4V 'd5V 'd \ R4hV'd \ R4hRVnRVnVPFp V R,Fp VPV ,pV R,'d5\P!R\V R,R7V PR7M\P!R\4R7VR&\P !V 4'd \#WU4MTp\P$!W\P&R7VR&K K R#)zTensor lr must be 1-elementzInvalid learning rate: zInvalid lr_decay value: zInvalid weight_decay value: z)Invalid initial_accumulator_value value: zInvalid epsilon value: rrr rrr!rrrz)`fused` does not support `differentiable`z0`fused` and `foreach` cannot be `True` together.Tris_fuseddtypedevicer1step memory_formatsumN) isinstancernumel ValueErrorsuper__init__ RuntimeError"_need_device_dtype_check_for_fused_step_supports_amp_scaling param_groupsstatetorchzerosr r2tensor is_complexcomplex full_likepreserve_format)selfrrrrrr r!rrrdefaultsgroupprB init_value __class__s&&&&&&&&$$$ r'r=Adagrad.__init__s b& ! !bhhjAo:; ;by6rd;< <h7zBC Cl";L>JK K//;>  $"3U"C#$88 #\\(:K:MNFO % '(DJJ--/0 l+q0  eoo OF #7 !!LL!F)$,=u,M& "r*c<V^8dQhRR/#)rr"Nr8)r%r&s"r'r(r)s--d-r*c VPF=pVR,F-pVPV,pVR,P4K/ K? R#)z6Calls tensor.share_memory_() on the state sum tensors.rr7N)rArB share_memory_)rJrLrMrBs& r' share_memoryAdagrad.share_memorys?&&E8__ 1 e **,%'r*cRRrvVR,EFpVPfKVR,'d&\VRR4'd\V4RVnWhPP,pV\ P !V4,pVPV4VPVP4VPV,p \V 4^8XdVR,'d \V4VR,'d5\ P!R \VR,R7VPR7M\ P!R\4R7V R &VPR ,p \ P !V4'd \W4MT p \ P !W\ P"R 7V R &VPV R ,4VPV R ,4EK Wg3#)Frrr?Tr.r0r-r3r4rr5r7r8)gradgetattrrr? is_sparserCrFappendrBrUrDr r2rErKrGrHrI) rJrLparams_with_gradgrads state_sums state_stepshas_sparse_grad has_complexrMrBrrNs &&&&&& r' _init_groupAdagrad._init_groups',exAvv!>>g8'' 2!4>CD;66#3#33u//22  ''* QVV$ 1 u:?W~~5a8!>>  "3U7^"L#$88 #\\#5F5HI&M15 31- !++A.. 9U6 $)??U5J5J$E%L!!%,/""5=1Q!T++r*c RpVe.\P!4;_uu_4V!4pRRR4VPFp.p.p.p.pVPW4WVV4wr\ VVVVVR,VR,VR,VR,VVR,VR,VR,V VR ,\ VR R4\ VR R4R 7K V# +'giL;i) zPerform a single optimization step. Args: closure (Callable, optional): A closure that reevaluates the model and returns the loss. Nrrrr r!rrr grad_scale found_inf) rrrr rmr!rrrnrrrrs)rC enable_gradrArorrf) rJclosurelossrLrirjrkrlrmrns && r'r4 Adagrad.steps  ""$$y%&&E-/ "$E')J(*K+/+;+;K, (O  ;">2z*%L /i(z*$%56'Gn"4t<!$ T:! ': A%$s C C )r?r@)g{Gz?rrrg|=NN) __name__ __module__ __qualname____firstlineno__r=rRrbrorr4__static_attributes____classdictcell__ __classcell__)rOr&s@@r'rrsbEE %E"EEN!F--,,\"*"**r*a[Implements Adagrad algorithm. .. math:: \begin{aligned} &\rule{110mm}{0.4pt} \\ &\textbf{input} : \gamma \text{ (lr)}, \: \theta_0 \text{ (params)}, \: f(\theta) \text{ (objective)}, \: \lambda \text{ (weight decay)}, \\ &\hspace{12mm} \tau \text{ (initial accumulator value)}, \: \eta\text{ (lr decay)}\\ &\textbf{initialize} : state\_sum_0 \leftarrow \tau \\[-1.ex] &\rule{110mm}{0.4pt} \\ &\textbf{for} \: t=1 \: \textbf{to} \: \ldots \: \textbf{do} \\ &\hspace{5mm}g_t \leftarrow \nabla_{\theta} f_t (\theta_{t-1}) \\ &\hspace{5mm} \tilde{\gamma} \leftarrow \gamma / (1 +(t-1) \eta) \\ &\hspace{5mm} \textbf{if} \: \lambda \neq 0 \\ &\hspace{10mm} g_t \leftarrow g_t + \lambda \theta_{t-1} \\ &\hspace{5mm}state\_sum_t \leftarrow state\_sum_{t-1} + g^2_t \\ &\hspace{5mm}\theta_t \leftarrow \theta_{t-1}- \tilde{\gamma} \frac{g_t}{\sqrt{state\_sum_t}+\epsilon} \\ &\rule{110mm}{0.4pt} \\[-1.ex] &\bf{return} \: \theta_t \\[-1.ex] &\rule{110mm}{0.4pt} \\[-1.ex] \end{aligned} For further details regarding the algorithm we refer to `Adaptive Subgradient Methods for Online Learning and Stochastic Optimization`_. z Args: a lr (float, Tensor, optional): learning rate (default: 1e-2) lr_decay (float, optional): learning rate decay (default: 0) weight_decay (float, optional): weight decay (L2 penalty) (default: 0) initial_accumulator_value (float, optional): initial value of the sum of squares of gradients (default: 0) eps (float, optional): term added to the denominator to improve numerical stability (default: 1e-10) z a fused (bool, optional): whether the fused implementation (CPU and CUDA only) is used. Currently, `torch.float64`, `torch.float32`, `torch.float16`, and `torch.bfloat16` are supported. (default: None). Please note that the fused implementations does not support sparse or complex gradients. .. _Adaptive Subgradient Methods for Online Learning and Stochastic Optimization: http://jmlr.org/papers/v12/duchi11a.html c"lV^8dQhR\\,R\\,R\\,R\\,R\R,R\R,R\R,R \R \R,R \R \R \R\R\R\R\RR/#)rrrjrkrlrNrrrsrmr!rrnrrrr rr")rWrr$r#)r%s"r'r(r(sGG LG <GV Gf G $; G  G}GGD[GGG G !G"#G$ %G&'G( )Gr*c \;QJdRV4F 'dK RM RM !RV44'g \R4hVfVf\W RR7wppVfRpVfRpV'd0\PP 4'd \R4hV'd0\PP 4'd \R4hV'd,\PP 4'g\ pM9V'd,\PP 4'g\pM\pV!VVVVV V V VVVV V VVR 7R#) zlFunctional API that performs Adagrad algorithm computation. See :class:`~torch.optim.Adagrad` for details. c3V"TFp\V\P4xK! R#5irx)r9rCr).0ts& r' adagrad..6s@Kqz!U\\**Ks')FTzPAPI has changed, `state_steps` argument must contain a list of singleton tensorsN) use_fusedz6torch.jit.script not supported with foreach optimizersz4torch.jit.script not supported with fused optimizers rrrr rmrrrnrrrs) allr>rrCjit is_scripting_fused_adagrad_multi_tensor_adagrad_single_tensor_adagrad)rrjrkrlrrrrsrmr!rrnrrrr r_funcs&&&&&&&&&&&$$$$$ r'rrs2 3@K@333@K@ @ @ ^   }1 e 7 }599))++STT ''))QRR UYY++-- //11$%  ! '%r*cPVP4p\P!WV4#rx)sizerCsparse_coo_tensor)re grad_indicesrXrs&&& r' _make_sparsergs 99;D  " "< >>r*c8V^8dQhR\\,R\\,R\\,R\\,R\R,R\R,R\R \R \R \R \R \R\R\RR/#rrrjrkrlrrNrsrrrr rmrrrnr"rWrr#r$)r%s"r'r(r(lsA=A= LA= <A=V A=f A=  A= } A= A=A=A= A=A=A=A=A=  !A=r*c NVfVe \R4h\PP4'g \ V4p\ WW#RR7EFOwrppV^, p\ V4pV 'gTMV)pV^8wd0VP'd \R4hVPWR7pV^V^, V,,, pVP'dVP4pVP4pVP4pVP\VVVP^444VP!V4pVP4P#4PV 4pVP\VVVV, 4V)R7EKQ\P$!V4pV'dC\P&!V4p\P&!V4p\P&!V4pVP)W^R7V 'dVP+4V ,pMVP+4PV 4pVP-VVV)R7V'gEK#\P.!V4p\P.!V4pEKR R#)N,Expected grad_scale and found_inf to be NoneT)strictz;weight_decay option is not compatible with sparse gradientsalphavalue)AssertionErrorrCrrr zipr rgr>addcoalesce_indices_valuesadd_rpow sparse_masksqrt_rF view_as_realaddcmul_sqrtaddcdiv_view_as_complex)rrjrkrlrrrsrrrr rmrrrnparamre state_sumstep_tr4clrr grad_valuesstd std_valuesrFs&&&&&&$$$$$$$$ r'rrls "!6KLL 99 ! ! # # ^*-zt++&Y ! &!#t$ 1 ~~~"Q88E86DAX--. >>>==?D==?L,,.K NN<lKOOA.s9 '3tNN|sTFrg?cpu)r2rr)rrUr r"_group_tensors_by_device_and_dtyperXrrWranyrrrC _foreach_negcompiler is_compilingis_cpu _foreach_add_rE _foreach_addr _foreach_addcmul_ _foreach_sqrt _foreach_mul_ _foreach_mul_foreach_addcdiv_)rrjrkrlrrrsrrrr rmrrrngrouped_tensorlistsdevice_params_ device_grads_device_state_sums_device_state_steps_r device_params device_gradsdevice_state_sumsdevice_state_stepsdevice_has_sparse_gradr4 minus_clrr numerators&&&&&&$$$$$$$$ r'rrs"BCC!6KLL 6{a BB#FF  0 & & (   T&\>: DL-8  f/AB!$v,0CD!0" " SS9 '39 SSS9 '39 6  " "!")! $!-'%#    -7H I  --l;L ~~**,,1CA1F1M1M1M   "ELLU$C3     2A 6 1 ##L-|T$11 -|  GY FXdRC1 4(1,88 9 9FX    1<UVW!!"34 C% 1     i 8$I**<CI  y#>Q )p s1LcNV^8dQhR\\,R\\,R\\,R\\,R\R,R\R,R\\,R \R \R \R \R \R\R\RR/#rr)r%s"r'r(r(sSS LS <SV Sf S  S } S SSS SSSSS  !Sr*cbV'gR#V 'g V 'd \R4hV 'd \R4hVeVPV/M/pVeVPV/M/p\V\4'd)\ VP4R8wdVPV/MRp\ P !WW#.4pVP4EFawwppwwppppp\\\,V4p\\\,V4p\\\,V4p\\\,V4pRRppVe$VPVVPVRR74pVe$VPVVPVRR74pVe'VV9d VPVRR7VV&VV,p\P!V^4\P!VVVVVVVV V VVR7 VfEK9\P!VV.\!V4,4EKd R#)Nz5`fused` does not support sparse grad or complex paramzr2r9rstrrritemsrrWrStorCr_fused_adagrad_ _foreach_sub_rU)rrjrkrlrrrsrrrr rmrrrngrad_scale_dictfound_inf_dictlr_dictgrouped_tensorsr2rrrrrrrrrdevice_grad_scaledevice_found_infs&&&&&&$$$$$$$$ r'rrs#" +RSS J  ,6+A  J'r*3)>  9%B&b&11c"))n6MBSW  BB  0O           T&\>: DL-8  f/AB!$v,0CD.2D+  ! / : : f4 @!   -88 V$ ?    6#8 ee6eEGFOB .2     %(&   '   "%5$6=O9P$P C !r*)NNNFNFF)typingrrCr optimizerrrrrr r r r r rrrrr__all__r__doc__rrrrrr8r*r'rs $ i KiK^4      5.fGT? A=Hl?^Sr*