+ &jz^RIt^RIt^RIt^RIt^RIt^RIt^RIHuHuH t ^RI Ht^RI H t ^RIHt^RIHtHtHtHtHtHt^RIHtHtHtHtHtHtHtHt^RI H!t!H"t"^RI#H$t$^RI%H&t&H't'H(t(.R Ot)]t*R ]PV]PXPV]PZ]PXPZ/R ]PXPV]PPV]PXPZ]PPZ//t.R t/R!R lt0R"R lt1Rt2Rt3R#Rlt4R$Rlt5Rt6Rt7]Pp!]&4R%Rl4t9Rt:Rt;]Pp!]&4R&Rl4t<]Pp!]&4R]PzRR3Rl4t>]Pp!]&4R&Rl4t?]Pp!]&4R#Rl4t@]Pp!]&4R'Rl4tAR(RltBR#RltCR)RltDR#)*N) _FusedModule)_is_activation_post_process)_activation_is_memoryless_add_module_to_qconfig_obs_ctrdefault_dynamic_qconfigfloat16_dynamic_qconfig!float_qparams_weight_only_qconfig&float_qparams_weight_only_qconfig_4bit)_get_special_act_post_process_has_special_act_post_process)get_default_dynamic_quant_module_mappingsget_default_qat_module_mappings$get_default_qconfig_propagation_list(get_default_static_quant_module_mappings2get_default_static_quant_reference_module_mappingsno_observer_set) DeQuantStub QuantWrapper)type_before_parametrizations)DEPRECATION_WARNINGget_qparam_dict)has_no_children_ignoring_parametrizations%float_to_observed_custom_module_class)observed_to_quantized_custom_module_classc\#)z'Defines the default custom config dict.)_DEFAULT_CUSTOM_CONFIG_DICTv/Users/jameslopez/projects/CWCArchive/cwc-podcast/.venv/lib/python3.14/site-packages/torch/ao/quantization/quantize.pyget_default_custom_config_dictr Gs &&rcVP\V4V4pVPW54p\VRV4p\PP P PWP4\WP4pW`nVP4FiwrxV'dVR,V,MTp Ve;WtPR.49dK;\V4VPR.49dK]\WWi4Kk R#)aThis is a helper function for `propagate_qconfig_` Args: module: input module qconfig_dict: dictionary that maps from name of submodule to quantization configuration qconfig_parent: quantization config of parent module, we will fallback to this config when there is no specified config for current module prefix: corresponding prefix of the current module, used as key in qconfig_dict prepare_custom_config_dict: dictionary for custom handling of modules see docs for :func:`~torch.ao.quantization.prepare_fx` Return: None, module is modified inplace with qconfig attached qconfig.Nnon_traceable_module_namenon_traceable_module_class) getrgetattrtorchao quantizationr"_assert_valid_qconfigrnamed_childrentype_propagate_qconfig_helper) module qconfig_dictqconfig_parentprefixprepare_custom_config_dictmodule_qconfigqconfig_with_device_checknamechild module_prefixs &&&&& rr.r.Ls2"%%$V,nN"%%f=NVY?N HH!!77O >~ V.N,,. /5 t+4 % - 223NPRS SE{)--.JBOP &%> /rc8Vf/pVf/p\WVR7R#)acPropagate qconfig through the module hierarchy and assign `qconfig` attribute on each leaf module Args: module: input module qconfig_dict: dictionary that maps from name or type of submodule to quantization configuration, qconfig applies to all submodules of a given module unless qconfig for the submodules are specified (when the submodule already has qconfig attribute) prepare_custom_config_dict: dictionary for custom handling of modules see docs for :func:`~torch.ao.quantization.prepare_fx` Return: None, module is modified inplace with qconfig attached N)r3)r.)r/r0r3s&&&rpropagate_qconfig_r:}s)  !)%'"9Src$VPV4#)z.Forward hook that calls observer on the outputactivation_post_process)selfinputoutputs&&&r_observer_forward_hookrAs  ' ' //rc2VPV^,4#)z2Forward pre hook that calls observer on the outputr<)r>r?s&&r_observer_forward_pre_hookrCs  ' 'a 11rFc\VR4'g \R4hV'dVP\RR7R#VP \ RR7R#)r=zGExpect activation_post_process attribute already attached to the moduleT)prependN)hasattrAssertionErrorregister_forward_pre_hookrCregister_forward_hookrA)r/pre_hooks&&r&_register_activation_post_process_hookrKsM 64 5 5 U  (()CT(R$$%;T$Jrcaa a Vf \4pVf/pSfP\V4p\V4^8d\RV 24h\V4^8d\ \ V44MRoR Rlo Ro R VV V 3RllpVP 4EFwrx\V4\PJdK%\\V4\P\P34'dUS !V4'dE\VR4'g\R\V4 R24hS !VPS4VnKK\#V\$4'dS !V4'd V!V4KKVe-\V4V9dS !V4'd V!V4EKEK\'V4'd\)V4p V!W4EK9S !V4'dq\V4V9daV\V4,p V P+V4p \-WV 4\V \/\1444'g V!V 4EKEK\3VVVSV4EK \5V4'dC\#V\6PP84'g\V4V9d V!V4\VR 4'dI\#V\6PP84'g\V4V9d V!V4R#R#R#R#) aGAdd observer for the leaf child of the module. This function insert observer module to all leaf child module that has a valid qconfig attribute. Args: module: input module with qconfig attributes for all the leaf modules that we want to quantize qconfig_propagation_list: a list of quantizable modules that will have observers added to them if they are leaf nodes device: parent device, if any non_leaf_module_list: list of non-leaf modules we want to add observer Return: None, module is modified inplace with added observer modules and forward_hooks NzR_add_observer_ only works with cpu or single-device CUDA modules, but got devices cfVfVP4MV!4pVeVPV4V#N) activationto)r"devicespecial_act_post_processrOs&&& rget_activation_post_process3_add_observer_..get_activation_post_processs=(/    )+    MM& !rcF\VR4;'dVPRJ#r"NrFr")ms&rneeds_observation)_add_observer_..needs_observations q)$>>$)>>rc<S!V4'd_\V\4'gGVPRS!VPSV44\ V\ VP4R7R#R#R#)z]Adds an activation post process module and register a pre or post hook that calls the module r=rJN) isinstancer add_moduler"rKr)rXrRrQrSrYs&&rinsert_activation_post_process6_add_observer_..insert_activation_post_processsa Q   1k(B(B LL)+IIv'?  35aii@ )C rr=zfunctional class z- has no pre-defined `activation_post_process`weight_fake_quantrN)r_get_unique_devices_lenrGnextiterr,rnnDropout issubclassnnqFloatFunctional QFunctionalrFr"r=r]rr r from_floatsetattrtupler_add_observer_rr( Sequential)r/qconfig_propagation_listnon_leaf_module_listrQcustom_module_class_mappingdevicesr_r6r7rRobserved_classobserved_childrSrYs&&&f& @@rroros, '#G#I "*&(#~&v. wH ($+  U/j 2&996588#6#677 ( 04L L&v. +,,6588#6#677 ( 04L L&v. M8 -rc8VP4Uu0uF+qPPR8wgKVPkK- upVP4Uu0uF+qPPR8wgKVPkK- up,#uupiuupi)meta) parametersrQr-buffers)r/ps& rrbrb6sy$//1 M1XX]]f5LHAHH1 M ..*Q*Qhhmmv.E*Q  MQsBBB9Bc\V4'd0\VR4'dVP'd \V4#VP 4Fwr\ V4VP V&K V#)aOWrap the leaf child module in QuantWrapper if it has a valid qconfig Note that this function will modify the children of module inplace and it can return a new module which wraps the input module as well. Args: module: input module with qconfig attributes for all the leaf modules that we want to quantize Return: Either the inplace modified module with submodules wrapped in `QuantWrapper` based on qconfig or a new `QuantWrapper` module which wraps the input module, the latter case only happens when the input module is a leaf module and we want to quantize it. r")rrFr"rr,add_quant_dequant_modules)r/r6r7s& rr}r}<s] 2&99 FI & & NNNF##,,.  1% 8/ Mrc\PPR4Vf \4pVP R/4pV'g\ P !V4pTpVf \4p\VRR7\;QJd*RVP44F 'gK RM RM!RVP444'g\P!R^R 7\VVVVR 7V#) aPrepares a copy of the model for quantization calibration or quantization-aware training. Quantization configuration should be assigned preemptively to individual submodules in `.qconfig` attribute. The model will be attached with observer or fake quant modules, and qconfig will be propagated. Args: `model`: input model to be modified in-place `inplace`: carry out model transformations in-place, the original module is mutated `allow_list`: list of quantizable modules `observer_non_leaf_module_list`: list of non-leaf modules we want to add observer `prepare_custom_config_dict`: customization configuration dictionary for prepare function .. code-block:: python # Example of prepare_custom_config_dict: prepare_custom_config_dict = { # user will manually define the corresponding observed # module class which has a from_float class method that converts # float custom module to observed custom module "float_to_observed_custom_module_class": {CustomModule: ObservedCustomModule} } z!quantization_api.quantize.prepareNrr0c3b"TF%p\VR4;'d VPxK' R#5irVrW).0rXs& r prepare..s&LOqwq)$222Os//TFzNone of the submodule got qconfig applied. Make sure you passed correct configuration through `qconfig_dict` or by assigning the `.qconfig` attribute directly on submodules) stacklevel)rs)r(_C_log_api_usage_oncer r&copydeepcopyrr:anymoduleswarningswarnro)modelinplace allow_listobserver_non_leaf_module_listr3rsrqs&&&&& rpreparerWsD HH  !DE!)%C%E""<"@"@/#  e$ *#G#I u40 3LEMMOL333LEMMOL L L  K    %$?  Lrca\SR4'd(\SP4'd \SR4RV3RllpV!RR7V!RR7R#)r=Fc<V'd SPM SPpV'd\M\p\ 4pVP 4FwrEWRJgK VP V4K VFpVPV4K R#rN)_forward_pre_hooks_forward_hooksrCrAsetitemsaddpop)rJhook_map observer_hookhandle_ids_to_remove handle_idhook_fnr/s& r remove_hooks5_remove_activation_post_process..remove_hookssp086,,f>S>S*2 &8N  #u"*.."2 I'$((3#3.I LL #.rTr\NF)rFrr=delattr)r/rsf r_remove_activation_post_processrsMv0116Q&&77 12 $$% rcVP4Fp\V4K \VR4'dV=\ V4R#)zzClean up the qconfig left in the module so that new qconfig can be propagated. Args: module: module to be cleaned up r"N)children_remove_qconfigrFr"r)r/r7s& rrrs9"#vy!! N#F+rc\PPR4Vf \4pV'g\P !V4pVP 4\VRR7V!V.VO5!\WRR7V#)aSQuantize the input float model with post training static quantization. First it will prepare the model for calibration, then it calls `run_fn` which will run the calibration step, after that we will convert the model to a quantized model. Args: model: input float model run_fn: a calibration function for calibrating the prepared model run_args: positional arguments for `run_fn` inplace: carry out model transformations in-place, the original module is mutated mapping: correspondence between original module types and quantized counterparts Return: Quantized model. z"quantization_api.quantize.quantizeTr) r(rrrrrevalrconvert)rrun_fnrun_argsmappingrs&&&&&rquantizersd$ HH  !EF:<  e$ JJL E4  58 ED) Lrc \PPR4VEfV\P8Xd}\P \ \P\ \P\ \P\ \P\ \P\ /pEMV\P8Xd}\P \\P\\P\\P\\P\\P\/pEM;V\P8Xd,\P\ \P"\ /pMV\P$8Xd\P\&/pM\)RV R24h\+V\,4'dV\PJd\ pMfV\PJd\pMKV\PJd\ pM0V\P$Jd\&pM\/R\1V44h\3\5V\6P8!V444pVf \;4pV'g\<P>!V4pVPA4\CW4\EWRR7V#)a*Converts a float model to dynamic (i.e. weights-only) quantized model. Replaces specified modules with dynamic weight-only quantized versions and output the quantized model. For simplest usage provide `dtype` argument that can be float16 or qint8. Weight-only quantization by default is performed for layers with large weights size - i.e. Linear and RNN variants. Fine grained control is possible with `qconfig` and `mapping` that act similarly to `quantize()`. If `qconfig` is provided, the `dtype` argument is ignored. Args: model: input model qconfig_spec: Either: - A dictionary that maps from name or type of submodule to quantization configuration, qconfig applies to all submodules of a given module unless qconfig for the submodules are specified (when the submodule already has qconfig attribute). Entries in the dictionary need to be QConfig instances. - A set of types and/or submodule names to apply dynamic quantization to, in which case the `dtype` argument is used to specify the bit-width inplace: carry out model transformations in-place, the original module is mutated mapping: maps type of a submodule to a type of corresponding dynamically quantized version with which the submodule needs to be replaced z*quantization_api.quantize.quantize_dynamicz5Don't know how to quantize with default settings for z. Provide full qconfig pleasez.Unknown dtype specified for quantize_dynamic: Tr)#r(rrqint8rfLinearrLSTMGRULSTMCellRNNCellGRUCellfloat16rquint8 EmbeddingBagr Embeddingquint4x2r ValueErrorr]r RuntimeErrorstrdictzip itertoolsrepeatr rrrr:r)r qconfig_specdtyperrdefault_qconfigs&&&&& rquantize_dynamicrs@ HH  !MN EKK  20/ 4 3 3 Lemm # 20/ 4 3 3 Lell "!B ?Lenn $!GLGwNkl  L# & & EKK 5O emm #5O ell "?O enn $DO@#e* C i.>.>.OPQ ;=  e$ JJLu+ ED) LrcV\PPR4VP'g \ R4hVf \ 4pV'g\ P!V4p\VRR7\WRRR7\V\VP44RR7V#) a Prepares a copy of the model for quantization calibration or quantization-aware training and converts it to quantized version. Quantization configuration should be assigned preemptively to individual submodules in `.qconfig` attribute. Args: model: input model to be modified in-place mapping: dictionary that maps float modules to quantized modules to be replaced. inplace: carry out model transformations in-place, the original module is mutated z%quantization_api.quantize.prepare_qatz1prepare_qat only works on models in training modeNrTF)rrremove_qconfig)rr) r(rrtrainingrGrrrr:rrrvalues)rrrs&&&r prepare_qatr>s~  HH  !HI >>>PQQ13  e$u40 EDG EW^^5E1FPTU Lrc\PPR4V'g\P!V4pVP 4\ VRR7V!V.VO5!\VRR7V#)aCDo quantization aware training and output a quantized model Args: model: input model run_fn: a function for evaluating the prepared model, can be a function that simply runs the prepared model or a training loop run_args: positional arguments for `run_fn` Return: Quantized model. z&quantization_api.quantize.quantize_qatTr)r(rrrrtrainrr)rrrrs&&&&r quantize_qatr]sW HH  !IJ  e$ KKMt$ 58 E4 Lrc \PPR4V'g\P!V4p\ VVRVVVR7V'd \ V4V#)adConverts submodules in input module to a different module according to `mapping` by calling `from_float` method on the target module class. And remove qconfig at the end if remove_qconfig is set to True. Args: `module`: prepared and calibrated module `mapping`: a dictionary that maps from source module type to target module type, can be overwritten to allow swapping user defined Modules `inplace`: carry out model transformations in-place, the original module is mutated `convert_custom_config_dict`: custom configuration dictionary for convert function `use_precomputed_fake_quant`: a flag to enable use of precomputed fake quant .. code-block:: python # Example of convert_custom_config_dict: convert_custom_config_dict = { # user will manually define the corresponding quantized # module class which has a from_observed class method that converts # observed custom module to quantized custom module "observed_to_quantized_custom_module_class": { ObservedCustomModule: QuantizedCustomModule } } z!quantization_api.quantize.convertT)r is_referenceconvert_custom_config_dictuse_precomputed_fake_quant)r(rrrr_convertr)r/rrrrrrs&&&&&&&rrrusUJ HH  !DE v& !#=#=  Mrc VfV'd \4M \4pVf \4pVPR/4pV'g\P !V4p/pVP 4FJwr\V \4'g"\V 4V9d\V VRVVVR7\WWe4Wx&KL VP4FwrWPV &K V#)aCConverts submodules in input module to a different module according to `mapping` by calling `from_float` method on the target module class Args: module: input module mapping: a dictionary that maps from source module type to target module type, can be overwritten to allow swapping user defined Modules inplace: carry out model transformations in-place, the original module is mutated is_reference: a flag to enable quantized reference module use_precomputed_fake_quant: a flag to enable use of precomputed fake quant rTr)rrr r&rrr,r]rrr swap_modulerr~) r/rrrrrrsreassignr6modkeyvalues &&&&&& rrrs, ? @9;  ")%C%E""<"@"@3R# v&H**, 3 --,S19TT *+E  % 5 -&nn& $' MrcTp\VR4'EdQVPEeBRp\V4V9d%V\V4,PV4pRpM\V4V9dV\V4,p\VR4'duVP'dcVPf \ R4hVPP 4pV!VP 4\V4pVPW4pMU\P!VP4p RV P9dVPWR7pMVPV4pRpV'EdVPP4Fp VPV 4K VPP4F p V \ JgKVP#V 4K" \%V4p \'V 4^8:g:\'V 4^8Xd\(P*!R 4V 9g\ R V 24h\'V 4^8d\-\/V 44MRp V 'dVP1V 4V#) zSwaps the module if it has a quantized counterpart and it has an `observer` attached. Args: mod: input module mapping: a dictionary that maps from nn module to nnq module Return: The corresponding quantized module of `mod` r"NFT _IS_REFERENCEzAmodule qconfig must not be None when swapping to reference modulerrrxzOswap_module only works with cpu or single-device CUDA modules, but got devices )rFr"r from_observedrrGweightrrlinspect signatureryrrrHrrArIrbrcr(rQrdrerP)rrrsrnew_modswappedqmodweight_post_processweight_qparamssig pre_hook_fnrrtrQs&&&& rrrsGsI3;;#: ' ,0K K1,S1mC  G )# .' 97<=Dt_--$2D2D2D;;&(['*kk&8&8&:##CJJ/!01D!E//#>''8/3>>A"oo.G#ooc2GG 7"55<<> 11+> ?--446"8811':7 +3/GG !LA%%,,v*>'*I$efmeno-0L1,.get_prefix4s2v76C<7rr=N)rFr=r,_get_observer_dict)r target_dictr2rr6r7r8s&&& rrr+si8s-..  ' ' Jv&)BBC))+ 5; 6*T1 5}=,r) r r:r}rrrrrrr)NrN)NNr)NNNN)FNNN)NF)NFTFNF)NFFNF)r)Errrtyping_extensionsrr(torch.ao.nn.quantizedr)rf quantizedritorch.nntorch.ao.nn.intrinsicrtorch.ao.quantization.observerrtorch.ao.quantization.qconfigrrrrr r +torch.ao.quantization.quantization_mappingsr r r rrrrrtorch.ao.quantization.stubsrrtorch.nn.utils.parametrizerutilsrrr__all__is_activation_post_processr quantizableMultiheadAttentionrr r.r:rArCrKrorbr} deprecatedrrrrrrrrrrrrrrrrs  ##.F   BC 9, $$ r~~@@.0 R\\.. ))2<<+J+J2 ' .b20 2 KH/V 612@3@F!4 , 123:12EKKuW3Wt123<123.12131h;|@F>r