+ &jt~a0t$^RIHt^RIt^RIt^RIHt^RIHtHtH t H t ^RI H t H t Ht^RIt^RIHt]'d^RIHt^RIHt.R"Ot]!R 4t] !R 4t]!]P2R 4'g^]!R4]P2P4R&]!R4]P2P4R&]!R4]P2P4R&^RIHtHtHtRRltRRlt !RR ]4t!!RR 4t"] R]R]#3,3,t$R]%R&]R#RRll4t&]R#RRll4t&R#R R!llt&R#)$) annotationsN)Callable)overload TYPE_CHECKING TypeAliasUnion) ParamSpecSelfTypeVar)Tensor) _POOL_HANDLE) _dummy_type CUDAGraphgraph_R_P_CudaStreamBase _CUDAGraph_graph_pool_handle_cuda_isCurrentStreamCapturing)rrrcV^8dQhRR/#)returnbool)formats"i/Users/jameslopez/projects/CWCArchive/cwc-podcast/.venv/lib/python3.14/site-packages/torch/cuda/graphs.py __annotate__r-s,,T,c\4#)zReturn True if CUDA graph capture is underway on the current CUDA stream, False otherwise. If a CUDA context does not exist on the current device, returns False without initializing the context. )rrrris_current_stream_capturingr!-s * ++rcV^8dQhRR/#rrr r)rs"rrr6s99<9rcP\PP\44#)zReturn an opaque token representing the id of a graph memory pool. See :ref:`Graph memory management`. .. warning:: This API is in beta and may change in future releases. )torchcudar rrrrgraph_pool_handler'6s :: " "#5#7 88rca]tRt^BtRtRRV3RllltRRV3RllltRV3RlltRV3R lltR V3R llt R V3R llt RV3Rllt RV3Rllt RV3Rllt RV3RlltRV3RlltRtV;t#)raWrapper around a CUDA graph. Arguments: keep_graph (bool, optional): If ``keep_graph=False``, the cudaGraphExec_t will be instantiated on GPU at the end of ``capture_end`` and the underlying cudaGraph_t will be destroyed. Users who want to query or otherwise modify the underlying cudaGraph_t before instantiation can set ``keep_graph=True`` and access it via ``raw_cuda_graph`` after ``capture_end``. Note that the cudaGraphExec_t will not be instantiated at the end of ``capture_end`` in this case. Instead, it will be instantiated via an explicit called to ``instantiate`` or automatically on the first call to ``replay`` if ``instantiate`` was not already called. Calling ``instantiate`` manually before ``replay`` is recommended to prevent increased latency on the first call to ``replay``. It is allowed to modify the raw cudaGraph_t after first calling ``instantiate``, but the user must call ``instantiate`` again manually to make sure the instantiated graph has these changes. Pytorch has no means of tracking these changes. .. warning:: This API is in beta and may change in future releases. c V^8dQhRRRR/#)r keep_graphrrr r)rs"rrCUDAGraph.__annotate__]s000$0rc "<\SV`W4#N)super__new__)clsr* __class__s&&rr/CUDAGraph.__new__]sws//rc$V^8dQhRRRRRR/#)rpool_POOL_HANDLE | Nonecapture_error_modestrrNoner)rs"rrr+`s)PP'PDGP Prc (<\SV`WR7R#)aBegin capturing CUDA work on the current stream. Typically, you shouldn't call ``capture_begin`` yourself. Use :class:`~torch.cuda.graph` or :func:`~torch.cuda.make_graphed_callables`, which call ``capture_begin`` internally. Arguments: pool (optional): Token (returned by :func:`~torch.cuda.graph_pool_handle` or :meth:`other_Graph_instance.pool()`) that hints this graph may share memory with the indicated pool. See :ref:`Graph memory management`. capture_error_mode (str, optional): specifies the cudaStreamCaptureMode for the graph capture stream. Can be "global", "thread_local" or "relaxed". During cuda graph capture, some actions, such as cudaMalloc, may be unsafe. "global" will error on actions in other threads, "thread_local" will only error for actions in the current thread, and "relaxed" will not error on these actions. Do NOT change this setting unless you're familiar with `cudaStreamCaptureMode `_ )r4r6N)r. capture_begin)selfr4r6r1s&&&rr:CUDAGraph.capture_begin`s& 4OrcV^8dQhRR/#rrr8r)rs"rrr+us  T rc $<\SV`4R#)aEnd CUDA graph capture on the current stream. After ``capture_end``, ``replay`` may be called on this instance. Typically, you shouldn't call ``capture_end`` yourself. Use :class:`~torch.cuda.graph` or :func:`~torch.cuda.make_graphed_callables`, which call ``capture_end`` internally. N)r. capture_endr;r1s&rr@CUDAGraph.capture_endus rcV^8dQhRR/#r>r)rs"rrr+sTrc $<\SV`4R#)zInstantiate the CUDA graph. Will be called by ``capture_end`` if ``keep_graph=False``, or by ``replay`` if ``keep_graph=True`` and ``instantiate`` has not already been explicitly called. Does not destroy the cudaGraph_t returned by ``raw_cuda_graph``. N)r. instantiaterAs&rrECUDAGraph.instantiates rcV^8dQhRR/#r>r)rs"rrr+src $<\SV`4R#)z,Replay the CUDA work captured by this graph.N)r.replayrAs&rrICUDAGraph.replays  rcV^8dQhRR/#r>r)rs"rrr+strc $<\SV`4R#)z1Delete the graph currently held by this instance.N)r.resetrAs&rrMCUDAGraph.resets   rcV^8dQhRR/#r#r)rs"rrr+slrc <\SV`4#)zReturn an opaque token representing the id of this graph's memory pool. This id can optionally be passed to another graph's ``capture_begin``, which hints the other graph may share the same memory pool. )r.r4rAs&rr4CUDAGraph.pools w|~rcV^8dQhRR/#r>r)rs"rrr+s++4+rc <\SV`4#)z/Enable debugging mode for CUDAGraph.debug_dump.)r.enable_debug_moderAs&rrTCUDAGraph.enable_debug_modesw(**rc V^8dQhRRRR/#)r debug_pathr7rr8r)rs"rrr+s..S.T.rc "<\SV`V4#)z Arguments: debug_path (required): Path to dump the graph to. Calls a debugging function to dump the graph if the debugging is enabled via CUDAGraph.enable_debug_mode() )r. debug_dump)r;rWr1s&&rrYCUDAGraph.debug_dumpsw!*--rcV^8dQhRR/#rrintr)rs"rrr+s(((rc <\SV`4#)amReturns the underlying cudaGraph_t. ``keep_graph`` must be True. See the following for APIs for how to manipulate this object: `Graph Managmement `_ and `cuda-python Graph Management bindings `_ )r.raw_cuda_graphrAs&rr_CUDAGraph.raw_cuda_graphs w%''rcV^8dQhRR/#r\r)rs"rrr+s--S-rc <\SV`4#)aReturns the underlying cudaGraphExec_t. ``instantiate`` must have been called if ``keep_graph`` is True, or ``capture_end`` must have been called if ``keep_graph`` is False. If you call ``instantiate()`` after ``raw_cuda_graph_exec()``, the previously returned cudaGraphExec_t will be destroyed. It is your responsibility not to use this object after destruction. See the following for APIs for how to manipulate this object: `Graph Execution `_ and `cuda-python Graph Execution bindings `_ )r.raw_cuda_graph_execrAs&rrcCUDAGraph.raw_cuda_graph_execs w*,,rr)F)Nglobal)__name__ __module__ __qualname____firstlineno____doc__r/r:r@rErIrMr4rTrYr_rc__static_attributes__ __classcell__)r1s@rrrBsx400PP*  ++..((--rcR]tRt^t$RtRtR]R&R RRlltRRltR R lt R t R#) ra1 Context-manager that captures CUDA work into a :class:`torch.cuda.CUDAGraph` object for later replay. See :ref:`CUDA Graphs ` for a general introduction, detailed use, and constraints. Arguments: cuda_graph (torch.cuda.CUDAGraph): Graph object used for capture. pool (optional): Opaque token (returned by a call to :func:`~torch.cuda.graph_pool_handle()` or :meth:`other_Graph_instance.pool()`) hinting this graph's capture may share memory from the specified pool. See :ref:`Graph memory management`. stream (torch.cuda.Stream, optional): If supplied, will be set as the current stream in the context. If not supplied, ``graph`` sets its own internal side stream as the current stream in the context. capture_error_mode (str, optional): specifies the cudaStreamCaptureMode for the graph capture stream. Can be "global", "thread_local" or "relaxed". During cuda graph capture, some actions, such as cudaMalloc, may be unsafe. "global" will error on actions in other threads, "thread_local" will only error for actions in the current thread, and "relaxed" will not error on actions. Do NOT change this setting unless you're familiar with `cudaStreamCaptureMode `_ enable_annotations (bool, optional): If ``True``, enables kernel annotation recording on entry and automatically calls :func:`~torch.cuda._graph_annotations.resolve_pending_annotations` before the capture ends. Annotations are **not** cleared on exit so that multiple graphs in the same workload can accumulate annotations. Requires ``cuda.bindings`` package and cuda-compat >= 13.1 or CUDA driver >= 13.1. .. note:: For effective memory sharing, if you pass a ``pool`` used by a previous capture and the previous capture used an explicit ``stream`` argument, you should pass the same ``stream`` argument to this capture. .. warning:: This API is in beta and may change in future releases. .. _cudaStreamCaptureMode: https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__STREAM.html#group__CUDART__STREAM_1g9d0535d93a214cbf126835257b16ba85 Ntorch.cuda.Stream | Nonedefault_capture_streamc ,V^8dQhRRRRRRRRR R /#) r cuda_graphrr4r5streamrnr6r7enable_annotationsrr)rs"rrgraph.__annotate__s<666"6) 6  6 ! 6rc VfFVPPf.\PP 4VPnVfRMV3VnVeTMVPPVnVP f \R4h\PPVP 4Vn Wn W@n WPn R#)Nzcapture_stream must not be Noner) r1ror%r&Streamr4capture_streamAssertionErrorrr stream_ctxrqr6_enable_annotations)r;rqr4rrr6rss&&&&&&r__init__graph.__init__s >dnnCCK49JJ4E4E4GDNN 1;?r)rs"rrrts  4 rc \PP4\PPP 'd\ P!4\PP4\PP4VP'd^RI H pV!4VPP4VP P"!VP$RVP&/R#)r)rsr6N)r%r& synchronizecompilerconfigforce_cudagraph_gcgccollect empty_cache_C_host_emptyCacherztorch.cuda._graph_annotationsrsry __enter__rqr:r4r6)r; _enable_anns& rrgraph.__enter__s  >> 3 3 3 JJL   !!#  # # # W M !!# %% YY  $66  rc V^8dQhRRRR/#)rargsobjectrr8r)rs"rrrts 1 1f 1 1rc VP'd^RIHpV!4VPP 4VP P !V!VP'd^RIHpV!VP4R#R#)r)resolve_pending_annotations)remap_to_exec_graphN)rzrrrqr@ry__exit__r)r;rrrs&* rrgraph.__exit__sY  # # # Q ' ) ##%   $'  # # # I  0 $r)rzr6rwrqr4ry)NNreF) rfrgrhrirjro__annotations__r{rrrkrrrrrs+!F8<4;62 @ 1 1rtorch.nn.Module.r_ModuleOrCallablec 0V^8dQhRRRRRRRRR R R R/#) r callablesr sample_argstuple[Tensor, ...]num_warmup_itersr]allow_unused_inputrr4r5rr)rs"rrr)sD #     rcR#r-rrrrrr4s&&&&&rmake_graphed_callablesr(src 0V^8dQhRRRRRRRRR R R R/#) rrztuple[_ModuleOrCallable, ...]rztuple[tuple[Tensor, ...], ...]rr]rrr4r5rr)rs"rrr3sD((,(/(( (  ( # (rcR#r-rrs&&&&&rrr2s%(rc 0V^8dQhRRRRRRRRR R R R/#) rrz1_ModuleOrCallable | tuple[_ModuleOrCallable, ...]rz3tuple[Tensor, ...] | tuple[tuple[Tensor, ...], ...]rr]rrr4r5rr)rs"rrr<sLII@IDII I  I 7 Irc \P!4'd'\P!4'd \R4hRp\ V\ 4'g0RpV3p\ P!\ \R3,V43pM5\ P!\ \ \R3,R3,V4p.p\W4EFqwr\ V\PP4'd\VP4^8Xd5\VP4^8Xd\VP4^8Xg \!R4h\";QJd*RVP%44F 'dK RM RM!RVP%444'g \!R4h\P&P(P*!V !p VP-\ V 44\";QJdRV 4F 'dK RM RM !RV 44'dEKi\!R4h VU u.uFp \V 4NK p p VUu.uFHp\ V\PP4'd\ VP/44MRNKJ p p\1\V44U u.uFp W},W,,NK pp \1\V44Uu.uF!p\P2P54NK# pp\1\V44Uu.uF!p\P2P54NK# ppVf \74MTp\P2P94\P2P;\P2P=44;_uu_4\WV4EFwpp pRRRppp\1V4Fp\P&P(P?V!V !4p\ ;QJd.RV4FNK 5M !RV44p\V4^8gKn\P@PCT\ ;QJd.R V4FNK 5M !R V44\ ;QJd.R V4FNK 5M !R V44RVR 7pK VVV3Fp?K EK RRR4\P2P94.p.p\WV4Fwpp p\P2PEVVR 7;_uu_4V!V !pRRR4\P&P(PGX4wppVP-\ V44VP-V4K .p.p \\IV4\IV4\IV44EFwpp!p"\ ;QJd.R V!4FNK 5M !R V!44p#\ ;QJd.RV!4FNK 5M !RV!44pRp\V4^8d\P2PEV"VR 7;_uu_4\P@PCT\ ;QJd.RV4FNK 5M !RV44\ ;QJd.RV#4FNK 5M !RV#44RVR 7pRRR4.p$^p%VFMp&V&PJ'd(Ve$V$P-VV%,4V%^, p%K<V$P-R4KO \ V$4p$VP-V#4V P-V$4EK VPM4V PM4RRlp'.p(\OV4Fwp pV'!VV ,VV ,W,W,VV ,W,VV ,VV ,V V ,4 p)\ V\PP4'd>RRlp*V*!VVPPV)VPR4Vn)V(P-V4KV(P-V)4K V'd V(^,#\ V(4#uup iuupiuup iuupiuupi +'giEL;i +'giEL;i +'giEL;i)a Accept callables (functions or :class:`nn.Module`\ s) and returns graphed versions. Each graphed callable's forward pass runs its source callable's forward CUDA work as a CUDA graph inside a single autograd node. The graphed callable's forward pass also appends a backward node to the autograd graph. During backward, this node runs the callable's backward work as a CUDA graph. Therefore, each graphed callable should be a drop-in replacement for its source callable in an autograd-enabled training loop. See :ref:`Partial-network capture` for detailed use and constraints. If you pass a tuple of several callables, their captures will use the same memory pool. See :ref:`Graph memory management` for when this is appropriate. Arguments: callables (torch.nn.Module or Python function, or tuple of these): Callable or callables to graph. See :ref:`Graph memory management` for when passing a tuple of callables is appropriate. If you pass a tuple of callables, their order in the tuple must be the same order they'll run in the live workload. sample_args (tuple of Tensors, or tuple of tuples of Tensors): Samples args for each callable. If a single callable was passed, ``sample_args`` must be a single tuple of argument Tensors. If a tuple of callables was passed, ``sample_args`` must be tuple of tuples of argument Tensors. num_warmup_iters (int): The number of warmup iterations. Currently, ``DataDistributedParallel`` needs 11 iterations for warm up. Default: ``3``. allow_unused_input (bool): If False, specifying inputs that were not used when computing outputs (and therefore their grad is always zero) is an error. Defaults to False. pool (optional): Token (returned by :func:`~torch.cuda.graph_pool_handle` or :meth:`other_Graph_instance.pool()`) that hints this graph may share memory with the indicated pool. See :ref:`Graph memory management`. .. note:: The ``requires_grad`` state of each Tensor in ``sample_args`` must match the state that's expected for the corresponding real input in the training loop. .. warning:: This API is in beta and may change in future releases. .. warning:: ``sample_args`` for each callable must contain only Tensors. Other types are not allowed. .. warning:: Returned callables do not support higher order differentiation (e.g., double backward). .. warning:: In any :class:`~torch.nn.Module` passed to :func:`~make_graphed_callables`, only parameters may be trainable. Buffers must have ``requires_grad=False``. .. warning:: After you pass a :class:`torch.nn.Module` through :func:`~make_graphed_callables`, you may not add or remove any of that Module's parameters or buffers. .. warning:: :class:`torch.nn.Module`\s passed to :func:`~torch.cuda.make_graphed_callables` must not have module hooks registered on them at the time they are passed. However, registering hooks on modules *after* passing them through :func:`~torch.cuda.make_graphed_callables` is allowed. .. warning:: When running a graphed callable, you must pass its arguments in the same order and format they appeared in that callable's ``sample_args``. .. warning:: The automatic mixed precision is supported in :func:`~torch.cuda.make_graphed_callables` only with disabled caching. The context manager `torch.cuda.amp.autocast()` must have `cache_enabled=False`. z_make_graphed_callables does not support the autocast caching. Please set `cache_enabled=False`.FT.c3<"TFqPRJxK R#5i)FN requires_grad.0bs& r )make_graphed_callables..sEA%/sc3V"TFp\V\P4xK! R#5ir-) isinstancer%r )rargs& rrrsHKS:c5<<00Ks')Nc3L"TFqP'gKVxK R#5ir-rros& rrrs$K1??QQ$ $c3L"TFqP'gKVxK R#5ir-rris& rrrs%';!AA';rc3t"TF.qP'gK\P!V4xK0 R#5ir-rr% empty_likers& rrrs(+9@AOO/E,,Q//s88)outputsinputs grad_outputs only_inputs allow_unused)r4c3t"TF.qP'd\P!V4MRxK0 R#5ir-rrs& rrrs)$ FT???E  Q  .__annotate__sl====6= = < = 1 =+=7=/= =rc aaaaaaaaaa !VVVVVVV3RlR\PP4o RV VV3Rllp V #)c<]tRtRt]RVVVV3Rll4t]]PPPRVVV3Rll44t Rt R#)Omake_graphed_callables..make_graphed_autograd_function..Graphedi&c$V^8dQhRRRRRR/#)rctxrrr rrr)rs"rr\make_graphed_callables..make_graphed_autograd_function..Graphed.__annotate__)s' A AV Af A9K Arc <\S4FSpSV,P4W,P48wgK5SV,PW,4KU SP4\ S\ 4'g\ R\S4 24h\ ;QJd.RS4FNK 5#!RS44#)z"static_outputs must be tuple, got c3@"TFqP4xK R#5ir-detachrs& rrjmake_graphed_callables..make_graphed_autograd_function..Graphed.forward..3s@AXXZZs)rangedata_ptrcopy_rIrtuplerxtype)rrrrrrrs&* rforwardWmake_graphed_callables..make_graphed_autograd_function..Graphed.forward's}-A+A.779VY=O=O=QQ,Q/55fi@.  "!.%88(.make_graphed_autograd_function..Graphed.backward..Js(0#$-AHHJQ6/s#%) lenrxziprrrIrrr)rrggradrrrs&* rbackwardXmake_graphed_callables..make_graphed_autograd_function..Graphed.backward5su:%8!99(%c%j\1NsSfOgNhi ##6>GA}::<4==?:GGDM ?   """4e<<(@FXAY@Z[u0uu0rrN) rfrgrhri staticmethodrr%autogradfunctiononce_differentiablerrk)rrrrrrrsrGraphedr&sN  A A A ^^ $ $ 8 8  9 rrc V^8dQhRRRR/#)r user_argsrrr)rs"rrTmake_graphed_callables..make_graphed_autograd_function..__annotate__Ps R Rv R& Rrc<\PPP!V!pSP!\ V4S,!p\PPP VS4#r-)r%utils_pytreearg_tree_leavesapplyrtree_unflatten)rflatten_user_argsoutrrrs* rfunctionalizedVmake_graphed_callables..make_graphed_autograd_function..functionalizedPsW!& 3 3 C CY O --%(9":]"JLC;;&&55c;PQ Qr)r%rFunction) rrrrrrrrrrrs fffffffff @rmake_graphed_autograd_function>make_graphed_callables..make_graphed_autograd_functions4( ( enn--( T R Rrc ,V^8dQhRRRRRRRRRR/#) rfuncrgraph_training_statergraphedzCallable[_P, _R]orig_fwdrr)rs"rrrks:  % &* * +   "  rc&aaaaRVVVV3RllpV#)c$V^8dQhRRRRRR/#)rrz_P.args user_kwargsz _P.kwargsrrr)rs"rrJmake_graphed_callables..make_graphed_forward..__annotate__qs&CCC CbCrcF<SPS8Xd S!V/VB#S!V/VB#r-)training)rrrrrrs*,rnew_fwdEmake_graphed_callables..make_graphed_forward..new_fwdqs2}}(<<& A[AA'BkBBrr)rrrrr sffff rmake_graphed_forward4make_graphed_callables..make_graphed_forwardks CCrzModules must not have hooks registered at the time they are passed. However, registering hooks on modules after passing them through make_graphed_callables is allowed.zIn any :class:`~torch.nn.Module` passed to :func:`~make_graphed_callables`, only parameters may be trainable. All buffers must have ``requires_grad=False``.zfIn the beta API, sample_args for each callable must contain only Tensors. Other types are not allowed.r)*r%is_autocast_enabledis_autocast_cache_enabled RuntimeErrorrrtypingcastr rnnModuler_backward_hooks_forward_hooks_forward_pre_hooksrxallbuffersrrrappend parametersrr&rr'rrrrv tree_leavesrrr tree_flattenreversedrreverse enumerater r)+rrrrr4just_one_callable _sample_argsflatten_sample_argscr flatten_argper_callable_len_user_argsper_callable_module_paramsr"per_callable_static_input_surfaces_ fwd_graphs bwd_graphsmempoolrr grad_inputsr outputs_gradvper_callable_static_outputs"per_callable_output_unflatten_specr func_outputsflatten_outputsspec per_callable_static_grad_outputsper_callable_static_grad_inputsrrrrgrad_idxrrretrr s+&&&&& rrr<srR   ""u'F'F'H'H m   i ' ' L  E&#+$6 DF {{5vs{);S)@#A;O y/ a ) )A%%&!+(()Q.,,-2$a3EE333EEEE$1 kk))994@ ""5#56sHKHsssHKHHH ^ )069L!L8K#d)8K!L"A",Auxx!?!?allnRG" s9~&*&A !;!>>>&'* 38I2GH2GQ%**&&(2GJH27I2GH2GQ%**&&(2GJH%)\!tG  JJ   5::,,. / /03 %G1 ,D$,26tT,K+,++--99$+F$u$K$Kuu$K$KK |$q("'.."5"5 ,$u%';%uu%';% &+U+9@+UU+9@+&%)%7#6 #K -|[9:'1  0. JJ#%)+&!$Yj!IdI ZZ  ig  6 6;L7!& 3 3 @ @ N#**5+AB*11$7 "J(*$&(#;>34,-<7ni$e$ FT$ ee$ FT$  uJJuuJJJ  | q !!)'!::#nn11( 5 T,@ T55 T,@ TT!&&W2E&W&W2E&W!W $!3 2 ; 'C   [%<"))+h*?@A "))$/ ( ##56(//0CD'../ABK<P%,,.#++-=@$&CY'40 qM qM & ) & ) .q 1 . 1 ' * ,Q / +A .   dEHHOO , ,  0dmmWdllDL JJt  JJw I(L1v :i"M"* IH 0 / /B7 6 60;::s]b4Ab9b>;'c:'cBc ($c  *c 8>c c! Ac5(c5 c ! c2 5 d )r!r'rrr)FN)'__conditional_annotations__ __future__rrrcollections.abcrrrrrtyping_extensionsr r r r%r torch.cudar _utilsr__all__rrhasattrr__dict__torch._Crrrr!r'rrrrrr)r:s@rrDs6"" $<<66 '   T]t_uxx*++&1,&?EHHl#.9:N.OEHH*+:E(;EHH67TS,9q- q-hk1k1^ %%6f8M%MN9N   ( (IIr