+ &ja+^RIt^RIt^RIHtHtHtHtHtHt^RI H t R RR/Rllt R Rlt R Rlt R RltRRR/R lltRRR/R lltR#)N)_flatten_dense_tensors_get_device_index_handle_complex_reorder_tensors_as _take_tensors_unflatten_dense_tensors)nccloutc"\V4pVRJVRJ, 'g\RV RV 24hVe:VUu.uFp\V4NK pp\PP W4#\PP W4#uupi)aBroadcasts a tensor to specified GPU devices. Args: tensor (Tensor): tensor to broadcast. Can be on CPU or GPU. devices (Iterable[torch.device, str or int], optional): an iterable of GPU devices, among which to broadcast. out (Sequence[Tensor], optional, keyword-only): the GPU tensors to store output results. .. note:: Exactly one of :attr:`devices` and :attr:`out` must be specified. Returns: - If :attr:`devices` is specified, a tuple containing copies of :attr:`tensor`, placed on :attr:`devices`. - If :attr:`out` is specified, a tuple containing :attr:`out` tensors, each containing a copy of :attr:`tensor`. NzFExactly one of 'devices' and 'out' must be specified, but got devices=z and out=)r RuntimeErrorrtorch_C _broadcast_broadcast_out)tensordevicesr ds&&$ n/Users/jameslopez/projects/CWCArchive/cwc-podcast/.venv/lib/python3.14/site-packages/torch/nn/parallel/comm.py broadcastrs*V $F _ - -TU\T]]fgjfk l  189A$Q'9xx""633xx&&v33 :sB cVUu.uFp\V4NK ppVUu.uFp\V4NK pp\PP WV4#uupiuupi)aBroadcast a sequence of tensors to the specified GPUs. Small tensors are first coalesced into a buffer to reduce the number of synchronizations. Args: tensors (sequence): tensors to broadcast. Must be on the same device, either CPU or GPU. devices (Iterable[torch.device, str or int]): an iterable of GPU devices, among which to broadcast. buffer_size (int): maximum size of the buffer used for coalescing Returns: A tuple containing copies of :attr:`tensor`, placed on :attr:`devices`. )rrr r_broadcast_coalesced)tensorsr buffer_sizerts&&& rbroadcast_coalescedr2sV.55W #WG5+237aq!7G3 88 ( (; GG63s AAc *\VRR7pV^,P4pRp\V4FwrEVPPR8Xd\ RV R24hVP 4V8XdTpVP4V8wgK]RPRVP444pRPR V44p\R V R V R V 24h Vf \R 4h\V4^8Xd V^,#\P!V4'd7\P!W,4p\P!WVR7V#\P!W,PPV4p \V4UU u.uFwrJWC8wgK V NK p pp W,V ^,P!V RR7,pV R,F%p VP#V P!V RR74K' V#uup pi)aSum tensors from multiple GPUs. All inputs should have matching shapes, dtype, and layout. The output tensor will be of the same shape, dtype, and layout. Args: inputs (Iterable[Tensor]): an iterable of tensors to add. destination (int, optional): a device on which the output will be placed (default: current device). Returns: A tensor containing an elementwise sum of all inputs, placed on the :attr:`destination` device. T)optionalNcpuz7reduce_add expects all inputs to be on GPUs, but input z is on CPUxc38"TFp\V4xK R#5iNstr.0rs& r reduce_add..`s6:a3q66:c38"TFp\V4xK R#5ir!r"r$s& rr&r'as; 1A r(zinput z has invalid size: got z, but expected zLreduce_add expects destination to be on the same GPU with one of the tensors)outputroot)device non_blocking:NN)rsize enumerater,typeAssertionError get_devicejoin ValueErrorr lenr is_availabler empty_likereducetoadd_) inputs destination input_size root_indexiinpgotexpectedresultdestination_devicernonrootothers && r reduce_addrHFs$K$?K!JJF# ::??e # I!JW  >> { *J 88: #((6388:66Cxx; ;;H23%xjQ $ Z   6{aay   !!&"45 F ; M#\\&*<*C*C*H*H+V!*6!2F!2ao11!2F#gajmm%D'4'  R[[E KK(:N O! MGs  H+HcRVUu.uFp.NK pp.p.p\VRR/Fp\;QJdRV4F 'dK RM RM !RV44'd7\Wq4pVPV4VPV^,4Ks\WGRR7F8wrT PV P'dV P 4MT 4K: VPV^,R,4K VU u.uFp \ W4NK p p \V RR/F]p V Uu.uFp\V4NK pp\W4p\VV ^,4Fp VPV P4K K_ \\WV44#uupiuup iuupi)a,Sum tensors from multiple GPUs. Small tensors are first coalesced into a buffer to reduce the number of synchronizations. Args: inputs (Iterable[Iterable[Tensor]]): iterable of iterables that contain tensors from a single device. destination (int, optional): a device on which the output will be placed (default: current device). buffer_size (int): maximum size of the buffer used for coalescing Returns: A tuple of tensors containing an elementwise sum of each group of inputs, placed on the ``destination`` device. strictTc38"TFqPxK R#5ir!) is_sparse)r%rs& rr&'reduce_add_coalesced..s3Nq{{Nr(F)rJ) zipallrHappendrLto_denserrrdatatupler)r<r=r_ dense_tensorsr* ref_ordertensor_at_gpusrDcollrritrschunkschunk flat_tensors flat_results&&& rreduce_add_coalescedr_|sr&.4 4VVM 4 FIv3d3 33N33333N3 3 3rusU 444DH(3l,9^0GPT0Gf*7D*7rt