Ph#UdZddlZddlmZmZmZmZmZmZddlZddlm Z ddl ZgdZ ee e d<ee Zee efZeegeeee ffZGddZGd d Zdd Zd ed eefdZdeefdZy)zManipulation of micro-batches.N)AnyCallableListUnioncastSequence)Tensor)NoChunkBatchcheckscattergather__all__c,eZdZdZdefdZedZy)r a Wrapper for a Tensor in :meth:`Pipe.forward` indicating that the tensor should not be chunked on the batch dimension and instead be replicated as-is across all micro-batches. This is useful for tensors which might not have any 'batch' semantics for the model. inpcXtj|std|||_y)Nz+NoChunk only supported for tensors, found: )torch is_tensor TypeError_tensor)selfrs uC:\Users\daisl\Desktop\realtime-object-detection\venv\Lib\site-packages\torch/distributed/pipeline/sync/microbatch.py__init__zNoChunk.__init__s(s#I#OP P c|jSN)rrs rtensorzNoChunk.tensor#s ||rN)__name__ __module__ __qualname____doc__r rpropertyrrrr r s' F rr c:eZdZdZdeeeefddfdZe defdZ e dZ dZ d Z d eddfd Zdefd Zd ZdefdZdefdZej.dededdfdZej.dededdfdZdeeefddfdZdeddfdZdeddfdZy)r zC An abstraction representing a microbatch in the pipeline. valuesreturnNc||_tj||_|js5t d|jDst d|jyy)Nc3FK|]}tj|ywrrr).0values r z!Batch.__init__..3sH<%uu-<!zNo tensors found in batch: )_valuesrratomicanyr)rr&s rrzBatch.__init__-sV oof- {{H4<<HH"=dll^ LMMIrcd|js tdtt|jS)z Retrieves the underlying tensor.znot atomic batch)r0AttributeErrorrr r/rs rrz Batch.tensor6s({{ !34 4FDLL))rc|jS)z-Retrieves the underlying values for the batch)r/rs rr&z Batch.values=s||rc|jryt|jD]\}}tj|s|cSt d)z< Retrieves the index of first tensor found. rzNo tensor found!)r0 enumerater/rrr)rir,s rfind_tensor_idxzBatch.find_tensor_idxBsD ;;!$,,/HAuu%0*++rc|jr|jjS|jD]%}tj|s|jcSy)z; Retrieves the device for this microbatch. N)r0r/devicerr)rr,s r get_devicezBatch.get_deviceNs? ;;<<&& &\\Eu%||#"rfunctionc|jrt||jSt||jS)zbCalls a function on the microbatch. It also wraps the output with :class:`Batch`. )r0r r/)rr<s rcallz Batch.callYs3 ;;$,,/0 04<<01 1rc<d|jd|jdS)Nz Batch[atomic=z]()r0r/rs r__repr__zBatch.__repr__bs t{{oR /?qAArc#nK|jr|jy|jEd{y7wrrArs r__iter__zBatch.__iter__es$ ;;,, || # #s +535cH|jrdSt|jS)N)r0lenr/rs r__len__z Batch.__len__ksKKq6S%66rindexcp|js|j|S|dk7r td|jS)Nr atomic batch allows index 0 onlyr0r/ IndexError)rrIs r __getitem__zBatch.__getitem__ns5{{<<& & A:?@ @||rr,cyrr$rrIr,s r __setitem__zBatch.__setitem__x rcyrr$rPs rrQzBatch.__setitem__|rRrcnt|tr|j||y|j||yr) isinstanceint_setitem_by_index_setitem_by_slicerPs rrQzBatch.__setitem__s, eS !  " "5% 0  " "5% 0rc|js.|}|jd||fz|j|dzdz|_y|dk7r td||_y)NrFrrKrL)rrIr,r7s rrWzBatch._setitem_by_indexsX{{A<<+uh6a!eg9NNDL  A:?@ @ rc|j|jcxur|jcxur tdtd|js||_yt |dk7r td|d|_y)Nzonly slice [:] supportedrFz5atomic batch cannot be replaced with multiple tensorsr)startstopstepNotImplementedErrorr0r/rGrMrPs rrXzBatch._setitem_by_slicesl uzz?UZZ?%&@A A@%&@A A{{ DL  u:?TU UQx r)rr r!r"rrrr rr#rr&r8r;Functionr>strrBrDrVrHrNtypingoverloadrQsliceTensorsrWrXr$rrr r (s>NuT#Y%67NDN***  , $2X2'2B#B$ 77 __  V    __  w 4  1sEz!21d1 s d   u    rr r'ctd|Dstd|tfd|Dr tdy)z Checks whether the input contains at least one tensor and each tensor is on the same device as the first partition. Raises: ValueError: input does not contain at least one tensor c3FK|]}tj|ywrr*)r+inputs rr-zcheck..s:6%uu%6r.z inputs do not have any tensors: c3jK|]*}tj|xr|jk7,ywr)rrr:)r+rg first_devices rr-zcheck..s. WPVu5??5 ! Belll&B BPVs03z>All inputs should be on the same device as the first partitionN)r1r ValueError)riinputss` rr r sG :6: ::6(CDD WPV WWYZZXrchunksc t|dk(r>t|dtr+|dj|Dcgc] }t |c}St |Dcgc]}g}}d}|D]}t j|rq|j|}|dk7r(|t|k7rtd|dt|t|}t|D]\}} ||j| t |D]E}t|tr||j|j2||j|G|d|}|Dcgc] }t |c}Scc}wcc}wcc}w)z7Splits an input mini-batch into multiple micro-batches.rFrz6Found different number of chunks produced for inputs:  and N) rGrUr chunkr rangerr RuntimeErrorr6appendr r) rlrkx_batches num_chunksrgtensorsr7rs rr r sa 6{aJvay&9"()//&"9:"9Qa"9::&+Fm4m"mG4J ??5 !kk&)GRJ#g,$>"%[\f[gglmpqxmylz#{||WJ&w/ 6 !!&)06]eW-AJ%%ell3AJ%%e, #,kz"G% &gE!Hg &&;;46 'sE% E*E/outputsc "|djr)td|D}tj|}|Sg}t t |dD]}t |d|}g}|D]D}|t ||k7rtd|dt |||j||Ftj|d|r%|jtj||j|t|}|S)z4Concatenates output micro-batches into a mini-batch.rc34K|]}|jywr)r)r+bs rr-zgather..s2'Q'sz2Types for microbatch outputs do not match, found: ro) r0tuplercatrqrGtyperrsr)ryrxoutput output_bufr7 output_typecurrent_outputsbatchs rrrs qz2'227#$ M!!# s71:'Awqz!}-K O $uQx.0#&XYdXeejkopuvwpxkyjz${||&&uQx0! wqz!}-!!%))O"<=!!/2(z" Mr)r'N)r"rarrrrrrrr torch.cuda.commrr`__annotations__rdTensorOrTensorsr_r r r rVr rr$rrrs% == GcG 6 ( _%uT#Y->'?? @"t t n [ 'S 'T%[ 'FDKr