Ph' NddlZddlZddlmZmZmZmZddlZddlm Z m Z ddl m Z m Z mZgdZde dee fdZde d eedee fd Zd e defd Ze j*fd e d edede fdZd e defdZdd e deede fdZd e dede fdZd e defdZd e dej6de fdZy)N)AnyListOptionalSet)_ShardingIterDataPipeSHARDING_PRIORITIES)DataPipe DataPipeGraph traverse_dps)apply_random_seedapply_shardingapply_shuffle_seedapply_shuffle_settingsget_all_graph_pipesgraphreturnc*t|tSN)_get_all_graph_pipes_helperset)rs jC:\Users\daisl\Desktop\realtime-object-detection\venv\Lib\site-packages\torch/utils/data/graph_settings.pyrrs &uce 44id_cachecg}|jD]J\}\}}||vr|j||j||jt ||L|Sr)itemsaddappendextendr)rrresultsdp_iddatapipe sub_graphs rrrs_ G(- $$) H   Ux 29hGH )6 Nrr!c~t|tryt|dr tj|j ryy)NTr F) isinstancerhasattrinspectismethodr r!s r_is_sharding_datapiper)&s4(12x)*w/?/?@W@W/X rnum_of_instances instance_idcFt|}dfd ||S)z Apply dynamic sharding over the ``sharding_filter`` DataPipe that has a method ``apply_sharding``. RuntimeError will be raised when multiple ``sharding_filter`` are presented in the same branch. cP|jD]\}}d}t|rs|td|d|tj|j }t |jdkr|j n|j  |}||}||y)NzoSharding twice on a single pipeline is likely unintended and will cause data loss. Sharding already applied to z while trying to apply to )sharding_group)valuesr) RuntimeErrorr& signaturer len parameters) r prev_applieddpr"appliedsig_helperr+r*r/s rr9zapply_sharding.._helper9s$||~ORG$R(+&(FFR^Smnpmq(stt''(9(9:s~~&*%%&6 D%%&6 Tb%c& Iw ' .rr)r )r!r*r+r/rr9s ``` @rr r .s'  "E(($ EN Orct|dr t|dsytj|jrtj|jsyy)N set_shuffleset_seedFT)r%r&r'r;r<r(s r_is_shuffle_datapiper=PsE 8] +78Z3P   H00 19I9I(J[J[9\ rshufflec||St|}t|}|Dcgc]}t|s|}}|s*|r(tjd|j }|g}|D]}|j ||Scc}w)aE Traverse the graph of ``DataPipes`` to find and set shuffle attribute. Apply the method to each `DataPipe` that has APIs of ``set_shuffle`` and ``set_seed``. Args: datapipe: DataPipe that needs to set shuffle attribute shuffle: Shuffle option (default: ``None`` and no-op to the graph) z`shuffle=True` was set, but the datapipe does not contain a `Shuffler`. Adding one at the end. Be aware that the default buffer size might not be sufficient for your task.)r rr=warningswarnr>r;)r!r>r all_pipespipe shufflersshufflers rrrXs  "E#E*I"+J)$/CD/I)IJ   [ ##%L W% OKs A<A<rngcDtjdt||S)Nz`apply_shuffle_seed` is deprecated since 1.12 and will be removed in the future releases. Please use `apply_random_seed` instead.)r@rAr )r!rFs rrrws" MM 4 Xs ++rc\t|dr tj|jryy)Nr<TF)r%r&r'r<r(s r_is_random_datapiperIs%x$)9)9(:K:K)L rct|}t|}t}g}|D]G}t||vrt |s|j ||j t|I|D]_}ttjdtjj|j}|j|a|S)a6 Traverse the graph of ``DataPipes`` to find random ``DataPipe`` with an API of ``set_seed``. Then set the random seed based on the provided RNG to those ``DataPipe``. Args: datapipe: DataPipe that needs to set randomness rng: Random number generator to generate random seeds )dtype) generator)r rridrIrrinttorchemptyint64random_itemr<)r!rFrrBcacherandom_datapipesrC random_seeds rr r s  "E#E*I EE d8u   t $  # #D ) IIbh  !%++b <DDsDSXXZ[  k"! Orr)r&r@typingrrrrrP(torch.utils.data.datapipes.iter.shardingrrtorch.utils.data.graphr r r __all__rrOrboolr)DEFAULTr r=rrrI Generatorr rKrrr_s'++ IH 5}5h5}CTRZ^H#6"="=X%( #CKD8XRZ>,,,,(t u8r