Ph ddlZddlmZmZmZmZddlZddlmZ ddlm Z m Z gdZ deejjde fdZdeeejjeeeejjfffdZdeeejjeeeejjffde fd Zy) N)UnionIterableDictIterator) ProcessGroupgroup)average_parametersget_params_to_average&average_parameters_or_parameter_groupsparams process_groupc||ntj}tj|ryt j |\}}t j|Dcgc]}|jjdc}}|tj|z}t jjrt jjtj||d}|D]N}||||jzj!|j#||_||jz }Pycc}w)z Averages all the given parameters. For allreduce efficiency, all the parameters are flattened into a contiguous buffer. Thus, it requires extra memory of the same size as the given parameters. N)rr)rWORLDdist_rank_not_in_group itertoolsteetorchcatdatareshapeget_world_sizecuda is_available synchronize all_reducenumelview_astype_as)r r group_to_use params_it1 params_it2p flat_paramsoffsets }C:\Users\daisl\Desktop\realtime-object-detection\venv\Lib\site-packages\torch/distributed/algorithms/model_averaging/utils.pyr r s%2$==5;;L |,&]]62J ))DAQVV^^B/DEK4&&|44K zz   OOK|4 F Vfqwwy&89AA!DLLQO!'')Es"EcBg}|D]}t|tjjr!|}|j6|j |Ht|t r)|dD] }|j|j |"tdt|d|S)z Returns a list of parameters that need to average, which filters out the parameters that do not contain any gradients. Args: params: The parameters of a model or parameter groups of an optimizer. r zParameter input of type z is not supported) isinstancernn ParametergradappenddictNotImplementedErrortype)r filtered_paramsparam param_datas r'r r ,s O eUXX// 0J*&&z2 t $#Ho ??.#**:6.&(@e M^&_` ` c@ttt||y)zM Averages parameters of a model or parameter groups of an optimizer. N)r iterr )r r s r'r r Cst1&9:MJr4)rtypingrrrrrtorch.distributed distributedrrr__all__r*r+r strr r r4r'r=s22 2 c UXX'' (9E<%1C1C(DhtTWY^YaYaYkYkTkOlFm(m"n.K5%((BTBT9UW_`dehjojrjrj|j|e|`}W~9~3KQ]Kr4