Ph:B ddlmZddlmZmZddlmZddlmZm Z m Z m Z m Z m Z mZmZddlZddlmZddlmcmZddlmZddlZddlmZddlmcmZddlZddl Z ddl!Z!ddl"m#Z#de$d e e$e$ffd Z%d e ed ejLd e e$e ffdZ'd ejLd e e$e fdej$jPfdZ)d'dej$jPd ej$jPfdZ*dejPd ejPfdZ+dejPdeejLdeejLdeejLfdZ,ejZej\ej^ej`ejbejdejfejhejjejlejhejnejpg Z9ejtejvgZ (`foo.bar`, `baz`) .r)rsplit)rparentnames mC:\Users\daisl\Desktop\realtime-object-detection\venv\Lib\site-packages\torch/fx/experimental/optimization.py _parent_namers3 MM#q)MVT6!9 ,,B ,,patternnodemodulescjt|jdk(ry|jd|f}t||D]z\}}t|tj sy|j dk7ryt|jtsy|j|vryt||j|uszyy)NrF call_moduleT) lenargszip isinstancefxNodeoprstrtype)rrrnodes expected_type current_nodes rmatches_module_patternr.s 499~"&))A,!5E'*7E':# |,0 ??m +,--s3   g -  ++, -] B(; r new_modulect|jtsJt|j\}}|||j<t ||||yN)r%rr)rsetattr)rrr/ parent_namers rreplace_node_moduler4,sH dkk3 '( '$T[[1K%GDKK GK $ 3rmodelcftjtjftjtjftj tj fg}|stj|}tj|}t|j}tj|j}|D]}|jD]}t|||st!|j"dj$dkDr7||j"dj&}||j&} | j(sot+|| } t-|j"d|| |j/|j"d|j1|tj2||S)z Fuses convolution/BN layers for inference purposes. Will deepcopy your model by default, but can modify the model inplace as well. rr)nnConv1d BatchNorm1dConv2d BatchNorm2dConv3d BatchNorm3dcopydeepcopyr&symbolic_tracedict named_modulesgraphr+r.r"r#usersrtrack_running_statsrr4replace_all_uses_with erase_node GraphModule) r5inplacepatternsfx_modelr new_graphrrconvbn fused_convs rfuserP2sN BNN+BNN+BNN+-H  e$  'H8))+,G hnn-IOOD%gtW=tyy|))*Q.tyy|223T[[)--.tR8 #DIIaL':F**499Q<8$$T*$ >>(I ..rctj|}Gddtjj}||j S)z5 Removes all dropout layers from the module. cDeZdZdedeedfdeeefdeffd Z xZ S)&remove_dropout..DropoutRemoverrr#.kwargsrct|j|tjrt |dk(sJ|dSt ||||S)Nrr)r% submodulesr7Dropoutr"superr!)selfrr#rT __class__s rr!z2remove_dropout..DropoutRemover.call_moduleVsG$//&12::>4yA~%~Aww*64@@r) __name__ __module__ __qualname__rr rrr)rr! __classcell__)rZs@rDropoutRemoverrSUsF Av AeHcM6J AUYZ]_bZbUc Ahk A Arr_)r&r@torch Transformer transform)r5rKr_s rremove_dropoutrcOsB  'HA--A ( # - - //r orig_moduler+inputsoutputscZ tj}i |D]"}|j|j}| |<$|D]}|j | fd}| |<|j |Dcgc]} | c}|j tj||Scc}w)z Given lists of nodes from an existing graph that represent a subgraph, returns a submodule that executes that subgraph. c|Sr1)xenvs rz"extract_subgraph..hs s1vr)r&Graph placeholderr node_copyoutputlintrH) rdr+rerfrLinputnew_noderrprks @rextract_subgraphrt^s I"$C((4E &&t-=>D 8fc&k89 NN >>+y 119s/ B(c,tj|Sr1) th_mkldnnMkldnnBatchNorm)a_s rrlrl{s!:!:1!=rci}|D]}|jdk(st|jtsJ||j}t |t vsPt t ||t j}t|tjsJtj|||<t||||S)z For each node, if it's a module that can be preconverted into MKLDNN, then we do so and create a mapping to allow us to convert from the MKLDNN version of the module to the original. r!) r(r%rr)r* mkldnn_mapr`floatr7Moduler>r?r4)r+r old_modulesr cur_moduler/s rmodules_to_mkldnnrs /1K 77m #dkk3/ 0/ -JJ:-'Z(89*ekkR !*bii898*.-- *C J'#D':> rr~c|D]R}|jdk(st|jtsJ||j}||vsCt ||||Ty)za Maps each module that's been changed with `modules_to_mkldnn` back to its original. r!N)r(r%rr)r4)r+rr~rrs r reset_modulesrsU  77m #dkk3/ 0/ -J[(#D';z3JK rc,eZdZdejfdZy) MklSubgraphfx_graphc<||_g|_g|_g|_yr1)rr+ start_nodes end_nodes)rYrs r__init__zMklSubgraph.__init__s  $& *,(*rN)r[r\r]r&rmrrirrrrs++rrcDdddtdtffd }|S)aW This generates a heuristic that can be passed into `optimize_for_inference` that determines whether a subgraph should be run in MKL by running it with the example_inputs. Example usage: heuristic = gen_mkl_autotuner(example_inputs, iters=10) fast_model = optimization.optimize_for_inference(model, heuristic) NrCrc|j} F|jj |jj t j |Dcgc]!}t j|j#c}tttj|jDcgc]}|jdc}}t |j || fd}|fd}t#j$j t'j) |fd}||kScc}wcc}w)NrctD] }| tj}tD] }|} tj|z Sr1)rangetime)frybeginoutiterswarmups r benchmarkz?gen_mkl_autotuner..use_mkl_heuristic..benchmarksG6]#IIKE5\c"99;& &rcDcgc]}|jc}Dcgc]}|jc}Scc}wcc}wr1) to_mkldnnto_dense)i sample_inputs submodules rrlz>gen_mkl_autotuner..use_mkl_heuristic..s@I_lGm_lZ[ _lGmgen_mkl_autotuner..use_mkl_heuristic..s =(Ar)rr owning_moduler~r propagater`randnshaper r r&r'rr#rtr+rrCrArB)rC input_nodesr output_argsrmkl_time no_mkl_timerrexample_inputsrKrr~rs @@ruse_mkl_heuristicz,gen_mkl_autotuner..use_mkl_heuristics''  ~~33H..44K h  ) ). 9=HI[TTZZ0[I 4=EOO*TOD499Q.E )rbool)rrrrrKr~s``` @@rgen_mkl_autotunerrs/HK&&&&0 rrCc2t|jdkDS)z This is a heuristic that can be passed into `optimize_for_inference` that determines whether a subgraph should be run in MKL by checking if there are more than 2 nodes in it )r"r+)rCs ruse_mkl_lengthrs u{{ a rc>eZdZdZdefdZdedefdZdedefdZy ) UnionFindc0dg|z|_dg|z|_y)Nrrsize)rYns rrzUnionFind.__init__s,06A: !sQw rvc@||j|<d|j|<y)Nrr)rYrs rmake_setzUnionFind.make_sets A ! rrc|j|}||k(r|S|J|j||j|<tt|j|Sr1)rfindr int)rYrpars rrzUnionFind.findsOkk!n 8H3 ACQ((rrxbc|j||j|}}||k(r|S|j||j|kr||}}||j|<|j|xx|j|z cc<yr1)rrr)rYrxrs rjoinzUnionFind.joinsoyy|TYYq\1 6H 99Q<$))A, &aqA A !  ! $ rN)r[r\r]rrrrrrirrrrs9'#)c)c)%c%c%rr pass_configtracercdddtid}|i}|j||dr t|}|dr t|}|ddur|St |dt s t d d|dvr t d |dd}|}|jtj|tj|j}t |j}Gd d t}tj D]} |j"} | j$d k(r|| j&} t)| t*vr|j,} t/| j1d} | | j2t4j6k(sJd| j8t5j8dk(sSJd| j$dk(r=| j&t*vr |j,} n| j&t:vr |j<} | |j"k7s| |j<k(rt?d| j@Ds>jC| 5tjD| j@fd} dddtGtHtjJjL | _ jO| 5jQdd| f}| jS|| f|_ dddtUtj |}|_+j D]} | j$dk(s| j&dk(s#| j@d}t| jX}|D]D}|j$dk(s|j&dk(s#|jS|j[|Ft]| jXdk(sj[| t]j }t_|fd}taj D]'\}} | j$dk(r(| j&dk(r|| _1je|>| j$dk(rA| j&dk(r2|| j@dJ|| j@d| _3| jhDcgc].}t |tjjs||'||0}}t]|dk(rt?d|DrJtm|}|d| _7|ddD]}jq|d|*tsfd}j D]} tu| dr7|jw| jnj jy| tu| dr7|jw| jbjzjy| tu| ds|jw| jfj|jy| |jD]q}||r |jz|j|zD]3} | j@d}| jS|j[| 5t|j ||sd}j D]&} | j&dk(s| j&dk(s"|dz }(tjtjd|jtj|}|S#1swYxYw#1swYVxYwcc}w) a Performs a set of optimization passes to optimize a model for the purposes of inference. Specifically, the passes that are run are: 1. Conv/BN fusion 2. Dropout removal 3. MKL layout optimizations The third optimization takes a function `use_mkl_heuristic` that's used to determine whether a subgraph should be explicitly run in MKL layout. Note: As FX does not currently handle aliasing, this pass currently assumes nothing aliases. If that isn't true, use at your own risk. T heuristic) conv_bn_fusercmkldnn_layout_optimizeNrrcrFz+mkldnn_layout_optimize config is not a dictz4Heuristic not found in mkldnn_layout_optimize configceZdZdZdZdZy)*optimize_for_inference..MklSupportrrN)r[r\r]NOYESUNKNOWNrirr MklSupportrs rrr!z)this pass is only for torch.float modulescpuz!this pass is only for CPU modules call_functionc3:K|]}|jdk(yw)rN)r).0args r z)optimize_for_inference..3sIy3::3ysc*jd|fS)Nr) call_method)rrs rrlz(optimize_for_inference..6sh>R>RS^ab`e>frrrrrct|drj|jSt|drj|jSy)Ncolor start_color)hasattrrrr)rufs r get_colorz)optimize_for_inference..get_colorSs@ 1g 77177# # 1m $771==) )rc3$K|]}|du ywr1ri)rrs rrz)optimize_for_inference..rs9j19jsrctSr1)r)rsrrlz(optimize_for_inference..ys H@Urrr end_colorzmkldnn conversions: )FrupdaterPrcr%rA RuntimeErrortracer>r?r&rHrootrBrlistr+rr(rr*mkldnn_supportedrnext parametersdtyper`r|devicemkldnn_supported_unknownranyr#inserting_beforemap_argr r rrinserting_after create_noderFrr~rDrGr"r enumeraterrrall_input_nodesr'sortedrrrrrappendrrvaluesrlogging getLoggerr[inforq) r5rrdefault_pass_configr cur_tracerrKrrrsupports_mkldnnrsample_parameter mkldnn_argsdense_xr~prv_noderDuser num_nodesrcur_idxr cur_colors other_color mkldnn_graphsrCprvmkldnn_conversionsresultrrs @@roptimize_for_inferencersV&#."?  {+>*U +,u%34= )*BCT JHII-.FGGQRR+,DEkRJ e 45H~~joox8H$()<)<)>$?GTX^^$$-- 77m # -JJ#33",..#' (=(=(?#F #/+11U[[@nCnn@+22ell56IIoLooI WW '{{..",.. 88","4"4 jmm +*"4"44ItyyII**40 jj4fg 1U277#3#34kBDI))$/"..}j4'R**73 $w 0/3%>$D$8'BK&H 77m # z(Ayy|H$E77m+ {0J..x8''-4::!###D)HNN#I 9 B&#8>>2  77m # {(B&D  KK WW %$++*CTYYq\*6 76&tyy|4DN040D0D{0D1 STVXV]V]H^bklmbnbz)A,0DJ{:!#9j99 :9 +J#ADJ)!"~  1 {3 .3&-88U,VM 4 ! "''$**- . 4 4 ; ;D A 4 ' "''$"2"23 4 @ @ G G M 4 % "''$..1 2 < < C CD I %%' '))EOO;iil**3/##D)< %++w < ( ;;+ % )B ! #  h$$'; collectionsrtorch.utils.mkldnnutilsmkldnnrvoperatorrrenumrr)rr'r.r}r4rPrcrtr:Linearr;ReLU MaxPool2d AvgPool2dAdaptiveAvgPool2drelu transposesigmoid avg_pool2dadaptive_avg_pool2draddmulr MkldnnConv2d MkldnnLinearr{rrrrrrrTracerrrirrr!s*3III 0 #&& -#-%S/-HTN"''DQTVYQYN$4bgg4S#X4EHHOO4 //588??/: 0")) 0 02"))2DM24PRPWPW=2cghjhohocp2"IIryy".."''2<<rOcOc JJFFALL!//%LL(,,7IIy%%IIy%%NN= T"'']T#ryy.5I$ Lbgg Lc299n1E LTXY[YbYbdfdmdmYmTn L++$L + $ %%8-1 iii 88??i$sCx.)i Oi XX__ ir