PhlFddlZddlmZmZddlmZmZddlmZddl m Z m Z m Z m Z mZmZmZmZmZmZddlmZmZmZddlmZdd lmZdd lmZmZmZm Z m!Z!m"Z"m#Z#m$Z$d ed ed e#e%fdZ&d ed edede%de%de%de%de%de%de'de'de#e%d efdZ(d edeeee%e%ffdeeee%e%ffdede%d ef dZ)dededed ed!e"ejTe+fd"e"ejTe'fd#e%d efd$Z,ded%ede"ee efd&eded ed'e%ded(ee%e$efd e"ee effd)Z-ded&eded ed ef d*Z.d+ed&ed,e'd e/fd-Z0d.e"ee efd/e#e"ee efd+ed&eded'e%d efd0Z1d.e"ee efd/e#e"ee efded&eded'e%d efd1Z2 d7d2e%d&ed3e%ded4ee%eeeffded5e/d(e#ee%e$efd efd6Z3y)8N) GraphModulemap_arg)GraphNode)get_new_attr_name_with_prefix) $get_node_first_input_and_output_typegetattr_from_fqnNodeInputOrOutputTypereturn_first_non_observer_nodeget_number_of_non_param_argsget_target_type_str get_arg_indices_of_inputs_to_logget_node_input_qparamsop_type_supports_shadowingget_normalized_nth_input)NSSingleResultValuesType NSSubgraphNSNodeTargetType)get_node_type_to_io_type_map)_is_activation_post_process)DictTupleCallableListAnyUnionOptionalSetnodegmreturncd}t|drw|}|jdk(rJt|jtsJt ||j}t |r t||d}|j|jd}|S)N_node_name_to_scope call_moduler) hasattrop isinstancetargetstrr rrr$name)r r!fqnnode_to_use_for_fqnmodules fC:\Users\daisl\Desktop\realtime-object-detection\venv\Lib\site-packages\torch/ao/ns/fx/graph_passes.py_maybe_get_fqnr0s Cr()# 77m #dkk3/ //%b$++6F*62&>tR&K#$$%8%=%=>qA J logger_clslogger_node_name_suffix ref_node_name model_nameref_nameref_node_target_type results_typeindex_within_arg index_of_argr,c t|j|z|} t||} |||j||| ||| | | }t|| ||jj d| |fi}|S)z Given a starting graph of prev_node -> node -> next_node This function creates a new logger_cls obj and adds it after node, resulting in prev_node -> node -> logger_obj -> next_node r%)rr+rsetattrgraph create_node)r r!r2r3r4r5r6r7r8r9r:r,logger_node_name target_type logger_obj logger_nodes r/_insert_logger_after_noderC.s4 K%dii2I&IJ2N%dB/Ktyy*h & c;J  B *-**(('$"6K r1*node_to_instrument_inputs_to_ref_node_name+node_to_instrument_outputs_to_ref_node_namect}it|j}fd}|jjD]}|j dk(r'|j tt||d|:||vs||vrt||} ||vr+||\} } t|} | D]} t||| }t|tk(rV|j}t|||d|j|| | tj j"d| |  |j<yt|t$j&j(j*k(rjt-|D]Z\}}|j}t|||d|j|| | tj j"|| |  |j<\|j/|||j<||vs||\} } t|j||d|j|| | tj0j"dd|  |j<|j/|||j<"t3||}|S)z Takes the graph of gm, adds loggers to the output of each node in nodes_to_instrument. Returns a GraphModule with the new graph. c"t|fdS)Nc"|jSNr+)r envs r/z8add_loggers_to_model..load_arg..gs s499~r1r)arKs r/load_argz&add_loggers_to_model..load_argfsq566r1outputr _ns_logger_r9r:r,)rdict named_modulesr=nodesr'rPrrr0rtyperr+rCr NODE_INPUTvaluetorchfximmutable_collectionsimmutable_list enumerate node_copy NODE_OUTPUTr)r!rDrEr2r5 new_graphmodulesrOr r,r6 ref_node_typearg_indices_to_log node_arg_idxnode_arg prev_nodearg_idxargnew_gmrKs @r/add_loggers_to_modelrjUsUIC2##%&G7 77h    W%=dB%JHU V ? ? @ @ r*CAA*TUY*Z'- &Fd%K"$6L7b,OHH~-$' $6 -F%r:}dii&-4??EE-.\ # .%HMM* h588+I+I+X+XX,5h,?LGS(+CHH I2K )2z=$)) *Hm 8 C C I I18|$' 3)C /-@-%74'00x@C NBB*UVZ*['-!: NB M499-,88>>%&QC "ADII'00x@C NqtY 'F Mr1 prev_node_cnode_agm_bgraph_cscale zero_pointdtype_cast_nameclt|jdz|}t||||jd|di|}t|jdz|} t|| ||jd| di| } |jdtj ||| tj fi|S)N _input_scale_get_attr_input_zero_point_ call_function)rr+r<r>rYquantize_per_tensorquint8) rkrlrmrnrorprqscale_node_name scale_nodezero_point_node_namezero_point_nodes r/ _insert_quantize_per_tensor_noder~s +% KK/ ) ++/ 1 D/5)$$OR_>J 0% KK. . 004 6 D& 3))("b2FHO   22 j/5<<@" r1node_cgm_anode_name_prefixnode_type_to_io_type_mapc d} d} d} d} d} d}t||||\}}t||||\}}|tjk(r|tjk(sL|tjk(r|tjk(s&|tjk(r$|tj k(rt j} n||k(r.|tjk7rt jj} n|tjk(r8|tjk(r%t|||}|t j} |\} }np|tjk(r&|tjk(rd} t j} n7td|d|jd|d|jdzt!|t"rt%||}| r+| |t'||||| ||S|j)d| |fi|S| r|j)d| || fi|S| sJ| }t+||||j)d ||fi|St!|t,rg}|D]|}t%||}| r(|j)d| |fi|}|j/|>| sJ| }t+||||j)d ||fi|}|j/|~|Std t1|d ) a Given a starting graph C (derived from graph B) of ... -> prev_node_c -> node_c -> ... And a corresponding related node_a, inserts the correct dtype cast node after prev_node_c to cast into the dtype expected by node_a, resulting in: dtype_cast / ... -> prev_node_c -> node_c -> ... For example, if node_c is an int8 op and node_a is an fp32 op, this function will insert a dequant. Ntozdtype cast from  z to z needs to be implementedrw call_methodr%ztype fz is not handled)r r FP32INT8FP16 FP32_OR_INT8rY dequantizeUNKNOWNnnIdentityrrxfloat16AssertionError format_noder(rrr~r>r<listappendrV)rlrrkrrmrnrr2r dtype_cast_opdtype_cast_mod_clsdtype_cast_methoddtype_cast_method_dtypedtype_cast_scaledtype_cast_zero_pointnode_input_type_a_node_output_type_anode_input_type_c_node_output_type_cnode_a_input_qparamsnew_dtype_cast_namedtype_cast_modresultsprev_node_c_innernew_dtype_cast_nodes r/_insert_dtype_cast_after_noders}6M" , D*&> @+* - D*&> @+* 388 8 388 8 388 8 388 8 388 8 3@@ @(( ..2:::"XX..27772777 6 D2 4  +!55M6J 3 327772777 "'--0163E3E3G2H M !6#5#5#7"88P Q RS S+t$ ; )*: ;D A  +0E0Q7w8H)+>@@**#][NB'))&&056&&2[NB#% % K &!, ?-.>?E &-&9&9#]5F4H"'')#23)))!3!51>B&-&9&9!#69J8Lb'')#23!"-"vd;&7%8HIIr1c |jdk(rt|jdz|}t||j}t j |r|j}t||||j|j|di|}|S|jdk(r|jdvsJd|jd|jdk(rctt||d |||}t|jdz|}|j|j|j|fi|}|Stt||d |||}t|jdz|}|j|j|j|t||d fi|}|Std |jd |jd) z+ Simple copy of node_a to graph_c. rt _shadow_copy_rurrrztarget  is not implementedrrrzhandling of node z with op )r'rr+r r)rY is_tensordetachr<r>_copy_node_from_a_to_crrr)rlrrmrnnode_a_copy_name node_a_obj node_a_copyarg_copys r/rrAsyyJ H )&++*G H N %dFMM: ??: &#**,J& 3)) II'R1AC  m #}} 44 9fmm_$7 8 94 ==L (-(q9dG%HM-fkkO.KLTR !-- 6==8+r;KMK -(q94wPHM-fkkO.KLTR !-- 6==3FD!DE$&K  2 2 45YvyykI\ ]_ _r1 subgraph_anum_non_param_args_node_ac`g}|j}||jk7r.|j|t||d}||jk7r.|j||j d}|D]}||dur|nd}|j |d}||\} } n|j |j} } d} | t| kr3| dk(rn| dk(r|dk(rn|| | |sy| dz } | t| kr3| jD]$} | dk(rn| dk(r|dk(rn || |sy| dz } &y)z This function returns `False` if the input subgraph cannot be copied by `_insert_copy_of_subgraph_a_after_input_node_c`. This usually means that there is a corner case logic for which copy is not yet implemented. rct|tr:t||}|jdk(r|jdvS|jdk(ryyt|t t fr|D]}t|tryy)NrrrtTF)r(rr r'r)rtuple) node_a_argrarg_aels r/ _can_insertz3_can_insert_copy_of_subgraph_a.._can_insertsp j$ '2:tDExx=(||';;;Z'  T5M 2 !"d+ !r1rTnormalize_to_only_use_kwargsF) end_node start_noderrreversenormalized_argumentsargskwargslenvalues) rrrrUcur_noderrllocal_num_non_param_args_node_anorm_args_kwargs norm_args norm_kwargscur_idx kwarg_vals r/_can_insert_copy_of_subgraph_arosq E""H j++ + X+HdA> j++ + LL MMO "q!+D'( ("66 t75  '%5 "I{%+[[&--{II&!|A"AQ"F"9W#5t< qLGI&%++-I!|A"AQ"F"9d3 qLG.1F r1 input_node_cinput_node_c_2c t|tr |j}n!t|tsJ|dj}|jg}|j}||j k7r/t ||d}|jd|||j k7r/|d} t||| |||} tdt|D]} || } | } t| d| |||} | S)z* TODO(before land): real docblock rrN) r(rr=rrrrinsert)_insert_copy_of_node_a_after_input_node_cranger) rrrrrmrrn nodes_of_ar cur_node_a cur_node_c cur_idx_arks r/-_insert_copy_of_subgraph_a_after_input_node_crs,%$$,---q/''%%&J""H j++ ++HdA>!X& j++ + AJ:   J1c*o.  *  >       / r1ct|tr |jn!t|tsJ|dj|j d}||\}}n|j |j }}g} i} fd} d} | t|krB| dk(r|} n| dk(r||} n | || } | j| | dz } | t|krB|jD]-\}| dk(r|| |<n| dk(r||| |<n | | |<| dz } /t| } t|}|jdk(rqt|}t|jtsJt|j}t!||j#|j|| | |}|S|jdvsJj#|j|j| | |}|S)a Assume that node_a from graph_a has args (input, (input2)?, arg1, ...), and kwargs {kw0: kwarg0, ...} Note: input2 is optional. If it equals to None, we assume that the op has a single non-param input. If it is specified, we assume that the op has two non-param inputs. Copies the underlying values of arg1..argn and kwarg0..kwargn into gm_b, and creates the corresponding nodes in graph_c. Note: observers are ignored, so if an arg is an observer we navigate up until we find a non-observer parent. If node_a is a call_module, points the module pointed to by node_a to gm_b. Creates the copy of node_a in graph_c, with input as the first arg, and all other args and kwargs pointing to the copies of the objects in gm_b created above. An example in pictures: graph A: ======== input -------------> node_a / / / (input_2)?----------/ / / / / weight -> weight_obs / / bias ---------------- graph C (derived from B): ========================= input_node_c --> node_a_copy / / / (input_node_c_2)? / / / / weight_copy ----/ / / bias_copy ------/ rTrcFt|trt|}t|}|St|tt t jfr|Stttfr D]}t|tsJd|Stdtd)Nz/handling of Node inside list is not implementedzhandling for kwarg of type r) r(rr rintfloatrYdtyperrrrV)rhrrrmrnrs r/ _copy_argz<_insert_copy_of_node_a_after_input_node_c.._copy_arg-s c4 0d;C(dD'BCJ c5%++6 7J  D%= 1%b$/FEF/ J -d9o->>QRT Tr1rr%)rwr)r(rr=rrrrrritemsrrr'r)r*r r<r>)rrrlrrmrrrrnew_args new_kwargsrrnew_arg kwarg_namenode_a_shadows_c_namenew_mod_copy_namemod_anode_a_shadows_crnrs `` @@r/rrs)f,%$$,---q/''22 431#!1 ;!'fmm; HJT"G C N " a<"G \n8$G ' 23G 1  C N ""-!2!2!4 I a<%1Jz " \n8%3Jz "%.y%9Jz "1 "5XH 8%&67=yyM! < )*: ;D A &----- v}}5'/".. II(( -/ yy<<<<".. IIv}}h -/ r1name_aname_bmatched_subgraph_pairsshould_log_inputsc.4| t}t}i4t|j} 4fd} i} i} |j D]\\} }|\}}t |j |}t |j |}|| ||f| |j<|| ||f| |j<^|jjD]:}|jdk(r)|jt|jd| <|| v}|| v}|s|r|r | |\}}}}n|sJ| |\}}}}t|jxr t|}|sUt!dt ||dt |j|zdz|j#|| 4|j$<t'|j|||\}}t'||||\}}|t(j*k7xr=|t(j*k7xr(|t(j*k7xr|t(j*k7}|sVt!dt ||dt |j|zdz|j#|| 4|j$<|t(j,k(r|t(j.k(rot1|j||}|sVt!dt ||dt |j|zd z|j#|| 4|j$<Ht3|j|}t5|||sVt!dt ||dt |j|zd z|j#|| 4|j$<t7|j |}t7j |} |r|rt9||d}!t;|!t<rV4|!j$}"t?|"||d |j$|||t@jBjDdd| 4|"j$<nt;|!tFr{|!D#cgc]}#4|#j$}$}#tI|!D]P\}%}#|$|%}"t?|"||d |j$|||t@jBjD|%d| 4|"j$<RntKd tM|!d|s|r.|j#|| 4|j$<4|j$}&|rt9&|d}"|rHt;|"t<rt9|&|d}"n*t;|"tFr|"D#cgc]}#t9|#|d}"}#tO|j|&|"||||j$dz|| }'|rd}(t;|'t<r2t?|'||d|(|||t@jBjDdd| }'|'})nkt;|'tFsJg}*tI|'D]E\}+},t?|,||d|(|||t@jBjD|+d| }-|*jQ|-G|*}'|'})d}.t3|j|}|dk(r t9|&|d}.tS|'|.||||&j$dz}/|/4|/j$<|r|/}0t9|0|d)k7rt9|0|d}0t9|0|d|)k7rt;|)t<r(tU||)j$}1|0j$|1_+n@t;|)tFsJ|)D])}2tU||2j$}1|0j$|1_++t?4|/j$||d|/j$|||t@jXjDdd| 4|/j$<|st?4|j$||d|j$|||t@jXjDdd| 4|j$<|j#|| 4|j$<=t[||}3|3Scc}#wcc}#w)a Creates a new GraphModule consisting of the graph of C, with the meaningful nodes of A shadowing the corresponding nodes of B. For example, Graph A: a0 -> op0_fp32 -> a1 -> op1_fp32 -> a2 Graph B: b0 -> op0_int8 -> b1 -> op1_int8 -> b2 matched_node_pairs: {'op0': (op0_fp32, op0_int8), 'op1': (op1_fp32, op1_int8)} Graph C (A shadows B): / dequant0 -> op0_fp32 -> logger_a_0 / dequant_1 -> op1_fp32 -> logger_a_1 / / b0 -------------> op0_int8 -> logger_b_0 --------------> op1_int8 -> logger_b_1 In a nutshell, this function does the following for each node pair: * copies the necessary attributes and modules from gm_a to gm_b, keeping names unique * adds a dtype cast op (dequant, quant, etc) * adds a copy of node_a in gm_b's graph * adds loggers to the outputs of node_a and node_b Nc"t|fdS)Nc"|jSrIrJ)r env_cs r/rLz6create_a_shadows_b..load_arg..suTYY'7r1rM)rNrs r/rOz$create_a_shadows_b..load_argsq788r1rPrz$skipping shadow loggers for node_b: z, start_node_a: z , unsupportedz, unknown dtype castz, unknown input qparamsz", unhandled logic in subgraph copy_ns_logger_b_inp_rRztype z is not handled yet _dtype_cast__ns_logger_a_inp_rrr _ns_logger_a_ _ns_logger_b_).rrrSrTrr base_op_noderrr=rUr'rPrrrprintr^r+r r rrrrr rr0rr(rrCrrWrXrr]rrVrrrgetattrr4r_r)5rrrrmrr2rrrnrarO+start_node_b_to_matched_subgraph_a_and_name)end_node_b_to_matched_subgraph_a_and_name match_namematchr subgraph_bref_node_type_aref_node_type_bnode_bnode_b_is_start_nodenode_b_is_end_noder6all_op_types_support_shadowingrnode_output_type_anode_input_type_bnode_output_type_bnode_io_types_known_a_and_brr fqn_base_a fqn_base_b prev_node_brkrhprev_node_c_listrgrdtype_cast_noder4 input_logger new_loggersdtype_cast_idxdtype_cast_node_innerdtype_cast_loggernode_c_second_non_param_argrrinput_logger_modinput_logger_innergm_crs5 @r/create_a_shadows_brlsH '#?#A gGE4%%'(G935/02-399; E!& J-j.E.EtL-j.E.EtL _o F 4J4I4IJ_o F 2*2E2EF <**"" 99 NN76;;q>8< =  &)TT#'PP $6#?GG Ho*))=fEG Ho+:+@+@A3*62 +2:;NvW[;\:]^&'::;P;PRV'W&XYZ#$%&-%6%6vx%Hfkk" 5))4,. 2 1 5D*,. 2 1 "%:%B%BBD"&;&C&CCD!%:%B%BBD#&;&C&CC ( /:;NvW[;\:]^&'::;P;PRV'W&XYZ*+,&-%6%6vx%Hfkk" "%:%?%??!%:%?%??'=))41I(K$+>?RSY[_?`>ab*+>z?T?TVZ+[*\]^123*1):):68)LE&++&-Z-B-BDI &1*dD]^:;NvW[;\:]^&'::;P;PRV'W&XYZ89:&-%6%6vx%Hfkk"' (?(?FJ' (?(?FJ#%":64"KK!+t4&+K,<,<&= 2K'z;N"KK?4??EE-.Q * 3,k../ $K6HS+S{E#((O{(+S,5k,BLGS*:7*CK6O +T:?R & VX 8 C C I I18q$. 70E+"2"23-C-uT+5F4GGZ-[\\$'9%,%6%6vx%Hfkk"v{{+$ 7vtQG $!+t4&>vtQ&O #K6Yd&eYdRU'?T1'MYd &e"?))6;dGKK.0*,#.%$&M!/48*C+T:?R)68_4??EE-.Q * +, AP )/4@@@&( ENE_AN,A0I 5tZI\ -vx 8 C C I I1?-.$. 10-(../@AF`+6'6 /3+,HI^I^`d,e),12J6SWYZ2[/#P#%@dFKK/,I$K 0@&++,% 0H28T1EU#;HdA#N38T1EU!,5+249J9J+K(9A(6),===2>./6t=O=T=T/U,=E]],:3? 0I*//0$ O$))68_,88>>%&Q" 0$&++,"&?&++&j/KK?,88>>%&Q" &$fkk" ")!2!268!DE&++ Y#\ tW %D KS,TZ'fs ;` `rI)4rYtorch.fxrrtorch.fx.graphrrtorch.ao.quantization.fx.utilsrutilsr r r r r rrrrrns_typesrrrtorch.ao.ns.fx.mappingsrtorch.ao.quantization.observerrtypingrrrrrrrrr*r0rrCrjTensorrr~rrboolrrrrrur1r/rs: )&H    GIII  ; 8C= % %%%! %  %  %%%%%% #% %NOO04T5c?5J0KO26dE#s(O6K1LO O  O  Ob    u$ %  ellC'(  :{J {J {JtT$Z'({J  {J  {J  {J{J{J#3,<(=#=>{J 4d {J|,_ ,_ ,_ ,_ ,_  ,_\GG G #G GR/d4j()/U4d#345//  /  /  / /d@ d4j()@ U4d#345@  @   @  @  @  @ TLPJ J J J  J !eJ ,B&C!CD J  JJ'tC5E1F,F'GHJJr1