Ph3ddlmZddlZddlmcmZddlmZGddeZGddejZ de fd Z Gd d ejZ Gd d ejZGddejZGddejZGddejZGddejZGddejZGddejZGddejZGddejZGddejZGd d!ejZGd"d#ejZy)$)BaseSparsifierN)nnc$eZdZfdZdZxZS)ImplementedSparsifierc &t||y)N)defaults)super__init__)selfkwargs __class__s qC:\Users\daisl\Desktop\realtime-object-detection\venv\Lib\site-packages\torch/testing/_internal/common_pruning.pyr zImplementedSparsifier.__init__ s &)c d|jjdjd<|jd}|j dddz|d<y)Nrzlinear1.weight step_count)parametrizationsweightmaskstateget)r moduler linear_states r update_maskz!ImplementedSparsifier.update_mask sN45&&q)..q1zz"23 %1%5%5lA%F%J \"r)__name__ __module__ __qualname__r r __classcell__r s@rrrs*Krrc eZdZdZedZy)MockSparseLinearz This class is a MockSparseLinear class to check convert functionality. It is the same as a normal Linear layer, except with a different type, as well as an additional from_dense method. c@||j|j}|S)z ) in_features out_features)clsmodlinears r from_densezMockSparseLinear.from_denses"S__%%' rN)rrr__doc__ classmethodr(rrr!r!s rr!returncd}|D]@}|t|kr/tj|||s|dz }n1|t|kr/yy)zW Checks to see if all rows in subset tensor are present in the superset tensor rrFT)lentorchequal) subset_tensorsuperset_tensorirows rrows_are_subsetr5!sX A#o&&;;sOA$67Q #o&&  rc(eZdZdZfdZdZxZS) SimpleLinearzModel with only Linear layers without biases, some wrapped in a Sequential, some following the Sequential. Used to test basic pruned Linear-Linear fusion.c Rt|tjtjdddtjdddtjddd|_tjddd|_tjddd|_y)NFbias )r r r SequentialLinearseqlinear1linear2r r s rr zSimpleLinear.__init__5sx == IIa ' IIa ' IIa '  yyAE2 yyBU3 rcl|j|}|j|}|j|}|SN)rBrCrDr xs rforwardzSimpleLinear.forward?. HHQK LLO LLOrrrrr)r rJrrs@rr7r71sV4rr7c(eZdZdZfdZdZxZS) LinearBiaszModel with only Linear layers, alternating layers with biases, wrapped in a Sequential. Used to test pruned Linear-Bias-Linear fusion.c :t|tjtjdddtjdddtjdddtjdddtjddd|_y) Nr9r:Tr;r=Fr?)r r rr@rArBrEs rr zLinearBias.__init__Jsn == IIa & IIa ' IIa & IIa & IIa% (  rc(|j|}|SrG)rBrHs rrJzLinearBias.forwardTs HHQKrrLrs@rrNrNFsO rrNc(eZdZdZfdZdZxZS)LinearActivationzModel with only Linear layers, some with bias, some in a Sequential and some following. Activation functions modules in between each Linear in the Sequential, and each outside layer. Used to test pruned Linear(Bias)-Activation-Linear fusion.c t|tjtjdddtj tjdddtj tjddd|_tjddd|_tj |_ tjdd d|_ tj |_ y) Nr9r:Tr;r=Fr>rPr?) r r rr@rAReLUTanhrBrCact1rDact2rEs rr zLinearActivation.__init__^s == IIa & GGI IIa ' GGI IIa &  yyAD1 GGI yyBU3 GGI rc|j|}|j|}|j|}|j|}|j |}|SrG)rBrCrWrDrXrHs rrJzLinearActivation.forwardlH HHQK LLO IIaL LLO IIaLrrLrs@rrSrSYsB rrSc(eZdZdZfdZdZxZS)LinearActivationFunctionala,Model with only Linear layers, some with bias, some in a Sequential and some following. Activation functions modules in between each Linear in the Sequential, and functional activationals are called in between each outside layer. Used to test pruned Linear(Bias)-Activation-Linear fusion.c  t|tjtjdddtj tjdddtj tjddd|_tjddd|_tjdd d|_tjd d d|_ tj |_ y) Nr9r:Tr;r=Fr>rPr?) r r rr@rArUrBrCrDlinear3rWrEs rr z#LinearActivationFunctional.__init__{s == IIa & GGI IIa ' GGI IIa &  yyAD1 yyAE2 yyBU3 GGI rc |j|}|j|}tj|}|j |}tj|}|j |}tj|}|SrG)rBrCFrelurDr_rHs rrJz"LinearActivationFunctional.forwardsb HHQK LLO FF1I LLO FF1I LLO FF1IrrLrs@rr\r\usB rr\c(eZdZdZfdZdZxZS) SimpleConv2dzModel with only Conv2d layers, all without bias, some in a Sequential and some following. Used to test pruned Conv2d-Conv2d fusion.c 4t|tjtjdddddtjddddd|_tjddddd|_tjddddd|_y) Nr rPFr;@04r r rr@Conv2drBconv2d1conv2d2rEs rr zSimpleConv2d.__init__sx == IIaQ . IIb"a / yyRAE: yyRAE: rcl|j|}|j|}|j|}|SrGrBrlrmrHs rrJzSimpleConv2d.forwardrKrrLrs@rrdrds1;rrdc(eZdZdZfdZdZxZS) Conv2dBiaszModel with only Conv2d layers, some with bias, some in a Sequential and some outside. Used to test pruned Conv2d-Bias-Conv2d fusion.c ft|tjtjdddddtjdddddtjddddd|_tjddddd|_tjdd ddd|_y NrrfrPTr;rgFrhrirjrEs rr zConv2dBias.__init__s == IIaQ - IIb"a . IIb"a /  yyRAD9 yyRAE: rcl|j|}|j|}|j|}|SrGrorHs rrJzConv2dBias.forwardrKrrLrs@rrqrqs6;rrqc(eZdZdZfdZdZxZS)Conv2dActivationaModel with only Conv2d layers, some with bias, some in a Sequential and some following. Activation function modules in between each Sequential layer, functional activations called in-between each outside layer. Used to test pruned Conv2d-Bias-Activation-Conv2d fusion.c t|tjtjdddddtj tjdddddtj tjdddddtj |_tjddddd|_tjdd ddd|_ yrs) r r rr@rkrUrVrBrlrmrEs rr zConv2dActivation.__init__s == IIaQ - GGI IIb"a . GGI IIb"a / GGI  yyRAE: yyRAD9 rc|j|}|j|}tj|}|j |}tj |}|SrG)rBrlrarbrmhardtanhrHs rrJzConv2dActivation.forwardsH HHQK LLO FF1I LLO JJqMrrLrs@rrvrvsA :rrvc(eZdZdZfdZdZxZS) Conv2dPadBiasaQModel with only Conv2d layers, all with bias and some with padding > 0, some in a Sequential and some following. Activation function modules in between each layer. Used to test that bias is propagated correctly in the special case of pruned Conv2d-Bias-(Activation)Conv2d fusion, when the second Conv2d layer has padding > 0.ct|tjtjddddddtj tjdddddtj tjddddddtj tjddddddtj tjdddddtj |_tjdd dddd|_tj |_ tjd d dddd|_ tj |_ y) NrrfrPT)paddingr<Fr;rgrhri) r r rr@rkrUrVrBrlrWrmrXrEs rr zConv2dPadBias.__init__s == IIaQ14 8 GGI IIb"a / GGI IIb"aAD 9 GGI IIb"aAD 9 GGI IIb"a . GGI  yyRAqtD GGI yyRAqtD GGI rc|j|}|j|}|j|}|j|}|j |}|SrG)rBrlrWrmrXrHs rrJzConv2dPadBias.forwardrZrrLrs@rr{r{sc &rr{c(eZdZdZfdZdZxZS) Conv2dPoolzModel with only Conv2d layers, all with bias, some in a Sequential and some following. Activation function modules in between each layer, Pool2d modules in between each layer. Used to test pruned Conv2d-Pool2d-Conv2d fusion.c t|tjtjdddddtj dddtj tjdddddtjtjddd|_ tjdd ddd|_ tj ddd|_ tj |_ tjd d ddd|_ tjd d ddd|_y) NrrfrPT kernel_sizer}r<rstrider}rgrhri)r r rr@rk MaxPool2drUrV AvgPool2drBrlmaxpoolaf1rmconv2d3rEs rr zConv2dPool.__init__s == IIaAD A LLQq! < GGI IIb"!QT B GGI LLQq! <  yyRQM ||!QG 779yyRQM yyRQM rc.|j|}|j|}|j|}|j|}|j |}t j |ddd}t j|}|j|}|SNrrr) rBrlrrrmra avg_pool2drbrrHs rrJzConv2dPool.forwardsv HHQK LLO LLO HHQK LLO LL!Q ? FF1I LLOrrLrs@rrrs8N rrc(eZdZdZfdZdZxZS)Conv2dPoolFlattenFunctionalaModel with Conv2d layers, all with bias, some in a Sequential and some following, and then a Pool2d and a functional Flatten followed by a Linear layer. Activation functions and Pool2ds in between each layer also. Used to test pruned Conv2d-Pool2d-Flatten-Linear fusion.c |t|tjtjdddddtj dddtj tjdddddtjtjddd|_ tjddddd|_ tj |_ tjdd ddd|_ tjd |_tjd d d |_y) NrrPTrrrr:r9 )rr r;)r r rr@rkrrUrVrrBrlrrmAdaptiveAvgPool2davg_poolrAfcrEs rr z$Conv2dPoolFlattenFunctional.__init__"s == IIa14 @ LLQq! < GGI IIa14 @ GGI LLQq! <  yyA1adK 779yyBAqtL ,,V4 ))B.rc0|j|}|j|}tj|ddd}|j |}|j |}|j |}tj|d}|j|}|Sr) rBrlra max_pool2drrmrr/flattenrrHs rrJz#Conv2dPoolFlattenFunctional.forward2sz HHQK LLO LL!Q ? HHQK LLO MM!  MM!Q  GGAJrrLrs@rrrs@ / rrc(eZdZdZfdZdZxZS)Conv2dPoolFlattenaModel with Conv2d layers, all with bias, some in a Sequential and some following, and then a Pool2d and a Flatten module followed by a Linear layer. Activation functions and Pool2ds in between each layer also. Used to test pruned Conv2d-Pool2d-Flatten-Linear fusion.c t|tjtjdddddtj dddtj tjdddddtjtjddd|_ tjddddd|_ tj |_ tjdd ddd|_ tjd |_tj|_tj"d d d |_y)NrrPTrrrr:r9r)rr,rr;)r r rr@rkrrUrVrrBrlrrmrrFlattenrrArrEs rr zConv2dPoolFlatten.__init__Ds == IIa14 @ LLQq! < GGI IIa14 @ GGI LLQq! <  yyA1adK 779yyBAqtL ,,V4 zz| ))B.rc&|j|}|j|}tj|ddd}|j |}|j |}|j |}|j|}|j|}|Sr) rBrlrarrrmrrrrHs rrJzConv2dPoolFlatten.forwardUsw HHQK LLO LL!Q ? HHQK LLO MM!  LLO GGAJrrLrs@rrr>s@ /" rrc:eZdZdZdedededeffd ZdZxZS)LSTMLinearModelzCContainer module with an encoder, a recurrent module, and a linear. input_dim hidden_dim output_dim num_layersct|tj||||_tj |||_yrG)r r rLSTMlstmrAr'r rrrrr s rr zLSTMLinearModel.__init__ds6 GGIz:> ii J7 rcT|j|\}}|j|}||fSrG)rr')r inputoutputhiddendecodeds rrJzLSTMLinearModel.forwardks,5)++f%rrrrr)intr rJrrs@rrras0M88*-8;>8LO8rrc:eZdZdZdedededeffd ZdZxZS)LSTMLayerNormLinearModelz9Container module with an LSTM, a LayerNorm, and a linear.rrrrct|tj||||_tj ||_tj|||_yrG) r r rrr LayerNormnormrAr'rs rr z!LSTMLayerNormLinearModel.__init__tsF GGIz:> LL, ii J7 rcv|j|\}}|j|}|j|}||fSrG)rrr')r rIrs rrJz LSTMLayerNormLinearModel.forward|s699Q<5 IIaL KKN%xrrrs@rrrqs0C88*-8;>8LO8rr)torch.ao.pruningrr/torch.nn.functionalr functionalrarrAr!boolr5Moduler7rNrSr\rdrqrvr{rrrrrr+rrrs, KNK ryy  t  299*&ryy8>299(*ryy8BIIDB"))D  F bii  ryyr