FPhaddlmZmZmZmZddlZddlZddlm Z ddl m cm Z ddl mZmZGdde j ZGdde j ZGdd e j ZGd d e j ZGd d e j Zdej,dedeej,eeefffdZdej,dedeeefdeeefdej,f dZdededej,dej,fdZdej,dej,dej,dej,deeefdeeefdej,fdZGdd e j Zy)!)AnyOptionalTupleTypeN) LayerNorm2dMLPBlockc#(eZdZdZddddddddd ej ej d d d d d fd edededededededede de ejde ejde de de dede edfddf"fd Z d!ejdejfd"ZxZS)#ImageEncoderViTa An image encoder using Vision Transformer (ViT) architecture for encoding an image into a compact latent space. The encoder takes an image, splits it into patches, and processes these patches through a series of transformer blocks. The encoded patches are then processed through a neck to generate the final encoded representation. Attributes: img_size (int): Dimension of input images, assumed to be square. patch_embed (PatchEmbed): Module for patch embedding. pos_embed (nn.Parameter, optional): Absolute positional embedding for patches. blocks (nn.ModuleList): List of transformer blocks for processing patch embeddings. neck (nn.Sequential): Neck module to further process the output. i @TFrimg_size patch_sizein_chans embed_dimdepth num_heads mlp_ratio out_chansqkv_bias norm_layer act_layer use_abs_pos use_rel_posrel_pos_zero_init window_sizeglobal_attn_indexes.returnNcht|||_t||f||f|||_d|_| r6t jtjd||z||z||_t j|_ t|D]@}t|||| | | | |||vr|nd||z||zf }|jj|Bt jt j ||ddt#|t j ||ddd t#||_y) a Args: img_size (int): Input image size. patch_size (int): Patch size. in_chans (int): Number of input image channels. embed_dim (int): Patch embedding dimension. depth (int): Depth of ViT. num_heads (int): Number of attention heads in each ViT block. mlp_ratio (float): Ratio of mlp hidden dim to embedding dim. qkv_bias (bool): If True, add a learnable bias to query, key, value. norm_layer (nn.Module): Normalization layer. act_layer (nn.Module): Activation layer. use_abs_pos (bool): If True, use absolute positional embeddings. use_rel_pos (bool): If True, add relative positional embeddings to the attention map. rel_pos_zero_init (bool): If True, zero initialize relative positional parameters. window_size (int): Window size for window attention blocks. global_attn_indexes (list): Indexes for blocks using global attention. ) kernel_sizestriderrNr) dimrrrrrrrr input_sizeF)r$biasr )r$paddingr))super__init__r PatchEmbed patch_embed pos_embednn Parametertorchzeros ModuleListblocksrangeBlockappend SequentialConv2drneck)selfrrrrrrrrrrrrrrr r!iblock __class__s rC:\Users\daisl\Desktop\realtime-object-detection\venv\Lib\site-packages\ultralytics/models/sam/modules/encoders.pyr,zImageEncoderViT.__init__s<J   %#Z0 +  26 \\%++aZ9OQY]gQgir*stDNmmo uA##!%#'"3+,4G+GKQ$ 2H 4JK E KK  u %MM II    " II    "  xc|j|}|j||jz}|jD] }||} |j|j ddddS)zProcesses input through patch embedding, applies positional embedding if present, and passes through blocks and neck. rr r&)r.r/r5r;permute)r<rBblks r@forwardzImageEncoderViT.forwardrsa   Q  >> %DNN"A;;CAAyy1aA.//rA)__name__ __module__ __qualname____doc__r0 LayerNormGELUintfloatboolrModulerr,r2TensorrG __classcell__r?s@r@r r s5 !  " !*,,,)+ $ %&* 35#T T T  T  T  T T T T T RYYT BIIT T T  $T !T ""'sCx#T $ %T l 0 0%,, 0rAr c eZdZdZej fdedeeefdeeefdedeejddf fd Z de jfd Z d e jd e jd ede jfdZde jde jfdZde jde jfdZd eee je jfdee jdee jdefdZde j(fdZd eee je jfdee jdee jdee je jffdZxZS) PromptEncodera Encodes different types of prompts, including points, boxes, and masks, for input to SAM's mask decoder. The encoder produces both sparse and dense embeddings for the input prompts. Attributes: embed_dim (int): Dimension of the embeddings. input_image_size (Tuple[int, int]): Size of the input image as (H, W). image_embedding_size (Tuple[int, int]): Spatial size of the image embedding as (H, W). pe_layer (PositionEmbeddingRandom): Module for random position embedding. num_point_embeddings (int): Number of point embeddings for different types of points. point_embeddings (nn.ModuleList): List of point embeddings. not_a_point_embed (nn.Embedding): Embedding for points that are not a part of any label. mask_input_size (Tuple[int, int]): Size of the input mask. mask_downscaling (nn.Sequential): Neural network for downscaling the mask. no_mask_embed (nn.Embedding): Embedding for cases where no mask is provided. rimage_embedding_sizeinput_image_size mask_in_chans activationr"Nc t|||_||_||_t |dz|_d|_t|jDcgc]}tjd|}}tj||_ tjd||_ d|dzd|dzf|_tjtj d|dzddt#|dz|tj |dz|ddt#||tj ||d|_tjd||_ycc}w)a; Encodes prompts for input to SAM's mask decoder. Args: embed_dim (int): The prompts' embedding dimension image_embedding_size (tuple(int, int)): The spatial size of the image embedding, as (H, W). input_image_size (int): The padded size of the image as input to the image encoder, as (H, W). mask_in_chans (int): The number of hidden channels used for encoding input masks. activation (nn.Module): The activation to use when encoding input masks. rDr&r)r$r%)r$N)r+r,rrXrWPositionEmbeddingRandompe_layernum_point_embeddingsr6r0 Embeddingr4point_embeddingsnot_a_point_embedmask_input_sizer9r:rmask_downscaling no_mask_embed) r<rrWrXrYrZ_rar?s r@r,zPromptEncoder.__init__s>, " 0$8!/ Q? )*!@EdF_F_@`a@`1BLLI6@`a " .> ?!#a!; !$8$; ;QAUVWAX=XY " IIa!+1 E  * + L IImq(-Qq Q  & L IImYA >!  \\!Y7bsE$cV|j|jjdS)a  Returns the positional encoding used to encode point prompts, applied to a dense set of points the shape of the image encoding. Returns: torch.Tensor: Positional encoding with shape 1x(embed_dim)x(embedding_h)x(embedding_w) r)r^rW unsqueezer<s r@ get_dense_pezPromptEncoder.get_dense_pes$}}T667AA!DDrApointslabelspadc||dz}|rtj|jdddf|j}tj|jddf|j }tj ||gd}tj ||gd}|j j||j}d||dk(<||dk(xx|jjz cc<||dk(xx|jdjz cc<||dk(xx|jdjz cc<|S) zEmbeds point prompts.?rr&rDdevicer') r2r3shaperqonescatr^forward_with_coordsrXrbweightra)r<rkrlrm padding_point padding_labelpoint_embeddings r@ _embed_pointszPromptEncoder._embed_pointss# !KKa!Q(? VM"ZZa!(FYY 6A>F--;;FDDYDYZ(+" %" %)?)?)F)FF%! $(=(=a(@(G(GG$! $(=(=a(@(G(GG$rAboxesc.|dz}|jddd}|jj||j}|dddddfxx|jdj z cc<|dddddfxx|jdj z cc<|S)zEmbeds box prompts.rortrDNrr&r )reshaper^rxrXrary)r<r~coordscorner_embeddings r@ _embed_boxeszPromptEncoder._embed_boxess r1a(==< !  ;;q> !rAcH|jdjjS)z@Returns the device of the first point embedding's weight tensor.r)raryrqris r@ _get_devicezPromptEncoder._get_devices$$Q'..555rAc.|j|||}tj|d|jf|j }|4|\}}|j |||du}tj ||gd}|*|j|} tj || gd}||j|} || fS|jjjddddj|d|jd|jd} || fS)a` Embeds different types of prompts, returning both sparse and dense embeddings. Args: points (tuple(torch.Tensor, torch.Tensor), None): point coordinates and labels to embed. boxes (torch.Tensor, None): boxes to embed masks (torch.Tensor, None): masks to embed Returns: torch.Tensor: sparse embeddings for the points and boxes, with shape BxNx(embed_dim), where N is determined by the number of input points and boxes. torch.Tensor: dense embeddings for the masks, in the shape Bx(embed_dim)x(embed_H)x(embed_W) rrpN)rmr&rrrt)rr2emptyrrr}rwrrreryrexpandrW) r<rkr~rbssparse_embeddingsrrlrabox_embeddingsdense_embeddingss r@rGzPromptEncoder.forwards<& ! !&% 7!KKQ(?HXHXHZ[  #NFF#11&&uPT}1V  % +<>N*OUV W   !..u5N % +?RHHrAsizecf|\}}|jj}tj||f|tj}|j ddz }|j ddz }||z }||z }|j tj||gd}|jdddS)z>Generate positional encoding for a grid of the specified size.)rqdtyperrrror&rtrD) rrqr2rvfloat32cumsumrstackrE) r<rhwrqgridy_embedx_embedpes r@rGzPositionEmbeddingRandom.forward:s1>>EEzz1a&u}}E++!+$s*++!+$s*A+A+   u{{GW+=2F Gzz!Q""rA coords_input image_sizec|j}|dddddf|dz |dddddf<|dddddf|dz |dddddf<|j|jtjS)z