From ed7a1b2ba80cf46c04dc359e7aef3afd8f66fd5f Mon Sep 17 00:00:00 2001 From: pytorchbot Date: Wed, 19 Apr 2023 13:34:08 +0000 Subject: [PATCH] Generate Python docs from pytorch/pytorch@a470c041b0f9f01f8ab62ec80bb0b47466fe8d34 --- docs/2.0/_images/RReLU.png | Bin 34208 -> 33856 bytes .../torch/cuda/amp/autocast_mode.html | 2 +- .../torch/distributed/distributed_c10d.html | 41 +++-- docs/2.0/_modules/torch/nn/functional.html | 40 ++++- .../_modules/torch/nn/modules/activation.html | 37 ++-- .../torch/nn/modules/transformer.html | 8 +- docs/2.0/_modules/torch/optim/adam.html | 10 +- docs/2.0/_modules/torch/optim/adamw.html | 9 +- docs/2.0/_modules/torch/serialization.html | 2 +- docs/2.0/_modules/torch/storage.html | 37 +++- .../torch/utils/data/_utils/collate.html | 3 +- .../_modules/torch/utils/data/dataloader.html | 17 +- .../2.0/_sources/notes/extending.func.rst.txt | 9 +- docs/2.0/distributed.html | 4 +- .../torch.nn.MultiheadAttention.html | 14 +- docs/2.0/notes/extending.func.html | 9 +- .../quantization-backend-configuration.html | 168 +++++++++--------- docs/2.0/searchindex.js | 2 +- docs/2.0/torch.html | 10 +- 19 files changed, 260 insertions(+), 162 deletions(-) diff --git a/docs/2.0/_images/RReLU.png b/docs/2.0/_images/RReLU.png index 75fa1b8507010c3e3bf9c3740ab2a6f87371b3a0..04f0001d9bc7375e90cad50c82d3795d3d7f308b 100644 GIT binary patch literal 33856 zcmd?RWmr{f)HS>)MNmQzkrYq?X+b&^2^9np>29REO94SrL_tXf1qA`=?ow%y?(P!l zdgs#PiSK%zKi~EKd%cd=(Jg!LweEY~bB;0QnCr=1Iq6e)6nF@NoVs;W;vRxv`XC6# zDDE+MYdnK~mL0(c0L-S8T46G_}1yjxd3SjZFIEPKbY)4DI^g;8+9zAQ^iPEOvCb%B}jgN=-j z%~$f9Wv^dXu)Q83d0mpokbF8}gto4Z)1Ju4X1Tdcr)^^W^HgiIz>0goPIK|rU^*@x zb|C!vY%m))gwVsUk{ng!G%+!;D><2u1pK0TjB_6T8WEs}aTESJsrUcDKfir&?%cW6 zjg9)1$|EZ^UZbz1d8QpTuA%hwNfg|#WCKG&>SC=c<`z@#USvgvs*cloAd(8KM`dMY z?H|>+WnW#vdG+cQ?*bt?XO)?!n3z~yW23>BXBY1(ZRGTG!lz+lV{7JxdLJDW4vmCU zT>WaLz~_DF@pG|{q}+Y0;-gbBmXgxicrXXOpddv>MMYqEcw?GUy4la(#YKze1QANC z;{+1Ay5~tHt5fd8T6D9pvo}y$mpy!QBY>Wj_2k2c4`&C9xrf~r*j~06r#p6bc4llQ zemxT1TjZ8@=$kM!HrB84J7L=X8spKUM>Yo=JrkcJ7_Pc4KElDb=~7amA)rWAO3lTit%nJ|q;|Clbd^ckIc~2+*s2OeZW%B`PWk z;Z#**JMr|{GyU~lLXn?DB@T0s9-pF2(k^#1?a8E0aGgFEC+0nDRt|TozrVc}_~gm; za<>hl)2APK9qf*IAMGW!*jH=~yU&hPipB6ez@(+6m5`P;nE(Faro4Qi`bv#Ie$hmj zQ2U!3$G>Ok&|+X<%zpnsqV2wP^`-2kbVokx;gF7GX@)CTPQ!he_UAEYmT#)3b3`}i zSzX!K+-#^N6bqS2^rpFZ@glD!!_2Ck6V_+j>9)l1*;&))&!3OBi5=C!rut-P6w+V3 zhyx#@ydWhZp*vRNkNA|834hcqK8@a(oxT0q{(75G@(wTh$FTs~arkHlC&jq=FN}$G z_#9({r}8fk4uJh#T+lJZJVUAhzFN6a9l%2v8NpUzR4Of$c~vrf)$ zbG|iRD1b4+1#@_K*kz+jrK8f@`@+SGZxoWHx_V)HhrdHNT80B;ai)>AY%LHUEk?>md`Yni#^OrBxdn=XUG$L-g0|l0__>o3K)Iv_Ltw$=Rz@>Rmve+o^!4eJ zl9RdIewtz*KRyBf=CV@0wF+<1-&!8RB_uq4SNr4QQ0Z9)sgP^IWNfcH=T;kOy+_{! z(9~=$4P@I*-EOj{5q2RGj3?L{v?q+=wG7HNX&aA}7rkETb+9_*JTlgAo@JEc_;T+1 z2SqOTO$&54k9OL;(}q&#@Q5dWD1TI?LjvJ;RN1`u*BWt1f{tm(`ETF8u{MR#h+IYjqqW`UuDDNWwP5vK(@b8yks;kJnmqla!Q9 zx)pgzzxR{gSx-4*^*veiqjxu#+&g-bSbPI~hE(CcFpvO3+KSl{I6o>0-^W@x_LveuK79D#M5}WDz8~yGu;@|!{W$u& z8#dq3Tuh(=NgtsWbf|(8*^#covAwfH2U*;@@}OX-F;lyoLSA?c8-fxK&RqzNNdNL- zspp=Pru$OZ{s13Zy)y z?NvF8J2Q!mo$_L#CrB=h!cmp5vSN>7*9wAcO~=L6VWuU0<&`WP?&Hho>SeO{RiS5k zA**scOlbLW#+|~cd0e_x$R;;ZJN25P z*vLiPco1FqL;IXV)H*?Ma4?$8pk|Fj!jf8_Zh?&GtqYO;xwW<7*+rR;{ki(}cxcv! z{P8F}Jp3Kf)YO!&FoK1JH8wGU^?LtPXl12Xf4;d%z8qvM*2*kPjC=8Maa70@j=mCn zU~O`^D4wK3d%n4f@X`J{xuAn-eEz!}$Qt@}fy6I4^)S)23u{aTX_MpRG2fQ4cWCtp zN*aZ0Y5bs=`pNLi>67QKVBflRYju4c3)%nC#6d3V!C&IE6jW?KBMx^~A9RMC$CRkR zqEG)zY0dW91TiILFb+PMYK0+rs1zv~S@-*cGp=cAK_c#(y&@O7lcfpwe|E}4x`aHV zo8QLhS2fp}s-Ud=I4_R}#Q<*QG+4(@ep1_lqsesna?Q^A%-Vo;CHh50P#oCQ?nfmi z5=MUUgF8io9tQ`9T-fDPi<^;=(dV`{=?4#(@7=q%`dv|;j+3+f65s26nOcARvo_0v z0v_uve5#dRB331Hmr4%=J$HGUjk&{U#lqk_uzZGkIM~=EuV2T2Jm^z-bVwO|mP6|P zeNy#&GoxH9c>T7FOhj~aK)C2GE;BQ;%lH$O0fd*Nl~v~F&!36VWQBX`)EU4L^b8EOW(RN_f(tAMf(8dQ0gA9}u8eTd zc>YwtxE4Ys@GVE%&8-A7-8g(gp__-DUC!!{A35sFvp1>&2&hjZb>H5}SaxT);ci3J z!C&p$L`?FB0i5_HF^vO!5XYr+TI_{v4_Qtdm#!{B#2p*q^4fRl$Tg8~nve>oVH0#- zw%FO}Gf6yicz8HITJ5XcCI&_0P0vF8+C-y2(Gd;BDLlN>sdDi-5BPj24m7g${gv3c zxVU)RlT;moAC?7^c5T}|5QC9h83{lelQ#T8XkL0`-FYhXKTo1loctC6RBTF2XA2|;Y$a>wwx@_$+avm1iyJJ5mH#Z|ejGn$``g=GY{#o`s zb?4R5>L|~mG+xVo)oy%TTq1dqjpv-PExZd*JzC}k2J#*-47Dv+>{&Ir3C7EXQ1Z#e z3D`F*m%xrjIZ|FVyP)lT;8?nlp`QPBexuZJ;nL<}pMmobBwDp@Zt|1pF(*6AuK7tV z{FR_+=f{t%7}r?U^FA-Hx4)<-zQ%XO@H0_*qDG$SLwV61Y+xq53(F&wv~S+LQEr0$ z3Ozd9s*FnP$<)GEQsUC9!A|c;hr~?J%S%>yxSKv??HXpY`||;Em9l*QaBl^+ajgoA zP?at5sySH&><{04vH?kS9EYw@<&1Pa}f1H>| zTVOR5q+wND|3lck`{Q*ecL0$rUerFNyRlk>&!G#crD$h5maPdER4Ud2L$~(vsjKtu zQ#Z2F+Y33StjzUc22Sr)x7H>ugNBof1BHYD^{C6r4i6O3FxcOo2v=SJ9G_u2QXyj8 z65}7iAlUK<{c^&Ty4#p|3+J`3Us%^tRzH z(OW#v%zWa(g9k42A5vNOwEU}mk55d5i#<+Diy~RU7)@|r&VJDONw3zdER^NGMPOxMC(|=IemTx-+%RxefAa-rkCA&(PqToSq(sY*P=l>0MSfBw$if(otX$s@eKD zkQb2LK8qJPNS^fw4JEI|H3bFY!|e&eeZb1!dJ+ZfFExZx2EfjG9PZ3O=zP2JZlNbj z`qr%z_Y@V^06D*Y|Nir`8$jebxQM5TiQ#ZBDE#~8sN*t*(;RaZAJD?~YAvCHx_VGf z&Xw4Wmmem$Gq&BeJCo_|H^yl_ytkwjB3!oC@aqEthj(FXWkd%k zow!Bsr|ak{j{|}rZHc)`qN3xO-{lwVzEWP;uMo1m?b1{dMvh_8k>o!$vouf$XAc&w zK&@~dcN7*&!qD(-iyMFqOdprY2x-=)%;IT^j#6i9IzJ-75Tucp?wmGhOE?4Pz+$jS z0Z<$+q7Ppa)d;XaDYh2Hh<@A7*zLonty6a0xTA2POh}+fqGv(FuPlR6Wn5Q$iQN(V zlk564Z(FwQgj9OuPZpXz*wlfK1BKQ)O}BnPjjxyu)T%eVU61?P!eQo~fEo6uv=Q$k zErjT3Z{&!*WEprHdaV$*nNH>0Car-`(E`opRsAV+SA*Dp1L9kgbeN4VOOzhJ39_ncT@zDKYikyoSf{{ zRAdzrqC{Aut!&;Bif$uNp7@uGXJ~{szAIk!(lj)@fM|A~7!FG6)zaRe@N;~lPv0iP zQGQ{lL5Y#RZ=eBA^7}R3Occa!Z*Qv>m>A{vH2{|tS5+k!J^0xfoGD~A$Zu$9h-QPX z%(K9uuKOPAniT-jhmTw>VV{Iz^vR%+fC_4J|Au;j#S=KkJw@^}ZHbKl{{d+q^TkM( z4sQSe!y)+XtE_qAgZdGB_Zia87mYWntZrxZ7yvCrIbal<#$zZpDy3c?9>OSFFI#UF zLgTU>a!H;=-)V7NVA08r3*Gr<@=&mSek^=bV$Zw`Urhvv_0#?!{2T=47=X8{Zp(?@ zhketAJ3xCh%6ac<_r8dV((g)B0tlCv_fDtM3(bi@&L07IOa>+p)rfutTy~*Ao!5t& zni{jb;h-}tsN8{|5HN~ECk&`3q=ey|R)Do^mCkRWrcpQ#yD~O}Q3vx4yG%Tz(XbuU zcMquKNEvUx+x5y9Rw=jC3&aPY0<1j-Sh_o((%r7F0R&1<^e5tXm?avYn80{$t%Km= z;iYx)g2He$MJ`@F*^vd$EnY4F!lb|a8nBW9ViFSjjuYpv0OjX*ULKU}N#A~&IahL3 zt<982T&YXY)ib$cc6EkS{Pm|#cXy{UFu60dQqFj{rOTNy%#7l`rljQ5MNrM-qD<8H z+4+TVkxg6Ta%Zp@2^ujo`1wMv~NCmKStJ$CG3#dvi*H_lw) z=H~L)wrRQo%+Fz=`;Obp)2~t?!YzP;WihN8PZ()T2nWUBxVS+u%ieZ?nppDF2Opwvp+wFV*<)u;)FX?1h6pv30LN%Wuw4CGCrdH}?52PiMWn-i5s2OOl5*(DP9x@4|nOsSjzPLteauiR{! zVYc&Q3enBaET7?dCA&SF96mZTW3ux|QCZp8dGZJXOB}#pm;LnARNua3$@8AJwzkG- z9j_;!Ki}lB8YD-!GlFGV76*$-f|6Drtx;41ve<%8LFK6JQD4Hvgd%OVz0D;q{W@G$ zoeC;>u>&6TaLy#S6WC8TUw3m8YzR5$TUB)p&>87McSfz2^N_zZjceTx7r@-`W3AJ! zQ2GL4?FeBl8c@w+d8l+ECDuwjSt|7D%a^9iRF`$~X_z>?5cd~)m-CHB_TM8 zYz)Texgekw4FPlyyfa*)UyrtX%e*bE$0`P#P_wIP59Bp9sWMCEFN4T< z+@v+`Iwa~fz}dHg7eE{W$cGDWHXSOts+^@A`u6PwGz`($RxbpoINO;@2pYQP?fmHx%=!&AUAP_WQi;wcndy+H?t&}4_pqLt9)@#VUrrjUU z+1c5lJEjkZ$Yp0Lnk}HOEm15S;(O4c^G;xJurIJ)*nJe7p&}_naNd-F*v=$L*}>)j zIk)jieSQ7As;Yss-uuK5sa2q*1VC+4>!ya)1F+i;6bL;kjp4M7kjS6Dc;VlbuACJ|%2`%QxTL)iGaeXMXwnBn(>;>+dbLJ#lv<~@i6V$m9J zuC@uZB}mkB7Z*Ul*vbkAEG0{dd?Gsm6~C_9%;tb~bi%cf93z<~H$a1D0sWDJ09rFJ zpaE4Y81zt8007-7Y>e^=q}tlryd>8bhfchOY%Tn={RX|Tu%?Te@FMjy{nS zCF9TdlsF{Y)09{+u2G44^iFfZ#-oy0l2W>IjV@;(h&rf3lMSj+laciNJi$5RTs2nJ zPuCaxUZ8ck@_^-%?B_$JVOr%(P3GN=1xE23H~gcc zqj681Am$si@$D_PPrVZh+6c&Vv|ig|Ne99n+n>m<7zVDctr6de;UNb7#}9I~@6fID zD2_tqH57CL(mJD{prGHEW2D@s(vb?+_(Z^dIuMXDK6DVc>?Uv8!l8@pk(ZSvHE0Yw z6H0OQG)j^$U%oB-lJh#?%Z?nm`U_772M2>UXYK*y2(+DOcmnbY@R;*bp%g}rTQBes z2?!T9v6<=Vpwi_MnLNt@ZYSSukiSgC3+Or}(&oWOYVBL)2 zbzas4N?1D`rTNxV9M_^h@5}N742emB%dKqQK({j>C@9qrXc;T z3%2w>SfR=*M@HYISw98 z;5BGV5P{ZGVTl+kSL{3|rGPO*qs8RQ&^Yeg9V>aV=>uzyAQ>f|k29gEk0lK}j3^4W^zeGNi|$<(mpMl!CMHLx z^|&yc*Rh}h1_lNF=#5lMuSGA7DmTIa{ifyH#_RR1Brdz@q?(+sh)<9D_FTI3+ByBH zptqJvjnKpI;-Ev#_hg0J&$J305OG5PGL?m$t+rEVsXn??4nf=s&HPlXZ(p0Zrjx$0 zz01%RynHnW$be07%;xU^`Qm%^b%FD-& zhksoo`V>nf{`&1(Y^wJB`hQsxNju4~EG1xn!`A_0)m1G$LD#A*C z_fy&D%Cb+&XUa76+)_h)4l{17ef5(wB^{lZjCw$ee+}}moq&68>)7FoC;ZP;u*>DD z8Ey=WE_-pc&k1jRV~>9H*D8gEPBn82{j)-F;R2T_>99rnyr|l|N(tW?U9Oqm)1Q0W zI@_J}##ep*oGMasLQ+B4=~8F?KYI@gY%jr!bMaEMT5QP5rXdz`tk5!~enN<@cVsm1 zk{hX)q%DbY?m198fw;F?1x^3wz3G;`?sAg`-xoC2h7v4X=17IF=1@nC42RZ_7>xsn z9;2oD6J@`*GCDGp2$Oa{A`E9FwZ@Rlw7ZJ1>5{Y5%yhM5yLekQ=2K`U^2}*wp9uFX zrW9KiIf0P(SDetM=sB^m;3`yXlo|f3JN-azd@c87_7&n+J7;y<7gp;63yR8B@R{&T zb_6l4aa8ctOJ2y<^?5?GZEPZVt z;Te!GDU&$TS_9bd4HMqUV^~L{z19EO;Y)pSY?td9%$*D-4iyof2=z?fM+S+->_Nyg7Nt)V5xwbk}x;9X)iBxTz7;`y%9EQ89=X$T_ zC4@S$mhC`zTmx3(#c{)j#G=}Q&6T1ABqfqR_P?;yC7WQFfR>I5*HFHIT!%n@Mn&UU z;BlceS@~)bu)S{O!+kC77ye2JzqblrzU$z!-gCocW5)L`PJ5ayGW>E7$1>)acD9r0 z()z$TVJGZWXx+>XmGT`UpbFer=wX0XGTgr!*2>-i=a<~thx&2?R zcX{o5ZSx?b4eZ1&PO?<<=Y$KWY$?Lyamq|P=pm({MF@!LakM$;Fnbp`#<#vZrL1Xb z`XIc4yxDiBgX%xmtMSc_%dL+W{nY)k4OQU^677WcC-sSuE*A469_yz7!8`^9cLIdg z8=#)r03Ouq&wKE#5mZ|Euc-|bbL4A49!p9-OAuzf+;Jr(g`L?Fxm4dIrs%S=c?Z`% zyVv^zN?lVR#X}cdOzMnf+4m`x68X)3F9FY-=+N}7FA4)En2xH_2IW*%A{h!5r?v`8 zf5MBm@V&S4N>-Fw8?<)sr?)Xm2K0D|2EMDLc|>$=p#eixkQu|WW?m>?KB3hBzluRR zY&4^AxCU)oqfK5Rr$xYt!iaCR4{*YvWdkyH%|H-{`dCMDO$aeDF%bmHA|y_6DXC*W ze*9ov@bmXaoAc0&Da zv%T32AyN#_o;_ooNtoSm1K!jl8x2{?$uDPH|)CAuP;A528kWunylHVCW} z<3sY3$ADU&f~IYNQ6DUj4`6S!KkYMu{&BRsEWjW8z-a0>NpmNwf z``)@n95>cNvs^x`Uk)M)6tLO$H@+7sVb_ui6xGzuIjszbf|#K0`N3{VRKQF~silE> z7p~9n%BxQ%Z8S2x%k8O6Lvpd5c3kFnuC$WAs2O?PxZknVF8(FgaH3gsYGwv{c)iJt z4_z!U$YK4>GmDQkMROFgx-VoDt`6F_(Q$LXa}A+wYqJQ*8p)9K;wq%eYnF&#;WQAik#lm*B36JCZmmM6TYRK z_h~^ud=h%ij{j2(UF~=)7sZ|sXIWQ|G`7;WH{wHfYOpsDtm!G;C&iO*l?oPiI(-us zFxzTK5M_4jGi}li4m?q;YLCD()7N*00ZQ7rOSwVOUcx@ke6$#dgN5?19Cx3=o3St? zZ6X`zJQ6~26%S~#of9{tM9ptca_iCTe(T&0;=^37uG<<|u(~!Le>bl+8ULDd5))pQ zFvq_#bHekMaCiIF*_a_FCgvMYv(9Q$jGU+)kGb8m_0~f3VPh+%6Lu?lZU~p|X>@dBpOVu*l`gK-#D3p_tD!u< z<=+0C1(Y2}Td53F`AIW#)Q?LbvSstdq#K@Y%;Mp86&{87m8u zQ33>Fl>#T~EMY}nyr}>0ilm#^&wh8+tXy>dxXXeSgelDV{*3Z)UngCCV!FwM)$dgS z>p;tY-^OCHKGjN6WHqNu#jh6l6ruIlPfT!eF{N}7nDr1-X>Fihqe4Ltp|Ym>lpWb zZ233V0fSM^f4@H6kioBN#@j@9na^E8|6T`w`|&LK`H*ydimpEMlH0rqk8*v?ALc>~ ze(s9niyHgj_+|StGd|Lu<;a9CARUINoK@e8+BR!BOk2vb`mjF$72wwckyuYP9P)Ap zmWtb7A8DtKW}6R;`Z5n?-CRrEun;~48&w@^(ew*3|Hj?QfJ8r+%0!M7I6R%+? zI7j@5FroIwX&L=1@d&U0m_01rVA58tq%}&rw*GkIlZBm0oitBT($af^ElSH@czUwt zN3hJ==a`{wfelhbSE4b+Wvsykdq>dwdw5>|yR;-32Iw%5)Q zB=L{Yx1DtU-w1}LWp1Ab=UE2rfk*J>hn3gEf%T30Q%-1AY-1ITn4XeGhHyvj{|j+D z3-mpIrBt@Ne!Z-Cj?)k6_s1Cv{Sr@1LUH#vn*+@?JzvIEjGsiYqw^s}+g82bI`8Ms z5-Sx3x>!}ZocsrO-bwyLrBxy=DNIP>5M%118kk5}d;s!W>sInp;EzGyCbwbfKJG&Xje;j$)%|{VVV2Jq2T=ru5;yp;;?>}RFV?K@dMe9*>FO!d!AoAAc!-Fpa(a6H+v5ka()AFb`aZ3S-F z_9erfJ!6qieQ)3CSojg1-Qm9LAh=afiy#pd)#Bo`CNCW}_)n_chROZ`Ehp$~IE85W zDmDV%ZI`Bc%Xm2TDADNb+rqI8=Ri{B6T_gduK)QdU8~jY49HoYdz;slluY;}?f>D^ z%L{L1LpPZ=rPQXTD6FZG%rtgzXc394mXBTT zclpqi<&e0Te_*$3$nhJ3lH5I%F18cW0)RRw_hMvV`5U9OB_rGZQ0KP_6*IDzN0yhB z4;-nGc9ArUsKMtxtD`^EDl&UhOWeei_J{jCe>l)+AjXDCkUFhP0-AXm8 zv7v#Ui3#`1moLiJDbQl8)#U_}!I!jC0K5Gx+Rt-Yef>^sxt&qkD)nn$kxU}Hvrs$}Q|-5SN&e@~7&(9*wjxDxBh#D10GLe=r$M#pQq3MoK278UPA z>zE}%a}m^gwm$w`c`z8@5)f?8-Z~GvNJd7c?pccXfbA>^95*iDQ0)NYcd+Pj6u!Bp zjDNe*eBj2>y(o-<%y#7vN6E@wo2ZJRR-m65pD8>}i%D?uWFGSmFoAY}4~ycAw>pF0 zH9b8#Oe`$6nR10cnItaN#D3L~F2%b9|3PZ&jZ}n?z$)~duln^YMPX4?EwmiVyTJD* z?J{^EUG&qb|D0a=7Af3^ZO(?eKXWo}3OUk*VtKC4#jA`}(`C0We8s1bZ%MiIGPJ;D z5%;x7EU%@V6A7r8zCj^IF7xT6A7z@ck)NTbs?+%{p1)Al_=Sz3MEF-WN{fim@&RyH z6H!J#>xf5Fk#co;Y2>sZHKDAt($e^y7Rmorx9xi@{D`(!!yQ6A*9qSyvP%KV+TKq{w^gZ(NnDx9 z?BmCTYx~6pXjTSsNQ}LUk1r_NAxKZ2^`rU;!8cwa*1(kBTYW;2UU-2J11-1ml6b=_ z{MBxjm)vF4DVtW+U+^Zo|AM`wq;+Ddf76%2i-4qtk3^_BKFa%lVI}Ho2g%Lxdg|w6 z(FbwM&+cgH^7hxLRpnQqR5X*hvWPVNW1nxLqC6%d8>W|+{pr8~ zBJuMqEWx?yi+^ffu^X>Yczk_ePr>NS_ejoJN0-fAeyDlDH-f}e+EUoLP^dP0>An&w z7W{ivx-9N1jV_H47}N+2Jax>(W~1n;?cZ}7>rSV3u1HKb2Dk=MCje6^lbgV>$H&?t zKh0fZM|2%hZCJ?%h%rR>$uG3ki6lZb>erEO8-1wsT3W>%5h3VefC1`Zoy>Bbd>vThRD) zk&V!6*sY9*ohEDWxs7CyScb`jY_pUGx=F)^8o!igIDM&I7R_l@&k_m8qoxKTG)yHH zE@5mNOIe4kkh#|PzwWZg^XW6iZ>)I|_<1V+GE=p0;zhlIh)%>OwnTFLRfp{=_iX&& z6?#j3bXdDZJI^8Ej>qBw@VK^)|}<8-J6cv@}Tkq5p~x0ijr(t;8BMKC*stHHUWnE7ZSYkVfy8&m^cR5dUq^D~Mpc`X@ubjDKJv1kN(55GA; zO!O;njIFFef33!K3`Ef_zbcN{a%^I1UqOssr4HqaJeIW;kuHWUR&k}e;;&J8;B2_L zke@)QnglxI3L~nBxE8 z&|RS++yI^I&N?LmbSlb~=bG9oG;u`dyO}d6AH4a66+s4v#EylZw8XM5-;_QG)Feh54R&cq zORY6=`Cs_0S#O{MK>`Q_13XSnxq*We|04^adU5B;myn&2<=U@HjTnfCI#1m~q+s#` zPaSEVPUb&5g68J6-dD0`|8#Jm!4P2IX0V%D`bjL6CP8UkaOyjY z|D6dG#~4OJ0X)azJZ57>?10BC& zZBCG{H>~i-Y51jFuh?G9Oh-kL-&CMW4^A4&2J8YsO-N9Uegpx4iJY_`35sUC04TQ_ z>TgoouN%*JXP8hM@}l&q*Kaj}z})wNr-88k_wdhWTz}Gyf^$3x#+;zTC;`4FF5_l$ zDt?=)z7|yL6UX%^W1}`0*~gko4et2(kXCub^Xl@Z&pJfjE9o}Oip_p=F57VlbgTi) z>8N2GX2%`@3a^Htnxycazi5K?%A&F9>A%ucs|D|b$A=&R^R3anNh+l3sneZLE*`=Z z-`5v!R18{Af{@WIS{YxRj9--ovsoQ546zRu1MobJNcQx6BY696ZbSV~M3RW>HjXL2 z+?difj4Ma1%O45U4(OS!kkKRphZbNnN`BX7ZL$WYa;117X95Iu`!NkTG|CvOh>hvj>Xzp0Yo+Pr&Kx`WVF2A?HOh*MRzi~_Z7J% zQsu|{{P3*&4_`>_w@5_=8xKo7-bEp3qjgtsX+zQ6I1-p#2=Yt|Ub^pEX&Wsn@Z;eA zn!Cx?m(FZtzwzf>X9R2oxUp{cs1@1Jp&qwf59k}?;^HERpP%3P%a`%Mi~G5$Dfc4@ z8QEiS?x0?ad;FN!z+osZF8(woCg|C-(@j6XnE^8hbOHi>^DW=O1P@|8OsJ{lzX3A? z%thV2eLE*k2kh7pfd6Z1#NWPqr`El(u>nQ`D!D|_Q1~gesN?Vf(5&d|>jT12!0N1= zc>45dXeyzMO)W}GE)gs(Al90^_}zM82A7R=IE_D1&9mPxe1IHLC~PTM<6 zW0BXuniGav>{(PYC1BVT>_$%`BYj~8fWP!XwCu0y>|!Ir@vGecY$2E7;G|Sp0S>eW zaA-NG$p+hrv*cy3z0N9ZG*P5Rh3LF`IE;nEZusD>ul2#eA|fL*-YJCHI%qaLgJt=5 zvmqfSG!3t8ZGjGLUQCicKY!^vtOP1wc$7Q7^!7X+vO=ME zZi9h-s|K_E4pUIllk{|R;$QhC;Tgzks=G22+9NNIDIwXRSgX@5T0{dE?(UMyf0?#J zTLH;zG$Qy#1^mI#py>NhbGmt-^@KN-=qco7I~(S@l(SD__0E>r#Wb1y{ms=|=a(gL z9|Dn0=rR2bI+qh#y7PoXTrBYq`8_B$PP&1h%~-~tsTdk0=G>OTi%Jh2WMn>8;z~+D zl|%kug;uRxX6|2t(%|l)(_}n{HvQDqGWZI`s0jfQ3F|SqR$T5qeipngrb@9-Z*VF%Z%RKn)NvFwXBc zR)*f*-;iVZS?HIK;}DnQfYh;&@fzRNhDml);&Jc3wmJ5`?>N79`VT<7*>q){1I6LJ zu8Sb7ZKfx4nn(!I{J0qY7djNjH8e7^b$aoeKLT|STM>E^0o|om0~6xe1^+^;y=v675fh_R%Rb|H9dGIP|reY{SNNksKr~Z;d(EIQe+S%YS>RPHz zip-dn7gcEEzF-yYCh$errjTqi4g9bRtiWQiNhY}bqS{a&BOB&S_<8BIL+pi-x`vgFG}E3 zmYoZO;-nQZF2B&nPl5((KN`KOmj-^zo)50Zo{l_+8cIUIF#!M4u)2c0w_xpz zxvz3=2j=fZMEeNEfwjdmTxcUHQ4v=XBZbh^|J4osvHBYo5)Mz$x?k**b((vvuEdZu zI)?bPZWZwI*VY#Ab7LhX{)pwc`1rnZjS)U+zqGQ>+q`FQx42Blma85(A45jxUrM=H z>O-SuM=;}#`WLC--(mfiTAoz$+$o2sQ)RiPI&$U>Q1UYtN?DJdVRT)J?cr+p@*;~PsG z>!Yal2fE+G0|)U@sLV(rKAKB?4!upzs1K$TlfgZVj#AJoYqg>vE@r+HheZyu74*gj zSiyHJEh`&BqCV*cq8}(|f4Ai_@AJ-t8o8|IUI4GnNrP8G7?$E|2fm(HwLqEtO&~RR zxG-NPBoM$fP41v64?Q<`=Lt4Dku!XKmo$FCJgp}d_pY?*HGI{&DW-K2%s~dV{krNS z zOQB?J>Z9QFbP{jU3@5_;H(hajap(Gz6Ssg~3>W5I>Ksgq3C66OU%qSAi2IL1P&zobE0{`>D?OOhE;3W+L`F>I8hC z%sZk|(tHRJCX1JRW6j0~&R00}j}Fl6G)vl7h4!5Z-#x*!Io#jrIy^i&Tsg|TE29|= zZ7>T9*1vTlX(Rj0UO(x|-q6I&cx*(IvWGOdp+31Hv5+BVj`Z(KyfcVj1lBTTu~^wl zr~w@OsHS~67oC>+r`|>>g>Rj{kTidRMlARo3rPf!(PQNV@}Xt{8d*W(karm;W(uV; zv0j1&e1iiXF93-QCNnu{+-B)a_}VUl;~s!fUwK?u>MyTbCU4*Sv?}-5aNYfZfv50k z3Ze6QOP=L616|9u70?~}y%&-s0X|D7@bGHNT-R-e9DByPHLM+->@_j}G(@!Icuj(D zbD6hT?dKdpZ|uGYIa%0J;5D5dkx6sHKOb`=7rO6pfO4To+oRogv&og8+(q6>^f_%W zsz%?{a`)8h-3z~2NRAfHq$ilG;H)m(Rl`N?{~jPlg7>;8Qe9HwK6l#X)o$y6wXctT zInyz@=nh>brrAx;PQRKVBaYa7pnv)%K&P%YnA3;ZJ#;n}CXi7>J!AeyH+8`$dA~^% zmE~`{7W7dforePh#706}j@x4(d=6HRXBr%QD#I^T==7ZUMRet4`b_862h@1h&Ia{h zd=!yX&;^C}Fy$LP8m*IN&sq|t{*@>Iy$uD9I<1c0OB+EK40xQE3pdC2hL1mw1PN-wu)*6EH+6X8pzq&n;Q~6;= zQh0AM7uZ6SkyJ7CI^w|^rh1TI<>7s-w_ga`L*D=XbQ}H8$ zAzA-p^R6wwI+0*G3*F1%+Rta(UYI?%P5BdbSg=wJbW8+`{SyfU7>t)L=>waE?*BCy z??n*syGwy9ADy^|$zAns7_rE>p`DwTM=IpJTnAIGtWA*P!R!18g!w+tkPg$l%lGi{ z@E|`Xe|XvTmI)93`My$q{};zU{T~yVFHu-k;N~HwrM2Ezf@!WBxEM-GY)$`Z16}a@ z^rfL|5Bx@NAy?nKhi;n;Z{{#+Ig*Mz2Le6Ap5}?LIQE7J#hJDGOj+cbrYezkZ10JT z80_NviR=X;hD{vZPO#e0_y8N%wXB{}Nt7pk{rdG|4r;kF6M6-YPO*p^Vw6vPg82ys z0e%h5MM+9Ybk?Y;NL||M8Ur8uBePm~{!AwVo@$TKb z-a_j{qrR!1%$F`5b9Hr9{)-kr^7Xa-Ijew6$BMpi*_x3KJH+@Qfu+X8-3q*`9MtD# zLR_ZW?D-Y$K8_(Ln+Rq;;l-cmiETQ*FREm-=_Cl9rYV8Jcc@M5MJ~#g^@T1nGHT>- zk)M=X>IB31=p2tN!E0^+wQpdE1SU(xSh>S7Mzb4fYT;Nsh7r;M;f7&cT^KBcNdyCd zfT~vvK2M%Jp~83`f+)A3oBWx$3IeMb8YFO5h-&PX%0u_;*IK6;dgCL)AziKZjhrLG zT>Zw(q-cJanyF}re}#71{g;kh2a(3ZS%)|eh@4Z zlV2j0o8%LOpTLkxzh?+MJ;TGp!y-y$@$-v0hfVn`Fh1r)8l|)I#kQam_yc5;3X7s` z@7OE9d6vZS_G@Ts@n-g?&PVB(4V0IcFBvEMi@s{UEC&&*QAsVeS&R=QBG@XV@vM;i zXjdX((yiAVH5A+9{@H82_VlzeUUD~#4{?yHH0mSmyNCMqiQpUIPM#nd`H0aUa2$G4 ztN{Y)#uIZZsX**h>nZ0keyM(itXZ?&O5Ak(F0ybN$WWdQqXb zOw)TU96IbEnUKwQNIOIPqZHAT z&)lo)178??;J{?>dxJF+o}`n#Z^0Y82Hwus5HXIMyYNsYPOHiTC)COadASOv4$$!_ z==XK^nRlO|!-l7*z;k>)6Xsq9uYPh)wu;2i`WLzVyp;>ecS`YED7$J?uu@kiOGuHb zU@@wy&)b}J{>R6}Kb#{x8CHL1#iBNIVkfFGOQyMOaq~X!R>>xG1{m6VC^IOI_YP*? z=(heV2cm+ytr^D4r%$2L56>bwc8bDq-5rKenr1b%isn*cpT@=pAMI5h-H?+b03inE zvBi32Puk-AlKfljtZ)v3`#vkRN~Ly=S!AC$b#Q9i@rYR>U^`hdP;7W-iF3Hm1KZ$q z%-Ed5603JnOIp=D^*JUcF1;^p$Jf4p-Yl9*W1{*Z60Q+tE!|!GBeH?9d@FUR=g>-? zw_-ri>&h0W&bB4*e?A)+;Vz*oII{Pwz_W}!rVscxWa-;SW!fu$;Wd;bjep1dq`#>r zD|-?iJi-CvBI;f+9nucvOmO5DT18TOC@y>}IeWw(T$eAFB<;XpA@V#RAnQS>3h(o_ z?*k&vOk|lHiDv*fO4|-}DF9>gq1{T5n3+Dc>aDYqFWjc6^KP`UTmZU4CR>haik;)u zx3kEiBC2(aH>7>tj+K29J!5>PDAoHUP~z8$5?HeiR9ffh)4~+-tKPq7LPj@S0;$el zT&IF2M||vYji>`#e!}#_uT^X!u?Cu8riwUircfV;`0z*^+K zc99aPH*!}$C!xi8SoN{HXdh11AzgnH^C5f6brEaq9k*l?ha1Aec~_wiampb`znFYI zl3_rj+mdQ>VYP>m-cCB^CC+|_G_BI!9#rrpy!<*x<{(T+wh1bF0W8Lf`d21UU)t6a z`=V?a?vtj92N@^uWn{_o9axa_JFnj0{@gf*?9cd8GjV;`9$98Zs^B22(%36GFi`Qk zR`kzg@S9|b*PDO$zivr65q*&QG0HK>!Y3Jv}V;IXap9W6R;rn8KR?!vug_sSWjcL5FKlg*68QeGFh*6ESLyqPDj+Pt!pqa_`o!OzFJgN)MI!;z*(4m zeUB15KO%T)6A#5zvpQ=PC4RwJP}3m`5GjwWJfqnH_`AK*ovix9en)AAPvi>XbxdUc zq#JCCM$6lS4`1KOFYmZ8hE1Ot+dIBjBa=GXXT9~N=zGY4fIq}o;4G1MQwjLC#_g6n60Qu9!#J5;YVoa&lV(yUGlmfD zFm{-czsJhL>>_oY4Gu4R48BtIW{KUN&h)%m>6u)XCxMT7o5%L$#~k0l6}G`XNy5pQ z(r;x1g~P$z^ES?quEKq-kk9p?dU{J5dG0AHr;nfS+xQ*8)q>~gJmHF@C zg~Sjo4-_3Gj2-iI)#!~#JpIr)Nus1Sn(q2-eKFJM!vs=?Zix}@Rhd-3X}!zs0;7?o zT=P%Ha8Jun(QrHWeG}qK&wL*yfBc74JLy-sY?&b_(Ry+oB6!V*ge9)zx64;CKdz|aI)5~kAi;B z0Z=z)j33KU(wi)<-*2T>GtNyxtbMO<0o2-$$_*Tza5B)+fOw#!edU(ATI=MS^s7#M z53zDb^F+^N+fhjZsqP=XUPz&Qsp3R8@X(al_aL4y!>^vT8I1Jop^b&ApSc(*q`mds;51)c zuTnS8v3!q$3(iaE7)Nkgx#F+Sjk}folxi7OGgdKvQW9tV8F|*ycpht19m(5?si-QH z!F*ht%4;mSS}v$F^_tF>XzvNb4ULWNLg@$@& z-p^S&eA-?K5Qyhi|I8X||F&24q*hP1P_vDR$zA`>(;n)UiK^CNsdF!UH$>$r5sM^m z6P%Zq?InVRk3t!(3)Diugn58Zqv|#LPAJ!EjDjk$VvF6B88Q1mw)?m?(KXWES(WCs z`SAHePT2#m46E`(RG${C6`^BeE_;d%lV7_e)dT*ryrWemsaMmwN@(_}U5+?E0uRmL zY+J2d2@QtS_k~Dg%!_FsUjwN_FVw0#52MxUi}13;2oo$~2ELU<5q@mGZzA%{>v?)Y zO(iGt4m+5b*Y?X*rV9%b)ADndED!g_?GxqMH+?Q66t~uE!CKSYz7aGtCL;Fp!PKa6 z$`sgi*s135le;b19n1&sUbs5O5Wk@b)%CoQmjy@|d@#Xs0VKf`*i+$9|6-hBrqk`R zUpOBeu+X@2YPwDL%mAhYQ0H^%eJLJFj@6Enz`Z8Sey9FRJnwv)38s&P1Ebd3EtsN5 z>3#$6GUh}kvJwlLUPu}q-)uO=8y+|i6YV3a zZQ*Ftdou9uE2mJ>JQ}m*_-g&$jvdSs-y(~?bP}wX2i>W>=5(9vE3p$bp9@Iv=jHh= zI&!WHEwd%lVDb%EXkOM2k@{a40Nu^UfOV8sA%OS{Jw!0*DOItb5RFqxui3?H&MHK1 zPzmV+5J*%fU$<-CMjNUyCv7^Yazn5?7Z|o8BP!}Is5F+hsUP4q`4+0 z4#8;P1aQ0HP$rA$bRm zE0B+RB!4Fv0Cxl-?Cm-d{hH@?m3~}s>l+|<;#EtoscN`9NJ;j@T}>pmE<~2khis*z}F!V#IH(;xIkTs zH4e#Nev@if?V&Vs^R`S`P*GG(D^)+&n>FP%15<_?+05{ZeN#VU4%yY(CzCS=&r^fx zrycc0!X4R7W6h%>ir|beZPi z{2tg`eANm{$<)VvCAZbZgdJP%9o+SNuZf{sq}=d~%Cu)g+-L8!qeXiIX|HL^g*G;f z^pw2>!SRU!QkmU3b$OB`vBQ0`^CqqA?{MVi?!?=xv_F&Q-iZNizw~|J?of4Vl7vAeMES1RR{&lHm!hVmPEy>gGhoHy{FMMe9Tw!wRoWi41KKt-k+|8z_R9n8f&!5{xsUR?%6aV^k&3g?S0;LtX6zyC>6m)U zG4qvC%PT{Hh2od8RSEGimOf{x%1_5kOtp!;BeOax;@P=Bur!v(vhYIeTHr#Cg^W(6 z`(2-7s%-)x5q;KzPLH6}svLSX?hV5Lu&bKRjwrs3cv*YQup#E<)xagXhiAd}n-MBg zV9mz}%MlWAcGw;DT{R=j&%^r;^#E=oM^kR~w=@zlOnz3eL_JjaaI&-#$9ZMU_h8`T z)shBwFO$xEU+%oQ*)PY_KkFvv#C>_c(nDhiq-@gey(7{w;6^1_exJU0`Rd0G{ui%- zDAMIdKu~s`tk0TrmvNU{+j-hU_IWfI1FPWCK;HKzttuWfy&)?+5Yp7+rpT6izU;P-Mc?fiN%d% zj>+u6v_1_eJ?ao<=r|T}!(@0V?UhJ5GiO+_=A*=P>PknUoBj!zDJJD{EuYWQFl8mt zQm>t9?zeuvcg18aUqND!`Y{O6bZ>Pwn(6s^JeW*yWM)qiH?K@xPQcw%{^dwW$Zl*+ zXr%4OfKhI~A8>xq{)e(fF3nuOT6wJ=Oz_To1*K1FDZ;R+yS=)P9HSEaw+phYHbUhG zhP+F)fv1@mTo^OcR9$6~M~`>iuo7FtbR~FOpNRX&>gZ(utiAIdkpmaGPeO*1iR<-= zJD1!IoCOvyD(!0*<9;!ATK>cQ&WinXZfb&VEe+kPUZqQsDFk&egpBRCRk7V=)- z677S!-bE;x4%X+S zsb+TE;Hwi$`4`^L(o#ij}T`de? zqR$*$94&mH7H4*+KW^DmB;fMsJ&AM#w4WM}XvO3wE=emIX%Jw-dz1%`Bw#&HIegh$ zN@}cnIPqCUW9RAUf_MIPnqrsVSnkjrGcmMW6}I{o>7JDSG`X6hsX}jnw4<4a5^RGq zYj2_j!!#%`UH1!;GDcVOn1+Wei5%_TiQwW+^W%B#T+3Z)u?)N&)deO@90iODyVgsh zd_O8HX7)FREM&b2_~~~?QgA+(UG-MH(AvZ_*2quVDfF2Du5L$*1;ze-k6jBqbtH%n zJin*5!^ZW6vgR2v*^^p~Pqdz9$5SY0WsEOkoUM(A`@j3v2YM)Rn92n)Z(ElR!R0j7 zYd#sTs(vQCYUJ?1W!y!6N1l3y{FC3>sH1!OM>L^fv|)%#an+h9o2-x~VAWfJiFOP@ z0KLFFLyZ)DDtGZmX<`l{Wy&}awncXHW25ab3R_blsDtTLDio;opRCY$#(TBbuQAZ# zuXU`$Kkw`(p6p~6JqOwRA0JqM>chbU<9ygf-Bdi7B4sI$W+Wlq#6Uc5ahMkjE%M<5 zvUqG6$|QS?8gUz~i7u4ZUC2nYCZDIQ@T?M8D&>vb$pS+K&^i6{V=kEvMFiJcr#0Oq zU`}G47`i)NWjUJIG%Pzlvp6Sb`Qh>*IL)fbc=r#I-@PaDoo|iw8Cl_vSG6^UG>0@5 zeTkbf&(Eohe>1nHgIflukTt?tp8Uqv*^dR5)mPWX@cy3R$Ls9qB#6)^Q}!e1U6p-2 zKlMqklD_+WMkIH#>(~!+m)3i!&ab4-U5|#v5pS(E(6uApCTJHZnfL5I`{CWsLpqiv zQ5El0-Lz+z*HrnVId00Hos-Vy7ppHzpPMUyca3=%pd}2qhnHT8G)lJnD+xudk4<%g zwtLvHO4GkZ`|2{XriTKBl33VDR`G=>@#b%4CnQ|C-V&)L$Q z)`$O~0yTmi3X@}MPTnTMn^x)_ZI_W6iDO?;``2qe-a`PpvW*bXkr-;g8IvA*M6jw? zX0E|v45|QF_T2_BS zF>05VPU#LB=f4S)fj;G)i35At-b-r0FJ;cTZ#p-w@fg?$%eo2Gjt(zKU$=-jyqeh` z7)GBFYK|)>F?xTHcuk+iN7ZF`-y{4MsTM$mxCF6#h0w1(J+?YKh%vBvN7F+~snF6L z*CQSEH55)!0k&H@cWgt*uv_k(#w)vf$$`>)3*LDKh|807sHC}>A2ogXO7?5Q2dYUt z=fp@L6{v(H@~Hdcep|&mdiFXgoLFLUURM0_tFG&lEJtK^=ZozSe$D<)HeR){2(Uecb-!*p zXO(RBsU4k4_Zulgd6YoyZ8P zwgZb;(#*uUR3lm_eE0~&F~bUdt+7G?A6XsCnk8gYAXnen@=zU~qWPgvN6ZMh@$AE! z<_EOyT^M2Qpz!YBJKP;%I>DW&dOYWTaNk;|$meviQFI>~d!FN|0qhN7g8)NSR9m|r zAd|%a&js|TU32XM$P27qb=J{QqRz9X{p-Fnr(b$9h1eICW+WWYU8&_WS6TEk^nE9m z7H;0~*zllxH8jrVjh_2zrQdF#x_VkUCsV#s%trm95;`pF($t(z1Fc#nt+yav71c2*2i_SeH!rrpysrA;i=7))&BOtCC8WqpR^ijrFksngFZg!jSSMX1yM=wi{Y z(*zp5i$BG*m;-&9(?DT`YiXMtwsK7?*^kk8*n*$l14mZ&!0lQ)Z4(;@k#{lrwl@Yt zS_Gxtpj&03Azm!J60W7F;6NkDpTZSHS$Ak=B`1m2Hm`d4$BZ4}@sOA`;ixcr5cdPV z!|X`x?2wGFeD#&IHkDS^DCzTrh;oNGRK`IG9mrk>tR>LCN(J&OZb&!ZfHDK7eO?Od z8rvA{-hGH=!3bo;iY*)jj8m9FtXW#5Yo}z z-OX!1&*o7Z^x+oWU!R@VO?4Q|6Rot)r&4{IcWQ&|hs&ZDkTMh#0_xXGy13Jir_Z$# z??gO~qEUdEfi5apu%@vQ#S)+nG{|MN6{$%{LFlQVsiQ+HAh6WgxNFxrP~rjz3JGHs zRWhYtdUnIS0kpdcGrKS_JOwP?LzoK3iV(we(yc)4eL347XQ;jsZJ}uJ`9HGY!DyLL zr)wNu)+ zbJ&t7t(w{$j)4XOX85sd!uOl5IkoW|EB!HUrmh~=Ku_L9-AQ?r9Y=u+%?WA6G1j9k zfR+|;fIGQ4G_`FoSpRiT`Ru~ZXH&jf!9W5ygfK+x$I?6~Kq7PD+A|#~8;jTj`7pe@ zQ}KeKA%TR1L}S4Tn5Bq;1MJOih@8kBoLZWn&o3)mjgZ%*OO;bj_&UV)-jD?K3Wje~ z&>6rvMM0ZTUa)`lT>Q1$`lUX!Eur;sw^-=D(s7eIs9(7L5{ALRnd_63eG^7+)oLAG zsg*vKS>MN1{llz9*VxvwIYU1TT~KK@NR?XL zO&j681}#FugaNN4WrMN&bI@V24DXq0^%2JieZZ}O?*;b|F=t_&qR7n5gc;{KDXD6f zP?-jQo>!_n-|IOH4=?3%vxl~nwb#+L+$jmh7Qb71%5Bv@wIar->HQA$DnTC*7Wez$ zSE|gbtV^_9INt1hCEekk-fh7IY2E&3v){mJhopj4r?y%5*LS>$PdFxUbBND3t;j1K z(U~dZXiP2B6=)oVI{3sFoOD0jFg)f9JHh1C)pHu9AMrhgb#ojdD-Rj@#B$rv~4lr7KFZ@EfF=dY83R(?ws z`)kYHGNB!KKHNRP1Ag~M#5jtIihsIv>()aM%J*+-C;#h*!lD3*r(Y)!q?+31eB1g^ zeB-3`FKuqx8x_~@JPYg>L7j@IR-cNnUd0oWY}?rrMqhrb+R)YL-sDT1XauST+ylU} zJ@@r3-PCdZx(`_0>+d4Z-@f?AkA7SidqoeIxkKH0V8gk~rS5JlZ<=Y$#EEyNwDBxM_jf z4rb(kz^$;gcGiK9(b&Jet`@~UaiePg5!2N{lyE4;;`AGj-3E(`i|}McrHS6%zVm+4 zV>rU>;3%*go+gMzU;`kh= z2g}1n%bH%D9U=hMh4-@v8aXH(2p%>{5xKj`?hr09ZI_$uA9{J~>MTvLtOW!-0lzy^XF&}S71Nw?;oo#t1JCh7h;eR2 zN)!mqwYdOU^EiU^dMo{3_u2SQkpVJ{Fucwkwy)UzvH!3A`2U+?AhUtb8G=?y!$dt} zlz93046vkhMAh0IsnXwIykt& z9vG*X45;!K`i-#J7qI3nBMDzYSy|Z+YiJM`K|H_{(B*voso?;!C+eNc0{qwBNa?Ii zS6o$Q5l~F~VFnxv7+hx?UMDU2Id%KT5|6g^0BF*49=d?u0f} zTDlad`Cu(cm7^O#v~^ruQ034KaQJ?slkSnDb>^~Mo;Qs>-VIF8Jx93rKr{*-(pAFB zLMgN#x&5)rRdzF>nk1KTN8679kEuC8?<)58NZ$p?BQ`qb7^&-=V%(++abtn3JM{EE zqEw-U^^N3~GkjxJXz6(lS2b0)4u{(2uK=`;k%}WC$i{j`PXv5dTNT%TY!f!$nbgF) z5gTZ0?LLkR_&FPv@PGVWw2H>W^Z+{U&s*2#$DwbwzX)s3c>pNeZFb* z{JSH4tE2Q!w5S5v41SyT&8=-5w%bCU4aN4aJA3p$@2pZm!vkNVlQ&H7OM=7+HsJa+ z-Mzot^^!hN6f`E}>vVbMEzBp$YL7jbHw4xrG$EFHK_pEJ#M0RDIf zGHh6caOfYN1^u$%`=C7f?JYOtm=T;L9tR2z22(3IwFjjdKs*W#tb=Fp_fR;y5bQ@m zXjU{joRoJsDVascNl7vEYhpLpV=rGP9kk$wqj;(h_JD3yUKv~C`!_RzGt^HfgoQhP zw7x`6Rh*cZTfVX>6Ed$wrI?Z%Rt1j#=FRDRpFd-nw!8(x6e8!%AqKn7Pdla-Vd zy?ptxmMx45fnoG4Ai!WoTOU$+fC>xeaTvfPX@~NmKU@ijZDs;f&NJSesgdv)^@Aa`24Om;(x7Nj*5Jd($4O86DHItiF4lrvDl92G>=thmg zu^AYmeiS>ZzyJlvZ}H%rgRrH6)dSMKatH0;e{}^FF=D?nU?fkGK;FRtGP08n0sqI+ z;@%3>lsQAOXQNDOQ(3RY*1gdqnE}GG)na!YCjTgWHeKiqGLy(sf zlQ(~@Vu(ikZ+q|v?7>(*-|u^H{%=4mw1WK$n`YZu3ZIJh-pz9q0+b=z_n!{9n>-Nr z+xvn1hcn0xbLonT?EsCab@9v>nU`ssJRa0)gBaxm3(KUAG_00(?J2&$rEfue)@}ITfN42r(#A(d;Ue zMuReBhYp279%7BrqZ^160|!Vb+aN)b-{IChICemo9n9UklQ{e|fBkf5)n))14^nhf zQ|}9`;i!GoOQ^;SD=O@nyxE84V66#QE&amy+j4!Yy!3|nDx-~d^Ho$86l`udun(i4 zOfaUwtL2^SAI7(}32vU`4-&%G^xW;=5c6A`b;uD&+ZaO;5GcSBeDoTnPfV?RI7`5V z2;D(=5`gs8Tx}XNJ#7IS(^1HkLy2f8A_Zas!ctP|n^DoA zW&Im0;>%l<7z<+-aDo9^{1gg(^i?uC2OwBTcR^u3C@ukjB!bjvyaOh~9x%C;K#XhB zy*VQk=k{9A_CvCN1OrMTfqa6%_C1nKc}^lXWwD_(up{*}$g7qBG}#L{%7ws>?m^)q zC~FrkS_$|ikh2EW+Tiywv-%qF1Dr%fU^bb6bPp7U1{N;V28T4FfObyFW=Op3upKxG z@y@6mF#&bL^w0Jq!1YJzi72`gbe4yqUkbL+U1kEOd@e&8i3G&Sw842k4t^CCl)nIN zXL$@U96M@^{_fA2@9~7x=a%N`j2^Hg90ys7(L%EUw$ z&-y17#->eJCJ^=)4amNgTk?)tLXlsP5%+{fgc^d|#Gp+&0ReSD*TIEw(W~N5sdiul zr>Zzm6deloJpqpwL|Nto4kMtAf8gMGHE7jkK!gtZCOSOg$@`A-KLl?EN(O}Ef5JPk z(L)`GRDk-n5U~bWg5I}sdfqWHkokJbZtkZS{13d@e17AQCUXZ)g$fxaq3|{^z$X{L zy6p<>aTNNN0dbxv8LtbQ+58nfRODwM+L7KvW;D;kJQLG}LY%#LI^5yGKobzKeM8Wn zsTu1JG09}bEDHvBZ`pNfsC0N1Jl^TCywrl{6|#j(Al(y{b#$o#DF%zvLvWP~AkwWL zFg6ItjqZ^B;!qSMetjX#h<;sF7O9bB91vM{)3!c)&20ip<83{Q?wbb9YP}-&Bdog zX5NI=xDa)CA@2m`=vcP#kAki0zDi;lB>2Vq2E&FV>CbH`6Puz1xkmGF_)fB&(#9@A ze17lE2c!b{Es0R(E)$1;byJQl2jm6ehu0D2s*DvM;+IxXc-^3L00SAH`ykyZ!spgS zGQdkcgOn(6TBum$!UaSt6Tv>Jv%$g+7z|Vg5bopd;Sn>e%`YmMYzra069B~*M zU|O(f0`Lxwz(!Dr`Z2JKod9Gw=Bl5Zm2H&Fr*7n zKsFc@5AR`XhhW58jstqMs{d}|;m_8S;sVBY9OO=`I<0VF$$J;SfngMm2~tLZ=6Lw{ zEdjfS#>TuMw#9at?{Abf4vj2VcuLeDxqTjzb;sdlu={=hKOQ#JNXUUci83M4Y6y!f zTDDOHZO-h$=F~ey_*DEBgwR3%!0^zOSW5q;ISL5+1U2E>O6S^fIywS)66PVh+~CwB z8$ZP7z_39SN^IY0xwwo$J^b&PS=M7uAvNv}IM!Z^I0*xR9|e`7r>n-uuk6Ods1-{P zC!Clx39|lsKX!jX6b0Cs?)=)B;6Jw?0{`{PnGujXuJCi9}wA?jo>;#rY~b%rtO|G+IE2#HW|25AEGGI%+tR z97calHnr5daj0VLHqmjWN(i3SpcKDo;fgTPi)E1Vr#EGuZ8eI)Ky8k3MmEYZA2`u` zn4?zE1t2c= zz>r6YRPofEaCQw4aiT`B0YE2uczOob?L86ZiP&`0-tZ=MgFpEKq!mJV8ExFHR3f`HSB} zA-x;k`-J)S3lO7mpZ530!-SdDPIvbdXPqMA=XC$I{`qqP&sKS~{^bRy$w_MS-BBsE Q82IP1sMN)T^J)+N7v7bp!2kdN literal 34208 zcmd?RWmr~g*EPHVDN#aFQoy7^>6A1G5fwyAKuWqBBm_wn5s*?)QBXpi`=*5RS`BShh$t);1jt*No@8T&`pHpUj_mpOPjF0-96 zvbD9e5#r=D`{x%pEUXPVn^^OIz(w#auidvn5Tb|ZKNz3HlZ+9>_vCfyt9Kk@76zTv z?sOjFE^o#@d5C*e;wtgg>)0{_DmMI0S2BO_^sjZt-Fwa6XV8}r9{$L^KlAQovo;FW zcKORT*fKKJ*fJ8>WLIB3j1(*t_kZzl3aJWg0X|w#13HB`L)_XE%^X%AYUVnM?5x4))E><$w9|WxC_an2;zRKR;#rM}^9= zCs-uJ#E6V?y8VKB%6fCGupc%7h0aXtJJ=x!2?<2VVKK;I*o{&)u}ycdu~9y7KWWZG z+hadlwBV=xx6Dkd@nHUh>tW~dPMoO9?$!#F@+0h=&EPEEjyna-R+6y9998cHv^+-n$r74Koj;^weN=)=mVj66K2 zw6wJR0|F`{x%JW?cXOn4drZ8LYyEJO(#OZAth<|Xp+C2FwM9HM)38PTORmeNc@Uj2 zg_V`n^7^`rkR{g+}52yvlXb)f@&3?5KE6$PyA0M;8`?r<$V2lT%W^ZCTgL z(_BzdQc@DxnMrh<{Sa|WUEQRU&-w(B9KYvJ!>%v)VZCLHjV~SSZ)&^l%qDH^E|q4$ zLpZv)JorLGPVU>AX^4gVT3yXBYGV@lRmWV{&`^1kvv|UP>OnpHt6{v`(-yJa#x@13 zPf79a)HYL%<%15x!Ha{1dfUGysl?qy$Yjd%tj0}Tf0nrIE`MiDJ0m7WPbX|&RTE5O zmg+Fm$|$lm;!P#u$VKnArhE(sXDoNvy%LK^#M9$&zk7-=Z+j}*YQ0nS0o#0Mit4`b z+{flP(ZDBIxMPh`mmTLmCymU`KI0#DCEefMlru45dBbO>K0cmn*8g;3LO+OxpAZ8B zCEtsjHCf4s-R>Cc}(FR!j*-C7Y7xBp316Li}9HT#1K9WI^D30u`u!rb z-Fx?9jE#-)2nlN{iNu3iTW`^dyVJ(I{=#=&n|TsU!=Ip%q&Q`lYuf7tkEJ_S^=x@# zL-xjv<5P`M-r@&aSo2+9y=7Inn=L2mNXRHD$3uh{CG_>rqH$|cx=)cP8(aZP9qO`} zU%MA`>Cz?Ghw4wtiNv(DL6v?)-J>COm`ZC`@y!MhF#~~#w?P{ zu>w|19~GjX$^=r39v&={KvbG$gGuCA`MgoNisPR`O9z5%ORx{8*TrG$Pp<0^j=rt{}(bbDzmiWps8U8Sw8 zGCCjM)mHs_0>`^`etq42FGH`^Cn+f@@`~|Gr>J`UQ*OH(8}j>^}rhK2^xLi>ep>SAyLFkt2MBO~av@ERY{T^Xy+P8GJExHdXHtvX*~ySHUEIy%a@ zSiCu;9&fWakPn~Dv(gKm508ilgah>q4tv*5%!6EWa-`~KZ{}l7hhG!*n4Wb_O<%2w zU%oum_T~O%ZquH0_4u$TK9Tkp%yI#+|6M!kw#LiE=>+rrxtz$zNV#`*>O7o7I0cD$ zc~lpa;qr$?=5bI65{p%9`-WW7VhXsAeV zZ*Pywq~l`r?h*u`W#?U0#)}v6N=iy*hD%F3-+h^m_n;3*(1ANdvxb+qw^cQ{>XlDR z4X@dyHWvrmid`I5zu7PJJzO3s-$Hsp8S@AH>c=GRk;e%;uECl;`qyYn?oE;!WBQ&Vly&)Cw^ zGPYZl3>)%1H>V4v%H^M#En_t_7FC)KgT6&YB8~Cl4EsClQa5e{R2OXMb5W%~YM>Z^ zB$b`Y>Qe@1cNv~wtXX(T+S;1OeRqLhp3!GK|A3tsn*tCc)|N6y4sk6PkZ8zrvMX;!-2)RR>v$DD>wY#yv-Zk5n=(oLj zl3l{Y8bTi-$JVvGY}i~i9x6^KquiSPShZ;F$4x3BTcgD656SIGO5<}WX*I$8L;i5j zr-xlvPcD`oy3g1OrJy-~?G(gHn3CADunS7MLq$$hJo338^oGqbDH4s25j3E_;&B!x`iaR`6tSS>OzG?bXM0J zzJI@U=Z??nbc-G&_DEp|4#;eM;w^{@F33R93yC-Do+O6qUkLY8J=_{6dJz$E0Wv_AW@bUBWtG0Gw zz!b9_@kGu{wZvOwo3Bfi0hsVc&S<%=$ONv#jxdwSkl+&15=-J`K-gkP?EPxsl*D7S zUmUoE_(O6Ih!b^r!@v*cvn}7oaJ7ZAXxR(uBCPGs@8W>)rCtXkigU!0jOM1&oRApg*td^HYt8?ATF6@GZYHo1GYo74)hI_dU#++ z6hZc*FjG)eq=4+5@a0S6SO&X#7O{+s%#>Z}_7n~L#(LKSN&f(;IY4l_fMF~2d7`?7 z9TsDtu9B;2Iel>I4LW`K7-F^B#D8=5R>UF83d22Mcq%_N;Ae z*yF@mSiaA7gx-J%%TaTHd^6>iR{*8qP50{NCM5(=5Epguy?L{NytXkxNPDbDMWSj`G{2mytH{CLs#5guub=PUzC|PkWjQ^zCn=4x z)kW4?r6?Mb6k`LRNQ4xN=X3z^;?wEb!&Tw0&c-& zU|>KKXC5qdQ%k1p?CrIMe#-Llatgr{r+JQ{$gjX|-m-x`H!@<_vhw7;Ud*!OyD4)W zo5_BuQwv(D}GOCKsMuK3~JWf2jMz0YW>L(^uUokigS-`TTg z(LzYYtt*|F{rRPzAMVi#w+O`&LSf-V^O9+Q&Pgb$`P&YF7z?ZM`T&3vlaM^ocHhP( z5?v#P-1OqbizkM0PBlftcc}qkmTZj?z#^?a+e9pf-8S!M8$+nr4b_CRGhLfvK()F6Loc$iRn2v#z^NphEkM_`0c)GGTj0j*foNN8f!IBTR8eZXK7@=yQU(G0 zY-YwNWgVafHnP&A7aA3z=`JTKywD-rH;Jq|0 zGpm>GlX~LY6Qn72P}ClaU3ag4c`iGo=WfWyJ0Zw-SqF~S#j;3BKQLG1j60kL_3q-` z#X=cd+bjWxMUCM3SRuO`R~~oC%f||3cTMj>jq{OBBmp9jo$4|ft{MO^?LkmUpY=!? z2Cyfy)lU+3XnI}V+)T5p@X|^{5%Abk8uL%Fkmmqj71AZOwn$DXNchf*_Kc#`*= zUesBq?m1)L=u?_>-Rk2g8XB(+&E5Xm*3v>G_4GtJBt5Ic{mpTR*K$aD?6Pa}=TYbe z^(z44<^KNW!R}IbzFBs#L5#3N^Vn_P@Q4xtw;kJ9?L>~$ys_h^z2C4ueE6VN>Mpvo zu@HD(;munJKJ>3$Y1%|1700OGGWoHl75y;4NxAp6;w1lo&I)x-^DYKUb5cr!$KA`o ziA#*`9UyQhLGzlScGHv9HP~3 zzN<4&%hHkyP{5s}X*7ppyjAS>rM+x$0yftRs6}_47L-#cIMYI#vycqbN?e_S=Vxt& zf~a|q!vWR&%KWu4Yc%TKfoQIsBDbcq$g!UZ00RtK;(}2I80#=po@Lx_q9K0i5`ONG zGwYDYp_|A?j~<#!)N{>GG9CjGZPoiIg7=dsH#axJl6q4$J1;rQ@`z$J=T zA>(z==A%_PC2SJF=Hrz<37LngP}i#iDVc?Z`v+dT@9#VW_-EAhb}<)Kp+Zn z0!B*+(X;2yVSMT-K>$ehyfrHVfGr;*kfAt#k7A8F7zYSrmeZ3pg=juTKv-S)XLulL z1Z<|{zs}!vzt(LxSnwsOd;TF$6!ql#x^wYhyb9hEKwPsCxi=cK9-=FqCiXd))|=Ur z!?es3W2P+;7vfblDJ4yOrHT|~4WOeScU;FiOtK>dZ9aD5z1p&Ps-bDO&_@o%d21#y z^vgld9#9$-`Yw-FHZ_Qdine`yz&{%+m0SP{&%-O!D}a)LXF@Fo5vh80<7Y2vTwI(Z zoJHbOr*y@JQ!R!|wjd>~K0_+|B!U07SBOjq+ki!m3~^7eFmWo>Qk9(Gbu(ZE#QA?(tN*RKs$Vt@?X^N8k{ zS|=bNSbbt=ziLPXCaYEJ$)`@8vV@F0 zo){`tk=Y^^h{ks{rFPZYZ2OC-sAp}7a?Buw`4KVHpz;@p7r3FK89INlhXC0hxd zt51D=fK}3$4dmN^KH%`yWpgoa_G5Hwmaz)d)JOuqWYw>@ECKQGg?+BQ6j6^;^Kr0n z39H(b#DhV!vz_a}FFiaEe$A;Ja!KW5WFW!YG3vXTh4$6()5j$^XKrz0ssZ!|oSA~T zyi&_h3&0&k*Yc5E81QQ{u&O_P{75(LQR@jj z*5K zWTY1qv#2_=&T(^-Km`!|s1zRp`Vr&Vv&T?{GAl16D5wgOw-C+BbcCKrbuh zxC~|O?d`gQ&z}=(03D75@kTL5;EB`f)H^ugc~Ipr?gUl1t+mNfiMa}#A|*uyLsXFM z?d?T{6cQ>bqt!0;P8n(Gj+8Ao*qY^0KL(UYIR4D2JyOnw9I-%etOSU`-eopiA`0Zw zqJ%?NRu<>!iPK)u<^}k`W4vV(%OMt4fW9R_xpLeZ^=k@s1CpHnuo4HAme7O?`iUV%`t*g>s_gufqwMPcqldkwEF$~ z_jLDtXIH33GhJV!tM!wC08;|UM>Q<wY8xk z_B=8YPh4F5{f7@IBEvg=91Y;d?#-a^o?0s2q?Zk)56;UIK(#njU<8z8*~Hte)B5)6 zlpO?g$fr*i*B}qUNfB^fyNFV3NWyIQzf~2{U(WXU7>*)CF}Ix#X9t=BXHiqTG^n@Dt)TtM~h!ri~pgvsy^+OsIt*ZCe!>T|sYMe&} zv>&81pf#CkSpW#v9VvUl1jNeG**O68lNC7a0YJ$gWKUK-Bdq{!!-UEItMC(efIvBV zCuwO}+2fG*NEjG6wOkf>qc2=l)%x{u=L23}mN7Hp?~O~OF8VPqPYsj zGFJgz{Gug(?wpK<$NqW(kkE>ThMVYzKxb6`qE*~iq&Ru@Ifx$WyEfb?mxg7R*3o$j z$c>U&E+i{&_r^IS!Q=2?Dt`6XuPV{a0cJqM=vMeBCXu34@%yVw_smdMQw>dtrll92 zo(MD~7&0tJB~v!*fy+rlT9&r7OkZAJCI&X4JCJ8}F9JglxOU$DM!$@PM$Gcc3JEk$ zn7Fx>8zZ?Z=LjzaQgTuN=p#WNo}QkbNkBji7?L!bcFBOQXt5S2z5CX~7q4FF9|)6> zk}@wAttzItxd0N;{n?X|0F(@U%B}6~XB?UZ{!sfW$9~92;-NLuAEHUp=jqeq==xJo zoKMZ4qRucM6bf9A{JejeJVEz%96T3;+uDoHr8;ghM$@46Q=h8f$EPYGp@w zL0yNEi>|zthng-Q??XQ8M%j?wI&K<4IfK>JX4=L*EB@y2DB`jCXNFI!bxjZTSEuuY zbuoI7a6dOJEUdfA$jG4?y$2OIWSLYcV3~$-*avPE9%!Nmct{%=C-`!KjrpL4= zk;HrT`n4BULD$jU!6%NElJ_2;BpzGuOW)J1_EOsBl9>{mMZBq(IYJ+sbaz|v9yL_! zF2#06jL$B$r;G67BCUttxQeXwE?imPw6<*M2*o}Fy;;H7K|#aikpwc@@0?HWC5~jg zGqbWN6)?K8=wciZSdQ5$V|0ZY^hBGfsFQzh;nD5?3=by8)HW9ipmgu^{$@?eZH0JG4;~+czx;(g5a)xG+gRbLG;3=Iv(>o3GtL94`S zaX=6XA`Ss12b$dH)*Q2&n0Tmzp?4Hw{*(oh!M_LLGIg+LLiyo=h$sJH4^bG>=d-I< zzQtkGFEa8nTgohcIAkP;*%L+Y@nUXVmLn2hH1bb9eE6`gu1>---A3!a2Hq2(4_eM; zIR9Q0wm=e3Kc1y#jbTH#vK$jBJE@tlTrnfPH8+q$YekhCi@k^TNddQ`_XtgAVj7yj zcT#={K&eB_)r-+%3(DF5c5LS_|Kgg94eqf_N5taP8|nJ-(}sJVRmXehnazRXS_XKH z3#9nH^=@sWwghZ?#}SMHx6Q#HwKMC^@W0g-fLjR5(=67cY}NqGU|FoNdY@#uZeHq> zWHR|PlbuNo4X?4thf|X|yG^o*CDr!0lTF8~IZ8X+zIATwtkvJ+YaY7bVs_!}7*>jj zaBQ*C#(?78;r`Sg=nr}K236)uRC#GsXH~q#;>k`9?dXY59XcVBNQCdM-a%8n5e>;i zt608pE;~X*C**_n^FzgUE~1J$DuAG>o~;nwV%|7mbZC^Q3!g!AjB&Q>baO0Jd|U)7 zBeWrLKt-=>Y2i;J1ud|y?i#I#W2=S$ri8rw2}vob7jbbRAn>jJ+S+o2W-}8zd(&9c ztLM*=SVg~%jyr*Twm*AFoQvr}9h$OH0`DiPZu)O4sD&rLSJa2mzX| zMNdWL50{yknquPT?=BipcDn&U_x}CUKxp-}?!}8$S64HNiw`r=7%qhs=coV#CS(vh z!RRCS*AJlazOgiHu^7s_Rv*{@*@W+CosyD!^KQh?6~-#Y@{c7BPWIStT|wSHPN!Gl zAgbOb^?QO>yKX#O;%4b!E{rdo)cy`2B-&+YY@m0ZVMN!7hTlAEo9Y3xCtzdfbG_O= zds{99fp#Y;+RMGbAiSC@>*7XAA*uFg{HNQuZhIc+!er>-3fa!+G}rX*>ZdK-(b59& zZvPh8A%%8wOuo`apJHdGm9JRuW8u>$aDEQtx5ostu6-`)H`bJ;|Hz?G`S~T!)@)WJ zZ?dTRrR^zG+TNs!3dv)*xP|LyiD5tWbagF(;Ew^d=z?CSn3eiW-rn-Vf|C#7hSqPhEpE_T{N3^hZJ?BdKt+KRxsz zzR1Gj72UPUtNU*$cW61Ev@BBNww#u~Rk;4^_13|YA%B8Zhh5W-8Z&}2z|O~jpK)rr zZ$F|HutWkW*eihTdxAAYu)ug-mnZ7^ix=gj56pD_xMZ_I(-3Li&cb?+^TZX*B9Z0h zt!76nTRBIUdohxu&Ap#q^ebY{0<$bRJa9pKb*Q<;)RYZKx!(ix{OUFZX7CeH<0Rsz z?dK3Nd=Wl|%=9qfp;<2SqdTGb>CP%M0ttjl(E6-w2<_|Tbd|KGe~lVl3jf;NtoI?BiR|TTx=Qn zwr&pJYfk4au<5vru%X5#{vB)PsV@V8zBD`-2-JQqD3ShzD|V)p9GFK-?Vmy-c}843 zJ~PGik7ZD8nKCLD403Wvgq1C{U@OVbRgDc0S8b{3O-F1=&WGX4h0u=vWTE*N5;TWr zZl7bPM3aBk&&x&ttU{M&o+inim6%q}yA-VK!r$Bh9cPOsXm-T)I=C zg&B@E6v$gSQ@kPi?$b@)MRWr^yQ9lrTml@bPf3XLWSu z+lysV-hDc4nxYH0_XJ(|QlDbK4djZF2lK$>e-|YxbKAb3TjTPf#ZO`7pj#fCt}7m* zD>JP*%jxag?#*zp#spEoX#P{+V1SVhaQ*j7#&?XoTkhV?a?mK2s)mJ}!si_YA!7Fl(pcE8qyfvgp>ov;|_!GCw4 zMQGmpp|Zt75!32)QT3nr=Cf$DsGtlL@1e!s^EX&uWy?ubv#XOx3$$O3DJ}Ho(ZD@< z@%O&|7Y+$#8vq=NIHWJ1IW#MKw|7eVnA7;Y3r8&$R-c5;v*t~e$9#(c(?<`RK6V@H zBkX0o!xSchlTA=NuP4mdqclD8`UP_a>e;haHm+^f`8Bp6aNClQo*G+8tCP?{i^Qc~ zcK=#28Y9qi_xHy$x+0RSP4_P-`+Sau@tD}|1yat#Zq%EXa$#7(t7&um=0F?!j6NM2hsyXf}5&@x=V>$m`Fz;-!N4RVb^7EINEqy!?WcRYdh^?d!!dhS>`kh^m8Wh#>!2#@7Co zMECbvM*u3OBZoN&Mbr~9mwcP0yba~fk95VcI~DRKlM_f#Lc@lbl(h0chiBl!_Zt0E zTRQuUMH~HMR0^0G@7x^?T@BC5;_%MR^Bvoyu^VL@!8uAP1s#CTxb0>MzYIp;B2=B1 zPH}1vlN@N#wtr@IcekNPZ-9f)Q|!WTsM=fhFBAoq@lgsJO{{8#oJ3mm)Q$Ni%tO=> zKgp^Y;PT3c^ipdOG>LCip7rc{=&ojYlQQe^qDZ3ZKdTZj zNj|?)_wvr3+ZhJLQYzwN5tk52iIBUG%D&YSBNNl@&mM#m?lHW}! zXf21B2h9Box;3OS@8^mWsg`EbAJPk9i^oz}1mCQQp?MiovW1m5{OP0|qKP+r=N~R> zz^oc?iWbxDTK7GRg{ycNS)q7UUMsOAlvBfa?6$Ov{BuVvqwa}yOH8d)pi26=)ho!z z>p1q6pLNF^U4#xlF?^;xGQHw%+bmrd*->CojK5qq+<;|cKI8x5`U`Z4Fouq&VszND zMtvtRb#`%CJwMX<$31Iia#JZp?)Hlnl`aAWmkN|%qZ-EMR@mpbtHLSByTEU^|o_iaQ<3^4`0D zKJo*;(j~DF5{tv&SBbuOYa3-Qx%SnoiKjIGStOvu)c5b_<7cXV)Ej*&k4PlVoOhi5 zyYkhU9c2&`X*q)Chmb*foPM4_Mixsl99`9fXG9!{Lumb6hCRUFhXooi=*e>YzW)9i zRo}BTf*f5$?dgTR&zHn)&1{%EI!j{)idzdbW8xfHgrH-BnohyN%55=JBqXY6mgyfP z%bca&Egt$_?2lS$bfrQEQ|sUyUA%<|CNe`!8kB3X7(}7qN;@D_d%M9sF{i~s&T=>H zjA2V$HGA%W7m@fLF*M*q%oU6A8Nr2u)3y2s9aqGAvY+ppH#+FiSUv2v{{YRZ4!dKa z5N&yG7y`yW&IVC&V}ej+WvfD6rO$(!!{C(uH~0pAb(m@#4xRtmd`Z$jo-khVXNYlS zf4R#121D+E#hse37oNE8uXp#9xCw(KbUO)}N}_Ad#oNqE$;`oi1$$Yq|$CO)ax z|F-oyvCZi1#qTnlX=0jMIkca)?C-ke-%If&t{6M1+FKK->Hc|FKV@NkHtefu3~+}& zMTu&L$>asw4Y6Ju;k#}Yg*W1VX2*6cJ18maYDFpQbScfM6>VQc-nz(>HWpqw{i2*a z(=|3jDR|vId9OHecVclM2WkBNy#h6Hf^ZEU-$KRz$sE%19y)*z7gQItZ3%kVst{y) ze~{$My>Xj-nGsMN&s@2Jw(2jkv-5I>hK9=N=$wH*1KW^1K0bcytnD=gg={0K+7P;T z-SaQryfG0gl}zbm5)?cQ3;?(wR~qUDtg4a*L(>QdjZ>3Po;>k~%Rq+;3@1$ube&1y zR(%y7j_$$KqxD%84Gmgw-LikPf=7QWjE##s4!tyD$;iy#KL~&QTKDi^f9fDlGX7_N zY_h>Pi(B%Nz0kcoA=~{oEYBQ@kRkh(&ErGO?*!Z(IWa|IGL?ceV&R?}cY_s^Y>eW; z=Zfnvof9U3HbXQV2JI*=B!YcqH5b2F%fc=p`722oH2YB}i|eV+%yNjs;=nU_f!WWC#;2VYlfa&Am`oY0U&=#bG>k@Kx#^q z>T7IRH%2s~NB~Df3V+1Hyz#B6S3oa*u3%4h7kqWkp^epC^TLgVjSW>^A1Gm3^<$>z z<@Jp9VVB9yLC}Y8Khzl6d@YNynkZ;P9EQ{Emq*k7As! zECxNj>*@1ECV*tDc1=wxOgntrUuyV(CJ7A^CDZIDN9rNye@BW+(`&XM=V=Rlz=X>P zr>kA}q@&|Ck*qfXUNxGTp9u^SIn0GIjT%6MrW-s4_2=CwH^I2=hm^1;ILKNsQFS!` zIp&VxhH`f1WQHTjkoQ^(dj(+L@mucrS=87l;9XoArlvkgr1x@F_o&o70lR!lO8P%{ z!U2#W-Dp)A#7v^^-xZALXM(rKg%e#;h-dW%&PDI&@Xd|oWO^<<$pq)ThKkj1-AhTB zHC2%RlcvB_$bt{Cc&zsW*^RQ_08bKcVtU$py@>a({2Y^Fn8~_(A~oue-@Px2fGF52 zDb?IB)O_h)(pXDprZFBoCn~hn>sNq){|91@NkO5WAt+xvhx^`*UN%N;Rk*_g-2|I! zDZAV5sc(Wsxd2n3(05{^Ifh@SFIPCjy4wmV3&(Hba*)4pY};8V?~!z{Hwt?4v+3^p zMiw~=e^ZS0OS)ISHoCx_p~R|q?@UAu2!s~{yvoWX4$8m%voI_uma2a$-*`WY&|CS8 zP!;adzyNk^LUhNjpys*gnljkjOivn5bX{UpM)P?@a%?XgTocDY@(T298uJHLnnhn3hxOth?-OXz5dT$Q(gexUVffa}k3@$!hXb8huxNNkHq)agf2{7{dLn1``EOKN+A# zVcx~#pn+`THvLUvVlwshc<=!wU=(GTh|6OFQLaRlK(T}s#T|#&rnFCbKN0*5em|>d zWzdJZhXt&k-bOs%aY5{1h`{ao?)_HYCM`7Tok|5K8)!9-Z@sX^wF4`^e9-=NoL)S+bbkx*79`2+CU+~EV| zz{aEBZ?l^Y6R|ZS@pmv^^t$)XJtB*achntv1iyK}XD!fzMe>Y*zMaTjYlw*AyVg!w?Ac_y1(2A1mZnADE+8n{gTO zEKK#U2L9*W>|Kq;OAG9Xpy#iiL-Zf0n%ADMbn$z2r$t(Ah@K(w#P~DFB~J}y&kM+` z)b2xr<`V%CMj@D{Z``~o_%Hv~(0i1~$v-w3-@ejY!;E-xut)10fHvD!RhydD9+@`` zeH}pKAe9qFL}`1~2j=Rj+LIt7LkCulrbZ7OO*+Nr}TNy^3h=jt_UWS@(x(f5xUGw16T-(ofRU(-IL~5T9r`delIh()PMWqbp3j zyyP&-^haL?Xu!7EBo&%=H|RNKKa~>3A8HfH)Ae=D28w&`Dkwmb0NLJmS>fE@J);yQ zzUhXb$P8V|l3wQ%NGogVB_LNp%M#O~6ORctLFrDUN7>=0wns4+Yp}G<!c`?MC`o#lXbPk*8x9Kpw%pVS? z?Y9s=_fo;xoa8)bUx>HUIk>qF=mmKkcD3ehF>3#gz8H`1A76X}5*E ztbcqnl;DIpG&YpI7ru*mLr61cj?hAPhn}H}!eKMmCQw zCF*BZG>Ol);$y{;@m-&Ez z?a~k(7?RO(CUk@hRrlT9-Lu8C{-z3F6QkeV_#2~uH$C&wdHF;03+~s22@$FvC8wuC z6eLNtH^tB>Qe^M_w6NMwGEjhg;o#&Xg1g$9Pb+Or(=Ju-R&6JI|8aKxpNJ%t)M+RW zi`-c-*N+t=Qu^vOTiLIt8-LF{o*d$X_fg_U;2c)-Qe~a7f;Kn^__M}FUTcUzTR_xh zqfDPiu;5biU(ubRfcGS{o1tPWG?!_9%}LsCOQ+YZc*R4tN@!0?^W^UXIePZ_AqXV$ zV2O3xS-a{-NN1Xw*gX%pxk;RooE+m5)sNJeuGEXip$`fc$z;M*lPlC01FYK`Qj{Vh z*l}47xKy=MJ53mU(j6=B=B(&OXwQlDnz*>_=(GM-)k$wdCOiQ}J@btRBrMN$$IdKh5uVedl$GIU+ufBg8SB@Oyds87bq ziW__~jq^aMnZYiEjg2i}J>l=<1dL;BeEdapbl|{%#;Ud4`X!(j;0Er=Gz@jR@3sTJ z5I^89fqxMvpoIdiTUsjW#*O_TPqA355nkSM->RjHDwqn3GU zpsUMAYS&)aES~zY@ZCWRjSos2J&G^`H)U!o6f3n`g&!}13sf9z+hAm%1s8{usp%J6 z(LcHjw3vT(8G3~NzY~R|9n+*=*34pk&!0grFw{CsBIRR=7p$rkMXGb^rN54VP6ZG-pjLRe5L}M^yR||L>Y%(6oq0`sjOHX`P41YD6 z@j*D?JW8HWfsMcr6o25MO+0~dqo2~iC%T* zRx}l)rF$IH5mbE&t9EvDG2xo;Bc$r*{Vp;mC!LCd(pzLQRbHj_QN1=H4(;VL2r6#X69qEbK z70W?-#Xua^0?OA?`d6QG=8OJDh{Td-E?n^Y8woY{W^U`X;G<$v4xL~yVnI~oa+9*L zD92kfK-{q9BR5(Q|6UYHP!9fvBQcou}~HRCO0Mg_5iWUxmg{C0F5Yw-$; zpJp1^T>5=uXse}8<~KxBNoP(WMl7evGaP><+qXI`7mAVGM=SjVru$%cU~R;Lw3V@- zm#YZ3gXsm0sMbaRLrmenDd$y?OaA?K{r?HWMi!!lnOS;r+qBy6f3rG<9+H2URox4o z)tg*SEA~IXb6_F^{o;h7C(Q_y*on4Jb^62)#~d5uaSWvUUHLoWnv{fXxU3I`d%`t+ zbhMS&Vp{3i{c=msR>${cTAlrUVF=$5g`CMF9SwQ~0L$VI#7>4$ei->uLOb{XdpyE# zS68QS`DPbRwm~8eT0}n)fQ^aC`qY21*p;^@Kc0BzhYPpTCUdkB$K0>g*Qg99jUI^= zLPbt0JUcahf(7@bC*J?rO`yg9^qWLv>)Vtx`cJF=ta4Myng;3b&klpb=Wl{3oV6&2+i@^%jfm!YYewbnl$VwqD1L+AV-XMICM;kPo)Nx#nb>j$IQGXLg_rqrL8Ba4iL0bX5AQC_Y}&&eyah?yhW zwnia1nx|;hkyN#eioK;__uZ3MuU?&s50+&HQ*|&HItAZ8YOdAiVH6dmgCYIt+!T|e zD0s#1F5hdn)O)r8C?0p~edk?i9iPg;Z?&bJ>ANLZ9Ak)CkA zV?{UuzGYNVN{|J8rQ&2%WHApNPNKw}bRsl8DYhxl4qjohR!P2y7&g7RJH8YT&h48W zr=~x^o-i=9tg`&>@-}Pr#t0ggjW^hjY>Hk$5Y=;hwXAwCfsq-98VEtZ=jd2=$dGSh zf|nq(x&>-4SZANYa2ELMgJB7UKRx0dFFy9{)A>tsN_x<{)~Af~1101#bN_fwgPvnu zL|ysT_Scv5$jA5Z)%J1$>w(&TPh2e0lQfl4$jCJP)hy$Kiz93jt8Q+e*7eRN{GRmGUE&BT0Zeqf3VtJ2 zN+(0^@y?O1?5PSFwoq*Is0GV!b3`gf@1Dyi1T9DPx}rs*ypeWhxIqillfRt)IV*R# z3G$M^d|?ArD_C$^5lYiesfBJ?dVGm=#Ww#rML)f3_4iS` zql~HB!0;ZhftgjV)O{ZY1G_J#p@Z%NFuv#rUh~#WL#4)OexjQZ9Mxd;CZ?y?nji1v z3#>D`0&M}Ig44-WXbUt3JcaJhFo-x5%cm-n5QKlz=ovHx%lg}kFOUD;(U-w}u%lNY zqbkOUoQ9cPOz5pRMQNqg!aTN5eb@#5zWEZc^rDjhsP!L4jYi_|lQ2_A$uadK>F>gC$$PzZ7JIPh-TE@Gc89 zOM)#PT_z+Qh7r;J1RPa#cByIJ9l)%;Qxmw`&^b&ns`GQU8(l$X!qHKkRc`0vp@eJf zEG(F}R)RlT-wR8)2t(nZiBSmWLpLblB4u&Wj_IXdYXt2I!|PtyN;HkL)$tg0c%m8` zSo$3^8aVCZO|y%|+)n++rUV1lLUwcds7>k4@WLyPd!b;H8MnQDdLvFC5bHwAm|q7i zAT#MRL)6{)W^b^C?#7$+(|?=H&K?9+g`NFmeSJMpEsi#{DS;07{KF&4nP}uT>jyt4 zAf`XwZ8;_$?p9WtR=p#u&kNlWi~+k1lCj zaoqaJ>y$6-|Pf~Lz;EbI?GZxU& zkjK$c6qthzb(-QvL|}dl)gjW2+s~)e_E2L4g@n|6R!M4{CKj+h#&zzTFSF-MlCrs8 z0hjonf|z-3yY>;(7$O#2x@a3fx7L^FSs*VMVRa>4TjgVF7Al?9Tt8cHZia!B>*OSu zV=>%JB0r((p86?|L819t{JFmd@8Z?K8 zYcp-|{(|5^?aR>RQ~ycS1g7Gru1OGU}SRgQFf8FTc(mCRX__uXQ3nvk7!wT-clSTa;5L-MnAz6!|}zXr08!KpiyYH{7LTDtkX&tu7F;vqz3_LL( zEO-Zr7=3TUjT<+(tj4PTN?xFcfcO9H#Z(X0Mb0YTKh@&`=7ouTSdTdpP1L-dAMtv zQeALe`RcWx6BPk;r3dCWnAzAG_k_W=3Zp5q!`ET^20nFDpS7lt#2eUIId3X( z%M)}@=wKMy8lMiLt8G35UovdkB$*ot@ZET&sY?&>3_TDFGI*jDX{Eq4q!s&cJ*Kwh z71Mat{neIj!-m$w>})#|CFq0nxD@4et&5c;(%|GelyI6!ONUIfVse4m7I74%Pr$jz z!8;OGdf7@L39stu&coguy=(p_q;>Ug1&6jr&?sw}S3QV6uDrY{j<>dQCEnYi=o1;g zM>>95utEhe0CkZ?zw2&n?qbUCYe9t0-KNG>jH~_`zD~1HxEi1)Qev9aX${gOyAn}3 z2=obFZu9E{($b5HI!&{}q(7=BkmOh_8%YRX;9 zG5JftWl88~%rb@ufGP7o2pz3l_e;0bb+Bp6a+%`Va&l4=8D3?zPgKYgcg`lPe$Ic8 z{EO|wr_@HHZ!8ChPV@IQq0M(<)$yPc#L+}~J{FQ0i^$RdIc23MJ;#V{D%9rDOJO@2K)doNZ<0})BLq&^4IGJY-c-Tf@D7_Y5783%a|AGaAe4mYR zPf5Oomr20z;Uusn+HiD8gzBJ6NSbe@JXvl>@8;B{frEMjR6wG&dv zj=PZVR_W^B_L<|TuGoC_d#nJ&eFZSyCj>~9<-iDt9*h#AucY{pYmi6)iZM8|y+%fK znp2afZtI7+Oo>SD7ms}YN);{~=|HjRmVfd*2Cc9gJ!&sXDb?%=y#S4cUo5_TMdBNJ z-=8zlHelD)M%Th6*XJruV5tK&?`eFS-H)1V{& zlXQOMb^VIy-#r&)%DdXf`ztH41O){{%yE>1a~MQ}wKBU$HbVKMBTl;SbJ8|*avy%- zM$ zmenuv$kxJer(SY1$(6a2*D2KaI;(xsUbn=Oi_0^A*KUN1mHI$^KOB*DF@5J&3wEo$~t zVum)cO`W)`Rjtz^$jQ`sd6|#>U|0NRlXi2i$$`Pa=wIISUmRnlaXRQ+m0&VBl#sRf z_>-8mdpTQ~tj2CCL`dAC9~>^cc>4h9f6bnnu`S>4>$`_rb11|WTQA$r4Sr`*Qp6Zn zZ==V`nq#`U>&Obl-(ce_u8Oz&TWZ62M~&B(>VZ@-_b*n_`CGRaDz zh#q14d+pOF%UOSu>+X3f?Zi|+JHSJ!>v`)jr7mDJBgJi9^m(W?(U7!pl$Dv=M_%Zf z+w`S38ooZdPunQ-clI5@D-U}pdG>E;)bq|Bd7EvhkRK+mCVv{(%1EtHL$=vAG_yn`pXlDQ>d zc?tEf^zsb5b<(Qy`k4iXc~(z#OH&G`1Q+p1KW2vyvEhoxBL1FPYtOx*c3v|>7G5J& zO+jG~+*&aO_wiz4i+o#9e4i|e=Y%u)rp@~{2gDqhpvb!=jh6TyO=vLed1x?xh_#WW zf-WEZalOpNU)KxM?+6Z$9oj7E7@K8PdS%H~cCA+Nw8OvDz5iywbh;oVK?~1|QixDwfRcmL(+{miR7%Q=>I#R|kb}H^+rZ^v$#}Gq z!!9+ZqtPCQpp?%sPDAOR-yLK0vXV`5ci@Hq-!`yH2eN7`Tif-Hd&uVG#7HkaDrU6m z*p9G6hb!+6pFDDf$ftX*4isHF6aUH4JY5GmvFBS3YW7TlG%B$ z>)=e@x|*R75jRgwT6ZV*r8bFoyKS4jc1ocP(C7KmUoHH2X25tSw}hU=SwfceU~Cga!#rO(Ym}6mJ}3y@C6@1h|r+s-3gChJt@4K+%M; z#BIGa^OK>5W?Qrje7C1w44w?j`qa#aVRd(tYqadn6*~DDOoD!&bRBinKiN2#IKG<_ z=GTkWJ5RxGacx1$CVu8;`&C&!=-6*=8=sXv2M1B`Ro;iXE&Q?19w1QvWU+MN@piBp`x7XBi2-WDRkIuSxvbs+WuyWmyrJYtY5E|}1QF&8j zQBnDRrWy zH}-TN9~#O`UBTSJyvfy$gi>v@4Cm9rU5ibd)Qxc&1u9 zfc=O7-w7?I&(gDy2O49frCq{9LCw#V{BTN`I5`%l?``o@S=Q~vsY;uDbQ9NK&8{B; zN^X_km}$GGKd91ZZ;)9yCUw+*c1JQ31y;OAs7w>|bAe_0Q?@GfYb;-^#J)O1WvfkD$?lVbe+ z{0&~|Us5W%9)w96WZk^^u=(7w#c$Kx%*n-n8tck13WM;BFTQ0?K*VnZ}`~pujpS

G1UriaFQ8S}jKb#NbprVE-cw2g-qvd`9YyG^tw zIZGC)-z)j3r&Bw%`ik={p{)li)kL<(KUv(zWTKq$J6*62$8T|tQ_bfOzf$ix-LYD! z!u*Sc>`TpObsmOJwM;ZsM%nCCD@<12=y`{l6$@Xo7Pd&$WpbspJZO2Lb3J+oCjZ;D z4rt$dqOFw?laF&Pcb!Dv#N;^TsC!J-Ru17=Vg;*36Pj-+vPh?=aJ|=BvGCTdTVYp@ z&K;vsC;k4r845w~1Py`@uHfzgW*)f!&=f)o9Fz+B<$^}clmb3B->!3-ZFc@;cKp4z zzu9#Mr8oVx929SkGG|lW!wxeK?{(O9F79LIm&#c}iM&(p8$2r)aBE2g3wMWp&)Dbm zt&0WyKQ*{dtnh3x%q-e5o6tIAd-mCaS9T``sx|8BGf)AycbQBZ{JAA+iO-12;ERqG z6E2n&;Mr}M-h$*zw!OXaq!ak~1b)7*gE>?ZMz-&8UKH^*wy(n+6 zX3ZV5diTk{{7$~=UwukEyP+wj<;A+CCNxG9Ho5)#A6|Iy{)XD6?RLW(Upn}UrE2`T zhT|98(F)fu72!kAuibGwG}DM_JV3-4yS&)L6qUPSe)T4&CO$T&u`rpy_y*nwc6khaf? zlQ-EY{`iiAzFN~F5#xt#ShYuIGW~BQA9zc|^7!|WzngpZ?{8-1nhuYoQlRBacWpIs zTDrogTy6A~CCk3u?Pe=zMGw`pQtVENPG+{AU!L1!;0b%OyOWN3p7`jI#UH9G zSe4ii*d*6_?o+ehNJXn|22YyT@0q&ydNbY!Xs%AyC$p6skyWMa3YOeG0c55C9hK313YxAiKfLwNUj`;6M!`}gnbW}jH6 z79kfH5a34}gKj|0`C}XE$L5JIO2<7deJ5q|cTAV^$rN&Wo6)$WW;MDx%uY&)u(wEZ zm7bHkzt32#k?y|p@!NH)J zH1?e)QMY7dw@GBy6kTko&Qy(=l)gaIx-!H}Z}zb~4%{P!mXedYTQg}{GD+(xYf~>W zw{nF9hGTXPi;WlDeai7hG~wGu_IS2-SBma3*-U^6=* zW^ugyk0fc{o{ygP+6By?#vV(KkEPo3TMlRHxH{uO!xmyJ zc4K2Djf`G2B_Ss-uF>YA`M_48EozMq_oSw!^t4FkS#FOI+dN#CV*lLX#N0s}KKOc~ir0!Tk+kiLQ(o4z4HTtF z*5gAwP0aOs-RQI8$o=3bOSI))q|4)XGX1XSvfK-;uyT)jCM2uLwcqYt5cJ%drQ;rG zGib29D;N&OUuU6(Bl+hgoq<7qN1FI;=P3gYP@;S*My(CwqBkux_6VYxz2r$7VJ*X# z|JLuu2U#P7=|3ch6vESOaH;>L9RHOEq9;DI3O95(ubX0M(m6RT1?yjiM#ZlzIwn>N z>>fq1J6q>5&*rx1srDY6O{_|h``f+M_;Hlh$2+%_dY<<>8x}pZ-c>F9idjFdS^Vu` z_^Y4Yc5&}4J>s76UZ{0C{xIiBBF-#zEAVvyC2YG zxv>y(;4)J?$e8v&lDj66$6b2TA64@&j9D9-V+CAa-5svmi_1hk7o9t%gRUd|brdDR z-@#0YE2gKs1ugZo#9dYwixuw~-``j;<80C$$KGJ%Ue(p@Rh1*bS|!pJsAIdQ=x*`7 z!TvG7+$9%bJ?%haRVzTjN)fLE%~+btrq!Ly;kW$VR~> zGSjfJ|9G5vv2EAY@17XLtc4%;dR_%g%qX0zTRR{0&dfAf2fYPHe6XwuLhc*Zfd z?XJ}doT-w7B#+MB)m!VP9-KDKRWebXTEb|-Y17F;`2U=E;2bWn7J4zSe040JM_}~P zyM>(PJA->yTujb9BCefQZZoj*Y?Wl=!{e2Qf|2J$s@JLSl7HUyZ<5E>K99o<_~>4 z>N53QY>v>@$;z=*1*Eu#&oLrY2gfhdvDTz_>?-%W9f|Rx3r0Wb*_~c#&eo(R2oCO% z68vGWAv{c%f*_4Xp#JczB8Q<4SY@p?6N zZGh)p;DkBNVDgDpBRq-?E-{pFeThd;@9YWK8#fy+OYO~#`#0q9e%qX>+JP<2l@eTb zfwU<&CRe0!Q7_1nmaWu?f0O)Vv2o`o^dZBsHKqrBruA2N$(O{mY3O7YFV1v|KNYnO z6Jf~VT!k7I{rnwS<*!&YZKO=Ay&4!h8mJf#LV0`@#Ri6knx5y8$rA5NcE_JiuB3Y? z9lTmp+vCt|U6z-;)w{AlGSisy(C`w)P{gTNdf&p?H`FmNIVGh^FqnfVaS9>V-Zbju zC|nTOM(4eHz@>el-(=*ZV$gxGbYmD4I(Sh1XT9 zQYbUS`wo$|jgqo`fk8ON1aeu48;EaoL`%TO#ZvUcf#P36^<-@(;qVd{t_~6r;N!dP zU$VPUxZNbGVYud4^f6>-!vTx}<@j@-9bT+x*667C-=DN(yr| z|6TcGOP5yR~SwREbfo$j$U{ZaqR3q^K_3# zy>SZ-RQbJ!+G)DId=|R27dPd$$LG4;7fyrC|t8cu?;YY(QgU;p;v;n1MaEld z-}Ak|w|dLfJggzRb?b6+C`<5ufRpiq)K4gsr2c;An4|J?5s$ja%~@NC4lCu{~F~i%^X;At2s7A^8U5+`dX@^ZM)BQ+BbDQXkvM5(oyT4v%tIM@PFQ{noH*4r>wpI&14sFI>mfg>STU49~|WT5cVV1?RTp_3UFWa(q}yKjn0Ti8 zmg1i!!>t%l_WkWSdMu}gbys9U0wc&7VIJDc=7(kTN|!R)7gvEJC&2kqr%&(=PBEkr zo_6nNH&u5T$bL{mp|sn3m%-12ByV2ZmJ(Xj^d#?j?3^afnEC|RzuKXo@fHmHPP>~GR(0&M(2->q1t96 z&NkQ*p>R-?2>N(A*OZ1HEH}DqJH0#;qr3o-E=5#@mBGn9Xesnc2zk|)OY*CL9MWdhWPkdROzELi1bU{^r_8w}+z=2|yGe0~C@ zhz27;1=$o#Yh*0;GwSqf96J{4>gw*Em#wZfD-TJ5{Jk1paq(Kd1+cW-W7VoBzutb5wDlr=nV9BWoJl+LL3LLV4z{`s^y+SBpaM-l}l zH1Y)QFU0<4b=zuCkacCrsj93pC3Eg<*@O5R@lHt5T9~?ieE@LCjQ9>Nn)K?EXCHav z9%%(o*x1+reOm>vJjU2-n=h#ZQUFI>iIkie`*1)AM+zKHoFKTMPaj)2dE5toORO5` z>RKo&+SfO#1yRLVkJq-cg%+;-1m`mEEIH{(L-EJtFRC)C0NfXFZxr8Mq<(%4ss%a#l9|L2T;x=ljpr{C!f6>e#G9f%W*>=Ub zIV(|z(XB+hWGjDTXX9Hoi;ku&_G=PzPOWl*ANr_oOO##Fxg{*S8fKE^l$>o$zBmUr z_^CXb=#@#CxLIbe)3*I}F-k;Vt|Ad;8p#WRHH&%sUb()X^uQLOU9EWK7ZVx4gk9Pq)zmnY$c>ZGg2FTDKp3SYJOEohN`Ktl1n?vy$a(u$ zjkLmBclaq;Y!{BCbg7y0HWqW({6s;^_-k2LMy zPD^MQr{OrC2>pX9f1Z=&et({}=l=22&7CV8$?JnBXzqI3Pt)#8uIGo}1rIScqUs8~ z)_j(ZHai(6fU{MC6ll}BI zvb0%<`Hss&f~ zjUvaSI|2m@c-T>~VYPSWT}zis)RMk|4)p(cz(tv_PW)|kATQcQ58=p2>T?OB55p=g zg8J6U>JjHy=2zixQp@x=p2vQ1ectPjg&kB|UMZ>9VS#WpL3ZUPk@VuflCNc((FmO@ zM1UJfI3Q@Sr{(1dHJt>Rj_K$@k+y6HLBs;kQ1D`6<^4@tn+=OhMISCWNty!5cpDl< zNVU&mY<}n7y{Po`S9vnX>;kf2kBUl~%hq+&ia;+s8OX4C`_e&o-F1S3BP z&XrpCFrnj;b6>TKe!lj)2VmQWwt}{19-ViWWivHuj}voJqncKl{S~>_raNG5x}H}3 z-jS)l26D?c9}RlB3ZZ0bzoKIlYab2*JGsv!GH4Jy@!tSuK-rENX2wlA3#$cU!0aIk zV@t&DH&0m$@eBn~{y>@lm1NLE|7LExk5WgSM;I5%6zh%2HvvAwZGaK!+D;WH&0hcO z&fkz&MdKLDS_pO9VQe4bNy{PG5x1%{?ko)i6*qOl5a2>Ua?rx_g6wA3)^qXww z-?m4Qqclwn-+nHK1ux9#dH;-vsB+-=5zt26O6T46|FFs(LqbfETL5Ge zUhe%-4rO|USj>n|g?J@)4CJM?Rz*F~QviMHAP%o9_O1I?#E`ujwk0a(^s0|H+Mpoh zk^#Nz__3J--wn|DERA4TEeU%_GNH$+WNyCpb&rqn*Iz>`=lG6=@&<7QPeAQH4M#vb z;G<8xY3Ndr!hwj)2KO+dWFMns$yozU%>|JioT^<30)RqQTR;lF@#G0WNEP>Qa&sMh zeIT`ql7bjSBw*;d=e{HWQaF&Whx}v%{`FkS^EreizC_H_kgq?R-#9Tgm{)**b;W)~ zS|a~X{|ISN!NOd{$o6`Bj{3*S=%qhzBO@2Q5RqwLyiAfpA%J*j;}Ha~0(nHq6;F^y z$cbS_`oV_|J!(ihrK5nf0lxQH_$k0o39ZK+`r2i0;-Xm#?IdJYXePo~< zZiFdPe$Y?f&zmoE1kqPh_vJ>5+(*XpxGYdLBiw?Oi|a!j+N4+X#=5$zii(OX4U-YY zfep3@WkC7DNc?4Ua{+Lj?gRt>ekLRY3X$jB*}{RPWACr9=Ng<$JwDon#~zG!iKg2^ zeic304P7BQ>Z8Y^61&9BIuktNj!l+4Xh8&nxdfGBBp9xC^a`~IE3=+Qr>0W>e0Ay* zsxr&I%RB6`@B5P^#*N4aA%UUz{6Pfhv?FWdF_H`LQybjtp+fWcdVo-5=cLJ7g;2}Y zGBPrDP1c$1Gm|4ogA~L}QR6?@bThf>Mn8RejQ$yt=_T+bn;<|6kXl$+dn@MJoUNfh z96`?Yd?lcblBUABnnHWD8or;IP&=;}|E5aX2aBnk%&)x+tgAbB@+9;m z{%d1Tpcg1_QTLtu4?C!Tz;@0zBYK6)3aMz=N}q@}E*eJ;s`TtRVrnMwyjdHWx=ZEr+QQV$KjE^1_$P8wmC%huG3`NJ2#wZMqiExfMf>AzlB z_=KGb2@}K=@ODfLJ#~fMzw+l5_W9jmho*34em#nWJml^ake?eLiJ-^1jRSCl06^W9 zC5w4U7fYIN(C%4S-V7e=$H6FaLY1_&uQpX${PhED5Pi;FsGmu(hOyqZ)#EapLHyp! zu%fc8T-o)`To*Vx#|ciA#Gg8y*ma!F+>`NdZ<>i>X}`odM;V{1&rqNu+kg&6Jx`s5;JbW8rb(tZ8cpyDxL$0$ZZUsr3BK&;Yhm!&b9+D+N zr$IK2>oQ5`DFT;~MzILClpyqSCaATyyAI!p3hmy9EDLtzcb!9YC_AQA2yiY$JQxz- zzygSUjx^#yv@87U*ApoJc^Iprkgqm~cO2~U`$wzRBC<>TnMupyyhMX-9Ifzb( zMRI8*5?E1z6arFrAA7E{r0Pl%R0Eq0CrQz9_we9u;i{E(=w{rtYZq5bfC_AejL0G? z#uiat2G$1&n?bCybc$Io?ia?kxgK`};Ls^T4#>ta9_LIFtJYVy4EIaX#brB5`v(0G zf>b`_?y3M>JzryNt&%xQV*0?pSS4mm#;#p&>ZGo+kcjd^R7E~8nn21P=Y%9JlITh{ z@2JodzjyWYAPj{_AKA)6#jP&jIA=$URxA_P{q$m}G`J*K&yJ+L-SSyegkVLOE%ll z<(%yIJI}xlLNAJlSC8^7xSSx6o0-IGTZ+%)Zh5P6J1JZ93dY_zkXedfyacAm{>ZDB zZy=+V$|gZO+yCxv%r5lsBvXWh=^=*tB!0@0BU^64en@ISlIKaHVyV*$Jf@&r+f)@- zFN@3xm39&q*n4eXr!*PkS5z#dVAy{xwKO?Cg0k$O67?QB84!;c)3TE-wuV2ShR_h8%WLYC z9f@O)+z&@DZN+wR2GyNCCNd_a0UTu224suG_D~8jMBxVB)ADs{bV=2hFJCC$2wKV= zwNN^FQUY9ZgV;$zVC){Rm`?k22pXwNFWcgxv`r( z8H8v)+*QU3;Gvx07T9mGgfiKLnPk zXTUWj?k}6{E;g+4{p*WbzU1HuV9k~q-|@@`i_6mrD%p+i#MXb?(Em9-=7*`J!AnfU z%Ew3YHRV2k(%5$F<=ej=3b5|?_+gU#v}+l-EF@w8(T+>dm_#CG%LSRfB6a&$EF5?^ zy$J!wOb9`JTvB54I@PSZEg4S&L>SP;=WPV83&TkamQaC~;JGK97X)Z1?oTi}>$^t; zv++y%6G)VPgUb)S9Pu`nBOct^@H*MP?y}7ubzlXz;ic zqq6M#G=rxc7z9k_=jHwF0|RZV40-u1#W1*k%SNoYob-Vs*Xz@|NXd^-r%{ZmA=N=OH2o{zB6Y&I=rJl>kefyH7Gg=BO<2TJY$ ziE$-OFR_P|bn3%cj&kZSem0yvK7pT#t9lvV?Spe4jic5fG?>^wm*9VqViPqAKmsvk zrO%)2_|N=P<9zrE^3j)4qmiE=?2!{rW*(cI_X0?^BKO5fLDJXPGTJvzFr9@pd%fcR zLF7Xn8qW1$VU?m)1}<%Gyh0tycecM<#KIa&Q+0tC#wpf+J$9K(Ir{HCt|PN%C(ay` UOWJUTf`5KjR8vUWWqjuU0UdM>?*IS* diff --git a/docs/2.0/_modules/torch/cuda/amp/autocast_mode.html b/docs/2.0/_modules/torch/cuda/amp/autocast_mode.html index c233fe991454..dd4a059ce9c2 100644 --- a/docs/2.0/_modules/torch/cuda/amp/autocast_mode.html +++ b/docs/2.0/_modules/torch/cuda/amp/autocast_mode.html @@ -504,7 +504,7 @@

Source code for torch.cuda.amp.autocast_mode

if isinstance(value, torch.Tensor):
         is_eligible = (value.is_floating_point() and value.is_cuda and (value.dtype is not torch.float64))
         return value.to(dtype) if is_eligible else value
-    elif isinstance(value, str):
+    elif isinstance(value, (str, bytes)):
         return value
     elif HAS_NUMPY and isinstance(value, np.ndarray):
         return value
diff --git a/docs/2.0/_modules/torch/distributed/distributed_c10d.html b/docs/2.0/_modules/torch/distributed/distributed_c10d.html
index 5e0538df744b..7694541ae08a 100644
--- a/docs/2.0/_modules/torch/distributed/distributed_c10d.html
+++ b/docs/2.0/_modules/torch/distributed/distributed_c10d.html
@@ -683,7 +683,6 @@ 

Source code for torch.distributed.distributed_c10d

def __init__(self, backend: Union[str, Backend]): self.device_backend_map: Dict[torch.device, Backend] = {} - # error check to make sure the config string is valid # Cases for when backend is a single string (without device types) if backend == Backend.UNDEFINED: @@ -700,13 +699,24 @@

Source code for torch.distributed.distributed_c10d

"cuda": backend_val, } else: - # custom backend string in format of "{device_type1}:{backend1},{device_type2}:{backend2}" - # TODO - pass - - required_devices = ["cpu", "cuda"] - for device in required_devices: - assert device in self.device_backend_map + # make sure the backend string is in the correct format + # "{device_type1}:{backend1},{device_type2}:{backend2}" + # e.g. "cpu:gloo,cuda:nccl" + backend_str_error_message = f"""The custom backend string argument is invalid: {backend}. + Custom backend string is an experimental feature where the backend string must be in the format: + "<device_type1>:<backend1>,<device_type2>:<backend2>...". e.g. 'cpu:gloo,cuda:nccl'""" + + # parse the backend string and populate the device_backend_map + for device_backend_pair_str in backend.lower().split(","): + device_backend_pair = device_backend_pair_str.split(":") + if len(device_backend_pair) != 2: + raise ValueError(f"Invalid device:backend pairing: \ + {device_backend_pair_str}. {backend_str_error_message}") + device, backend = device_backend_pair + if device in self.device_backend_map: + raise ValueError(f"Duplicate device type {device} \ + in backend string: {backend}. {backend_str_error_message}") + self.device_backend_map[device] = Backend(backend) def __repr__(self): # string with all the device:backend pairs separared by commas @@ -1293,7 +1303,9 @@

Source code for torch.distributed.distributed_c10d

.. note:: Support for multiple backends is experimental. Currently when no backend is specified, both ``gloo`` and ``nccl`` backends will be created. The ``gloo`` backend will be used for collectives with CPU tensors and the ``nccl`` backend will be used - for collectives with CUDA tensors. + for collectives with CUDA tensors. A custom backend can be specified by passing in + a string with format "<device_type>:<backend_name>,<device_type>:<backend_name>", e.g. + "cpu:gloo,cuda:custom_backend". """ global _world @@ -1444,6 +1456,9 @@

Source code for torch.distributed.distributed_c10d

backend_type = ProcessGroup.BackendType.MPI if not backend_class: return GroupMember.NON_GROUP_MEMBER + # create new process group with accurate rank and size + if pg.rank() == -1 and pg.size() == -1: + pg = ProcessGroup(backend_prefix_store, backend_class.rank(), backend_class.size(), base_pg_options) elif backend_str == Backend.GLOO: # TODO: remove this check after lazy initialization is supported # if pg_options is not None: @@ -1527,15 +1542,15 @@

Source code for torch.distributed.distributed_c10d

timeout=timeout, ) - # only create single backend pg when backend is set to gloo, nccl, mpi, and ucc - if backend in [Backend.GLOO, Backend.NCCL, Backend.UCC, Backend.MPI]: + # register only a single backend when all get_device_backend_map values are the same + if len(set(backend_config.get_device_backend_map().values())) == 1: for device in backend_config.get_device_backend_map().keys(): pg._register_backend(torch.device(device), backend_type, backend_class) # break out of outer loop to not create any more backends break - else: - pg._register_backend(torch.device(device), backend_type, backend_class) + + pg._register_backend(torch.device(device), backend_type, backend_class) # update global state _world.pg_map[pg] = (backend, prefix_store) diff --git a/docs/2.0/_modules/torch/nn/functional.html b/docs/2.0/_modules/torch/nn/functional.html index 0fdcfaeb1ba0..1737bcf7a48c 100644 --- a/docs/2.0/_modules/torch/nn/functional.html +++ b/docs/2.0/_modules/torch/nn/functional.html @@ -5513,11 +5513,21 @@

Source code for torch.nn.functional

             be ignored by the attention. This is an binary mask. When the value is True,
             the corresponding value on the attention layer will be filled with -inf.
         need_weights: output attn_output_weights.
+            Default: `True`
+            Note: `needs_weight` defaults to `True`, but should be set to `False`
+            For best performance when attention weights are not nedeeded.
+            *Setting needs_weights to `True`
+            leads to a significant performance degradation.*
         attn_mask: 2D or 3D mask that prevents attention to certain positions. A 2D mask will be broadcasted for all
             the batches while a 3D mask allows to specify a different mask for the entries of each batch.
         is_causal: If specified, applies a causal mask as attention mask, and ignores
             attn_mask for computing scaled dot product attention.
             Default: ``False``.
+            .. warning::
+                is_causal is provides a hint that the attn_mask is the
+                causal mask.Providing incorrect hints can result in
+                incorrect execution, including forward and backward
+                compatibility.
         use_separate_proj_weight: the function accept the proj. weights for query, key,
             and value in different forms. If false, in_proj_weight will be used, which is
             a combination of q_proj_weight, k_proj_weight, v_proj_weight.
@@ -5617,8 +5627,33 @@ 

Source code for torch.nn.functional

         target_type=query.dtype
     )
 
-    if is_causal:
+    if is_causal and attn_mask is None:
+        raise RuntimeError(
+            "Need attn_mask if specifying the is_causal hint. "
+            "You may use the Transformer module method "
+            "`generate_square_subsequent_mask` to create this mask."
+        )
+
+    if is_causal and key_padding_mask is None and not need_weights:
+        # when we have a kpm or need weights, we need attn_mask
+        # Otherwise, we use the is_causal hint go as is_causal
+        # indicator to SDPA.
         attn_mask = None
+    else:
+        attn_mask = _canonical_mask(
+            mask=attn_mask,
+            mask_name="attn_mask",
+            other_type=None,
+            other_name="",
+            target_type=query.dtype,
+            check_other=False,
+        )
+
+        if key_padding_mask is not None:
+            # We have the attn_mask, and use that to merge kpm into it.
+            # Turn off use of is_causal hint, as the merged mask is no
+            # longer causal.
+            is_causal = False
 
     assert embed_dim == embed_dim_to_check, \
         f"was expecting embedding dimension of {embed_dim_to_check}, but got {embed_dim}"
@@ -5748,6 +5783,9 @@ 

Source code for torch.nn.functional

     if need_weights:
         B, Nt, E = q.shape
         q_scaled = q / math.sqrt(E)
+
+        assert not (is_causal and attn_mask is None), "FIXME: is_causal not implemented for need_weights"
+
         if attn_mask is not None:
             attn_output_weights = torch.baddbmm(attn_mask, q_scaled, k.transpose(-2, -1))
         else:
diff --git a/docs/2.0/_modules/torch/nn/modules/activation.html b/docs/2.0/_modules/torch/nn/modules/activation.html
index dac76c8328b2..d152507f86df 100644
--- a/docs/2.0/_modules/torch/nn/modules/activation.html
+++ b/docs/2.0/_modules/torch/nn/modules/activation.html
@@ -1353,9 +1353,8 @@ 

Source code for torch.nn.modules.activation

 
     where :math:`head_i = \text{Attention}(QW_i^Q, KW_i^K, VW_i^V)`.
 
-    ``forward()`` will use the optimized implementation described in
-    `FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness`_ if all of the following
-    conditions are met:
+    ``nn.MultiHeadAttention`` will use the optimized implementations of
+    ``scaled_dot_product_attention()`` when possible.
 
     - self attention is being computed (i.e., ``query``, ``key``, and ``value`` are the same tensor. This
       restriction will be loosened in the future.)
@@ -1510,8 +1509,13 @@ 

Source code for torch.nn.modules.activation

             corresponding position is not allowed to attend. For a float mask, the mask values will be added to
             the attention weight.
             If both attn_mask and key_padding_mask are supplied, their types should match.
-        is_causal: If specified, applies a causal mask as attention mask. Mutually exclusive with providing attn_mask.
+        is_causal: If specified, applies a causal mask as attention mask.
             Default: ``False``.
+            Warning:
+            ``is_causal`` provides a hint that ``attn_mask`` is the
+            causal mask. Providing incorrect hints can result in
+            incorrect execution, including forward and backward
+            compatibility.
         average_attn_weights: If true, indicates that the returned ``attn_weights`` should be averaged across
             heads. Otherwise, ``attn_weights`` are provided separately per head. Note that this flag only has an
             effect when ``need_weights=True``. Default: ``True`` (i.e. average weights across heads)
@@ -1530,8 +1534,6 @@ 

Source code for torch.nn.modules.activation

         .. note::
             `batch_first` argument is ignored for unbatched inputs.
         """
-        if attn_mask is not None and is_causal:
-            raise AssertionError("Only allow causal mask or attn_mask")
 
         is_batched = query.dim() == 3
 
@@ -1684,20 +1686,27 @@ 

Source code for torch.nn.modules.activation

             check_other=False,
         )
 
-        if attn_mask is not None:
-            mask_type = 0
-            merged_mask = attn_mask
         if key_padding_mask is not None:
             mask_type = 1
             merged_mask = key_padding_mask
-        if (attn_mask is not None) and (key_padding_mask is not None):
+
+        if attn_mask is not None:
             # In this branch query can't be a nested tensor, so it has a shape
             batch_size, seq_len, _ = query.shape
             mask_type = 2
-            key_padding_mask_expanded = key_padding_mask.view(batch_size, 1, 1, seq_len) \
-                                                        .expand(-1, self.num_heads, -1, -1)
-            attn_mask_expanded = attn_mask.view(1, 1, seq_len, seq_len).expand(batch_size, self.num_heads, -1, -1)
-            merged_mask = attn_mask_expanded + key_padding_mask_expanded
+
+            # Always expands attn_mask to 4D
+            if attn_mask.dim() == 3:
+                attn_mask_expanded = attn_mask.view(batch_size, -1, seq_len, seq_len)
+            else:  # attn_mask.dim() == 2:
+                attn_mask_expanded = attn_mask.view(1, 1, seq_len, seq_len).expand(batch_size, self.num_heads, -1, -1)
+            merged_mask = attn_mask_expanded
+
+            if key_padding_mask is not None:
+                key_padding_mask_expanded = key_padding_mask.view(batch_size, 1, 1, seq_len).expand(-1, self.num_heads, -1, -1)
+                merged_mask = attn_mask_expanded + key_padding_mask_expanded
+
+        # no attn_mask and no key_padding_mask, returns None, None
         return merged_mask, mask_type
diff --git a/docs/2.0/_modules/torch/nn/modules/transformer.html b/docs/2.0/_modules/torch/nn/modules/transformer.html index 4ea7205f7fff..02cb886a2e77 100644 --- a/docs/2.0/_modules/torch/nn/modules/transformer.html +++ b/docs/2.0/_modules/torch/nn/modules/transformer.html @@ -1022,21 +1022,21 @@

Source code for torch.nn.modules.transformer

x = src
         if self.norm_first:
-            x = x + self._sa_block(self.norm1(x), src_mask, src_key_padding_mask)
+            x = x + self._sa_block(self.norm1(x), src_mask, src_key_padding_mask, is_causal=is_causal)
             x = x + self._ff_block(self.norm2(x))
         else:
-            x = self.norm1(x + self._sa_block(x, src_mask, src_key_padding_mask))
+            x = self.norm1(x + self._sa_block(x, src_mask, src_key_padding_mask, is_causal=is_causal))
             x = self.norm2(x + self._ff_block(x))
 
         return x
# self-attention block def _sa_block(self, x: Tensor, - attn_mask: Optional[Tensor], key_padding_mask: Optional[Tensor]) -> Tensor: + attn_mask: Optional[Tensor], key_padding_mask: Optional[Tensor], is_causal: bool = False) -> Tensor: x = self.self_attn(x, x, x, attn_mask=attn_mask, key_padding_mask=key_padding_mask, - need_weights=False)[0] + need_weights=False, is_causal=is_causal)[0] return self.dropout1(x) # feed forward block diff --git a/docs/2.0/_modules/torch/optim/adam.html b/docs/2.0/_modules/torch/optim/adam.html index d9c8680f64e7..885d39ff2447 100644 --- a/docs/2.0/_modules/torch/optim/adam.html +++ b/docs/2.0/_modules/torch/optim/adam.html @@ -984,7 +984,6 @@

Source code for torch.optim.adam

     capturable: bool,  # Needed for consistency.
     differentiable: bool,
 ) -> None:
-    grouped_tensors = _group_tensors_by_device_and_dtype([params, grads, exp_avgs, exp_avg_sqs, max_exp_avg_sqs, state_steps])
     grad_scale_dict = {grad_scale.device: grad_scale} if grad_scale is not None else None
     found_inf_dict = {found_inf.device: found_inf} if found_inf is not None else None
     grouped_tensors = _group_tensors_by_device_and_dtype([params, grads, exp_avgs, exp_avg_sqs, max_exp_avg_sqs, state_steps])
@@ -997,16 +996,15 @@ 

Source code for torch.optim.adam

             device_max_exp_avg_sqs,
             device_state_steps,
         ) = grouped_tensors[(device, dtype)]
-        if grad_scale is not None and found_inf is not None:
+        device_grad_scale, device_found_inf = None, None
+        if grad_scale is not None:
             if device not in grad_scale_dict:
                 grad_scale_dict[device] = grad_scale.to(device, non_blocking=True)
+            device_grad_scale = grad_scale_dict[device]
+        if found_inf is not None:
             if found_inf not in found_inf_dict:
                 found_inf_dict[device] = found_inf.to(device, non_blocking=True)
-            device_grad_scale = grad_scale_dict[device]
             device_found_inf = found_inf_dict[device]
-        else:
-            device_grad_scale = None
-            device_found_inf = None
         torch._foreach_add_(device_state_steps, 1)
         torch._fused_adam_(
             device_params,
diff --git a/docs/2.0/_modules/torch/optim/adamw.html b/docs/2.0/_modules/torch/optim/adamw.html
index 204bf9a06a32..5506ca655aa2 100644
--- a/docs/2.0/_modules/torch/optim/adamw.html
+++ b/docs/2.0/_modules/torch/optim/adamw.html
@@ -1057,16 +1057,15 @@ 

Source code for torch.optim.adamw

             device_max_exp_avg_sqs,
             device_state_steps,
         ) = grouped_tensors[(device, dtype)]
-        if grad_scale is not None and found_inf is not None:
+        device_grad_scale, device_found_inf = None, None
+        if grad_scale is not None:
             if device not in grad_scale_dict:
                 grad_scale_dict[device] = grad_scale.to(device, non_blocking=True)
+            device_grad_scale = grad_scale_dict[device]
+        if found_inf is not None:
             if found_inf not in found_inf_dict:
                 found_inf_dict[device] = found_inf.to(device, non_blocking=True)
-            device_grad_scale = grad_scale_dict[device]
             device_found_inf = found_inf_dict[device]
-        else:
-            device_grad_scale = None
-            device_found_inf = None
         torch._foreach_add_(device_state_steps, 1)
         torch._fused_adamw_(
             device_params,
diff --git a/docs/2.0/_modules/torch/serialization.html b/docs/2.0/_modules/torch/serialization.html
index 828eefb3877e..37b75a1bf0a2 100644
--- a/docs/2.0/_modules/torch/serialization.html
+++ b/docs/2.0/_modules/torch/serialization.html
@@ -1533,7 +1533,7 @@ 

Source code for torch.serialization

         def restore_location(storage, location):
             location = map_location.get(location, location)
             return default_restore_location(storage, location)
-    elif isinstance(map_location, str):
+    elif isinstance(map_location, (str, bytes)):
         def restore_location(storage, location):
             return default_restore_location(storage, map_location)
     elif isinstance(map_location, torch.device):
diff --git a/docs/2.0/_modules/torch/storage.html b/docs/2.0/_modules/torch/storage.html
index 808613bfc3fe..9f1ee809d93e 100644
--- a/docs/2.0/_modules/torch/storage.html
+++ b/docs/2.0/_modules/torch/storage.html
@@ -761,14 +761,37 @@ 

Source code for torch.storage

     else:
         return isinstance(x, int)
 
+_always_warn_typed_storage_removal = False
+
+def _get_always_warn_typed_storage_removal():
+    return _always_warn_typed_storage_removal
+
+def _set_always_warn_typed_storage_removal(always_warn):
+    global _always_warn_typed_storage_removal
+    assert isinstance(always_warn, bool)
+    _always_warn_typed_storage_removal = always_warn
+
 def _warn_typed_storage_removal(stacklevel=2):
-    message = (
-        "TypedStorage is deprecated. It will be removed in the future and "
-        "UntypedStorage will be the only storage class. This should only matter "
-        "to you if you are using storages directly.  To access UntypedStorage "
-        "directly, use tensor.untyped_storage() instead of tensor.storage()"
-    )
-    warnings.warn(message, UserWarning, stacklevel=stacklevel + 1)
+    global _always_warn_typed_storage_removal
+
+    def is_first_time():
+        if not hasattr(_warn_typed_storage_removal, 'has_warned'):
+            return True
+        else:
+            return not _warn_typed_storage_removal.__dict__['has_warned']
+
+    if _get_always_warn_typed_storage_removal() or is_first_time():
+        message = (
+            "TypedStorage is deprecated. It will be removed in the future and "
+            "UntypedStorage will be the only storage class. This should only matter "
+            "to you if you are using storages directly.  To access UntypedStorage "
+            "directly, use tensor.untyped_storage() instead of tensor.storage()"
+        )
+        warnings.warn(message, UserWarning, stacklevel=stacklevel + 1)
+        _warn_typed_storage_removal.__dict__['has_warned'] = True
+
+def _reset_warn_typed_storage_removal():
+    _warn_typed_storage_removal.__dict__['has_warned'] = False
 
 
[docs]class TypedStorage: is_sparse = False diff --git a/docs/2.0/_modules/torch/utils/data/_utils/collate.html b/docs/2.0/_modules/torch/utils/data/_utils/collate.html index 87f898a5af4e..e3907f7d8d88 100644 --- a/docs/2.0/_modules/torch/utils/data/_utils/collate.html +++ b/docs/2.0/_modules/torch/utils/data/_utils/collate.html @@ -524,7 +524,7 @@

Source code for torch.utils.data._utils.collate

< return elem_type(*(default_convert(d) for d in data)) elif isinstance(data, tuple): return [default_convert(d) for d in data] # Backwards compatibility. - elif isinstance(data, collections.abc.Sequence) and not isinstance(data, str): + elif isinstance(data, collections.abc.Sequence) and not isinstance(data, (str, bytes)): try: return elem_type([default_convert(d) for d in data]) except TypeError: @@ -653,6 +653,7 @@

Source code for torch.utils.data._utils.collate

< default_collate_fn_map[float] = collate_float_fn default_collate_fn_map[int] = collate_int_fn default_collate_fn_map[str] = collate_str_fn +default_collate_fn_map[bytes] = collate_str_fn
[docs]def default_collate(batch): diff --git a/docs/2.0/_modules/torch/utils/data/dataloader.html b/docs/2.0/_modules/torch/utils/data/dataloader.html index 8faf4e767bf3..a9ec570aff46 100644 --- a/docs/2.0/_modules/torch/utils/data/dataloader.html +++ b/docs/2.0/_modules/torch/utils/data/dataloader.html @@ -489,7 +489,6 @@

Source code for torch.utils.data.dataloader

     Dataset,)
 
 from torch.utils.data.datapipes.datapipe import _IterDataPipeSerializationWrapper, _MapDataPipeSerializationWrapper
-from torch.utils.data.datapipes.iter.sharding import SHARDING_PRIORITIES
 
 from . import _utils
 
@@ -559,6 +558,7 @@ 

Source code for torch.utils.data.dataloader

 
 
 def _sharding_worker_init_fn(worker_init_fn, world_size, rank_id, worker_id):
+    global_worker_id = worker_id
     info = torch.utils.data.get_worker_info()
     assert info is not None
     total_workers = info.num_workers
@@ -566,10 +566,10 @@ 

Source code for torch.utils.data.dataloader

     assert isinstance(datapipe, (IterDataPipe, MapDataPipe))
     # To distribute elements across distributed process evenly, we should shard data on distributed
     # processes first then shard on worker processes
-    torch.utils.data.graph_settings.apply_sharding(
-        datapipe, world_size, rank_id, sharding_group=SHARDING_PRIORITIES.DISTRIBUTED)
-    torch.utils.data.graph_settings.apply_sharding(
-        datapipe, total_workers, worker_id, sharding_group=SHARDING_PRIORITIES.MULTIPROCESSING)
+    total_workers *= world_size
+    global_worker_id = global_worker_id * world_size + rank_id
+    # For BC, use default SHARDING_PRIORITIES
+    torch.utils.data.graph_settings.apply_sharding(datapipe, total_workers, global_worker_id)
     if worker_init_fn is not None:
         worker_init_fn(worker_id)
 
@@ -857,8 +857,7 @@ 

Source code for torch.utils.data.dataloader

                             ('multiprocessing_context option '
                              'should specify a valid start method in {!r}, but got '
                              'multiprocessing_context={!r}').format(valid_start_methods, multiprocessing_context))
-                    # error: Argument 1 to "get_context" has incompatible type "Union[str, bytes]"; expected "str"  [arg-type]
-                    multiprocessing_context = multiprocessing.get_context(multiprocessing_context)  # type: ignore[arg-type]
+                    multiprocessing_context = multiprocessing.get_context(multiprocessing_context)
 
                 if not isinstance(multiprocessing_context, python_multiprocessing.context.BaseContext):
                     raise TypeError(('multiprocessing_context option should be a valid context '
@@ -1122,8 +1121,8 @@ 

Source code for torch.utils.data.dataloader

         # Adds forward compatibilities so classic DataLoader can work with DataPipes:
         #   Taking care of distributed sharding
         if isinstance(self._dataset, (IterDataPipe, MapDataPipe)):
-            torch.utils.data.graph_settings.apply_sharding(
-                self._dataset, self._world_size, self._rank, sharding_group=SHARDING_PRIORITIES.DISTRIBUTED)
+            # For BC, use default SHARDING_PRIORITIES
+            torch.utils.data.graph_settings.apply_sharding(self._dataset, self._world_size, self._rank)
 
         self._dataset_fetcher = _DatasetKind.create_fetcher(
             self._dataset_kind, self._dataset, self._auto_collation, self._collate_fn, self._drop_last)
diff --git a/docs/2.0/_sources/notes/extending.func.rst.txt b/docs/2.0/_sources/notes/extending.func.rst.txt
index b3a044f8c2f4..21603ba1becf 100644
--- a/docs/2.0/_sources/notes/extending.func.rst.txt
+++ b/docs/2.0/_sources/notes/extending.func.rst.txt
@@ -37,8 +37,13 @@ Only the latter is supported with function transforms:
   (by calling ``ctx.save_for_backward(*tensors)``), or save non-Tensors
   (by assigning them to the ``ctx`` object).
 
-Any intermediates that need to be saved must be returned as an output from
-:meth:`~Function.forward`.
+Because :meth:`~Function.setup_context` accepts only ``inputs`` and ``output``,
+the only quantities that can be saved are either objects (such as Tensors) in
+the inputs or outputs or quantities (like ``Tensor.shape``) derived from them.
+If you wish to save a non-input intermediate activation from
+:meth:`Function.forward` for backward, then you'll need to return it as an
+output from :meth:`~Function.forward` so that it gets passed to
+:meth:`~Function.setup_context`.
 
 Depending on the transform,
 
diff --git a/docs/2.0/distributed.html b/docs/2.0/distributed.html
index 52d99629322a..76e14e919f3e 100644
--- a/docs/2.0/distributed.html
+++ b/docs/2.0/distributed.html
@@ -800,7 +800,9 @@ 

Initializationgloo and nccl backends will be created. The gloo backend will be used for collectives with CPU tensors and the nccl backend will be used -for collectives with CUDA tensors.

+for collectives with CUDA tensors. A custom backend can be specified by passing in +a string with format “<device_type>:<backend_name>,<device_type>:<backend_name>”, e.g. +“cpu:gloo,cuda:custom_backend”.

diff --git a/docs/2.0/generated/torch.nn.MultiheadAttention.html b/docs/2.0/generated/torch.nn.MultiheadAttention.html index c398bca93d78..33e9c36bad23 100644 --- a/docs/2.0/generated/torch.nn.MultiheadAttention.html +++ b/docs/2.0/generated/torch.nn.MultiheadAttention.html @@ -469,9 +469,8 @@

MultiheadAttentionMultiHead(Q,K,V)=Concat(head1,,headh)WO\text{MultiHead}(Q, K, V) = \text{Concat}(head_1,\dots,head_h)W^O

where headi=Attention(QWiQ,KWiK,VWiV)head_i = \text{Attention}(QW_i^Q, KW_i^K, VW_i^V).

-

forward() will use the optimized implementation described in -FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness if all of the following -conditions are met:

+

nn.MultiHeadAttention will use the optimized implementations of +scaled_dot_product_attention() when possible.

  • self attention is being computed (i.e., query, key, and value are the same tensor. This restriction will be loosened in the future.)

  • @@ -549,8 +548,13 @@

    MultiheadAttentionbool) – If specified, applies a causal mask as attention mask. Mutually exclusive with providing attn_mask. -Default: False.

    +
  • is_causal (bool) – If specified, applies a causal mask as attention mask. +Default: False. +Warning: +is_causal provides a hint that attn_mask is the +causal mask. Providing incorrect hints can result in +incorrect execution, including forward and backward +compatibility.

  • average_attn_weights (bool) – If true, indicates that the returned attn_weights should be averaged across heads. Otherwise, attn_weights are provided separately per head. Note that this flag only has an effect when need_weights=True. Default: True (i.e. average weights across heads)

  • diff --git a/docs/2.0/notes/extending.func.html b/docs/2.0/notes/extending.func.html index 2aa252938bed..2d7fca29adb3 100644 --- a/docs/2.0/notes/extending.func.html +++ b/docs/2.0/notes/extending.func.html @@ -483,8 +483,13 @@

    Basic Usagectx.save_for_backward(*tensors)), or save non-Tensors (by assigning them to the ctx object).

-

Any intermediates that need to be saved must be returned as an output from -forward().

+

Because setup_context() accepts only inputs and output, +the only quantities that can be saved are either objects (such as Tensors) in +the inputs or outputs or quantities (like Tensor.shape) derived from them. +If you wish to save a non-input intermediate activation from +Function.forward() for backward, then you’ll need to return it as an +output from forward() so that it gets passed to +setup_context().

Depending on the transform,

  • to support reverse-mode AD (torch.func.grad(), torch.func.vjp()), diff --git a/docs/2.0/quantization-backend-configuration.html b/docs/2.0/quantization-backend-configuration.html index 26a7d09ff41f..4265b51b7c55 100644 --- a/docs/2.0/quantization-backend-configuration.html +++ b/docs/2.0/quantization-backend-configuration.html @@ -479,7 +479,7 @@

    Default values for native configurationsOperator Tags class torch.Tag

    Members:

    -

    nondeterministic_bitwise

    -

    pointwise

    view_copy

    -

    nondeterministic_seeded

    dynamic_output_shape

    +

    nondeterministic_bitwise

    core

    -

    inplace_view

    -

    generated

    data_dependent_output

    +

    generated

    +

    pointwise

    +

    nondeterministic_seeded

    +

    inplace_view

    property name