From 649e4368ff786e3d02eb2a06b1493fb217d74408 Mon Sep 17 00:00:00 2001 From: "Paul E. McKenney" Date: Wed, 7 Oct 2015 13:32:08 -0700 Subject: [PATCH] documentation: Record RCU requirements This commit adds RCU requirements as published in a 2015 LWN series. Bringing these requirements in-tree allows them to be updated as changes are discovered. Signed-off-by: Paul E. McKenney [ paulmck: Updates to charset and URLs as suggested by Josh Triplett. ] --- .../Requirements/2013-08-is-it-dead.png | Bin 0 -> 100825 bytes .../Requirements/GPpartitionReaders1.svg | 374 +++ .../Design/Requirements/RCUApplicability.svg | 237 ++ .../Requirements/ReadersPartitionGP1.svg | 639 ++++ .../RCU/Design/Requirements/Requirements.html | 2799 +++++++++++++++++ .../Design/Requirements/Requirements.htmlx | 2643 ++++++++++++++++ Documentation/RCU/Design/htmlqqz.sh | 108 + 7 files changed, 6800 insertions(+) create mode 100644 Documentation/RCU/Design/Requirements/2013-08-is-it-dead.png create mode 100644 Documentation/RCU/Design/Requirements/GPpartitionReaders1.svg create mode 100644 Documentation/RCU/Design/Requirements/RCUApplicability.svg create mode 100644 Documentation/RCU/Design/Requirements/ReadersPartitionGP1.svg create mode 100644 Documentation/RCU/Design/Requirements/Requirements.html create mode 100644 Documentation/RCU/Design/Requirements/Requirements.htmlx create mode 100755 Documentation/RCU/Design/htmlqqz.sh diff --git a/Documentation/RCU/Design/Requirements/2013-08-is-it-dead.png b/Documentation/RCU/Design/Requirements/2013-08-is-it-dead.png new file mode 100644 index 0000000000000000000000000000000000000000..7496a55e4e7b41becdb658a2bd34765fbe80013f GIT binary patch literal 100825 zc-nltcRZHuAOC+MWQFW4Wn^YWMwH745!pK60&zCyX=wC z?|r&I-#>nj@8j|9zI(WDUDxG2kK_G*y`Hb*IL;z&X(*EsGZG^RLUuz1qm3YV<_Lo8 ze3}5h^D(5`AHLvtYAatyO205K!v7FhswrcTf9OA%&3WGlWyWc+MaQJ8}baO~-d^W%6FY&E0>5Yr`MSt`nS9{XC+``N}BA zp(aNxVXUymo4s%+|1YD!s%DX$ZV97efq>p>+|blHr7qUW_|%ErKe3W|yWKZ0P+B*o z$94wRyL<`A@O1scm*FYLhL6EuaWkb5S7aNf9@mXvW~;09GCvK??M1{A!Qg~n-jiT# z^G3LFUJ^WguT4p*-zGRo2^(023icvw;*z^B|H zdOs)cJQo*%mX_AD=g*%H{`n$?%(D?x#3Meq^}S1!l$7msTI&I=5_n(Lv=U0g?i86c zczj=6Eb#U7%hJkL%D?pr_vOo%Ge3Wx^7r@Gd(wr~$W!q060@|lq+n%zQf1rwZhD&S z`1rWqYw>LVz`)>*ZDa?>fXL!JR#GoEJe+k(Z|69q1 z?L)p)UdO<%&{TZvxyY*u48(Y{rujnOf=kPlyKokPBF0f^5fr1 zXEVfLItYzh!XuNCkz)K?1Kr)KR}WT1Z6@m7d2|b@`4n{Ii;e5f{P+;l9>XM&$)ox# zJQ{yZmQMkrvz1bH3Ztl`uc%dv>-MKFoA9#p*z2Z$#|PyVcmAwl@Nc9^d0(>bqNuO0 zpV?dgLHqIJM^~JDVhchMKlKY7Ssal9=9th72J9m8VQgfi>w8C3;#9ML6)z^V?Q^O` zUxk&jNxd5ma){GJ$R3_}j>B?~BkN%&BZH|ytLdxj_@mYK@f#aGLg*V*u3yLfyR#Za z#&lU;E5VKOK1rKgF)5{1U;Fr(2;#a7953gorYk!uquDw|dNDNTc&%Yz*NzYVw7+|Y zd$6-wrB9uMhn>cGUnPaXkeXw3E(H) zy-16$VRXdSq+wQi3_U$JlS@q-!+3QI%Na3>6HKm65AX=jAZ1}=3tP*4g7Q1(^brokF+FAb7RgmH3m-Wjd5(RAL2 zO7NXa)GrVIcRc?lBPb{+JR;)0;IhVz8`lPMZ&uj$vbIn~QikmMd+UXShtEcZr?ZeF z`FbTr55~*;vX!&VTZ0U$xLjo|&FkIHjb}i~{gJ%CalvnUX?tR49cs;YxuouE!R@@f z%EQh}_l1yaAg4JENcfI`3-W<{RPF27orxn60>lgr^(8 z@M5F&a`Px<&YnG+&J&Pgl>X+^;pyiNJ?80h;ThKufhS&4ELmAu(PkuMWP@MyIp7Xr zKt>W=XZa*bu3LSGej7qa{b1@wm9X)sM9FF($P-spL z=kfkL8>)xNf~MDR-J$?pU;o|s=lk}sd2h8)K|ukM>2pxBzPT9^8%ypmTpB~gt?{$a z#j5j(BACe=pVdfmxkC!jVmSh%>PU&L~8aas1J%Z}8!vhA#nKQ1r7Bq=5J zZeW1QbN)+cYHI37Z+HdjKY5xNA4aQeXBQVE(q;X#tUo@}3}AhE#gm{&ujE~`|8C~M z;@5h2E3cIi;qh6@>k0~JE19$eFxQj->AzyP$bXGz}a)oq~kZtxi=of!j11=`s=hIcnLO8CRc++MbcFVuzyOh;PAT zKiK_~{P4rzpf1uiSfuYMb_Catk42us4M&7`Z7mMe41ep4pyI}0-h;6qUA7s4Trf$L z+}wHU1BJ1bHa)5V0Zp}~&_={a0sQ>%@b@1+z<-)GzY(+VfB#d2QQYAI5fPEnxXiuZ zCOGV6O&gIrW1xOD&P4_lmZc80{<|*DfiRS3DdP;Q?%IqcomYMm9vAV(ZGYTT z@BZWc=;)OmMksAnXp>y;Qu9_b=*?t%KVxI#6q)-T>9POd>gQ^P%|sgnBD1)hNdkGXE(ga=So zg?c50HB$>-Qv4?M6n@RDRd+vMhk`-cK7S?)As~DAK7vZCNKXJ^5V%;RUp6$9Q~RP| zGe7O2ufP9!US47!AD^|I9VKY^8rm1Z$NPkV2Rq_pcftI?-@X><3wiySVu9)`udC~1 z3qFi1dZDPK^bRiQ_wV1QiHYCEomYPP@?|t=y`6ZzPQkOwrKuWu?NF+Snwr|D=fVn& zX{G8ZBJ`4O>|7cz!==2J?6yrCJaYQ}UOn2n?X}pCzx!vpvqz{k8ZGaZmKM8-df{jB z@xtCqIstpLj~kW-g*iF#qv(V_z>A7w2JYI}gaR#Cz-KQE@V@gYw=-MmkJnJHq7fne@;{_Io!Kaqt9xzlvIj@E2~lrV`xGGHQ+_2Zjs)? zw*o3&{ZdlOw@`M%UJJLJ{c1SXWeKkyt|>T8zM%p0I)$je5F;Wy!#u6RKk3-Y?tdEDhr&@GlE=KCm@s+w?%n(0VSUFm z;~IxV5K_AVlSZGLE;4DQvalum+U7i`Y#J=Fzu8T{WaDFC(b1c?whlhaJi>o=?A&{&SVrqPDYK z(9yO&C1pfxY_f<=x7`6NE31o657@1vfJJ948y#ib#oM{35fF-)V)NE1yT2}zZz4iN zL&46Tg9O7b9p87!j7qq8c%HJZFrYUH$P+d4sF19IMQgCMN{2z-?3pZ8m|(#XGFP%g zpkI7!ePg3z*%c`%#>wBwgU$(I*KB@v#r4CeuCK4h-DwYt2nh)Z*dEjes^5FR?SHUZ zhmVb>7v|`(M^r2=vjX57l>z%6@#0?>y~b9(VTQ(*Z>CJU8Hw*^76Lao2lwwm?$Z65J~?Xe((jYTBp%Y zm%8li^T-TTl|f6u{iFeL=P_y*7njpSL~jB0s#yhrhx*FQ<>EI3!1mTQH{16&=P7Au zLNhbj5&2MJny;m%((wmZ1NPjY*pwO*zjk!sBNVi>cRg5EYBNp9o^b=TSu(6znV@fcWzkO@^BeCX!O!P{Fqa~ z2L|B7xvUhM(%7YKzwpysH(xwr3)-8l5I;>qGT=Z#Pk$Qxz1m@r_)qh$*^^q(LFoo! zbHh-j#4H{j^A?%w*RMm0D`km)#;JC9DMfoQ^BUgj9`NY73l~lyw`v>)`5S-f&ucXv z>-)Qj$vpWV#5lDN|9JTLj-8(gUpx#kxTTZW@1%mYIY^1fX>xLjqqlo*vSB~270mW@ zb`qMt@p?P+He`L+Ja{(k-qhP)pHpj&x}rMqZ?!uIPt@DN*h=2}%?x~&IlN5C$Qb$b zDFrkSJrK*6zMY?9>V=b2MU!i`%4Bx-_V!~NRV2<1gxpIz`JETIH}iY+V*~ZPIse1U z>&h%9SA;@_ul`PwZZBW2pP5}*DRx#?RV9XUsc{x@63}Yb*Q?7e*rfj*>lc+3A5TSa zhWRv91wf_cpQ&cwe@D9&Ha$tqWb|pt$ctn~uoij1-bXbx;txYY+-KSz1h5RNI`4~+ zs^(5KcoG7;;v*2AZaS_;#>8|$S@)ISX;3l2Vpz#UVXB{rTTXhW66;85evZVB#r+kzUi$(J6&NZqsgm>DTyT zmi?WByzrq$^0{qGQ?kUi;JVU+x6W%bZ}(dN9mpYVPgtZcIgOOd4EO5v=v!E@hn-=G z3=coe6PXp7+gWogYlH`$E(V1>TxQ-`Y*@9iU0)<%9#(h%>Jr59y_pbl1cBiE-m$Ei zfSjB>fkz7$>=rDxoB9cASbke}h&<$l*_oNl&@yP$Xh!jF>$X(D9&GfV=W`dR&r+hf zMyrC@e96EJyhI*)6omNkSdF9A?7L8U373~zGkzv(mz)N*6uQ!+?;&jqjfxJ~Zr*?b z0kg2WvxPq#7;5>+*sd~zQ6gByDB3&g-#hLXW)l1fUYqJOz zp!pT>9)bnTw}SDn_g&}UIDfw4-XF7S*J;*>h=@y~qU}%5DIkZ#mG<;a5vZYhWVQK- zPZ#R);jlvv?%8e<78cen)VYYm4!P9m-uIg?#NxoyDJe@1wrU)Q?=}|0q&RMJb8<$6 ztH3+(u>dNjsV*xc6}a*Mt2YJb2ldTC%Yho~%0@>=i4eeF&D`}j0RaK0uz=$n3Bi?6Q1+RbH+b08_AE^{)?IGBqrcQ1MmAu!ySDaJsP#}A(Bp#W z-A_;NO$X{P2eDzj7QUWS9&dL4p@hQ@C^7g;eA~^(hgMFmRo&Zhyw6%~LbO@~2(1Rr97C|H z%gMzRnVg)AJWUa`{Z&9<9(+`zMP^jz;!K?mdN1(h%a>S|t~ajJEn_02j`>ggbHY?^ z$8OB^^gLz@dj7-nQ*W;(qOv+xBP!7I<%EM6Ov@L^=v$L~5 zrAlb__xFo@reb2^L-CxCuSk8o#br_|+dg=Ijt++%pu&0&h5oE8$`+UD76J6#Lt_ba z()WHR03!52{oFj0x$@`FzY&}KYdrw_jc%GZZzh7RTEBTJ=kZgecASZt+Is6>LPCO5 zyE&K?bf?nq>DG*S#+9jN$}QYk9o)IT$7=kZ&+Ale)P8G6c^VOG;889u62ro7v#rKUX|*m<}!6^Nbfbm%P7#H{Jwf&iilwP>@wG}uu~02s0X z39mbnhA;i^8~DWh8W~yNH7qwf`W9J&#uwaK9NCzNvm=!@Y89mqnww=WyG|W`S}IXh zQTh5No%_H1%+ak3@}F;9C#bEhoo@Hk{(s-jaxyS7vIIPVAaJLM?qr=3KK7=#!#4{` zw4=g9_^HrKLq+HR_m%R$Oa8yV{=Z&Ga}`Z~!N&n{<7F@qbak%Nzi_w!c^%z;Ph6pW z_}{9?KD~_ZC9}Y7-MxbYx|G2F}o-Krjwqc zyat~Ej%eZP`tN;Dqv!k?z(-J`#6?ObCK3P^a1(h9GfwgSu*TFg>;&@P&c~`CbT~3I z4A)Lp-uA5_IX@S(KMUYR2`%UG0zm3@P7V&j%*?z|oepvX1wx%42x>o^p3I~5kh@(} zhoYA-F~#u1XAY?}!Qss%FDW!cLx*-i5^~#G(3)txCnOP7j`Otyn{b_ch8f$AoRK}; zz1nMbGHu=Gw+}QRPZ?@Z(bDc<&e=>g-KQ6Kp#JqKB}6M*iKc5UaroAahWS5#V)e}f zX-bT0-qkvf6Hn}Ct@vDqDJHnvzu^t0eidk247-N7KjI4ae(5)3&u z@UWp>^@?#Pg~KEFPT%@x@UY%}`*V=nsa2EsBZOM^vgB7AXM=pFjajPGmk zoX0&%?-C4fo@!!HczmYg=z!tgENBHO$TYfi!Y?2Y^{K+c%}pM1MBf{4N2CpeUijW` zlVXF4EGJdfZZAcx50PD$kRIQ=RFp@Ln_14rEs{TGKGdCBYqYpebTN@v_Zd%f!^%G zP>FFIu~{wzCla}ThoX?_ve5*2`SN8SBnYpZ7<<0g<-dYZ(lBZvZEe?1Xkp3;6RP#2 z5dws(zrV@#<>eKtO6mP#qn6{tO-&e3LNvs}ZMtv#yuoi&o!=JLl~(}a`ni-huSUkz zlBUCr9^|WnR)Q!Zb3crda=li8!w&;wmzA9z_u@rlrA^Pfi3#S57cZ9A)_xIr%@b+) zZQBPV?D%Nc9Hp`Hii!tOQ6y7A2dkX~Zybk9?;IcQ1C}sPOisq7r$3pQv3TFn@#^m9 z7luCaZo9utUrKos11>3RYoGO!Vqs=snf?C#A}ZWBd9<+ zWFW81{Qms}Jk+{@8HjRi&FxWGSQbbix{1Qi@3C7^1bln;EUucGT7sadlzFW8ty>h` zNdk_Qy1yW;?)_aZgA%Y6gOEKw2i4iu-+vaZlaCp4zH7*Z z5+-Q+Ci6x-7fME9WK3?e@9adK_PXpMPd?F#@vV zYzX;PEF{#ANo8Q2G!Wl}i;IgdS$!alW|fs1?yTo2)XF9RqzogX;zEN|K`~Fa8Kr$> zw9698GsI1QkgnsSgE@OfNw-YoDa4AXa1{rKLIeUEle?#9JP2nu(6J)M8e$EuJV*`( zV-4W7G3Ec~#>Z!z_}IQ|5Co5-zuwv2pI=_N4Icax44#8yqfeQQ2w5C1%Rtx5KYjjO zv)_}*#|_pOH!@=2d<-17I9iqWJSD}(C}XxA4E68!GB+r}mBP1J9K2IlG|fZCw1HCU zha6T>Rn=3yA~79s8^Vr=fEA=@rEzk3DE9XQ1Ghnal2-ph-kk%ILYD_35)%_Ix2C0~ zd993Sy?gg=!^>%4U|=pUBj}l4iBUvUR21^GKSwR9tgH;8YLDGm>%U{;PoF-$pPDj* z>rb8&NftEa{q^ftKQN`boV42v*6a7gs2v-WPmXb&iw6*l`oN!whTBFp4xE6=5T9A3 zeNH0njBgj!Nr~XSJvmL2E9es}v&6eOZdQ^3;kA zB$>=|-RLyVE}z?eSOaLFG~2V*SLW@NjfJOG(bt+dJ`%_i{HB%|$@DBxo1< zqr)Gua$mH_{I-`k&zw1fE@BjD=X6``_6!e8^iVjuxCo+)O%PDB_u*1qMVJF2S6eze zr_CKVHa6N4S6p%5->|Xoe|;mV#~XeCXfDHNQO6;E9QNAU+VAu8$@VEAO*D#H?*|8U z!6<8n&vS5CFMNHaB<{C$$I9xw+|gEi-^SnFKSIdsdw-@lIL^}3^B}LG7qqi5L^$lg zAli*S&d#G%cR%<4LYJ^knlMnt{rjUnsn&zv(yQdB0i=e_THR(JqN_)#!*AbSGxJ;I z#9;^4vtH}YasTn5phgA{ySsT7g^FaqYez@B(>*JrRdcI#(@AwxeriL-hUcT`gp#g! z{^|m~=aiH*z+hVTzLPLq@pyfTkdVn#XLos66O8N}5fPCGXi&keR}Y79a|RZxy>*KX zw2&^4x#~Hbr*P#rz&=rK^#H-V;fNqExZy-zTXsaIB*i^{&Jz?A1higO zQ|>g~VoLG|s#@uqeA%$$EFN}mZx6kcpKx~`|BeHRh>4%U@HqDL#DI^Y3)Vv~>Vl5^ zh)78jpkvoJH{+o@90eSqJ46`5(FBY;GHf1PQuAwdvQgjv2uilUCD1=RbGS%f2)&yN zTwL8yK-MtnmQWyf@7}Gn?PYZ~6Ib6|98fNbC@CqCl$MTy6ri4Q)f6HsXbmJcTzTLf z&MQ|80Vk`+hDuG1hsw-_AO`mDtf$?Z>f2d|7{I{+WW@m~p9ZP4>>!S>IoKXHuc56I zFs|()5o%)pcQF00`qt-}M9*fwZSCHO&_hYURmEa8s{eBq0K5$k?w1Zhm$>J|vXx?< z0qOTN`|qOb`Ov1Ez_9~fORJBNXcvjw3Zw8-;R3r8F5=4jZ7ra)Ygo(Wz3}zg6Xq*f z0t3_2a^r#aW7P#i51&y%NJ`8IJ~n*9CMO8}A`yib+yxYU<7*TyH*j5;pqD8N`moeq?M67Xf*;u(eILze2_+h9IX12pGL_{?$K` z`8GH2wKlc2Cj1Go?+cUZ@;mL9yq9!<0cH;OH}mz&%yKtz5EBy<3JMAfvEm1QpANQ{ z^NxZ?D{TO4-j3BcR;{dA!ekp%+ffGnJGvLJJ9$q z8}IeysL_GF#j#O0td-wBn*%8H3%QHC@V=1ZBokRs2SMt!u(3(Be*rn=Q@SkE&d$zh zQqoTzhGDZkZpe?~clvNGwLL^6BnqL#G}XgGCiR*4Rmzadzk}^7%?fbOE?4zlRRCM6 zVp)?YIzqAc*O#=^xd7;q4~qkN!j|t(uZj~A5*C{_(xO{Ake1y6H`O-{jf}G3hSB96 zS_?L10IUx&ObgSka&VW)t6odO&F(+KXj_mr7~b((=k}2btMztrIg6RL2k4e10U2Y0 z{gtKX&JbaGW^%%+!H&>9LT-(hT_OPm<97_Em2+})IWAqIfVia$pWxXP8^b964muy3 znaPf>Ji=x7cuYZC(BJz#f$p22tJBapn+hD{Q^EfNef|A^fPS;!G!e{PD!P8MD7wPn z!%{Y|s2(OndCk{X5_~shwUX30oU~KZZ_CoS*)29QQlx>|TNc7QkLepRpiW9==4f== z9va@w+xxRvFL-KJXafMvyxsKRQ6X4!?EpiVkaugTdFzw=o4;@nDW8?oLPA2{N{s0c z3P#4ruU~InxNw0T`X)~!O~l9y^mVP9uMFMVM=2W(jSLJCdwaezK#u*nkpB^otLZDRdhh-D7~ZbCQyh$_M}Ko@=P6$iq0JYXbER4YQBgf)GT^b!uW^ zNUnV?aqx`dA0a&#PC10Mra>ofzN>9)TEGkM=hN?0YcF z_);V#C4mwV%CV=n0WuLOD=SCD!~~`evS)q&5Y5S+HSwtI=TGYr8{Rx%v;#vJ!ZFWn ze;J46u(7inGjH)gRjrMl_?;^?dmYOtPJH$1RRp^^-)jY(l&@PviymNDSO|xqvn%0^ z4QpGG`508N3^BrRir3cJIR;=;pnOgTC_*^s&=*|}Mau2^vO2%qe&b~iNs#>b`1mvl zNl0qyxE0&dYDOWtk2(J(4YDcACD$pLt<`Zl`479bf@Wf@qWUbK{<g(&*@V*ln7#L(akJl25d^jxZjXV)28vn3579|V@LG;SZplcmwz}2s*sz%ak zunu3h9x;48R!B)jl|6aJ0PX;}^fx`OU`S{i*Muw#F36xYNEY4A^Y!($esKmds%tFFV-$g`!x45y%8UGqR;6p%cUa9H8ghSY`d_@omHdWXGwBC)q_;`%>=#k8p5!?5vEcPxel?2Alh@0v4tuRoV2=;;@4mc6+eT zq8o)DV%dal=c>oWFiAve(DO%O6SL(kNWmfs0^DHm)8A}0vQIc8uG znbid%=vgfXTB%*6CxFBL<;xe8Cg2W+2z;*I`~C70gvLVsGC}0EViYaJ#$LI9;Wafi z(5dK&izyl1|m^8iYRhnAKWR~|iIm3{x}{?B;z zA2(C7!mc43%Vn*1iSg8u1ry=ZB4c7=w8$XOdxL_|ASoXv|IQ_wzn`3x?V-Rv#$zh- z{!{?vXosV2s&w>sy9VbG8Jn!%)>A@qR=aD8ET;~%mWRuRKO9JT{Df?{GG3>Ngg<_) zoHTA;pv{?b>CVHx7H)2CN4bA6fYrl;tNvax!nXmR%J9oOSAS&xNGZT^0=d_QwsV#e z6&1bzr+N3aNrMM`W6^eJ%DCCjJIOnE>ku81x#OdQ{mQAXE|t+5$Kl)4t@ZViH6IA~ z`ZS<;@wGG&5!){>ZU53OfjdS)7}qJ(xdEul4-BJ(x8?)UT+sAQ!m0?VaYL?VMk!hS z7-WZim_o&}VJ0%W@di)Z`)f1Ta_7JsyY|+9boBOW`maI)5doDmtaC~0^U!I$_h(4( zJz1TsbVWr)KcruxGiScIOCCX1t8`8rt#i!?INBcWZ}z{(AnBHA((I=zBO{}J#_uIe z2#Uf18y~s9udWJ+I}TC1y1Lp=zNz_?B066&o;Vr$SKmC4Q^2U2LRwn7qP8{(Fi{N{ z=V$ZoR6ITjt%|%nHaz@MXy^^pfe0R6ws{{;xd8yu)e?As!Jyd;SH4iEfJ5Bj8wI%i zeNe|7&5WxefP_~d4b+BQlW-hTfB5iW^>C7qxgh98(#XgN^PeBp_5-h>@>6#DIy;pC zG1(E36&^QihwA(W$BRFq8Kmz5ep@%6T+f0Rwin)P0kq_Pu{! zcHPfgDh(JXDvJk-|9f6b1^|K|lKj8sB*Mshl>tZydZ1aHpW9#A3(fn%%&#Bts(;;(s{83AQ#FkS8t zwg866pl-hmHie}Pgx>@=$^5rJ@wJuvEO2E!xwnF00D>H5%CqH|Zg&vvXK2Z4dG=e+wWX@sj z#MF0}?r#<2R!LR2Pc{3;y?hyc@pi8Ds9FwV3#&w_kp_6Hoov9>sI~3#bmD1obcp7n z^4<111qEHcXDLQSziAcxt(w5|0MZ;S6_I62=A4$?qW#`Gi!7{0cB zKQ+Y)sw2MT*Q~YsFTvA+ZwPGSR@L2`?>~I71l!^2^uGOX<|&(ixxCV{zNMvQtpW=L zx~W8$L!)@ZEygGrwJyBfRK={Ct1i=JUo)UU9T9t zx2>ZCUn=d4=-p3UVkJ^-_k(Hz3PKF!{`|Rc_GI7s+PVO8^oWS+CyZ7xffl9J-jM+4 z9SS(e+PG`$xeFKG+Zk;ic<-;L3C*->NYJjRo{Z-ey{2i535`b<2s8gy+Vm{x1IR*v_*~~66TYyo6%xl-j97rqjMTfI=iuNd_gOXC zS|0A*!@DN3YrH<65o`$o3Y5QVOVrlZw%D+W0CMVOKT|SyfOL!|dKR9z>=eR6qeyhT zR8)o@5*fn~cJw4RFYatl`5Xy?o-0Gz&|Rmw?nM6f z;nndvVb5RctN+lb2pU5{P5tP+@)HZFFLaFk{G?ElFlh}kO-(`%UBNXcG#?;F$lmt% zj;OV*t%Qu=umdh+~-o> zg_nb2`jnn97kn-Xa!KGfd`9^2cmeezun(4ATtO8HueC76WJ@9E7B>@mZ3Pg z>MeS?NEH?&RnVj!>$5uc3bG-(xxlBO)AepRTfiJyTkm9RpO|1ickbNU=4Q{9XR}!> zKn%h#O|`kUcIl{R`V(nJ;0^;edZ;v8Io6?<;eUlXUyAEXE-#97!pDXJUj5h}Ht&SN zp=4qCOiW2*u{08x`)(Lb!S|-_6C&l1P_k(u%vxK&9`e&IF{*JI9w@(~uzKIaLj?QJ zs0BixFZ+KVO?L**jyN+aoLSmyfdT^fBPhVg*jVz*E))5a4qzF|dUY&?1mvuw$Vjz4 z6=*>9@U~y`WhDa*`X_XEZEWxn9{o~gCuiq%aKSZs3QAdg=J}}nh4HmDEf#s#G>ze9 z0KDQuDAVg&)PV18Jv}5-Q&Vdj8`oj%Ev&5C3n^vi*K7)*cE)O*qf`@lXNO9R0r;DD z-h^iYhoG7JU3)tv6I1G{AN+@CXjzJ=?YqyPV_-T@&eDJNM&c25f;C303rGLYrNN@@ zQnO}<8jm^E)rY;KqlW&HSFYIiIy*ZXcHoxkmzm`o*Zrmvn*aGR9_=X-V!P>Aygai# zpFWY|5sl>uhdgSQj+45lEN57@7nznGOp!EG6CXb zX3(>OnsGFUCZd4whGNJH?yW2!pU1Oa0`YQ+f4 z8#u@|(uJPSRX2JBbIXUkhUOgAsI%>5Mc2)pM#`@tD%#r5cB+74(A_1r@fY*U%SE3( zmxmxaM9~WPaUCqq&2b`IGR@)nl6PDmV=#)HXAn)9dLIJ=dQk6k9JYqqh&&H+ijYt) zK2f(&ho~)hy1iQ!JPzIUW0G)b`|{|1QKC{Hh=%} zif8q(S-`sDrwlnZw!p1E(a~NY+)Fd)IR&jd+ak#RObB^;z9M?2-Jz%J9{)u|9ss2# z=;&|nYegObftw$mu<-5vp7dX-u5+DEilE}|h8nJV`3t(W67)Jn#3m9w*!g{ao*STb zqiJ+}d>O~&=c3}t^G)Ix-rx82mFZJlpJ?zLTpP^b9GviScc<)TgZQ#^r8YK?mXe0M zY4vG%{sILhQB&YSZGM{2&(LeOOiin6a@XSG;=&`pn|kce{+?`H8n~nvf;M+}e)*e3 zpp8e59?976sS>&piF)B8omYFf{da%sW*oOW{I_aIGX|Aj=Q7zn@X4>4l}^xv0vSdw zRK0wmiPxD{W2GT1NQVzj$p4jvxRh-qdfXPx( zSJ!FN6FM>2tgN*L7s+3@UCOIsoCI{KIfItI6?n{lVbMEJ-ajxP?EW3Yo}JguS$z7x z55JR(T)I;3qVZC;$VRX>z{bZM^MUKa1s;Vk(mUcC765CCcX8}r=#Q(Ndg7L{F(_ip5nu8hdrEeDtjA{yxXRmml zkrznkIyo}^?NNP0!`xrX26&OfUg77nzanI@|L~rkk%v6V0lvxa^NNq1T^>DXUr}Cu zD`Yqpaq{&$nHMcm#7{`5%*6B(_k|1SssQWC13|ea;+g*TnRD>t;`TS^)zsD13hGsz&C70%I~1d7TbrENHPv7jz#AHHk?P4 zBncay-(`n`Kr-waR3EQ%)z7Gm+CO|nSpH)+8yEWkx`1tUs(CyRG=O0dd;}=CLrtjp z6dfI%q_ngnIr>#f>aMh2YG^3FjI8Wt(*qFydhnGY>CEBtQc{d3JsjD8Oh!i5QE)rY z4XVBU;{eVI&Ke8Wlw5%xol&hgA8PEQu&`f+G}+ffiL-cAaS_L)wSq!|ObL+|!fhee z+Q_+WN(zdn^GQW)L3`(Vm!Wyiw7DShIMui_Z3=3b1W^>?co|o4e1}R+nNTEOSBE)9 zw-<2Q$mNN9_H^)N0*Wt5-qnp0E`33T)h6gwL%IqVi=?o1;4M+^KQ=C#Rm8`~-?5Oh zv3%x*R3KSw_c7rrZS5H+2fJw0?*DB;*=&$DCf4xX^>!Bdzc3aF^GRg@gGHTzQav{? zTPx2iPC;#7Ct<%gy{oFJ*$J4yADMLlZ=~%kE#q6{OosHmGy0__9|v$rjarxuHU2wZ z=GQ&zxI9#HnwVG{V@)RFaxwdz6$?^q+IU%0bbNfgNOP3$+{sY6l;+^P9xAxatzVa3 zV?UtOyj?7;X0T9?qf(|@q?hg@JyGMB?6+cDk$}PUo&hfnCvMfpG5D1`2*n9@8HBho zwh<+jeV$!4DZT5QjX@uW;pSWZIUDxK`~qk{HBLX@MuqSgg|etzY;mzQ*dS71CxB&N-@_x zJSrbX>-CBy#hu{L!u^X6qOZB6#r2WhyVfe}&x4?kp+N4Je{%Qv70=wi-`L1q4PRgX z$b|6fm%jYL0HP8KLX6Xfhu9LYuhSv!T?;v)#0+@XNi>N#co(Rl{3Bm&AWf`hO@27} zYnPm^JP1z6y3Td(9FC7q>>uGEh{0>?>krJzgd%NJ2q`u%MdQmIw!gcGaN|tO7+%NV z;|qy^fAipFGGUX*@$|2>vyF|8qC@fh=g(8yT7^0n0nyJ!zH8OedOJ5W^E7)VXJB#W z&!1S$tNX9JR{Tv&n6t98DXFMJ78dL_;de6po*x+*aX?-a3|x#n%gK?4FrO-ClxOc{ z=I}G~)xCyMOk}}6LHuy+LLLV@0k|jl`uXvK>D*!yYT4r`Qq;jbdi=N@eF#k>?IGl{ z!=ocgX6E8!1B4l;{44{WuBV&(i5>ajCL-!O)Hy}rl88Lu^w-9FuIM8|gGKtbq2ZA! zV5fIr1Pm%H@d2LF-Em&?NKuaBT1c__jBB23lO%}dAkq5NbGIogJiNUV(dVcG|8Akr zQ5Pv-5D@};b9k>_sYyhtgexvX$(Bi14AYjPC(cKT}D5Aou^E;^5Eox3hObg+foVC}~e`g7;Gl{V`k)=@>>$0*k5x*@v zNlD3a5lq5)dwcu6xny(n+!zrVnJ%>=5fvqyb$~>4IFC8{6ts>?jW%@$d0P}*=i4L! zQu&~>WZ90j%`B2dE_eQqQZ%@wUE1p z1_sYRXULf|$oO6YC)>9KmtPpI%6s3@aS@?K_DJO!`4moe;FymP3j-)|^yrUj)!okq zlpNjq(EHcWEt~BnU1Wr0#u+InCZ&|UCkI!)_8fc z6N9+o?#Y4WZs0*v=a<`+@^nvW=cuOa9~|)M7Vgl{5C5OD06KRR!xswt)@E>;nw#gw zY8;8N50Id{x+h%}`G$^0ab2If!v;i8wMef7F3OJ!as{!gt7}A5lp0msFXTxDFD5}B z1vBE1e^ivit}XjvZy}Hrp!*xU)4}@GiWNH4HZ&u*#VBQQg0K`?*Yru+cF7pUZUr5z zN&u&&Um+(aM;~Ye?sCI*V`#%iFa04TVV$pmIPwrQNcpZ|djEthLFN%2{^XAtjV7LKg>r5K4REEp5=?XLKLH_u4!>FKFcYLb%3o;CmF-8)jzmptW9AjHOZ?@t+<|VP{C1LTx>wf!Al_{Bg4*d8R4yD{O@`H(A02AoiejDcsnjl z3+wmCR3~dSaazO(q|c2e?CaMTpbqBlA4Bul_m-KrqG$N8VSKGC%e|NNip`o|g!0Uw zQ5s%I%Mq)Z!1L;AjqQI|xA5v?Ja@MHj}PDi-P=ttH`nL76DxezEfH*4nGly|di4Fx zUo<^NT-@ApJ}+J=JkC;%WknxG)oj6a0RBYx6d5l&by=P~(M>s*0r)iw8b8u3z1zO- zG}Xi)m8QWmK$t9KPJmqC;-YPo%FW4n1+RL3HPA0<;KGFqGtk3Gn_eRB3gkl-HMRB{ z$6*+WlZHLKouf*Fw3QgwdC0n=4?Si=a73_cYi@{A9WxCR^Z~lGCv5xWC4ZjQ^6d$S zglEql#xO~6aB=0{aw^N>K@iFZ8hL!>@iabDR#IfJHB<)$e0(-@Y0d6y~mD zrX5}rxBbHG^?QN=jDS~XmITqheVgRxWdZ_%ERb}&Z~2t1RyR)G9jZ83sJqGBl9iQJ zy+#8L$sZEB6ZLLoy_J`=UEiHJzqYm(NiVjd4->MYrKK&euEva3+QdD3_KsWxCY;{@ zmxH(PTT)qDTboI*7amK3wk|05k^-7F{1LooZoYB7%;$QDn z;A2Jil3?WDm&&$&f1#|?sx`=2|EOli=!y=5tPghTl%6-EFFIKA3oFI@#xOQ(S;a^& z!XKX6dW^4#PkHS%(Zj7bU#49V-Tbn$(&Yc;CHgpdm$gXC zQ{fMi`vsySyH8X7D&bZ^~aZ`tLzEM2Vjc{qykd|+VU7O==i zC)e95EeQLiOHYL?qATV%r4tp}=>5$a>g(ls6CZ{zZLbBnxRmY{e{BCqHykkdGJxnI za{fH)ad^DY2YRb9x6gD|EBnoDH*(O&)BvYlCh8w#X0oq01Lm^x@sS7$3hKVNoRgOq z6)Y_!1=Fi^7(8c`2Hu5ks6r)%U^pHW6kN>J%s3Mq{ExAY58edyf<7$&^qfL9mzP?y zV8?9x;|l*>mxZOJ?~mDn+FVvUN zh=Iu)FCqp8271ts*V-2WlnF>kNS>#rq6<(-Nl8ytb9ngqPa*US3=tB0#G$y!f~N1# zM`kD~aq#f)z*uLRcc((Kv#nxI{t^QW<`gwCKKjI!si|qqZlWqD`q*rLzqZqOEg2CJ z5xPJ7g!#(Dwl;b7G%1>9zwN@>Lug&}uscUwX7In`r>acH-UdcS#4z?He<&$`Pc=t9 zdUOh1l^z+PcX4r9+ukmz)D*vX@#5P=KK-XjNnzF1Vmyhe)pmWjV7k}AVV6xb=8V+6 zHxl6w+aHso*bM1KMMFagkPtnS0l^zRN{rmn(+gYw(W#)UOlVx^qUhwrKU8Lpp7=f| zDM=3^%>^WtThS67O^Q)g{sFxR1OB+8LU>|wa`wj$JV(c6sSYuiifrXrrSV$l3Wq^n zbn5`Jy^^XbF^C=dK>otw;yQk;FciWHWi^4k)raU;%Ih0FLR1Jb5mDds7Qbx=4vsg_ zz=<}%UqZO@TwIWls;xTD;L0OBTwGy##Rk{Vhja6EgBF*T`XQs$xb&pU`sYLEGHx<4 zF!TXLFT%7kZtg4%Mj>B|4Xe5a2Wz_hg8nY%@y@x9*TzgtOo;sw6&8M#B5JD*qu6XU zT;r&ZD^JCx&d!kqf66UbPE1U+|5PRzJo_%x+R4dePV%MnJq@d%1``daN=_)%nENcu z%wS9{nHw7$ew_3g}*Kt@fz!Zy~{IyWqNr`cUcD?`EA z?6#K%t!-`d+<(~G*{OhDH?QjHCG3(w)`u_>hngN%ZtzLPmy=;6R0yat!_c{(fK6{mriN@l@n#R8*9&)1Qe3 z1&BA6hCV(%)D#q%aSLQjm!l&iBbR8-2L}h^%1;L!lt``)l^E+j^OTmB-q_l@aPQtd zhpDD3H`8Sr%$6H`*Ev2~i#SZa5l7#NVqhFbN=i!a#!L-FY6+Z7a6WB$dF=BhQdXJg z=;P{RZ@l?dCmP}bB)QV%fZD!qELow44{_z4$7^%It+l|#Pb1>4Q!gtkE8p+89_<+S z4-V$c%vgX?(>EV(^t=XZci9D1EOhz-UQLAH-wX9)~ut52DY67lf#%m#w6U+J0aNiOi5 z|8fc|<-Md+HWC^ZrfqF~!Fi%Sj+B(NYi#UAe}6x!4}d@N!T*jVE>aUiCCK_;4urO+ zrlaHjkO@*&?L2G@-5}E44IXxM!ce*P8PaYt1$H{?9k7I7cl2-@~|ctIb0MM0j}vpr+MAt{Y!#&dvjRHao z3AWmstwKc@`dz-#ZS`poa5Oc|{&ZJFfA!+f1F+i!y!n4(ez4d=cHn<=I4;>#J(E>8!=I5WW`uls)`+xxQ0^iibXauH^o-P{%>v+CW zN_TD!w|fI*)fY)=nhgU1Kqa@;UYxQYI_m0li`TbDebi*U#Bhm--1Ov0hJbSb78VvW zJA35s-@p66ehqi{*;-b69T0%*;^8qZC$l5z+vNa|=9lmj%Wt{&She6IA(s~yqrk;| z{>E=T&cG!zHa12!EUfq0>KU+Zx`|ER=j;ziNuxDc@&IMh(Ou27{gsBS#&e&)*ccfe z&I2?RIy|eY)}+(0QCbZcFQ&Yl9i(XE>jT3yv(?>Q1<2~xTrF824g?<1NTrrB-=N8( z_M7+CD6R9K&%8IK`+Z@6FKvDJfD$o$wYaypXA{-Lw?mCai-c_*eU={kwsnxg$RH9VK(;F#$9=s4$1x3 zuU|cFW@ctc(CzgJ9{9e)0j7-hu`vRWm%53+W~=Do3PO+wcgc9}09gYeCMPu`sCJ+i z=Nm2F=jaf863C^20+{5d0MLscw@C)!0=>}t#s0@A7&Wh+(a}+SL~&Q(4l^-U{Kt=` z=lcuwfVgW#7z70bthx&UcM14i3osC3wq74AK%0nW$?BslR1`l$&zHr{!+FIJJXp`~uY?wZ_x3dG({bat?mmFYq*AK@wgM!&Y?;cI+ z9Dr{?)@7HL-qpxg>;T|qY-M$ao}OMuPp@lkjtr;+Nhc?#mdFlZH>zxB*nys+;o!jY z^70y;m;myO9`4rz0&sQjdrj>}jfS|WB-E(Qe17Qr>D2IyPNa*GjUN- z(c$rNr)7rkhY`^~;T(u?E;zIHtgGwlg7Wen0@!u}2zhaJMaRQK3~~BgIFXW*o5AZM zEnd7E-_o#?6hGA03qZHzOo~~m#z3(_2t3h?%wG4RqCU(f&l5&N?_azW0!~mE2}ilR zHxVHrB(B&Bt}8C}f{?Q|h2PP1XR-+1hP=J?!nSFO}Z+3k>DbUEp?s`)TizWW2+RVhN%*0b3fqvlkRUW16>-@P2 z4({5gQmBCvYG!wB$cYs7!a?ptCnaI>^Yb6|HnC4~u{?fU=G+SAC_X+u6C|NKi*}JF zpTMO$lNP)f10sH9bu}hB8nL%$M|}VO3XnJg?mNWn2956l@#Ib4JZ52mOXwkJB44Si zP%UTa-~bQUj@*I*dJz#yxPbwDg0{O`%KX)vpjHj;X zSy@^5s7OD3PH>P>PcXG&D5UHa4_U^rzM_kb6m2*TMkY z1b_gBPrL%h-V2}w2?BY3&v9S$a*YTs)`k1@w7*5^F}j=@&|ZI^eJt=Cuco&4-mO-Y z_xi725kQ2PfPj1h&~dkUIKkG+M!vPVrKJjp_q~OtI^|ksoUGQW7eD816e=etY!q6H z!^VJ6$A9qP(PHKdlT7WDg{Qao>VLaJM7B?;N$dv_1w-~nvrq)xwldw5ZvLFl2xz6u z!K-ztsT4XoI>sQk$@)5nhjHKzCAhdB6d5J0tsemzY9SlaU@}rK`mD@r1n9r+;z2RR z?i`_}`^4f~6pWRH*FUG;jC}oiH$6QajFhOT=!@hdo3J8~gFs^N_Ac4>H?)JGN`4P} z*~OOM;|fsb(bp+(bG)%Ne{cQ(ctHUI7|Qhe);%!roIb12o@R0Ipv2Z}a>T+H41sC2 z+RwF;d~yT#OSnNxTJj#R)cUhz))_Ya1{buMaszKWuB+b@D!a7?>+9?8@~D&R9J-&k zs`_LWuFFMHUSbCzV(4XUgsESonRI=4hL5_owic{pRIA31SNb?hea1&QnFrIQp}AML z3ycLAU=nn6^f#wd+Pn8$-Nzr0k%fZ!nzl`;?a@rjTxj!e>m5;WA9e-e9k$)xg99@_ z-xJ{D!=dndXG*~=oul;vIe-SP6jD}J21fFpoo`!9=U- z>+4rsBz^i6{Kpkg|9qQ2HNfj<;^GioGm)B_iryo4bKW0AZ}RF@Aed`<28P_Ze6yaO zp4nDkp-3u`P~fkQUUCo-#gWZCwfIW z&2NnaUDi`Ru~eUjjEq_#MOT-tJ2tJ81IE6w#=Wq7Iunb^EAiw7P%+rwpBacv68a8Z z%&zl2CTwScc8=@)^c=o>flL?sRnDyJp;q2}se8SI9GNk^R=B|XP?s2(sqe&UoUzFg@=cC zpVr!aj|CT%$?!SG1coRQCJ&qM)ekAf{F;@2>>V7a(949=K){r^|C&qiyI96(bl<&y zb21{v^%m-YP~g@ z^`cj6Vgp#d6_*4eV!G^!0u>DaLbm$aK$35R4t_zh=>Wk91KX==-J055_m176+P6 z23}>l`n~c*}K0dxroF)MRA_4-WK)=BK>}>UA*Ef}@{LXSMN7Cy( zevzFW{DjNE<{RCy;bqsVs<#mlIDj<}hAmzpN?Y)W8Nj#UltN#W|ID=c`@^MjY+~%} zxIpikUUIbjCiMEfrDp3|wS$U=W^()Y2HuBiZf=IVu))uAIEJ0h!U2^u!XMnmX!abA(=4 znBvu|SDd`OaL36HIOOOmDkEN$C{La|`I`Ea2>77)>P6jfk4C`t;Q1&oFE8QlcRoHo zHf(USJXnImDB9q<+1cNJ2Sh+0xC;LL`yW6Gbk5F_f_Ur#{Ev^1pILPc?qzvn<3n_G zw4mebu=|fIfl91;U?wGn3ZKRTNNjFlVG5>-Yy=r-I})+K7xW-e8I|>_%`xxd;K1dI zWxL=hCEvUe0OELvLIIKv5f2&j^@ZY5WlZgL*gpJjIKmHRv|9*tmy#M=Axm5u5e?R_z zefS@D{{MdX-zTbi9{~h>*ZzN>+j~AZI4A*Vip&464=%d*9`LpJl$677jsN%iOBtyP zU_P=mSxVA20sGba---hJ-G^irwE*PbXqF^~l#GlE=l^cq+wZaN=Q}kP!=2TyhKkcR zK|Dba$VOaJ?>$&V0*eeVJm_7=f44Y$0+$Y=LJUky+CnNaGU!nI?CeEg=y0>=$!M*u zHUlH$amX<$@VO8|B5TQ*vp1RTQ4oTwSI6GX&CMoFHqEESir1*=^?{7KIw8A$Jp`sn z^kq6eF#S+r)i&$@9;%5f8Aq7lq|ZHK;^yc95^uptUL*Li{|)k) zeu=_xqTuED^d6^>5IOMoa?hW?)o1s&`*Hf`pe>Bo0Rmz&WTELbJqJe=-khr!?)198 z`F|d(G~^SXJsAvE1)fn9PXqjJuHZtimCx%#i8%nQ4uAiC-_Y7M(oo-wJ z;QUgfHmc?2W!)Jr<8}xpdVrE-6F~k|!DsHsc(L~P_er@-QGpt|p0z z*U&QK=K42$`Ecd^#8mQx>4{%9@&YX=#*#9(%!$H1e0OPIqxwTu*nV zNJV|>FXj!EfNLCUdhG~s(3&D_vBkTquMa~@OUuc_L#EDQ;SgvGnn#bG+dc)3a_dJO z$Hv#BoYR}pQPtnT>%(mz%nUBC|4XAN6DE?fY1JE1x zv9gg}%UwY|+<26NQc_Z4*N2h`r|^<$aIl@d{d0WOWFBim_k+cjInqKSc=>3Tm5nVI zsH|A|Jl)NM*(&;178ao=K|w)8kj;-TkCReTw3Ccj62`YD3VJS2PENMdUN6#yNM+Yv zQ;B$~0vX+6>UMMG?tB>@e%JX5;Qcnxn~W$Z?`+QfM>C({2(kk$TJc3OL`niEoaZe= z6d;n2GU3JJY0pG<19`{_IGmNw1942+O-kHJ;eH@4S=I|j@<1rq7>r<0`-nqUXM5kJ zq=<=$r68Wj?V3YibBoXI|1H#?_E0)-_RdNI4uD&_=(l*?1J^5YHataCBy9d@_Iv`1 zc!0ui!*gJ&XIs4a{jRo|U~C7vwBGBq_yXJSXREJ?xjEb-2;S9oY&rd?GYxDj5?tWp z6kk&|@qe`dZf>YhI4Ku$UlesvYpW<+5`#cMv$L}_JUqmDU+kXg=#UD&-VX;8ucNEW zCx8pM8V(1x$B&)Y2k*hg2tYsFocZ|pI65&ALqS0SUX+4V4GkmP{$61NUpw3A&I(P{ zn8HQIvsGp$j*d~3LLSB^8^fll@M)lO$H_u9xNFhx(^Hs)P|?vldwbDMOiV5=H#3~~ z=byqs4*nC~n+0|i336$m$j{H;0z3?_PU*aS`R|ya^9m~5zYhSPgsdzETz3zs*x1+@ z0sfT@tZrp>b?5gQ%c@sH_u=(1gC-AYD=VusTkT-EYdw5asF?91T=g+kqD^}T`G1n; zji=``a54EjLc$PKkegPc&YqiTui+>4_xFGE=8brk>e4~mZExLAzp(gtVm8BOQn>F@ zo8Qfj`?rIiZPeNohGp(epVLl&E_?@c22c*DmX?FI^&-fOSCzV#i@M`-^x5rDk))I0K#`r(8a~2&~~Q$-~Q6?SyOJG z3nyf8phQYNsDLbymy^o`<2LkNdcFNvvZXWJ-qMYd<8t=9jKwcUKNCA zLba=_Ymr`^1FCqAMc>?9G6Nyz&0!Z_t|m**HpuZDOd`hMA5CDik;NbDeaQuXSk}ef z93p_o11a#m`jva6g1LEaz1YrN?^up35d`miR903Ncr03&o2&l#VsBfjRQwIZIwpie zE!sTo2|!;7Vy2_Zp7wMEh6NWF7Zm}deZ`R>81Fj}FjOyhs{w2>L7xHCN06=0&g#HK z#`_Z1C1!ZQ0U<%~$+%RHv%QzZ_wUmIfeIoTLNqrEn_F6T4G$~j5FzI0_2C6a78Wch zv!DP2FbA+E6e8Ydz<)D2?M^?AaQ~Lfi%rgFCl1gY?j#PIXlrY0cv%QCzHplIUDP)6 zMGKJ%Xz@Pp93Czk{F0KCWCDKSr ztm!ZjH2dSnYNCa%K>(};;t|8L#3rsfHw;wMCZe?QBn zw&b_VwKa42cmPPG!^1=T0wx9qXtWge;_#XoTr$+4k1HY~(r?aP59D!zwjtd2p~Nok z>LsVQw|Ci(-119qZf?R64uoy%e_CnQwP$9gZZN_Mh{O+g)RB&@m*^mZk)U706|J{_ zy}zdLy3bx2o4K-S5Z+hTY3y2b*fy2J&~S|4{ixTm?CVG!w#?c0n*fs zYj0#Qf;Zh^z<&Q86~u_N4=`HPz#Re4@k~_}A3i+}CfAEH2n@9G-fR_F$ZpqjO~8Zq z*>s>l3AN31Brt%q3G(prBZ$#Y$;m67f!Oe(QI*TOJUkqT;*|65ZmCb7J_U9lhb^+N$SXdYm1UK(Y)OfwrIgI<@f8+giXb2lXo3pcX|L}02 zL{>YnGr_PHI$G(0<=5A*UyB)FX8k1M&G)Z`1)z|_v$GHS?C>F`@$qq|+rPd#dV28Y zt<$VCieC? z{QPw`-r|#5ev)w&U+jOt)qoejCsg3(gx!68^uXW&c7h8fTwJ)}Gi1)IGU4R>xa9l} z#2EPH!|{(aoIr-bc{ehG2Of;`$rJe15aSaQ&VYPD;I6H&!#0tfo!y;9TSWygJv|*> z6bGIPR*2GJ_u=T*0l^5LlJfX9NBi{Y17P`pG$ke~bYAVHWo5+%hV}6LJSZcB+Rfd) z$DW0m84(ilJRzNQN=#Lg|F<~6F8HA05h-he{MOc1uox<5FDa7|P|p;cJUs6!Dmduq z>ESlRKq}FINCQzV?cl%;CMhUN>~FpN30(7x1OXF>0JhNPu{Riec!j^Zy1KZT-(^h> zL=7DWhsvHYG6e56a2*YN`v$*$;App^ZQ9ysGEH4L@)Mvx$0@a$>d9&S3YRA}ot+Z# z2??}JOc(_!8RaK_MMU!M=ZC!Z+Am)|0K)^98gCviEG+EuWPDk}R^^QYuv&ZJpQj!I zJ_#eMs3;ZOhq2Lp7v891P8hVYt)>H{eY+PJPnQxhL z5H2PDy?cQ*mLn1%PvC`fAm3x+;@&+}_-G6?&5LiBV7S4UDqM4QPELN7ZvgoT1g${1 zenU3bLQfAG+$$FlINX)a*-K50VB=d_7?66BDk>_9is16P1^Jo(vwgf=@FD^jTIY?g z58%2aXlZ$w_Kf9xzv&>fZLK6DNR3JJ@87??jL1SBd!vF;ii}-bV>$3C1-J!_6e=Db z9x5s-q{@rq>gGnAvQ?DgD^m7ev*o7tp;0Zbd&#_AX<^|VDk>_VsU1fBic+$tcw`dW z+QdprOX0_S_6}WS+QJq@=rJZ%*+V z?bsh59wOepeO9{}=%rsbs6^Z3wjFSI=mf(Dm?LR0vN0(sVW+z@^dNkJ=EZ=3Sx1KM za~SGxl1cQE+LpeFe$`7lkqtySg!t)yOG(164?$m(o?s9$DvATJ)+*P3gv(7%N_uC> zKh5;|`eXz?Iyvn!#UDGhK&p}|KmZ+nx6F8kj~Y%Xq_uhjug3#oA!1UVEOak|&s4$9 zhv1h&TnH-}6s1sVl`DO(bEF131D}jcw_;6!?((O0lo$2Y`9Tyg%!;2lvkd>d(A3o2 z6?{mD3BNK31CPoW7`UiYVSv5_M4N-d%fQDMm)&TV%pV0^H5U#HakSk9z>%Amx4fQc zbRVP6?~@>AX)5XTW@?+yB{y7}3KehG0!4vG*9n}i=C7yzM=R2Bn-*IaL(iWqjuY^P z8vMqN`g&6Eeee+~c$EcMbu(ZN;LS6d#G#(`r$_&sk+)0wOYIXWDKMj^{4+8#GHVA1 z93E+9IXRL*x=a?oVCClHV+@pdCM5;0eV@-l{Sy;$8^bBGmLutTdwX_tbXTV{Mg#5b zh-7}pa`zVbyx95qc{%n(jS{W0;=59*si}$<78a-o07lBdwH2%PHn?tDfl)BGwN*H1 zIZqc@0Fq|@3!rSTo>dUNrsvz{Oe(z&D>e^+h&!%jfZ(AY=_>9+a#M(l=1MKJ-1ge(rk zbJ2KOi+WM6|KH0XMx|sa;BG$~cLWIbyqYS0VLEf0Fq|%;M; z=;`Ts0VdqeOEjbdT4-=&qyX4|z#t?T+34c@j*G;J@Y)t;fH>5@xR?&NNV!^#659e_ zl&i`NmztP!Kfny2@xx`o#=|QBt}$7_nQk~uDDhjmhyobwxjIJ`Xse>_Z#D?I9U$F) z&DDmb^X}WE|F}#+>bi-??#H_cr>l#7BD)KK>V=`r3p9($V5^+ zZ)|KdjYUELkF!0@|1)=YrXnsbPNR0$puy!K0fFzIUv;`af3|qny+L8WgN%lbK0e>z zY7HctDRQnK$jqqVV7SypWPg93EJWO($ngVfo1y=WI}0l-Jslkw31o4gM+>V`ff3Qu z7v<%BVOCnvxS!}1&IuS4SsajSx%;2nlZ@Nw^mHmD8sJ|$Iy!j3Snm9&tN3(LWCYwz zp@t#=Oa+LUhDO}X3s^s(cC6#Ajg7@UJUqdJ>SZSE7Ox+>8*&eEgeTqg(i`Y#|c>%vFH9b_2BP;^Z_+su2;q1=yO$ ztoJTA7uVPfNvQgvLmnTKQG?>47B@MzoADe_d8c5>W zA!%7z4ItgY7Z%TX0-^h?#p{$XE@Y5{m6et3iN#&}xz8c$>gvJ4@I0>k!6_3F@Vl=6 zlnYeWXdktIsWVY4@HGHdGeJI*4XMpwA!UC4%v1I!Pr2A;jw4U~IqX6J>6D=`*}4Lw zyaU)ep_}wym0rE`sueJUZh)o*6JB2&nt-PzvR$E~QoU+AXq`wF=Chp+7yWy&Doe(x zQ>3X5#0D^Qp>PEun0E^UOHdxq03bzxvLx0X8I0esFq^|c=!r269VOBY!jNTK+Qj9a-sd73OGk8>D%dU}9X zkby4+H>*tMoR(T}tL7^zDvD1_GbmV{Qw2cC@4I=Z`2A3S()@vH79tQ>(G zHalJ$;KdEUVQU8Nc*o>nBZ@$mZ0tWhv3FT7j@ju8j7`eqq$X}JegD8fM=$}s)7B_0 zRpv)tGH@j)7nfp>3uGoHCV1flsPPD!ra!;8-vJtg5F!XS(b)f1K1ag*AvbnrX8a~W z%TsMlO;oxS(rOu|$B%J<$LTCLXo`-Bc?+aHDH)k5%sW@R6(*LJGh1K2Pfgte(cmd% z-YXmZ-)(swKSA&V%{BVPUM~{eY%ht0g(aY>iYu6zLY!0fJ(re|a6h+MDOVF}A^ORy z@G47t5SKg8+UW(z!mBk%&!7O0rg0k2SWl`$}LEEi=(7h7!6) z(qAp)?A-~2ZCu1lQ3BtLxHx*d*~+&$9i-kC!2gsgH{-gl^pM&z|Fh#BUjk_0LZdBU z!LYBct|$n4XEopgUbyWs^qOmg)EgCc?6!ORL}MFZIG4-n7FVn|d(#`&hZ0MOD*Oci z!-`;0;v$)qVhW%bZzZs5+m!)XMe#KaE_#au$!`j15ITg!{hG3CYQ!ytdPDrH7`jF3X(&d}%3aJS@ zmSzPg#uSJF$P2=Eb8{OlP@#r55CC(+7B~`D?09hYBr@0u(9rv!pw;%bC~%L?g!uTv zir0xqc4(CZ;sml`{}NFt47D8<%7CmciDybm-xwfO;kf|dDIyyQrK0XZqBU)zWJAnB<}7UY^^N`#^;!!p&>~R4?b{1G>O_j1O;*0dqQU(6ec^e z?m8*GMjOQvuaTe8^4dBWRO(t~Ew6PE<^BhF-Y$*eQ+9dM^b4ZW(gAMh;?$EBUrN3( zRMgWU{mBcV650)AV9TWAszo(E8g0w>NwX7_kd!zi!F`;rQaLSVlC8V?*;8V&#o9_ zbyi{FP5UGfaRCZraK)Hi)$cy;vGi<)s)6=%crRDz?@<6frmIbjmu*GH- zqbaUARa!1V2-@xK?aJC(dnejguU=V=WupNPd{1@CkHN_*qh$At^&ZFT2y)tI2%0-I zbijUt&1*L*x6;CKgV2ic^WVOkby8nfSMqS?m-i@ zbP%dd`uX+MGW5beRR|$bXmh%cB!{Nj9+Xg(FFQzUTx#eg&Wj#S5VYcd&aTS)qX}v1 z$%|zQ!n6375F#+JhyGr*SsyO&HYmv>^40le^oR<`ZRVrX^E~yc8cLXIE2QXR#90zL z`}^f=O&Jr1)O2-20fw$^Y)G&tDm)~7E}N(RJtkN*VCQAkVQ8YSbmw|Va!M$PCWGlF zbdB|*qHf)%|AFl_zfyQrEJhg}&&+@7rZwmKnQyUvBkY$!1ou)p91N;daASHOl z7}wF>4#5klGc!S8sx(=ofu;aP6gGzb*jQMqfX<)cPY&o1B3?kP7&9OKLx-=#kl3@S z8_tr|7O5VkJyuBmDa9V<5RjOa_3pb`@37%ldEinCq829K&)A6OdtQNZK~Rhv0~Ojt z+{M9JURgPo8^%frsIKNVx3q+dF!J(>hsxE9!qGT!im4p`t@llD4Wm^ii?QNJK;-}i zqzQuFi9Fked)_L^$yv>Y$fE)~I0zggJ)JDBWn2#4k`Oe(%47g2+R5ELu<9smrpnmb z`VovcA3x?hl-C!m{Mn<*U=J73yH*f2o6@_pq^I0WrU)%zyihT`PNndU9keAiuhZh< z;sSbzAecE^^7|T3P;r5HMEB6Z04DHH9Of%%5c~~JPKb;QY5;CDF>*4Jyo<9GqOblj zMz0l{<_J!utV%mF5(SzVM_+m>cZ2@o(l@kkX+i~mPlSW2pFcpIG9LVe6r5)L*=A}d z<^#nWt>8PrzQ8Yp4GpDIF+(cS3^N#g4W?{C)dQqh#$u4G_=_K}OihugsWTWa?X(RIsZ32B=-AnD zxs&#)4&KGY5cv4`EbZ;x4UytTF6Qd}ddXu7*B_+w`W80s@Gh$gpV=z;LqZ+H0^e7) zb+xpnmjYc~U8np-msI&MZ;#!;^Ga+o$S1zK6*T2LlPibU}{TGPcM_A3qcgr+a96`hH}jlA}39Ct+#|H${iHz;#}} z6iLCZio_Kb5iv0_$)b;uQ*ML|9p>-CO#Z87kR9R!S05b20v1)tK~`Qtp$FHnxuqqm;?_=f86OqN!NEb5nYhu@ z)003xZ~UG(dxlH6$WvIO9^OOXQhcyO;B?lI$Q$?#B*@UR_%=kB^VnJO~WA!^p_U%*GauvGi@(PE$sFY91d^T$CCW zh3Do@?(Ziq)epGX!~|)4d_1e50Nc`%N6gOlw zH3=yyDU}#xmX_WP3=E7OS_eZEE%%19=R>6j-FmXFu>#R&Q3bwdn7rPmH3{QXkg%E{hrYt zpPa;`#f3G``>d_4Wz)3b2TDuR(2@3=!#iP8vH{1(F7X3HngeO-n7+OuQFl;0%S!9WXzE)Kh)`EGBo3@11D z`>Gg%yLZiN*>Cro17{T@uiyP&Ex`Ft$x2((g=I@~bDF158{~>KleP5|{}IQT+=k>S zKEG`$=4=V{cPPvF^jb7*Vrt4=K}qReX<6B>K|9>Q$N$z3B1J8K>-+IHB!z5+R%p`# z=cIWq*@v>`W(r_BoSZ$nchLfF)-*izlwg)PcxC&A)b$yq(~hgCGP2YdE#O~ zhDJls{zBB?r3mSxguz8H`&%G3T1XtGOW{7&s8A&I#YG?^2_G_>oDsps#{SonKQt^1 zZm+Q@(zG&P)OBd}y*?;^?);}%bbM;|AoJ;OnmbCH{EiwL8s_Ha^+aRtg}R}fy;OZS z6hVj<#5BsMv00+WAySCi+I!7ws*C!OA0VVL`3BB)Vq6rc8HlQ z3;6t2=^q-5&M+FCjM8|D@Pwuh9}VsFBZ-@qXqDxTjg8?nj{RC}5&D+;6s6EhqNSVG zcV`dkM&U%&Y#|36DZ!GEU08Uh&S62KV&A~&dNiG}>Iqk!gW1gDPv@nmoO%LE6u>B- zl9G6nk}KiU*j!vynnjwrNX%;!aj?`b;ShretlrnQsuvMYYkmB5Gp!j;Du};mSVpE9 zDUsElMelFgvvc6Qef%Pxm6(~;`#0mC9jXrCMT67lRyo&IBf7o>s~Bj>{(-{htIRNf z2h%$U#G@2^-|-FurP!i^ULaLvL0c849XCV@+0xQ-c6JsK?6lg8Q_t(#ajIZRlfc*V zw-XCH>~JVjM5lU>JbrM!vbx%V@VSni9UB%FmNE;NJs&S_SQeeSQ;28uR&!ll6Yc*=a6uW800g!@*7I-?xM@dOp-zZ=v{&TUisw$u%=|QeK zW$C3Kii1>-a2e+#$PVHpN5ra_Xkl{zxCQFqIS4vbpk%!sD@xtL#q9;g37WcaO?jHD z5`l?e(ERPhoFKn^aDev=KRQ1C{q?nQZntb_c1&d43mDOa&iDFCUyHtP2z**vW>0o{ z_Aw!0`Ra5xSChrJeHiZnOl@AIcbAOKX7Rzjz459yTWqS~+(=jkiRhEP8#lB*+jL9Q zs(VDaKi?1*JFd=59LjP61UA4KPJaHbN&a55mHIKOvF4;{ZvWd=W|^pzD;*$kT-uPA zw0gM0S2dEROmPse;jU6XXWuPP4Fv#j|C+1K2?z*Kr^7;aN|U_Od=Kg1f?TAaq%^U# z4C?HZfIFQ|Tsd-CcYfNmuq4`NG&$@)>L2_Dq=#gLS*VC@QBjecib_C%kti>t| z-D0#~CiyL)B6S7bZI2zf&r|<$Ij51(lk#@)f;v@{DD=k>CNl>|P-G-7+vBfXetWEM z%sOok&rixVUOq^|8y&oTXl}VFS~i?0`2zdwF_TFt`bGfF{Pe^`&0TD4qPe;X|A^ys z5%0sDk_rj@_oQ@Kkl4X-?lw}rhI82mDY|DVeD**Y;!{)OnbAu!^vVdo7BbLf>HGTh z#AzwY%*n~^y&|J+{Es3{7J>&4FassVaN1vr8vL7tWKv`S6q==%_6TqtRdFE-p?Yi! zi~NWk>=!M*I0qCcib7MwGa5YUI4zB0-ul>Iqbf%t*1?1$!&F1-; zx!(Tu&CShaU`>!G0XOCS-dh3}$s5Vfc=n8qhyw-X!A6S^_T~tMwPr}-72~E_O5>-L zq-FVwpOtkr-pLEhyO32MxTOvUh?S6tC~(HmTgmpxkGWFA7Bc%EU!naJ!|^u`y?K&SaIBu(T@`G+KRd zrek4cj5yHp4XH>J{h4<9eNKn|I>W5;OSMljrj9_yV(EQs==yTT$RA!?I`tD(1BSHe z^}&a#DlW``N~(4z+-)K*ihC$5*a7gn#;f?d!o%j9j@B2HrvkA~shG=izyfHplYzcr{>e1ZshOWiD&nXu*zl%Cur%dr}X9NBF zIA^b=fLT(C4|xz@`s3Y>7N?eaPovY#`}fo|#R&-RNt+Kux4cR)m{V`u{%V!%?B|~w zQAW&6sruuOtAYOT^KXM`L2>h&!kd2kv6Ua2z6VC7Wo2a{N{(~C+ga=xOfhZxzWQcI zw>v}o@L`_%zLBvp4Os*u{uceP)}80A@0s_smOiAWP1x|FqN7*(PB_l{=^fbI`z?b* zf9cv)adCNN7&UDF(Slu`2BjWnw%KBs?CR{-P=o7cxi#9&pL5Hp;ukC`Dk>53c~|UW zSw!nOB%`%7N5`k9J;2C7O$(7xb^L;9KKoh`s7^XS)LC(Vt81P$&B^a&+p>$ za^+w%<#q{w%tA;7mdFUj9ycaOIbv7Yak%6SUwnOrQEfi&HRg_ zREQN@ytXD0rJ+9oZaOzNKkNLMj)&*HE-mdO7G|$QU0gz%l7|QG%kNLkU%dkV5z)1r zY4+!f+reB&O5^J+?-}Ly(^w5Vt#;oV-#zO(=(YQz*{!#L{QKhuyS%f{Ct#TuHHmxb z^y(bo*OdX=mEFw>moTDW;A$!|#vv=Bw|=brsEZ$sDETT3{40;`^w%3x8yl8|CXb~i z>5LacOY^=zdA|Ib-((k6PwMXOj!sV(HByUzFdhy+m8RqQQnACE1y67C+4>FHMzWzP15`oArBe{b-nZt^(j`}s35 zrO|yrrC)F&|3#ST$T#-eBN9esU(%X}`3gh?vR^cKmPZs9q@?*1+wmlZZ=Rj*)r@?+ zxb2Rl6e8s`M#RR&8u25007fhBr1K(`4oQugUa@Fgi zZ+`vm`uW#5yq~MtSYmWPERcvvc}*})*|GhR(n}EIQm45X%E&XmcM0Ehv{(0JaG8(( znz>QR5bz|!!^5))L|y%5UwHWIQE2CYhUIL7x6$9WmQuai)XLG7k#C~>o9?0)Z=QL2 zJQK#|`AFTAEgbtMLH;x*o0+yuuTBdNFESo0)sRLy_ERE@+AJVok zdcRdh8_@2^99$xMclbt=_|az6>h7+FJqpiRtM%S&ZANr-w0xZ-usk|!AH5Ndd5^GZ z2NDa@4I#spMe@@#zC-_8-;f6yOoL2LN#^!Tl&@%|(uy4+$!!eO{}eR-mW8DyNJqA`6X{{#zB3xx@?+-tZv`4Y z_H->uhYl<5gjc-dJ<^(=jg3R)^Ss2>;v1n~Kk6(uz9xlJJdJH{Z#N}@8;eX7sA#j@ z55u}3euLT{=X75IB^9F+`mFmHFqV6pRW1B_!iP`!usAqIY^Y10SdNUf{wU*dn9mhf zHclv6?TccysPPmy4CAq-m6s{^!BmrPZ53_4xQ-(XCM?$G570L9LU(jL7%w*!J4};O z*R0fB@(<0HLVgYH;!+70Dy9i0L`UPpn>;Cwt+g~Y`-l4o+2@4uHsoS~`6 z$jES;_v3%SrG)$O9v&Xju(F0dk7Is9-pLGe#YjBasIi~#w;;t=#BcUTP7tf{W49tz)gf$B9=o9M;hW- zZA{FRI8Z!0I@_pCOka<|D0nevEI@(FT~qyBWrRkA{>iN)dML7B+Wj=;Amx->?Seml zo~GR|(+X2A=>o`dIA<~=+<$!U$r>*?=@UN|8MBZkY70mNu{z!@{SY7bF8YT;&Eg-# z(qkxZyuIOuerl&%WAA`+bJ_U%fXe^c8Gf-Ni+ke8wd_wrZH3NOdcX zbbO?RyM7%*3f7Z_F+0Gs@1M}N3$;_w)RFbsp&w$_mwQbWX%xu47@(JpJe>A7T3Jpy zSJ=G@TFzlQoLMd`(OPa^{FWi6Vq>$z6q@}N1R1OxJ|8Gw8l4e6>hq>Z;kT8T?6AN0 zWz~q|Wp4p?Yo{@1=8QKWzE0!OoSEG*Z_lh55=C#5QS!s{^M{-?G-6{|44jqdfXvs& zACB7$uiE{7s+<cKomal#-Ggm!Lop_ynNsc#S4C%PZ;jgP*MQJG;Fj1WO zJxkl$v}N1WT4jcPlP^{9si@S7&RB(nQd~E`rPcoU6|ts42tt$=>of8q%GF1AY2C69 z1=Rk*Ge&;px){Grz0gx`=fCAY{xSRcl~72?K&fFvyn}(Dq5JpvNgfN~-wd`hj-7U2 z^sT4G-Wrbk^t5Td$&%W;uWzP@#M8%1$BBUQ9f zPgj}Cj*py|K3T2J&%DT+QghOeQp9U=!%gs^+5K#WHu{IC*~tQ-C8T|f`sa8F$c%Kh_8HNX37JMbFSiH%)n8mKlRF?!o#_&GXE_Lb6`u)}_Xo60I`beG<0&*bg2& zm@Fewe%J4nwp8{12a#@0SQ|je?+`ET~cRr%je3!ti=7^Az4`#{gucvtX_U(RO zRaF&$yL`pw0!4DGpI>;J>dH;P@NX^J(l|EXm^^1GzJD~coFjbQ?JC7%*F|FF+tqz! z=D89xsL{>ANrOO-Ips`JOxWohPEVDFw95HZRYlxH65q^0d3>^PQdWwS^P z)j+0|nj*RD#{LPU$?K>W2}wyIK$1>Oylaw;VmtZ3LxvM2!t+ij2yZ#7FT-p<Ot3;EfxupeZM*I+ria?t3j?{aNHp4WRAI1xU1f}||!KoDV-sjZj=Zh#l4OWky zQ2E6E4p)>2uq8AU3J*Ty1Zp7oPQ@_B!PoVTl*W&{naB4`4%_{F&U#h`PtDXzq}7Ny ztal{jjvq=n2Mks_wRjN_cDNXGCsheiXs0|@p?lcfmX0Hb-bpM?%g;|Dtp>l@5;q8i zLq&yAg;blC{Ot5}fI95({<*7iL5+T|c(Yogn@!Yr(_V|37s;_FWD)4nYCH(`+mrhe z`XA;`9v0`QpHGe2)J|4tYt!lAe!Z{oe7~IH!P!JRpiQ7$5J!E z-Eyw?vIj5jeR`hN@umf5@%Q$WB%N!7Zko;>-}xNm2nb^phg1B<@$779_ao}`-1`sipm!*m>JH7CKe$rAF8EujPByL|tTrmGCgqWStaf{3&rAt{Y?cPQN<-AH$LBi#bh-QC^Y z-QC^Y_3rKSf4MF`@PXZ(*>ip|b9Rmy>8TqRMGUMok~CEVx=Dba&0JsxJT`;ZsRpOU zO<}B55w+-LjY47(vVs_B{Z!XzU?fK^PD;>#4I7sqVA`8QA+SkYcKj~;830f`yth^Q zWd#Z}FmG%h*b4SaG({#$<-OY$L&X3?`|G#TYPj76Gd%=Med|5f-E|h%)pi%j!K4!5#GcN> z*wEWno|XrH(Tom5LdbDI&*oz(HwJ2-E53X0GcEdt6&ogt6Nr}QjaV{?iGt^0@<2wb zA=|bQ2o!mCR*Y1m>2w}wA3Q6C?=7awdMs}ZxU|x=#m-nl-kIP>M%e)X;Yn2Ez&BjA4jSdIO zG?t{av=MM0MfLOzqhn$u4Ge0ZDl*BZi_`-8AMFdY$`Cy~T6eJed0Je~pT%t{M?NHJ zeHR5rO8kJbiE`of?ym{)$lt%yKP}nF`ucy}t@+YZlw6srjjOR8gVyM_WLBBwPk{p*)!`x_3a6!xLk6YTTLORB$a8W)3c@2H}3 zDFk9rRh=>&l&L|A&&|Ojo~Kqy-)9c&oUq~9(8z?cK4wjHi)Y3j}6q zSIY5|YRwFn<0ErCAceAm*5+P1HZ9|>`f_>x}>Px=-z?`8L)n-NBn_Q;5e1W`^D|Yb7WZ2A}AD#Jrn`^O>baw zCtseT(c!e;R+=|hC~G%pf!5}5SjEvhLmJ&}zz_}tc-)YSEg|}HPc2mzOaHpM4C0UP z)ap)jUWMP3HgQJ>2T2dRzaiqY_uf%@{CXN)adL5qN^!rju7xdKZ-^qEdoeT2(w(m` zrHvSSMGC`a)&@vwnca>2=4hs2SNryOBCM-V3gxq%ZV3GDnrAU|Wp)tQ9V7S9$Fe+# zjn&tpx>i_UNF?6U0TbUfH3ezzk?}EaDo831NXhy%NXI!qP&1pvV7*sDu`N+b7mFtL zXfk^rF_vn(05((`M@K$W2if9zh=87noh?xv2UIl|_Z!cZNzLvjRO~(g8#2Swx|f!| zx3#uZES>DzH9q5OHd2yk@rt6NWr`KvI9~4jkXc{Tz4Un8P`hr-;Ve;Q%9A`oKi#`K zQ62+pfh_43uK_)WI}`x4x{fTH`#AC4#DEmRdiAI_n+TO93b205c3tjE>_o)l;1dxU zAyi7%+1ga8`-Qr|&VEWW8zee#%xe`WmzPDKFoJpQf+O{Ton7dT{#5MRg%$w&;LP+1 zBN}#p4B{x^h?MmQjJ?ZkM%sxAcePg4bAl387D{Wn0Qkm~7f;-V4gA6p1O5_4wUoFf zdKECyd zo7OQrI!e%IL`*~!%`Y4dUKT=pwf`Kw;2Rf*K^omK|5Of4JG4(K*r`3Q^2FDHq7Tk0 zblu#V2ZpKuR2)C3hClSgzLRc@^Kf_7xY@lcP2p6(uvfaYUx>>Y%0O0#cI>;P3@Z+c_)E1jZ!NFlLguGPO2Y=6e z(&&#hs_rs2|4=D&Il63n+q7*@yhKrsj4&A>`0K`Pve)&-}AZidGg8{L%4M7@f)B#Xth3U}QHQ)miR2Jf>-~El&B?gdjSU9z} zuP-Um5uWC+oB7si(-{TBTZ`jb&sbY2jm_qP>civX`p&_CF4CCx;r7Ke{A1IzYnFc` zJnnK^|8qlw;^t^y)|{&>Cspc5ge^Y4O@*eno8|^iwb?Q|=`|j`&sx;?e-?iU9(vGOAM(4k>EyWX zhoX2UIAI$tmqV1bTvI_l-BAzB`|VgP`+hN7tBF`&U&p@9zylnVMU86HyAJ?rSnHd7 z^v?G;iP8VnvKlgFm{p{w&1dST9@h=m-zF?!Lq$-SkmW}<`gcM!Qz4#}a|EH^sy)b$TtR5w*1u8<-S1{2&`XbMWM;;=Z)cA8hmA>c% zwYA}Exjo`xmE_J-1gx$?`y=6Y1LBE%;<32}q4?#-wUf^@W6o9MqF$eFAMh6m-6^jj zURcadkI-wjGe@yWKwia5VO)P5iT-c>nYiGxW&C%`HQd=6VqM^NLSiP}mxw8+)CjRmJHr zA#S1a*A7~Q_t>a2f|Z6okQ&qHcVtA2(SSM0*1yW-;@o4tTfYWo<;$J)_DF+ma4{Jp zC0vT4<3iMRi5m0Xbfktv!UcI-sb+{`5eZpeU|d|@lSiR(bl=rJ|I|v~*4o_b4;h(u z1cZFCoXk0D)(X?HAAsE$BG~{RJxrYxZ~yuKYXPVcl19KsTSP`i#LOvgh*S#? zFHZh5Wj1BGN@5|hf)vd;;yFb<1>Wh~5whzoZn}8hCS`WvjLG3#kq;~yU?IYUx*Fn@ zJyaVKA)8*)9hAu-E!tu}>5|&8JmSFZwT;C$`D<0C=B2tP0JvrapgV1oEdZlbYL#Aa zAA^pUTOzuIGxJeplX4}oCQ8Kn#8LiOXmB#TbdagkxZq-i`5!0aG$TM`F`Knd58TBT zw*NHV=WZ#rTz%4D&ODJ?CHf{M!Sc#0#L#L}YoIxI0aPMMLBk)53_ zuz4s;LV^SiJs8%qXZ%}zn@zogGB*tok?j#j9-FidYW3Cm13N$)29-sy)?M%Wld#6I zmo;}RDzLulZTk5NB(hjQ)tCOpWp}{In;KY$cqyHi+^Zaj%OjK8sa%_I6a!3J%nigj zlni9MS?xK%ITw5WTEWA1yT|M((NUseQXTwzKUO$4sx4-IyVw(foE~_qR{Bx_6t!_W zGXOZw7+K|vp${jb2z{R%8A!V(A^4rrPyA(UYrqWJ_WkibUaX>hn?K(|5Ck9?=%?Xnl zqZ-wi5d+h(i~ zd)YqeirqQ=YSXDh{|^?Uw$fHdUXlBDs$5)L`}3^`vCHZP+zd%UTYU_=hXRC0bJQ1) zTwHcaO_wCT`H7X@j^1${_BTiES^?}Ho1aILn61aAJ?E8&4d7}_{J ziHQ*`iUsoJ9+_0Seo2FzKWjpr<>KA>^?7%qA@$X0o}pg3@sI-wYP6_?7%Ba?_W*>E zhGC6oTk|I}S@d@?=3ValOgVe58p5|+X(FC3DrdZBf90?%Ru>ytI=R&uTK(|;z4gUr z58;54kPtLrxZa-cvEiXBD=Srjm-uf=W*fN_H?7zP`iF_rg~CiVYYk++B8?@w%sd$I zULZ^5UHL1FKjt0WNo)6~FUfxP3p`&u-O~kRO1TyNiYvl)LLT1k#D}~%wvYB4$u?#v znJzB)b2GHOR`z?ZJ9Rau+egu3t%TpceMuS-a&TZFlS>V{&w_W0-i9GbHCR;( z!{E;a=g-f6WTXJ@Fh0i^;D?_lGV-iFh~!&aCt+!2NUg0r03L4a*zYg~0>XHB_&E3t zsjyW2O|Z)}08O+RbT|{XUK5w9KUQ1g_SPlla7L}N!yt`>gcDbmP2xiZCtITrVG2tM z$o$r)SF*bh%hoFbgDPP7yRT2zQ__&g$jINI=thzFjn3c3g9JyWbElR%wde-?vq-xAMlTTOK5m9?8&{ zQ*%CAan{pU7YNAtk4{;uGmgDIfnZyRUcCE@VdKG_qq!h zsa|(o0bMNcgMfC-VIY_)Hkh(Ej~o`f@N;ylF>mGofY;;Q5r1?qf~6A(aGRWtX8^Mk z0%DE@5ZQpgIh+mBe#%{wDG+0Ehzkh#AeEQZZ%YwVm`=HgWn9Q7@7xBF1xFg5#wF<@ z1g)8>NlZ$DkuY&!XLG%u;}_1y&+mMY6Bo~BO$hk`@hb;uEUQ+U+_824&`?Nd=rF^H z72w(dZ+|GYS*%EDQj=J>>Jl=i<+;M0*4D;bWiX;MTdZ=t+z{=2#ghrloJzA59_yNX zYfGWh^D}nikUU}Y>{-gPIvlpg8$^vN|MG)VEcIJL4K~Xsb;kJeayqFr9!VP;hK-Gl z(^LQVW-LkXvc;>=|M_sYHV`UE>#IVV2amE2KimP$mu}b$n zyr?L-V7DMv)5Usq$K(4Txn^EVh=2&ZJ>_ZZ!76Q+6OCY1_s80 zwQ8bP4-ZUuXdom%!opHi7SEXUY*@`d^nG3A3ltZv^MHJLLRv4#Gu)DQrGH#C2Fxam!PQI|Y@s{_6BgLy@U&R$c4wpOvFaOVrL(j=Z=Z2Fs{u?;pE>}aCD7t>11`x9>G?f%K+M6BI8&s==ytgXCZot{ZQ zr4@gnteuGfgY!YpREM|t2EeG@ytUk2Y&SPI^JW*o%H|Y!wN+uhXTnmtL{n!Bj2UV( zcpBx#VRH)9?UTN^*NylS87V4cK&ohp_i3ZXcsUW`H-VCN^YT;p{1oUbzu-0m;-;?yrFlzB)< zdQR;)+WwGI3k)E-@yWB&LeM@wJXX5wOb&T0e6H1<(_DSqXEcyV`JSf98n+Y<-UUl~ zLfpB(Ks}|cS@V2L`QZbt;gF!)7e0|G2nRk2;PtB07H7U8L;ZH=C4xt{8`o!fYhs!4 z00N82WVEQP^)NM$KL8PjOWBP&)G*CGw!wA}FmA^4qqy;G9>WT#-KLQ#Df}nUCBU6Oaio;W zAAwiZ+1Z1ByjbdPJl_RVGiqr+Z>u~vm}7Jh5D2F7y51H?rKGgX9@ruNC8h438Hk~k z#SMc};(eh>S&TFCPHME@&9z7wau;?uS&%+kZ3#)1e7@g`D61bPOLOZ49}pvz*EKd) zw|u*kbf!37ElLFN&3D<)91Vf7~^TCOldq_ih4ByMwW&#|YQl86WrylN{6RVqT@gkBVe3GXb6XuPL43qR6CMANx>JGo?gjc^( zFC|}`tWRltITMt@LRkgp-EiwCcf1{IZ@oX2MWNY&B7T%c9vu@QA72&CcV&|u7P!WBMVJV)d= z)U$7O?jJvv9!RxIE$|r<0dW-98-_Je?97P8Wc1z6&aPsJ)p*i3l1RAR_0FQ! z;(n)4H^=*Ea4sOtZM6HT7yaNgiq}Cx1H6oF+y#qw|L{oXv~G^pNS`&~TUI6=1GDTO z=BZP$je=tHKoNi*;+NFu`0J%`=XCGbq3Nhk>-GDfR9>_+hTBL~&QMTTNS=98Z1d}+ zZMC`WK|>kGr;YhbGd3JVCM*PYc6K(LaCz-2CjEa4*fx(KC#U^D7$nL|J_iTHDSWr6 zm?Don*)EY)*CS?&6O9YOJAe4b6apWMz5_cHRM*rkGO1q`m@D8OT0UNn`( zIIRITC7v36^~tF9>9{D9Jed;%?hok*API)BjegSos@YYzznC>*+SuCCAo+cF@Vkr- zv-DY$3Y{_KmHt}z$E3n}%ADuE(PzPp8H}`?A9~!g#Y_F>KBsy+NFlTbCS^YhVBp|- zOloxKFh#_~Y=EIFW&bN|LRLb7Pv=v}9?FcQHCJP_7*1(EA4YY1d{A<$vU}gr@k1iP2`QQ^G|S-IY0mk$v(-`wxAUFP0~doS zmTF1t%Lk+lD-MXvoCa7~{T03pHGE(~QPdgp`B>w1nNVc9-n8)90=haHd09}uEUX7# z=@}X_K9+I@yKBgPCS88}n!-`L_44MrqOs{Y8#*vuf;tL}gUvzvh(rliZfeqKJe>e= zAXyVeAJE0+{wc)LZn$7hf2D5&Urlgp`Y}Q!0 z^+*J(p72tg`AGBKYQ4o0HpjgKp6la#hN{Z^4bKOBa%wfEoi4}MAx_KYpSP~N^6OS? zKw_R+1PDi@v>({1dJ#>k`{h^qf98@Ynt{igFY9E*bf<|0CiV5kn-d$?N2;i4^MV!7 z3V`v>abKG4X?;BnNQ-iyPuW@XupWqy;s0=FxG}eOxiz>JZ*_)wcLiADO*;-ih-c^K zX8tUgCAf3>Xgvb1-n1C?IVxTsR4`qU_PCf~<+^YGOQ^v>Dvj9}nI$XG)Yz`z3=r|b zas@Z<+ofyJE^UMkiLt3IJR+I_j@(y^exupqe4O3pGD8;w6;WynL zQ&x^#$Qb}y<(NatP!CALzMsau;C=AT`IR-V{_)N&dx0G(2#(nTN%?gFaUBw<1ct)==Egdl>fwv?O?oA($C^tT>{7c zNI=~tW^YZT?5@+G3d5FbgovI10x2bSMQu*AmeL9EA{-7z;(9Mbb4-VTk7CS|bda`O zX!MDnznU)o0*I;i3P^Ke|BH@Zjh;@O0ejwuTWnfzIL8fhnQYT}JU`T#peD;YBI-`X znyX8U)$L@np-etgX zPT!R!sU*Fa3-qnP7>C0RKHIJWSwW?#6EzG=DAq{kwvx?$mnsZPK5TnPd)(IOjP!AB zHsrhDy@G%+Ye+;q|p$&w(Ay zUjr)e>cw9H^oKi?gO=v`O8Y)E!`J5_5_l{twHnxQqN?1B2{Ns9I|ZEL^X;PZ^t?2; zTM~+z#gD~*z;9E57!DRb{~jW!&U{U>t`xUO0z}klT4nY6WTtt?xJLVsJb>D(abYsY z%YD%gnaqPK`O{`lVeiGGM{IKo_2S~+O>@v@l56luPdYj5a`_H<9y*aqlmNjJqy zMez8@7BE9WHx`oa?(9THMsCP_78D!)M<#Q6oK~3aQAtC>X&yfu9Hb(@*w~h2Ye`hx92**1t8P%vLOsh(-e0p#&@(Ef}!Oe z!c1#PR{kR~4_QGo(uBuwG_|N4;bhen6hacW%Izxul}1+BaXmCN=(mh|QzC@(rYa5Byni#U+02Ls9&Xre1XZEMySX>6Q|3zH z(SG_sLrhG6*1~$fe$wEikz2HPKTsty)}IMDNWs*IurPbJl+g+=1;TIt^3NajUaP}b zEJx=BKPhDR{Q0<#4b?rIw)HW05!$%(o}TK}!TCLj7KY4cKP{y)60*K{sN+8d-vnx` zGS7Vn%zBNO6uwtR_77^!(E)EPi0yn@x2j%u-)ktm;I^arz1c#ok7`CnT(d#l(oB0A zir~NPHY)TjiUe^$Jq>wMsqTTRr2MYd_AVA|3(fDktaVTIx_YUBV4%IT@9Rww&YCTE zcp>#7nZ)tSX9;A@$v$3(emyl ziv3sKkjYg~>&TAg_ws>OlA$-OA^KE?-V3L04{lJa;i|DOGq;9F{}M! zSe4-I^Y7XU^->-X=@Ulx4|A0oFD@sZXI9VM#<6Y>1NxMY3S@awX*~v28n-uB7Z^@} zVu7JyN}bE;J5&cq^72o#-oJaXtMb2wb*qN?ug$#@eELRATu&-0`faCiv}%GL6W%3R zY7afA9C_v^X{p7Um#Nj7;F_D(VGQ-lt8nZ6A-2bh^+%VMH~|byE*%{1NBCcJsV_aP zJo=AWP*6~swf+7p7G$KPy#?fhAzsT@z}+UqV0^#gDtX=}g_5EUd8!YD7N8PSJRHsy ziAx+m*l6tKQZHG}o605aX>)DbX+lW|GlA$D4 z&P*(`OiQ&pnvAqY&5Z_Jc-$!N{$c|WVt^0uR=zSaCb`@kX;mg?k`ogX|A~rvS1v01 z*O4TV?=OA>4Xy9cRl6w&u*Q0M$vs_N{F%Rtjs_)LK14cf=smoUW3oj_s?}zZ2P2k? z|Mkm{HG#Z%-zMnh?jC=)iXDp2dYGulptsGv%<=KTSLqpcHVUsOSN^jsv?@h~qdBSlQO>r3_@6A~-z2E8Xi(&59fJ z4-Yo#;glbyYp!r)8|vvWB@XJl1H;j1y2~OLOC;9hlQ;eNxiB*>S{GcvGFSck_!wvQ zt>S%*n8)a|+>eBp6VAK!oH94D_C1)UvCd-{MT2i~W(WEzc`oL$T?Kzoar% z1ISXe3N>@_so@-q)}1d%n)VsGvSbI&OLTayMjTbS(i<&El()AN+kfSM{+tTty6Z1C zHe0C;fga(!*yY5JOi7XZ2HEVmrsd@7s%4E=G$W>?lew~CC9)>@&)7CP!j_K6^7oTQo2)?$!ewZBzThJsfv?Bw1}lX-I3XNAy3kX=C3=)>4f5>2T_35i!ghoEECpo)!^0gY zf-m1$NhN0g(@ibO?=3o+Q$=C_NP2v9NgC;`#P=wy=Jc?QD1ELpLHHOXidOkrd~Ejo z>^9hpSGB-5eGv^>%5L~ba48Aege3)%Rhs*W;Ry+k!eQ4M)!=*`Qm)?hTBKbeU-t;V ztp#n+*e1c!{QEp;p&12j)0j!}a2# z<+6g4fB>Rs#zjQ-FKf;>*ne^-iTy3$J9Rt0Q|Y?jbae^|tpiE~_lX z%R`-zdiwfXXM0M!H)gN4oG!Gqf1X-=kv+vf2+0|Z>jSgu6A;m*$AFPw{erL7-#@%v z&}V?=%mV;h(%j3icN^kVqnz~X)9llZ1C{z)ANnoseTanyr>1plqdlSWe0GcTGjvK@NKfLl0*TSovTjf`^Q zyLDouwJLpcygp&o?Cs(gj-FlFA1`Hgf4c7O>q`OveRH%A6&oX>Esa)#(=po=W6p^- z^5!U(>S1uScdhMsU{399l!%g3I0Mr@v*-V70c7k&34GfXZqC20xBT7)o9pW*{07g- zmeT}Hw;Kdar`-#kCaevfDpW4S7EMIj7NV1J0cK0#gUjhnboyFwjW&0cmMS?k+2CUB4gqZZ!l??*>Ok zNO*Wslaq1eDPjgTtePR3DMq9NkA!=Hj2N`5FZDNuxOU+dr5C=qx+% z;XLi`?U`y2<^`|#)?M%Kdc58T$x*31Law>pK>}VEi`C>iFbnh*dly$X84f2hQPFY} zM=kXaxfa9jJM4$E4Ywx2ZN;ixl(uUIYvDJn#D7#&7JO$d1w+uOWeVi8b8@U-pPd0$ zW&ydHJ>KHL2aLZING^RMxHn$eA_Z_gcmx+)$>gb(_IHqD80Me`kw5y=&jS^P-$2)+Kcu<S~0bkzl@XFoD-d|Fe>ktcI-}; zy;Y{j81=?a%vM@MK4p%7R|NShI$E9J7p-Fk``8Y~B?9&b)Y z?ya**pK4vNzetFwLW_!uN<|MasmcC*i_ZO5NFZ;p>hA>r@xa=P2ol@PIczq(TgR>4lKaQUzX!n>^}RQZmX}naq|skKP zods+Qp6BOnmto8FcJ21k&7jQiG`_PPzX>CO5=G0HZgH^uzh|}b)U7pNiUoIqz&WO1 zHe*qnZ!zR$z4;;_WRNRx$dJo>t@q=%3$&9KA)zlh1lhS#hwp@u&R%H?N{ngwev<5 zao|cOGC{28CRykl&6fBE1V92R{irS;7ectRU0wC3z^?WQ2)D9L zBT&A&WOLA_#I($)nBSL2X{Epn zz72}iE-bLak)Go#q-x3vpeyzZ4?zXqJzg)!Idqk_ha4RRY{(D*m7MGDV8i zQBhIBDFb>8^fucfvn6VK9#Um@IvdYOn4Bg~ejWpwA|l&-d?^2Y&eQJaD_~<2dVlW( zu2L!cuS&(qRj$kem&4vnT>JIK1dW9LTS9J0Q?xGAyJo0nDoIfoSut0yQ4mE;Z~$I? z+TEH*%P?h)>eXd*g~TKyLr_rImh!Y{rc^TV;r+KwTH^8iU&Pdj%sK8&v+&I=v;ef` zUw2^H^{tpKH^(&E6V12jLISr{ml#Wz%5n(q(JARNZ}TD^ZE)i5dbJ36pD`<#IbS$NScJV^$a4jwTiwJvlK3O|NcjNpT<_PY#3j77Y2D=#vS ztnZijAL?Ugjk`8_tG26GsV-ygo6Z!T`q4^8XUF!d*P>a zjLZ+Y3Dqty=)-FD@PKldZ_uaqN5YmrzAMk$u(C6cONope_^<>$s!ey%qb)D1`mP=t z6o$>(+v0kc4a?;D>PDqnogWY4wBC2LGn&H!{Yj4ceNctIIk|JRzgh!nhRy6%A_+NN zSNMUMxpX>+(`p}+R5~dn%rq{;b1ikpLxhm;{tBZW{}K7_l(#!Rl3%v3vUd1FpetsCNdqz^4mT7a-4r0ml2aS|y7Gc!V5sd`_6ZJgEXCYrKp z{MPIYV@zszaC{!RWZDz}%e6fvsYXocv@_rni`P5+UL z%N5Vt7UDyjvCyYa$w`jwFXO_{_tnnOeR6ct$(t8Kapbf4N9acyoMsg=u}dcv-+L(z*>=`q*P zclSa%$P)a=nJ)X#2W@P(xnR5)%AU@w({x48Mgxo^|@qh*MHEZ9keT>)W4rzt`=|Wgm*qi4$?EDJ(}8084g!c$2h0UV>k1q52+GH_I|nbgYwz z&a%T&WUL?Lbpcxa@1go%I%#z(Fj?OkU%CvspOAtuIdr%d?T-Djv-QX+DL3v;u*)rW zHW?(hlvtU(B(oj|$*r8)u->a3eSnw`co#7HSX@Yt83TEB)ua6vzKw7b4n##om0l!= zkQaUy7qUM%G}KjSpH!j$gqO@!i7_gPMJq~ z_#ith)eo_&i_9 z$zjgc+5&*vhBr1oS}U$#!uND30+GKN{V5d((U3W!Z|ZolhVpQ(Osg=WY)n){z0#;Tej+tuN zscy`hlA}T9EIxdmUSQ6_Jt9G=peU7_Uv!wTC|ch@Oc0;z7uamxiKCCcO7mXVw5y$~ zRKFw0i1;HrKGxYB`)#&VS^m&9oj58gdV|~b6OCr8KY%ouN@bco9vCoC^>zm+!^6WI z@93bA`wDS=GT7p4_@T4Bc0mrXethC^aB#=V4X6M<3Gwm0JDe#Bz@XLq+wcw@7dH%$ zbM3ZPp2JBy3cs|NT%Qqt%{KR7a+TA%?MeiEZ>PO^tii6^9B$z4*t=@qyJ+7jB-~hW z<$5K9AJItC*;>>epPDJK63WTV&B)-l?e1=c_@n_f?yZ0i?UV8SHlv*m3e-NI53NIp z_q>P4Kue1x9R3E8CP;~l|5kcVMSbsvXCRdncp%ryuc0BC%XWv)1%1ZemZ{bD3T|s% z0G$nugcnLA9DY1HGJ-o=i1RPaGRRFJ&e33?a39=wz=zQxAw}q5z<*(Y0=LTLus>8H za(;W|R@cy=GhT)X#bqw$`xqP|-QI7l>2yVk=;~l5&;_lC7EH^?2-ilV;rGkuDN9ph z2)2hLQe?c5pLV&`7T@{d0T~8L8j!%A5Xk+7C_toW9*lEptD-jVD z?b$tP)16V(+TPto3!*^`@LA<<)V{g&%Qx1cl1ttg)6zcf{vt9q+&Npaq;05F`{7Rj z5`Je-4+$d?5q~={o(#761lrz+V3>9jnz*Oru|3D@+uPnErLtCJO%R>OvlF=IAWBen zcOU};!+3)&+4b2}OK0cNevn+btw~s~153U86`!9Uh76tQ>)q#H4S{$uA6pIC0;bh$eTXnRA96?1~{3JXK< z<;&pk5-J{~$CPlR$Gj}QB4Wdlf=?T}J6x#Bvk?%00{4Njx4%Jw+&?-Z;^f5M8%ulJ ztb2ZTyFQr0zGb8^X%x^3cjm=$$7kxP8Rbi?_3)MT=pwEK$c*zs$ z%a?@nn;XCAXjE`Zv-o&SeJw35aXGoEDl9Cl36P6dNg*R9{6nOVv|_7`?Nzz?9*-}t z=Y2q`9u@tjj|6B`*|jxR3B)>;@E9`3xH!Tz-{ z{>2g{>U_O}AKc|iOso>&3Yf*5?99wd#v&Vd|4nU3;@$`WI1ncf@01jBCFP~Ah1$;C31HLkHzZbqe`}07eB%(J>$;M;~r?0P16@L_Foy8rZnd^A!x9;36 z4m_gw9SeUaL}Lf!ai*}3K4D)5`>layS?79 z6ReHV67e;`%^(=hDJ;p5Yg~|Le500ggqlK|z{CkO2X=?V z$-;be_YI7TG*VuJMGQ0lrLu%$Pi2~AfeROpIY%DHHas@g7yog}`rZfxg!Oo?i)_$-2Kd{85f9g1>I`ME^961m)u`K9~Rd5T(vaDO@U* z%Fl?1XpW{)X=zB3l9HkZ2D$WJb>LKihvr8Y0L@x8EW3Gs{|<*ilU-a)MG?{!fE1mP zK|rlu|L5}ZGWeCpa1;mJnf24B7MGjg>ubAjw>|us#lAM~KT$QKf+{{WG_XaB%aOhN zjz##%IM8?dggG^{i(fCCk?{g;rP0NIwnSY*R|qj@mU^YXdp;2A9na12Y!Kpud0r^e z#Lu3XdzR!xcFw5p?PMT5U0wcxfdM9yY4XJyQw-6r9PzmRl@*QQtqU7wX67+SBgop7 zgbwY$NcNylp!{5GM(=$=%%o zSajMT6*aY~m#4e7du~!ve>*!na&GRF$D6D4oVL+#cOmSDDljTDMf_R*NEtaH7*QhR z*{TN2#!tJtO&B9oJ}QB8*;RI?*(rm!rvPIV%r5Vs>t>5U9VW$8U0S zGBD8S=5U6|{!l(aa~Ge-EghiCsF)aq*6&~Y*RLLuK+08xgW7a@*(-T~mb!T0TC6oz zh(Xq{%z7m#6V(;{x{jJQMj6{1)E1iEpLaKecE)>WIT>1jIxycKh=Yp@K0Y;@{B*g2 z|DE^O4_*{S7Vv<8ArCI`&?k$j~H8oQIt0WoZxXAV#31}5`M!AIU^XQ8TGX$nlH z^H*M@bsL2cUyM%^35eO9h+fidfRiWw>m&#yfyph?_&ZAlZcWM4)hh}Bfb5kax$0uCnP*BD%hvZ z9yst>-|kPhJ&{Dh+Gb{<5RlO6T5TTrnk|jf#(zYJ0e{0$Yqr2=IF=fdUr_K8ot)f^ z*DeYgC>CSH4E?!}*pB}_^3EAA;sd0+#{2@y@6=Mn>l?d_l9G?ql9EQo#(d&(kdH1d z<@p-o28Mu!NJJmZtKgH@3HZr>Ca}K(kecz5i_HXTo>oe};CX6{;@MY>sBB;c+>@ot%1s%xWuNRYEzME$J%o+NyWi@E!gbBSO5tvWCjR zF;i|kv}l=V99Opbea-v*tgqbEWJp38Mi3BJ4p6yLcyMq0Wasw>;$};=E&GlY%JBB~ z_S@UrAW*4R8=|MDClnM^dv7l^p{x+S1cM-P5I+l~%~-c-5P8Z*GM^sDTGiZ|D3gvT zYlM6F3s}f{SKiN#XWZ>6PZ>_<=k)68>U2)$$k+Q5kX(3YN&2T?-U3u}C;*@YMJC|v zlX*uM)wPig1NH83t&O+E{YkO#uw$|{?=4ogFF8SRaxXl?uX`$LleCh18{ zN(%8k6qHPx$BT%#cnG+`TmT|&=Ks-j)^SmEZyR4gKw4=~Lb@BJK{}-*q(fRlxM@+}njKt;fksvKLlo!8y~dHuR3x>tcnxO0xR; z8P+pg)HqRXwbnCC=$M%KU|34zEv>CX5t6H|`-}!~>Q8#F$gR+fe#+8R*Vfj;-*#wZ zZ_lpLF+Q$>NiIZu{&%(?G;K~^9<@f}_36&WZcKNh`;AIdJsQYFB&gWr{Sm`Ol=$c+ zl5`g&$J}5N5u-$wCc4T=^ip33fN`^y!eo_HwOFQ2-(xux7i7Zeq3jHGZ&F`Iyyl~Yrjl!pN4sxIsR<3%#o zh{S!2{^#Vz&EOfLfPT_i$2^JoMKSN$a2^% z@_qZh*gY4A3Ie&(wq%xo6M`wd=f<+3Uv6{G-JRz4&JHyrKXOQjT-IG$cuR{n@rxNT zB|Pe6;uit^%pMrqlDm3JL0d>+a6i)!hv{nBx}my(10?2mqPyt*h&6fKt&J8R{&!IVuoZS~IxL zf|>jh8HxP_@$E!Va5PZD^z@e?8o{?i7SE}a{g}`{W}>k=o+ClbYmE3l(sdvH{EvYRQ<=&sQJsgfs+Rou0;y=^j{9Uv=r zZv2HI^H1ay9SpAyJXnT8meg9mS&tv~3n<18D*U!s=U%vu?+|<_9 zsVM1U%#>J3=qg|^F%1R#+@nBE45;ZYc)uYcSf>&b1#{x%(1F(t-FW&+q0Gyp_TOXbX$l))pi#sRbw3goWOPFSFdtx!OR68cj%bTFnU?+8d zbHUQEw;q2*^d4G4`|nRn;o=$&fI?p%&*=I@!n#j~;&9Tih3m2WTp?E$1A5KwqV zNGN@}Ge)nVpirXQz{aXw=?`kkERvpv=9_dBxe`?jgQ2djAZCY=w+TW-r_tT`Vt>EW z2W)KYf(u%OB$gbx4Vk1rQ5OfxfryBRyD~H!9MPaP-+5gg76c%n;%1G}V_`*fo!XH= zE|7Mghy`lA0Rn&JTLp2DpnMjcl0t%vf--bg9UK}8UltfsTFTt&4ihh|L_{Gku#ewFl|_er;e4+K zk>lFfJANR0ztBQhXTJh})Kg)U8V3hQ!27Cl(csYSXt{tIrz9(9~H?Ny8x<8L^X)EZ!{rXk7(I2G_yag`sl2ocW&lDh) zB*+#=$zRRkfCy)?*pT&*;&N6Si{o}RTPz38}MyOlQv$wiz3oE%L=XbC9v}LMKT&bV0z%5OB#& zP9{z;7Z`#`W#(@W;?=V3s_No3iUL_a<)>$_X<+`;oOXaINV-Y2}1M)5n<+s>Q^>{7KT%WwLev+A1y6q zowi4EK=q|`?`&_!mX>0$CjZrc#mNn_QFVQZLKQo?JQ*EGkac!@((vMMPHLDsbrN5@7L^?TjYeuqX4y@ zg4*_k%xKq`a6sZ( zVd_NGRW5YJ*%wtMqd*rEFo-%{5)dTliDW4QB`P{N_>^MsR8CHg$8wyqM7;f3@1PV(g`}_OypfQSecP&Y|J_c`Y zq*T^HB+E#+L@uNXA)g?%khBaa`cni=xrcO3~-q>Aia-yvW zFDfXAiHL|$yjS$GVIETOQlHjhJ4umq+WB>F+^Yr1#1eqZ7J6D@qBJ;@LN@*8XHoss zN=i!W4t!6>(E!Q&?$ono}7bPbKS zkI&%m!dGVA!E8zXg9=^}Ur{d#5=ZqJc-IOPyGr#J5~l%w+f%hLFb6zfwMxOXY z4J9yJfoQqnE%po#PXaUsm6w-SiE18-QVLh#;c~0)L8?p?c}#J!73XI{5)!%loAbD* zPrl?weYNe23gkk6dGrhMn7&OWCyB1B`BB%|Mnkb7Md7Y$<)-; z{IW7Vh90{dBPqo!UL-IO(G)WX>6@q5`9(!qY*Vd!dwV>t2ihecYUuRayqmB+&{0aQbA_hwOhLFB&$l@)`f(9sIbi=C*v+aI1T5-F#)=le4n zc9bW8Uru1K908a8Xb_i1vqZBDTHN27_DA0DyXJsdQ?0X}Ir%2OrioANWqjv0fx99lj87 z-W7X%xP>Yjza%3iN>f~D@kNdx6Wk8MGvMdve_!;Dtl4oZbaCU?*jUluS?E+Ur|A!! zI_rGUc>svNAz~yXLEI6?KaPF{2sk7Q!^LJe^ zI~;(dr3$z(kqI~lJG;0rzIwG*I!i}Krv^1>r4i3LX7#@*D}E?i&Fz?0+oQW7K9p=`ztLzf`vm>%n`3=KtxU3*(k z|9F0VeXXjgso8Y<)tCtQr)7+gb=5}R!1sNI^u4vXZ9N!-avP^#M&j$TzGUn zIXi=w2q`IPpe~*hWG0~?aLUZcSV zj1N>>0ljkOKta@>#?RE{+EwD)4xo*CS$-daQ3QtlAc4kiZ*TAD>RJebfPAaHfdP4j zs9)yI#XkiQv7CZ_p+cPR`Kkc55{HkCBd?Baz+`oG)3C{fv@0U{czEJTUw@Jn_PStk zL6#t-qN0KaQcwqy0s`}nc;b^+u*s>Jobd&%VY2?=O0Qb)L2pOrN@@)ajde)RmZKa4 zysckryy2mgKoZ6cc`{w7;gtHuhA9JsPx}*LgA=M-Dz6W&swQH_fg92%JGm=dqfkDc-=q#01kAUs_fsl+3$wglhxoS8?#ow^U(YJ(bf} zikPpnl|zuZ7~J$@|EEtgQ)(l}#KaWz;bI6G15$K6_W=wn7ASi4K+PukuE9Y$^9EYU z|2AJgm~E0eH`+<8#Z8*VH5=6ZK6Oa6PzclGt&B5JJXH zl|n)BKxiSR@$vDK9zx`xAM5qFEkJ43ejLOFKpr>5U&vBQ3&~V>hof9U5k9bJUtOUn(uIZSfB^Cca?W@_fNJ~vHI2LU&L;x~Cz%2Pi+brH zV>{}t0*TWjb1^K|vrO|R$w?RA&(6;on3&}Eg+5IcV(I^MEtki{U0hn)ixPccfW1*x zR-Ui1h*E?F1_s6i1iWR7>QC!VJ#pjBUWrn^6aLKiave53IA6kMyIi@BQPfoJO_jO$ zN+gQ@ZD%NH-rZS0{eoyhe0=cO&J;ijMn*=tL!a(n8G=?<$0y~QO^uBzqp5VYzB9PF zWl!c?ZG}BNv2Vx1Dgbl9Z)fuYr#0WX2&DsYm6 zhz7koYnWbcCGhZyNa80=E6d5S~s;Q|-Ra-k*{3Z~XKTs#t zCb|a`Q`5nTiJ08nTo6qYJ)Qqb>b|bX&+;Wb%OW&TRaFHtE7|IKt`NWe9mE}9tr>CVUc#CA6oSv5UDDd&|C6n@49?!6bY<+IlZ{*cbSLY@Y zQ~2Fd`cfXU4H@XQK;b#CeK8p^T=dbJ=*T2%vX6K8hixh0CjUpHkToZD!??CgHiX@- zYD06nwYc(ZFzEm`2!7K8(2p%IXVq`_ZPi-?%%?2qdcd$RWBw~$4&d!a3ya$0NKZ+x|^#)gNf{rvnuAS4fw%oVo@iGpAv2om)faXfTmt~u~A0s#Gv^+Gx% zBxGw}(y-N23KY;QdU|@#i-Qz#!A}+z^f!XX-c_A5Ul(S76ox!Mzuc8ubHo*lPY`Rj zUlBXs*HM$d!zk{4DS9Tjx4$1fmcHj$7o41cFQLcQ@W&&OGfzm)&@jtgDD(Y-Vpmt! z4}(@ujqRdG?9|e0Bj*SS}Q5e6+S!#OsT=%l&uX=7SU>+4V&82^VKW zquBP;!~HG6Z;Ef?4# zgx)YBt)bA+q8$#<(>@_c)d|CdaAjgm-_bX&asutgH9nhn+~27d#P2c^Wj)Oi|2MH% zIZIh7LZR~w+xW5ZS6KyxxOta-#eTLsxnFD*&R554T&HZ5l>W^we>DpXn)REVa{y`( z+%AAvH|(>#p?-INFGu;z8$xNK^pLg@uFOJQ^5aLeEKNQ&s;m4G%9o0af~+7+8e3ad zNBf1?i+?8twQe!J74Wa5^aUI82rEVz*ly&i{&n|B^FD=e4^WW_ix3%{RBPVy`rho( z&)3`PHg!{gL`SEbskd$~EiH}z;>B&&Gkw217%D#>UvxwS7F^#!1MH2Ya8EZmS(Ip& zlPV>1ytN~wXHEm9I`u;}4@e|TeGkG#N z*5la-f#1Jt>g=?5UpuCXZ2jGVc2S_JE8*p#3v&DhNi*BVq5vE3YKH>E8->fqwzBAd zfU_y$Qc06WMn-mbs@lf`M1PH6>vlRnXmUhk@}>AbTw4^M*zHdjYwVl-G#@1;<*#nU z$wb4W8?eO~^+!OWK@2cLwYgY{pNHTchah(szQaDd*`h9HAv|2J&jjwGLN+{K>sYLu zvIdB*)?pbFc=oFmm*KE_ObY(WI7upFF6KzC=ZL|b^gG8OSL7eJ!Pib7{QGAG=+61? zZ`PsV;W$QbK>TgZr@2aLubpA*y+@i3ax}Q`&EDtc=3+83$TEaHs(Z@sfx_`v{CWxR z;qB41xb}9@t<6nux3RS3WLXd+Sshm~SFs=xzj;V7`z~b#y@fbheQqxt-NNbLrBg~s zNSs_;6doKL$p4F127wy@IlIfTnNhaPXVu3~u;=JBi$X6MmX|^VgbS8 zH@LmIIX$St?T+6f&u!9GxBg@=;bx~#BWEu3;rDBLNPDFTW5S24p7=Wu?uI^DfC@?) zrU9%c0;gjX9CAgM{plP~|5IRm_LhFS6@2$P*+gsl`P1+G67=BI6>Otj@Of-XirmA) z1Mn1T>%~TxR(W?~E%NC=4E1!gi_PThEIr^lPcRQ|mv_L0Hg|Sp2nh)RO{zmIYqkfC zINBQKMi>YP2pm6uX4H~NPES_^>AP^g&Su!wK`+x5t!~sNL14E;wL2Qur0XLgzu;?V z4(SZm9<23+T9901SavF%wDbBgc@tVWoJSsyhMnCXv2V;q9Jsj4<>+{oc8}-6XhWf= z4PWF|IN}nw9{ml7UF_9KpZ4IY55Ldriy$ znET>F?Oh3sbnF$VJl~mxXWdxf`CxC8Qq5@<>vp+`IF-*~+{8)gJhi;czU5lgeS|-h}%7^Mvy5@dFu;-Pzdt zmHE&{JvNQfLZt{&-uw#vpX1=VibLY?Sas?yO1u8vl^gAk7ucleo`VCFY&Hb}D43Y5 zu?Puf5E4pfVP)N7I|ew@J2W(u_u-97vE?eB`iTpRje1|uu9_ZGOHO*cc3>GfeZQ2- z*kQ(xf;5bjl;1RvZf#h+YVrCAz zI9yTyVnItyO|4UD^fo~h)$L?cj_b{vB4DBuc`{MxSm8pb*+3KtFX`_zpWCDANob_ieYnn zeCvzXr{D~|L)zPYZV5r80gj?k;h371wy`yw1o0mqSNStHI7`76)ZGa@I058_Mj>Hg zpWFUUS_v2rySEnF6Yr7q*U!BlW)})eO4bc*Qd3F!e2?8m9$omms13v@SM5g!1{R0+ zyNGI8?x>zU^jPS5L2r`5Y*} zeEn){ZccaURa{zXIuQK|zLz_>h<9+gZbPnd_jk*QT;#rR5;%!6wqO>}f@z_aiIn{G ziKX6tCGxc`x}N`1<7XsSS64z93`PVRY@vF4$Y?!GRYMiEQbA5GBriYz+v(2O2FTSM zyu4DQsk{@F#uE8OMH~0Gm+lXD{NM~=$7>Qabac5EW0?*P4r12UOyr`zn702)+Lv^S zQeQ8(D?fQYct&M)v@BnE+v4#e2&Qhneg?Z=g-sV=U+%)%lZAZ9W7X8Ee7ostt!GN! zcV0f+du{G)z5SUACU0Kwb->Yn0_gCsB_;br?V{dS&p`Uqs2%}78#pi^3lW3Ansc^- z_0p@cxmjFGODjGzbD6Lst4#L?erS53JPmHG?b2Nu-S_S_qnI(p>4u7+n2o|8pZU^_ zs2ojt-(!|`v8SY@fRkr9U(e5{Nf&TIFV%up)II^MxIS5+;Na{m>F&-4$+lgnKR!Qa zh28W=QFQnBqvFvic7XqniHSi75Ia^1>;;sxdUO;8J_P>F9s&Y_<)2F9f`ZPzJ`~;h zPn|#Bl+#$LWAjYi^uWFN!Qul6qtLxv!9un zH#q{AEJnUr5%2L6V_!_7rNYe8&lBlK*L%dUSktIUQPYm%YO)`I=mBw5=V4 zy?|lXrP+UNKDXn=>h!z&`qwDczAf95B_kz3L=uPA*K^ z+1Ux--w;Uc1YpE)67Iqtw~*Z6@bI4FweFa>xUVnX*m+ldxh}q1uCc%YHs1oY@}2?W z`aMq7ilK_dNcOET#6XA~2TWL;l~iX}7b3V`2^E!R*yKV_DJUqM0peMDX>0V^H?V7# zFJX>N;Kdvtwk~dndamPB8Auu*jxeg}Q`|EVD(O0Z5tBew&JyVkeomdgynMAgL7n`Y zD`{URtuM0C9cB{JuHgdH(1tFz`Gk@3S~F~lNPNe9>+9Qw1fl-T%kWYdKb-A|E^_K^>RVI7h$L{;w2=6U=&kL?@=2tJ(>undlPvo_ast&<6`ZwSMiCD!0+d4SF zgb?#sKD#)W%R8>!+S%E-__v^4Wr7T(?kNTa1{n3Og@qUP%PpM%aMt$sb>52z%~M>UGEwA~3-#qp zQT?WPR`At`8j*pJ_=Na)4x8U)jxv*FAGp7L|K2q^iUsU|&+Skz!PnBJwAtPLQULQD zYkWM1pO97etJ{(20PSJ^=10X;0aU?;GFB%|Vo0WlSLC+sPvd!K`!bz6bgGz7555xO z62^1|mi11bzMK-V#a}IS2jP^KUEMe@tfjrN$ERzPqPt7Z4C27WU*ZX!nhXp_ZWnYKw~fQf%i} zMoz9;-5WM58yiyq=I%FVjIV9yzkvbk92vo+mJaV|^}5{jgf=$vfm%Y7iG0}!lz`LY z)RMDq zj3?eKx;&_tkImtKW}&Jp%LErn3msk;;)^$b--U4JPjcAyA)f6|kc_624Y$1J+9%L) z*$rGtgm!?$H007AghP>&7+COr_P70?r)y)GuNWDN|EzFxbHi@VZKi>@$R{S$*T0UB zkB?4H{-FXxhLDq!KSK}vs(qR5z=GZ`D~l+Y_FaQ5;9l6X%V_{Y8{qGkKtSZLtl9bm z5dqZo`~6;29P)2PMHUHi-g3-_jEs!#r`uTYrTu4mZ|oNRD=OGhd91#$>ejEG|D9D^ zp=M{tQO*?V1oTm5)QNcZZ-MCd@89FUe}7L(B8rWRgFh-JDI>Gd`4v^W-B;N2eE(c& zELr25X1UH9Xmj|GkBp8^6e&}{ix^}HfFNZmi$%&=(x9zDt%r7Whyg%RC_Dy}_@~Z> z=$rW3JIGMkH5U_->votbVHm^#{?)(jJWrF9wS`3x1`%5}P^E#v!ME+c_a)AB#4lz# z>Xk~X%(~?lF+BWFiJ&HnH1;&UvR%qZx9hXl%W9*Uh#hqux{codP#&86YDR3hxogI2 zRyanIIq{HCFd4MwdrdeEBqg&8i&LC4z5wZ-pI-xx0i;cNVg(~Yd%$F{YI1n+4{=ph zRd86C#QA>c0`m6y`j?Rr#k?pVOOd~H_|g*wa|Z`6r+JD~1%=;~8H950u#uR3{D_TE zd4$SdliO+_FE3vS(C~cy_VUPdwWFkU*J0-upxMPn?Unv5$12dodW{YULD*znUyw13 zK7GOj;sEp+7%LGeDXFZovJ9|zHFfov+`RNnHkxXIT?Q%OaIE%m`9`2Z+ESc`t zXmCSD#Kq<0(3D;ttZ`eAk}tJJ4iiPL&7JPR*P2gGP8N8@tYGveVlh+VCJ2Y|dmnt{ zsjlkX9E=;jPv24a8vMVt0HEC7y5y0nFm`&b&00?XLD(51TJa=){v5%{>6A&|nExfV znwlCVK6*}JVPWO1rIl5VbOh;3$VFM23WJoFnTtz_#)ma10JtrMZ3zh)o!-#0#)9t2 zUrWKgCPE%3@Mj}5N^q{dxBdbhmeR!*g}-q15AYx${uqXahGz$Jc%aBt)YO<9M*-x+ z@1{Qi)D;AxIVgG9*`7N5ZS`PKS#+*%*fF4ED|s0LE?poKJtrkaXJut={P{D_<8=G@ z@>1sAyLZKFQDlNBHa0fk_}{*LdoSxwQ0=%S|MbZh5~EVXaC!f7Z%%nrG?hvX2f(N3 zO#yXvyl~|Q9JvPi3iNlvi4i|PzpRqd*NS&_=)LODNZQEjqrNdyI^*Nx&rkW(F7hwK zi}CV9y`olLeUp)V#jp!SFPKN18^-VJYvcUQVsNz-!X6^|+L%?DMb_p0bI5HsAZHZKA zsmkLT0E2=kV_R$g3M zn#hqrRZQV3m?UQt5Fn-NlXgsz)6&$GHr0LcekwNWu^MhyDI!E5kOnk2dp2#v$kH4y zv{RJN4{uM;mnHH8qz?jPe;tE z`iH=`2L3D|$US^c+lmE3dLKX1Sf`{6(XxSw()Ys2)=k28R9GqVP*`2x|NeQS&-mYb z-LtUXtH^r*0r}tz1-Dy*aLwP98jqkJ8o;R>_bgnY9EIZBn`i^69=~y+EN7H!-17R8lH+?oE)Tk(HJGfe-6lb1<^A`}HgLx;&(Xx+!b1G@70DpYx@@VpDDGf8T7XsuaU|6B(IIN_^I;Bhg!-o$g zgZ0tT&mceWDZXYwy#v2}i{C2^&mksc(T0eMiY~Tz;F)iEM-9q1*ey}U$c7&edZ?p- zYyF<9ioo#_0*Yd2@imn_=+w%OgSfIZxX5A9q#v|JNOwM-u2M;n|BQB~N#lyfI!ph| zZbkluuKsjCef@w@bAp75-Lugj6%!q;@{v}J^l-5i!)f>=D}m&-`AfT{=D?N~Ar`|n zJ>v!8O~y7*>ROEJf8#o{1^>PVM(a3>%FoaDm>wRk=stE$0Md=hR!P^y+Y5w;vpEbH z2(~fT*Vlr+$0sEh#g$A7-k+QFis|CsrSkYcPq&Sm z^S=qdvxt5V0@eLIJw2URsRmzb{)43%i39)X6UXzvw5-7CzDGu4M^cDZ7{O@UxeXB! zC@^+BHnO7AR5nVwmy3B1gG5-g^c1wp`p@>3RWu(?TA<(;;9?Hc*7)F0)$S~m;&7$? zai$T}erOO5d3L16H7fRu4e={_deobn8{#*1dI``LlPY2~G_(ZAZ4AguGO}0r==vWT za@=7Wm3hh!EFaqH2L`@ZS92ZMnmKJA?D*j?xE?AgC@I*WQ(mI7L#gUOS!CzssqgnH zrSfzd`X$(o)^YXD?l2pIz?M+G6an*t2c5OU%ghN9^!kw7*|Me?w zUceV_!j&P|)<_C>6&MxyXND)d`ua=bkm00}?%_(W$_85+rz@GLN2%TJeqsS<%i`HO z+XccX3eoRCZvgpYXlZEyr!4z(k@4dP!`;a+JICwd4|WO(lHXYWYUXK_Y68WT0^9;F z7&qhz3IQZ@kdduhsCK^~au$mfqB17PzaN4z= zDe0B3$GtrU=4uM&#KFnwmd*;GnRvNQ-FpP*kF;Gtq;o8-5iEz-9LDQxX4UR%7-9h| zPzwpYjERXs4-sW$#rp7JZhUr1L-vl8G#7W2AgR~B^o`NUCZifZ>NdJKU0ht8_m+wl$6``qB5xbMe!jc<3{DZ{@bC}>!qQ0s zZ5F}}P?ZVwCo^C;KAM^BRaQtC2K22^e&xv*z^sd72PSeB>^FcXEv zkPa10py0q&k_9gp=39=>&NiP0inD7{!6Rg7ArU0<`*)O^%cI9p+??E8L>wF(5dFno z!H9?#e&}H9nwl7Hj4=lgbDM{M!qS#Q6ZX20G%}(DiRgGt^s%F-=Lx`pz}Q&aML`!v zx71wfyp9c6Hi%DezoNXQ!8Hbgs$t-b_5{(PX;Ho@D|WfheDuvR&l3#mdAn;jh;vkM9|%>BUyt3gW- zP7SguyoG&k;%)iM;4jfVUa)V6uR&%L;bRs*T5f$s7^XkRm(tloBr)NP>VI(Xsn%xp zFpX}rD9g7C{#_zn+h*%lWp3F@EmCHts00mef*`h_py0vM&&v(5KqL*e2JxESh6=;Z ze7QIXzBb!W$1QYtvlpuutpgg=s7Bn`*^y9JuTeX)va%XP82{!%f&S`xDj=n$$%j6F z_qLXCGQ-OI1rbh=`C+gx0fvUgOsby73tg@3cyR0K>z95Dzyo0sfJ!mMvQz#${K8as5iL<5z{-)c z9&PWx3Gnm31+@h_-!bZENeFa$n&)87?!-9{j*0`ZKPPBbd%<{fDn@>O5~`TpQlFW@IQofl z-8Vme{CGc)GZ1}B zaMAYvR8b=$Ara8DDV8Yq*{}QyiX;>4`lBsk`UVRNYjf+rRrmaJCojqz`Ck5+pI<+i ztAX!8{4PrbOGZZKCBNg7$A??{gn{9*vNGteTxN7HFE2|=OCD4GV3S^=q+uyX`OrnG zcXCRBll=)Kd8B4p&oTWT5F^m+OVsF%zyq|L&|#{1i;;-yk#UJ%3DK zx2k_hM@N_F$#!rB+gR!8@y@w2NyKK30s$LNPFSQoKe((;j*lTh-NqG?|6Q82Tei_w zw#{sX2Gv_OHnyke=&|irp4V!2=G$AHzsbBfJ*w~MX1|HQdMftL^>u7q>}IBcgoFgh z*OVL_&oVPJwHqB?>C{;x0~AdM00T8>y`j5K&dkIH)df@#9MeBGHWvA>IKKEqy#xV) z84dlgvbU9uRgQv;8~ph(rsgvNuUosP@E2Fy;)JX^YJ5KHU<*IKE4EZyZ|FVkKv;wj z->q(G)b41dx?X`|g6c?yhlOtDxa79wlkKM2fX~z^r&ScuVhl>A;Vx2 zLrxYFrxK?k{v1&jf}om;F01~GQG7bVM|H);Wr@ILrTq1$(ndBm${#$>$00d^%V^); ziDU47n;ujwZ1uxTLGDcW#^`4L>+*E1JAS0KXwmENI%9`_jQUU0vqq7cc9DTT1$5$Wwo1Z8XszM^3jf%zZTX0wJ2@c*HyE#y3YHH%XeCfS# zM96M1;*WqdZe2%CWa#=cgWKZQc=p?`o<;l57Wr>4kNPcnChKgp1{3~;&}T+l)y*4* z8Q?yJ(K_(@_)`XocSgUGpi%$v?c0<0ObNphzlbsgq^c!Z42S)+M>_((>UoaLw-Rzz zRq3@kPyBNU4heZBESynR#!_8VGuP%L2#&THr!2CTpk)BR<3(;Y4FA@MG@`#(OIw@x z)vH%rlpio@*>)x);?{?g^wZ>-*}*biWy5#0F7$Rm0D{s5-45$ld`?H- zgcEai*%0u%9wb+KdIUE)?Rc_6ciA~jdY-;#nt3*%fT)#=uSB`(nH`axO_`))r1!S` zZ%%2fo3k{*0jN}ihIK}I7rNF{kPh`xH7!}wn?O#=G6GY_@9OZPTSnPv zxIQD_Z7)KnTslv4ftiVU*x$i_HJj3W?8g7C1jUYy93wr>ktFmnN+aKE_Sc`Xdf99= z2RgCaVaB$@o_`#Sj3Ho@jp}slNp08+S_ayDZryiZjqNm+*Li0MdnI}uHpwgo*!7v$ zaQZ!X1FNI?&<2nsZ))$1fPlbCtZkPwKw;Y|7ww6d0N;g|%kKDFHDsr{zs7IhA`Hrf z#QsPdeBWKMbh|q@RTuM#nTaVlDvBOVGV9;jnOi>kNU|LLpROWK&yx}pbKf%w+g&5d zIKOMsTUG$`T9o>?B4|dM+G32LX^RGiwE^GqA#JwY=B&Q1&Es6x5>jqnK)aNY<0O zJY3ReYD34skb4)~XT4n^d@HM@^w~q(uhk$*kuDG5wjOImR+^@g5*BiQ)Kq~2QC3qh zyI$ieFDxN7Ro=nD!HYA#G4Ql*V+s}*XeX@N5bkw7Yr)OP?-%YjnktvIEIe*i2QIE8 zVy35$C)?04sL!Crb%V8?g4+>Pd?y}KZPrCk1|M)WmThG1k;#8}MYg=QC#;a7TJdei~X^%T`t&o)J9&Bg|K)2=btP5DS@@Y z8;{Nu8%2f$CeV*#dIRw1h@rDyz5h5kIiClLPiSM(+u<2nL5h^K&IizVNqHqG$Op!Q zP?t|KM$M@itI*m#T>4m7&Rg5*>Eaj>WCqS-A%E$MhI8Z#nm4||kWesJ|23=>Drbot zth7@k$oB>yq3*7&#LoZy4Lqnj!p3829@Aav+mRG?z{j7~bz|c|YD?>VZDan+Voq#(G<2w!qfxc64~M@q7D+oU*b^v6`wa z$CIAG7OzV?AaPnOh9My#$)npUoav2$L-RzvTCGrvO-KF=f!dAr0_JoUK9?yso$~MD zzcM~{WJ3<}ld@ZDJ2An@gJK-7T724WC#S&mae+3)ZP2+$fNh~O_Aet5WSU%B|X zHJmh6Whz7Z`V+&or(!b4+rp_m13Y0SxZQBU2=MV~J`mLy&R_C^X5IOFeC+TmQ+RmR z&~Nl?Zz?1#%ycl87XGR+9{c6=h85p5Kko$#_X=`Mh0@K$D^as#Jy)&f5dbei9u&i!OVK{4&_vOH~Chr1N9zB zd!u~97E}j+&qs>lMKtjSa)nqU8G8Ed`}u~X?ZkxN<+`VPMOhEIS8Am*Lg&-!J$-$X z?S7AGi<+1yNFE*@fQ^&>Ei}~l+`6{uw_w)k070bVp-KL2RlYlu_rm9JjE1#<*1B%q z)D2K>Fy6xLTGI*4+I74(@g4L0i=(cX*x7ehJHHAYcOV&*2hV*fEG+y!u3|jmnsilCQetwxKjVG9l_a7X zmB^|a8y%f3;Cpw4)De|yf$ZjD7) z*xkB$wq}NhV}h&!zx57(?4-Fll}ym}crA$HWOv+HuSusb{Lw3|LKyX#m$J=z+(Pu6 z+f(+(L#5xgf1DQeB^>Z^e~(g#`k&sNyLklX0q3B-Xxu!IcMQEsuwE8 zAfuwTGkIzN^=GbQUr_wm1YlaCQT}@q` zgxehRsO^>w{)|stggEeDt9AjyM}T zJ3Tk|ROx!D)l}iHUd#Zmo+J(Nb&TxXYTOjP3p^M>#NbD@84=SN>+_!-?MSO0FP!pZ z+v0=_A4%{cEO|(wKVL30w1JDafiD5MTyk4s&Dz=A94}I)$SN65<}|I;X3z`ACKebSTH)`R|?$mV^L2oLgEA)RRg6rg8i6Y_dv)#eLz@X;ljvEP#8J6B`a{NwU$g0GL~WW5XNg6qLL^vGM%?vO63 zZs|{za;5Q<`S&P~^+>B^NPm zblbV~{B;zjZ?Ny>r6hEA>CD5@^63^q!r9SEma^!5JU9g|ht&?dgoFfC+t<2UtFaUp zaq6}+GyldNgsE&EAMRC)-y4)mmTxJP*j=^tfm{O$Yg8#qztuAVM4Pynm^G(!_|A#D z6M2ui1_mNpXBF9PPluD983IrkL|M2hzG{wnp^vRwXT1eWkWY}8^7Iq{zqzv!C2GFX z=0l8$f=w(8V$*>X#fXv=Z_v)#uZ)m_0l9Pkw3vh?GDt4wLE*AsCCMQ|nzKzV^ev><6{S*sp zD~!Dj^76T^+rN3KPoF-;$`=TrU*!c(Gq&xMo)wmqpojIY*DV@|7PPeJX}9SynEQG9 zm4R6VxoE7^>vn&5JV&CqxL6KZl85kZ>+?kRS8W``n7@R@+o;;YrK!V+*xN*|(M;Ybb2ocX{u%U}j`@JE*DDDaWJ`(VN_g3JuK#7}RpL9>${I9A9HGreuAs3y=z>J&HM>D_{AI;5|PxPIgdEm+k4pjoO5JiB+y{h*NZ=iJfLQcbl&2oj1 z3mP%il^M*1x((opbQC!&1~#^@&RK<6F#B75D_4F0=OiSI*Il4*kADNQQM@=axLyZQpK3n2F*7s3Q8vN2bRkU>9)8K|p?MvR9YAGtgI9?YFCvI`4#}q#Bc6bO^BfWG=JN6=!)3jRf73Od5K6r-wy;=rkh>OZeoZ zT0>U#@4ePhGyYR8w{{#5E;SlwSI#wJ%KN8}Mv=XfH&TY=MUpYyOV2sjn^euIFx zUbGrKzKqN-Z+qFMgC*(TJsFdntX)L$0uu!)&2(?I@?KLD9kx{>x2?=#_;(gs1ZJ?| z0O^!AHa6CBJUak)m&H>Xm}0iUdZ}^u_gMB%VAkm3LcoQBq#izD`J}_7isR$Jv-Umw zubnD8oB3`xm5tu8T&?~1m>BpvA7QGtmezpVo6H1k6Zg;jr~agnCh=d3P(t|eMh09GBnF20MpzY`w7=E-X(k>akc`fwcmUp{^u$lbNrWJ&Su+%FM(zL{aq4DwfAGto*JgF!ozFMvlQymrL7VWgI zlg~Ah&I<{mc+sCf827jCvexhM;nQ4o;K0DZ`vNr&eA$c6htDOfbvpVMusXj-Uodtp zc1;h;S)@zr0hB#N_=JS9@$s8D>lhjD-YA|`!s|DqH@Nsc`Ty1eB*_d^S|T`JF@y+D3-HEy})-^Ac=0QB{mHp%{vr?-xaa{J!D2?+rKX+gS?mK0DV zr9)a^P*S>c2nmrE5NS|Cy1NDt5D{sHknZm8XW!@Z{XOR&Uq@l)-g~cgt@pL>9leP< zMY%tI{MuPNIL+X3+;$t#gKG|*y9*EG}r`gKO0Y%hti*9y{` zHV;_E<&~A#FJDF_vxk<^XW+#S@BE(WjE9VlYGLA0(1nF3DYJkeg#Y7G-CR_{(cs$q zYCqHXwnC2qB{p6X3IcmbtHwn+%Z0nfL42(l9@I|SLXhbTv zbK_=Pe0o1DWF=xq!S8P}A-*W)s@+4p_&R$6r#?!y{9A#c~+6p}twP_s3;df}4 zTJ>#7>WWCL-);md1V$+IA*0;y6Ym?tc@-EjOi)nJ&F_bnwzh&C9636QeXzoEV9pDj zQgxJEg5TPiuaONZKsu+lKGcu>1WK@nFzcF>pFgG6n1CXN`5~bW1l7J_Wx~6KCbjS5 zr0uC^3(zE>>W08;f?w;iz?GV>6~vj>#Xm5A>$^+l_q+eVliqo3&oOGkkUrs1o}_X}qzshbC|c zFQ}pZ+qKQW#vhk}If;q8ZH}rC?|?4pv2FC|DXz>4oS!$TqlOh|d6%>R`kid`Z=GUQ z(R386*5h1!xo!M)9uM)>Ct0rnj0kq(G#MW^^?k_dI2l6$Hvh}1AuH=A*ATp==XWfl`LlHU@b&kQq>GWk zEUG41ucrpmM1y+FoDpTk#UOQ|?fm_jFF?7+!@$63KKMmdZ8a2FUoUcbywQQn)7&=Q z;L_9Ed-STarHK?)itS)%XsE2EHOiGK9BK2rrbxF%Q#f69?Qh@ez@k#wIHSw?DE7Ft zFv0z{Jybj-5`H*oi%%=MI!4TWbBo0y@icvBjwI3ioYTsNqc&V*!`k?&tBz)MZg!DYrR z_CzGva#4-Un{UA%6uc%VP;|n4Pe?(f&Z(~cBI>d_*y`_JYS{3zX%C^)9Zekuun1H~ zJQh*?_RKokPr&ycEtUU&Bqa z*A07@qXA|E!jEA3`F#Kpa7X$gB;_cno%~AKZ1G89XD+Y0@TriM9 z14iwitx#R=HMYPPMP|w<=v_FX48Sd!w2s?CcFCP2Y+>*-*$URCr9|*BEd; zeE1L&)t$!{Rr*8)Sfx#z%wcbTG5%fhUiXUV`kCtQ-EEFlJGJxod};1tXYVdX(8k$n zrm;r;NiB#M4i1j73rSk54n?>Wzh%iBnS=(8EWzIsI06!KVOQOtbQNwqOG>MSfn9^Hm+Fe zU9va5b81yL#k?TpBhhY4%=gzbm~!4I^t^Weq)tgh*Y&oPz`)qJosltv$+CmYeRo<` zG2Bujys)rvqS71}z{nj+P3KkH`;%cS0XNlhPQe578iSB?XZnHXSDd(HoG6kk@k0f# zS!?a4Rz1*0tOQt47+$8$N2`akV805qPb0h{;^X7{oB3ky;#;P``?#rKDQjyE;8!9k z1^--ly?@W{<>dtmdWFN>^V^#};2ks7w6sV-m2cUAfsx-DMU0mkxfD_5YZi1`^MVF( zlHn@m-Vun76Mtg)^!AtOHM;z9hfD zg>RG;6&cvsdw6E3Uu$Zzv9UQK97cXT0I6jL^1|=6e7dSU@Db(>4Glp74ewr&VIMHp zQ>#Yg8ScLMH2~>}MIYtb)xlD?&++xb!UFIMammRW*U;q*hsz3iY;Vd6n>U$TUt};)!)-#lBRsv9qz!iHc5gbti_eJbC_{rooVAuVkJD^%W`aeLVn>lMOyR zE{Myo6{edRspnUulgi!}&-^b(GLw1V2I_K{)=gc^2N9eDZ3KRgUP3~mrM})XWX7JB z&h3CXv3HB~hmoQWf|D#_>d#@n*q$SJ9voa;@U8Rv3mwIJ6@h_)X#HuT(q?8kuOn}? z(wLc;&dr<`e??Kb?EDk7l!Ugujg`UnJc0(3_)N_Jf20~{XwY{iLONyVlwu0+J2V%d zOmF8+5q3}o{<6iGC0SVxM6K0W@gwO_{Fe7p&=wES3C*5-(E9JZk)a41>GGNPEJ+L)|jr4q9J<}iow^K}ul8bnD~CD+x~W|pHIh(-U4#-gkJ$n$i( z^}B!TY^Ol|(WmT`BYf(ul@;60R9&9$)oC5v5?DSCj;wFr0zh|w{?Tvot~v76*MD?2 z?J}SexGndA1!);-$j!>k9MTa=Fj8tH1{!d&o*i>-k3uQpNH>zFwijvY^rpD^KBR5U z=ZAvADcF1b6ATQIhE3VA%K%UCEj(4RY(JC>cztpH{iL=7hhbgPReU-J?-$uF;k- zw@y7ZHqFOV!g@__iD~`LAEIO7AA4g&)4u5MPZ!e%WTKL3X=cXje+!q3qCEN7ZX$wp ztyH*JRQk}hsHkYjx+bn5a?JT<%J=Wz*WKKLN+KLq)t4#^X*Os>gX;!S z6s2HeiI}OgbLAU!2~_dJ-;B){tEtdLI?&yR0P)_w{mvw=YuE#lW@WGS?sfzaG~iD# z*3*G%jnsZeeZqPr(c;od-U#p)hlhs~t^O~d*)c4vs~y=q%>sL( z)LTX1OodWaf|ZW&U!YWdIKY$^B+k1{F;1Qd3vU+npKZp#_1Z`frjYQ)!y;H!VwvdJ zDr@ck!v5O+i5w;Y&0n9rmhLjZ&i<5mACDw!M>`W$^NNn1$|`b))-3jtXH);`>Uu~{ zE)P`I;iB}%kB_hOR7Fe70M-E%7`b>3RMNcnYeR{x^hdmflW~|O@IrWx_=ADHIy^eM zQH4uDSFW5+S`v(pj(&=TTy@)v&T#;M>E^)r77hp@`YfPzYS1g76P(^_rEY}hK~`_f z7S3+VnX$3&2{M2XK&F0F+nZ~GeH22N(GH=BQtJg#e8I;ugmGsREj(N&&+_7s)OCIE zC4Y!<#XF2Qb@nS&pyEunLND%s=Ek?&oK8IfRq7eqRW1{vl$D~a>O4;KPdotCR$Rd{ zoucE#QYeZR8=2^gzdh8c)T)ej53Z>%7BB%gmcB!el;EX!D^KuswE?rtD#5myu$c;jY-PtnZTZ z_JgO^07)vFn=?<$}uPz%@#Fq}tm#Fx*%P-@eh z7nUTDJ0 zU&E2CdSRo}_%}m`%ot|;k7C3Rv7xg)1O*d-@?V%s$VXAidU~Fl4J`%UeRLEfS?p!S zG2C*_AK6%+ZD}$qethz_FGaBKq_?cgO28S=HMddY^0l!2biJ73Y2o;mMiCN-$sw8l z<=~)tWJuobz5=_)ZkY=+wV1mZFhAAyGylpQ%E3PZ@+HFk61MQmoFd?Xh#}6VqO=xMJ>46W-6CKT86>+CUbOLoAxZT>mvK?Xga&VNZJy*50|Wb3|0(27noCv91TgvJ^^Jmcn$ z2Ht!+7J79jBPaJxDAnN!b|ez%q4leC`3DG{jup4Pp=_*FVTaWo)=bxnid?4Oj0_9_ zGZpzAmGOkRnVWRr$$b_&WyVoGW{dx7Y`!EU#3m&{bDzE}jw%4H<5|^K!Gn|@`LRdI zNrRyUmCpgb`Zsw;r>F)!V}K_I?PvSjNvr94rg!OdP=%Kuhz=lK_*b;g;Eqmudjp5aE!HV9 zsDUm50Tn|OBe5yY{Nu-un!XY*Pk4qJjh^9J_%9st%gD-#p5}o1#5F&NaT0 zD$%)Wp8DQyf?a=OJewu7_`NlZCqN@xt|yvCBE_7S0xHgD3iWi~_8;{oPTH;lO?5Xc zb{1Xgjyc=-2sm8q{{8zPaAv)?xH5cpML!bI_WPdA$OA&1VQFrAD=3h>`xlo%fb&au zIF3`^X#7_E;IAxZ!~U1^Y?89F7_|P^6>2&^?dzU;9hz}Gd6Fz>GiF2$z%C<-N|=a< zXqfhVx8qjYVM zFSYM)5?@Ci&fPC6hjRBF1D#^M17?shg7s2pR--S z6q(gJqtJmxuFs4((LB`An^B{?vVmMz*o<>$vYi9#4}2E>leeQ=@R5{UWD%2Gi-sjSrXE38*`P{fXJ>1$k;Gz8 zcX67y>;!uquMfr5Ew+NsxZZK_H$B-zp5XNO<0j)w*+ey z*n29=;a<@|*DO1w{d0i-OH%&fSt^fCiVdDD$pLPnnDB4fxV^ny{OIq~=X2|za5A&^ z_@M5C7^`=_7^c1{QOwQKkN9dkRjV@1bwUE;Pv(@V4xTDUe{J1ed9*fgRM&c=5{FSz zSojV++}{j|)CW{To=okRk&m8Ih&ty;m5n=~rg%!h*%SVFhrAc2z>E8L#N3&qXVJ?ZRco0DT1crDGcuLEcSv&!+R`3ST#8vOU0m!dru0E!5vozM>amj z#K3^wP+!4=62E*=H#FRS*qb;CVEzG6?xU?|a_NKq0 z79Fzx2|JLls4t2u_`-%Ust4M0EybF+f;aX~2Kl!S)m$)O=%Dix0!(RNW0!L?`Iyne}Uk#6(1Bbk4qBAITeb{q&5C`b=Xa zR)=h$Em3$Bya5nkZ5EYP0_~^^jw|CiI5@DKu8#$!->;3FTfBNCk)MI#=UP`H$gi&; zzZe)8u7wuq+sfZ|pj@Kx4=n)|3O+~6vAn39w1uA_bptv!S5y+Ibt{3#$HafU{@NVT zdMC|kQ2m>Wp@-V1=Zj))&d;dOkPuX+@P%KDTBxR|MP2Xcd3a)g_%z<+$jZtpr-{V+ z-CXiH&Pzhuy;$GGL7ObhEG>yiNRCP73;&KbW^xLRwk$D`SdPNPLCTctJ*|aVZS94eprC4;I z$UCCslTo(|(q9o!KO4=(Rlc~mXudw|g=U;1$ayOL@BBa&F7d`CaNq^mJL`Yvd+dFa z*A~zu@?}IOM~h?2jxR3dKLPKW1KBTrI$_F|@TWbi4$A%mj90yfpy2B@d-BbflPR#< z9?#5gEdK%FzXI~5j+9k%-2ngw2z3#c)kvk7A}tPPX2d&FQ`AL5Pj6c)+$1|8D$D+~ zg%PzEEk)?Y&7QH)J4)aucum?+NN_NygzcA)241i}pB1h-$x(>++{jHdd-?KZZvei> z69o7H1G}5yfA{O8=@Hj6hsavHH^-B zh_FVtt<{R7klbq)J!Th13tnFz#Q5iU9j!i}dsWtY`|SUm z2iNF;#3{>xoA6hj1$T=2ds3a!0YD%BXmcQ+W_4ryr&N*do~HI?#l@Vh?`_qXb#J>; zjksCG83GS0hsB7WT0d}a>f_>$1bFk~+qbrrMJ2^?7hr9HMUcZ2HePt}?eXJMq(jTe znE#DwYM-2+-*XuJ00Ww_1*!QrT`Um@frge=Cs5Rv&?&3&om1L>Qv@S2Bz&x0pUt3t zD_y|ZGk{Op3jMWku`|F__*8ajuz&A_k&uDLm>dQgq4D37-alCRn(uP5Fr2Fd9fjbi zx0FN4S6uo92&<;1)LQ>AwrdX3s(^21^7#vBdbwM**sx)EWu+d}xzH)L>gt3ujc!X9YndK$=kT?358%y;#U7Q#5T%l1iANTt=d7+;@+SwMAEJFI-zx%mjI!_vyZ%3= zi3>kBC^tdXDI!f)mTUF9!G7@IB;w<2tzD`bt8R3d&H7=B{=B20lN-}B_HdpMsfXch zZGruGJ}=P&EhqMh1L->_Ib_C zX}dE`F4yNytqx58f1x-~%yrhKUd1rb{lvrGaFMLc(xbP47AE=0lP$i-b2(NX_#JrM zy3+?j*oQEekrpyL&O_$hoL^FKuTV+sM%4ZDr5IvL z%F8eRJt8VMBFE(7ne#GslgFMr!hgRV>*fb^2VkWf z0+mQbKR<-p9xNg*SqTT_;jK_=0(Ns4wI0Eju?6AWHuN?G6~@G5mE{eh z!l2ex!QEZRyT$ME{%U$SPty$dG(N@&bR=m~#KaW*`_;fN!&k3folk~kWn$RGSD%M}`t|mEr=lw^ zX2}=x-;mZTo5&*9jo}c$hgZH`Px<-JS5qAZw@zBoGZ4hk!kFNx@84!e6Pds4{lB#U za^m8?m(_FQZlGodhlW~)8Ivz65qiG*by)L$gk>#&ij0gUIX|G(WNWO1iIvsox*3v+ z{QULWISdS}*LJ+v2P>#REv;)P>6GzApDXYUVu{#R=mC3+W7O!nu-Fyh2RrzHj>Ye? zstgLii-4EA05@I4NQUrRV*r$>*n24(WT{ zQB-TXQO)MPWC~2Wh}isfwm-dpw&<)z^*AjOBT_8-QW~ybLe+-U5&iL)A)NwfbYIsN z+ICLYyM^nEBWQ*B;^Mp3vU6y{FJB{@1dKj$-72ed%BNx-n-)$v(D>qlf`^N}O83H7 za80hy4?xGOk+*Pz@h|_}nNddiL`U6Va{_BFOBT1Re!U^0NViPWz<@(kSh2wn7#82s zF{@X~sulnnK9sc< za!zy)U$r+;ndQi!w9Fu}9B84R(0pD zyk;YT)*z4iDupi=y$r%cx$h_Bm;b>+1Wi*J{W+EpT8Lt9rDhS5Qjk(5N~cKheT_ko zP7&9*bq%vY$!Qjl;B3I;MaWHU=Z5Puk?Bd2=bC#};}{dk0M}JBpR1hh&ZOhM@)J|| z+^b({23<+`oib{b?ykaf(g7YlYDurA=mRZ7^#BmM0?ddX?LoPF|5Lys4N~6;$I4J- zgJ0r1LT_0r3`*EYGScsWWI(T$+bR}C-gRQPTAMhq=tjrH+E-RK#Hd7vRybLC?w@Tf=`|dmo81v76nF7sN=iyFa8cgJ z>(HX6`wvd$ZjLCUJ__254StsnJ%4K1zg?-vY4;&@RL7U459m;s3DqpKfWVjieMhKo z-o4A38y9}^Bm#h{3fGe-KC;IMLRcUQF>JOH4sH4w9UVpaoSeLI;JCdDLR!$w+>15E z765B^S2ZE5F7Iu}$KPg^6tr*pRGPVKG0}X#D4KsZPR=!UJr_UB!aqqgXE`ICAwA2UNtAS5B#wH3p)t>z_gWJ?&{%F8Gpm9v&Sn zmp#-;86~H%rSn*)y<=w=R}#y?b2#~Rfe9|NRRRK#ItXGaO&+_VM_ z`nLy^d!e-dCB5{2VM3ri|i~kDf!e4ZBxGg|1fZf{BTD&`BLV97SoEh|d~7%Il=*Lr z+S+i~wwm5~36qeKReBv+F$zH#hd{R~8UzhkzYjXR9m+5`lbF z6=@Z!$Ymc$n=q^ItelF$jRH!ZQbr8{mY?>&yD5HCS8Dur+&2VTxSyV`0xagUN2P0c z%~ZXKUrylwj`!kHQvQI>L;HasW>qy?{hN&LSnu(siAjMWyf|d{Uf{Q6Y45lvXsk-z(0!Qs2LU2sE>>7_NT* zT0z!l#7dyrW;_}?_zw>gJ(PfUx5jcHgC>#NsFFS5k05A)sR^n2QZ56{y-PHkC^=Iz zV`G||%M(Uc*6y+rHqc4%|Lt@OG(gpvlYnX77v$lD%|bKtYASLqd_5Ix6ER^e*DleC(I`UxiKZ5dOGxMnzE5m5R-ByEVY_wW4XvwK z?ujRb4x2mKow@oxF)_gm6j7=O$&LOImEM@`pWowHTBV|*qRA!?9)joR-(1v+TKRzq zh0Y!_E<1~`3I!yUIXbF^Py75a;X?u&uib_E_)BX|(M+diQC@=vI|dW8+jJpGd=N*B{E0|h=On}QL8)R-*)s91|La>h1ub&8-j=X1=}oZQ?$ii;yv z)1N8SM!MM@jE>ofcrSlC#(H`mbNW+w_)l9QADc94{AMhse57??U?7FKXE}`dDHb*X z0V^Y8P-G-tNRQbf7=waPd}?oKy5Ap5u|#KUoGfC+#sqiw!P{1JPwd}96nO1#YHi>C z{!9YtY$YD<+IzsO1BAsN$&3UW~KKlm(Q$y z-2kDkqzN4-An<-Fs6)$5+R@r#&CUw)G7g(Qp13=k*L@OfaKIzh#H$B(@uiANR5MM= znY|UxH43~)osDSJS`bw1W!=`xgt_}=nOQ?tuT)gb&-WK96JdV(e4sz`BO@a%85kIx zfkr3DW$QnPM4dL@okGJyDFn#^vRAme2lt`_({(gpF>2RtQV{hD*~7;-37vXW|3&HWL$BZS4g6oqsh@x;nJp z=kpbd=E`J->p>3cNIl;2~#HXfd_Nt6p$uXtCC> zvUm`L`G80_De?0nqHHV7!*m9og5@YP>jUbojkJ^$bhw=9ugLqa7Tm<7q@_1T3y7Dx zV-j1h4|~Ib@Ce#W)$U|g8`vQXO1O2{NcK>ToT3t(srPwJiVF*&D_@4P<*~HrP+`Qd zpF8T5qm9&uGS&gn6{9FP8uGxyYg_;FgFm&{Wx8DnCiZMT;34&pUR)+v{ejn8x z9m4IeJbUp#t794E(RQB$H6sF8^_GtZ(Efa@=k$)_P?oHgu9$O{JMa%lNffO%qriZI z0&K%NWz5XN@=+;~d*;x~%*-rZy6=j}y2ikn9#JMwY`gtY+7k6DV2Cmbv&DFzGmLm+ zrl}5w<>@F`vl~VrzK=4Jr=|_a|2zBzW1aGWT)DpU%ko71qH0LJ~W22dVz zl@j@#7G;Lk$755bzLx`K8yr*zaDc|K__k!-b~q;thp_ai6#V|R5q!h-SxlN^f0}3j zqfB*cBXqWnHb3eEA|fIY|J!S>OnF&Z4gH1IMWwtEQCWo6r-%r2N4NFTjkaz{ml^{V zU}B+zL@FvOuy>>L^VxtI=zlsOo}KjEx3SXy?lw6rEUbzYr^`G_&V?dLN1pFSf)}S; zOGoE$r*4klZF4lh!LkS$j-)bU+Azsp9&Y*anPFk9SRPra(4IqS}(Ia+Kva%S) z%3uT{hWNC8=WknWM*+XRVsV%pXaL>`0-c%LV`fdz`Kbt+Hm2>@kYGn3E;pap3Q++D zX%E6u+WfGknDg22LN&kWNsD3ulQ=^lP@Pu@d+!!w=dWd*oieV^(u+>vEf1P+UoSB4 z{|%G_oWv*>`T1t=?zYT&@Y~BXA?r~qgkmm-b`dfY#m;Yma1Lb&<&C*H^WEimtLH%% zIeM+c1I!?^Q=bpz(0q?C10&-jLF+Hdc_Un;x*z-*iB;&{leBF)H#CnGDJiL8qic_C-4yQbst7QKPC}_% z+j0vhgVF?H8pFL`5)vwW&g=nlnAc6(k`*Bh7!rGX&H0meEd7Wq>ZX?3D&{yD7y{SV z*8^;Xt0w<8xU9WWRE(4H3Xh10?OoxfUb=VE@xRyN&~+q^Zs21`NJvcTrUua5+?RNh zm0=S*r>)^duRig-y!nh?m7usXQD0vl;Gyvl8!Z`pTwG!zmPlZ@>-yjaELym_KxA;NV zkb*p9A;g~k(-Y6SfhT6C#zWfFM45}St*iqglNhj^zV7LFmXx5%{U0hHe>Twl%IttMM$v|Chp z$#|lsN%(yI{Fx9`03$zJ5Z<5^O|LyhBo7+(CR?Yra%<4)9p#|tL-WpQe)_RWBcBSAL?u%U$Qi^<)A%n zIXOArZ$_>kBrD6svjD1mHSgemMI|dMd%QdIrk)<>9$F&Nqend*1TQ$>l$*R%i|+}Z zv7e(e=;~c+wL&X9EL3Lkc>=t5jm>x*FgpK@>#XO}gWK!17wF$K3!q>)oAJnmArAm= za>5X4pjb!9)V}!M*qE}1xWZ3Y?LJ)T4g2(oFp7Z?+YlO2>yE*J@=4pLG7;DZq@S*N z_7g!DgVst<)!NBMfOwZvQF)-RuMaJoWp9Rxv*zp9H;U@CZ(I!KkZi()sT`)nv85%x@_Q?VGd@Szy z8I+t4t9_{vRI+r0*r=nclTBHBdv3+tIMA{TzsvWn zBlH)y=Zrc< z2>=hj4_T#U3foU7fqVoh9ynyh#LOH*Fa43%dHL1h;bCVV^a$bQ0iek3MWR~(T$#+0 zjkx9xPnM)RH%FW=;%0a*-j9^ae){D)Un5(cBLQ`s%)2E6U{ZzK)>uH{8BqV<0|U_N zRsq|I!HWw(M?i=AN6}J+9WWuGINlnM0pAE+jqXHz{~nqLPWd<#rz<>>n76qj8_RR> zqX(dbll&J?w7$m=2PDoOyEjoMC_<-vY|OR3;Mbr3C2l=c%LSc{YhQ;Y<7+2G6+y-& zm8%KCdfP)+)2_o6XbmB!LH%c-1)z*Mk7U$WR|`PyH2@VIi1I_>crfe@(BY)%oSdAt zA0a`%K#xInaeI~yzrDTy$|MiziwXStpu72aSgFn5*)vR*0$4QQbv3oLNC;168r_~G zBqTr!H-T3}f%16vp5R??8M$^e>x$d zuh5Lk`BHI7iTTlL-}&Vw5jFLLfGoBz84|r!G0zqE>}OuRDG!JY!n%L|etTC}NOg5J zW_BR7l}v*KnrGb*k;1-jLWJmy5y$4_r@%`YcyX5o1VAYQ+bI<2y~%Knf?kP^76&Oy zd>GJv809Kfqx+7cg@r}vi+;PQT2#k5-!swO-v0g{S;dhLgD_D|bV;-lcWFf&=`5;} z)6yPyA8B)tLYe&j{rh6$7K3Logi@fDJWf5nZLI*>PGNVKQ96o-@yNj1C&7_ zmv3QFPI8nd8?R4kYm+3AT>!d3eM&Kq`2t$7+uq*(V9cq?pyW6_f{}sgUI3xLKiSRP zU4_e)J78{nNdp6eAiMW926B-U?Mgf?GQj-0($ULAVDd)E9f`evcptVi)6&Kddim;n zDdv%wd+~o?Pt2;ikJN=3T{9QyKxzVq!*kxI5Q6fsF_I^?ePH*m`hJ2U(qt5uPM9?` znA?bQ`fUC)y_T-7sZi?1a4v3l3@xu=Md|EszD{6zl# z{@pj>S)UUUx@^=Wko&m$B^I4+A0CUjM*_7t_LcxeAgVU?%a_jW$*P_|f8y|&f~^iv ze&`hWKjC6nC~9e-(lh6!NCN|A#G;zU#>LP{y}8^IAFaUyGFL|C-t6q`Ft3Nn`M>hF z=o|@uD*VRH^VBkb^_u&hLnp`m%67T7Ef8U`xYV+rr-As0gh3Otj)~XS+gsFbI;pRpuU*J8?xjS@8vsV++xXVO!2!Ln@c2-s9~*!bAoZpddO5`R z=?w$|j0~`pp{#NLPFp?(-69Uk^3hS|ogpZJV$(nzAHR4KE+r!qEbvG0r>;EJMPX^F z1rX-*t1Drzp7%o=Bh8niuT?(sz<78XyE{K8Co4!vp`lz{TtsD9@+a>f8~|N@FdPPw zl;hd6e-GOPD5LK2-iO)!5H6oJ;jM?xmstxTH~ue8J@)5c%5S)W{v0ng(kNOP-$vFq zHfrkXjxVqG@8ly8(D6c9|IIW>#iJID*;?q`-Re&ND)T>mdio<5x_Ci=@tt9B?$OaPvz7g>o+d4T0!>)3cS6d6L ze`H>sWs`$g)6DL<|I3%CUS3`#q@-_!QZ-00puZz%GvS?ISojbVL*VD+0&_ zvl-*Gwz1)(z;i~!p*LU-ctHK^m`7V%d&0K%Ek>4g_fgw1pmg z06??ym*<<)4KAs>BjYgtsIoF1z}Y)Oj(}ANnr8Zo^{aSO`;z)B($dop4i6dV=mMa# z=dA^;Ybc}SmM9d8OG}}7I_Xe+4I`tz5M1&>D}nlkhK02?c>pXx`K7amP}KGH73AbF z0rrd{ks7+Xp;>Ie3C(YAhWD8LG~-`bSqZ7D6ZUSoZH)e&l?7ST&*|yV%2@ER=4}C} zx{nXUT`mP)Ty3b`KR#($6as37?sNRDxRFv>4YmShR5Fjc`+uP9IMc`h}&(G&~c6>)7G<0=zG}YB50b5ZC+nZe+t^i+J ztn=jLmZ9ODk_qce>~ZR79hXiiFqM@Bk=aa2VIG;=lN_F#wo(?+Y@~K>0;w% zNM!;70stdS@cZ=tI0D#(QdP)kzU(6z-Ual5=B};1xD$moB0<54GGQxpdl2I|v_Fc5<4Qg*L!gKPJd4Q%3x5(<`?tFKhsP9%NhIya##;w9{57&jPo8*c{}s2 zaMI=FWnRB)&qmjcVUE2GFmhU2qbe;rc_ZIPHXL1CY-(@VxjEiVe|os?Cs3(tU>rRV zv$|5C#Q`EZBO_zo^_ilg;-`oRn`#v`H8r4#+^pXTvsBO~0BUOk;JldI+_Yv2FrsDe zaI;%T4Ag~gMei2URontk0h#wC7+NB*VkQNlz{DK|GuN48Fg^;i+Gu zBQFUZJPMr81DZ0BI|u}#<1BtSj_C3>-vOSUqWHV~t?kXOShp{^wGDM<(U6s^AX_%`*jEtYVxpHU=UQh!6=GnKK|xi-8Ok zS62bRHJvjv=~-C}WWbfwP3SRXWdW?){EWQ1JlP2PXT?-kPw^r??1ld|-;r)9Q-wUi zM~Ndm2gEi1MsjlTY;BZuZ{pJ&*bA}}{h%y1T|Ahos;ZW@c25{vu+>vOKEA}YfF5j6 zHQo?8vwMa-2Y3iS9+DqskoqJCVzZz{KogaajBJc!?+pw#P&en-3!vx`-@AJf-p5vT zb~%~_6oS?yM{5HfB!$2Kcfic?CK5NID-dbT{M#%G(VXV*cg=@-d+X`#{qAs|?m7KJ zVF;%^d3uxU+p7vw+tz0L#5`8D=O-kXK_9+-qq}V}$F&R(=N%u9uk;plKpYb(jg0q91;ZG}_#wgbbGl65Q~gvOE@G)_7pnC93qYK{<4;Z!abo!X{re6gU73X##6wPLY4qaayY3VtW@ct!O^C_JzcMA=uZjzEHLwtPO6BW>P zH##{#7MB*<8>z*Bkzfn|+4cdQ%FYy6fiDROe0Ki?L?49^+WqP74#lGJl&mpn4}41& zk*lilWoMWpN+HYY@15p;_#>!0SNm+@!;YjrO?5mS5MR=p>+R_Y<1_swpil8#6nyUW z#gVAnCJA6g^`ZeT8*WiiQD`w@5%MyVhcN-CL?@!XCyYD&S?u*$E3xnUbOBk7R|9{| zZc2?fjq9R^|0rV4_zDjV4N;fANba+6eMSu(d}C_Ov5+q2vU&U-)V1@QJ^#7&D>g>P zpmZ^hxYX1qtl5NFM!);}E!GFWE4sOzIM@nJ!E$9OvX! zW{)KlW8`U9Vrcy%8%ps(@0Qhz%zB>cBa{>tVE*Ump{IM6{g#=))J9`ww(s;$PCkl~ zlLlt|QO1|fAk$y-i;K=(UN8*}jWjV2{M0^{{~o#?em^TUb?Oae)T-_>)pW&aaqtZy4an@$sV}dD?}gr4Ar04!v_K0D<6VGagU@ zfo2JybLRp4v2k&k1AqKj#4!Mr6`;gq^D_f4EXRI^T>JeK*rgRu+E*%j#OFxIh>34D z)1NNRYexw>9Z0i#zF)3p!8-7Vj!s-SaET24DCwTPDXy78);c=Y~jP(d(b5op!zNSin8%& zO$9^v`}c22Ai3jPLt7_6Q^DV`@$f*)+^(0gtmxfqah-Vi?LI=>Dm_y}8|JSl&p5(~_LCi%UUSSzdYhXTZPe8X6yr%ev?0 zGH-9Lp1Q892L=U|QPLo-1#fR}kJpESf`ji>8*r`lr)U4DC5!0#S@)){@}y}mzfMnY zf2Xc^swLGaMq+G%hkyVZn-0WwU?7_J>2{c0Hob%dwWXyc9ws^xiTv{Q>*0J5!Jqr= z!B)D!_|)IsUTNc!b3f*m6=8y-)}wCWVM@=Q>o|bD!?b5g0zc zaCURcwHYt%tGxk)g9AQQE{YOqKSjsO`x*4?Q>R54@8k8CfMx)gF)}a&0+ySep0@W) zmCKF=B7%uW5s;ZlOU`XLZ~LD8B2TT#Z=4%3@F%OX5QoQM&;5~NDnJBcCnqQ9xCW34 z4}mEIbPpYQ05mD6T~bjIho|+r(y)d;*lG{;Z!z1SXt4%~-DPW{(l%>L%eC9Eu&`pi z3h0^-QR!ep;pJ}9Gr(`Dc2fX^0M59#&X?*iZiWDWD)|LnA2LdI-2EXk<4MNs0o><@NQoUX67^+GnXx?ggMv zpamQ&tE;EAa@k(eKgZL%v5Q^Z-DyZ@X$K0mi_Bsb>1C>d%Rt4pC=%CIR}WClTLNzP z!Rpr<`HoP6yu!k83|z8my_5D&pRl35N55idV*qe^w;;O4wQSTtzkbjuS^pmgZPuc3uo)&1mcB2oiN!RaaLZWZ28rEC6uQ(bxB&2cUM>J*ROIK8nJ0#fO^vh+?=RBF!3-rcSb|FJ3CX7QB(K*tFZxN7yI-j zw$ljoUUaslY02Hl2-0A?uXat1jamQgON3s818Mo@>;F9;32obW8H9OoG7dYsSj%*| zK0mPVr%mkx-}DgFbBd^6Vuh{gFZ?qyBz%M#VP)gQ#KawmFh6$I-2dhn%G_TPa3Z%( z7#SED^;`W~mb|^_gX>Av#1CYGrZ6xtM#sj4l+G)scG|jX5YTQv@Q`nE(tKT@8&-J` z=M27Ar{*`8C-B1_R-h0$lc!gpw}3^*d_d_ce^LcH?XN|Z>Dx_OrB|=!BX|&>|Lb*( za+u=zrl)J$yB6aY&&0*kMn|=PgUemRrBCSsoHX|BPnto=>fm?jx@IF745r1AuvFXi zZlP%pVG8&NRLz<_9`c$L;Oar~_yeGGqQQlg7NE|Y|6L2C`YfKT2DE?9pyaFH&E<^7 zO>s_6a6mu+rte}XwDSU*;p^}J8V{c3&AoE)$2DG;i2Zm(NJz-KJ47FJ&BGt#HBlS2HX@9%v9as; z(XF`S5l~8qs;G-Bn5twDJ0*TczA$1=k&|2D*0jI~Nl8f|*~7}py7_&$3Ha@^RoV8= z&e#freh3Bg^khQ-I|m091A`|t7RuJIdj&{J+h&PCa023x-f2^vZKjba^#UadeXU#O zFxAK&-lCqt-L7)Vu>5G%C$n$i<4ICdWErZft6yhB3npvjnGP45Mva2wf$=%vBuLoW zLg!yVX*nJ4drVG$d-`hu1$xhM^Sch9<*3hq7K}>-K;SVJ78YJ-Wu>dQ{u z$Dm{nZw9PbeAcm}#*T9+Q=Gawx*Y4mg$rP>N_Fwdwo5?W;E(cxRgzU&D_xgRjxz}N zpVMc$u?z{>eG8l?PMj#R9Th>dMj@x!bY*2_Ph2zac6qL&oudh41ng;tk6+IucLRrS zGTj{d$lP4R&~Sn~5IJ2?+3fs0;b3b*;*dyyhX=|yeEFH3AOGQ%%e-Wte?PUzIN3(dh` zgZX-^TU+rgR{|{@Rc~%+W=J>V-s<%6jV=$DaDhEI!z%R>NC}-0U+a;cFnZo$RwCBk z>$g1>0>~WF+1aVb^{Gl{78Mm?ok0wQv=~2}{q$cH_M)8HP_b#p zbzVi^J$!j!V4$!rV_wDOu*T>CMeZ-r`3Zt#hw& z)LMn+%!LUF^l|+9JY26Sr{7aEGI~{zwNBjQ8L(zvrI(oggp>TP>9igdFjTD1b=^`+I@oJ;?fednnEEigBxvR z1WA7BspK^MM)A-2z~}e!w5AppM`ooNV4H=015o<}nQFqsr!udwRqsEiH`}zm$|Z@8Lw9ot%ujUR>WN#MeDX7&clVNgNj) z?PRa7KjvW+A6U3)9ju%0i|yzLw;g>y<``QPNNHQBnkv(E37Ow$A$> zi;H)kvfKnyQoS*#2L_W6BEA3kvPRZF+S;w4xR}?v`AKP#x;`deGao9qXH85@ydfqw zg~yNapVoSil3j`x>6gE>wZ(Ad6clhuNF)KzoM&dnyBFx@bMo+np^MNSHj#jIJ$3rD z^NCK-!?b+&Kl7}m0*aoF7kF*w$b5oGu0G!X_^e!qSnE}=g|A}J|pXF+jK@iYa68Sp?tmsxv9M_Wh7Ul~|5!GB%s-*5Bt z@`VO99~`EL4gC(TJ2=ufwS-+j$%pc^3u!05{uM&;A3i$LcDIG&7&q}e@%4CL-;7YY ze9{AfjR+z@Taw@>8?3f$7{sHX<)y zzAVt4`X^rtZRJKn3uHJ-qI1K8zz+4ZjrAG#vytBD!5w|2P@NOesm#I6iT?dxQ!AGALbf>M6u z<(Ik~NSWW#(h?*(;a$?;15&grNwoaa9$38Pimxn(dv!;QKwT-Giot~Uk&DeD($(> zy@dNYTZ!{7NarqZ5x0;bb3A_R`&blZz1>y^gSwtt({FGFOLIqP; zj@GYlDX7taj=$l(Ey%0&_D@mK_z+g#?MnVUXnJ!nXzDI^$*^qTVVyvq=0UXHNC(aG zVs!hw?11gQ*#6y2^%^|Djg=M8>gsCmwIE77%F_9VxC=KJGRIVKE*jY?wgV>}czJnI zMq-dgXm-SX#R$v=x-uJ`q+gwGLF+1QxWx@m5GBAvUtDqe`QjlKJt;TQeN%F3ANH23 z3ay*Du3d{c%P4|wt5UGs0gs4I7wjJ#6yC7^u7x(43|}r0OLb@IusOZ&U}bpWjd?D% z`}6e7eACUEA8eeQ^I@an&huwb=B=+^HNbLUEnWk_xZp=Nf(yPUK$Jn3$LHM+q-4Hp zX2uDzqLG`MYoJXtcf{^pmV)zl)7RHW4Gf;Dml0~)3$*BgFFWG92ReRrWyNTu)bdL0 zvZe;I-f+*~Dk_*Sg9|RQa&oIfCQr*2d;x`H z;^P%C7!h#uLwF1c1=HVvFVkV3YeOe)vv`LbE(tp8msrp?WCSjKprhJqr%H!gp|O-xNB#(v%|wr zLx80l*7^4C1ybS-Y;CVVpQ0o^XNfz(h!l#w=zGD#!*c`JUq?p=EzCB>) zVSLVj6{vR@dgy*o9f3Q1JUqsr`J-08E-tn&71lu`JSgBC9$D+N>j%C>w@QH2M_sv> zi7td;|MO3O80u|pT!G(;qL{=QL891K5n&{P3cvcRM#yI@CfV0oYWo$Du{D=LOdNOoT2J*XDFx@n2#?$TVuVWft2OR;I81o zzCWu$-h(Sd8HGkRYPGesk=_*&5(0esE4xn(qziYTWtpzAv9bH)V^RkUrgmy8Cok_3 z2S>|0TAe$0LT9dm6AV7f_z{5`AKz;*{U9emab2Z90~Q3hF6ZsrW*ZS|O3GWQQa<0n z<})ad(rxAE=g$S4lbRehPqls59*%a`fEz3=6&9Nc{J97<_B^znPCi4e=22TT=7=UhB!L2`YMFQDn8@7^2v8iA?ISi%tM4gc%6+>$QwxSTR>+% zpF~B~F5_wG=we>J>;yywUFGiXjuLSun2-RxBNSs}V{`ZLcqm8@HMHxugI_?I{mgq< zx>If{f~QJ)dLlC`3)wW(Y8OWZ`1l@;SEIeZUq0_Xl=wUGqePL7V2QCE*% z=Ldyg-xkfW|J__VqqcV3F;yk|5+7g0DQeErJuyY;=SfK&poX})x$Eu9O-hCtm^74> z&OmrqEsmIh%SWgD8yjVS+;8|DR?l25)9peAXltu=baXGPaRF(?YWJ&Pfx)cmyE2BQ z-;f^wk^&TSkUS!!_lsUjCtZQ|y=vV2L_U^2iI>uk(8g7LxVr=zT33 zm6JP(cDhZ}`rOagYW+JwD^D&{9X^wRI;g0qSd{51WVTFC8*B1h0Byi>(^JrTkZ*VE z$3&X4fyZ2I5`TrS4wxr^(gq7*LSO1(DRw609P2y~3{Wzd+<$U30h4It8pg($Bjx{s zXQ7iM54UGT%g&+AJ4YLaM+4u(ZM3ztPy#A!p`mo`&OD=71qz?sT;EsR?Gc+Fp1#10 zU`+hwpkE~H$0Y%Pq!yHAo={{0t)Xce(EeicD#LgTppe(ra?2y3F{i1p!P%e7U^HmT zkBf^mZy>D8tjVriA}X(_=-K|hH-SJv+6pZe<1t^x#ymC#w2{PVXlU>TtJOa+aEwl% zEg~YKxv6O;ZWFM8ij}noI{UL#S>)YH+`s_Z>4oN?!&}tEz_6<8=_x`Dq2ZJg1gC-r zMR~cojbOkcQD{FQwF}1J?NUZ;Z0zbncItUb%KG`PB(=N&C#O?v%lCiJLc~T#3l+ov zi7niyjYdtjD_OV88;m}8rQA#kVe$OiGa9vr$i0ji2s!O}fO3G>781{v2vBcOMoy*n%k2 zGknU&-6nkSBK~FN9TW=0yk|%P4r5fMED61oY=Pe_zzupF?tImGZ=;h@lr~^hu{G`m zgQdq-En+4F8KdD6bC2mzF2SEC5Z66#-t=5ggF0)U5sKkb>Rql};+K!0$J*?b zft!NyPk!!7ebqqR|J$RW z4P51=N12(Kd9(MXViDd*nCmeN*KRoRf?3OeV=d+lZ*iklj~Ro zRz%Ww3aMniPyvSi0dD=&sZ%JUt(wC$f&0F%uN2t6z~JCbZG&9`lLj2@-Av5$>nhWU2S}_ebW)?tAT+_j~ydkOVZWjC=jK zYT3)aa8>4sgu1CIhkf-b30g~|Ap^>Wp@in(&YySEV^Vc=yb8XZ36x1C|L}1%Yfa6m zB}1}G$bgk=Cu=}>M&i`e6ncYbX`OlZ%ZCfS znVOoKlra3VQ+?Y3spbNyWS&@C*k3Zjvw5E-9{l4B$qbalEyZV3XMqE%H4duLV=9R5 zQ%{dQO(SSU7@w$GEG*;8PfdYAP_oM^@I93&_9)MK=&LtxJ~?(mV@s|!Q+K^rg-(Vj zA`NJ`WnjR9#bTRVTHwnny1EQi7(C!Q6-X6gqmp560fDd0q0~qwMn|6qWWiL8ttJlP zH%U53AB3u8XMlu4m)mIjq>R~dNk}B+>y@kP>H@Brfa2ZnIodx5T&r>()Z!}9W>`8m z>zB4EYoekCBf$@77D6deNX|0^IM`{b@gz$7Ep@nU0t1cl<}$d@wnxU>D^o7eK4V&y zdKFF_&^<^=;|^Y+OQ-7g=LF6YcPI+fxQu0(omej|8P&=Hav_A|h);djV2eQ@sgou60?ZrO)9t zH|};@3p4C);yd1`WxOy=MVpXnJU3h;HT65@dN*LSo?#Y$F6FcPEFobijqD@%A zVT*_Oj#ND(qd#G+fVFVmUU}F$Vh#!wnm7n&;H5l|f#E_*W@LMz){{61l-q)6-Oky< z+TGlB66l`bkr7>~qkU)83FN0wGv4&A5Kk{DxmM$~#d-DWY8$UmrWJO=!T9d2amH^JkPM^LcA`&-V<3&wJr{Llu3`C2& z7wQC31o3Tuf4}hV7h#68ATS2ymarM5J<>;#$N}WXF;L1p*<1!8j$+G}wtnQIw=1a# zW2AhK;UOU&rCaYk4ZT3p4^{)zSGC95P>q*cOb^yV_|1Toq@?7N$Vd)x@fT@nX(AQE zj)(g@5GsCCQz5LiuW-zg?$5xCihk$;H$!LOJ{31FnJne=Vs`e?xc8beFgEZd5EcB4 zV4~6u2?-NHdO?OrcTZ1Hek*Hh)^SeDC}B*O=)|Lq#Q`)Jo`RMI1=qCJBP|e5N6sd@ zbfV};i)OAVimA2|Z3xNO;w`Lh{NcMLv9lgTIZX>%q}Zr|I65WM$ zrlx2P)@|G)1mL5xOR60F@3ZLWe}`@4Gq983N48gI7~h`e45Un-IaxG##!&xpK76vZ z^ELHrkKbX5_o56LHUjOsa&i&^;bd8StJPFInLaX$J+dnx1)Y8zkAQktxqp6yO4P;U?rCgNnk)S;q>XafDi;up>`-AI+} zCbN=PSPN+OOTxm>Cnt^2z03qcWr3KobX-hK({xKXXv>DazO}5UXpsX1P%#W@dA9TX z{I3_5mJ|V^LCT&b*ML`$lar$)-sor&G&4}^7_wXIpo5v=5yA$@#pr@k>B9MkymBLq<;CI5BY)_PEom zFyrKrp;44g3n29PiFCJSF^OVv@l8gzJxMr6&iE_8Z*9qH3y;>^gP-wq>-g$ zxmjTY#zq`vxp(?ev-RGB$U0}`g7;GpLUu3m9It3$jFNZU!aBCv9}!O)rjcw z7%(1SFSqyCdN`0+Aa&ebw-m7*X@;8Z9Pa&CU0rqTJh=qk(WEshR&fuU?cYS!nJaUx zEw0s-QS?{;MJwpawCglb;y{E+wDm8pdG8PFL}@HXSQ-6gGnNXMFa655g|+GMR=1{dz3@5=4Lz5Oq0Iy&Z>l1@TgH zax&@q;s8oSaYx7S|1T2%&-MN)YHDgEth&3q387@aK4I@1|y~vNztn0-|#$WN|tal z>xjFOpu}lu`KyG@%s(K^k#`h>wgdTbkb6wYC(_o^|538amIKf@kW%y{$M3?#gKj|? z_hR$%mRhaKJVypn@+#$5UKFC$NK(=g+fflf%}m$dub)=%6(ruQ99s!cPeNZp9D(vPSqHXyj|D){L)Y5=#wOm8tIT?V)8@Zy%q`>;L;TTH=Jt z(8=o>8Cc1gnHjVI450Z;>wnRscg9FMWo_d>`3#Q+Fuk^;@2MCVMiwRVxZ5w-8!miJ z@_Ebg9{XR^9A@-mSLs1=Z&v<0A4_|&bF-4@1&5a|T?(+TJD` + + + + + + + + + + + + + + + + image/svg+xml + + + + + + + + synchronize_rcu() + + + + + + + WRITE_ONCE(a, 1); + WRITE_ONCE(b, 1); + r1 = READ_ONCE(a); + WRITE_ONCE(c, 1); + r2 = READ_ONCE(b); + r3 = READ_ONCE(c); + thread0() + thread1() + thread2() + + + + rcu_read_lock(); + rcu_read_lock(); + rcu_read_unlock(); + rcu_read_unlock(); + + QS + + QS + + + QS + + diff --git a/Documentation/RCU/Design/Requirements/RCUApplicability.svg b/Documentation/RCU/Design/Requirements/RCUApplicability.svg new file mode 100644 index 0000000000000..ebcbeee391ed7 --- /dev/null +++ b/Documentation/RCU/Design/Requirements/RCUApplicability.svg @@ -0,0 +1,237 @@ + + + + + + + + + + + + image/svg+xml + + + + + + + + + + + + + + + + + + Read-Mostly, Stale & + + Inconsistent Data OK + + (RCU Works Great!!!) + + (RCU Works Well) + + Read-Mostly, Need Consistent Data + + Read-Write, Need Consistent Data + + Update-Mostly, Need Consistent Data + + (RCU Might Be OK...) + + (1) Provide Existence Guarantees For Update-Friendly Mechanisms + + (2) Provide Wait-Free Read-Side Primitives for Real-Time Use) + + (RCU is Very Unlikely to be the Right Tool For The Job, But it Can: + + diff --git a/Documentation/RCU/Design/Requirements/ReadersPartitionGP1.svg b/Documentation/RCU/Design/Requirements/ReadersPartitionGP1.svg new file mode 100644 index 0000000000000..48cd1623d4d49 --- /dev/null +++ b/Documentation/RCU/Design/Requirements/ReadersPartitionGP1.svg @@ -0,0 +1,639 @@ + + + + + + + + + + + + + + + + + + + + + + + image/svg+xml + + + + + + + + synchronize_rcu() + + + + + + + WRITE_ONCE(a, 1); + WRITE_ONCE(b, 1); + r1 = READ_ONCE(a); + WRITE_ONCE(c, 1); + WRITE_ONCE(d, 1); + r2 = READ_ONCE(c); + thread0() + thread1() + thread2() + + + + rcu_read_lock(); + rcu_read_lock(); + rcu_read_unlock(); + rcu_read_unlock(); + + QS + + QS + + + QS + + + + synchronize_rcu() + + + + + + + r3 = READ_ONCE(d); + WRITE_ONCE(e, 1); + + QS + r4 = READ_ONCE(b); + r5 = READ_ONCE(e); + rcu_read_lock(); + rcu_read_unlock(); + QS + + QS + + QS + + thread3() + thread4() + + diff --git a/Documentation/RCU/Design/Requirements/Requirements.html b/Documentation/RCU/Design/Requirements/Requirements.html new file mode 100644 index 0000000000000..36de7aaa941e2 --- /dev/null +++ b/Documentation/RCU/Design/Requirements/Requirements.html @@ -0,0 +1,2799 @@ + + + + + A Tour Through RCU's Requirements [LWN.net] + + +

A Tour Through RCU's Requirements

+ +

Copyright IBM Corporation, 2015

+

Author: Paul E. McKenney

+

The initial version of this document appeared in the +LWN articles +here, +here, and +here.

+ +

Introduction

+ +

+Read-copy update (RCU) is a synchronization mechanism that is often +used as a replacement for reader-writer locking. +RCU is unusual in that updaters do not block readers, +which means that RCU's read-side primitives can be exceedingly fast +and scalable. +In addition, updaters can make useful forward progress concurrently +with readers. +However, all this concurrency between RCU readers and updaters does raise +the question of exactly what RCU readers are doing, which in turn +raises the question of exactly what RCU's requirements are. + +

+This document therefore summarizes RCU's requirements, and can be thought +of as an informal, high-level specification for RCU. +It is important to understand that RCU's specification is primarily +empirical in nature; +in fact, I learned about many of these requirements the hard way. +This situation might cause some consternation, however, not only +has this learning process been a lot of fun, but it has also been +a great privilege to work with so many people willing to apply +technologies in interesting new ways. + +

+All that aside, here are the categories of currently known RCU requirements: +

+ +
    +
  1. + Fundamental Requirements +
  2. Fundamental Non-Requirements +
  3. + Parallelism Facts of Life +
  4. + Quality-of-Implementation Requirements +
  5. + Linux Kernel Complications +
  6. + Software-Engineering Requirements +
  7. + Other RCU Flavors +
  8. + Possible Future Changes +
+ +

+This is followed by a summary, +which is in turn followed by the inevitable +answers to the quick quizzes. + +

Fundamental Requirements

+ +

+RCU's fundamental requirements are the closest thing RCU has to hard +mathematical requirements. +These are: + +

    +
  1. + Grace-Period Guarantee +
  2. + Publish-Subscribe Guarantee +
  3. + RCU Primitives Guaranteed to Execute Unconditionally +
  4. + Guaranteed Read-to-Write Upgrade +
+ +

Grace-Period Guarantee

+ +

+RCU's grace-period guarantee is unusual in being premeditated: +Jack Slingwine and I had this guarantee firmly in mind when we started +work on RCU (then called “rclock”) in the early 1990s. +That said, the past two decades of experience with RCU have produced +a much more detailed understanding of this guarantee. + +

+RCU's grace-period guarantee allows updaters to wait for the completion +of all pre-existing RCU read-side critical sections. +An RCU read-side critical section +begins with the marker rcu_read_lock() and ends with +the marker rcu_read_unlock(). +These markers may be nested, and RCU treats a nested set as one +big RCU read-side critical section. +Production-quality implementations of rcu_read_lock() and +rcu_read_unlock() are extremely lightweight, and in +fact have exactly zero overhead in Linux kernels built for production +use with CONFIG_PREEMPT=n. + +

+This guarantee allows ordering to be enforced with extremely low +overhead to readers, for example: + +

+
+ 1 int x, y;
+ 2
+ 3 void thread0(void)
+ 4 {
+ 5   rcu_read_lock();
+ 6   r1 = READ_ONCE(x);
+ 7   r2 = READ_ONCE(y);
+ 8   rcu_read_unlock();
+ 9 }
+10
+11 void thread1(void)
+12 {
+13   WRITE_ONCE(x, 1);
+14   synchronize_rcu();
+15   WRITE_ONCE(y, 1);
+16 }
+
+
+ +

+Because the synchronize_rcu() on line 14 waits for +all pre-existing readers, any instance of thread0() that +loads a value of zero from x must complete before +thread1() stores to y, so that instance must +also load a value of zero from y. +Similarly, any instance of thread0() that loads a value of +one from y must have started after the +synchronize_rcu() started, and must therefore also load +a value of one from x. +Therefore, the outcome: +

+
+(r1 == 0 && r2 == 1)
+
+
+cannot happen. + +

Quick Quiz 1: +Wait a minute! +You said that updaters can make useful forward progress concurrently +with readers, but pre-existing readers will block +synchronize_rcu()!!! +Just who are you trying to fool??? +
Answer + +

+This scenario resembles one of the first uses of RCU in +DYNIX/ptx, +which managed a distributed lock manager's transition into +a state suitable for handling recovery from node failure, +more or less as follows: + +

+
+ 1 #define STATE_NORMAL        0
+ 2 #define STATE_WANT_RECOVERY 1
+ 3 #define STATE_RECOVERING    2
+ 4 #define STATE_WANT_NORMAL   3
+ 5
+ 6 int state = STATE_NORMAL;
+ 7
+ 8 void do_something_dlm(void)
+ 9 {
+10   int state_snap;
+11
+12   rcu_read_lock();
+13   state_snap = READ_ONCE(state);
+14   if (state_snap == STATE_NORMAL)
+15     do_something();
+16   else
+17     do_something_carefully();
+18   rcu_read_unlock();
+19 }
+20
+21 void start_recovery(void)
+22 {
+23   WRITE_ONCE(state, STATE_WANT_RECOVERY);
+24   synchronize_rcu();
+25   WRITE_ONCE(state, STATE_RECOVERING);
+26   recovery();
+27   WRITE_ONCE(state, STATE_WANT_NORMAL);
+28   synchronize_rcu();
+29   WRITE_ONCE(state, STATE_NORMAL);
+30 }
+
+
+ +

+The RCU read-side critical section in do_something_dlm() +works with the synchronize_rcu() in start_recovery() +to guarantee that do_something() never runs concurrently +with recovery(), but with little or no synchronization +overhead in do_something_dlm(). + +

Quick Quiz 2: +Why is the synchronize_rcu() on line 28 needed? +
Answer + +

+In order to avoid fatal problems such as deadlocks, +an RCU read-side critical section must not contain calls to +synchronize_rcu(). +Similarly, an RCU read-side critical section must not +contain anything that waits, directly or indirectly, on completion of +an invocation of synchronize_rcu(). + +

+Although RCU's grace-period guarantee is useful in and of itself, with +quite a few use cases, +it would be good to be able to use RCU to coordinate read-side +access to linked data structures. +For this, the grace-period guarantee is not sufficient, as can +be seen in function add_gp_buggy() below. +We will look at the reader's code later, but in the meantime, just think of +the reader as locklessly picking up the gp pointer, +and, if the value loaded is non-NULL, locklessly accessing the +->a and ->b fields. + +

+
+ 1 bool add_gp_buggy(int a, int b)
+ 2 {
+ 3   p = kmalloc(sizeof(*p), GFP_KERNEL);
+ 4   if (!p)
+ 5     return -ENOMEM;
+ 6   spin_lock(&gp_lock);
+ 7   if (rcu_access_pointer(gp)) {
+ 8     spin_unlock(&gp_lock);
+ 9     return false;
+10   }
+11   p->a = a;
+12   p->b = a;
+13   gp = p; /* ORDERING BUG */
+14   spin_unlock(&gp_lock);
+15   return true;
+16 }
+
+
+ +

+The problem is that both the compiler and weakly ordered CPUs are within +their rights to reorder this code as follows: + +

+
+ 1 bool add_gp_buggy_optimized(int a, int b)
+ 2 {
+ 3   p = kmalloc(sizeof(*p), GFP_KERNEL);
+ 4   if (!p)
+ 5     return -ENOMEM;
+ 6   spin_lock(&gp_lock);
+ 7   if (rcu_access_pointer(gp)) {
+ 8     spin_unlock(&gp_lock);
+ 9     return false;
+10   }
+11   gp = p; /* ORDERING BUG */
+12   p->a = a;
+13   p->b = a;
+14   spin_unlock(&gp_lock);
+15   return true;
+16 }
+
+
+ +

+If an RCU reader fetches gp just after +add_gp_buggy_optimized executes line 11, +it will see garbage in the ->a and ->b +fields. +And this is but one of many ways in which compiler and hardware optimizations +could cause trouble. +Therefore, we clearly need some way to prevent the compiler and the CPU from +reordering in this manner, which brings us to the publish-subscribe +guarantee discussed in the next section. + +

Publish/Subscribe Guarantee

+ +

+RCU's publish-subscribe guarantee allows data to be inserted +into a linked data structure without disrupting RCU readers. +The updater uses rcu_assign_pointer() to insert the +new data, and readers use rcu_dereference() to +access data, whether new or old. +The following shows an example of insertion: + +

+
+ 1 bool add_gp(int a, int b)
+ 2 {
+ 3   p = kmalloc(sizeof(*p), GFP_KERNEL);
+ 4   if (!p)
+ 5     return -ENOMEM;
+ 6   spin_lock(&gp_lock);
+ 7   if (rcu_access_pointer(gp)) {
+ 8     spin_unlock(&gp_lock);
+ 9     return false;
+10   }
+11   p->a = a;
+12   p->b = a;
+13   rcu_assign_pointer(gp, p);
+14   spin_unlock(&gp_lock);
+15   return true;
+16 }
+
+
+ +

+The rcu_assign_pointer() on line 13 is conceptually +equivalent to a simple assignment statement, but also guarantees +that its assignment will +happen after the two assignments in lines 11 and 12, +similar to the C11 memory_order_release store operation. +It also prevents any number of “interesting” compiler +optimizations, for example, the use of gp as a scratch +location immediately preceding the assignment. + +

Quick Quiz 3: +But rcu_assign_pointer() does nothing to prevent the +two assignments to p->a and p->b +from being reordered. +Can't that also cause problems? +
Answer + +

+It is tempting to assume that the reader need not do anything special +to control its accesses to the RCU-protected data, +as shown in do_something_gp_buggy() below: + +

+
+ 1 bool do_something_gp_buggy(void)
+ 2 {
+ 3   rcu_read_lock();
+ 4   p = gp;  /* OPTIMIZATIONS GALORE!!! */
+ 5   if (p) {
+ 6     do_something(p->a, p->b);
+ 7     rcu_read_unlock();
+ 8     return true;
+ 9   }
+10   rcu_read_unlock();
+11   return false;
+12 }
+
+
+ +

+However, this temptation must be resisted because there are a +surprisingly large number of ways that the compiler +(to say nothing of +DEC Alpha CPUs) +can trip this code up. +For but one example, if the compiler were short of registers, it +might choose to refetch from gp rather than keeping +a separate copy in p as follows: + +

+
+ 1 bool do_something_gp_buggy_optimized(void)
+ 2 {
+ 3   rcu_read_lock();
+ 4   if (gp) { /* OPTIMIZATIONS GALORE!!! */
+ 5     do_something(gp->a, gp->b);
+ 6     rcu_read_unlock();
+ 7     return true;
+ 8   }
+ 9   rcu_read_unlock();
+10   return false;
+11 }
+
+
+ +

+If this function ran concurrently with a series of updates that +replaced the current structure with a new one, +the fetches of gp->a +and gp->b might well come from two different structures, +which could cause serious confusion. +To prevent this (and much else besides), do_something_gp() uses +rcu_dereference() to fetch from gp: + +

+
+ 1 bool do_something_gp(void)
+ 2 {
+ 3   rcu_read_lock();
+ 4   p = rcu_dereference(gp);
+ 5   if (p) {
+ 6     do_something(p->a, p->b);
+ 7     rcu_read_unlock();
+ 8     return true;
+ 9   }
+10   rcu_read_unlock();
+11   return false;
+12 }
+
+
+ +

+The rcu_dereference() uses volatile casts and (for DEC Alpha) +memory barriers in the Linux kernel. +Should a +high-quality implementation of C11 memory_order_consume [PDF] +ever appear, then rcu_dereference() could be implemented +as a memory_order_consume load. +Regardless of the exact implementation, a pointer fetched by +rcu_dereference() may not be used outside of the +outermost RCU read-side critical section containing that +rcu_dereference(), unless protection of +the corresponding data element has been passed from RCU to some +other synchronization mechanism, most commonly locking or +reference counting. + +

+In short, updaters use rcu_assign_pointer() and readers +use rcu_dereference(), and these two RCU API elements +work together to ensure that readers have a consistent view of +newly added data elements. + +

+Of course, it is also necessary to remove elements from RCU-protected +data structures, for example, using the following process: + +

    +
  1. Remove the data element from the enclosing structure. +
  2. Wait for all pre-existing RCU read-side critical sections + to complete (because only pre-existing readers can possibly have + a reference to the newly removed data element). +
  3. At this point, only the updater has a reference to the + newly removed data element, so it can safely reclaim + the data element, for example, by passing it to kfree(). +
+ +This process is implemented by remove_gp_synchronous(): + +
+
+ 1 bool remove_gp_synchronous(void)
+ 2 {
+ 3   struct foo *p;
+ 4
+ 5   spin_lock(&gp_lock);
+ 6   p = rcu_access_pointer(gp);
+ 7   if (!p) {
+ 8     spin_unlock(&gp_lock);
+ 9     return false;
+10   }
+11   rcu_assign_pointer(gp, NULL);
+12   spin_unlock(&gp_lock);
+13   synchronize_rcu();
+14   kfree(p);
+15   return true;
+16 }
+
+
+ +

+This function is straightforward, with line 13 waiting for a grace +period before line 14 frees the old data element. +This waiting ensures that readers will reach line 7 of +do_something_gp() before the data element referenced by +p is freed. +The rcu_access_pointer() on line 6 is similar to +rcu_dereference(), except that: + +

    +
  1. The value returned by rcu_access_pointer() + cannot be dereferenced. + If you want to access the value pointed to as well as + the pointer itself, use rcu_dereference() + instead of rcu_access_pointer(). +
  2. The call to rcu_access_pointer() need not be + protected. + In contrast, rcu_dereference() must either be + within an RCU read-side critical section or in a code + segment where the pointer cannot change, for example, in + code protected by the corresponding update-side lock. +
+ +

Quick Quiz 4: +Without the rcu_dereference() or the +rcu_access_pointer(), what destructive optimizations +might the compiler make use of? +
Answer + +

+This simple linked-data-structure scenario clearly demonstrates the need +for RCU's stringent memory-ordering guarantees on systems with more than +one CPU: + +

    +
  1. Each CPU that has an RCU read-side critical section that + begins before synchronize_rcu() starts is + guaranteed to execute a full memory barrier between the time + that the RCU read-side critical section ends and the time that + synchronize_rcu() returns. + Without this guarantee, a pre-existing RCU read-side critical section + might hold a reference to the newly removed struct foo + after the kfree() on line 14 of + remove_gp_synchronous(). +
  2. Each CPU that has an RCU read-side critical section that ends + after synchronize_rcu() returns is guaranteed + to execute a full memory barrier between the time that + synchronize_rcu() begins and the time that the RCU + read-side critical section begins. + Without this guarantee, a later RCU read-side critical section + running after the kfree() on line 14 of + remove_gp_synchronous() might + later run do_something_gp() and find the + newly deleted struct foo. +
  3. If the task invoking synchronize_rcu() remains + on a given CPU, then that CPU is guaranteed to execute a full + memory barrier sometime during the execution of + synchronize_rcu(). + This guarantee ensures that the kfree() on + line 14 of remove_gp_synchronous() really does + execute after the removal on line 11. +
  4. If the task invoking synchronize_rcu() migrates + among a group of CPUs during that invocation, then each of the + CPUs in that group is guaranteed to execute a full memory barrier + sometime during the execution of synchronize_rcu(). + This guarantee also ensures that the kfree() on + line 14 of remove_gp_synchronous() really does + execute after the removal on + line 11, but also in the case where the thread executing the + synchronize_rcu() migrates in the meantime. +
+ +

Quick Quiz 5: +Given that multiple CPUs can start RCU read-side critical sections +at any time without any ordering whatsoever, how can RCU possibly tell whether +or not a given RCU read-side critical section starts before a +given instance of synchronize_rcu()? +
Answer + +

Quick Quiz 6: +The first and second guarantees require unbelievably strict ordering! +Are all these memory barriers really required? +
Answer + +

+In short, RCU's publish-subscribe guarantee is provided by the combination +of rcu_assign_pointer() and rcu_dereference(). +This guarantee allows data elements to be safely added to RCU-protected +linked data structures without disrupting RCU readers. +This guarantee can be used in combination with the grace-period +guarantee to also allow data elements to be removed from RCU-protected +linked data structures, again without disrupting RCU readers. + +

+This guarantee was only partially premeditated. +DYNIX/ptx used an explicit memory barrier for publication, but had nothing +resembling rcu_dereference() for subscription, nor did it +have anything resembling the smp_read_barrier_depends() +that was later subsumed into rcu_dereference(). +The need for these operations made itself known quite suddenly at a +late-1990s meeting with the DEC Alpha architects, back in the days when +DEC was still a free-standing company. +It took the Alpha architects a good hour to convince me that any sort +of barrier would ever be needed, and it then took me a good two hours +to convince them that their documentation did not make this point clear. +More recent work with the C and C++ standards committees have provided +much education on tricks and traps from the compiler. +In short, compilers were much less tricky in the early 1990s, but in +2015, don't even think about omitting rcu_dereference()! + +

RCU Primitives Guaranteed to Execute Unconditionally

+ +

+The common-case RCU primitives are unconditional. +They are invoked, they do their job, and they return, with no possibility +of error, and no need to retry. +This is a key RCU design philosophy. + +

+However, this philosophy is pragmatic rather than pigheaded. +If someone comes up with a good justification for a particular conditional +RCU primitive, it might well be implemented and added. +After all, this guarantee was reverse-engineered, not premeditated. +The unconditional nature of the RCU primitives was initially an +accident of implementation, and later experience with synchronization +primitives with conditional primitives caused me to elevate this +accident to a guarantee. +Therefore, the justification for adding a conditional primitive to +RCU would need to be based on detailed and compelling use cases. + +

Guaranteed Read-to-Write Upgrade

+ +

+As far as RCU is concerned, it is always possible to carry out an +update within an RCU read-side critical section. +For example, that RCU read-side critical section might search for +a given data element, and then might acquire the update-side +spinlock in order to update that element, all while remaining +in that RCU read-side critical section. +Of course, it is necessary to exit the RCU read-side critical section +before invoking synchronize_rcu(), however, this +inconvenience can be avoided through use of the +call_rcu() and kfree_rcu() API members +described later in this document. + +

Quick Quiz 7: +But how does the upgrade-to-write operation exclude other readers? +
Answer + +

+This guarantee allows lookup code to be shared between read-side +and update-side code, and was premeditated, appearing in the earliest +DYNIX/ptx RCU documentation. + +

Fundamental Non-Requirements

+ +

+RCU provides extremely lightweight readers, and its read-side guarantees, +though quite useful, are correspondingly lightweight. +It is therefore all too easy to assume that RCU is guaranteeing more +than it really is. +Of course, the list of things that RCU does not guarantee is infinitely +long, however, the following sections list a few non-guarantees that +have caused confusion. +Except where otherwise noted, these non-guarantees were premeditated. + +

    +
  1. + Readers Impose Minimal Ordering +
  2. + Readers Do Not Exclude Updaters +
  3. + Updaters Only Wait For Old Readers +
  4. + Grace Periods Don't Partition Read-Side Critical Sections +
  5. + Read-Side Critical Sections Don't Partition Grace Periods +
  6. + Disabling Preemption Does Not Block Grace Periods +
+ +

Readers Impose Minimal Ordering

+ +

+Reader-side markers such as rcu_read_lock() and +rcu_read_unlock() provide absolutely no ordering guarantees +except through their interaction with the grace-period APIs such as +synchronize_rcu(). +To see this, consider the following pair of threads: + +

+
+ 1 void thread0(void)
+ 2 {
+ 3   rcu_read_lock();
+ 4   WRITE_ONCE(x, 1);
+ 5   rcu_read_unlock();
+ 6   rcu_read_lock();
+ 7   WRITE_ONCE(y, 1);
+ 8   rcu_read_unlock();
+ 9 }
+10
+11 void thread1(void)
+12 {
+13   rcu_read_lock();
+14   r1 = READ_ONCE(y);
+15   rcu_read_unlock();
+16   rcu_read_lock();
+17   r2 = READ_ONCE(x);
+18   rcu_read_unlock();
+19 }
+
+
+ +

+After thread0() and thread1() execute +concurrently, it is quite possible to have + +

+
+(r1 == 1 && r2 == 0)
+
+
+ +(that is, y appears to have been assigned before x), +which would not be possible if rcu_read_lock() and +rcu_read_unlock() had much in the way of ordering +properties. +But they do not, so the CPU is within its rights +to do significant reordering. +This is by design: Any significant ordering constraints would slow down +these fast-path APIs. + +

Quick Quiz 8: +Can't the compiler also reorder this code? +
Answer + +

Readers Do Not Exclude Updaters

+ +

+Neither rcu_read_lock() nor rcu_read_unlock() +exclude updates. +All they do is to prevent grace periods from ending. +The following example illustrates this: + +

+
+ 1 void thread0(void)
+ 2 {
+ 3   rcu_read_lock();
+ 4   r1 = READ_ONCE(y);
+ 5   if (r1) {
+ 6     do_something_with_nonzero_x();
+ 7     r2 = READ_ONCE(x);
+ 8     WARN_ON(!r2); /* BUG!!! */
+ 9   }
+10   rcu_read_unlock();
+11 }
+12
+13 void thread1(void)
+14 {
+15   spin_lock(&my_lock);
+16   WRITE_ONCE(x, 1);
+17   WRITE_ONCE(y, 1);
+18   spin_unlock(&my_lock);
+19 }
+
+
+ +

+If the thread0() function's rcu_read_lock() +excluded the thread1() function's update, +the WARN_ON() could never fire. +But the fact is that rcu_read_lock() does not exclude +much of anything aside from subsequent grace periods, of which +thread1() has none, so the +WARN_ON() can and does fire. + +

Updaters Only Wait For Old Readers

+ +

+It might be tempting to assume that after synchronize_rcu() +completes, there are no readers executing. +This temptation must be avoided because +new readers can start immediately after synchronize_rcu() +starts, and synchronize_rcu() is under no +obligation to wait for these new readers. + +

Quick Quiz 9: +Suppose that synchronize_rcu() did wait until all readers had completed. +Would the updater be able to rely on this? +
Answer + +

+Grace Periods Don't Partition Read-Side Critical Sections

+ +

+It is tempting to assume that if any part of one RCU read-side critical +section precedes a given grace period, and if any part of another RCU +read-side critical section follows that same grace period, then all of +the first RCU read-side critical section must precede all of the second. +However, this just isn't the case: A single grace period does not +partition the set of RCU read-side critical sections. +An example of this situation can be illustrated as follows, where +x, y, and z are initially all zero: + +

+
+ 1 void thread0(void)
+ 2 {
+ 3   rcu_read_lock();
+ 4   WRITE_ONCE(a, 1);
+ 5   WRITE_ONCE(b, 1);
+ 6   rcu_read_unlock();
+ 7 }
+ 8
+ 9 void thread1(void)
+10 {
+11   r1 = READ_ONCE(a);
+12   synchronize_rcu();
+13   WRITE_ONCE(c, 1);
+14 }
+15
+16 void thread2(void)
+17 {
+18   rcu_read_lock();
+19   r2 = READ_ONCE(b);
+20   r3 = READ_ONCE(c);
+21   rcu_read_unlock();
+22 }
+
+
+ +

+It turns out that the outcome: + +

+
+(r1 == 1 && r2 == 0 && r3 == 1)
+
+
+ +is entirely possible. +The following figure show how this can happen, with each circled +QS indicating the point at which RCU recorded a +quiescent state for each thread, that is, a state in which +RCU knows that the thread cannot be in the midst of an RCU read-side +critical section that started before the current grace period: + +

GPpartitionReaders1.svg

+ +

+If it is necessary to partition RCU read-side critical sections in this +manner, it is necessary to use two grace periods, where the first +grace period is known to end before the second grace period starts: + +

+
+ 1 void thread0(void)
+ 2 {
+ 3   rcu_read_lock();
+ 4   WRITE_ONCE(a, 1);
+ 5   WRITE_ONCE(b, 1);
+ 6   rcu_read_unlock();
+ 7 }
+ 8
+ 9 void thread1(void)
+10 {
+11   r1 = READ_ONCE(a);
+12   synchronize_rcu();
+13   WRITE_ONCE(c, 1);
+14 }
+15
+16 void thread2(void)
+17 {
+18   r2 = READ_ONCE(c);
+19   synchronize_rcu();
+20   WRITE_ONCE(d, 1);
+21 }
+22
+23 void thread3(void)
+24 {
+25   rcu_read_lock();
+26   r3 = READ_ONCE(b);
+27   r4 = READ_ONCE(d);
+28   rcu_read_unlock();
+29 }
+
+
+ +

+Here, if (r1 == 1), then +thread0()'s write to b must happen +before the end of thread1()'s grace period. +If in addition (r4 == 1), then +thread3()'s read from b must happen +after the beginning of thread2()'s grace period. +If it is also the case that (r2 == 1), then the +end of thread1()'s grace period must precede the +beginning of thread2()'s grace period. +This mean that the two RCU read-side critical sections cannot overlap, +guaranteeing that (r3 == 1). +As a result, the outcome: + +

+
+(r1 == 1 && r2 == 1 && r3 == 0 && r4 == 1)
+
+
+ +cannot happen. + +

+This non-requirement was also non-premeditated, but became apparent +when studying RCU's interaction with memory ordering. + +

+Read-Side Critical Sections Don't Partition Grace Periods

+ +

+It is also tempting to assume that if an RCU read-side critical section +happens between a pair of grace periods, then those grace periods cannot +overlap. +However, this temptation leads nowhere good, as can be illustrated by +the following, with all variables initially zero: + +

+
+ 1 void thread0(void)
+ 2 {
+ 3   rcu_read_lock();
+ 4   WRITE_ONCE(a, 1);
+ 5   WRITE_ONCE(b, 1);
+ 6   rcu_read_unlock();
+ 7 }
+ 8
+ 9 void thread1(void)
+10 {
+11   r1 = READ_ONCE(a);
+12   synchronize_rcu();
+13   WRITE_ONCE(c, 1);
+14 }
+15
+16 void thread2(void)
+17 {
+18   rcu_read_lock();
+19   WRITE_ONCE(d, 1);
+20   r2 = READ_ONCE(c);
+21   rcu_read_unlock();
+22 }
+23
+24 void thread3(void)
+25 {
+26   r3 = READ_ONCE(d);
+27   synchronize_rcu();
+28   WRITE_ONCE(e, 1);
+29 }
+30
+31 void thread4(void)
+32 {
+33   rcu_read_lock();
+34   r4 = READ_ONCE(b);
+35   r5 = READ_ONCE(e);
+36   rcu_read_unlock();
+37 }
+
+
+ +

+In this case, the outcome: + +

+
+(r1 == 1 && r2 == 1 && r3 == 1 && r4 == 0 && r5 == 1)
+
+
+ +is entirely possible, as illustrated below: + +

ReadersPartitionGP1.svg

+ +

+Again, an RCU read-side critical section can overlap almost all of a +given grace period, just so long as it does not overlap the entire +grace period. +As a result, an RCU read-side critical section cannot partition a pair +of RCU grace periods. + +

Quick Quiz 10: +How long a sequence of grace periods, each separated by an RCU read-side +critical section, would be required to partition the RCU read-side +critical sections at the beginning and end of the chain? +
Answer + +

+Disabling Preemption Does Not Block Grace Periods

+ +

+There was a time when disabling preemption on any given CPU would block +subsequent grace periods. +However, this was an accident of implementation and is not a requirement. +And in the current Linux-kernel implementation, disabling preemption +on a given CPU in fact does not block grace periods, as Oleg Nesterov +demonstrated. + +

+If you need a preempt-disable region to block grace periods, you need to add +rcu_read_lock() and rcu_read_unlock(), for example +as follows: + +

+
+ 1 preempt_disable();
+ 2 rcu_read_lock();
+ 3 do_something();
+ 4 rcu_read_unlock();
+ 5 preempt_enable();
+ 6
+ 7 /* Spinlocks implicitly disable preemption. */
+ 8 spin_lock(&mylock);
+ 9 rcu_read_lock();
+10 do_something();
+11 rcu_read_unlock();
+12 spin_unlock(&mylock);
+
+
+ +

+In theory, you could enter the RCU read-side critical section first, +but it is more efficient to keep the entire RCU read-side critical +section contained in the preempt-disable region as shown above. +Of course, RCU read-side critical sections that extend outside of +preempt-disable regions will work correctly, but such critical sections +can be preempted, which forces rcu_read_unlock() to do +more work. +And no, this is not an invitation to enclose all of your RCU +read-side critical sections within preempt-disable regions, because +doing so would degrade real-time response. + +

+This non-requirement appeared with preemptible RCU. +If you need a grace period that waits on non-preemptible code regions, use +RCU-sched. + +

Parallelism Facts of Life

+ +

+These parallelism facts of life are by no means specific to RCU, but +the RCU implementation must abide by them. +They therefore bear repeating: + +

    +
  1. Any CPU or task may be delayed at any time, + and any attempts to avoid these delays by disabling + preemption, interrupts, or whatever are completely futile. + This is most obvious in preemptible user-level + environments and in virtualized environments (where + a given guest OS's VCPUs can be preempted at any time by + the underlying hypervisor), but can also happen in bare-metal + environments due to ECC errors, NMIs, and other hardware + events. + Although a delay of more than about 20 seconds can result + in splats, the RCU implementation is obligated to use + algorithms that can tolerate extremely long delays, but where + “extremely long” is not long enough to allow + wrap-around when incrementing a 64-bit counter. +
  2. Both the compiler and the CPU can reorder memory accesses. + Where it matters, RCU must use compiler directives and + memory-barrier instructions to preserve ordering. +
  3. Conflicting writes to memory locations in any given cache line + will result in expensive cache misses. + Greater numbers of concurrent writes and more-frequent + concurrent writes will result in more dramatic slowdowns. + RCU is therefore obligated to use algorithms that have + sufficient locality to avoid significant performance and + scalability problems. +
  4. As a rough rule of thumb, only one CPU's worth of processing + may be carried out under the protection of any given exclusive + lock. + RCU must therefore use scalable locking designs. +
  5. Counters are finite, especially on 32-bit systems. + RCU's use of counters must therefore tolerate counter wrap, + or be designed such that counter wrap would take way more + time than a single system is likely to run. + An uptime of ten years is quite possible, a runtime + of a century much less so. + As an example of the latter, RCU's dyntick-idle nesting counter + allows 54 bits for interrupt nesting level (this counter + is 64 bits even on a 32-bit system). + Overflowing this counter requires 254 + half-interrupts on a given CPU without that CPU ever going idle. + If a half-interrupt happened every microsecond, it would take + 570 years of runtime to overflow this counter, which is currently + believed to be an acceptably long time. +
  6. Linux systems can have thousands of CPUs running a single + Linux kernel in a single shared-memory environment. + RCU must therefore pay close attention to high-end scalability. +
+ +

+This last parallelism fact of life means that RCU must pay special +attention to the preceding facts of life. +The idea that Linux might scale to systems with thousands of CPUs would +have been met with some skepticism in the 1990s, but these requirements +would have otherwise have been unsurprising, even in the early 1990s. + +

Quality-of-Implementation Requirements

+ +

+These sections list quality-of-implementation requirements. +Although an RCU implementation that ignores these requirements could +still be used, it would likely be subject to limitations that would +make it inappropriate for industrial-strength production use. +Classes of quality-of-implementation requirements are as follows: + +

    +
  1. Specialization +
  2. Performance and Scalability +
  3. Composability +
  4. Corner Cases +
+ +

+These classes is covered in the following sections. + +

Specialization

+ +

+RCU is and always has been intended primarily for read-mostly situations, as +illustrated by the following figure. +This means that RCU's read-side primitives are optimized, often at the +expense of its update-side primitives. + +

RCUApplicability.svg

+ +

+This focus on read-mostly situations means that RCU must interoperate +with other synchronization primitives. +For example, the add_gp() and remove_gp_synchronous() +examples discussed earlier use RCU to protect readers and locking to +coordinate updaters. +However, the need extends much farther, requiring that a variety of +synchronization primitives be legal within RCU read-side critical sections, +including spinlocks, sequence locks, atomic operations, reference +counters, and memory barriers. + +

Quick Quiz 11: +What about sleeping locks? +
Answer + +

+It often comes as a surprise that many algorithms do not require a +consistent view of data, but many can function in that mode, +with network routing being the poster child. +Internet routing algorithms take significant time to propagate +updates, so that by the time an update arrives at a given system, +that system has been sending network traffic the wrong way for +a considerable length of time. +Having a few threads continue to send traffic the wrong way for a +few more milliseconds is clearly not a problem: In the worst case, +TCP retransmissions will eventually get the data where it needs to go. +In general, when tracking the state of the universe outside of the +computer, some level of inconsistency must be tolerated due to +speed-of-light delays if nothing else. + +

+Furthermore, uncertainty about external state is inherent in many cases. +For example, a pair of veternarians might use heartbeat to determine +whether or not a given cat was alive. +But how long should they wait after the last heartbeat to decide that +the cat is in fact dead? +Waiting less than 400 milliseconds makes no sense because this would +mean that a relaxed cat would be considered to cycle between death +and life more than 100 times per minute. +Moreover, just as with human beings, a cat's heart might stop for +some period of time, so the exact wait period is a judgment call. +One of our pair of veternarians might wait 30 seconds before pronouncing +the cat dead, while the other might insist on waiting a full minute. +The two veternarians would then disagree on the state of the cat during +the final 30 seconds of the minute following the last heartbeat, as +fancifully illustrated below: + +

2013-08-is-it-dead.png

+ +

+Interestingly enough, this same situation applies to hardware. +When push comes to shove, how do we tell whether or not some +external server has failed? +We send messages to it periodically, and declare it failed if we +don't receive a response within a given period of time. +Policy decisions can usually tolerate short +periods of inconsistency. +The policy was decided some time ago, and is only now being put into +effect, so a few milliseconds of delay is normally inconsequential. + +

+However, there are algorithms that absolutely must see consistent data. +For example, the translation between a user-level SystemV semaphore +ID to the corresponding in-kernel data structure is protected by RCU, +but it is absolutely forbidden to update a semaphore that has just been +removed. +In the Linux kernel, this need for consistency is accommodated by acquiring +spinlocks located in the in-kernel data structure from within +the RCU read-side critical section, and this is indicated by the +green box in the figure above. +Many other techniques may be used, and are in fact used within the +Linux kernel. + +

+In short, RCU is not required to maintain consistency, and other +mechanisms may be used in concert with RCU when consistency is required. +RCU's specialization allows it to do its job extremely well, and its +ability to interoperate with other synchronization mechanisms allows +the right mix of synchronization tools to be used for a given job. + +

Performance and Scalability

+ +

+Energy efficiency is a critical component of performance today, +and Linux-kernel RCU implementations must therefore avoid unnecessarily +awakening idle CPUs. +I cannot claim that this requirement was premeditated. +In fact, I learned of it during a telephone conversation in which I +was given “frank and open” feedback on the importance +of energy efficiency in battery-powered systems and on specific +energy-efficiency shortcomings of the Linux-kernel RCU implementation. +In my experience, the battery-powered embedded community will consider +any unnecessary wakeups to be extremely unfriendly acts. +So much so that mere Linux-kernel-mailing-list posts are +insufficient to vent their ire. + +

+Memory consumption is not particularly important for in most +situations, and has become decreasingly +so as memory sizes have expanded and memory +costs have plummeted. +However, as I learned from Matt Mackall's +bloatwatch +efforts, memory footprint is critically important on single-CPU systems with +non-preemptible (CONFIG_PREEMPT=n) kernels, and thus +tiny RCU +was born. +Josh Triplett has since taken over the small-memory banner with his +Linux kernel tinification +project, which resulted in +SRCU +becoming optional for those kernels not needing it. + +

+The remaining performance requirements are, for the most part, +unsurprising. +For example, in keeping with RCU's read-side specialization, +rcu_dereference() should have negligible overhead (for +example, suppression of a few minor compiler optimizations). +Similarly, in non-preemptible environments, rcu_read_lock() and +rcu_read_unlock() should have exactly zero overhead. + +

+In preemptible environments, in the case where the RCU read-side +critical section was not preempted (as will be the case for the +highest-priority real-time process), rcu_read_lock() and +rcu_read_unlock() should have minimal overhead. +In particular, they should not contain atomic read-modify-write +operations, memory-barrier instructions, preemption disabling, +interrupt disabling, or backwards branches. +However, in the case where the RCU read-side critical section was preempted, +rcu_read_unlock() may acquire spinlocks and disable interrupts. +This is why it is better to nest an RCU read-side critical section +within a preempt-disable region than vice versa, at least in cases +where that critical section is short enough to avoid unduly degrading +real-time latencies. + +

+The synchronize_rcu() grace-period-wait primitive is +optimized for throughput. +It may therefore incur several milliseconds of latency in addition to +the duration of the longest RCU read-side critical section. +On the other hand, multiple concurrent invocations of +synchronize_rcu() are required to use batching optimizations +so that they can be satisfied by a single underlying grace-period-wait +operation. +For example, in the Linux kernel, it is not unusual for a single +grace-period-wait operation to serve more than +1,000 separate invocations +of synchronize_rcu(), thus amortizing the per-invocation +overhead down to nearly zero. +However, the grace-period optimization is also required to avoid +measurable degradation of real-time scheduling and interrupt latencies. + +

+In some cases, the multi-millisecond synchronize_rcu() +latencies are unacceptable. +In these cases, synchronize_rcu_expedited() may be used +instead, reducing the grace-period latency down to a few tens of +microseconds on small systems, at least in cases where the RCU read-side +critical sections are short. +There are currently no special latency requirements for +synchronize_rcu_expedited() on large systems, but, +consistent with the empirical nature of the RCU specification, +that is subject to change. +However, there most definitely are scalability requirements: +A storm of synchronize_rcu_expedited() invocations on 4096 +CPUs should at least make reasonable forward progress. +In return for its shorter latencies, synchronize_rcu_expedited() +is permitted to impose modest degradation of real-time latency +on non-idle online CPUs. +That said, it will likely be necessary to take further steps to reduce this +degradation, hopefully to roughly that of a scheduling-clock interrupt. + +

+There are a number of situations where even +synchronize_rcu_expedited()'s reduced grace-period +latency is unacceptable. +In these situations, the asynchronous call_rcu() can be +used in place of synchronize_rcu() as follows: + +

+
+ 1 struct foo {
+ 2   int a;
+ 3   int b;
+ 4   struct rcu_head rh;
+ 5 };
+ 6
+ 7 static void remove_gp_cb(struct rcu_head *rhp)
+ 8 {
+ 9   struct foo *p = container_of(rhp, struct foo, rh);
+10
+11   kfree(p);
+12 }
+13
+14 bool remove_gp_asynchronous(void)
+15 {
+16   struct foo *p;
+17
+18   spin_lock(&gp_lock);
+19   p = rcu_dereference(gp);
+20   if (!p) {
+21     spin_unlock(&gp_lock);
+22     return false;
+23   }
+24   rcu_assign_pointer(gp, NULL);
+25   call_rcu(&p->rh, remove_gp_cb);
+26   spin_unlock(&gp_lock);
+27   return true;
+28 }
+
+
+ +

+A definition of struct foo is finally needed, and appears +on lines 1-5. +The function remove_gp_cb() is passed to call_rcu() +on line 25, and will be invoked after the end of a subsequent +grace period. +This gets the same effect as remove_gp_synchronous(), +but without forcing the updater to wait for a grace period to elapse. +The call_rcu() function may be used in a number of +situations where neither synchronize_rcu() nor +synchronize_rcu_expedited() would be legal, +including within preempt-disable code, local_bh_disable() code, +interrupt-disable code, and interrupt handlers. +However, even call_rcu() is illegal within NMI handlers. +The callback function (remove_gp_cb() in this case) will be +executed within softirq (software interrupt) environment within the +Linux kernel, +either within a real softirq handler or under the protection +of local_bh_disable(). +In both the Linux kernel and in userspace, it is bad practice to +write an RCU callback function that takes too long. +Long-running operations should be relegated to separate threads or +(in the Linux kernel) workqueues. + +

Quick Quiz 12: +Why does line 19 use rcu_access_pointer()? +After all, call_rcu() on line 25 stores into the +structure, which would interact badly with concurrent insertions. +Doesn't this mean that rcu_dereference() is required? +
Answer + +

+However, all that remove_gp_cb() is doing is +invoking kfree() on the data element. +This is a common idiom, and is supported by kfree_rcu(), +which allows “fire and forget” operation as shown below: + +

+
+ 1 struct foo {
+ 2   int a;
+ 3   int b;
+ 4   struct rcu_head rh;
+ 5 };
+ 6
+ 7 bool remove_gp_faf(void)
+ 8 {
+ 9   struct foo *p;
+10
+11   spin_lock(&gp_lock);
+12   p = rcu_dereference(gp);
+13   if (!p) {
+14     spin_unlock(&gp_lock);
+15     return false;
+16   }
+17   rcu_assign_pointer(gp, NULL);
+18   kfree_rcu(p, rh);
+19   spin_unlock(&gp_lock);
+20   return true;
+21 }
+
+
+ +

+Note that remove_gp_faf() simply invokes +kfree_rcu() and proceeds, without any need to pay any +further attention to the subsequent grace period and kfree(). +It is permissible to invoke kfree_rcu() from the same +environments as for call_rcu(). +Interestingly enough, DYNIX/ptx had the equivalents of +call_rcu() and kfree_rcu(), but not +synchronize_rcu(). +This was due to the fact that RCU was not heavily used within DYNIX/ptx, +so the very few places that needed something like +synchronize_rcu() simply open-coded it. + +

Quick Quiz 13: +Earlier it was claimed that call_rcu() and +kfree_rcu() allowed updaters to avoid being blocked +by readers. +But how can that be correct, given that the invocation of the callback +and the freeing of the memory (respectively) must still wait for +a grace period to elapse? +
Answer + +

+But what if the updater must wait for the completion of code to be +executed after the end of the grace period, but has other tasks +that can be carried out in the meantime? +The polling-style get_state_synchronize_rcu() and +cond_synchronize_rcu() functions may be used for this +purpose, as shown below: + +

+
+ 1 bool remove_gp_poll(void)
+ 2 {
+ 3   struct foo *p;
+ 4   unsigned long s;
+ 5
+ 6   spin_lock(&gp_lock);
+ 7   p = rcu_access_pointer(gp);
+ 8   if (!p) {
+ 9     spin_unlock(&gp_lock);
+10     return false;
+11   }
+12   rcu_assign_pointer(gp, NULL);
+13   spin_unlock(&gp_lock);
+14   s = get_state_synchronize_rcu();
+15   do_something_while_waiting();
+16   cond_synchronize_rcu(s);
+17   kfree(p);
+18   return true;
+19 }
+
+
+ +

+On line 14, get_state_synchronize_rcu() obtains a +“cookie” from RCU, +then line 15 carries out other tasks, +and finally, line 16 returns immediately if a grace period has +elapsed in the meantime, but otherwise waits as required. +The need for get_state_synchronize_rcu and +cond_synchronize_rcu() has appeared quite recently, +so it is too early to tell whether they will stand the test of time. + +

+RCU thus provides a range of tools to allow updaters to strike the +required tradeoff between latency, flexibility and CPU overhead. + +

Composability

+ +

+Composability has received much attention in recent years, perhaps in part +due to the collision of multicore hardware with object-oriented techniques +designed in single-threaded environments for single-threaded use. +And in theory, RCU read-side critical sections may be composed, and in +fact may be nested arbitrarily deeply. +In practice, as with all real-world implementations of composable +constructs, there are limitations. + +

+Implementations of RCU for which rcu_read_lock() +and rcu_read_unlock() generate no code, such as +Linux-kernel RCU when CONFIG_PREEMPT=n, can be +nested arbitrarily deeply. +After all, there is no overhead. +Except that if all these instances of rcu_read_lock() +and rcu_read_unlock() are visible to the compiler, +compilation will eventually fail due to exhausting memory, +mass storage, or user patience, whichever comes first. +If the nesting is not visible to the compiler, as is the case with +mutually recursive functions each in its own translation unit, +stack overflow will result. +If the nesting takes the form of loops, either the control variable +will overflow or (in the Linux kernel) you will get an RCU CPU stall warning. +Nevertheless, this class of RCU implementations is one +of the most composable constructs in existence. + +

+RCU implementations that explicitly track nesting depth +are limited by the nesting-depth counter. +For example, the Linux kernel's preemptible RCU limits nesting to +INT_MAX. +This should suffice for almost all practical purposes. +That said, a consecutive pair of RCU read-side critical sections +between which there is an operation that waits for a grace period +cannot be enclosed in another RCU read-side critical section. +This is because it is not legal to wait for a grace period within +an RCU read-side critical section: To do so would result either +in deadlock or +in RCU implicitly splitting the enclosing RCU read-side critical +section, neither of which is conducive to a long-lived and prosperous +kernel. + +

+In short, although RCU read-side critical sections are highly composable, +care is required in some situations, just as is the case for any other +composable synchronization mechanism. + +

Corner Cases

+ +

+A given RCU workload might have an endless and intense stream of +RCU read-side critical sections, perhaps even so intense that there +was never a point in time during which there was not at least one +RCU read-side critical section in flight. +RCU cannot allow this situation to block grace periods: As long as +all the RCU read-side critical sections are finite, grace periods +must also be finite. + +

+That said, preemptible RCU implementations could potentially result +in RCU read-side critical sections being preempted for long durations, +which has the effect of creating a long-duration RCU read-side +critical section. +This situation can arise only in heavily loaded systems, but systems using +real-time priorities are of course more vulnerable. +Therefore, RCU priority boosting is provided to help deal with this +case. +That said, the exact requirements on RCU priority boosting will likely +evolve as more experience accumulates. + +

+Other workloads might have very high update rates. +Although one can argue that such workloads should instead use +something other than RCU, the fact remains that RCU must +handle such workloads gracefully. +This requirement is another factor driving batching of grace periods, +but it is also the driving force behind the checks for large numbers +of queued RCU callbacks in the call_rcu() code path. +Finally, high update rates should not delay RCU read-side critical +sections, although some read-side delays can occur when using +synchronize_rcu_expedited(), courtesy of this function's use +of try_stop_cpus(). +(In the future, synchronize_rcu_expedited() will be +converted to use lighter-weight inter-processor interrupts (IPIs), +but this will still disturb readers, though to a much smaller degree.) + +

+Although all three of these corner cases were understood in the early +1990s, a simple user-level test consisting of close(open(path)) +in a tight loop +in the early 2000s suddenly provided a much deeper appreciation of the +high-update-rate corner case. +This test also motivated addition of some RCU code to react to high update +rates, for example, if a given CPU finds itself with more than 10,000 +RCU callbacks queued, it will cause RCU to take evasive action by +more aggressively starting grace periods and more aggressively forcing +completion of grace-period processing. +This evasive action causes the grace period to complete more quickly, +but at the cost of restricting RCU's batching optimizations, thus +increasing the CPU overhead incurred by that grace period. + +

+Software-Engineering Requirements

+ +

+Between Murphy's Law and “To err is human”, it is necessary to +guard against mishaps and misuse: + +

    +
  1. It is all too easy to forget to use rcu_read_lock() + everywhere that it is needed, so kernels built with + CONFIG_PROVE_RCU=y will spat if + rcu_dereference() is used outside of an + RCU read-side critical section. + Update-side code can use rcu_dereference_protected(), + which takes a + lockdep expression + to indicate what is providing the protection. + If the indicated protection is not provided, a lockdep splat + is emitted. + +

    + Code shared between readers and updaters can use + rcu_dereference_check(), which also takes a + lockdep expression, and emits a lockdep splat if neither + rcu_read_lock() nor the indicated protection + is in place. + In addition, rcu_dereference_raw() is used in those + (hopefully rare) cases where the required protection cannot + be easily described. + Finally, rcu_read_lock_held() is provided to + allow a function to verify that it has been invoked within + an RCU read-side critical section. + I was made aware of this set of requirements shortly after Thomas + Gleixner audited a number of RCU uses. +

  2. A given function might wish to check for RCU-related preconditions + upon entry, before using any other RCU API. + The rcu_lockdep_assert() does this job, + asserting the expression in kernels having lockdep enabled + and doing nothing otherwise. +
  3. It is also easy to forget to use rcu_assign_pointer() + and rcu_dereference(), perhaps (incorrectly) + substituting a simple assignment. + To catch this sort of error, a given RCU-protected pointer may be + tagged with __rcu, after which running sparse + with CONFIG_SPARSE_RCU_POINTER=y will complain + about simple-assignment accesses to that pointer. + Arnd Bergmann made me aware of this requirement, and also + supplied the needed + patch series. +
  4. Kernels built with CONFIG_DEBUG_OBJECTS_RCU_HEAD=y + will splat if a data element is passed to call_rcu() + twice in a row, without a grace period in between. + (This error is similar to a double free.) + The corresponding rcu_head structures that are + dynamically allocated are automatically tracked, but + rcu_head structures allocated on the stack + must be initialized with init_rcu_head_on_stack() + and cleaned up with destroy_rcu_head_on_stack(). + Similarly, statically allocated non-stack rcu_head + structures must be initialized with init_rcu_head() + and cleaned up with destroy_rcu_head(). + Mathieu Desnoyers made me aware of this requirement, and also + supplied the needed + patch. +
  5. An infinite loop in an RCU read-side critical section will + eventually trigger an RCU CPU stall warning splat. + However, RCU is not obligated to produce this splat + unless there is a grace period waiting on that particular + RCU read-side critical section. + This requirement made itself known in the early 1990s, pretty + much the first time that it was necessary to debug a CPU stall. +
  6. Although it would be very good to detect pointers leaking out + of RCU read-side critical sections, there is currently no + good way of doing this. + One complication is the need to distinguish between pointers + leaking and pointers that have been handed off from RCU to + some other synchronization mechanism, for example, reference + counting. +
  7. In kernels built with CONFIG_RCU_TRACE=y, RCU-related + information is provided via both debugfs and event tracing. +
  8. Open-coded use of rcu_assign_pointer() and + rcu_dereference() to create typical linked + data structures can be surprisingly error-prone. + Therefore, RCU-protected + linked lists + and, more recently, RCU-protected + hash tables + are available. + Many other special-purpose RCU-protected data structures are + available in the Linux kernel and the userspace RCU library. +
  9. Some linked structures are created at compile time, but still + require __rcu checking. + The RCU_POINTER_INITIALIZER() macro serves this + purpose. +
  10. It is not necessary to use rcu_assign_pointer() + when creating linked structures that are to be published via + a single external pointer. + The RCU_INIT_POINTER() macro is provided for + this task and also for assigning NULL pointers + at runtime. +
+ +

+This not a hard-and-fast list: RCU's diagnostic capabilities will +continue to be guided by the number and type of usage bugs found +in real-world RCU usage. + +

Linux Kernel Complications

+ +

+The Linux kernel provides an interesting environment for all kinds of +software, including RCU. +Some of the relevant points of interest are as follows: + +

    +
  1. Configuration. +
  2. Firmware Interface. +
  3. Early Boot. +
  4. + Interrupts and non-maskable interrupts (NMIs). +
  5. Loadable Modules. +
  6. Hotplug CPU. +
  7. Scheduler and RCU. +
  8. Tracing and RCU. +
  9. Energy Efficiency. +
  10. + Performance, Scalability, Response Time, and Reliability. +
+ +

+This list is probably incomplete, but it does give a feel for the +most notable Linux-kernel complications. +Each of the following sections covers one of the above topics. + +

Configuration

+ +

+RCU's goal is automatic configuration, so that almost nobody +needs to worry about RCU's Kconfig options. +And for almost all users, RCU does in fact work well +“out of the box.” + +

+However, there are specialized use cases that are handled by +kernel boot parameters and Kconfig options. +Unfortunately, the Kconfig system will explicitly ask users +about new Kconfig options, which requires almost all of them +be hidden behind a CONFIG_RCU_EXPERT Kconfig option. + +

+This all should be quite obvious, but the fact remains that +Linus Torvalds recently had to +remind +me of this requirement. + +

Firmware Interface

+ +

+In many cases, kernel obtains information about the system from the +firmware, and sometimes things are lost in translation. +Or the translation is accurate, but the original message is bogus. + +

+For example, some systems' firmware overreports the number of CPUs, +sometimes by a large factor. +If RCU naively believed the firmware, as it used to do, +it would create too many per-CPU kthreads. +Although the resulting system will still run correctly, the extra +kthreads needlessly consume memory and can cause confusion +when they show up in ps listings. + +

+RCU must therefore wait for a given CPU to actually come online before +it can allow itself to believe that the CPU actually exists. +The resulting “ghost CPUs” (which are never going to +come online) cause a number of +interesting complications. + +

Early Boot

+ +

+The Linux kernel's boot sequence is an interesting process, +and RCU is used early, even before rcu_init() +is invoked. +In fact, a number of RCU's primitives can be used as soon as the +initial task's task_struct is available and the +boot CPU's per-CPU variables are set up. +The read-side primitives (rcu_read_lock(), +rcu_read_unlock(), rcu_dereference(), +and rcu_access_pointer()) will operate normally very early on, +as will rcu_assign_pointer(). + +

+Although call_rcu() may be invoked at any +time during boot, callbacks are not guaranteed to be invoked until after +the scheduler is fully up and running. +This delay in callback invocation is due to the fact that RCU does not +invoke callbacks until it is fully initialized, and this full initialization +cannot occur until after the scheduler has initialized itself to the +point where RCU can spawn and run its kthreads. +In theory, it would be possible to invoke callbacks earlier, +however, this is not a panacea because there would be severe restrictions +on what operations those callbacks could invoke. + +

+Perhaps surprisingly, synchronize_rcu(), +synchronize_rcu_bh() +(discussed below), +and +synchronize_sched() +will all operate normally +during very early boot, the reason being that there is only one CPU +and preemption is disabled. +This means that the call synchronize_rcu() (or friends) +itself is a quiescent +state and thus a grace period, so the early-boot implementation can +be a no-op. + +

+Both synchronize_rcu_bh() and synchronize_sched() +continue to operate normally through the remainder of boot, courtesy +of the fact that preemption is disabled across their RCU read-side +critical sections and also courtesy of the fact that there is still +only one CPU. +However, once the scheduler starts initializing, preemption is enabled. +There is still only a single CPU, but the fact that preemption is enabled +means that the no-op implementation of synchronize_rcu() no +longer works in CONFIG_PREEMPT=y kernels. +Therefore, as soon as the scheduler starts initializing, the early-boot +fastpath is disabled. +This means that synchronize_rcu() switches to its runtime +mode of operation where it posts callbacks, which in turn means that +any call to synchronize_rcu() will block until the corresponding +callback is invoked. +Unfortunately, the callback cannot be invoked until RCU's runtime +grace-period machinery is up and running, which cannot happen until +the scheduler has initialized itself sufficiently to allow RCU's +kthreads to be spawned. +Therefore, invoking synchronize_rcu() during scheduler +initialization can result in deadlock. + +

Quick Quiz 14: +So what happens with synchronize_rcu() during +scheduler initialization for CONFIG_PREEMPT=n +kernels? +
Answer + +

+I learned of these boot-time requirements as a result of a series of +system hangs. + +

Interrupts and NMIs

+ +

+The Linux kernel has interrupts, and RCU read-side critical sections are +legal within interrupt handlers and within interrupt-disabled regions +of code, as are invocations of call_rcu(). + +

+Some Linux-kernel architectures can enter an interrupt handler from +non-idle process context, and then just never leave it, instead stealthily +transitioning back to process context. +This trick is sometimes used to invoke system calls from inside the kernel. +These “half-interrupts” mean that RCU has to be very careful +about how it counts interrupt nesting levels. +I learned of this requirement the hard way during a rewrite +of RCU's dyntick-idle code. + +

+The Linux kernel has non-maskable interrupts (NMIs), and +RCU read-side critical sections are legal within NMI handlers. +Thankfully, RCU update-side primitives, including +call_rcu(), are prohibited within NMI handlers. + +

+The name notwithstanding, some Linux-kernel architectures +can have nested NMIs, which RCU must handle correctly. +Andy Lutomirski +surprised me +with this requirement; +he also kindly surprised me with +an algorithm +that meets this requirement. + +

Loadable Modules

+ +

+The Linux kernel has loadable modules, and these modules can +also be unloaded. +After a given module has been unloaded, any attempt to call +one of its functions results in a segmentation fault. +The module-unload functions must therefore cancel any +delayed calls to loadable-module functions, for example, +any outstanding mod_timer() must be dealt with +via del_timer_sync() or similar. + +

+Unfortunately, there is no way to cancel an RCU callback; +once you invoke call_rcu(), the callback function is +going to eventually be invoked, unless the system goes down first. +Because it is normally considered socially irresponsible to crash the system +in response to a module unload request, we need some other way +to deal with in-flight RCU callbacks. + +

+RCU therefore provides +rcu_barrier(), +which waits until all in-flight RCU callbacks have been invoked. +If a module uses call_rcu(), its exit function should therefore +prevent any future invocation of call_rcu(), then invoke +rcu_barrier(). +In theory, the underlying module-unload code could invoke +rcu_barrier() unconditionally, but in practice this would +incur unacceptable latencies. + +

+Nikita Danilov noted this requirement for an analogous filesystem-unmount +situation, and Dipankar Sarma incorporated rcu_barrier() into RCU. +The need for rcu_barrier() for module unloading became +apparent later. + +

Hotplug CPU

+ +

+The Linux kernel supports CPU hotplug, which means that CPUs +can come and go. +It is of course illegal to use any RCU API member from an offline CPU. +This requirement was present from day one in DYNIX/ptx, but +on the other hand, the Linux kernel's CPU-hotplug implementation +is “interesting.” + +

+The Linux-kernel CPU-hotplug implementation has notifiers that +are used to allow the various kernel subsystems (including RCU) +to respond appropriately to a given CPU-hotplug operation. +Most RCU operations may be invoked from CPU-hotplug notifiers, +including even normal synchronous grace-period operations +such as synchronize_rcu(). +However, expedited grace-period operations such as +synchronize_rcu_expedited() are not supported, +due to the fact that current implementations block CPU-hotplug +operations, which could result in deadlock. + +

+In addition, all-callback-wait operations such as +rcu_barrier() are also not supported, due to the +fact that there are phases of CPU-hotplug operations where +the outgoing CPU's callbacks will not be invoked until after +the CPU-hotplug operation ends, which could also result in deadlock. + +

Scheduler and RCU

+ +

+RCU depends on the scheduler, and the scheduler uses RCU to +protect some of its data structures. +This means the scheduler is forbidden from acquiring +the runqueue locks and the priority-inheritance locks +in the middle of an outermost RCU read-side critical section unless +it also releases them before exiting that same +RCU read-side critical section. +This same prohibition also applies to any lock that is acquired +while holding any lock to which this prohibition applies. +Violating this rule results in deadlock. + +

+For RCU's part, the preemptible-RCU rcu_read_unlock() +implementation must be written carefully to avoid similar deadlocks. +In particular, rcu_read_unlock() must tolerate an +interrupt where the interrupt handler invokes both +rcu_read_lock() and rcu_read_unlock(). +This possibility requires rcu_read_unlock() to use +negative nesting levels to avoid destructive recursion via +interrupt handler's use of RCU. + +

+This pair of mutual scheduler-RCU requirements came as a +complete surprise. + +

+As noted above, RCU makes use of kthreads, and it is necessary to +avoid excessive CPU-time accumulation by these kthreads. +This requirement was no surprise, but RCU's violation of it +when running context-switch-heavy workloads when built with +CONFIG_NO_HZ_FULL=y +did come as a surprise [PDF]. +RCU has made good progress towards meeting this requirement, even +for context-switch-have CONFIG_NO_HZ_FULL=y workloads, +but there is room for further improvement. + +

Tracing and RCU

+ +

+It is possible to use tracing on RCU code, but tracing itself +uses RCU. +For this reason, rcu_dereference_raw_notrace() +is provided for use by tracing, which avoids the destructive +recursion that could otherwise ensue. +This API is also used by virtualization in some architectures, +where RCU readers execute in environments in which tracing +cannot be used. +The tracing folks both located the requirement and provided the +needed fix, so this surprise requirement was relatively painless. + +

Energy Efficiency

+ +

+Interrupting idle CPUs is considered socially unacceptable, +especially by people with battery-powered embedded systems. +RCU therefore conserves energy by detecting which CPUs are +idle, including tracking CPUs that have been interrupted from idle. +This is a large part of the energy-efficiency requirement, +so I learned of this via an irate phone call. + +

+Because RCU avoids interrupting idle CPUs, it is illegal to +execute an RCU read-side critical section on an idle CPU. +(Kernels built with CONFIG_PROVE_RCU=y will splat +if you try it.) +The RCU_NONIDLE() macro and _rcuidle +event tracing is provided to work around this restriction. +In addition, rcu_is_watching() may be used to +test whether or not it is currently legal to run RCU read-side +critical sections on this CPU. +I learned of the need for diagnostics on the one hand +and RCU_NONIDLE() on the other while inspecting +idle-loop code. +Steven Rostedt supplied _rcuidle event tracing, +which is used quite heavily in the idle loop. + +

+It is similarly socially unacceptable to interrupt an +nohz_full CPU running in userspace. +RCU must therefore track nohz_full userspace +execution. +And in +CONFIG_NO_HZ_FULL_SYSIDLE=y +kernels, RCU must separately track idle CPUs on the one hand and +CPUs that are either idle or executing in userspace on the other. +In both cases, RCU must be able to sample state at two points in +time, and be able to determine whether or not some other CPU spent +any time idle and/or executing in userspace. + +

+These energy-efficiency requirements have proven quite difficult to +understand and to meet, for example, there have been more than five +clean-sheet rewrites of RCU's energy-efficiency code, the last of +which was finally able to demonstrate +real energy savings running on real hardware [PDF]. +As noted earlier, +I learned of many of these requirements via angry phone calls: +Flaming me on the Linux-kernel mailing list was apparently not +sufficient to fully vent their ire at RCU's energy-efficiency bugs! + +

+Performance, Scalability, Response Time, and Reliability

+ +

+Expanding on the +earlier discussion, +RCU is used heavily by hot code paths in performance-critical +portions of the Linux kernel's networking, security, virtualization, +and scheduling code paths. +RCU must therefore use efficient implementations, especially in its +read-side primitives. +To that end, it would be good if preemptible RCU's implementation +of rcu_read_lock() could be inlined, however, doing +this requires resolving #include issues with the +task_struct structure. + +

+The Linux kernel supports hardware configurations with up to +4096 CPUs, which means that RCU must be extremely scalable. +Algorithms that involve frequent acquisitions of global locks or +frequent atomic operations on global variables simply cannot be +tolerated within the RCU implementation. +RCU therefore makes heavy use of a combining tree based on the +rcu_node structure. +RCU is required to tolerate all CPUs continuously invoking any +combination of RCU's runtime primitives with minimal per-operation +overhead. +In fact, in many cases, increasing load must decrease the +per-operation overhead, witness the batching optimizations for +synchronize_rcu(), call_rcu(), +synchronize_rcu_expedited(), and rcu_barrier(). +As a general rule, RCU must cheerfully accept whatever the +rest of the Linux kernel decides to throw at it. + +

+The Linux kernel is used for real-time workloads, especially +in conjunction with the +-rt patchset. +The real-time-latency response requirements are such that the +traditional approach of disabling preemption across RCU +read-side critical sections is inappropriate. +Kernels built with CONFIG_PREEMPT=y therefore +use an RCU implementation that allows RCU read-side critical +sections to be preempted. +This requirement made its presence known after users made it +clear that an earlier +real-time patch +did not meet their needs, in conjunction with some +RCU issues +encountered by a very early version of the -rt patchset. + +

+In addition, RCU must make do with a sub-100-microsecond real-time latency +budget. +In fact, on smaller systems with the -rt patchset, the Linux kernel +provides sub-20-microsecond real-time latencies for the whole kernel, +including RCU. +RCU's scalability and latency must therefore be sufficient for +these sorts of configurations. +To my surprise, the sub-100-microsecond real-time latency budget + +applies to even the largest systems [PDF], +up to and including systems with 4096 CPUs. +This real-time requirement motivated the grace-period kthread, which +also simplified handling of a number of race conditions. + +

+Finally, RCU's status as a synchronization primitive means that +any RCU failure can result in arbitrary memory corruption that can be +extremely difficult to debug. +This means that RCU must be extremely reliable, which in +practice also means that RCU must have an aggressive stress-test +suite. +This stress-test suite is called rcutorture. + +

+Although the need for rcutorture was no surprise, +the current immense popularity of the Linux kernel is posing +interesting—and perhaps unprecedented—validation +challenges. +To see this, keep in mind that there are well over one billion +instances of the Linux kernel running today, given Android +smartphones, Linux-powered televisions, and servers. +This number can be expected to increase sharply with the advent of +the celebrated Internet of Things. + +

+Suppose that RCU contains a race condition that manifests on average +once per million years of runtime. +This bug will be occurring about three times per day across +the installed base. +RCU could simply hide behind hardware error rates, given that no one +should really expect their smartphone to last for a million years. +However, anyone taking too much comfort from this thought should +consider the fact that in most jurisdictions, a successful multi-year +test of a given mechanism, which might include a Linux kernel, +suffices for a number of types of safety-critical certifications. +In fact, rumor has it that the Linux kernel is already being used +in production for safety-critical applications. +I don't know about you, but I would feel quite bad if a bug in RCU +killed someone. +Which might explain my recent focus on validation and verification. + +

Other RCU Flavors

+ +

+One of the more surprising things about RCU is that there are now +no fewer than five flavors, or API families. +In addition, the primary flavor that has been the sole focus up to +this point has two different implementations, non-preemptible and +preemptible. +The other four flavors are listed below, with requirements for each +described in a separate section. + +

    +
  1. Bottom-Half Flavor +
  2. Sched Flavor +
  3. Sleepable RCU +
  4. Tasks RCU +
+ +

Bottom-Half Flavor

+ +

+The softirq-disable (AKA “bottom-half”, +hence the “_bh” abbreviations) +flavor of RCU, or RCU-bh, was developed by +Dipankar Sarma to provide a flavor of RCU that could withstand the +network-based denial-of-service attacks researched by Robert +Olsson. +These attacks placed so much networking load on the system +that some of the CPUs never exited softirq execution, +which in turn prevented those CPUs from ever executing a context switch, +which, in the RCU implementation of that time, prevented grace periods +from ever ending. +The result was an out-of-memory condition and a system hang. + +

+The solution was the creation of RCU-bh, which does +local_bh_disable() +across its read-side critical sections, and which uses the transition +from one type of softirq processing to another as a quiescent state +in addition to context switch, idle, user mode, and offline. +This means that RCU-bh grace periods can complete even when some of +the CPUs execute in softirq indefinitely, thus allowing algorithms +based on RCU-bh to withstand network-based denial-of-service attacks. + +

+Because +rcu_read_lock_bh() and rcu_read_unlock_bh() +disable and re-enable softirq handlers, any attempt to start a softirq +handlers during the +RCU-bh read-side critical section will be deferred. +In this case, rcu_read_unlock_bh() +will invoke softirq processing, which can take considerable time. +One can of course argue that this softirq overhead should be associated +with the code following the RCU-bh read-side critical section rather +than rcu_read_unlock_bh(), but the fact +is that most profiling tools cannot be expected to make this sort +of fine distinction. +For example, suppose that a three-millisecond-long RCU-bh read-side +critical section executes during a time of heavy networking load. +There will very likely be an attempt to invoke at least one softirq +handler during that three milliseconds, but any such invocation will +be delayed until the time of the rcu_read_unlock_bh(). +This can of course make it appear at first glance as if +rcu_read_unlock_bh() was executing very slowly. + +

+The +RCU-bh API +includes +rcu_read_lock_bh(), +rcu_read_unlock_bh(), +rcu_dereference_bh(), +rcu_dereference_bh_check(), +synchronize_rcu_bh(), +synchronize_rcu_bh_expedited(), +call_rcu_bh(), +rcu_barrier_bh(), and +rcu_read_lock_bh_held(). + +

Sched Flavor

+ +

+Before preemptible RCU, waiting for an RCU grace period had the +side effect of also waiting for all pre-existing interrupt +and NMI handlers. +However, there are legitimate preemptible-RCU implementations that +do not have this property, given that any point in the code outside +of an RCU read-side critical section can be a quiescent state. +Therefore, RCU-sched was created, which follows “classic” +RCU in that an RCU-sched grace period waits for for pre-existing +interrupt and NMI handlers. +In kernels built with CONFIG_PREEMPT=n, the RCU and RCU-sched +APIs have identical implementations, while kernels built with +CONFIG_PREEMPT=y provide a separate implementation for each. + +

+Note well that in CONFIG_PREEMPT=y kernels, +rcu_read_lock_sched() and rcu_read_unlock_sched() +disable and re-enable preemption, respectively. +This means that if there was a preemption attempt during the +RCU-sched read-side critical section, rcu_read_unlock_sched() +will enter the scheduler, with all the latency and overhead entailed. +Just as with rcu_read_unlock_bh(), this can make it look +as if rcu_read_unlock_sched() was executing very slowly. +However, the highest-priority task won't be preempted, so that task +will enjoy low-overhead rcu_read_unlock_sched() invocations. + +

+The +RCU-sched API +includes +rcu_read_lock_sched(), +rcu_read_unlock_sched(), +rcu_read_lock_sched_notrace(), +rcu_read_unlock_sched_notrace(), +rcu_dereference_sched(), +rcu_dereference_sched_check(), +synchronize_sched(), +synchronize_rcu_sched_expedited(), +call_rcu_sched(), +rcu_barrier_sched(), and +rcu_read_lock_sched_held(). +However, anything that disables preemption also marks an RCU-sched +read-side critical section, including +preempt_disable() and preempt_enable(), +local_irq_save() and local_irq_restore(), +and so on. + +

Sleepable RCU

+ +

+For well over a decade, someone saying “I need to block within +an RCU read-side critical section” was a reliable indication +that this someone did not understand RCU. +After all, if you are always blocking in an RCU read-side critical +section, you can probably afford to use a higher-overhead synchronization +mechanism. +However, that changed with the advent of the Linux kernel's notifiers, +whose RCU read-side critical +sections almost never sleep, but sometimes need to. +This resulted in the introduction of +sleepable RCU, +or SRCU. + +

+SRCU allows different domains to be defined, with each such domain +defined by an instance of an srcu_struct structure. +A pointer to this structure must be passed in to each SRCU function, +for example, synchronize_srcu(&ss), where +ss is the srcu_struct structure. +The key benefit of these domains is that a slow SRCU reader in one +domain does not delay an SRCU grace period in some other domain. +That said, one consequence of these domains is that read-side code +must pass a “cookie” from srcu_read_lock() +to srcu_read_unlock(), for example, as follows: + +

+
+ 1 int idx;
+ 2
+ 3 idx = srcu_read_lock(&ss);
+ 4 do_something();
+ 5 srcu_read_unlock(&ss, idx);
+
+
+ +

+As noted above, it is legal to block within SRCU read-side critical sections, +however, with great power comes great responsibility. +If you block forever in one of a given domain's SRCU read-side critical +sections, then that domain's grace periods will also be blocked forever. +Of course, one good way to block forever is to deadlock, which can +happen if any operation in a given domain's SRCU read-side critical +section can block waiting, either directly or indirectly, for that domain's +grace period to elapse. +For example, this results in a self-deadlock: + +

+
+ 1 int idx;
+ 2
+ 3 idx = srcu_read_lock(&ss);
+ 4 do_something();
+ 5 synchronize_srcu(&ss);
+ 6 srcu_read_unlock(&ss, idx);
+
+
+ +

+However, if line 5 acquired a mutex that was held across +a synchronize_srcu() for domain ss, +deadlock would still be possible. +Furthermore, if line 5 acquired a mutex that was held across +a synchronize_srcu() for some other domain ss1, +and if an ss1-domain SRCU read-side critical section +acquired another mutex that was held across as ss-domain +synchronize_srcu(), +deadlock would again be possible. +Such a deadlock cycle could extend across an arbitrarily large number +of different SRCU domains. +Again, with great power comes great responsibility. + +

+Unlike the other RCU flavors, SRCU read-side critical sections can +run on idle and even offline CPUs. +This ability requires that srcu_read_lock() and +srcu_read_unlock() contain memory barriers, which means +that SRCU readers will run a bit slower than would RCU readers. +It also motivates the smp_mb__after_srcu_read_unlock() +API, which, in combination with srcu_read_unlock(), +guarantees a full memory barrier. + +

+The +SRCU API +includes +srcu_read_lock(), +srcu_read_unlock(), +srcu_dereference(), +srcu_dereference_check(), +synchronize_srcu(), +synchronize_srcu_expedited(), +call_srcu(), +srcu_barrier(), and +srcu_read_lock_held(). +It also includes +DEFINE_SRCU(), +DEFINE_STATIC_SRCU(), and +init_srcu_struct() +APIs for defining and initializing srcu_struct structures. + +

Tasks RCU

+ +

+Some forms of tracing use “tramopolines” to handle the +binary rewriting required to install different types of probes. +It would be good to be able to free old trampolines, which sounds +like a job for some form of RCU. +However, because it is necessary to be able to install a trace +anywhere in the code, it is not possible to use read-side markers +such as rcu_read_lock() and rcu_read_unlock(). +In addition, it does not work to have these markers in the trampoline +itself, because there would need to be instructions following +rcu_read_unlock(). +Although synchronize_rcu() would guarantee that execution +reached the rcu_read_unlock(), it would not be able to +guarantee that execution had completely left the trampoline. + +

+The solution, in the form of +Tasks RCU, +is to have implicit +read-side critical sections that are delimited by voluntary context +switches, that is, calls to schedule(), +cond_resched_rcu_qs(), and +synchronize_rcu_tasks(). +In addition, transitions to and from userspace execution also delimit +tasks-RCU read-side critical sections. + +

+The tasks-RCU API is quite compact, consisting only of +call_rcu_tasks(), +synchronize_rcu_tasks(), and +rcu_barrier_tasks(). + +

Possible Future Changes

+ +

+One of the tricks that RCU uses to attain update-side scalability is +to increase grace-period latency with increasing numbers of CPUs. +If this becomes a serious problem, it will be necessary to rework the +grace-period state machine so as to avoid the need for the additional +latency. + +

+Expedited grace periods scan the CPUs, so their latency and overhead +increases with increasing numbers of CPUs. +If this becomes a serious problem on large systems, it will be necessary +to do some redesign to avoid this scalability problem. + +

+RCU disables CPU hotplug in a few places, perhaps most notably in the +expedited grace-period and rcu_barrier() operations. +If there is a strong reason to use expedited grace periods in CPU-hotplug +notifiers, it will be necessary to avoid disabling CPU hotplug. +This would introduce some complexity, so there had better be a very +good reason. + +

+The tradeoff between grace-period latency on the one hand and interruptions +of other CPUs on the other hand may need to be re-examined. +The desire is of course for zero grace-period latency as well as zero +interprocessor interrupts undertaken during an expedited grace period +operation. +While this ideal is unlikely to be achievable, it is quite possible that +further improvements can be made. + +

+The multiprocessor implementations of RCU use a combining tree that +groups CPUs so as to reduce lock contention and increase cache locality. +However, this combining tree does not spread its memory across NUMA +nodes nor does it align the CPU groups with hardware features such +as sockets or cores. +Such spreading and alignment is currently believed to be unnecessary +because the hotpath read-side primitives do not access the combining +tree, nor does call_rcu() in the common case. +If you believe that your architecture needs such spreading and alignment, +then your architecture should also benefit from the +rcutree.rcu_fanout_leaf boot parameter, which can be set +to the number of CPUs in a socket, NUMA node, or whatever. +If the number of CPUs is too large, use a fraction of the number of +CPUs. +If the number of CPUs is a large prime number, well, that certainly +is an “interesting” architectural choice! +More flexible arrangements might be considered, but only if +rcutree.rcu_fanout_leaf has proven inadequate, and only +if the inadequacy has been demonstrated by a carefully run and +realistic system-level workload. + +

+Please note that arrangements that require RCU to remap CPU numbers will +require extremely good demonstration of need and full exploration of +alternatives. + +

+There is an embarrassingly large number of flavors of RCU, and this +number has been increasing over time. +Perhaps it will be possible to combine some at some future date. + +

+RCU's various kthreads are reasonably recent additions. +It is quite likely that adjustments will be required to more gracefully +handle extreme loads. +It might also be necessary to be able to relate CPU utilization by +RCU's kthreads and softirq handlers to the code that instigated this +CPU utilization. +For example, RCU callback overhead might be charged back to the +originating call_rcu() instance, though probably not +in production kernels. + +

Summary

+ +

+This document has presented more than two decade's worth of RCU +requirements. +Given that the requirements keep changing, this will not be the last +word on this subject, but at least it serves to get an important +subset of the requirements set forth. + +

Acknowledgments

+ +I am grateful to Steven Rostedt, Lai Jiangshan, Ingo Molnar, +Oleg Nesterov, Borislav Petkov, Peter Zijlstra, Boqun Feng, and +Andy Lutomirski for their help in rendering +this article human readable, and to Michelle Rankin for her support +of this effort. +Other contributions are acknowledged in the Linux kernel's git archive. +The cartoon is copyright (c) 2013 by Melissa Broussard, +and is provided +under the terms of the Creative Commons Attribution-Share Alike 3.0 +United States license. + +

+Answers to Quick Quizzes

+ + +

Quick Quiz 1: +Wait a minute! +You said that updaters can make useful forward progress concurrently +with readers, but pre-existing readers will block +synchronize_rcu()!!! +Just who are you trying to fool??? + + +

Answer: +First, if updaters do not wish to be blocked by readers, they can use +call_rcu() or kfree_rcu(), which will +be discussed later. +Second, even when using synchronize_rcu(), the other +update-side code does run concurrently with readers, whether pre-existing +or not. + + +

Back to Quick Quiz 1. + + +

Quick Quiz 2: +Why is the synchronize_rcu() on line 28 needed? + + +

Answer: +Without that extra grace period, memory reordering could result in +do_something_dlm() executing do_something() +concurrently with the last bits of recovery(). + + +

Back to Quick Quiz 2. + + +

Quick Quiz 3: +But rcu_assign_pointer() does nothing to prevent the +two assignments to p->a and p->b +from being reordered. +Can't that also cause problems? + + +

Answer: +No, it cannot. +The readers cannot see either of these two fields until +the assignment to gp, by which time both fields are +fully initialized. +So reordering the assignments +to p->a and p->b cannot possibly +cause any problems. + + +

Back to Quick Quiz 3. + + +

Quick Quiz 4: +Without the rcu_dereference() or the +rcu_access_pointer(), what destructive optimizations +might the compiler make use of? + + +

Answer: +Let's start with what happens to do_something_gp() +if it fails to use rcu_dereference(). +It could reuse a value formerly fetched from this same pointer. +It could also fetch the pointer from gp in a byte-at-a-time +manner, resulting in load tearing, in turn resulting a bytewise +mash-up of two distince pointer values. +It might even use value-speculation optimizations, where it makes a wrong +guess, but by the time it gets around to checking the value, an update +has changed the pointer to match the wrong guess. +Too bad about any dereferences that returned pre-initialization garbage +in the meantime! + +

+For remove_gp_synchronous(), as long as all modifications +to gp are carried out while holding gp_lock, +the above optimizations are harmless. +However, +with CONFIG_SPARSE_RCU_POINTER=y, +sparse will complain if you +define gp with __rcu and then +access it without using +either rcu_access_pointer() or rcu_dereference(). + + +

Back to Quick Quiz 4. + + +

Quick Quiz 5: +Given that multiple CPUs can start RCU read-side critical sections +at any time without any ordering whatsoever, how can RCU possibly tell whether +or not a given RCU read-side critical section starts before a +given instance of synchronize_rcu()? + + +

Answer: +If RCU cannot tell whether or not a given +RCU read-side critical section starts before a +given instance of synchronize_rcu(), +then it must assume that the RCU read-side critical section +started first. +In other words, a given instance of synchronize_rcu() +can avoid waiting on a given RCU read-side critical section only +if it can prove that synchronize_rcu() started first. + + +

Back to Quick Quiz 5. + + +

Quick Quiz 6: +The first and second guarantees require unbelievably strict ordering! +Are all these memory barriers really required? + + +

Answer: +Yes, they really are required. +To see why the first guarantee is required, consider the following +sequence of events: + +

    +
  1. CPU 1: rcu_read_lock() +
  2. CPU 1: q = rcu_dereference(gp); + /* Very likely to return p. */ +
  3. CPU 0: list_del_rcu(p); +
  4. CPU 0: synchronize_rcu() starts. +
  5. CPU 1: do_something_with(q->a); + /* No smp_mb(), so might happen after kfree(). */ +
  6. CPU 1: rcu_read_unlock() +
  7. CPU 0: synchronize_rcu() returns. +
  8. CPU 0: kfree(p); +
+ +

+Therefore, there absolutely must be a full memory barrier between the +end of the RCU read-side critical section and the end of the +grace period. + +

+The sequence of events demonstrating the necessity of the second rule +is roughly similar: + +

    +
  1. CPU 0: list_del_rcu(p); +
  2. CPU 0: synchronize_rcu() starts. +
  3. CPU 1: rcu_read_lock() +
  4. CPU 1: q = rcu_dereference(gp); + /* Might return p if no memory barrier. */ +
  5. CPU 0: synchronize_rcu() returns. +
  6. CPU 0: kfree(p); +
  7. CPU 1: do_something_with(q->a); /* Boom!!! */ +
  8. CPU 1: rcu_read_unlock() +
+ +

+And similarly, without a memory barrier between the beginning of the +grace period and the beginning of the RCU read-side critical section, +CPU 1 might end up accessing the freelist. + +

+The “as if” rule of course applies, so that any implementation +that acts as if the appropriate memory barriers were in place is a +correct implementation. +That said, it is much easier to fool yourself into believing that you have +adhered to the as-if rule than it is to actually adhere to it! + + +

Back to Quick Quiz 6. + + +

Quick Quiz 7: +But how does the upgrade-to-write operation exclude other readers? + + +

Answer: +It doesn't, just like normal RCU updates, which also do not exclude +RCU readers. + + +

Back to Quick Quiz 7. + + +

Quick Quiz 8: +Can't the compiler also reorder this code? + + +

Answer: +No, the volatile casts in READ_ONCE() and +WRITE_ONCE() prevent the compiler from reordering in +this particular case. + + +

Back to Quick Quiz 8. + + +

Quick Quiz 9: +Suppose that synchronize_rcu() did wait until all readers had completed. +Would the updater be able to rely on this? + + +

Answer: +No. +Even if synchronize_rcu() were to wait until +all readers had completed, a new reader might start immediately after +synchronize_rcu() completed. +Therefore, the code following +synchronize_rcu() cannot rely on there being no readers +in any case. + + +

Back to Quick Quiz 9. + + +

Quick Quiz 10: +How long a sequence of grace periods, each separated by an RCU read-side +critical section, would be required to partition the RCU read-side +critical sections at the beginning and end of the chain? + + +

Answer: +In theory, an infinite number. +In practice, an unknown number that is sensitive to both implementation +details and timing considerations. +Therefore, even in practice, RCU users must abide by the theoretical rather +than the practical answer. + + +

Back to Quick Quiz 10. + + +

Quick Quiz 11: +What about sleeping locks? + + +

Answer: +These are forbidden within Linux-kernel RCU read-side critical sections +because it is not legal to place a quiescent state (in this case, +voluntary context switch) within an RCU read-side critical section. +However, sleeping locks may be used within userspace RCU read-side critical +sections, and also within Linux-kernel sleepable RCU +(SRCU) +read-side critical sections. +In addition, the -rt patchset turns spinlocks into a sleeping locks so +that the corresponding critical sections can be preempted, which +also means that these sleeplockified spinlocks (but not other sleeping locks!) +may be acquire within -rt-Linux-kernel RCU read-side critical sections. + +

+Note that it is legal for a normal RCU read-side critical section +to conditionally acquire a sleeping locks (as in mutex_trylock()), +but only as long as it does not loop indefinitely attempting to +conditionally acquire that sleeping locks. +The key point is that things like mutex_trylock() +either return with the mutex held, or return an error indication if +the mutex was not immediately available. +Either way, mutex_trylock() returns immediately without sleeping. + + +

Back to Quick Quiz 11. + + +

Quick Quiz 12: +Why does line 19 use rcu_access_pointer()? +After all, call_rcu() on line 25 stores into the +structure, which would interact badly with concurrent insertions. +Doesn't this mean that rcu_dereference() is required? + + +

Answer: +Presumably the ->gp_lock acquired on line 18 excludes +any changes, including any insertions that rcu_dereference() +would protect against. +Therefore, any insertions will be delayed until after ->gp_lock +is released on line 25, which in turn means that +rcu_access_pointer() suffices. + + +

Back to Quick Quiz 12. + + +

Quick Quiz 13: +Earlier it was claimed that call_rcu() and +kfree_rcu() allowed updaters to avoid being blocked +by readers. +But how can that be correct, given that the invocation of the callback +and the freeing of the memory (respectively) must still wait for +a grace period to elapse? + + +

Answer: +We could define things this way, but keep in mind that this sort of +definition would say that updates in garbage-collected languages +cannot complete until the next time the garbage collector runs, +which does not seem at all reasonable. +The key point is that in most cases, an updater using either +call_rcu() or kfree_rcu() can proceed to the +next update as soon as it has invoked call_rcu() or +kfree_rcu(), without having to wait for a subsequent +grace period. + + +

Back to Quick Quiz 13. + + +

Quick Quiz 14: +So what happens with synchronize_rcu() during +scheduler initialization for CONFIG_PREEMPT=n +kernels? + + +

Answer: +In CONFIG_PREEMPT=n kernel, synchronize_rcu() +maps directly to synchronize_sched(). +Therefore, synchronize_rcu() works normally throughout +boot in CONFIG_PREEMPT=n kernels. +However, your code must also work in CONFIG_PREEMPT=y kernels, +so it is still necessary to avoid invoking synchronize_rcu() +during scheduler initialization. + + +

Back to Quick Quiz 14. + + + diff --git a/Documentation/RCU/Design/Requirements/Requirements.htmlx b/Documentation/RCU/Design/Requirements/Requirements.htmlx new file mode 100644 index 0000000000000..1168010c39fe1 --- /dev/null +++ b/Documentation/RCU/Design/Requirements/Requirements.htmlx @@ -0,0 +1,2643 @@ + + + A Tour Through RCU's Requirements [LWN.net] + + +

A Tour Through RCU's Requirements

+ +

Copyright IBM Corporation, 2015

+

Author: Paul E. McKenney

+

The initial version of this document appeared in the +LWN articles +here, +here, and +here.

+ +

Introduction

+ +

+Read-copy update (RCU) is a synchronization mechanism that is often +used as a replacement for reader-writer locking. +RCU is unusual in that updaters do not block readers, +which means that RCU's read-side primitives can be exceedingly fast +and scalable. +In addition, updaters can make useful forward progress concurrently +with readers. +However, all this concurrency between RCU readers and updaters does raise +the question of exactly what RCU readers are doing, which in turn +raises the question of exactly what RCU's requirements are. + +

+This document therefore summarizes RCU's requirements, and can be thought +of as an informal, high-level specification for RCU. +It is important to understand that RCU's specification is primarily +empirical in nature; +in fact, I learned about many of these requirements the hard way. +This situation might cause some consternation, however, not only +has this learning process been a lot of fun, but it has also been +a great privilege to work with so many people willing to apply +technologies in interesting new ways. + +

+All that aside, here are the categories of currently known RCU requirements: +

+ +
    +
  1. + Fundamental Requirements +
  2. Fundamental Non-Requirements +
  3. + Parallelism Facts of Life +
  4. + Quality-of-Implementation Requirements +
  5. + Linux Kernel Complications +
  6. + Software-Engineering Requirements +
  7. + Other RCU Flavors +
  8. + Possible Future Changes +
+ +

+This is followed by a summary, +which is in turn followed by the inevitable +answers to the quick quizzes. + +

Fundamental Requirements

+ +

+RCU's fundamental requirements are the closest thing RCU has to hard +mathematical requirements. +These are: + +

    +
  1. + Grace-Period Guarantee +
  2. + Publish-Subscribe Guarantee +
  3. + RCU Primitives Guaranteed to Execute Unconditionally +
  4. + Guaranteed Read-to-Write Upgrade +
+ +

Grace-Period Guarantee

+ +

+RCU's grace-period guarantee is unusual in being premeditated: +Jack Slingwine and I had this guarantee firmly in mind when we started +work on RCU (then called “rclock”) in the early 1990s. +That said, the past two decades of experience with RCU have produced +a much more detailed understanding of this guarantee. + +

+RCU's grace-period guarantee allows updaters to wait for the completion +of all pre-existing RCU read-side critical sections. +An RCU read-side critical section +begins with the marker rcu_read_lock() and ends with +the marker rcu_read_unlock(). +These markers may be nested, and RCU treats a nested set as one +big RCU read-side critical section. +Production-quality implementations of rcu_read_lock() and +rcu_read_unlock() are extremely lightweight, and in +fact have exactly zero overhead in Linux kernels built for production +use with CONFIG_PREEMPT=n. + +

+This guarantee allows ordering to be enforced with extremely low +overhead to readers, for example: + +

+
+ 1 int x, y;
+ 2
+ 3 void thread0(void)
+ 4 {
+ 5   rcu_read_lock();
+ 6   r1 = READ_ONCE(x);
+ 7   r2 = READ_ONCE(y);
+ 8   rcu_read_unlock();
+ 9 }
+10
+11 void thread1(void)
+12 {
+13   WRITE_ONCE(x, 1);
+14   synchronize_rcu();
+15   WRITE_ONCE(y, 1);
+16 }
+
+
+ +

+Because the synchronize_rcu() on line 14 waits for +all pre-existing readers, any instance of thread0() that +loads a value of zero from x must complete before +thread1() stores to y, so that instance must +also load a value of zero from y. +Similarly, any instance of thread0() that loads a value of +one from y must have started after the +synchronize_rcu() started, and must therefore also load +a value of one from x. +Therefore, the outcome: +

+
+(r1 == 0 && r2 == 1)
+
+
+cannot happen. + +

@@QQ@@ +Wait a minute! +You said that updaters can make useful forward progress concurrently +with readers, but pre-existing readers will block +synchronize_rcu()!!! +Just who are you trying to fool??? +

@@QQA@@ +First, if updaters do not wish to be blocked by readers, they can use +call_rcu() or kfree_rcu(), which will +be discussed later. +Second, even when using synchronize_rcu(), the other +update-side code does run concurrently with readers, whether pre-existing +or not. +

@@QQE@@ + +

+This scenario resembles one of the first uses of RCU in +DYNIX/ptx, +which managed a distributed lock manager's transition into +a state suitable for handling recovery from node failure, +more or less as follows: + +

+
+ 1 #define STATE_NORMAL        0
+ 2 #define STATE_WANT_RECOVERY 1
+ 3 #define STATE_RECOVERING    2
+ 4 #define STATE_WANT_NORMAL   3
+ 5
+ 6 int state = STATE_NORMAL;
+ 7
+ 8 void do_something_dlm(void)
+ 9 {
+10   int state_snap;
+11
+12   rcu_read_lock();
+13   state_snap = READ_ONCE(state);
+14   if (state_snap == STATE_NORMAL)
+15     do_something();
+16   else
+17     do_something_carefully();
+18   rcu_read_unlock();
+19 }
+20
+21 void start_recovery(void)
+22 {
+23   WRITE_ONCE(state, STATE_WANT_RECOVERY);
+24   synchronize_rcu();
+25   WRITE_ONCE(state, STATE_RECOVERING);
+26   recovery();
+27   WRITE_ONCE(state, STATE_WANT_NORMAL);
+28   synchronize_rcu();
+29   WRITE_ONCE(state, STATE_NORMAL);
+30 }
+
+
+ +

+The RCU read-side critical section in do_something_dlm() +works with the synchronize_rcu() in start_recovery() +to guarantee that do_something() never runs concurrently +with recovery(), but with little or no synchronization +overhead in do_something_dlm(). + +

@@QQ@@ +Why is the synchronize_rcu() on line 28 needed? +

@@QQA@@ +Without that extra grace period, memory reordering could result in +do_something_dlm() executing do_something() +concurrently with the last bits of recovery(). +

@@QQE@@ + +

+In order to avoid fatal problems such as deadlocks, +an RCU read-side critical section must not contain calls to +synchronize_rcu(). +Similarly, an RCU read-side critical section must not +contain anything that waits, directly or indirectly, on completion of +an invocation of synchronize_rcu(). + +

+Although RCU's grace-period guarantee is useful in and of itself, with +quite a few use cases, +it would be good to be able to use RCU to coordinate read-side +access to linked data structures. +For this, the grace-period guarantee is not sufficient, as can +be seen in function add_gp_buggy() below. +We will look at the reader's code later, but in the meantime, just think of +the reader as locklessly picking up the gp pointer, +and, if the value loaded is non-NULL, locklessly accessing the +->a and ->b fields. + +

+
+ 1 bool add_gp_buggy(int a, int b)
+ 2 {
+ 3   p = kmalloc(sizeof(*p), GFP_KERNEL);
+ 4   if (!p)
+ 5     return -ENOMEM;
+ 6   spin_lock(&gp_lock);
+ 7   if (rcu_access_pointer(gp)) {
+ 8     spin_unlock(&gp_lock);
+ 9     return false;
+10   }
+11   p->a = a;
+12   p->b = a;
+13   gp = p; /* ORDERING BUG */
+14   spin_unlock(&gp_lock);
+15   return true;
+16 }
+
+
+ +

+The problem is that both the compiler and weakly ordered CPUs are within +their rights to reorder this code as follows: + +

+
+ 1 bool add_gp_buggy_optimized(int a, int b)
+ 2 {
+ 3   p = kmalloc(sizeof(*p), GFP_KERNEL);
+ 4   if (!p)
+ 5     return -ENOMEM;
+ 6   spin_lock(&gp_lock);
+ 7   if (rcu_access_pointer(gp)) {
+ 8     spin_unlock(&gp_lock);
+ 9     return false;
+10   }
+11   gp = p; /* ORDERING BUG */
+12   p->a = a;
+13   p->b = a;
+14   spin_unlock(&gp_lock);
+15   return true;
+16 }
+
+
+ +

+If an RCU reader fetches gp just after +add_gp_buggy_optimized executes line 11, +it will see garbage in the ->a and ->b +fields. +And this is but one of many ways in which compiler and hardware optimizations +could cause trouble. +Therefore, we clearly need some way to prevent the compiler and the CPU from +reordering in this manner, which brings us to the publish-subscribe +guarantee discussed in the next section. + +

Publish/Subscribe Guarantee

+ +

+RCU's publish-subscribe guarantee allows data to be inserted +into a linked data structure without disrupting RCU readers. +The updater uses rcu_assign_pointer() to insert the +new data, and readers use rcu_dereference() to +access data, whether new or old. +The following shows an example of insertion: + +

+
+ 1 bool add_gp(int a, int b)
+ 2 {
+ 3   p = kmalloc(sizeof(*p), GFP_KERNEL);
+ 4   if (!p)
+ 5     return -ENOMEM;
+ 6   spin_lock(&gp_lock);
+ 7   if (rcu_access_pointer(gp)) {
+ 8     spin_unlock(&gp_lock);
+ 9     return false;
+10   }
+11   p->a = a;
+12   p->b = a;
+13   rcu_assign_pointer(gp, p);
+14   spin_unlock(&gp_lock);
+15   return true;
+16 }
+
+
+ +

+The rcu_assign_pointer() on line 13 is conceptually +equivalent to a simple assignment statement, but also guarantees +that its assignment will +happen after the two assignments in lines 11 and 12, +similar to the C11 memory_order_release store operation. +It also prevents any number of “interesting” compiler +optimizations, for example, the use of gp as a scratch +location immediately preceding the assignment. + +

@@QQ@@ +But rcu_assign_pointer() does nothing to prevent the +two assignments to p->a and p->b +from being reordered. +Can't that also cause problems? +

@@QQA@@ +No, it cannot. +The readers cannot see either of these two fields until +the assignment to gp, by which time both fields are +fully initialized. +So reordering the assignments +to p->a and p->b cannot possibly +cause any problems. +

@@QQE@@ + +

+It is tempting to assume that the reader need not do anything special +to control its accesses to the RCU-protected data, +as shown in do_something_gp_buggy() below: + +

+
+ 1 bool do_something_gp_buggy(void)
+ 2 {
+ 3   rcu_read_lock();
+ 4   p = gp;  /* OPTIMIZATIONS GALORE!!! */
+ 5   if (p) {
+ 6     do_something(p->a, p->b);
+ 7     rcu_read_unlock();
+ 8     return true;
+ 9   }
+10   rcu_read_unlock();
+11   return false;
+12 }
+
+
+ +

+However, this temptation must be resisted because there are a +surprisingly large number of ways that the compiler +(to say nothing of +DEC Alpha CPUs) +can trip this code up. +For but one example, if the compiler were short of registers, it +might choose to refetch from gp rather than keeping +a separate copy in p as follows: + +

+
+ 1 bool do_something_gp_buggy_optimized(void)
+ 2 {
+ 3   rcu_read_lock();
+ 4   if (gp) { /* OPTIMIZATIONS GALORE!!! */
+ 5     do_something(gp->a, gp->b);
+ 6     rcu_read_unlock();
+ 7     return true;
+ 8   }
+ 9   rcu_read_unlock();
+10   return false;
+11 }
+
+
+ +

+If this function ran concurrently with a series of updates that +replaced the current structure with a new one, +the fetches of gp->a +and gp->b might well come from two different structures, +which could cause serious confusion. +To prevent this (and much else besides), do_something_gp() uses +rcu_dereference() to fetch from gp: + +

+
+ 1 bool do_something_gp(void)
+ 2 {
+ 3   rcu_read_lock();
+ 4   p = rcu_dereference(gp);
+ 5   if (p) {
+ 6     do_something(p->a, p->b);
+ 7     rcu_read_unlock();
+ 8     return true;
+ 9   }
+10   rcu_read_unlock();
+11   return false;
+12 }
+
+
+ +

+The rcu_dereference() uses volatile casts and (for DEC Alpha) +memory barriers in the Linux kernel. +Should a +high-quality implementation of C11 memory_order_consume [PDF] +ever appear, then rcu_dereference() could be implemented +as a memory_order_consume load. +Regardless of the exact implementation, a pointer fetched by +rcu_dereference() may not be used outside of the +outermost RCU read-side critical section containing that +rcu_dereference(), unless protection of +the corresponding data element has been passed from RCU to some +other synchronization mechanism, most commonly locking or +reference counting. + +

+In short, updaters use rcu_assign_pointer() and readers +use rcu_dereference(), and these two RCU API elements +work together to ensure that readers have a consistent view of +newly added data elements. + +

+Of course, it is also necessary to remove elements from RCU-protected +data structures, for example, using the following process: + +

    +
  1. Remove the data element from the enclosing structure. +
  2. Wait for all pre-existing RCU read-side critical sections + to complete (because only pre-existing readers can possibly have + a reference to the newly removed data element). +
  3. At this point, only the updater has a reference to the + newly removed data element, so it can safely reclaim + the data element, for example, by passing it to kfree(). +
+ +This process is implemented by remove_gp_synchronous(): + +
+
+ 1 bool remove_gp_synchronous(void)
+ 2 {
+ 3   struct foo *p;
+ 4
+ 5   spin_lock(&gp_lock);
+ 6   p = rcu_access_pointer(gp);
+ 7   if (!p) {
+ 8     spin_unlock(&gp_lock);
+ 9     return false;
+10   }
+11   rcu_assign_pointer(gp, NULL);
+12   spin_unlock(&gp_lock);
+13   synchronize_rcu();
+14   kfree(p);
+15   return true;
+16 }
+
+
+ +

+This function is straightforward, with line 13 waiting for a grace +period before line 14 frees the old data element. +This waiting ensures that readers will reach line 7 of +do_something_gp() before the data element referenced by +p is freed. +The rcu_access_pointer() on line 6 is similar to +rcu_dereference(), except that: + +

    +
  1. The value returned by rcu_access_pointer() + cannot be dereferenced. + If you want to access the value pointed to as well as + the pointer itself, use rcu_dereference() + instead of rcu_access_pointer(). +
  2. The call to rcu_access_pointer() need not be + protected. + In contrast, rcu_dereference() must either be + within an RCU read-side critical section or in a code + segment where the pointer cannot change, for example, in + code protected by the corresponding update-side lock. +
+ +

@@QQ@@ +Without the rcu_dereference() or the +rcu_access_pointer(), what destructive optimizations +might the compiler make use of? +

@@QQA@@ +Let's start with what happens to do_something_gp() +if it fails to use rcu_dereference(). +It could reuse a value formerly fetched from this same pointer. +It could also fetch the pointer from gp in a byte-at-a-time +manner, resulting in load tearing, in turn resulting a bytewise +mash-up of two distince pointer values. +It might even use value-speculation optimizations, where it makes a wrong +guess, but by the time it gets around to checking the value, an update +has changed the pointer to match the wrong guess. +Too bad about any dereferences that returned pre-initialization garbage +in the meantime! + +

+For remove_gp_synchronous(), as long as all modifications +to gp are carried out while holding gp_lock, +the above optimizations are harmless. +However, +with CONFIG_SPARSE_RCU_POINTER=y, +sparse will complain if you +define gp with __rcu and then +access it without using +either rcu_access_pointer() or rcu_dereference(). +

@@QQE@@ + +

+This simple linked-data-structure scenario clearly demonstrates the need +for RCU's stringent memory-ordering guarantees on systems with more than +one CPU: + +

    +
  1. Each CPU that has an RCU read-side critical section that + begins before synchronize_rcu() starts is + guaranteed to execute a full memory barrier between the time + that the RCU read-side critical section ends and the time that + synchronize_rcu() returns. + Without this guarantee, a pre-existing RCU read-side critical section + might hold a reference to the newly removed struct foo + after the kfree() on line 14 of + remove_gp_synchronous(). +
  2. Each CPU that has an RCU read-side critical section that ends + after synchronize_rcu() returns is guaranteed + to execute a full memory barrier between the time that + synchronize_rcu() begins and the time that the RCU + read-side critical section begins. + Without this guarantee, a later RCU read-side critical section + running after the kfree() on line 14 of + remove_gp_synchronous() might + later run do_something_gp() and find the + newly deleted struct foo. +
  3. If the task invoking synchronize_rcu() remains + on a given CPU, then that CPU is guaranteed to execute a full + memory barrier sometime during the execution of + synchronize_rcu(). + This guarantee ensures that the kfree() on + line 14 of remove_gp_synchronous() really does + execute after the removal on line 11. +
  4. If the task invoking synchronize_rcu() migrates + among a group of CPUs during that invocation, then each of the + CPUs in that group is guaranteed to execute a full memory barrier + sometime during the execution of synchronize_rcu(). + This guarantee also ensures that the kfree() on + line 14 of remove_gp_synchronous() really does + execute after the removal on + line 11, but also in the case where the thread executing the + synchronize_rcu() migrates in the meantime. +
+ +

@@QQ@@ +Given that multiple CPUs can start RCU read-side critical sections +at any time without any ordering whatsoever, how can RCU possibly tell whether +or not a given RCU read-side critical section starts before a +given instance of synchronize_rcu()? +

@@QQA@@ +If RCU cannot tell whether or not a given +RCU read-side critical section starts before a +given instance of synchronize_rcu(), +then it must assume that the RCU read-side critical section +started first. +In other words, a given instance of synchronize_rcu() +can avoid waiting on a given RCU read-side critical section only +if it can prove that synchronize_rcu() started first. +

@@QQE@@ + +

@@QQ@@ +The first and second guarantees require unbelievably strict ordering! +Are all these memory barriers really required? +

@@QQA@@ +Yes, they really are required. +To see why the first guarantee is required, consider the following +sequence of events: + +

    +
  1. CPU 1: rcu_read_lock() +
  2. CPU 1: q = rcu_dereference(gp); + /* Very likely to return p. */ +
  3. CPU 0: list_del_rcu(p); +
  4. CPU 0: synchronize_rcu() starts. +
  5. CPU 1: do_something_with(q->a); + /* No smp_mb(), so might happen after kfree(). */ +
  6. CPU 1: rcu_read_unlock() +
  7. CPU 0: synchronize_rcu() returns. +
  8. CPU 0: kfree(p); +
+ +

+Therefore, there absolutely must be a full memory barrier between the +end of the RCU read-side critical section and the end of the +grace period. + +

+The sequence of events demonstrating the necessity of the second rule +is roughly similar: + +

    +
  1. CPU 0: list_del_rcu(p); +
  2. CPU 0: synchronize_rcu() starts. +
  3. CPU 1: rcu_read_lock() +
  4. CPU 1: q = rcu_dereference(gp); + /* Might return p if no memory barrier. */ +
  5. CPU 0: synchronize_rcu() returns. +
  6. CPU 0: kfree(p); +
  7. CPU 1: do_something_with(q->a); /* Boom!!! */ +
  8. CPU 1: rcu_read_unlock() +
+ +

+And similarly, without a memory barrier between the beginning of the +grace period and the beginning of the RCU read-side critical section, +CPU 1 might end up accessing the freelist. + +

+The “as if” rule of course applies, so that any implementation +that acts as if the appropriate memory barriers were in place is a +correct implementation. +That said, it is much easier to fool yourself into believing that you have +adhered to the as-if rule than it is to actually adhere to it! +

@@QQE@@ + +

+In short, RCU's publish-subscribe guarantee is provided by the combination +of rcu_assign_pointer() and rcu_dereference(). +This guarantee allows data elements to be safely added to RCU-protected +linked data structures without disrupting RCU readers. +This guarantee can be used in combination with the grace-period +guarantee to also allow data elements to be removed from RCU-protected +linked data structures, again without disrupting RCU readers. + +

+This guarantee was only partially premeditated. +DYNIX/ptx used an explicit memory barrier for publication, but had nothing +resembling rcu_dereference() for subscription, nor did it +have anything resembling the smp_read_barrier_depends() +that was later subsumed into rcu_dereference(). +The need for these operations made itself known quite suddenly at a +late-1990s meeting with the DEC Alpha architects, back in the days when +DEC was still a free-standing company. +It took the Alpha architects a good hour to convince me that any sort +of barrier would ever be needed, and it then took me a good two hours +to convince them that their documentation did not make this point clear. +More recent work with the C and C++ standards committees have provided +much education on tricks and traps from the compiler. +In short, compilers were much less tricky in the early 1990s, but in +2015, don't even think about omitting rcu_dereference()! + +

RCU Primitives Guaranteed to Execute Unconditionally

+ +

+The common-case RCU primitives are unconditional. +They are invoked, they do their job, and they return, with no possibility +of error, and no need to retry. +This is a key RCU design philosophy. + +

+However, this philosophy is pragmatic rather than pigheaded. +If someone comes up with a good justification for a particular conditional +RCU primitive, it might well be implemented and added. +After all, this guarantee was reverse-engineered, not premeditated. +The unconditional nature of the RCU primitives was initially an +accident of implementation, and later experience with synchronization +primitives with conditional primitives caused me to elevate this +accident to a guarantee. +Therefore, the justification for adding a conditional primitive to +RCU would need to be based on detailed and compelling use cases. + +

Guaranteed Read-to-Write Upgrade

+ +

+As far as RCU is concerned, it is always possible to carry out an +update within an RCU read-side critical section. +For example, that RCU read-side critical section might search for +a given data element, and then might acquire the update-side +spinlock in order to update that element, all while remaining +in that RCU read-side critical section. +Of course, it is necessary to exit the RCU read-side critical section +before invoking synchronize_rcu(), however, this +inconvenience can be avoided through use of the +call_rcu() and kfree_rcu() API members +described later in this document. + +

@@QQ@@ +But how does the upgrade-to-write operation exclude other readers? +

@@QQA@@ +It doesn't, just like normal RCU updates, which also do not exclude +RCU readers. +

@@QQE@@ + +

+This guarantee allows lookup code to be shared between read-side +and update-side code, and was premeditated, appearing in the earliest +DYNIX/ptx RCU documentation. + +

Fundamental Non-Requirements

+ +

+RCU provides extremely lightweight readers, and its read-side guarantees, +though quite useful, are correspondingly lightweight. +It is therefore all too easy to assume that RCU is guaranteeing more +than it really is. +Of course, the list of things that RCU does not guarantee is infinitely +long, however, the following sections list a few non-guarantees that +have caused confusion. +Except where otherwise noted, these non-guarantees were premeditated. + +

    +
  1. + Readers Impose Minimal Ordering +
  2. + Readers Do Not Exclude Updaters +
  3. + Updaters Only Wait For Old Readers +
  4. + Grace Periods Don't Partition Read-Side Critical Sections +
  5. + Read-Side Critical Sections Don't Partition Grace Periods +
  6. + Disabling Preemption Does Not Block Grace Periods +
+ +

Readers Impose Minimal Ordering

+ +

+Reader-side markers such as rcu_read_lock() and +rcu_read_unlock() provide absolutely no ordering guarantees +except through their interaction with the grace-period APIs such as +synchronize_rcu(). +To see this, consider the following pair of threads: + +

+
+ 1 void thread0(void)
+ 2 {
+ 3   rcu_read_lock();
+ 4   WRITE_ONCE(x, 1);
+ 5   rcu_read_unlock();
+ 6   rcu_read_lock();
+ 7   WRITE_ONCE(y, 1);
+ 8   rcu_read_unlock();
+ 9 }
+10
+11 void thread1(void)
+12 {
+13   rcu_read_lock();
+14   r1 = READ_ONCE(y);
+15   rcu_read_unlock();
+16   rcu_read_lock();
+17   r2 = READ_ONCE(x);
+18   rcu_read_unlock();
+19 }
+
+
+ +

+After thread0() and thread1() execute +concurrently, it is quite possible to have + +

+
+(r1 == 1 && r2 == 0)
+
+
+ +(that is, y appears to have been assigned before x), +which would not be possible if rcu_read_lock() and +rcu_read_unlock() had much in the way of ordering +properties. +But they do not, so the CPU is within its rights +to do significant reordering. +This is by design: Any significant ordering constraints would slow down +these fast-path APIs. + +

@@QQ@@ +Can't the compiler also reorder this code? +

@@QQA@@ +No, the volatile casts in READ_ONCE() and +WRITE_ONCE() prevent the compiler from reordering in +this particular case. +

@@QQE@@ + +

Readers Do Not Exclude Updaters

+ +

+Neither rcu_read_lock() nor rcu_read_unlock() +exclude updates. +All they do is to prevent grace periods from ending. +The following example illustrates this: + +

+
+ 1 void thread0(void)
+ 2 {
+ 3   rcu_read_lock();
+ 4   r1 = READ_ONCE(y);
+ 5   if (r1) {
+ 6     do_something_with_nonzero_x();
+ 7     r2 = READ_ONCE(x);
+ 8     WARN_ON(!r2); /* BUG!!! */
+ 9   }
+10   rcu_read_unlock();
+11 }
+12
+13 void thread1(void)
+14 {
+15   spin_lock(&my_lock);
+16   WRITE_ONCE(x, 1);
+17   WRITE_ONCE(y, 1);
+18   spin_unlock(&my_lock);
+19 }
+
+
+ +

+If the thread0() function's rcu_read_lock() +excluded the thread1() function's update, +the WARN_ON() could never fire. +But the fact is that rcu_read_lock() does not exclude +much of anything aside from subsequent grace periods, of which +thread1() has none, so the +WARN_ON() can and does fire. + +

Updaters Only Wait For Old Readers

+ +

+It might be tempting to assume that after synchronize_rcu() +completes, there are no readers executing. +This temptation must be avoided because +new readers can start immediately after synchronize_rcu() +starts, and synchronize_rcu() is under no +obligation to wait for these new readers. + +

@@QQ@@ +Suppose that synchronize_rcu() did wait until all readers had completed. +Would the updater be able to rely on this? +

@@QQA@@ +No. +Even if synchronize_rcu() were to wait until +all readers had completed, a new reader might start immediately after +synchronize_rcu() completed. +Therefore, the code following +synchronize_rcu() cannot rely on there being no readers +in any case. +

@@QQE@@ + +

+Grace Periods Don't Partition Read-Side Critical Sections

+ +

+It is tempting to assume that if any part of one RCU read-side critical +section precedes a given grace period, and if any part of another RCU +read-side critical section follows that same grace period, then all of +the first RCU read-side critical section must precede all of the second. +However, this just isn't the case: A single grace period does not +partition the set of RCU read-side critical sections. +An example of this situation can be illustrated as follows, where +x, y, and z are initially all zero: + +

+
+ 1 void thread0(void)
+ 2 {
+ 3   rcu_read_lock();
+ 4   WRITE_ONCE(a, 1);
+ 5   WRITE_ONCE(b, 1);
+ 6   rcu_read_unlock();
+ 7 }
+ 8
+ 9 void thread1(void)
+10 {
+11   r1 = READ_ONCE(a);
+12   synchronize_rcu();
+13   WRITE_ONCE(c, 1);
+14 }
+15
+16 void thread2(void)
+17 {
+18   rcu_read_lock();
+19   r2 = READ_ONCE(b);
+20   r3 = READ_ONCE(c);
+21   rcu_read_unlock();
+22 }
+
+
+ +

+It turns out that the outcome: + +

+
+(r1 == 1 && r2 == 0 && r3 == 1)
+
+
+ +is entirely possible. +The following figure show how this can happen, with each circled +QS indicating the point at which RCU recorded a +quiescent state for each thread, that is, a state in which +RCU knows that the thread cannot be in the midst of an RCU read-side +critical section that started before the current grace period: + +

GPpartitionReaders1.svg

+ +

+If it is necessary to partition RCU read-side critical sections in this +manner, it is necessary to use two grace periods, where the first +grace period is known to end before the second grace period starts: + +

+
+ 1 void thread0(void)
+ 2 {
+ 3   rcu_read_lock();
+ 4   WRITE_ONCE(a, 1);
+ 5   WRITE_ONCE(b, 1);
+ 6   rcu_read_unlock();
+ 7 }
+ 8
+ 9 void thread1(void)
+10 {
+11   r1 = READ_ONCE(a);
+12   synchronize_rcu();
+13   WRITE_ONCE(c, 1);
+14 }
+15
+16 void thread2(void)
+17 {
+18   r2 = READ_ONCE(c);
+19   synchronize_rcu();
+20   WRITE_ONCE(d, 1);
+21 }
+22
+23 void thread3(void)
+24 {
+25   rcu_read_lock();
+26   r3 = READ_ONCE(b);
+27   r4 = READ_ONCE(d);
+28   rcu_read_unlock();
+29 }
+
+
+ +

+Here, if (r1 == 1), then +thread0()'s write to b must happen +before the end of thread1()'s grace period. +If in addition (r4 == 1), then +thread3()'s read from b must happen +after the beginning of thread2()'s grace period. +If it is also the case that (r2 == 1), then the +end of thread1()'s grace period must precede the +beginning of thread2()'s grace period. +This mean that the two RCU read-side critical sections cannot overlap, +guaranteeing that (r3 == 1). +As a result, the outcome: + +

+
+(r1 == 1 && r2 == 1 && r3 == 0 && r4 == 1)
+
+
+ +cannot happen. + +

+This non-requirement was also non-premeditated, but became apparent +when studying RCU's interaction with memory ordering. + +

+Read-Side Critical Sections Don't Partition Grace Periods

+ +

+It is also tempting to assume that if an RCU read-side critical section +happens between a pair of grace periods, then those grace periods cannot +overlap. +However, this temptation leads nowhere good, as can be illustrated by +the following, with all variables initially zero: + +

+
+ 1 void thread0(void)
+ 2 {
+ 3   rcu_read_lock();
+ 4   WRITE_ONCE(a, 1);
+ 5   WRITE_ONCE(b, 1);
+ 6   rcu_read_unlock();
+ 7 }
+ 8
+ 9 void thread1(void)
+10 {
+11   r1 = READ_ONCE(a);
+12   synchronize_rcu();
+13   WRITE_ONCE(c, 1);
+14 }
+15
+16 void thread2(void)
+17 {
+18   rcu_read_lock();
+19   WRITE_ONCE(d, 1);
+20   r2 = READ_ONCE(c);
+21   rcu_read_unlock();
+22 }
+23
+24 void thread3(void)
+25 {
+26   r3 = READ_ONCE(d);
+27   synchronize_rcu();
+28   WRITE_ONCE(e, 1);
+29 }
+30
+31 void thread4(void)
+32 {
+33   rcu_read_lock();
+34   r4 = READ_ONCE(b);
+35   r5 = READ_ONCE(e);
+36   rcu_read_unlock();
+37 }
+
+
+ +

+In this case, the outcome: + +

+
+(r1 == 1 && r2 == 1 && r3 == 1 && r4 == 0 && r5 == 1)
+
+
+ +is entirely possible, as illustrated below: + +

ReadersPartitionGP1.svg

+ +

+Again, an RCU read-side critical section can overlap almost all of a +given grace period, just so long as it does not overlap the entire +grace period. +As a result, an RCU read-side critical section cannot partition a pair +of RCU grace periods. + +

@@QQ@@ +How long a sequence of grace periods, each separated by an RCU read-side +critical section, would be required to partition the RCU read-side +critical sections at the beginning and end of the chain? +

@@QQA@@ +In theory, an infinite number. +In practice, an unknown number that is sensitive to both implementation +details and timing considerations. +Therefore, even in practice, RCU users must abide by the theoretical rather +than the practical answer. +

@@QQE@@ + +

+Disabling Preemption Does Not Block Grace Periods

+ +

+There was a time when disabling preemption on any given CPU would block +subsequent grace periods. +However, this was an accident of implementation and is not a requirement. +And in the current Linux-kernel implementation, disabling preemption +on a given CPU in fact does not block grace periods, as Oleg Nesterov +demonstrated. + +

+If you need a preempt-disable region to block grace periods, you need to add +rcu_read_lock() and rcu_read_unlock(), for example +as follows: + +

+
+ 1 preempt_disable();
+ 2 rcu_read_lock();
+ 3 do_something();
+ 4 rcu_read_unlock();
+ 5 preempt_enable();
+ 6
+ 7 /* Spinlocks implicitly disable preemption. */
+ 8 spin_lock(&mylock);
+ 9 rcu_read_lock();
+10 do_something();
+11 rcu_read_unlock();
+12 spin_unlock(&mylock);
+
+
+ +

+In theory, you could enter the RCU read-side critical section first, +but it is more efficient to keep the entire RCU read-side critical +section contained in the preempt-disable region as shown above. +Of course, RCU read-side critical sections that extend outside of +preempt-disable regions will work correctly, but such critical sections +can be preempted, which forces rcu_read_unlock() to do +more work. +And no, this is not an invitation to enclose all of your RCU +read-side critical sections within preempt-disable regions, because +doing so would degrade real-time response. + +

+This non-requirement appeared with preemptible RCU. +If you need a grace period that waits on non-preemptible code regions, use +RCU-sched. + +

Parallelism Facts of Life

+ +

+These parallelism facts of life are by no means specific to RCU, but +the RCU implementation must abide by them. +They therefore bear repeating: + +

    +
  1. Any CPU or task may be delayed at any time, + and any attempts to avoid these delays by disabling + preemption, interrupts, or whatever are completely futile. + This is most obvious in preemptible user-level + environments and in virtualized environments (where + a given guest OS's VCPUs can be preempted at any time by + the underlying hypervisor), but can also happen in bare-metal + environments due to ECC errors, NMIs, and other hardware + events. + Although a delay of more than about 20 seconds can result + in splats, the RCU implementation is obligated to use + algorithms that can tolerate extremely long delays, but where + “extremely long” is not long enough to allow + wrap-around when incrementing a 64-bit counter. +
  2. Both the compiler and the CPU can reorder memory accesses. + Where it matters, RCU must use compiler directives and + memory-barrier instructions to preserve ordering. +
  3. Conflicting writes to memory locations in any given cache line + will result in expensive cache misses. + Greater numbers of concurrent writes and more-frequent + concurrent writes will result in more dramatic slowdowns. + RCU is therefore obligated to use algorithms that have + sufficient locality to avoid significant performance and + scalability problems. +
  4. As a rough rule of thumb, only one CPU's worth of processing + may be carried out under the protection of any given exclusive + lock. + RCU must therefore use scalable locking designs. +
  5. Counters are finite, especially on 32-bit systems. + RCU's use of counters must therefore tolerate counter wrap, + or be designed such that counter wrap would take way more + time than a single system is likely to run. + An uptime of ten years is quite possible, a runtime + of a century much less so. + As an example of the latter, RCU's dyntick-idle nesting counter + allows 54 bits for interrupt nesting level (this counter + is 64 bits even on a 32-bit system). + Overflowing this counter requires 254 + half-interrupts on a given CPU without that CPU ever going idle. + If a half-interrupt happened every microsecond, it would take + 570 years of runtime to overflow this counter, which is currently + believed to be an acceptably long time. +
  6. Linux systems can have thousands of CPUs running a single + Linux kernel in a single shared-memory environment. + RCU must therefore pay close attention to high-end scalability. +
+ +

+This last parallelism fact of life means that RCU must pay special +attention to the preceding facts of life. +The idea that Linux might scale to systems with thousands of CPUs would +have been met with some skepticism in the 1990s, but these requirements +would have otherwise have been unsurprising, even in the early 1990s. + +

Quality-of-Implementation Requirements

+ +

+These sections list quality-of-implementation requirements. +Although an RCU implementation that ignores these requirements could +still be used, it would likely be subject to limitations that would +make it inappropriate for industrial-strength production use. +Classes of quality-of-implementation requirements are as follows: + +

    +
  1. Specialization +
  2. Performance and Scalability +
  3. Composability +
  4. Corner Cases +
+ +

+These classes is covered in the following sections. + +

Specialization

+ +

+RCU is and always has been intended primarily for read-mostly situations, as +illustrated by the following figure. +This means that RCU's read-side primitives are optimized, often at the +expense of its update-side primitives. + +

RCUApplicability.svg

+ +

+This focus on read-mostly situations means that RCU must interoperate +with other synchronization primitives. +For example, the add_gp() and remove_gp_synchronous() +examples discussed earlier use RCU to protect readers and locking to +coordinate updaters. +However, the need extends much farther, requiring that a variety of +synchronization primitives be legal within RCU read-side critical sections, +including spinlocks, sequence locks, atomic operations, reference +counters, and memory barriers. + +

@@QQ@@ +What about sleeping locks? +

@@QQA@@ +These are forbidden within Linux-kernel RCU read-side critical sections +because it is not legal to place a quiescent state (in this case, +voluntary context switch) within an RCU read-side critical section. +However, sleeping locks may be used within userspace RCU read-side critical +sections, and also within Linux-kernel sleepable RCU +(SRCU) +read-side critical sections. +In addition, the -rt patchset turns spinlocks into a sleeping locks so +that the corresponding critical sections can be preempted, which +also means that these sleeplockified spinlocks (but not other sleeping locks!) +may be acquire within -rt-Linux-kernel RCU read-side critical sections. + +

+Note that it is legal for a normal RCU read-side critical section +to conditionally acquire a sleeping locks (as in mutex_trylock()), +but only as long as it does not loop indefinitely attempting to +conditionally acquire that sleeping locks. +The key point is that things like mutex_trylock() +either return with the mutex held, or return an error indication if +the mutex was not immediately available. +Either way, mutex_trylock() returns immediately without sleeping. +

@@QQE@@ + +

+It often comes as a surprise that many algorithms do not require a +consistent view of data, but many can function in that mode, +with network routing being the poster child. +Internet routing algorithms take significant time to propagate +updates, so that by the time an update arrives at a given system, +that system has been sending network traffic the wrong way for +a considerable length of time. +Having a few threads continue to send traffic the wrong way for a +few more milliseconds is clearly not a problem: In the worst case, +TCP retransmissions will eventually get the data where it needs to go. +In general, when tracking the state of the universe outside of the +computer, some level of inconsistency must be tolerated due to +speed-of-light delays if nothing else. + +

+Furthermore, uncertainty about external state is inherent in many cases. +For example, a pair of veternarians might use heartbeat to determine +whether or not a given cat was alive. +But how long should they wait after the last heartbeat to decide that +the cat is in fact dead? +Waiting less than 400 milliseconds makes no sense because this would +mean that a relaxed cat would be considered to cycle between death +and life more than 100 times per minute. +Moreover, just as with human beings, a cat's heart might stop for +some period of time, so the exact wait period is a judgment call. +One of our pair of veternarians might wait 30 seconds before pronouncing +the cat dead, while the other might insist on waiting a full minute. +The two veternarians would then disagree on the state of the cat during +the final 30 seconds of the minute following the last heartbeat, as +fancifully illustrated below: + +

2013-08-is-it-dead.png

+ +

+Interestingly enough, this same situation applies to hardware. +When push comes to shove, how do we tell whether or not some +external server has failed? +We send messages to it periodically, and declare it failed if we +don't receive a response within a given period of time. +Policy decisions can usually tolerate short +periods of inconsistency. +The policy was decided some time ago, and is only now being put into +effect, so a few milliseconds of delay is normally inconsequential. + +

+However, there are algorithms that absolutely must see consistent data. +For example, the translation between a user-level SystemV semaphore +ID to the corresponding in-kernel data structure is protected by RCU, +but it is absolutely forbidden to update a semaphore that has just been +removed. +In the Linux kernel, this need for consistency is accommodated by acquiring +spinlocks located in the in-kernel data structure from within +the RCU read-side critical section, and this is indicated by the +green box in the figure above. +Many other techniques may be used, and are in fact used within the +Linux kernel. + +

+In short, RCU is not required to maintain consistency, and other +mechanisms may be used in concert with RCU when consistency is required. +RCU's specialization allows it to do its job extremely well, and its +ability to interoperate with other synchronization mechanisms allows +the right mix of synchronization tools to be used for a given job. + +

Performance and Scalability

+ +

+Energy efficiency is a critical component of performance today, +and Linux-kernel RCU implementations must therefore avoid unnecessarily +awakening idle CPUs. +I cannot claim that this requirement was premeditated. +In fact, I learned of it during a telephone conversation in which I +was given “frank and open” feedback on the importance +of energy efficiency in battery-powered systems and on specific +energy-efficiency shortcomings of the Linux-kernel RCU implementation. +In my experience, the battery-powered embedded community will consider +any unnecessary wakeups to be extremely unfriendly acts. +So much so that mere Linux-kernel-mailing-list posts are +insufficient to vent their ire. + +

+Memory consumption is not particularly important for in most +situations, and has become decreasingly +so as memory sizes have expanded and memory +costs have plummeted. +However, as I learned from Matt Mackall's +bloatwatch +efforts, memory footprint is critically important on single-CPU systems with +non-preemptible (CONFIG_PREEMPT=n) kernels, and thus +tiny RCU +was born. +Josh Triplett has since taken over the small-memory banner with his +Linux kernel tinification +project, which resulted in +SRCU +becoming optional for those kernels not needing it. + +

+The remaining performance requirements are, for the most part, +unsurprising. +For example, in keeping with RCU's read-side specialization, +rcu_dereference() should have negligible overhead (for +example, suppression of a few minor compiler optimizations). +Similarly, in non-preemptible environments, rcu_read_lock() and +rcu_read_unlock() should have exactly zero overhead. + +

+In preemptible environments, in the case where the RCU read-side +critical section was not preempted (as will be the case for the +highest-priority real-time process), rcu_read_lock() and +rcu_read_unlock() should have minimal overhead. +In particular, they should not contain atomic read-modify-write +operations, memory-barrier instructions, preemption disabling, +interrupt disabling, or backwards branches. +However, in the case where the RCU read-side critical section was preempted, +rcu_read_unlock() may acquire spinlocks and disable interrupts. +This is why it is better to nest an RCU read-side critical section +within a preempt-disable region than vice versa, at least in cases +where that critical section is short enough to avoid unduly degrading +real-time latencies. + +

+The synchronize_rcu() grace-period-wait primitive is +optimized for throughput. +It may therefore incur several milliseconds of latency in addition to +the duration of the longest RCU read-side critical section. +On the other hand, multiple concurrent invocations of +synchronize_rcu() are required to use batching optimizations +so that they can be satisfied by a single underlying grace-period-wait +operation. +For example, in the Linux kernel, it is not unusual for a single +grace-period-wait operation to serve more than +1,000 separate invocations +of synchronize_rcu(), thus amortizing the per-invocation +overhead down to nearly zero. +However, the grace-period optimization is also required to avoid +measurable degradation of real-time scheduling and interrupt latencies. + +

+In some cases, the multi-millisecond synchronize_rcu() +latencies are unacceptable. +In these cases, synchronize_rcu_expedited() may be used +instead, reducing the grace-period latency down to a few tens of +microseconds on small systems, at least in cases where the RCU read-side +critical sections are short. +There are currently no special latency requirements for +synchronize_rcu_expedited() on large systems, but, +consistent with the empirical nature of the RCU specification, +that is subject to change. +However, there most definitely are scalability requirements: +A storm of synchronize_rcu_expedited() invocations on 4096 +CPUs should at least make reasonable forward progress. +In return for its shorter latencies, synchronize_rcu_expedited() +is permitted to impose modest degradation of real-time latency +on non-idle online CPUs. +That said, it will likely be necessary to take further steps to reduce this +degradation, hopefully to roughly that of a scheduling-clock interrupt. + +

+There are a number of situations where even +synchronize_rcu_expedited()'s reduced grace-period +latency is unacceptable. +In these situations, the asynchronous call_rcu() can be +used in place of synchronize_rcu() as follows: + +

+
+ 1 struct foo {
+ 2   int a;
+ 3   int b;
+ 4   struct rcu_head rh;
+ 5 };
+ 6
+ 7 static void remove_gp_cb(struct rcu_head *rhp)
+ 8 {
+ 9   struct foo *p = container_of(rhp, struct foo, rh);
+10
+11   kfree(p);
+12 }
+13
+14 bool remove_gp_asynchronous(void)
+15 {
+16   struct foo *p;
+17
+18   spin_lock(&gp_lock);
+19   p = rcu_dereference(gp);
+20   if (!p) {
+21     spin_unlock(&gp_lock);
+22     return false;
+23   }
+24   rcu_assign_pointer(gp, NULL);
+25   call_rcu(&p->rh, remove_gp_cb);
+26   spin_unlock(&gp_lock);
+27   return true;
+28 }
+
+
+ +

+A definition of struct foo is finally needed, and appears +on lines 1-5. +The function remove_gp_cb() is passed to call_rcu() +on line 25, and will be invoked after the end of a subsequent +grace period. +This gets the same effect as remove_gp_synchronous(), +but without forcing the updater to wait for a grace period to elapse. +The call_rcu() function may be used in a number of +situations where neither synchronize_rcu() nor +synchronize_rcu_expedited() would be legal, +including within preempt-disable code, local_bh_disable() code, +interrupt-disable code, and interrupt handlers. +However, even call_rcu() is illegal within NMI handlers. +The callback function (remove_gp_cb() in this case) will be +executed within softirq (software interrupt) environment within the +Linux kernel, +either within a real softirq handler or under the protection +of local_bh_disable(). +In both the Linux kernel and in userspace, it is bad practice to +write an RCU callback function that takes too long. +Long-running operations should be relegated to separate threads or +(in the Linux kernel) workqueues. + +

@@QQ@@ +Why does line 19 use rcu_access_pointer()? +After all, call_rcu() on line 25 stores into the +structure, which would interact badly with concurrent insertions. +Doesn't this mean that rcu_dereference() is required? +

@@QQA@@ +Presumably the ->gp_lock acquired on line 18 excludes +any changes, including any insertions that rcu_dereference() +would protect against. +Therefore, any insertions will be delayed until after ->gp_lock +is released on line 25, which in turn means that +rcu_access_pointer() suffices. +

@@QQE@@ + +

+However, all that remove_gp_cb() is doing is +invoking kfree() on the data element. +This is a common idiom, and is supported by kfree_rcu(), +which allows “fire and forget” operation as shown below: + +

+
+ 1 struct foo {
+ 2   int a;
+ 3   int b;
+ 4   struct rcu_head rh;
+ 5 };
+ 6
+ 7 bool remove_gp_faf(void)
+ 8 {
+ 9   struct foo *p;
+10
+11   spin_lock(&gp_lock);
+12   p = rcu_dereference(gp);
+13   if (!p) {
+14     spin_unlock(&gp_lock);
+15     return false;
+16   }
+17   rcu_assign_pointer(gp, NULL);
+18   kfree_rcu(p, rh);
+19   spin_unlock(&gp_lock);
+20   return true;
+21 }
+
+
+ +

+Note that remove_gp_faf() simply invokes +kfree_rcu() and proceeds, without any need to pay any +further attention to the subsequent grace period and kfree(). +It is permissible to invoke kfree_rcu() from the same +environments as for call_rcu(). +Interestingly enough, DYNIX/ptx had the equivalents of +call_rcu() and kfree_rcu(), but not +synchronize_rcu(). +This was due to the fact that RCU was not heavily used within DYNIX/ptx, +so the very few places that needed something like +synchronize_rcu() simply open-coded it. + +

@@QQ@@ +Earlier it was claimed that call_rcu() and +kfree_rcu() allowed updaters to avoid being blocked +by readers. +But how can that be correct, given that the invocation of the callback +and the freeing of the memory (respectively) must still wait for +a grace period to elapse? +

@@QQA@@ +We could define things this way, but keep in mind that this sort of +definition would say that updates in garbage-collected languages +cannot complete until the next time the garbage collector runs, +which does not seem at all reasonable. +The key point is that in most cases, an updater using either +call_rcu() or kfree_rcu() can proceed to the +next update as soon as it has invoked call_rcu() or +kfree_rcu(), without having to wait for a subsequent +grace period. +

@@QQE@@ + +

+But what if the updater must wait for the completion of code to be +executed after the end of the grace period, but has other tasks +that can be carried out in the meantime? +The polling-style get_state_synchronize_rcu() and +cond_synchronize_rcu() functions may be used for this +purpose, as shown below: + +

+
+ 1 bool remove_gp_poll(void)
+ 2 {
+ 3   struct foo *p;
+ 4   unsigned long s;
+ 5
+ 6   spin_lock(&gp_lock);
+ 7   p = rcu_access_pointer(gp);
+ 8   if (!p) {
+ 9     spin_unlock(&gp_lock);
+10     return false;
+11   }
+12   rcu_assign_pointer(gp, NULL);
+13   spin_unlock(&gp_lock);
+14   s = get_state_synchronize_rcu();
+15   do_something_while_waiting();
+16   cond_synchronize_rcu(s);
+17   kfree(p);
+18   return true;
+19 }
+
+
+ +

+On line 14, get_state_synchronize_rcu() obtains a +“cookie” from RCU, +then line 15 carries out other tasks, +and finally, line 16 returns immediately if a grace period has +elapsed in the meantime, but otherwise waits as required. +The need for get_state_synchronize_rcu and +cond_synchronize_rcu() has appeared quite recently, +so it is too early to tell whether they will stand the test of time. + +

+RCU thus provides a range of tools to allow updaters to strike the +required tradeoff between latency, flexibility and CPU overhead. + +

Composability

+ +

+Composability has received much attention in recent years, perhaps in part +due to the collision of multicore hardware with object-oriented techniques +designed in single-threaded environments for single-threaded use. +And in theory, RCU read-side critical sections may be composed, and in +fact may be nested arbitrarily deeply. +In practice, as with all real-world implementations of composable +constructs, there are limitations. + +

+Implementations of RCU for which rcu_read_lock() +and rcu_read_unlock() generate no code, such as +Linux-kernel RCU when CONFIG_PREEMPT=n, can be +nested arbitrarily deeply. +After all, there is no overhead. +Except that if all these instances of rcu_read_lock() +and rcu_read_unlock() are visible to the compiler, +compilation will eventually fail due to exhausting memory, +mass storage, or user patience, whichever comes first. +If the nesting is not visible to the compiler, as is the case with +mutually recursive functions each in its own translation unit, +stack overflow will result. +If the nesting takes the form of loops, either the control variable +will overflow or (in the Linux kernel) you will get an RCU CPU stall warning. +Nevertheless, this class of RCU implementations is one +of the most composable constructs in existence. + +

+RCU implementations that explicitly track nesting depth +are limited by the nesting-depth counter. +For example, the Linux kernel's preemptible RCU limits nesting to +INT_MAX. +This should suffice for almost all practical purposes. +That said, a consecutive pair of RCU read-side critical sections +between which there is an operation that waits for a grace period +cannot be enclosed in another RCU read-side critical section. +This is because it is not legal to wait for a grace period within +an RCU read-side critical section: To do so would result either +in deadlock or +in RCU implicitly splitting the enclosing RCU read-side critical +section, neither of which is conducive to a long-lived and prosperous +kernel. + +

+In short, although RCU read-side critical sections are highly composable, +care is required in some situations, just as is the case for any other +composable synchronization mechanism. + +

Corner Cases

+ +

+A given RCU workload might have an endless and intense stream of +RCU read-side critical sections, perhaps even so intense that there +was never a point in time during which there was not at least one +RCU read-side critical section in flight. +RCU cannot allow this situation to block grace periods: As long as +all the RCU read-side critical sections are finite, grace periods +must also be finite. + +

+That said, preemptible RCU implementations could potentially result +in RCU read-side critical sections being preempted for long durations, +which has the effect of creating a long-duration RCU read-side +critical section. +This situation can arise only in heavily loaded systems, but systems using +real-time priorities are of course more vulnerable. +Therefore, RCU priority boosting is provided to help deal with this +case. +That said, the exact requirements on RCU priority boosting will likely +evolve as more experience accumulates. + +

+Other workloads might have very high update rates. +Although one can argue that such workloads should instead use +something other than RCU, the fact remains that RCU must +handle such workloads gracefully. +This requirement is another factor driving batching of grace periods, +but it is also the driving force behind the checks for large numbers +of queued RCU callbacks in the call_rcu() code path. +Finally, high update rates should not delay RCU read-side critical +sections, although some read-side delays can occur when using +synchronize_rcu_expedited(), courtesy of this function's use +of try_stop_cpus(). +(In the future, synchronize_rcu_expedited() will be +converted to use lighter-weight inter-processor interrupts (IPIs), +but this will still disturb readers, though to a much smaller degree.) + +

+Although all three of these corner cases were understood in the early +1990s, a simple user-level test consisting of close(open(path)) +in a tight loop +in the early 2000s suddenly provided a much deeper appreciation of the +high-update-rate corner case. +This test also motivated addition of some RCU code to react to high update +rates, for example, if a given CPU finds itself with more than 10,000 +RCU callbacks queued, it will cause RCU to take evasive action by +more aggressively starting grace periods and more aggressively forcing +completion of grace-period processing. +This evasive action causes the grace period to complete more quickly, +but at the cost of restricting RCU's batching optimizations, thus +increasing the CPU overhead incurred by that grace period. + +

+Software-Engineering Requirements

+ +

+Between Murphy's Law and “To err is human”, it is necessary to +guard against mishaps and misuse: + +

    +
  1. It is all too easy to forget to use rcu_read_lock() + everywhere that it is needed, so kernels built with + CONFIG_PROVE_RCU=y will spat if + rcu_dereference() is used outside of an + RCU read-side critical section. + Update-side code can use rcu_dereference_protected(), + which takes a + lockdep expression + to indicate what is providing the protection. + If the indicated protection is not provided, a lockdep splat + is emitted. + +

    + Code shared between readers and updaters can use + rcu_dereference_check(), which also takes a + lockdep expression, and emits a lockdep splat if neither + rcu_read_lock() nor the indicated protection + is in place. + In addition, rcu_dereference_raw() is used in those + (hopefully rare) cases where the required protection cannot + be easily described. + Finally, rcu_read_lock_held() is provided to + allow a function to verify that it has been invoked within + an RCU read-side critical section. + I was made aware of this set of requirements shortly after Thomas + Gleixner audited a number of RCU uses. +

  2. A given function might wish to check for RCU-related preconditions + upon entry, before using any other RCU API. + The rcu_lockdep_assert() does this job, + asserting the expression in kernels having lockdep enabled + and doing nothing otherwise. +
  3. It is also easy to forget to use rcu_assign_pointer() + and rcu_dereference(), perhaps (incorrectly) + substituting a simple assignment. + To catch this sort of error, a given RCU-protected pointer may be + tagged with __rcu, after which running sparse + with CONFIG_SPARSE_RCU_POINTER=y will complain + about simple-assignment accesses to that pointer. + Arnd Bergmann made me aware of this requirement, and also + supplied the needed + patch series. +
  4. Kernels built with CONFIG_DEBUG_OBJECTS_RCU_HEAD=y + will splat if a data element is passed to call_rcu() + twice in a row, without a grace period in between. + (This error is similar to a double free.) + The corresponding rcu_head structures that are + dynamically allocated are automatically tracked, but + rcu_head structures allocated on the stack + must be initialized with init_rcu_head_on_stack() + and cleaned up with destroy_rcu_head_on_stack(). + Similarly, statically allocated non-stack rcu_head + structures must be initialized with init_rcu_head() + and cleaned up with destroy_rcu_head(). + Mathieu Desnoyers made me aware of this requirement, and also + supplied the needed + patch. +
  5. An infinite loop in an RCU read-side critical section will + eventually trigger an RCU CPU stall warning splat. + However, RCU is not obligated to produce this splat + unless there is a grace period waiting on that particular + RCU read-side critical section. + This requirement made itself known in the early 1990s, pretty + much the first time that it was necessary to debug a CPU stall. +
  6. Although it would be very good to detect pointers leaking out + of RCU read-side critical sections, there is currently no + good way of doing this. + One complication is the need to distinguish between pointers + leaking and pointers that have been handed off from RCU to + some other synchronization mechanism, for example, reference + counting. +
  7. In kernels built with CONFIG_RCU_TRACE=y, RCU-related + information is provided via both debugfs and event tracing. +
  8. Open-coded use of rcu_assign_pointer() and + rcu_dereference() to create typical linked + data structures can be surprisingly error-prone. + Therefore, RCU-protected + linked lists + and, more recently, RCU-protected + hash tables + are available. + Many other special-purpose RCU-protected data structures are + available in the Linux kernel and the userspace RCU library. +
  9. Some linked structures are created at compile time, but still + require __rcu checking. + The RCU_POINTER_INITIALIZER() macro serves this + purpose. +
  10. It is not necessary to use rcu_assign_pointer() + when creating linked structures that are to be published via + a single external pointer. + The RCU_INIT_POINTER() macro is provided for + this task and also for assigning NULL pointers + at runtime. +
+ +

+This not a hard-and-fast list: RCU's diagnostic capabilities will +continue to be guided by the number and type of usage bugs found +in real-world RCU usage. + +

Linux Kernel Complications

+ +

+The Linux kernel provides an interesting environment for all kinds of +software, including RCU. +Some of the relevant points of interest are as follows: + +

    +
  1. Configuration. +
  2. Firmware Interface. +
  3. Early Boot. +
  4. + Interrupts and non-maskable interrupts (NMIs). +
  5. Loadable Modules. +
  6. Hotplug CPU. +
  7. Scheduler and RCU. +
  8. Tracing and RCU. +
  9. Energy Efficiency. +
  10. + Performance, Scalability, Response Time, and Reliability. +
+ +

+This list is probably incomplete, but it does give a feel for the +most notable Linux-kernel complications. +Each of the following sections covers one of the above topics. + +

Configuration

+ +

+RCU's goal is automatic configuration, so that almost nobody +needs to worry about RCU's Kconfig options. +And for almost all users, RCU does in fact work well +“out of the box.” + +

+However, there are specialized use cases that are handled by +kernel boot parameters and Kconfig options. +Unfortunately, the Kconfig system will explicitly ask users +about new Kconfig options, which requires almost all of them +be hidden behind a CONFIG_RCU_EXPERT Kconfig option. + +

+This all should be quite obvious, but the fact remains that +Linus Torvalds recently had to +remind +me of this requirement. + +

Firmware Interface

+ +

+In many cases, kernel obtains information about the system from the +firmware, and sometimes things are lost in translation. +Or the translation is accurate, but the original message is bogus. + +

+For example, some systems' firmware overreports the number of CPUs, +sometimes by a large factor. +If RCU naively believed the firmware, as it used to do, +it would create too many per-CPU kthreads. +Although the resulting system will still run correctly, the extra +kthreads needlessly consume memory and can cause confusion +when they show up in ps listings. + +

+RCU must therefore wait for a given CPU to actually come online before +it can allow itself to believe that the CPU actually exists. +The resulting “ghost CPUs” (which are never going to +come online) cause a number of +interesting complications. + +

Early Boot

+ +

+The Linux kernel's boot sequence is an interesting process, +and RCU is used early, even before rcu_init() +is invoked. +In fact, a number of RCU's primitives can be used as soon as the +initial task's task_struct is available and the +boot CPU's per-CPU variables are set up. +The read-side primitives (rcu_read_lock(), +rcu_read_unlock(), rcu_dereference(), +and rcu_access_pointer()) will operate normally very early on, +as will rcu_assign_pointer(). + +

+Although call_rcu() may be invoked at any +time during boot, callbacks are not guaranteed to be invoked until after +the scheduler is fully up and running. +This delay in callback invocation is due to the fact that RCU does not +invoke callbacks until it is fully initialized, and this full initialization +cannot occur until after the scheduler has initialized itself to the +point where RCU can spawn and run its kthreads. +In theory, it would be possible to invoke callbacks earlier, +however, this is not a panacea because there would be severe restrictions +on what operations those callbacks could invoke. + +

+Perhaps surprisingly, synchronize_rcu(), +synchronize_rcu_bh() +(discussed below), +and +synchronize_sched() +will all operate normally +during very early boot, the reason being that there is only one CPU +and preemption is disabled. +This means that the call synchronize_rcu() (or friends) +itself is a quiescent +state and thus a grace period, so the early-boot implementation can +be a no-op. + +

+Both synchronize_rcu_bh() and synchronize_sched() +continue to operate normally through the remainder of boot, courtesy +of the fact that preemption is disabled across their RCU read-side +critical sections and also courtesy of the fact that there is still +only one CPU. +However, once the scheduler starts initializing, preemption is enabled. +There is still only a single CPU, but the fact that preemption is enabled +means that the no-op implementation of synchronize_rcu() no +longer works in CONFIG_PREEMPT=y kernels. +Therefore, as soon as the scheduler starts initializing, the early-boot +fastpath is disabled. +This means that synchronize_rcu() switches to its runtime +mode of operation where it posts callbacks, which in turn means that +any call to synchronize_rcu() will block until the corresponding +callback is invoked. +Unfortunately, the callback cannot be invoked until RCU's runtime +grace-period machinery is up and running, which cannot happen until +the scheduler has initialized itself sufficiently to allow RCU's +kthreads to be spawned. +Therefore, invoking synchronize_rcu() during scheduler +initialization can result in deadlock. + +

@@QQ@@ +So what happens with synchronize_rcu() during +scheduler initialization for CONFIG_PREEMPT=n +kernels? +

@@QQA@@ +In CONFIG_PREEMPT=n kernel, synchronize_rcu() +maps directly to synchronize_sched(). +Therefore, synchronize_rcu() works normally throughout +boot in CONFIG_PREEMPT=n kernels. +However, your code must also work in CONFIG_PREEMPT=y kernels, +so it is still necessary to avoid invoking synchronize_rcu() +during scheduler initialization. +

@@QQE@@ + +

+I learned of these boot-time requirements as a result of a series of +system hangs. + +

Interrupts and NMIs

+ +

+The Linux kernel has interrupts, and RCU read-side critical sections are +legal within interrupt handlers and within interrupt-disabled regions +of code, as are invocations of call_rcu(). + +

+Some Linux-kernel architectures can enter an interrupt handler from +non-idle process context, and then just never leave it, instead stealthily +transitioning back to process context. +This trick is sometimes used to invoke system calls from inside the kernel. +These “half-interrupts” mean that RCU has to be very careful +about how it counts interrupt nesting levels. +I learned of this requirement the hard way during a rewrite +of RCU's dyntick-idle code. + +

+The Linux kernel has non-maskable interrupts (NMIs), and +RCU read-side critical sections are legal within NMI handlers. +Thankfully, RCU update-side primitives, including +call_rcu(), are prohibited within NMI handlers. + +

+The name notwithstanding, some Linux-kernel architectures +can have nested NMIs, which RCU must handle correctly. +Andy Lutomirski +surprised me +with this requirement; +he also kindly surprised me with +an algorithm +that meets this requirement. + +

Loadable Modules

+ +

+The Linux kernel has loadable modules, and these modules can +also be unloaded. +After a given module has been unloaded, any attempt to call +one of its functions results in a segmentation fault. +The module-unload functions must therefore cancel any +delayed calls to loadable-module functions, for example, +any outstanding mod_timer() must be dealt with +via del_timer_sync() or similar. + +

+Unfortunately, there is no way to cancel an RCU callback; +once you invoke call_rcu(), the callback function is +going to eventually be invoked, unless the system goes down first. +Because it is normally considered socially irresponsible to crash the system +in response to a module unload request, we need some other way +to deal with in-flight RCU callbacks. + +

+RCU therefore provides +rcu_barrier(), +which waits until all in-flight RCU callbacks have been invoked. +If a module uses call_rcu(), its exit function should therefore +prevent any future invocation of call_rcu(), then invoke +rcu_barrier(). +In theory, the underlying module-unload code could invoke +rcu_barrier() unconditionally, but in practice this would +incur unacceptable latencies. + +

+Nikita Danilov noted this requirement for an analogous filesystem-unmount +situation, and Dipankar Sarma incorporated rcu_barrier() into RCU. +The need for rcu_barrier() for module unloading became +apparent later. + +

Hotplug CPU

+ +

+The Linux kernel supports CPU hotplug, which means that CPUs +can come and go. +It is of course illegal to use any RCU API member from an offline CPU. +This requirement was present from day one in DYNIX/ptx, but +on the other hand, the Linux kernel's CPU-hotplug implementation +is “interesting.” + +

+The Linux-kernel CPU-hotplug implementation has notifiers that +are used to allow the various kernel subsystems (including RCU) +to respond appropriately to a given CPU-hotplug operation. +Most RCU operations may be invoked from CPU-hotplug notifiers, +including even normal synchronous grace-period operations +such as synchronize_rcu(). +However, expedited grace-period operations such as +synchronize_rcu_expedited() are not supported, +due to the fact that current implementations block CPU-hotplug +operations, which could result in deadlock. + +

+In addition, all-callback-wait operations such as +rcu_barrier() are also not supported, due to the +fact that there are phases of CPU-hotplug operations where +the outgoing CPU's callbacks will not be invoked until after +the CPU-hotplug operation ends, which could also result in deadlock. + +

Scheduler and RCU

+ +

+RCU depends on the scheduler, and the scheduler uses RCU to +protect some of its data structures. +This means the scheduler is forbidden from acquiring +the runqueue locks and the priority-inheritance locks +in the middle of an outermost RCU read-side critical section unless +it also releases them before exiting that same +RCU read-side critical section. +This same prohibition also applies to any lock that is acquired +while holding any lock to which this prohibition applies. +Violating this rule results in deadlock. + +

+For RCU's part, the preemptible-RCU rcu_read_unlock() +implementation must be written carefully to avoid similar deadlocks. +In particular, rcu_read_unlock() must tolerate an +interrupt where the interrupt handler invokes both +rcu_read_lock() and rcu_read_unlock(). +This possibility requires rcu_read_unlock() to use +negative nesting levels to avoid destructive recursion via +interrupt handler's use of RCU. + +

+This pair of mutual scheduler-RCU requirements came as a +complete surprise. + +

+As noted above, RCU makes use of kthreads, and it is necessary to +avoid excessive CPU-time accumulation by these kthreads. +This requirement was no surprise, but RCU's violation of it +when running context-switch-heavy workloads when built with +CONFIG_NO_HZ_FULL=y +did come as a surprise [PDF]. +RCU has made good progress towards meeting this requirement, even +for context-switch-have CONFIG_NO_HZ_FULL=y workloads, +but there is room for further improvement. + +

Tracing and RCU

+ +

+It is possible to use tracing on RCU code, but tracing itself +uses RCU. +For this reason, rcu_dereference_raw_notrace() +is provided for use by tracing, which avoids the destructive +recursion that could otherwise ensue. +This API is also used by virtualization in some architectures, +where RCU readers execute in environments in which tracing +cannot be used. +The tracing folks both located the requirement and provided the +needed fix, so this surprise requirement was relatively painless. + +

Energy Efficiency

+ +

+Interrupting idle CPUs is considered socially unacceptable, +especially by people with battery-powered embedded systems. +RCU therefore conserves energy by detecting which CPUs are +idle, including tracking CPUs that have been interrupted from idle. +This is a large part of the energy-efficiency requirement, +so I learned of this via an irate phone call. + +

+Because RCU avoids interrupting idle CPUs, it is illegal to +execute an RCU read-side critical section on an idle CPU. +(Kernels built with CONFIG_PROVE_RCU=y will splat +if you try it.) +The RCU_NONIDLE() macro and _rcuidle +event tracing is provided to work around this restriction. +In addition, rcu_is_watching() may be used to +test whether or not it is currently legal to run RCU read-side +critical sections on this CPU. +I learned of the need for diagnostics on the one hand +and RCU_NONIDLE() on the other while inspecting +idle-loop code. +Steven Rostedt supplied _rcuidle event tracing, +which is used quite heavily in the idle loop. + +

+It is similarly socially unacceptable to interrupt an +nohz_full CPU running in userspace. +RCU must therefore track nohz_full userspace +execution. +And in +CONFIG_NO_HZ_FULL_SYSIDLE=y +kernels, RCU must separately track idle CPUs on the one hand and +CPUs that are either idle or executing in userspace on the other. +In both cases, RCU must be able to sample state at two points in +time, and be able to determine whether or not some other CPU spent +any time idle and/or executing in userspace. + +

+These energy-efficiency requirements have proven quite difficult to +understand and to meet, for example, there have been more than five +clean-sheet rewrites of RCU's energy-efficiency code, the last of +which was finally able to demonstrate +real energy savings running on real hardware [PDF]. +As noted earlier, +I learned of many of these requirements via angry phone calls: +Flaming me on the Linux-kernel mailing list was apparently not +sufficient to fully vent their ire at RCU's energy-efficiency bugs! + +

+Performance, Scalability, Response Time, and Reliability

+ +

+Expanding on the +earlier discussion, +RCU is used heavily by hot code paths in performance-critical +portions of the Linux kernel's networking, security, virtualization, +and scheduling code paths. +RCU must therefore use efficient implementations, especially in its +read-side primitives. +To that end, it would be good if preemptible RCU's implementation +of rcu_read_lock() could be inlined, however, doing +this requires resolving #include issues with the +task_struct structure. + +

+The Linux kernel supports hardware configurations with up to +4096 CPUs, which means that RCU must be extremely scalable. +Algorithms that involve frequent acquisitions of global locks or +frequent atomic operations on global variables simply cannot be +tolerated within the RCU implementation. +RCU therefore makes heavy use of a combining tree based on the +rcu_node structure. +RCU is required to tolerate all CPUs continuously invoking any +combination of RCU's runtime primitives with minimal per-operation +overhead. +In fact, in many cases, increasing load must decrease the +per-operation overhead, witness the batching optimizations for +synchronize_rcu(), call_rcu(), +synchronize_rcu_expedited(), and rcu_barrier(). +As a general rule, RCU must cheerfully accept whatever the +rest of the Linux kernel decides to throw at it. + +

+The Linux kernel is used for real-time workloads, especially +in conjunction with the +-rt patchset. +The real-time-latency response requirements are such that the +traditional approach of disabling preemption across RCU +read-side critical sections is inappropriate. +Kernels built with CONFIG_PREEMPT=y therefore +use an RCU implementation that allows RCU read-side critical +sections to be preempted. +This requirement made its presence known after users made it +clear that an earlier +real-time patch +did not meet their needs, in conjunction with some +RCU issues +encountered by a very early version of the -rt patchset. + +

+In addition, RCU must make do with a sub-100-microsecond real-time latency +budget. +In fact, on smaller systems with the -rt patchset, the Linux kernel +provides sub-20-microsecond real-time latencies for the whole kernel, +including RCU. +RCU's scalability and latency must therefore be sufficient for +these sorts of configurations. +To my surprise, the sub-100-microsecond real-time latency budget + +applies to even the largest systems [PDF], +up to and including systems with 4096 CPUs. +This real-time requirement motivated the grace-period kthread, which +also simplified handling of a number of race conditions. + +

+Finally, RCU's status as a synchronization primitive means that +any RCU failure can result in arbitrary memory corruption that can be +extremely difficult to debug. +This means that RCU must be extremely reliable, which in +practice also means that RCU must have an aggressive stress-test +suite. +This stress-test suite is called rcutorture. + +

+Although the need for rcutorture was no surprise, +the current immense popularity of the Linux kernel is posing +interesting—and perhaps unprecedented—validation +challenges. +To see this, keep in mind that there are well over one billion +instances of the Linux kernel running today, given Android +smartphones, Linux-powered televisions, and servers. +This number can be expected to increase sharply with the advent of +the celebrated Internet of Things. + +

+Suppose that RCU contains a race condition that manifests on average +once per million years of runtime. +This bug will be occurring about three times per day across +the installed base. +RCU could simply hide behind hardware error rates, given that no one +should really expect their smartphone to last for a million years. +However, anyone taking too much comfort from this thought should +consider the fact that in most jurisdictions, a successful multi-year +test of a given mechanism, which might include a Linux kernel, +suffices for a number of types of safety-critical certifications. +In fact, rumor has it that the Linux kernel is already being used +in production for safety-critical applications. +I don't know about you, but I would feel quite bad if a bug in RCU +killed someone. +Which might explain my recent focus on validation and verification. + +

Other RCU Flavors

+ +

+One of the more surprising things about RCU is that there are now +no fewer than five flavors, or API families. +In addition, the primary flavor that has been the sole focus up to +this point has two different implementations, non-preemptible and +preemptible. +The other four flavors are listed below, with requirements for each +described in a separate section. + +

    +
  1. Bottom-Half Flavor +
  2. Sched Flavor +
  3. Sleepable RCU +
  4. Tasks RCU +
+ +

Bottom-Half Flavor

+ +

+The softirq-disable (AKA “bottom-half”, +hence the “_bh” abbreviations) +flavor of RCU, or RCU-bh, was developed by +Dipankar Sarma to provide a flavor of RCU that could withstand the +network-based denial-of-service attacks researched by Robert +Olsson. +These attacks placed so much networking load on the system +that some of the CPUs never exited softirq execution, +which in turn prevented those CPUs from ever executing a context switch, +which, in the RCU implementation of that time, prevented grace periods +from ever ending. +The result was an out-of-memory condition and a system hang. + +

+The solution was the creation of RCU-bh, which does +local_bh_disable() +across its read-side critical sections, and which uses the transition +from one type of softirq processing to another as a quiescent state +in addition to context switch, idle, user mode, and offline. +This means that RCU-bh grace periods can complete even when some of +the CPUs execute in softirq indefinitely, thus allowing algorithms +based on RCU-bh to withstand network-based denial-of-service attacks. + +

+Because +rcu_read_lock_bh() and rcu_read_unlock_bh() +disable and re-enable softirq handlers, any attempt to start a softirq +handlers during the +RCU-bh read-side critical section will be deferred. +In this case, rcu_read_unlock_bh() +will invoke softirq processing, which can take considerable time. +One can of course argue that this softirq overhead should be associated +with the code following the RCU-bh read-side critical section rather +than rcu_read_unlock_bh(), but the fact +is that most profiling tools cannot be expected to make this sort +of fine distinction. +For example, suppose that a three-millisecond-long RCU-bh read-side +critical section executes during a time of heavy networking load. +There will very likely be an attempt to invoke at least one softirq +handler during that three milliseconds, but any such invocation will +be delayed until the time of the rcu_read_unlock_bh(). +This can of course make it appear at first glance as if +rcu_read_unlock_bh() was executing very slowly. + +

+The +RCU-bh API +includes +rcu_read_lock_bh(), +rcu_read_unlock_bh(), +rcu_dereference_bh(), +rcu_dereference_bh_check(), +synchronize_rcu_bh(), +synchronize_rcu_bh_expedited(), +call_rcu_bh(), +rcu_barrier_bh(), and +rcu_read_lock_bh_held(). + +

Sched Flavor

+ +

+Before preemptible RCU, waiting for an RCU grace period had the +side effect of also waiting for all pre-existing interrupt +and NMI handlers. +However, there are legitimate preemptible-RCU implementations that +do not have this property, given that any point in the code outside +of an RCU read-side critical section can be a quiescent state. +Therefore, RCU-sched was created, which follows “classic” +RCU in that an RCU-sched grace period waits for for pre-existing +interrupt and NMI handlers. +In kernels built with CONFIG_PREEMPT=n, the RCU and RCU-sched +APIs have identical implementations, while kernels built with +CONFIG_PREEMPT=y provide a separate implementation for each. + +

+Note well that in CONFIG_PREEMPT=y kernels, +rcu_read_lock_sched() and rcu_read_unlock_sched() +disable and re-enable preemption, respectively. +This means that if there was a preemption attempt during the +RCU-sched read-side critical section, rcu_read_unlock_sched() +will enter the scheduler, with all the latency and overhead entailed. +Just as with rcu_read_unlock_bh(), this can make it look +as if rcu_read_unlock_sched() was executing very slowly. +However, the highest-priority task won't be preempted, so that task +will enjoy low-overhead rcu_read_unlock_sched() invocations. + +

+The +RCU-sched API +includes +rcu_read_lock_sched(), +rcu_read_unlock_sched(), +rcu_read_lock_sched_notrace(), +rcu_read_unlock_sched_notrace(), +rcu_dereference_sched(), +rcu_dereference_sched_check(), +synchronize_sched(), +synchronize_rcu_sched_expedited(), +call_rcu_sched(), +rcu_barrier_sched(), and +rcu_read_lock_sched_held(). +However, anything that disables preemption also marks an RCU-sched +read-side critical section, including +preempt_disable() and preempt_enable(), +local_irq_save() and local_irq_restore(), +and so on. + +

Sleepable RCU

+ +

+For well over a decade, someone saying “I need to block within +an RCU read-side critical section” was a reliable indication +that this someone did not understand RCU. +After all, if you are always blocking in an RCU read-side critical +section, you can probably afford to use a higher-overhead synchronization +mechanism. +However, that changed with the advent of the Linux kernel's notifiers, +whose RCU read-side critical +sections almost never sleep, but sometimes need to. +This resulted in the introduction of +sleepable RCU, +or SRCU. + +

+SRCU allows different domains to be defined, with each such domain +defined by an instance of an srcu_struct structure. +A pointer to this structure must be passed in to each SRCU function, +for example, synchronize_srcu(&ss), where +ss is the srcu_struct structure. +The key benefit of these domains is that a slow SRCU reader in one +domain does not delay an SRCU grace period in some other domain. +That said, one consequence of these domains is that read-side code +must pass a “cookie” from srcu_read_lock() +to srcu_read_unlock(), for example, as follows: + +

+
+ 1 int idx;
+ 2
+ 3 idx = srcu_read_lock(&ss);
+ 4 do_something();
+ 5 srcu_read_unlock(&ss, idx);
+
+
+ +

+As noted above, it is legal to block within SRCU read-side critical sections, +however, with great power comes great responsibility. +If you block forever in one of a given domain's SRCU read-side critical +sections, then that domain's grace periods will also be blocked forever. +Of course, one good way to block forever is to deadlock, which can +happen if any operation in a given domain's SRCU read-side critical +section can block waiting, either directly or indirectly, for that domain's +grace period to elapse. +For example, this results in a self-deadlock: + +

+
+ 1 int idx;
+ 2
+ 3 idx = srcu_read_lock(&ss);
+ 4 do_something();
+ 5 synchronize_srcu(&ss);
+ 6 srcu_read_unlock(&ss, idx);
+
+
+ +

+However, if line 5 acquired a mutex that was held across +a synchronize_srcu() for domain ss, +deadlock would still be possible. +Furthermore, if line 5 acquired a mutex that was held across +a synchronize_srcu() for some other domain ss1, +and if an ss1-domain SRCU read-side critical section +acquired another mutex that was held across as ss-domain +synchronize_srcu(), +deadlock would again be possible. +Such a deadlock cycle could extend across an arbitrarily large number +of different SRCU domains. +Again, with great power comes great responsibility. + +

+Unlike the other RCU flavors, SRCU read-side critical sections can +run on idle and even offline CPUs. +This ability requires that srcu_read_lock() and +srcu_read_unlock() contain memory barriers, which means +that SRCU readers will run a bit slower than would RCU readers. +It also motivates the smp_mb__after_srcu_read_unlock() +API, which, in combination with srcu_read_unlock(), +guarantees a full memory barrier. + +

+The +SRCU API +includes +srcu_read_lock(), +srcu_read_unlock(), +srcu_dereference(), +srcu_dereference_check(), +synchronize_srcu(), +synchronize_srcu_expedited(), +call_srcu(), +srcu_barrier(), and +srcu_read_lock_held(). +It also includes +DEFINE_SRCU(), +DEFINE_STATIC_SRCU(), and +init_srcu_struct() +APIs for defining and initializing srcu_struct structures. + +

Tasks RCU

+ +

+Some forms of tracing use “tramopolines” to handle the +binary rewriting required to install different types of probes. +It would be good to be able to free old trampolines, which sounds +like a job for some form of RCU. +However, because it is necessary to be able to install a trace +anywhere in the code, it is not possible to use read-side markers +such as rcu_read_lock() and rcu_read_unlock(). +In addition, it does not work to have these markers in the trampoline +itself, because there would need to be instructions following +rcu_read_unlock(). +Although synchronize_rcu() would guarantee that execution +reached the rcu_read_unlock(), it would not be able to +guarantee that execution had completely left the trampoline. + +

+The solution, in the form of +Tasks RCU, +is to have implicit +read-side critical sections that are delimited by voluntary context +switches, that is, calls to schedule(), +cond_resched_rcu_qs(), and +synchronize_rcu_tasks(). +In addition, transitions to and from userspace execution also delimit +tasks-RCU read-side critical sections. + +

+The tasks-RCU API is quite compact, consisting only of +call_rcu_tasks(), +synchronize_rcu_tasks(), and +rcu_barrier_tasks(). + +

Possible Future Changes

+ +

+One of the tricks that RCU uses to attain update-side scalability is +to increase grace-period latency with increasing numbers of CPUs. +If this becomes a serious problem, it will be necessary to rework the +grace-period state machine so as to avoid the need for the additional +latency. + +

+Expedited grace periods scan the CPUs, so their latency and overhead +increases with increasing numbers of CPUs. +If this becomes a serious problem on large systems, it will be necessary +to do some redesign to avoid this scalability problem. + +

+RCU disables CPU hotplug in a few places, perhaps most notably in the +expedited grace-period and rcu_barrier() operations. +If there is a strong reason to use expedited grace periods in CPU-hotplug +notifiers, it will be necessary to avoid disabling CPU hotplug. +This would introduce some complexity, so there had better be a very +good reason. + +

+The tradeoff between grace-period latency on the one hand and interruptions +of other CPUs on the other hand may need to be re-examined. +The desire is of course for zero grace-period latency as well as zero +interprocessor interrupts undertaken during an expedited grace period +operation. +While this ideal is unlikely to be achievable, it is quite possible that +further improvements can be made. + +

+The multiprocessor implementations of RCU use a combining tree that +groups CPUs so as to reduce lock contention and increase cache locality. +However, this combining tree does not spread its memory across NUMA +nodes nor does it align the CPU groups with hardware features such +as sockets or cores. +Such spreading and alignment is currently believed to be unnecessary +because the hotpath read-side primitives do not access the combining +tree, nor does call_rcu() in the common case. +If you believe that your architecture needs such spreading and alignment, +then your architecture should also benefit from the +rcutree.rcu_fanout_leaf boot parameter, which can be set +to the number of CPUs in a socket, NUMA node, or whatever. +If the number of CPUs is too large, use a fraction of the number of +CPUs. +If the number of CPUs is a large prime number, well, that certainly +is an “interesting” architectural choice! +More flexible arrangements might be considered, but only if +rcutree.rcu_fanout_leaf has proven inadequate, and only +if the inadequacy has been demonstrated by a carefully run and +realistic system-level workload. + +

+Please note that arrangements that require RCU to remap CPU numbers will +require extremely good demonstration of need and full exploration of +alternatives. + +

+There is an embarrassingly large number of flavors of RCU, and this +number has been increasing over time. +Perhaps it will be possible to combine some at some future date. + +

+RCU's various kthreads are reasonably recent additions. +It is quite likely that adjustments will be required to more gracefully +handle extreme loads. +It might also be necessary to be able to relate CPU utilization by +RCU's kthreads and softirq handlers to the code that instigated this +CPU utilization. +For example, RCU callback overhead might be charged back to the +originating call_rcu() instance, though probably not +in production kernels. + +

Summary

+ +

+This document has presented more than two decade's worth of RCU +requirements. +Given that the requirements keep changing, this will not be the last +word on this subject, but at least it serves to get an important +subset of the requirements set forth. + +

Acknowledgments

+ +I am grateful to Steven Rostedt, Lai Jiangshan, Ingo Molnar, +Oleg Nesterov, Borislav Petkov, Peter Zijlstra, Boqun Feng, and +Andy Lutomirski for their help in rendering +this article human readable, and to Michelle Rankin for her support +of this effort. +Other contributions are acknowledged in the Linux kernel's git archive. +The cartoon is copyright (c) 2013 by Melissa Broussard, +and is provided +under the terms of the Creative Commons Attribution-Share Alike 3.0 +United States license. + +

@@QQAL@@ + + diff --git a/Documentation/RCU/Design/htmlqqz.sh b/Documentation/RCU/Design/htmlqqz.sh new file mode 100755 index 0000000000000..d354f069559b8 --- /dev/null +++ b/Documentation/RCU/Design/htmlqqz.sh @@ -0,0 +1,108 @@ +#!/bin/sh +# +# Usage: sh htmlqqz.sh file +# +# Extracts and converts quick quizzes in a proto-HTML document file.htmlx. +# Commands, all of which must be on a line by themselves: +# +# "

@@QQ@@": Start of a quick quiz. +# "

@@QQA@@": Start of a quick-quiz answer. +# "

@@QQE@@": End of a quick-quiz answer, and thus of the quick quiz. +# "

@@QQAL@@": Place to put quick-quiz answer list. +# +# Places the result in file.html. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, you can access it online at +# http://www.gnu.org/licenses/gpl-2.0.html. +# +# Copyright (c) 2013 Paul E. McKenney, IBM Corporation. + +fn=$1 +if test ! -r $fn.htmlx +then + echo "Error: $fn.htmlx unreadable." + exit 1 +fi + +echo "" > $fn.html +echo "" >> $fn.html +awk < $fn.htmlx >> $fn.html ' + +state == "" && $1 != "

@@QQ@@" && $1 != "

@@QQAL@@" { + print $0; + if ($0 ~ /^

@@QQ/) + print "Bad Quick Quiz command: " NR " (expected

@@QQ@@ or

@@QQAL@@)." > "/dev/stderr" + next; +} + +state == "" && $1 == "

@@QQ@@" { + qqn++; + qqlineno = NR; + haveqq = 1; + state = "qq"; + print "

Quick Quiz " qqn ":" + next; +} + +state == "qq" && $1 != "

@@QQA@@" { + qq[qqn] = qq[qqn] $0 "\n"; + print $0 + if ($0 ~ /^

@@QQ/) + print "Bad Quick Quiz command: " NR ". (expected

@@QQA@@)" > "/dev/stderr" + next; +} + +state == "qq" && $1 == "

@@QQA@@" { + state = "qqa"; + print "
Answer" + next; +} + +state == "qqa" && $1 != "

@@QQE@@" { + qqa[qqn] = qqa[qqn] $0 "\n"; + if ($0 ~ /^

@@QQ/) + print "Bad Quick Quiz command: " NR " (expected

@@QQE@@)." > "/dev/stderr" + next; +} + +state == "qqa" && $1 == "

@@QQE@@" { + state = ""; + next; +} + +state == "" && $1 == "

@@QQAL@@" { + haveqq = ""; + print "

" + print "Answers to Quick Quizzes

" + print ""; + for (i = 1; i <= qqn; i++) { + print "" + print "

Quick Quiz " i ":" + print qq[i]; + print ""; + print "

Answer:" + print qqa[i]; + print ""; + print "

Back to Quick Quiz " i "." + print ""; + } + next; +} + +END { + if (state != "") + print "Unterminated Quick Quiz: " qqlineno "." > "/dev/stderr" + else if (haveqq) + print "Missing \"

@@QQAL@@\", no Quick Quiz." > "/dev/stderr" +}' -- 2.47.2