tag:blogger.com,1999:blog-35942276055473715582024-02-06T21:16:52.153-05:00VirtualBeangeeks + virtualization + coffeeCraig Herringhttp://www.blogger.com/profile/05719502208585022033noreply@blogger.comBlogger14125tag:blogger.com,1999:blog-3594227605547371558.post-12162969657057131162020-06-03T16:27:00.000-04:002020-06-04T11:36:27.603-04:00Storage and VMware vSAN design tipsThese days storage, even local storage, is more complex to understand with all the different options. These options include everything from Storage Class Memory to spinning disks. So this begs the question, "how do we choose what to attach to our servers?" Companies like Dell with it's VxRail product do provide a jointly engineered solution, so no matter what your requirements are an architecture can reliably be created. If your use case if for VDI, common server workload, or databases with heavy I/O, a solution can be created with success. Solutions like ready-nodes or simply picking parts off the VMware HCL are good solutions, however the success of the solution is up to the engineering prowess of the architect.<br />
<div>
<br /></div>
<div>
Storage is one of those critical pieces of infrastructure. It is the last chain in the data path from where we can listen to downloaded music, view favorite family and holiday photos, run that app on a daily basis. If a CPU or a memory stick dies or even if a network cable breaks typically no data is actually lost. However, if a drive dies all of our memories and productivity for at least that day is gone.</div>
<div>
<br /></div>
<div>
Desktops were typically backed up to some external tape or disk. Typically today, backups are sent to some type of remote or cloud resource. For servers, these can use larger variants of these resources however because more risk and expense can come with failed hardware. A little extra caution and effort placed when considering storage architecture. This includes the quality and built-in redundancy of the designs.</div>
<div>
<br /></div>
<div>
The other consideration is performance. Because of the number of drive choices, when evaluating performance and reliability, we have many different types of drives to consider. Our desktops and laptops typically use SSDs or NVMEs and now servers are typically designed with these. Considering performance below is a memory and drive performance table that displays latency with the 'human relatable' translation. ( #geeks #> <b>ls -lh </b>) Most of this information was retrieved from Frank Denneman - <a href="https://frankdenneman.nl/2019/10/14/amd-epyc-naples-vs-rome-and-vsphere-cpu-scheduler-updates/" target="_blank">AMD EPYC Naples vs Rome and vSphere CPU Scheduler Updates</a>. I like how he correlated from 1 CPU cycle all the way to a SSD I/O. I added a typical 15K disk drive for additional impact on the comparison.</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody>
<tr><td style="text-align: center;"><img alt="" src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAeoAAAEFCAYAAAAsf6RtAAAgAElEQVR4Ae19O24sTY+l9tOWjHG0hd9vme3JmB38WsC0K7vltdkLEDACZgnjXaAh3D3MFmrixYhDBiMyq1SPrKwjQF9mxoNBHh6SEVl19T39n//+fwf+EgNygBwgB8gBcmBbHDiUnyc6ZluOoT/oD3KAHCAHyIHIAflhoeYbBb5RIQfIAXKAHNggB1ioN+gU7qK5iyYHyAFygBwQDrBQs1BzB00OkAPkADmwYQ6wUG/YObKb4pU768yB/3v4n89Ph6enfzv8O3nLwnJ1DvyWfzL/fxz+5/9mTB+T1/dZqP/3/zr8y1NMaPn3X/75f38R1EKuayXH/zr8A3QXG57+9b9+YcOaoLi2nWt02tIYweeYQilzzsWdmbzCm+f/dfjP1Ql8Jm9L2N9SFw/XO8JtU7lQcPMKtYfzkt9F3rnia2m92/Xvr1ALMWvCis78jSOvTYZTCHsOAl3bznPofEUZhVf/8vw/0gbwH/+xZu1zYzqTdwpvZvLW2PcIYzxc7wS3zeVCwY2F+pjTdBwrP/v51vd//Fs+SXcnUI8kOgj//V/bKTyfYGUOtKsNALQ/Afkkqf/zv8qryjwuJndcw0/2WqfmUNGlX+dpjU7/neX+i6PT//lvkQ32VJmPkIyXbfzPf8YCHTZ8kvwsv6S9vA35x38MMJVxdb6MA78Kh7s3QjLW23iOeBNsc+WJLM/ntg90m/Ko4Cg2dlj0chp3l33QYuGaYz1cBZ/iB7F35NPS78XeMB+4Pot2z+LY4CIyql6mX/Sufir9Mm8V/wQL4RH6OMhDWeHA9I/00Y0Zk94AeTjP9LHrhvVrzrJ9uN4K/DpcRF4vp61psF39VmvdvEP52U+hRpCr4zIYOdk+Heqr8EKi9Iz3AeT//Oe/lc9RxEmYHPu2HHAmcKV4g0557UJK97PGMWGn+tdi2/RUOpUAj6/S8wbB6tDbdJvEuI6419WtYJUSnoeT9Vl4TtxzxgoXavKUMSUJxP7KW+kTn9pnxMrqUPqOltevcRyPrB4Zi39PG51B7J05qZ2XG9aeiKvBaI1PU9EDH6siWNaQfDD1mYwdxTFwQvSKa1VOSb+1KzzHMdO1jd0Wh+BHlyuydsmxT5IXld+tPuQv8rjU6cN+CnV0PhJUkVSTIZOqBI+QSEhVSWTJCfJrsg1tZX4qgrJ+ldXLUGvXtSI5i44lkPNn1CbAi1wlQ9Yc6SRyq04SVEW2E3RIlIe+L76VDd58wxR9KL+93ys3q59kjPhB5uar8vHUR5rbTYcj5f2WRwarpofWT9uldWxzttDuxaOcHs3GfORTwbTGnvhcNmA2FrXdGiuNY8RK9+u5lW+ST0SHoZ/0fC3b6C12VbvDXMyD3Roy3+N6b5fHg6k+Me6WdFrKg53OgofWT+shYy53PZSffRVqSZTitEBSSbIN4EIaIW4lvAlCLzkWZ9YvekkQhOvZCjXo1QgrRI8BrokjATLUyY6v9krQoOzLEa7Zcj9rNM4UnYVXxUdSuPuPMhxMZW5NbjJG/CCJV3gYr9InY1uCb3gaPkgMVD+vlLfE7QUejbEQu6ItxQ6X41vjhYer8cOST6W/2mvmVx+JnwUrz2e9Ph0/wfeVH6JDyYXLfvLWNnovcEXWkNzb3kQ0O6t+Dq+kL9u3Qp9o94JO9SBUfSFYZ51E5z6Wcdz1+XsoP/ss1Og4SYziyPCloPjN8EailiDEWbnPkBNkenMTuSQoKhl6GePg6gNRCBuvots//pm/2V51KHbV56in+u3lah16HfV8K+9RngtusBlrmyEd3D32DqbCDeFj3QhqWZIo1vuo96/iS/ny26K8X/JI+NljEfiyIva2xzkPV+PXBZ/WU97KfCAY+hzo9dE+ncSl4B+4J2tYP0m7v7axe4krXb/MX1+oj9JnTW52NgSIn6xncUm8FPwmteNS/D2Un90U6gg0gpydIJ/nRBILWeIOrRFGzTMEQ0dmR0jy9k42YQ0J3JWBqZ3bB6LqF9mpcDT9605RPudSRTra3cu1dtlntW4nb5IQ9jTWcEEwUQEtPqn+DliX+x5Tw50iX7iY5YpfZaw8C3c93vX+jbrO5eEpQfwpa3prrODRBItR7Amm27x6uFo/GMyMT4/NB3Of9fr0HMu+jHKGuXDgp/xdAuGb2CXPC3bbuLdrWFzU+N4u8ldico/f+laFOL8ukd2hJIIcCKGvJtYIiJCwvGKpJ57QJ4SLxbHOERKX8alwluQm4+tYkd2S3yi4akG1J7iqj8hCXcShE51WFGrfTpH9iFfBWhIVYCA+lo1RfY58gPHYLnyoCSv70E+OmVf/+Nf4rxhEnujTeCSc9nkT52lOaHnBHk8/4YrioKzZJ9SOyyiz6p6x82MPcFXJewvtvb0tVwgmQc+ZTwUP8X/NNW2+xnDms14fPRcxE75kLsU3QSoXil7Jz8Kx2doir+nt8w76Z7goX+t181sr8lfi+1B+dnOiFsOm10Ie3G1OxytCYSDwnriRA0dxgLFnPpIif47iz61z8Y34W+r0zr71veDMvAM1u8uFOXdFJtrCZLhRDjD2WJjvOZfeir+PV6jr6x54PbPRpHbPhKbuTMgdBxh73EDec669IX8fr1DfM1GoOxMdOUAOkAMPx4FaqNs/OWlfPGAbsSAHyAFygBwgB27LARZq9c3W2zqDwUD8yQFygBwgBywH/vz5c4i/Tz8/Pwf+EgNygBwgB8gBcmBbHKgnajpmW46hP+gPcoAcIAfIgcgB+eGJmm8U+EaFHCAHyAFyYIMcYKHeoFO4i+YumhwgB8gBckA4wELNQs0dNDlADpAD5MCGOcBCfQbnfL6Gbym+fp5M9O/358Pz+/fJ82XXdbvr5+H16fnw/s0d8O18QOx3g/3na/h/Crwfvs+Qm3aDyYNjMS7U3++HZ/NPly5eTOyaQNZUDJU+r4fP6rzvw/vz0+H10ySrKxH+d4W6L3KxcOPX8zu7qt0/Bz0WMTFYxDkW34Bnk50xtD5O8sEPo8C/q82GwaFh4GBWsBb+WXxGeLB9jOUlsZF4uGs/XSlvXdIPlH1e/vuFOhJFJfGW5C8WAGlNfSrD5B8TJa6dA1IK0x0XahuUsYhAYdR2WueHIg9jpxsGz6c/cZOgMUSMY7Cl9WGNYQBaO2AzMZxzkzHRZuCSw7tO32Tba8Aa5t1Ed+t/PjdfiV8/06bd8riNuwPM7iaW7gDLncSpU6j9k1UiekpqOrG/fuYAySdAXWh/frKsdjqUudbBWcbsZGML9U8qMnIiPKFQO6eqvtAZLJw5EZfRvGW7vbkGm7SmxdWMKWQcF9VlfMVXNsEpmcb+J/W6O66xTs+bJk3F4YjjgDs1wKXfcKH2o4xxLOQNl/whhzvASdnn8+2mfhzqN/NTtmPsizx3GLeG/y1fzeblvmmeVHIDN9756nub3LpdHPSFeloYMBkLOVvSsQXLPqukj4GW1hwV8QxOlKWKiNJTgsEAOdyZ5oTa5IXneGq0SVytMZgT7LB22ueh3aVIND2M/hEjqxPipu6tfiBrlYyModVlpHvfPvCB0hF0ulH7SG9rtySKND59/8DHJ4/LfbhxURxYwW9Zj9ffcmTmpyB74gvls8BPzRUbXyVnyGEEvqOS5pk3VUNudDkgr8PPqH/Lg33NHxTqUdHMJMo7SScxY2FURU5Ai/NbYa9JaUUhiUGEyTQFVX0l6+gSCwHqg4VhuB7aVwJVAnA4xxTqY+xWbwUEI7wuJB2xUT67F13R1jqm+TQnEjndSXteq50mpD9cK86im+9H66PqX6vPDZ918s32DPVWSX3mC4d/yL0kR97+CIa8XoYfMz8FzEe+WIrbUfwrjohPMT7WcENisMxH7twwVi7jH8GI12PwHRRqp5gmwhxLQEj2Ukx+UahVEVHFwwmGqO+I8KP2MCclcjhB1ddbkzlp0yBFsiQCpWuy3cNUbwys4/RmZJnYeicP46PudYc/avcT3Kiwed9yHxa8DSWb3h7f7v6V+GhcxNPhn+UL8AI3nNbnfAZ+nsSbmZ+KbM8X0KZjt8St9afo5sYW8gHvy/ooy5uP/bIOr3f8r2J+y2n3L5NNioci1QIBE/HNTnFItsmaZc68CPjB2SdlCBSvcMW1RG+5is7Kdg18X6h/b/exRTol+KTzERsCZdNKDNWcHofNF6FOf4fHyeeZkzppl42n2iRGDBwZcZ1uXBy7zHUWa82r4/DweezLAF/YeJe4l2vHm6KjOy/KlThc4IY3f8id3+DCuT4H7gOX/kQdiJlPZkI0JCS+fl4gYPfZyxwQb83YJol/XqhFZyyQEIQSbPWa+0R2Sp41qWa7UoKWU3KaN56jCvVRdntJpaxf9UHcSp/oFYNc7qvfEIM2N+NrXr+q5OPpUnCtuswwzfPrG4iKddNhG4FibHAw8N4WSDFunEG7HNuHydYZu1ms0MZ7ufd57HMPfbE0bxT/eR5yJsVajRlco2CouGHkpsLtfdx0L/hTT59rv8PFLdRpoZTA8NW1KdyrThGFxPW195yAUkzqKaaSPX8O7CfJBoCdPx0vAeG8lhY5XdEZzNGFOuqz3u5ubod78UEqyDYp2HWsjxo2yadK/yy3YZRltec8VyUdR7eGUUw4C+tvpSApHFBniy/i5+OTgzL3NSzCPEzGFjfYXF0iqB9TZi54NXeUnGP5nL+cCXlN+aL4f5Svhrwxa0Peklww5EaMCSU3bLTjs5KBPOT9I/J7XKi3klRvoUdKrP7J9OwkwYR+C1vPteY1MTuXzpTz0J/7nT2WySfy6UIcYKF2gO1Ouc6Y8wX5HZ1EJzhcFzOeKs7HP2JJLMmBrXOAhborPvk1lnpV1Y05L7Hj6+XuFd2F1zwvMfex2TgvJuflCHUjnuTA43KAhfquCuLjEpVJir4nB8iBR+UACzULNT9XIgfIAXKAHNgwB/78+XOIv09SsXklAntDIBKcP/tDgH7dn09pUY8A8pyFuseHLTtBAIm+E5NoRkCAfiUNHgEB5DkL9SN4/EFtRKI/KAS7NJt+3aVbaZRBAHnOQm3A4eN+EECi78cqWkK/kgOPgADynIX6ETz+oDYi0R8Ugl2aTb/u0q00yiCAPL95of56C3/W7+3LqHirx7+Hj5enw2bUuRUMO1kXib4Tk2hGQGC3fv16Ozy9fBz+0stEwPAcCvVtitRSof778RL+V41Ph5ePa9D3NhiQlZdBwEvox/BJxua/I/1yuAoFLwPFrqR6fo0GplxytVxxAUhZqC8A6v2KRJ5vuFB/Hd5S0H2lUy4L9f0S7laaI9FDGj+KT6lI83RzK9dN19V+LUNTkXs7vIU3YtfJFVMVT+tkoT4Nt53OQp6vLNT5pNn+DzVvIe3ln/5EnMe2YBnPjRL6+UVwvVh5taPd/P04vMD/+aa+uh61x5luX17r7Ssndf8kNbenKTVao4xw15/Nma87Ov2N2pWeO31AojcTV/ApFfU1J+hlvjwy/g3z8971fhU/zH079kWe5+W3pPkoVsNL6vhRmT9PdJrkEiU38O1Dv/oe63tePCltmwggz1cValtME4HktBF3gU+tcOcC2JLcdG7Ax/b3kM2Dr52U5NV4CIykm1yzRKVzPV3ZORJ4Y/2tvlouan/K+jmo2yanyZiumwIefCBqjNqlf+dXJHozdYlPYWTC7eXwgkm47v6apDCwJOoBXx4cf0TqnPfWrykGk38mvp34YhpbS7kCeJH0qLlwgRuFOyrW42FD8upE33NiSVnbRQB5vlyoS9LSn8/FgiLJKRcX4asqXItzz1Co7UZhhDsSfzgnB5fYkkTh66gV9oyWz8m/FNPR+qP2pXVTv/MluFH7UMl9dSDRm2WTZC6Dkh+E36FxiOMavjh+kXV4PQkB5dfkG9mkTnw78mFqB18njSC/TWNS1hUzYF4pxPNcYubHtVShJncE2Ue8Is9XFmp8vSP3jdxtR2oSVwmO9mpoNnfkiknwxSlIbiMi73BlzXiFQikBoeYY/a38FfaguKPXH9myZl0Y03bpQZtROyq603skejNxgU9xYJecR3MW+BJlPTD+0fxL/DS/WvxHfipaeL6ANp2nSn4bxWTHkbgG6oP3ZX2U5c3H/iSufaSnYrqI42XfCDSeH/BvfTvEqmQxOz+LTyJ7GCNX6bfP0g7XVuShUd0uBJ9H+DjftqMutq+u52CAwYMy6pzBjV0D59o+ETFqx7kydnjVbzjasFF7G7G3OyR6s22BT3FgwrttRCUB98lygS9t0XD3ePgr88/40PyaMdUFtmzM3Y24KAG+WIqto2IyyhXeLHDDWzeu5eoN+ooJvO4egcbzNYW6+yzFw6ckv5fwT6nUu57lpPjrQj36DMkEWFpHTtSjOWpHXOxUwbNsT0XnpPVzQLaCEJ5T4B6xrmdDUirLUO6pyu7zBoneLPSwzG2Nu2ZM8qUk4CZJCrjCVPFlYSx28341Ar5f43Tjt6HEPC77bWnOPCYbZ+L+LuS/WmhxjaKI4oaRmwp32GTU+ai8Iwu7eb9LBJDn3atvvTuV5JSJovoMoeQ1r0paCb753HGhzkRWa47+jaSQPH3zW3SOh+r22vvl4yPsduHNgDvHCQgVXNGguT3ImOPXj+Lb666nujtfWDcVkmZrTR6jdlRyx/dIdDnR+nwqPkXyKj+MPitc4MuD438pamm/4irZH22jC31TXyzEtOJCyy8dp1ROXOBGVE3JLW8kRcZUX7CLt7tFAHkOhXq39tKwB0UAif6gEOzSbPp1l26lUQYB5DkLtQGHj/tBAIm+H6toCf1KDjwCAshzFupH8PiD2ohEf1AIdmk2/bpLt9IogwDynIXagMPH/SCARN+PVbSEfiUHHgEB5PnTz8/Pgb/EgBwgB8gBcoAc2BYHYrGOvyzU3Khwo0YOkAPkADmwQQ7ImwMW6g06h7vabe1q6Q/6gxwgB27BARZqFmjuoMkBcoAcIAc2zAEW6g075xY7N67JEwM5QA6QA9viAAs1CzV30uQAOUAOkAMb5sBmCvXna/jTl6+fGyHL9+H9+enw+rmtXRV3ufQHOfDgHPh8PTw9vx++N1xUyNHzc9Qp1LcpUrNC/f3+HP7XlO1vWF++gN4GAxL8/ASfYZo4F3j1/P493iDGxAjcm45l8hzjeCVsJFdYP0m7+PLyOeRCXGahvjnHZjnlUn3bL9Tf74dn2EHmgHs9fF408FmoL0W4zchNCe/18BrenNik3nT8PLw+PR/ev0vSjVzE54ty8EKJfrc6R19FX36mt2HKpzfJIRfyHws1C3VOULMilftkV/r01ApmfyLOY1vAjOfGdfv5A6LPkmXqc07eo/aYtNw+wSAHf7YXEnZKdnN7WrIfrVHsc9efzZmvq08OTedRu9Jzt0m8YF3tE//ma+OoGZd80zj+82MKdydvzBfib7Ct2J27fcGncd3k1xYbGANjP83jzs8j0bbZvNz3+jnmjZYbdH7Xr77H+p4bV8pDnlz7/qgTtS2miSRy2k2vCCGpmWCYzg3BY/uHQNh1asBnsrekG56TbnLNRFM6p8SLJyoZK8HVgtnqZ5+1XCS1yDxm/TBHvZJtMqbrdoWl6DFqr9ihvvu/T75K34fIfm6c6W1PeKcN6WzsAl+I/xVPQTM/Ff+OcsjET9O4W8oj8N2bxL16wFngTSnyjZ85L9TPqCf6DvPng8b8veOxvlAnUrTClQ2PxJG2TCL57EcVrsW5awv1JAhHwWeJieQezsnriC3J1jhWNiUr7BkSY836I72W1k39zpfgRu0Wm0d4Rvy7RNgX6ujHXKzDmxrxf4fTGr44funk+OsPucT5zgZgkiMSXpP+UZykdslz4iPIfdN4hcNLWh/mFf7N84yZ3+Uh8uoR4uPIQt1eK7fX343AbddpElcJgDZH5HhzJRD6a5I/SpZIYJPA8i5W1ozXQv7hHKN/lIdjV9iD5Dl6fVwLbVmzLoxpO/Gg/6gd5e/+3vp1krQTFs7ms25MkZ9WruFLlEX8naKKGJ7rfu7TaQ4Z+Ql8p3NYyV+jeI3tkmtqbCFX8L7Yj7K8+dg/0reudS5MKQfz+S3ujyzUZndnCZEIHcbIVfrts7TDtRV5nxSLAeaROsq37aiL7av6LAQQyqhzfL1PWn+k1zHrltdxareedNXF5xaku92a2XadbMsGztkAqrdCCbtREVjgi+LII+M/iBGFz2/HjHxU3ow4fvb5CH5airuj4jXKlQPKAm+8deNarg2g71nx/K0/ON/n13G4rC/Uq14TliB5Dv+cCj6XkS9UqBOeIdO4UGeZPjnR2EzUtkZ4joQ2QZTWqbvcwZylV1KrsCi6nbT+XK9mI9pv750kkDAftdv5j/Bc+Fr/eVbhmnDX+E5Oxf3mx8F0mFCdsSYWzhHYjyvD+jTyuPjVLXAjnqOfPJk4bx6vmAv15g/XKPIUb4zcVLjDxtK1w5FFXl3pLQ5y4TL3w0KtTx16B6j6DGnkNe8omY3mDgt1Spb42rrcSzJFMgqR0797FZ3hM8b0Ba33sKOFNwPuHIf0KoCiM/KYkT2YKPPmIOv9/L5m/SDf1WthXYuVYDRqR+we8j77sG18ik8Ft4AJ+i76uo3FgFzgC/G/QsLMRU3Fo/jL4p/yQ4hH8HOKVztO9S/E+zBejV4qXy7wJsakklveVoqMqb7IT95jPr7He6dQ06n36EjqTN6SA+QAObBPDrBQP+RJcp9kZpKiX8kBcmCPHGChZqG+wmtJJo89Jg/aRF6TA9fhAAs1CzULNTlADpAD5MCGOcBCvWHncLd6nd0qcSbO5AA5sGUO/Pnz5xB/n6Ri80oE9oZAJDh/9ocA/bo/n9KiHgHkOQt1jw9bdoIAEn0nJtGMgAD9Sho8AgLIcxbqR/D4g9qIRH9QCHZpNv26S7fSKIMA8pyF2oDDx/0ggETfj1W0hH4lBx4BAeQ5C/UjePxBbUSiPygEuzSbft2lW2mUQQB5fvNC/fUW/qTf25dR8VaPfw8fL0+HzahzKxh2si4SfScm0YyAwEP69evt8PTycfhLBjwMAshzKNS3KVKzQv334yX87+LK3/h+ejtcvpzfBoOHYd6VDUWiy9LCqZcPTHnZ741r882jyJDx3NgJute5Wr9qf5g8EQtczSFPB+336+h7llVYqM8C4z0JQZ5vuFB/Hd5gBzkr6OcDn4X6fFjeXhIS/RC2eW8hYb98fKW3JjphH+H3vx+HF+BlLhKmONze9F1r0PkV/KHzRPT5y6HuyaLv8PmeUGKhvidvnUVX5PnKQm1PHC0x6cCI+uWxLRGO58bR/fzY2v+khAgBqUakAJSTN7y6HrXHyW6fJOyc1PNOHAI9LTq3Z5Vew/VHeqUJqcC000HzQRaHbx+azvq00drjnL3/INGbrZafsUf83katvhsmf5E55tIj+2Y1vs5A3695oMoTyTcYJ6Zwg+yxLxbi3c0jUfBs3jI3dH4KcfuhX32P9QWjeHvXCCDPVxVqW0xVMKRXSxAMJnFN5wYYbb+PbE52rfjjKNsXnlNBl2seq3Supyt5/SljJbhaQbP62Wct1+gFGws9bqJzOvVZvXqclLwuIRU9Ru2o5o7vkejNzOxjzSXxu2z2mv/bvMGd5X8dJjKbLMWdB/dNhemEG9+vUZCNqxI36WMzz+9l8YkvlM/CcBV33Xomj8BnImle/fhugRvdYSfbVT+jnuhbLOJlBwggz5cLdSJFSzbZ/kgcacskEk4qIi/O7QuQwjclwZI8ZQE1IDwME6UZiOQezskBpJaKY6XgrrDHrNoe16w/0mtp3dQPbxJk1VG79O/8ikRvpk4Sdhmkk2qb2d/NZK3hkuOzfhG2GAQ6v6a4GeeJVGzj59QSx0aenF5V3McxS3E3jVc4vKT1MGeu4YaZ3+Uhcse6cW/PyPOVhVpOGniVQo3F1hCwFIr2ulbme3PnMA+TJxLYiMhzZM14LeQfzjH6R3k4doU9qMLR6+NaWlD4bA3tkPuGoySbiLU6LYLOqh3l7/Qeid5MzD6eY6E3n22uvksFYJT8vdfp1r8P7BuN5HFPvl+zDJ0ntB9zH8QMLuv5Atp0DisyrD9FXmyXXCNtig8Lecabb9cC3eZcrgrw5s4QQJ6vLNRmd2cNTqQJY+Qq/fZZ2uFqXy1Bl75Nspwg80gdZ9p21MX21ZUWAghl1DmDG7sGzrV9ImLUjnNl7PCqk1MbNmpvI/Z2h0Rvtq0o1Alvh2tNSP7IZlik48AFLoGsQNb0RbfuRKfG8EEQ8P1aesF3qTArH63wPfpiKe6OitfoY+HUAje8deNayhZBg9wRJPZ2RZ4vF+ru8xIPjhIAL+ELTSrbLAfGsFBHsoIsvVNGHTJR264yPEdCmyDKr79kwzGYs5hcl+2pmp20/lyvZmNdxblxkkAaNWp3ROykCYneTOp9+PUmCTSP0iflPL7xujy7SbOtclyhfjzfIFLH3iu/zvKEiUF56wRpxVkafZHvx3E3j9fGGfvZNq5RVFCF2MhNhTu8RXM558hyrGLT/SGAPO8KtfuKpxQw1WdIkwup97lJJtJo7rBQd2vqZKpgFyKn18NtXC7O+TXxy8dH2NFKoQ6z3TkO6VUAxVXn9qBex68/0mth3ZSQ5HV4uEomGrWjkju+R6LLqVXxUD4msDgpbhd/jzCVjySkv+K5wCW7Zje/CuKNQUD71cZji/84DWMw+t4tulNfWPmmYLp5JK2c3pJUvjmcUi6POuAYJbe8rZT+qb4GLD7eLQLIcyjUd2sPFScCLgJIdHcAG+8SAfr1Lt1GpY9EAHnOQn0keBx+Pwgg0e9Ha2q6hAD9uoQQ+/eAAPKchXoPHqUNLgJIdHcAG+8SAfr1Lt1GpY9EAHnOQn0keBx+Pwgg0e9Ha2q6hAD9uoQQ+/eAAPL86efn58BfYkAOkAPkADlADmyLA7FYx18Wam5UuFEjB8gBcoAc2CAH5M0AC/UGncNd7bZ2tfQH/UEOkAO34D2YlekAACAASURBVAALNQs0d9DkADlADpADG+YAC/WGnXOLnRvX5ImBHCAHyIFtcYCFmoWaO2lygBwgB8iBDXPgIQr152v4c3+vn1ch4jXXWt71fh/en58Or5/b2h0u6019iRE5cDQHPl8PT8/vh+8NF5yjbaItqW45hTon9/q3adPfMH49fN4QsO/3518V2msWz9layQ75m9BP18CUhXqUGLQvng/v37kw6PZr+IgFaeSjU9rFf8/v37Axd3KabNxjcasxCX8n/yrxeWbfs1CDz8+M7Q3rX4wD+YFvfWdSI9FnxeeUYDp2Tgo+CawTALum/uO1Pg+vsNsdjzsnwVioPa4lPoEv2phb+Oic/n5kWcF3oeA+v3+mt0iYv35+jouD68TmBXzFQv3YhfqnI4DdoeLJI/bFE0oOnLxbjf04p51gcpLEvrirFXm2PQai7JRtn8zpA6APvPHc0di16/bze32izeNiEcZ/vx+eYZdfX12P2uPmxe3Ldr5+oi/WYu/o7a5Rxo36Ru2KD+jzLE9ORpk/TedReyu2jt5qcxexaPJm88Y+Wsb193ou2cF+33fZNy1eI07irxWYJb76/Bj7NMtvJ3OTi06KAdF5ErtKbtD5Xb/6Huu7AgcVMxzvc+16uKw4URcSwonWFiOd0IS0QlZ5bp+VuuNBfiaYzC9FDfojaHMdNIB2rH1W+qRXYW3tXABb4E7nOnr5Ds7Bp5OJ6Gz7wnM6/ck1j1M6/4zmCPZj/Zfsafqfsv5IrwX/pQQEPpCkMWqX/jXXJOP58Bw+u6+J1XAr22x1F//E6wKu59BzjS0c45ygsm90bIm/xOctHhq/s39tPNT+iU/tnKNiE7in857o3HTV61g7M1/rZ9QTfatN5I/DH4zz7dwPC3VNYul1kpxig+Il0clnei2pCaEygeopMJAhERAImU7ocmp2CRVJJ/Kc+Ys6aIAVwRfnZsKL/iroFuf2BUgFRdoElGSBeGDA2I0C9uE94jac0/tCvR1ZYY/S/9j1R3otrZv628au6jBqR72W7pNOjVvyJkL8nbm54CPvhBblyuv0c+i5ZAf7B0nWFjCdCyKXdEHEfh37lXcR65FPUzvwKfkF8tc0BuxmFOat4piZTw4OOIE+vs/7YaGuO1JLtEJYLOT5XsjaF4dpobbyE9G1jG7+og7aGX2hlp01XkV/LLZaDwnWse04V+uggn6WLDDYEhZNTk4wqHMJ1OEco3+Uh2OPxPHo9XEttGXNujCmcjHKGLWj/Nl91Ek2iWlcxkitUeZne00yhDm1uMc2a+tv9Sw6WN7wucWDj8XYn228X5CTv2Wz5eHv+RTadF4o+cTyQuR2PIx2YbzifbEZZXnzsT+uA7p5/G54LGHK/ltjtVyoC3mqo5PzveQlzuwJlgIAT5BIMlce7ixHJ+qZDqJLvvaFemGu6CRXCS77LO1wVWtBe+foJKttDmo/YoPzbTvqYvvqvN4XqqCgjDpHYzfUC+eO1h+149yldctrfVUU0xw/2VZ9R3LT2oh7xqjyG+d1YwWbBVxRxlB/kcXros8Unkt4Tfwpcly/HsMnGLvE5aNiAPPeAse8deNa7kYD9BUMeL2r0/eKQh0CQ5FtKRB6gk0LddkI4L9ztjtb+yw7Tze5OgTUxXNJ/5gIypjwpTjUq7bXL7T1SUOvBf0xsGCzkmxSJzsZm4Oq2RaeY/ApH5STe50/mKN26EW+CuY1WMC8ut7a9ed6NRvFdu/a8ykn9lG7JwPbjM0J11K4V/vIWVvh2q/XbzRwDO/PV6yNf0M++HzFjVnhrilo43j0fIP+79fTtsxjAPOLznO4RtFBcczITYU7vG0zdv0uVjzb2ab9ex081hXq7lSQSaRe9VSC9ARLBIQipQt/NDSTrsqrsgoIQkL1eflMBw1eXzyX5+bAdT4nlY0FfCsbg6NfS3Sxa+rkoZwP9j7BZ/VJdln3+f09fI4PbwbcOb0v1Ik6bWqsXqNgl+KcX72vWz/Y7uoVMZmsm4onvOIX7ozanc2ZwtP2K53Qx1ankY8WcD2XnlZvPk9OQSaH1DgJ36+x/rD5peSf6cbRyhBOJp9Y3pgYUnxDThmdlV4LHIvrKrkhF8RnkTHVV3ISr9M8saF4cwo1nXcvzqOe5Co5QA6QA/vnAAv1hnZNDLj9Bxx9TB+TA+TAsRxgoWahnrxOZEAdG1AcT86QA+TAuTnAQs1CzUJNDpAD5AA5sGEOsFBv2Dnn3pVRHnf65AA5QA7cHwf+/PlziL9PUrF5JQJ7QyASnD/7Q4B+3Z9PaVGPAPKchbrHhy07QQCJvhOTaEZAgH4lDR4BAeQ5C/UjePxBbUSiPygEuzSbft2lW2mUQQB5zkJtwOHjfhBAou/HKlpCv5IDj4AA8pyF+hE8/qA2ItEfFIJdmk2/7tKtNMoggDy/WaH+egt/bu/ty6jGRyJwPgSQ6OeTSkm3RoB+NR74ejs8vXwc/prmpce/Hy+Hl49jZy1JPaX/6/D29HLYhCqnqH+hOchzKNR/Dx8vT4dr1U4W6gt5l2IrAkj0Q0xm+Hfa6/3b4SukuMh91X+tQKja8mYtAsqvZVIsOtF/XeExflf9s761ymxh3EmFui+OgmGOgxgXp/+k/H7E5mE7m4bTbT73TOQ5C/W50aW8zSCARPeUapvF625SPV3Yth4B7ddYcGKB/kqbLVWIQ6lRJ7W/H4eXenKb9a3XZRMjTynU3ZyABxTWFhtXsrDT50rrbngZ5PnKQm1PHGa3lQKgnUjqYWTUHsAZEyGuFV+D5ABsuzvUwb4mwb6oB+p3bnklwD/khPZ2eOte42d9dNIojJhgchj1jdq7kyDafQji8ikjY9gwG7VvmLMnqYZE7wQkTAWT7K/K226wNMg45KbIyGMeBVtB5BZX369OzCUfY0yU2I1ve2d9xqixT/OaOb5s3pE1nLw4jdtljuk8EfgXcxEU2bG+zbBx/s1jkowqM+oUeY68j7ii/ToOtPwVNtlNVVP1Ye+Q5ysKdXEGZLFMBAmA7LxWlMJzcrBcPccvFWokfSODqKBJ1MvS/TJf9JXn9prfHS+LBfU9e9VmIO4GcXOgigDybIbJBEf1Sq/J0MFQ9JTg6hJR0WPUjmru5B6Jbk3S2DVO2E2NnifjWlJSch4IW43LdZ98v2bftDyUdUr+SbHZ98/6qkUTnyrfhwk6j4ziuXBomF8WOFaKY7Mzr1ML9UTfalMno/XkO6u76LQ2h9qcLPMHcZMWzWMAFqvUwz0jz5cLtev46MgCui1SIziNHEvyNq13WAoA9CCumeQ2AmQ5oF8hJU5flieEFK1QXiYxygu0TK/fpK2TL2LsFTFBm3DcqH3J7tTfNiNV5Ki9DtjPDRJdW6X9pfvsxgx7e26mz77V5sjBHEXw/tcI+H7NvmkFrC2TC3LY/IufWld+sxe/r+D0pWGjeFmKv2nczvLLGo6Z+XEt0X+kL9hs81XtSjqXNwCSzDII6WMFbOpynLFX5/cFm4oCcY7nv6rfg90gz5cLtXFAxgqAR5IYIJMzYxDU30Yw7UicCLJL85QUhZhtDVlPiveR8pbsNUVZNG869uvJmHgdYjLCcdS+aHdaLHwml/FQAQBzVTsquoN7JDqak3wgiQ076v2okDu+tf55EGwrVDe48f2afaP5rP2YY0/ywqzPGOX5FNp07oEDjMexxfyywDFv/tEc1LYba9Njxkryda9T6sfKbfTS+b2frza4RQEWau0J5PlyoU6EFIeJoOhoICS+9q1DvNfBTY52pEyK196pU1K4+p1bHtg7KNT5c6Ng30wfQ+Y6J6pr+8SEUftsHZlbr6PAHLXXiXd9g0RvhqywOWErCb3N9LjpJZw8Y8U6KJr3qxHw/ZrzBhbqlDdUsWxjZn1jRcCnS/F3VNxGucK3Pv8pjnnrxrWUnWIB6CtN6Tpqh0EqBnqdpjk5iNH5vZ+vbCrLslAD/uEWeb5cqEvhxH/zrEmend4CJDxH0hiiJsdBQdeORAV7p85Jkce39VFWvD9N3pK9uJnMK+Z10u6678xDppgMcCwbg2ZfwbfY1drzEv5/ewxQ55G6vqz7aUWii9aJS8DD2P71Jkkyj0rcdBOfg+MwSTpjRQlef4WA51eJcxUPJt7yxrh8NDHrG2qHPs33aj01bxTPed44v+AaRaDimJGbCmp4a7aWr0mko3uUA4lAx0mv0zwnn1Ko+zUUnA/4gDzvCrX7GqcUi9pnSSFkSa9ZW9LLxVlevX6EXeMFTtRAvKpf1KPq2BNgiWSBZukz5yqvyoqL5T7gdKVQJvf8M8oZJpJI8roNx2G7bKLqRwtgd0pEGfskTxQetVcr9nODRM9WmSQnplpMlL9lULz2XFInAytHMEcRvP81AtqvJlbNRz0YbzEOsLDO+qqSU59mPtQ8ofJOkDDIi5JD6jzFtwWORcWU3PIWT2RM9a1WmRNvEpo+h6461RN+60M6L+XQhG2dsMKmlFch5zVVH/YOeQ6F+mHxOJ/hKUjaZuR8ginpFASQ6KfM55xtIkC/nsEvMVdJcT+DuF+LYO7sIESes1B38JzeoHeRp8vhzPMggEQ/j0RK2QIC9Os5vBDfRGznBMvc2fsUec5C3eNzYsv4lfiJAjntlwgg0X8pitM3hAD9eh5nxNfX+FHAeaSeImVbm4ZTLLjEHOQ5C/UlEKbMTSCARN+EQlTiLAjQr2eBkUI2jgDy/Onn5+fAX2JADpAD5AA5QA5siwOxWMdfFmpuVLhRIwfIAXKAHNggB+TQz0K9QedwV7utXS39QX+QA+TALTjAQs0CzR00OUAOkAPkwIY5wEK9YefcYufGNXliIAfIAXJgWxxgoWah5k6aHCAHyAFyYMMcYKHesHO4q93Wrpb+oD82zYHP18PT8/vh+8ic9v3+fHh+/95Aof48vD49H96/yTPLM6dQfx/en+FvQ6e/mft6+DzS+Xah3zxHIj29fm6ASCTQb/y4pbmJU4HbNkFJe/5bx4b33++H5xQPOT5eP8mHLfrU853269Oh+c7Jd/eaa04q1LY4nhePz9cQK0dsHqKfbExuiWO30mVYqBGsBPYNyZuC7Ibr38o5XPcShTAmpligP9OGFHn+8xP6IKlo3su8cvKISZG7/w1tnie+ixss8Gsu2rIJy4WpFe5LcO5KMk8p1N2cG+PR6XMl7G54EF2T51cV6p8OPLvrEtJHUGNffH2RE1vb3eIc+3oD++JpReTZdjwB2T6ZYx0bx51Ln3PbF+VFnIJ+7zHxZ9tf4y5UbUyyrbqoFDtnp7xR36g9+S7qIL8aU30qaT4cta8h4O3GTDAtQZvskgQfY6DyUniAJ7Pij4Lh6yfyv2EV7b1PvMS++7gq39kknPgvPsk8WFOox37LMkZx83NSvIleYx5puSWHCF9X8kxvRme8lr6IG+oUcwTaL7hmnmj5K2ySfMjX32oTvKJQFydA4dDgl8RTCSJOkyQvzy2p6SDq5eeAkPlFPqwfk91cB0wmsr7Ik+eRPkuyZf5aeUv2CelFXtDdFgWVWNC2MLfibv2Q5bbiLmNH7Qt2Jx1AR0l+o3bp3+w1+6Xhg7jGe42T5mzsH80XfrSEpbh6t3hZfLb8rH3XbQZVfIm/ZHPa/KbmTfym/Bv4rrlidQnPKWbLupDXdN4TvZo+ep3c3/ib16mvmSf6NrusjMbrtulo6wvn24ZVdFybS2V8k6ltauuv2Tg1O7bMxfPoNizUzVF4ig2LJgI0oDNYkSTSlp2BQCcCAiFVIXIJhfKcQr2oA4Jzij5ii8hBfU6RZwscyov3jegNz9bW4TcqfoilSkZiR7iO2pcwTf1Npxoko/aRjptpz35sia5glPApSRs4q5NvHhuTTDe/nqgN5rKhulu8wJ7N+NDoNPBd5WrSe+D3YpMuliB/5LfUPskX03ib5YU+z6g3m2ldMz+udRTPvNwDNgdMNB69Tl1uMvbqQtzPVzYVH/hxpfXSPt1/37BQ1wRkgNevW2QXGq9C1t4ZU2da+RBMUuy7+SVocDOR70UHdNyR+izKPlLeon1+sDSb+/WQpGlcfVUd/VCCN64rQVsCIM0btS/aHTCFMZUfUfaoHdfd3H3GVdlhdMzYZjzTvcJzNN/xl8X8LvHCmNr+PfoO4yUVDuVHa4sfj0mG5zdo0/mo5CLre+FYbJdYlTa1yVvgkTffrgW6+Tyf2Fp1wjG9Tgln2NDagwALteXXac/Lhdq+4kvONzu56tSoxJHOdOVFcrSi25HBnTMC4Bz6oOxzyEP7MBBgHbFRrgrjMs4GK461fTJ/1I5zZezwOtC5vC6WDRYmyO3dZz/6CazgmzCBhKsSa8+DbKPTbhNoxXWEI/CgjmXbURxC3xUMl4t0wNiZ168LfluKm6PiLcqVvLfAI2/do3kGdox4pvDodepys7GXhfo8cbuiUIeFFPjZWeMEd6wz83j88lRyPux67bNsBsY6IDin6TOWfZq8sX2jYCm4xNMy7lgxoJRfymfMtZhkuc2O8JwwHbXn9dp4xNDe9xgMixTqu6l7x96YlADrxDuDZ92EGOxbMnewiWOBz9Oxm8LI+n3Dz1PfZZ94Pvh8lcKYbVtVzNVhxOGR8uE83jC2dZ5b4pGRmwpqyBVH8azXfY5Hr1PSGWJG1wr7vZd+fv/q2xmj8NwwBy+o57pC3Z2UMpjqVU8lSA/0kjPliztVXpVVnCIkDEWrFZKZDujMU/SZyT5FXg4q377cVwsAODsXCudzYRiTEkt59f38/h525PC2A3BrH00EbEbtJQFVPaNc8UUqTPHVevmV4By1g46tMKFfbnVvfFGxi//syvpdJ/ExbmhLzw+VjO4OL7Rty/cT31nMkcO2T/hu+WvHCf/TOLs2xE3sH8ab4aJae4FHndwQ93EdkTHVt/lRn3hDu50n8sBOzFVLuV3LX2ETv/Ud8lDzj9w7hbofJIN5vTI2KWig8DoOpE+u7BP6wE0k5OGd8jDmGFWMb2wHc54bXyzUG068ejd64wDaME4sEuQGOXAqB+Kp3rw5umGsM+f5fmShviEp58ll/Ep8Ps93NOcQF3KAHPA4EF9ft48Ub4nRtjYNHla3amOh3myhvmXAcO1bBSTXJffIAXLAcoCFmoXa/UzEEoXPTB7kADlADtyGA3/+/DnE3yep2LwSgb0hEAnOn/0hQL/uz6e0qEcAec5C3ePDlp0ggETfiUk0IyBAv5IGj4AA8pyF+hE8/qA2ItEfFIJdmk2/7tKtNMoggDxnoTbg8HE/CCDR92MVLaFfyYFHQAB5zkL9CB5/UBuR6A8KwS7Npl936VYaZRBAnrNQG3D4uB8EkOj7sYqW0K+GA19vh6eXj8Nf07z0+Pfj5fDyceysJann7v86vD29HDav5rnNDvKQ532h/vtxeJG/iVuuxzozEuDp7esCqi+L/HqDv0ud9H87NE3+Hj5ecr+rXiR8nFNJ38bXv3Ud+jMerW+dLF930deV4U9h60oEkOhxSuJl4fTTk8+L6md0iIkJ7FqpCoedEQHr14PEbfFtzVemvfo2+b7Fb22/V8eeVKj9AigxUjFMfjsvVinn1Ry7TIz72FAs23HsCOS5LtSF2IqvJUlpx82XvHWhRl0z8SQpZ8K9vISNhEOUSKC3N9yd5vEKj2r6sbLqRLjJwfL2druNDSizu1sk+iFs197A5ylZVMfO/Bx9JJuzAFGKkcfc4W+FIJ1f8cSV8tXYP83vM59vxdKVepxSqLs5wvOvdJjBHBq2uKmthstKtc42rNP1bJI3LQh5DoW6FB7vHUNKTljsYiBkx+bdqARGllF3qJjghqcSIYEnT3C0ckUX6W/XGIiaZFluJhmuJTqXuUm/IFeRQsY3+e1O+qL8NbLazHona8nataPcjDAbtZeAavhrnGS3rH1mT5rGFqvTHT0j0a3aCYtauMWXdlR4VtyP/aOx0j7m8Qh/Z1U2TRBQfu1ix4lHkZXGCr/FX9I5vo79lmWM4i28wlFvJ1uhm80TvcY80nKDPR94uFgXz23DYu3O6+scKjrZsfE59kVMUd+Yd/Icm2viDL22yMb54qM4Ov5MfJoH7PK/yPNWqBWJrd0IlDiggZmJ3IpCem6szEDXpFiIVJ97edqR1rF2vta1K9TKLiFFlolkrDpL8Uxi23i9SnxqfXZNX1Yvoc3LslCfTE7cdAQfJMwyodtYaV/AKeHQfFS1GbXXAfd7g0TXVlgMM/4t4RpuV65GKZ6vWvsTbNoUj3eMs8b28k/WrwlneJ3dYkProvyhCkn8OKz5XM2a+E3Ls3nJckzitHAN8qPOn8LFpo9ex/Ivr1PfEE70bXZZGa3H57foJB8rNt1kfPsoqY0VE5N9EEOePYi/7o+6ZZkiD7Xd8z3y3BRqJ5EnJDIZMlAeaLG/Oa8WqhGKikyOPCyWaWyTnUXq9XCZ6GQM1OT0ShJYS52UQB6u3QVzJKpgdKws1DLcG7ssmfuTXJmv9AaZRl7uAbtSf3i13z6wz0NG7SD6Xm+R6MmGhF1JNh0QzUpMnJ1fwjDLsTwT+CCikEs7xlnMvda182tYOMV5/Iy6xrrVJsaCw/8yDH2uZo78ltoneWkap5JDZCWIU68odTwy87v+sZ15xRkWmceYQ0VLuWqset6nfowvg0XyVe3v56fcZ/zox5xotM8r8twUaks8AWCBSOnVRCNH56ggJjtXdmSDgleXg1c5JVDaaUdk+LrWgI1B2wUukgLIikTHey9oREfVt0ZWnZhuEh5IRhv4Sg+YO2pfgxOMUYE4aodl7/EWiW71z3w0Ca8Oav7s/FT8rvBL85BbRZD11U5xrrBd6Ub7tfkqLp/92ueG3o9WWS1H9Xp+gzadm8ra1vciMLbXzb40InfwvvSjLG8+9scpoFvP0zhgYuuQ33Ge/OD8Xt+EdS3EYY7RmYVacJxfkeetUM+cp4DuHZMd34JjyVGZSJIkHXlIvEQ6GTs3LPbOd156LdFTzcG1VTG2ax8rC+fnuTrA88aiBpbCHOaO2o/CCQMNZM84gMPu5B6J3qmc8GqcVf3Y1+Gt/d7mOe2KS23kPFHiON57CKBf+wKc/VDjKAkY8R2ko8+hWd+CnKV463hTJLnzolzh4gKPvPlH8wzs0AaGJw8/M0hh1esrebXOMliwUFdkpjfIcyjUg91ocgq+Ts6Owc8UbLDYZ3dHVXeVvaP1q48VxAFzVdGF9nxr1iq2qR2uIr0Zr+SZvkVZMDkRVwKztSfcKi45mFrCCc/pBD5qPwYno3tVYdReB9zVDRI9bQ5hl49Yf4Vv3eN3KFMiqW87TFIzSacB4mCnuNRGSjIEdbCT9wsIKL9af5Q4RGzR1yJ67nMZZa/o43zf4tOOnccp/vNVnS9xjSJT8cjIlbxT+Yp6OLJS90z3vm+OVb9GsgcdYHx0fKHu10Ar93qPPFeFOhmcQJXXy/Gqk1hLMpkw+VRoTrxCnvDqWYicnFNeR798fIQdpMxxnKCIGbXKY9QJ1CXmcSfqKteSqsp21q02Wb3L2KGsRidN1NbenbQAR+WHUfsMJ+tX0XPUjmrd6T0Svfq6cFDhaTGo/i+GD/FGYCwfQh/y2K4h+KMI3q9CQPs1xzzmBsk5WZgpbLKC9Yf1+Wic8puTH1DOkDeYO0OOxTneWzzkUdRLyQ15ND6LDGuX0leMKpipPqNTzdXhD6JYmbJWEtfz/vyFOupm61CzZa93yPO+UC9a3TtmcQoHEIEbIIBEv8HyXPJCCNCvZwDWFv8ziLyYiLRRkIPdxVbZnGDkOQv15txDhc6FABL9XDIp5/YI0K/n8MH9nFLHbyDPgcN2ZSDPWai36ydq9ksEkOi/FMXpG0KAfj2PM+Irav0xwXnknlfK/Wwozmv30t/6PvdqlEcEboQAE/qNgL/wsvTrhQGm+E0ggDx/+vn5OfCXGJAD5AA5QA6QA9viQCzW8ZeFmhsVbtTIAXKAHCAHNsgBOdqzUG/QOdzVbmtXS3/QH+QAOXALDrBQs0BzB00OkAPkADmwYQ6wUG/YObfYuXFNnhjIAXKAHNgWB1ioL1ioP1/DXxt6/eRO9YIYM6FsK6HQH/QHOXB+DrBQX6yIfB5ew5/fe/08v9MYCMSUHDiVA9+H92fGJflzKn9uM29cqL/fD8/17yHnv/n9/P591Onw+/35difKz9f8v7csNqiCOezLQYx/Lzjeq7mmsA9PzXGN5/fDN44/A6YMsPMESuJm8G3HaeOjme/pi/P44iw4mphuMfx6+MQY/GGhPgveCtMN8WCnevmFupBeJamSwLrENgHmZoW66Nr0D6dbKZqzvhLEysaChWoDm0eFOrarOWfClEH226SQ33Q8v3+mk5Xy0Y/0lQ1p8tnz4f37t2ty/i1468cmC/UtfME1f5cDnEKdiawTWFkkJS7ZocZxMYnl5JZ3sJLUsoy2q4WiNTyxSAB58sRIK1d0kf5yTWuILkf0eYU6FmVlt5bnJ4NoA66f9V7GVMsmuS+Jh+OTzs/CSauH0x7nymYwcEZO7DourBw+X4zjwxwgvjs1z8h88J3yfYn9d3mjl3MU+QB4wUHnYv7f2Rp9oR4SPAKNBSgT9gkKUiZjK579iTrMt8msPvfybBG0z0l+nY9E6GU1Qiz39QU1B3U7obe1rE5pHRW4YexqTJvcpi/bLoNF5gH6uudTPybrktsVH9Dnyd8tDi6jP3kxw9WNy5S8+/jXY0s/fAlU57UF35e3Mk9P4H/y4aiPTGd+fdS+QaEGkqmdCRYsh7CqkJdTBRC+A1kR2JHXJT88pcZEhRuHPnGlAEyfUdt5Pwe/L+uAyTvrjHbrdXSQxz5HhrJTz882zD8H73BTPrHy+LwOr95PfaHOPOn5sIar9Ok6P1yCr+N4lfg8bpOFeWbB96VQK/kp/smH2/HhEhy7rsxBoe4LWwZ5gbCGpCnxmUKdd6f5y2n5taBsChYCoJAdX6evfa2Y1/Rt0n198m52Ld+0KQAAFBpJREFU+4HWFeqkp1nLa6vFFjG9rvMfO3B6XycuqDc0/ZiM2QJXo2+Br32hp58vyb3ej4j3gu/i4QBPwylOcQ7eF7l4oDA5sNpJPvBUXXM+8nHdfV+oR0SLiygSO4RNc1uRSgGDhVrND/ISeY8p1DJ2nXE1SKbf9EQ78n2XWK3eALgt1J3Naexkhz+R3fQ/1l6OX8bO8XXnC+QGYuq0x7mqyMv4ie+BR8v6ijxe51gt4b3gO5WTBGvcTC/Mn+XPpVxAPrCYDzjgFGr5IkwruCkwyo6wFbFM2O4zakhW3c7WJML8+lmK71IA5P62vgRRf43rqnFp3WzPrE9ei6m5nd16PV2oHRsK8AkL+Dzfx1TLnickjv0dPh6fTJI3fMX1lN8LR/xCPeYEyuP9efic40xyiifT8YfaZOV+/ENFSSbktbnvDYe6xOus343x9GbbI8eIW6gTIClJ4StqU7jrKTUTU7/GLqSSBAb/XjUXZ/l32e/h29ESVA6BVQBFmSWI0ufORTcIoOZIOw51P6YvroFz+2Dpg1bs6cfmNxIzTJ05DOIz77KRr80XdXMGnJ37HuUEnyNXbezgWyX688z+xJjJPqm+dLFek2fQt4EjXY7BfuN770RNPlzQ5+j//d6PC7VLcgTCIfziHJy/v/u082ZSZlA+eBy0DfP+Ypy20ae34AAL9dmSKjcutyAw12TiJAfIgb1zgIX6bIWawbL3YKF95Dg5QA7cggO/KNR02C0cxjXJO3KAHCAHHosDLNQ8UfMzZXKAHCAHyIENc+DPnz+H+PskFZtXIrA3BCLB+bM/BOjX/fmUFvUIIM9ZqHt82LITBJDoOzGJZgQE6FfS4BEQQJ6zUD+Cxx/URiT6g0KwS7Pp1126lUYZBJDnLNQGHD7uBwEk+n6soiX0KznwCAggz1moT/T411v4y0VvXyfO5rRrIIBEv8Z6XOM6CNCv18GZq9wWAeQ5C/VJvvg6vIU/Z8o6fRJ4V5uERL/aolzo4gic369/Dx8vjOeLO44LHIUA8lwV6nRKDAXo5eOvEvj346WeHtP909uhP0si2fN9/DvgbjH7egt/SzucSF8+Dnoltez4QeZHGXaNYV/TKf99cmeuWXF4ao5rWN3/fhxeij4i3+JoxPPxwggg0eNSmbvZ75o3Djdc4l5YYYpfhYD1a5wkvlUxZ3KBxOVTl78wd61SgYOIwMURQJ53hfrlJRRlQ2Qs1IdQot3TZAoKKeCZ+EmWLWjBvFgA396cYrfG9FIQWx4N+sgas76wJYi7Zi+QVRvoMCrUsV3NKQmh6RSEFF3UOJDN28sjgERP/hCeJPcgz5moL++N862g/Fry0cvHVx/fzpJ+TNP/DlRsujECyPO+UBfC4+evulDnQov90R4dAEL8WNRfDuqAngpYKOjdqTTPGe96C2ppvpEpgM76vEKdFe82JiJO21RbjU1Zb7cgq82LzOf1Wggg0bs1FVeEr90o0+CMMzyWk13m8YCnRiofj0PA9+skDkW88rk0xqv4NR9CfN/lMX5+kvkgU/Gi5MGP8iaxHITIFcCLtx0CyHOnUIeX0YbQtlCHqmyKWyZ4O1E24trTZ5WliGwLfXmVBSegZoUEjJcEl/v6gmp1byu5hdrobbFqs+NdCdCT3u9rSXw6HgEkejdbcVh4I6/FPW5FCY3XVR7yIcWNvFWqI3hzZgR8v2bf9PHdFnfjOXWL/5vf9djS3xJcCPuFNzLIi3LqV28qyZXmGN65CCDP/UIdpiWilkJZi2sVZ4qbImUcBAlNJUQoXDjHbAzyMjC2rttukn7pM+EWXNLr940C2dgiQsJVB2vscGRMA24sG5bh7YUQQKLrJRw/wgCdhKEDeS3NHY8H38uQ8bz+GgHfr3Of5k3zyDd5LtThGPzteyhujGN+WpjvfVyYZI70+TVEFLADBJDnw0ItxI471L5QlxNvYXYsaHoni8SFYoXkx/tC2vZaaelk07yQk2pfrOMI3TcKZNCviU13XaFOepq1vLYqB4O5NvLmSggg0XHJ5Ff3bY2MGnECeV3GIo9jE3BZx4TI5vW3CPh+HcV3Xi3lgqHPF/wafWy+tyOb9pwCF+Z7hTqqRa78lgq7no88nxTqgEEh6Ad867siUwuUV4w0caXQq4KOCS7JOvWVoV6r6pdusC/fd8nTDcIsxRZqsUOvMUrqYdREtpbBp0sggEQX+ctFOoys3JZZckU+lTbksQxL1wkv1Dg+HIuA51cpnF18r/LFgl/d/IR5b2H+qFBXw8mVCgVvKgLI83mhLq/60klXvReKskrhi98S73aqhriJ6PGUDMVYJbgiS33rrOqrbmKxVMGYimE+5c76qr64RtFLyYPVdKE2NsG4VMDdL83ZNw0wibcXRwCJLv7vuRr3U/otyayYK04Irzv+R9PGfLm44TtfQPtVjB3nkByfkHtkSr06vnLyE36BNskEv895sVSInfWrbrx5VASQ5wuFOkAkyagr1LErfqHC+5zFEi8/I9HVZ0DJE2UM/ltkCITmLDsOk+wxfXHjgHPbCnLXB98k2NOGQV7ZL8uWNXi9HAJI9Px2A/1T7iOvre9c3omeOenmj2kCHzChWzlOzIgUXk9HQPm1flFL+7ZtvrO/2rO3bs4byl3o1zQF/R7W6jiC/YYX3omaXPEcwTZAAHmuCjWM4a1BIG1KVCSbAXzcHAJI9M0pR4VORoB+PRk6TrwjBJDnLNSrHOfsuFfN46BbIoBEv6UeXPu8CNCv58WT0raJAPKchXqbPqJWZ0AAiX4GcRSxEQTo1404gmpcFAHkOQv1RaGm8FsigES/pR5c+7wI0K/nxZPStokA8vzp5+fnwF9iQA6QA+QAOUAObIsDsVjHXxZqblS4USMHyAFygBzYIAfkrM9CvUHncFe7rV0t/UF/kAPkwC04wELNAs0dNDlADpAD5MCGOcBCvWHn3GLnxjV5YiAHyAFyYFsc2Gyh/nwNf/Xn9ZO7PG4kyAFygBwgBx6aAxst1J+H1/AnRF8/t7Wr4S6T/iAH9syB78P7M/MOOb49jvuF+vM1/e1u+d9NqoI57MsklzlyVXPNrnB4ao5rPL8fvnH89/vhGf/+d7h/fv9+6F0WA+r0gPp+f04c7zhkeKb4O+tDrvL+ZnEpfs355/nw/l04YvJW5/fkMxZq5pTTc8olsesLdUlGLUGF060UzVnfTya5CoASHKoNktioUMd2NafIaToFMIsuahzIviRolL1NMq/zS35b8/z+mU5Pmj/SVzaAiXeS7Gd994zHfnRPRVpylcoF0XfiR8kd8FzHslCvi6H9cOZe7B0Uao/EM4JHxzmFOgZASnavh88aDM3JfqE2QTWSuyD7XhxAPRsfrouFw9eOq5C4Z32K2zBH2uNcKCDDU5+M5/WEE7nNG8CrtKnHHDQaK76L/fJ/4bK5MI+RN4ZPTyhX5sPayvdl3Xd5Y5nnkg+AF7nvcr8v1KUwPuEOtIInJLXkjUDnPn1Cie2Z9Oo0XOS5hVoRO8xPQeatJ7JHfbGfv8RgxIGer/2JrI2Z9WmM8xzFd+R0VzRG+rFd47qAR8kTz+Ez5lpE4cuoKdekotp82svPfZj7dI4q/SA3F1kp1gu+L7lQFXfygXl6RZ1yCnUOiEzsSPq+EPp9owA4plA7MqZEHsvug3Ah0FeARZl7wrDnWl+Mfw6R63HzOevTvFhI1onP/MKSxuwMvIqbIcxVDs41b8HbDa3HGt9JURadyyk5fRa+MN87tDh6ap1kHV4fGZdhoRZQ8o6xL9axX/f1iS/LGBfTFDiwO3VPz4nI/vr5tD7qI7HFh7x6XOj52hfjNmbWp/FdStZBl5Kc48mvfwPl6co2jbGDRyrUWESb7+xbPZ23UNaC77o14lycg/dFLr5N8Qp1PCCQDzxVLxwUFwu1JiKS2idpl3hccmc5tlCnAMLCnZQfF/rZ59+Lgb0ADOdbX+/tGRN5sa3jKiTeWZ/iEsyRdpWsEccJt2Uur+uSeLehb/492yYrrYGbgejLX56olX/JB+ZdzA/tvivUkdSq2KYElU+tsz4p6Gpu2SmqNiCmLtROgitjU6Dha63YviCbDm9OJhYeFi2RN3xMolTFedan5SteF57il8naemPOtzFaNttHeBh/Qt7qNvTFJ+p7BCnXOP5Qm6zcj3+IyW4C5r43HIJcmP3qrN+NGdnP9j3HRleopeDWL2SoAlmI6n4j0vb5n28jmD2p7W4VyJcCD74oovSCcST2uhPIw+KUk2Xjd+ZU3UxKYU0cNx+rzPoUnrhG4DQme8vj7g0SuYw54qh75R/9PYCUa2reGn3k4BRK9F3yMfo2cKf7vBv7je+9V9/kA/OVyh1+/DuF2h94VMCsWNjKSztTJi2S9gTuWC7x+TYxTNyJOzlwGQ5spFA7O1kmbBZtcoAcIAfIAXJA6vSB/z9qkoEJgRwgB8gBcmCDHJBKzUK9QefwNdJlXiMRV+JKDpAD98QBFmoWaO6gyQFygBwgBzbMgT9//hzi75NUbF6JwN4QiATnz/4QoF/351Na1COAPGeh7vFhy04QQKLvxCSaERCgX0mDR0AAec5C/Qgef1AbkegPCsEuzaZfd+lWGmUQQJ6zUBtw+LgfBJDo+7GKltCv5MAjIIA8Z6G+sce/3sJfOXr7urEW+1weib5PCx/TKvr1Mf3+aFYjz1mob+r9r8Nb+POGrNOXcQIS/TIrUOotENiuX/8ePl4Yz7fgxB7XRJ53hfrvx0v6H7C/fPwF2zMB1d9JhuoST4Vq/NdbkPFyUCKSNEfO09sBz5PphAl/m/dJ9bf5sHzTM60bTqgvHwfUPg/Ic5WeoSPZW8c3+WirXmvNmKZSvBuemqO+de0y5+/H4UXZb7DVovk0QQCJjsOEY5YLOCYQQ/lBceDUPrUAH05FwPpVcpbErPIVLiL5wcSX5JipnJnP6xo5NwzXr+N4QwSWEUCeQ6HOp7uXj6+0K9RJbE5AVagLoX2yZjko2xYxJSvYkoNHinmZ/xI2E7bAhbFx7tubU/wSJv3asdkr1KhfEGo2Lo6cbkxasP7H2igd1lZZS2FX8FQ6iQBepwgg0evAtDl6O7yFk88YU4mFst1L/pWN56l9VQPe/BIB5dcYH5ALdL5YXqjG5lTOzOe4Rs4NKn6xm/dE4AgEkOdQqEWCU4jC+XT2SqcVnEzoMVEd2SlxthNwkyX6oEzRI7ZJ4izjUkELBd3IEymhJDsbkBWFOgpIidpsFuzrAjWmrZqne59DWxt8/ZKkiWy9Ep8QASR6bhf+TLCOAzu8Zd4v+rIC5b8gT9rjml3BCbxJpz/DdZnzoNferwBEygMr8ZqNxb4ZH2BpyTFvXzln+b7LvpfTv5zms5glXpSc8ZEPDzJXvwlYabvSmw9bRAB5flShbuTSZMjF1TuJW/MzEdtJppAWKntXqDFgYMNgxyWyRjkm4TUN7Nq5Z/FEnYblwMtq+nLCwsPPm6Ou3RfGrJ7KzqZ1visB2r/PtwP5DAgg0WNz5chg0yZTNSfSzLrJO7VPZOdr5hDQXvM2cUE2hnomnxb+HXVXVMeIuXEpw0HOzOcyPF9LPoNDhF6j9IPjk+z68d4CL0qOkQKd1iRXtAt29IT5a2Wh1tZrcsUcs3bnL0SW8f3rxyirFfIiu540gMgQSLlIls2DLYBV9X7tuvEw8nH9PB2LcJYzH1MXTTc6WGOTI2MacLi+ls2nMQJI9PyZsxQ/B38Q0yfmzMPo81P7QHy4zetDvo4LtBN14gK/lKQxa0/Kr6254trHphpUHmYxpfkx87mWvMavwkGZiZvwhfneYYBcESB3d0Wen1Soc2FsiUSKqy3gPXI6APpXjDkh1gIaX/vVIhqlIZEh0DDJ4b1SwKxd+nQQ+mO0vWvGqIXzRgazcgou/VYiFxLTVsVgMNdG3iwg0IiOvImTRj7MAjUn9PhT+7Jk+a/VJ7Rb3pYEHGNhXeER2fu/Nr9qW9OGWOUL3Y9PvR9br5XTjx3xZ8Gv0cf19Czr4Ry8L/2KF5DzZHq8kiuIxm7ukeenFWpTaKRQSwLsXvNW6CzB7XMu1OPEpImcAigUwLZ+WEgRuy4cbvq1Yq8OQn9MklkDbM0YXDfbhJiI3mbU8NW5Xl/P4tMYgUb0nODUBlC++esl9i6hAu9O7VNqgjxpH/J2kJxl3gNem1+b8ba4th7vboypK2fmcyV+wa8pb575RK3WH9ulhvHhLhBAnq8q1F9v+qRnyawKZXk94xdbp8iZINCyLJ4mEOpOEsg/THjO2kH8YqEuazR7HDndGK13wqueqI0NMDTpAp9vpa4F2TCdtwYBJLru6n2ofWQSnuLoqX1aA7We8NjbNKi3SFrGoz5pv2Zf6jdvDRmFc2nOcQY5I7XP5Mx83taSw0AN9dil8lFZAwbo/GM29R0vjB64dLrP8kF8N4IN94MA8hwKdSaBPXWkApUSVftc2QZFDIZWyAIQhWA9YTKR1FjzuUsnS+FqidgTXwcGTvbW9gu1xkBvUiQY52NwXS/4bJKA8RZrW7hhKG/nCCDR9cieC11ClyTpfev61D6lBMZb4AMmdMuBPpCUpEd7UH61WMmbkoJZ59fRQWJBjuS0HPc2J4gHbH4K7ejXNAz9bj/aiwOw3/DC5MosTr4BXvIzuZJg2cN/kOdQqPdg2vZtSDtoBtNVHIVEny6YCu9k8zSdzM5rI0C/XhtxrncLBJDnLNRX9YCz477q+o+1GBJ9ZnncPOm3PLPR7Ls1AvTrrT3A9a+BAPKchfoaiHONmyCARL+JAlz0IgjQrxeBlUI3hgDynIV6Y86hOudDAIl+PqmUdGsE6Ndbe4DrXwMB5Hn4bgR8SYz35U82EhPyghwgB8gBcuC2HJANwf8H84y+s1nEsG8AAAAASUVORK5CYII=" style="margin-left: auto; margin-right: auto;" /></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Memory and Drive Latency</td></tr>
</tbody></table>
<div>
<br />
Next I would like to delve into VMware vSAN because many of our datacenters are now turning to hyper-converged architectures that run vSAN, I thought I'd hit on some of the salient points.</div>
<div>
<br /></div>
<div>
Disk groups should be a key considerations when architecting for vSAN and how many to use per host. Another is all-flash verses hybrid. As the cost of flash based storage becomes even less expensive hybrid arrays do not make as much sense to implement. vSAN limits the feature set of hybrid compared to all-flash. Hybrid arrays are not capable of erasure-coding (RAID5/6) or compression and de-duplication. Hybrid designs will consume all cache you provide and use 70% for reads and 30% for write caching. 10% capacity of the capacity tier is the recommended capacity for the cache tier. However, a relative relationship exists between the cache tier capacity and the host memory consumed. Increasing the cache tier will cause and increase in host memory consumed. </div>
<div>
<br /></div>
<div>
All-Flash typically makes more sense considering cost, heat, performance, and reliability. All-Flash is a little different in the case of features and cache. Specifically for cache 100% is dedicated to write caching however it's limited to 600GB. Larger capacity drives are supported and will enhance reliability due to write leveling. Keep in mind the goal is to flush cache to capacity and thus data protection. Read caching is not necessary. Flash drives do not have mechanical limits so I/O can occur more rapidly. For performance and to limit the amount of memory consumed away from VMs I prefer the Optane (375GB) drives matched with either SAS or SATA SSD capacity drives. VMware recommends architecting cache tiers with faster drives compared to the capacity tier. For example, leveraging all NVME drives in the capacity tier, Optanes are recommended in the cache tier.</div>
<div>
<br /></div>
<div>
Another consideration is that when using NVMEs Dell VxRail systems require dual processors. Check the vendor specifications for NVMEs and other directions with vSAN as using different drive technologies may require other server host considerations. I also prefer to use at least 2 disk groups per host especially in production due to the fact that if a cache drive fails the entire disk group fails. Using 2 disk groups per host will increase the availability of the architecture.</div>
<div>
<br /></div>
<div>
Ultimately, isn't that what we are after? Availability, reliability, and performance.</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
Craig Herringhttp://www.blogger.com/profile/05719502208585022033noreply@blogger.com2tag:blogger.com,1999:blog-3594227605547371558.post-48410241222153541032020-02-18T08:32:00.000-05:002020-02-19T16:28:40.514-05:00My HomeLab<span style="font-family: inherit;">Current Lab configuration</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">vSphere 6.7 P1</span><br />
<span style="font-family: inherit;">vSAN All flash FTT=1 Raid 3+1</span><br />
<span style="font-family: inherit;">VRA 7.6</span><br />
<span style="font-family: inherit;">vROPs 7.5</span><br />
<span style="font-family: inherit;">vRLI 4.5</span><br />
<span style="font-family: inherit;">NSX 6.4.6</span><br />
<span style="font-family: inherit;">VLC 3.9.1</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">Total of 6 VMware Hosts</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">Supermicro X9DR3-F</span> (<span style="font-family: inherit;">Ebay for $200 each)</span><br />
<span style="font-family: inherit;">Supermicro 2U Chassis, 8 hot-swap 3.5"</span><br />
<span style="font-family: inherit;">128GB RAM each ($160)</span><br />
<span style="color: #565656; font-family: inherit;">Dual Intel(R) Xeon(R) CPU E5-2650L 8C @ 1.80GHz ($140)</span><br />
<span style="color: #565656; font-family: inherit;">Dell H310 LSI 2008 HBA (flashed to IT mode and Q-depth 600) ($40)</span><br />
<span style="color: #565656; font-family: inherit;">Emulex OneConnect OCe11102 Dual port 10Gb NIC ($40)</span><br />
<span style="color: #565656; font-family: inherit;">WD Raptor 300GB - Boot Drive</span><br />
<span style="color: #565656; font-family: inherit;">Misc Cables ($40)</span><br />
<br />
<span style="font-family: inherit;">Supermicro X9DRI-F+ (</span><span style="font-family: inherit;">Ebay for $160 each)</span><br />
<span style="font-family: inherit;">Supermicro 2U Chassis, 8 drive hot-swap 3.5"</span><br />
<span style="font-family: inherit;">128GB RAM each ($160)</span><br />
Dual Intel(R) Xeon(R) CPU E5-2650L v2 10C @ 1.70GHz ($140)<br />
<span style="color: #565656; font-family: inherit;">Dell H310 LSI 2008 HBA (flashed to IT mode and Q-depth 600) ($40)</span><br />
<span style="color: #565656; font-family: inherit;">Emulex OneConnect OCe11102 Dual port 10Gb NIC ($40)</span><br />
<span style="color: #565656; font-family: inherit;">WD Raptor 300GB - Boot Drive</span><br />
<span style="color: #565656; font-family: inherit;">Misc Cables ($40)</span><br />
<br />
3 hosts based off each design.<br />
<br />
3x E5-2650L based hosts = $1,650<br />
3x E5-2650L v2 based hosts = $1,530<br />
<br />
vSAN Storage<br />
Cache Tier<br />
Intel SSDSC2BX40 400GB (5)<br />
Samsung NVMe 960 (1)<br />
<br />
Capacity Tier<br />
Samsung SSD 860 EVO 1TB (16)<br />
Intel SSDSC2BX40 400GB (3)<br />
Crucial CT240M50 250B (6)<br />
Crucial CT480M50 480GB (1)<br />
M4-CT256M4SSD2 250GB (1)<br />
OCZ-Agility3 250GB (2)<br />
<br />
Storage SAN / NAS<br />
<br />
FreeNAS - 69.6TB<br />
X8DTH-6F - ($400)<br />
Supermicro 4U Chassis, 36 drive hot-swap 3.5"<br />
Dual Intel Xeon L5630L 4C 2.13GHz ($50)<br />
48GB RAM ($75)<br />
Boot Drive<br />
10K SAS 500GB<br />
Disk Group 2 - RAIDZ 6 - 18.1TB<br />
10x various 2TB 7K RPM disks<br />
Disk Group 1 - RAIDZ 6 - 19TB<br />
7x various 3TB 7K RPM disks<br />
ARC Cache 40GB<br />
Disk Group 3 - RAIDZ 6 - 32.5TB<br />
9x various 4TB 7K RPM disks<br />
ARC Cache 60GB<br />
<br />
Networking<br />
2x IBM G8124-E - 24 port 10Gb SFP+ ($850)<br />
4x SFP+ 1Gb GBICs ($80)<br />
Cisco SG300-28 ($528)<br />
Cisco SG200-26P ($250)<br />
<br />Craig Herringhttp://www.blogger.com/profile/05719502208585022033noreply@blogger.com0tag:blogger.com,1999:blog-3594227605547371558.post-5848190070387873472019-08-08T09:29:00.002-04:002019-08-08T09:29:50.093-04:00VMware vSphere learning paths<br />
<div class="MsoNormal">
These days there is plenty of training for all things vSphere. The issue has become what is a good path to either a specific certification or simply learning because you want to become more proficient with day-2 administrative activities. A colleague came to me the other day with this delema so I decided to put together a quick list of free and paid training resources.</div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
VMware Hands on Labs can be a useful free tool in learning
about many different VMware products in a safe isolated environment. The
following are a couple useful labs for learning more about vSphere.</div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<a href="https://labs.hol.vmware.com/HOL/">https://labs.hol.vmware.com/HOL/</a></div>
<div class="MsoNormal">
HOL-1910-01-SDC - Virtualization 101: Introduction to vSphere</div>
<div class="MsoNormal">
HOL-1911-91-SDC - vSphere 6.7 Lightning Lab</div>
<div class="MsoNormal">
HOL-1911-01-SDC - What's New in VMware vSphere 6.7</div>
<div class="MsoNormal">
HOL-1911-02-SDC - VMware vSphere with Operations Management - Getting Started</div>
<div class="MsoNormal">
HOL-1911-03-SDC - VMware vSphere with Operations Management - Advanced Topics</div>
<div class="MsoNormal">
HOL-1911-04-SDC - VMware vSphere Security - Getting Started</div>
<div class="MsoNormal">
HOL-1911-05-SDC - VMware vSphere Automation - PowerCLI</div>
<div>
<br /></div>
<div class="MsoNormal">
VMware Learning Zone provides some free and paid on demand
classes.<o:p></o:p></div>
<div class="MsoNormal">
<a href="https://www.vmware.com/education-services/learning-zone.html">https://www.vmware.com/education-services/learning-zone.html</a><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
All VMware vSphere classes<o:p></o:p></div>
<div class="MsoNormal">
<a href="https://mylearn.vmware.com/mgrReg/plan.cfm?plan=112281&ui=www_edu">https://mylearn.vmware.com/mgrReg/plan.cfm?plan=112281&ui=www_edu</a><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<o:p></o:p></div>
<div class="MsoNormal">
VMware ICM</div>
<div class="MsoNormal">
One of these following classes are required for a VCP
certification<o:p></o:p></div>
<div class="MsoNormal">
<a href="https://mylearn.vmware.com/mgrReg/courses.cfm?ui=www_edu&a=one&id_subject=84058">https://mylearn.vmware.com/mgrReg/courses.cfm?ui=www_edu&a=one&id_subject=84058</a></div>
<div class="MsoNormal">
<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
VMware ICM Fast Track<o:p></o:p></div>
<div class="MsoNormal">
Additional material is taught in this class compared to the ICM class. The classes also typically run from 8AM to 6PM for the week.</div>
<div class="MsoNormal">
<a href="https://mylearn.vmware.com/mgrReg/courses.cfm?ui=www_edu&a=one&id_subject=84058">https://mylearn.vmware.com/mgrReg/courses.cfm?ui=www_edu&a=one&id_subject=84058</a><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
VMware vSphere Operations</div>
<div class="MsoNormal">
<o:p></o:p></div>
<div class="MsoNormal">
These are not required but can help with the test although the focus is on day-2 administrative tasks on the vSphere platform.<o:p></o:p></div>
<div class="MsoNormal">
<a href="https://mylearn.vmware.com/mgrReg/courses.cfm?ui=www_edu&a=one&id_subject=86055">https://mylearn.vmware.com/mgrReg/courses.cfm?ui=www_edu&a=one&id_subject=86055</a></div>
<div class="MsoNormal">
<o:p></o:p></div>
<div class="MsoNormal">
<a href="https://mylearn.vmware.com/mgrreg/courses.cfm?ui=www_edu&a=one&id_subject=84216">https://mylearn.vmware.com/mgrreg/courses.cfm?ui=www_edu&a=one&id_subject=84216</a><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Certification Learning Paths.</div>
<div class="MsoNormal">
This will provide you the path
you need to follow to obtain a certification.<o:p></o:p></div>
<div class="MsoNormal">
<a href="https://mylearn.vmware.com/mgrReg/plan.cfm?plan=108323&ui=www_edu">https://mylearn.vmware.com/mgrReg/plan.cfm?plan=108323&ui=www_edu</a><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Other resources include blogs and community forums.
Purchasing VMUG Advantage provides lab licenses for most VMware products.
Building a ‘HomeLab’ can be a good way to practice without using the corporate
environment to practice on. Also, your company may not own licenses for all
products and all features of those products where VMUG Advantage does provide
full featured licensing.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<o:p>VMUG Advantage (there are codes all over the interwebs for 10% off)</o:p></div>
<a href="https://www.vmug.com/vmug2019/membership/vmug-advantage-membership">https://www.vmug.com/vmug2019/membership/vmug-advantage-membership</a><br />
<div>
<br /></div>
<div>
List of blogs and other resources</div>
<div>
<a href="http://thevpad.com/">http://thevpad.com/</a></div>
<div>
<br /></div>
<div>
Hope these help you in your VMware journey!</div>
<div>
<br /></div>
Craig Herringhttp://www.blogger.com/profile/05719502208585022033noreply@blogger.com2tag:blogger.com,1999:blog-3594227605547371558.post-80670642433198691902019-07-24T20:01:00.000-04:002019-07-24T20:01:24.831-04:00Dynamic DNSDynamic DNS, even standard DNS services that offer many configurable options can be expensive. The free ones or the ones that come with domain name registration are typically limiting and most do not support dynamic IPs. I have been using one for a number of years from a co-location and service provider called Hurricane Electric <a href="http://he.net/">http://he.net/</a>.<br />
<br />
I learned about this provider while living in the Bay Area outside San Francisco. They would host a Linux user group and as a matter of fact still do 20 years later! EBLUG <a href="http://www.eblug.org/">http://www.eblug.org/</a><br />
<br />
One of the many great services HE provides is a free DNS service that contains the ability to configure dynamic DNS entries in the event you have a dynamic IP on your internet connection or need an easy way to failover some internet facing service with something a little less expensive than a GSLB. The service has been limited to 50 zones for the free version. Com'on!! For real??? Everyone owns more than 50 domain names... NOT! This is super cool of them to not only offer this service for real but then allow users to host 50 zones!<br />
<br />
Getting started is easy. Once you have a domain registered, either new or existing, simply point your root name servers to HE's servers. Let's use the domain vmuglabs.net. I use GoDaddy for my domains so once in DNS management browse over to Nameservers. Once there change the GoDaddy name servers to HE's. They are:<br />
<br />
ns1.he.net<br />
ns2.he.net<br />
ns3.he.net<br />
ns4.he.net<br />
ns5.he.net<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjJgZLH4fsX7UjRtkaUqxxIlrh8iQJmXS83uQyPH9GxbzcEZNs-Na3MVFAxTodyFs92O8rLT_VWh7wW9WUw_ad16ldLkIKdEac64sVBSQjF7NEz4MweaaR7OHmWohxn4vGR0896q3mzI1c/s1600/Untitled.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="863" data-original-width="1525" height="226" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjJgZLH4fsX7UjRtkaUqxxIlrh8iQJmXS83uQyPH9GxbzcEZNs-Na3MVFAxTodyFs92O8rLT_VWh7wW9WUw_ad16ldLkIKdEac64sVBSQjF7NEz4MweaaR7OHmWohxn4vGR0896q3mzI1c/s400/Untitled.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Now if you don't have an account at HE go over to https://dns.he.net and register for an account.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvSw_GIQrm1KTXTfh4Pe1mpV6BKj2rmfyUyKk9QTd3tXII4-DNu2Komp40G3iw0bCak10B2mOtYDQL_iujyR3o60FcCZ3DpRlxBeAS_CD-m_900ngHRt0KoIXPyXZTh7u1shDApBX3sLk/s1600/he1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="577" data-original-width="1034" height="222" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvSw_GIQrm1KTXTfh4Pe1mpV6BKj2rmfyUyKk9QTd3tXII4-DNu2Komp40G3iw0bCak10B2mOtYDQL_iujyR3o60FcCZ3DpRlxBeAS_CD-m_900ngHRt0KoIXPyXZTh7u1shDApBX3sLk/s400/he1.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Once logged in you can add a new zone or domain from the menu on the left</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGUYR6ufHqJt51egwnyeCgIa1ZpikWMD1SBFVrUS9lS3V8hdYQ7fjAP3E12-0mOHt9G3jlOkRmBRrueVM3W0RHeTNW-AsCd3ZIzuUWAf7lDUtULhGd1TU6d8WkiInqEkd5ZcR18Eks614/s1600/he1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="175" data-original-width="938" height="73" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGUYR6ufHqJt51egwnyeCgIa1ZpikWMD1SBFVrUS9lS3V8hdYQ7fjAP3E12-0mOHt9G3jlOkRmBRrueVM3W0RHeTNW-AsCd3ZIzuUWAf7lDUtULhGd1TU6d8WkiInqEkd5ZcR18Eks614/s400/he1.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Once created you can edit the zone by selecting the edit icon just to the left of the domain name. Within the zone you will find 6 total records, 1 SOA and 5 NS records. Next is to create an A record and investigate how the ddns option works.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEidr2m0muTneJUxhZv3z1yFY3Gn_4SeBxmgrxr-5sMFvnZwHDAgWwDyyIbHtLFUkSvUBg_IvU9ccU4w2c_vA6XoNMmSHPqn_iZ0lVn1S3gSG1Yw3bkWkClN6J550SSmQ7gO2hhc866e8vs/s1600/he2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="347" data-original-width="630" height="176" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEidr2m0muTneJUxhZv3z1yFY3Gn_4SeBxmgrxr-5sMFvnZwHDAgWwDyyIbHtLFUkSvUBg_IvU9ccU4w2c_vA6XoNMmSHPqn_iZ0lVn1S3gSG1Yw3bkWkClN6J550SSmQ7gO2hhc866e8vs/s320/he2.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Once created you will need a way to authenticate to dynamically change the IP for the A record. HE uses a DDNS key, not your login account. To generate one select the change symbol.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEibA7VtOUaJ7osCWDKqvaCPwSrPLXu62sIUIPoXLzyE-QijMJcMCNDAzuhBjCpeadPJ0AxtNNVZWfOg3Lg52yGEuAOZoxyzwELDnofREiGGN-7tBOAU2eyd9NyPCcxF7nbOvGNVy2rKyJk/s1600/he3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="389" data-original-width="841" height="148" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEibA7VtOUaJ7osCWDKqvaCPwSrPLXu62sIUIPoXLzyE-QijMJcMCNDAzuhBjCpeadPJ0AxtNNVZWfOg3Lg52yGEuAOZoxyzwELDnofREiGGN-7tBOAU2eyd9NyPCcxF7nbOvGNVy2rKyJk/s320/he3.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Generate a key and copy it.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhOadfVsclpHNlX1CcfvV1aPPQi82OUuYNKqq0lb_0yQVURo9Id63aWOyRPC_713uPX365OodDhgEJkK6Qr-tMeghjTL89rjRzV7h7yiJrQvP44sVHupsy9TzEov2zIVcgzXEtSl3mQijc/s1600/he4.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="305" data-original-width="618" height="157" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhOadfVsclpHNlX1CcfvV1aPPQi82OUuYNKqq0lb_0yQVURo9Id63aWOyRPC_713uPX365OodDhgEJkK6Qr-tMeghjTL89rjRzV7h7yiJrQvP44sVHupsy9TzEov2zIVcgzXEtSl3mQijc/s320/he4.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Once you have the key its time to build the bash script to facilitate the ddns change. the code can be located in github.</div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://github.com/craigeherring/he-ddns">https://github.com/craigeherring/he-ddns</a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Run this script and if your IP changes your DNS record will be changed. To test you can manually change the IP within the HE DNS console and observe it changing back when the script is run.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
Craig Herringhttp://www.blogger.com/profile/05719502208585022033noreply@blogger.com1tag:blogger.com,1999:blog-3594227605547371558.post-86482619708175118672019-03-12T14:43:00.000-04:002019-03-12T14:43:35.846-04:00Fix MSDTC for VRA Install Wizard ValidationDid you use a template to create the IaaS servers for VRA? This is a quick post on how to resolve the errors from the VRA validator step. Perhaps like you I had some trouble locating a concise KB article or post on an easy way to resolve these issues.<br />
<br />
<span style="font-size: large;"><b>Reset the CID/SID of the Server</b></span><br />
<br />
Log into the IaaS and DB servers as Administrator.<br />
<br />
Opening <b>REGEDIT</b> can show what the CID/SID values are. This is located:<br />
HKEY_CLASSES_ROOT\CID\(CID)\Description\(Default)<br />
<br />
Open a <b>powershell</b> prompt as <b>administrator</b> and run the command:<br />
<br />
Uninstall MSDTC<br />
<code>msdtc -uninstall</code><br />
<br />
Reboot<br />
<code>shutdown -r -t 0</code><br />
<br />
Re-install MSDTC (login with same permissions as above)<br />
msdtc –install<br />
<br />
<b>Warning</b>: The msdtc command does not give any return response when running this command.<br />
<br />
<b><span style="font-size: large;">Open the Firewall</span></b><br />
<br />
Enable the firewall rules for WMI and DTC on both computers by using the Netsh utility. This<br />
<br />
<code>netsh advfirewall firewall set rule group="Windows Management Instrumentation (WMI)" new enable=yes</code><br />
<code>netsh advfirewall firewall set rule group="Distributed Transaction Coordinator" new enable=yes</code><br />
<br />
<br />
<b><span style="font-size: large;">Testing</span></b><br />
<br />
Basic checking can be done by opening the Component Services MMC. You should see something similar.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgSVJZyYOqC99nPsYBuP1kLJpGF14Jhw1Ckb_ulGrPooL8GerWYUzo9Q9AnVfDspSuxo9tvclULR1EXBlq77eXqFytk3a8guA0yuyREGENcdmfOmpv0XewSGTChr0uqUZFmrT2ceBQ_SdI/s1600/Screen+Shot+2019-03-12+at+2.31.34+PM.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="392" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgSVJZyYOqC99nPsYBuP1kLJpGF14Jhw1Ckb_ulGrPooL8GerWYUzo9Q9AnVfDspSuxo9tvclULR1EXBlq77eXqFytk3a8guA0yuyREGENcdmfOmpv0XewSGTChr0uqUZFmrT2ceBQ_SdI/s640/Screen+Shot+2019-03-12+at+2.31.34+PM.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Component Services MMC for MS DTC</td></tr>
</tbody></table>
<br />
<br />
Run the DTCtester to test the state of MSDTC. Below are some example tests that can be run to test local and both local and remote DTC connectivity.<br />
<br />
<b>Test MSDTC on the local computer</b><br />
<code>Test-Dtc -LocalComputerName "$env:COMPUTERNAME" -Verbose</code><br />
<br />
<b>Test MSDTC on the local computer and a remote computer</b><br />
<code>Test-Dtc -LocalComputerName "$env:COMPUTERNAME" -RemoteComputerName "remote-server" -ResourceManagerPort 17100 -Verbose</code><br />
<br />
<b>Test MSDTC on a local computer that blocks inbound transactions</b><br />
<code>Test-Dtc -LocalComputerName "$env:COMPUTERNAME" -RemoteComputerName "remote-server" -ResourceManagerPort 17100 -Verbose</code><br />
<br />
<b>Test MSDTC on a local computer that blocks outbound transactions</b><br />
<code>Test-Dtc -LocalComputerName "$env:COMPUTERNAME" -RemoteComputerName "remote-server" -ResourceManagerPort 17100 -Verbose</code><br />
<br />
<br />
This is the result if the first test partially fails. The 3 local and remote tests will also show the CIDs for the communicating systems. Referring to the REGEDIT above will display the UIS and the XA values that are contained in the CID subkeys.. From this output you will be able to determine if the CIDs are unique as another method to validate the registry values.<br />
<br />
<code>
PS C:\Windows\system32> Test-Dtc -LocalComputerName "$env:COMPUTERNAME" -Verbose<br />
VERBOSE: ": Firewall rule for "RPC Endpoint Mapper" is enabled."<br />
VERBOSE: ": Firewall rule for "DTC incoming connections" is enabled."<br />
VERBOSE: ": Firewall rule for "DTC outgoing connections" is enabled."<br />
VERBOSE: IN-SQL02: AuthenticationLevel: Mutual<br />
VERBOSE: IN-SQL02: InboundTransactionsEnabled: False<br />
WARNING: "IN-SQL02: Inbound transactions are not allowed and this computer cannot participate in network transactions."<br />
VERBOSE: IN-SQL02: OutboundTransactionsEnabled: False<br />
WARNING: "IN-SQL02: Outbound transactions are not allowed and this computer cannot participate in network transactions."<br />
VERBOSE: IN-SQL02: RemoteClientAccessEnabled: False<br />
VERBOSE: IN-SQL02: RemoteAdministrationAccessEnabled: False<br />
VERBOSE: IN-SQL02: XATransactionsEnabled: False<br />
VERBOSE: IN-SQL02: LUTransactionsEnabled: True<br />
</code><br />
<br />
This is the result when things look good for the installer to proceed.<br />
<br />
<code>
PS C:\Windows\system32> Test-Dtc -LocalComputerName "$env:COMPUTERNAME" -Verbose<br />
VERBOSE: ": Firewall rule for "RPC Endpoint Mapper" is enabled."<br />
VERBOSE: ": Firewall rule for "DTC incoming connections" is enabled."<br />
VERBOSE: ": Firewall rule for "DTC outgoing connections" is enabled."<br />
VERBOSE: IN-SQL02: AuthenticationLevel: Mutual<br />
VERBOSE: IN-SQL02: InboundTransactionsEnabled: True<br />
VERBOSE: IN-SQL02: OutboundTransactionsEnabled: True<br />
VERBOSE: IN-SQL02: RemoteClientAccessEnabled: True<br />
VERBOSE: IN-SQL02: RemoteAdministrationAccessEnabled: True<br />
VERBOSE: IN-SQL02: XATransactionsEnabled: False<br />
VERBOSE: IN-SQL02: LUTransactionsEnabled: True<br />
</code><br />
<br />
<b><span style="font-size: large;">Summary</span></b><br />
<br />
This is only one example of how to resolve these errors. If you used a template and a customization spec as you deploy while selecting "Generate New Security ID (SID)" your experience might be different.Craig Herringhttp://www.blogger.com/profile/05719502208585022033noreply@blogger.com0tag:blogger.com,1999:blog-3594227605547371558.post-76728168294075569992018-08-23T13:48:00.000-04:002019-03-12T14:48:46.379-04:00IBM 10G Switch - The Home Lab GemI came across the IBM G8124 while providing some pre-sales architecture to some of my clients. As HomeLab'ers it's difficult to afford some modern datacenter switch that we can afford. Most all 10G switches are over $1000 unless you are looking on the used market and then most of the switches are old and power hungry. It's easy to locate these on EBay and the prices have been dropping as they get a little older. Because most of us are using SDN (software defined working) they work very well in low cost lab situations where 10G offers some really nice benefits. When paired with some Emulex OCE11102 dual port 10G NICs it's possible to get a full 10G network for less than $500.<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhyazZJPl4ZjitiyhFaSf6FVv_-LYVRbePByRZaC_GWTW66MEXRe0AfOwz3EPEVYsB3fQElR0YrW8Dz48o6alBIA2M20Ra7WTxwbyvbCS-3WNF2FJjebDAu5O8qaDlA2K4hFdDeY4PylXw/s1600/Screen+Shot+2018-08-23+at+10.27.31+AM.png" imageanchor="1"><img border="0" height="37" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhyazZJPl4ZjitiyhFaSf6FVv_-LYVRbePByRZaC_GWTW66MEXRe0AfOwz3EPEVYsB3fQElR0YrW8Dz48o6alBIA2M20Ra7WTxwbyvbCS-3WNF2FJjebDAu5O8qaDlA2K4hFdDeY4PylXw/s320/Screen+Shot+2018-08-23+at+10.27.31+AM.png" width="320" /></a><br />
<br />
<br />
The G8124 is considered a Top-of-Rack switch that maintains some incredibly low port to port latency, about .600 nanoseconds. It also supports Virtual Fabrics and L3 routing with OSPF. This switch offers some really nice features when fitted in a HomeLab where VMware vSAN, NSX, VRA and other goodies want to be learned. If this solution is of interest one thing to note is for connectivity to the rest of your network you will need to have either a 10G interface in your existing switch or you will need a 1G SFP interface. The 2 x 1G interfaces are strictly for out-of-band management. Many IBM systems feature 2 dedicated management interfaces that require a different network than any SVI assigned and each management interface are required to reside on different networks as well. It is possible to only use a single management interface or manage the switch through one of the SVIs. While I wouldn't recommend this in a production environment for a lab, have at it, knock yourself out.<br />
<br />
The config is a little different than the Cisco language but not very difficult to get past if you are familiar with the concepts. Documentation and firmware are still available from IBM. Below are some links for this and model information.<br />
<br />
Firmware and Docs<br />
<a href="https://www.ibm.com/support/home/search-results/5422459/IBM_RackSwitch_G8124,_8124E_-_7309,_0446,_1455_7309?docOnly=true&sortby=-dcdate_sortrange&ct=rc">https://www.ibm.com/support/home/search-results/5422459/IBM_RackSwitch_G8124,_8124E_-_7309,_0446,_1455_7309?docOnly=true&sortby=-dcdate_sortrange&ct=rc</a><br />
<br />
Model Info<br />
<a href="https://lenovopress.com/tips0787">https://lenovopress.com/tips0787</a><br />
<br />
Example code (shortened to remove redundancy)<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">version "7.11.9"</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">switch-type "IBM Networking Operating System RackSwitch G8124-E"</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">iscli-new</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">system timezone 145</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">! America/US/Eastern</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">system daylight</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">ssh enable</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">snmp-server location "CloudRoom"</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">snmp-server read-community "HNET"</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">no system bootp</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">no system dhcp mgta</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">no system dhcp mgtb</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">no system default-ip</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">hostname "10gNET"</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">no hostname prompt</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">system idle 60</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">no access telnet enable</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">interface port 1</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> switchport mode trunk</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> switchport trunk allowed vlan 1,3,5,10,70-71,80-85,98-102,201-209,250,252,301-339</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> bpdu-guard</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> flowcontrol send on</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> flowcontrol receive on</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> exit</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">interface port 11</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> switchport access vlan 98</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> bpdu-guard</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> exit</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">interface port 12</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> switchport mode trunk</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> switchport trunk allowed vlan 1,3,5,10,70-71,80-85,98-102,201-209,250,252</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> exit</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">interface port 15</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> switchport access vlan 202</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> flowcontrol send on</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> flowcontrol receive on</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> exit</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">interface port 16</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> switchport access vlan 201</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> flowcontrol send on</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> flowcontrol receive on</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> exit</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">interface port MGTA</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> shutdown</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> exit</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">interface port MGTB</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> shutdown</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> exit</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">vlan 10</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> name "LAB1"</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">vlan 70</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> name "VLAN 70"</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">vlan 201</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> name "iSCSI-201"</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">vlan 202</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> name "iSCSI-202"</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">vlan 205</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> name "ESXi-vMotion"</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">vlan 206</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> name "ESXi-FT"</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">vlan 250</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> name "Home-NET"</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">vlan 252</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> name "GuestNET"</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">portchannel 13 lacp key 100</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">portchannel 14 lacp key 101</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">spanning-tree mst configuration</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> name "local"</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> exit</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">spanning-tree mode disable</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">no spanning-tree pvst-compatibility</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">spanning-tree stp 1 vlan 1</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">spanning-tree stp 1 vlan 3</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">logging host 1 address 192.168.98.48 DATA</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">interface port 13</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> lacp mode active</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> lacp key 101</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> no lacp suspend-individual</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">interface port 14</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> lacp mode active</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> lacp key 101</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> no lacp suspend-individual</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">interface port 23</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> lacp mode active</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> lacp key 100</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> no lacp suspend-individual</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">interface port 24</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> lacp mode active</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> lacp key 100</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> no lacp suspend-individual</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">interface ip 1</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> ip address 100.64.254.254 255.255.255.0</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> enable</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> exit</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">interface ip 3</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> vlan 3</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> exit</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">interface ip 70</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> ip address 192.168.70.254</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> vlan 70</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> enable</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> exit</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">ip bootp-relay server 1 address 192.168.98.21</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">ip bootp-relay server 2 address 192.168.98.22</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">ip bootp-relay information enable</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">ip bootp-relay enable</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">ip igmp snoop vlan 1</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">ip igmp enable</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">ip igmp snoop enable</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">ip igmp snoop igmpv3 enable</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">ip route 0.0.0.0 0.0.0.0 100.64.254.1</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">ip route 192.168.251.0 255.255.255.0 192.168.250.245</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">ip route 192.168.8.0 255.255.248.0 192.168.250.250</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">router ospf</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> enable</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> area 0 enable</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">interface ip 1</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> ip ospf enable</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">ntp enable</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">ntp primary-server 192.168.98.21 DATA</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">ntp secondary-server 192.168.98.22 DATA</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">!</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"></span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">end</span>Craig Herringhttp://www.blogger.com/profile/05719502208585022033noreply@blogger.com1tag:blogger.com,1999:blog-3594227605547371558.post-5655466404446898162015-06-19T09:34:00.001-04:002015-06-19T09:34:31.947-04:00vSphere (and others) LAB storage<div class="p1">
<span class="s1">Some of you may know I have been building and using a vSphere lab for a number of years now as most VMware professionals. Recently the SAN platform I've been using for a couple years, Nexenta, has removed/disabled VAAI support from their software because of some issues so I decided to try the other popular FreeNAS since it's been rapidly maturing.</span></div>
<div class="p1">
<span class="s1"><br /></span></div>
<div class="p1">
<span class="s1">For the most part my 3 Nexenta SANs have been running fine until a HDD dies at which time the SAN would lock and require some coaxing and perhaps a power cycle to come back alive. With some of the recent changes to the platform, removing VAAI, I decided it was time to give FreeNAS another try.</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">For those of you involved in some way with VMware vSphere you know that VAAI was a very important advancement in storage function and management. It provides primitive functions to allow the storage controller to do the work only sending progress updates to the hosts cutting down on latency and storage fabric utilization. Nexenta used to provide 3 of the commonly used and 1 of the uncommonly used primitives. https://v-reality.info/2011/08/nexentastor-3-1-adds-second-generation-vaai/</span></div>
<div class="p1">
<span class="s1">They have removed VAAI in the recent patches 4.0.3FP2 due to "kernel panic issues". What they failed to realize is this is a SIGNIFICANT change to a storage infrastructure. It's easy to introduce from a traditional non-VAAI design but once a storage architecture is designed for VAAI it's nearly impossible to go back. FreeNAS 9.3 supports 5 primitives, you get a bonus one. http://www.ixsystems.com/whats-new/freenas-93-features-support-for-vmware-vaai/</span></div>
<div class="p1">
<span class="s1">1 particular primitive, ATS, allows us to make LUNs much larger in size since only VMDK operations happen at the file level instead of the entire LUN. This allowed us to make larger LUNS since having more then 10 or 15 VMs in a LUN since the host would not lock an entire LUN for a single file operation causing the rest of the VMs to be impacted. Further FreeNAS also includes Warn&Stun which provides the host with some more intelligence about a thin provisioned VM reducing crashes.</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">FreeNAS has also been making many other improvements to the platform. One major one was the migration from iSCSI target software being moved from user space to kernel space. After some 'seat of the pants tests' compared to earlier releases this seemed to provide a nice 30% improvement in performance.</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<div class="p1">
<span class="s1">Installing 9.3 FreeNAS is as simple as it's always been, a couple presses of <ENTER> and it's installing. One nice feature is you have the ability to install to USB where Nexenta cannot. However make sure you create SWAP on a disk once you have it installed. Being BSD based compared to openSolaris you have a much wider array of hardware choices. Going from Nexenta to FreeNAS you should have no issues. The community forms and docs provide some good direction for hardware and firmware versions. For example using the standard LSI HBAs you know to use the P16 firmware version. The other cool feature is FreeNAS does not limit you to 18TB of RAW storage.</span></div>
<div class="p2">
<span class="s1"></span><br /></div>
<br />
<div class="p1">
<span class="s1">I've now been running FreeNAS as the main LAB storage san for a couple days now and I'm rather impressed with it's performance and stability. Nexenta, I couldn't always say this...</span></div>
Craig Herringhttp://www.blogger.com/profile/05719502208585022033noreply@blogger.com1tag:blogger.com,1999:blog-3594227605547371558.post-71630301884268181332015-05-29T21:33:00.001-04:002015-05-29T21:51:09.620-04:00Backup of vCenter and other vSphere components<!--?xml version="1.0" encoding="UTF-8" standalone="no"?-->
<br />
<div style="font-size: 14px; widows: auto;">
</div>
One of the many questions through the years while deploying a VMware virtual environment has been “How do I backup vCenter?”. The response is the typical next-gen IT answer, “It depends”. Some of these dependencies can relate to how large your environment is to what is your organization’s maintenance process? Other contributing factors can be leveraging other departments within your organization that maintain databases. Through my experience from small to large business, these clients have many different operational procedures and many have dedicated SQL teams to manage this process. All of these can contribute to various scenarios within your own organization.<br />
<br />
To begin the search I considered how VMware would currently address the issue, however did not turn up any real meat in terms of official support or KB articles. Considering they have their own backup product and do not provide much guidance in this area leaves me to believe they recognize the thousand different ways this can be accomplished. Next I searched around the different backup vendor sites and this lead to the same lack of ‘official' information. The information I did find was info from other blogs or lists and as you can guess opinions varied as much as the search results I was typing in Google. Considering there are many ways to accomplish this goal I wanted to find information directly through supportable channels to have a good base for this endeavor.<br />
<br />
<h3>
Plan B…</h3>
<br />
What would be required if my entire virtual environment were trashed and I had to rebuild from scratch The key requirement would be to create a backup that would save the vCenter database but also the ESXi configs and the specific build numbers. If build numbers are not at least noted then firmware compatibility or specific vSphere builds may introduce issues into the environment. It’s easy to stand up a new fresh environment that is fully patched but this can break stuff.<br />
<br />
Let’s consider what specifics we need to account for. The typical components of a typical vSphere environment are vCenter and its database, ESXi hosts, datastore connectivity and network connectivity. If there are other services such as vRealize Operations or vRealize Log Insight these services can be saved and recovered either with a replication technology such as vSphere Replication or with a backup technology, vSphere Data Protection or Veeam. We can also use these tools to protect vCenter however we do not have a guarantee of database consistency.<br />
<br />
Starting with vSphere and the database if VCSA we can refer to the KB articles<br />
<a href="http://kb.vmware.com/kb/2034505">http://kb.vmware.com/kb/2034505</a><br />
For vCenter 6<br />
<a href="http://kb.vmware.com/kb/2091961">http://kb.vmware.com/kb/2091961</a><br />
<br />
This appears to improve the process by adding an online method of saving the database. If you are using a Microsoft SQL server embedded with vCenter your experience may vary using standard backup tools with MSSQL VSS aware plugins. A sure method is to leverage MS SQL Studio to perform SQL backups. This will use the appropriate VSS provider for consistency and then backup the exported DB backup file. Upon recovery this file can be imported into a fresh vCenter deployment for recovery. If the MSSQL server is dedicated the same method can be used however this architecture has shown more reliable while performing backups using the standard backup processes. Below are some references for MSSQL backups.<br />
<br />
MS SQL Database backups<br />
<a href="https://support.microsoft.com/en-us/kb/2019698">https://support.microsoft.com/en-us/kb/2019698</a><br />
Migrate MSSQL Express (unsupported) to SQL Standard (supported)<br />
<a href="http://kb.vmware.com/kb/1028601">http://kb.vmware.com/kb/1028601</a><br />
<br />
Next we need to save the config for the ESXi hosts. Yes, this config can be saved as well. Be sure to save any drivers you may have added outside the standard patches. I’ve noticed over time specific versions of drivers become unavailable so it is important to save these as they may have a dependency with the respective card’s firmware version. This is important due to newer CNAs, 10G, FC adapters and their dependency between firmware to driver versions.<br />
<br />
Backup ESXi host config<br />
<a href="http://kb.vmware.com/kb/2042141">http://kb.vmware.com/kb/2042141</a><br />
<br />
This provides ESXi build references for use in manually creating baselines for recovery for your current ESXi build level.<br />
<a href="http://kb.vmware.com/kb/1014508">http://kb.vmware.com/kb/1014508</a><br />
References for manually creating update manager baselines.<br />
<a href="http://kb.vmware.com/kb/1019545">http://kb.vmware.com/kb/1019545</a><br />
<br />
Another best practice is to keep a current config exported of your vSphere dVswitches. This is the only critical piece in the event of a catastrophic failure that would cause downtime. Sure, you would loose some configs and some historical data but these are not critical to the functionality of the virtual machines running on the hosts. Obviously this is very simplistic and other monitoring, automation, and compliance systems do need to be considered in the grand scheme of the design but this provides a second backup type for this very critical information if all else fails.<br />
<br />
Export dVswitch config<br />
<a href="http://kb.vmware.com/kb/2034602">http://kb.vmware.com/kb/2034602</a><br />
<br />
In the case where a SLA must be maintained for this data and other management systems a dedicated a management cluster becomes the reference and preferred architecture. This would remove the backup circular dependency created when any backup system attempts to quiesce the vCenter database. This also provides a solid architecture where a highly or hyper-converged architecture is implemented. When management systems are integrated with the hardware being managed there are times when manual juggling is required removing some of the automation SDDC provides. Updating, patching, providing maintenance, and unplanned failures often require this juggling effort. For example if vCenter is running on a host that decides it’s time to reject a stick of ram and PSODs while automation tasks are occurring this will impact these tasks while vCenter is non-functional. Here is a link with some great reference designs.<br />
<a href="http://blogs.vmware.com/vsphere/2014/12/creating-vmware-software-defined-datacenter-reference-architecture.html">http://blogs.vmware.com/vsphere/2014/12/creating-vmware-software-defined-datacenter-reference-architecture.html</a><br />
<br />
Bottom line… Since many vendors provide tools to accomplish these tasks of ensuring these management applications are recoverable prudence still is required while merging these technologies together. The community forums of each vendor typically provides real world experience and is a valuable support tool. However always reference release notes and documentation as these provide officially supported architecture, behavior and tips for dependable operation.Craig Herringhttp://www.blogger.com/profile/05719502208585022033noreply@blogger.com0tag:blogger.com,1999:blog-3594227605547371558.post-2387375765867723792011-05-11T23:18:00.000-04:002011-05-13T16:42:17.280-04:00Cloud apps<div><p>Been a little while since my last post. Well... Time to come back after spending some time at a new job.</p>
<p>Some cool things I've come across. For 1 I'm writing this from my phone (the little things in life). Watched a video for Google IO. You should check it out. Also VMware anounced a new cloud platform. This should lend itself for those attempting to create a private cloud beyond simply running virtual servers.</p>
</div>Craig Herringhttp://www.blogger.com/profile/05719502208585022033noreply@blogger.com0tag:blogger.com,1999:blog-3594227605547371558.post-48964536111670591502010-11-08T22:53:00.000-05:002010-11-08T22:53:51.055-05:00Virtual backupsA short note about backing up your VMs.<br />
<br />
One of the next (and sometimes forgotten) issues after you have virtualized your life is now how do you save it? You could keep performing backups the same way you have for years however I would recommend staggering them as if they all start at the same time you stand the risk of creating I/O contention on your SAN. Now you have an alternative method since your virtual servers now are living in essentially files or possibly a LVM style partition, depending on the technology you are using, let's take advantage of this situation.<br />
<br />
Using methods provided by traditional solutions as in <a href="http://www.symantec.com/business/products/family.jsp?familyid=backupexec">Backup Exec</a> with the VMware agent or even looking at newer offerings such as <a href="http://www.veeam.com/">Veeam</a> or <a href="http://www.phdvirtual.com/">PHDVirtual</a> you can achieve successful backups easier then sticking with agent per-server (virtual server in this case) methods. The new style software that specifically supports VMware or Xenserver are agent-less and are gaining features that can either equal or even exceed what physical server backups are capable of. Missing in the physical server world compared to the virtual world is the visibility at a lower level from the volume where the data or files reside you are concerned about. On one side we are dealing with platters inside of a physical disk compared to the virtual side where we can easily see a layer under the operating system's disk. Some of what is built into VMware, and to a lesser extent with other solutions, allows us to intelligently deal with this data.<br />
<br />
Bottom line - if you are having trouble getting good reliable backups in the physical world perhaps virtualization can assist along with other cost cutting reasons.Craig Herringhttp://www.blogger.com/profile/05719502208585022033noreply@blogger.com0tag:blogger.com,1999:blog-3594227605547371558.post-74635707132594160032010-10-26T23:09:00.000-04:002010-10-26T23:11:45.538-04:00I/O (part 2)<span class="Apple-style-span" style="font-family: Arial;">In part 2 of I/O we will consider how to observe some pain points in your overall storage design. These concepts could be applied to any technology once you understand how they can be applied. The concern areas include any connection between the application running in the operating system all the way to the spinning platters inside the disk drive. In this I will speak specifically to iSCSI as this is becoming increasingly common in storage networks.<br class="Apple-interchange-newline" /> <div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><br />
</div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><u><i><b><span class="Apple-style-span" style="color: #ffe599;">Servers</span></b></i></u></div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">Lets begin right at the server where the application or files are presented from. There are some things to tune here but nothing that will make a significant difference. If using a physical server ensure the NIC(s) you are connecting to the storage with are 1Gb server type network cards. Most popular ones these days support some type of TCP offloading and the associated drivers are a better quality in the supported OSs. If this machine is virtual the VM itself will not be performing the iSCSI translation rather VMware will be handling this piece. If you find yourself needing to use an iSCSI initiator from within a VM use a dedicated vmxnet 3 virtual NIC if supported. One of the methods to check if I/O is the issue, check PerfMon or iostat (with respect to OS) and look for queue depth, length, or hold time. This measurement can indicate if the OS is holding SCSI requests waiting to be processed. One potential solution depending on the root cause is to enable MPIO as this can assist with performance issues and also provide iSCSI redundancy.</div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><br />
</div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><b><i><u><span class="Apple-style-span" style="color: #ffe599;">Virtual Host</span></u></i></b></div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">The next link in the chain is normally VMware, Xenserver or some other virtualization technology. In a physical environment this can obviously be skipped. In a virtual host environment some of the same rules apply however keeping in mind you now have many servers using the same iSCSI connections. In a local storage environment you had a direct path between the controller and the disk drive using a 68pin or SAS cable and was typically capable of more then 1Gb/sec. Now you have many servers using perhaps a single 1Gb connection to it's respective disk as well as the latencies introduced with the other components. Evaluating the performance here can be done in a similar approach by checking for disk latency and queue. Make sure latency is less then 50ms and queue is less then 50. If using an application, like a SQL database, some vendors have much stricter limit of between 2ms and 10ms for latency. Using such technologies as MPIO, better network cards, updated drivers, fully patched hosts can assist to provide the desired performance. Also providing dedicated iSCSI interfaces should be one of the first things considered in a properly designed host.</div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><br />
</div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><b><i><u><span class="Apple-style-span" style="color: #ffe599;">Infrastructure</span></u></i></b></div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">Moving to the switch infrastructure can also play a significant role in the overall performance and is often overlooked. The basic rule is to use a good quality switch with plenty of port buffering. This will ensure the packets flow through without becoming blocked due to the buffers filling. This could be seen from the VM and the host showing high levels of latency however the SAN showing low overall utilization and no signs of stress. The switch itself may not show a high CPU level or any other stress as it may not have a lot of traffic on all ports or the configuration may not have CPU intensive tasks. Also to ensure the switch will not be asked to perform some of these other functions or pass non-iSCSI traffic it is recommended to use dedicated switches. In some designs or budgets this may not be possible so ensure the switch you are using is a good quality switch. Some examples include the <a href="http://h10144.www1.hp.com/products/switches/HP_ProCurve_2910al_Switch_Series/overview.htm">HP 2910al</a> or the <a href="http://www.cisco.com/en/US/products/hw/switches/ps5023/index.html">Cisco 3750</a>. Obviously there are many full Gb switches on the market even in the sub $200 range and may be fine for lab/test situations I would caution using them in a production network as these may not have enough buffering to maintain a non-blocking state.</div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><br />
</div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><b><i><u><span class="Apple-style-span" style="color: #ffe599;">Storage</span></u></i></b></div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">Considering storage, this is one area that is not as clear. Due to the amount and diversity of technology these vendors use one must understand the architecture and hardware used. Typically most vendors will have some method to measure CPU, memory utilization (often local cache), disk queue depth and latency. Virtualized systems will always perform better (as most systems) when RAID 10 or RAID 50 sets are chosen over RAID 5. Using SAS, SCSI or FC 10Krpm or 15Krpm disks obviously will always perform better then the SATA, SAS 7Krpm disks. Another philosophy concerning the number of spindles or amount of disks used can also prove to be beneficial however as SAN vendors use different technology this may or may not help as much as it used to. One consideration to support this is if the disk controller can handle many disks in a large RAID set. Recently Intel and others have <a href="http://www.tomshardware.com/reviews/ich10r-sb750-780a,2374.html">shown</a> processors are becoming so fast software based RAID can outperform hardware based RAID sets. Also as you are designing your disk system do not add parity disks (or equivalent of a disk) in your write I/O calculations as this stripe when written will actually increase write time. Read times will lessen however also keep in mind especially in virtualized environments the platters are housing blocks of simply more blocks of data. Each time the virtual OS writes a file it changes a block (VMware example) in the .vmdk file, then changes a block on the VMFS partition, which in turn changes a block on whatever filesystem the SAN uses to store data. In the world of virtualization this can be virtualized, not sitting directly on platters, also. ;-)</div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><br />
</div><div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">Enjoy!</div></span>Craig Herringhttp://www.blogger.com/profile/05719502208585022033noreply@blogger.com0tag:blogger.com,1999:blog-3594227605547371558.post-81820192875564769722010-10-14T23:23:00.000-04:002010-10-14T23:23:58.624-04:00Personal VirtualizationHere are some tips to make virtual workstation technologies perform better. Some of these are specific to VMware but could be applied to other virtualization platforms like Virtualbox.<br />
<br />
For a new VM that you are creating select:<br />
<br />
> Store Virtual disk as a single file.<br />
<br />
If you have an existing VM make sure all of the snapshots are deleted (if you have taken any) and do this:<br />
<br />
> vmware-vdiskmanager -r sourceDisk.vmdk -t 2 destinationDisk.vmdk<br />
<br />
In this case the source disk will be the large VMDK file. After you convert you will need to edit the vmx (text based) file to reference the new vmdk file unless you used the same file name. Obviously you'd have to convert the disk to a new directory in this case or change the name. Once it's converted you will actually see 2 new files, one is the very small text file that defines the raw virtual disk file and the other is the raw virtual disk file itself. DO NOT LOOSE THE TEXT FILE! It is essentially impossible to remake as there is a special code in there that references the large raw file.<br />
<br />
If you run the 'vmware-vdiskmanager' itself you can see all the options you can do.<br />
<br />
Another tip is use multiple partitions to reduce the level of fragmentation. If you are using Linux format the partition with XFS or ext4. I normally give each partition 3-5 VMs and have partitions of 25-50GB.<br />
<br />
Another tip is if you can use RAID 0 or RAID 1 of very fast hard drives. I am using 2 WD Raptor 150GB drives at home. I can run 4 VMs at once running a RAID 0 with 4 GB of physical ram. The key here is not necessairly MB/sec but I/O persec. This is where the 10KRpm drives rival any other SATA drive on the market by far. These disks are 50% faster. However if you use RAID 1 you will not loose too much if you use a quality drive like the WD RE3 1TB drive. This is one of the faster ones on the market. Do not worry about hardware vs. software RAID as the current processors have enough performance to lessen the need for hardware RAID (unless you have the money to burn).<br />
<br />
I've also done a little research on whether or not to use Enterprise of 'RAID' type drives. There can be a sight advantage beyond the (in some cases) longer warranty and build quality. RAID supportable drives are designed to intentionally fail and even can send commands back to the RAID controller (software or hardware) telling the state of the drive. A standard disk will attempt retries for a number of minutes (typically 2) before it will announce a failure ultimately confusing the RAID software as it may have already declared the disk FAILED even if the disk recovered. Considering RAID type SATA drives will declare themselves failed in a short period of time (7-10 seconds) if it cannot recover and send the message to the RAID software. This behavior is specifically evident in the Western Digital line but are similar with other manufacturers and may not be a critical reason to choose these disks for home/test.<br />
<br />
Enjoy!Craig Herringhttp://www.blogger.com/profile/05719502208585022033noreply@blogger.com0tag:blogger.com,1999:blog-3594227605547371558.post-17284364925216809972010-10-14T22:08:00.000-04:002010-10-14T22:08:04.684-04:00Resource ManagementSaw something very interesting today... In setting up a little demo environment with some colleagues we only had a server with some very limited resources. In particular 8GB of ram and we needed to check out the latest version of VMware View. Once everything was finally booted up I found the virtual machines' balloon driver taking effect, memory sharing, memory compression, and memory swap on every VM. Things were a bit slow but perhaps we'll chalk this up to a test of ESXi 4.1 resource management and even better - <i>nothing crashed</i> :-)!Craig Herringhttp://www.blogger.com/profile/05719502208585022033noreply@blogger.com0tag:blogger.com,1999:blog-3594227605547371558.post-66243549896683990282010-10-05T22:29:00.001-04:002010-10-06T07:38:32.347-04:00I/OWelcome!<br />
<br />
For a first post I figure I'd talk about one of the main issues I've found while virtualizing machines within any of the technologies from VMware ESX , Citrix Xenserver, ... I/O capabilities of the storage where these virtual files or partitions reside whether connected by IP networks (NFS, iSCSI), Fiber channel, and local storage. Storage medium usually consists of either SATA, SAS, SCSI, FC, and SSD. I'm going to make an attempt to speak about these different technologies.<br />
<br />
First let me dis-arm the idea where FC is faster then iSCSI and NFS is the worst. This all depends on how it's implemented. When iSCSI is configured to use 10Gb/s networks it can easily surpass 4Gb or even 8Gb FC just as NFS can easily be as fast as the other technologies. The real difference becomes whether or not multi-path is enabled and if the storage is capable of these performance levels in the first place. Multi-path brings a couple benefits; the first being the aggregate bandwidth of all the connections added together. If mating the 2 appropriate technologies each I/O request made can be channeled through a separate path. The other benefit can be realized from not having a single point of failure. Typically if the technology supports this configuration it will have the ability of failing over to another path, or re-issuing I/O requests if the request never comes back with an acknowledgement. If configured properly your virtual machines will not crash but simply hang for a short period of time then regain activity when the requests have timed out. I will speak to this in greater detail in a later post of how to configure some of these technologies.<br />
<br />
Another issue relating to storage is how it itself is configured. I've noticed a huge difference in whether SAS or SATA or if it is configured as a RAID5 or a RAID 10 or 50. I'll let Wikipedia ( <a href="http://en.wikipedia.org/wiki/RAID">http://en.wikipedia.org/wiki/RAID</a> ) define RAID for me :-) however understanding the differences and trade-offs of each type can lead you to disaster or complete success even within the same disk technology. What I mean by this - it is possible to see a RAID 50 or RAID 10 SATA storage array achieve closely the same performance level as a RAID 5 SAS array. Let's say for example a SATA disk is capable of 100 iops (input output operations per second) and a SAS disk is capable of 175 iops. Keep in mind there are other contributing factors but these are averages.<br />
<br />
If we take a 10x SATA 1TB disks RAID 10 and show for every spindle capacity we actually have 200 iops<br />
RAID 10 = double the spindles for 5TB capacity = 2 x iops = 1000iops for 5TB or storage<br />
<br />
Next we take 9x SAS 600GB disks RAID 5 and show for every spindle capacity we actually have 155iops<br />
RAID 5 = 1.125 spindles for 4.8TB capacity = .889 x iops = 1400iops for 4.8TB of storage<br />
<br />
In this example we can see SAS still leads by 140% however the cost difference could be an interesting story. From this example we could also see if we configure 6 SATA disks in a RAID 5 for 5TB of capacity our performance is substantially less.<br />
RAID 5 = 1.2 spindles for 5TB capacity = .833 x iops = 500iops for same amount of storage.<br />
<br />
This is substantially less then even our RAID 10 configuration. There are some other considerations due to relative read and write performance of each technology. Write performance comparing a RAID 5 and 10 could alone even out the numbers in the above equations between SAS and SATA. In a RAID 5 all bits must be written to all disks, the parity bit calculated and written. This must happen for each I/O request and can only happen in succession. When the process is applied to multi array type RAID levels as in RAID 10 or 50 these operations can happen simultaneously with most current generation controllers. This can also improve the performance of the overall system. Considering this with the above equations we could potentially realize an additional 25%-75% penalty depending on the amount of writes.<br />
<br />
Considering a virtualized environment where we are not simply dealing with documents and SQL databases, we are dealing with virtual disks and every read and write occurring corresponds with virtual disk blocks the virtual operating system is changing. The point here is these disk blocks can be larger then the standard I/O chunks we have been used to traditionally dealing with.<br />
<br />
Bottom line - when planning a storage system for virtualization consider the number of virtual machines and each machine's I/O requirement in the physical world or the performance level desired and add about 10%-20% for the extra virtualized layer in between.Craig Herringhttp://www.blogger.com/profile/05719502208585022033noreply@blogger.com0