不为所动 發表於 2015-7-2 15:58:17

在Linux系统中将SSD当块设备缓存的方法

<p><strong>原理</strong></p>
<p>&nbsp;写操作先缓存到ssd硬盘上，然后通过一定策略写到普通硬盘上；</p>
<p>&nbsp;读操作热点数据可以缓存到ssd硬盘上，提高读取数据的速度。</p>
<p>&nbsp;</p>
<p>软件 ssd 缓存开源软件主要有bcache ，flashcache</p>
<p>&nbsp;</p>
<p>bcache 需要编译最新的内核，要求比较高，配置比较复杂，目前主要用于测试环境；</p>
<p>flashcache 是facebook内部使用的ssd cache软件，开源出来供大家使用，目前有人编译出rhel6的rpm包，在centos6上可以直接安装使用。</p>
<p>&nbsp;</p>
<p>本次初步测试了flashcache</p>
<p>&nbsp;</p>
<p>测试环境</p>
<p>sdc为 ssd 硬盘&nbsp;&nbsp;&nbsp;&nbsp; 160G*4 raid 10&nbsp;&nbsp;&nbsp;&nbsp; 用fio测试 4k 写iops为 5433</p>
<p>sdb为 普通sas硬盘 146G*2 raid&nbsp; 1&nbsp;&nbsp;&nbsp; 用fio测试 4k 写iops为 447</p>
<p>&nbsp;</p>
<p>初步简单测试结果：</p>
<p>&nbsp;&nbsp;&nbsp; 4k 随机写可以达到 5014</p>
<p>&nbsp;&nbsp;&nbsp; 4k随机读可以达到45874</p>
<p>&nbsp;</p>
<p><strong>flashcache 安装配置方法</strong></p>
<p><strong>&nbsp;</strong></p>
<p>安装</p>
<p><br><div class="msgheader"><div class="right"><span style="CURSOR: pointer" class="copybut"><u>复制代码</u></span></div>代码如下:</div><div class="msgborder" id="phpcode162">rpm --import http://elrepo.org/RPM-GPG-KEY-elrepo.org&lt;/p&gt;
&lt;p&gt;rpm -Uvh http://elrepo.org/elrepo-release-6-5.el6.elrepo.noarch.rpm&lt;/p&gt;
&lt;p&gt;yum install flashcache-utils kmod-flashcache</div></p>
<p>配置:</p>
<p>用sdc 做sdb的缓存</p>
<p>&nbsp;<br><div class="msgheader"><div class="right"><span style="CURSOR: pointer" class="copybut"><u>复制代码</u></span></div>代码如下:</div><div class="msgborder" id="phpcode163">&lt;/p&gt;
&lt;p&gt;flashcache_create -p backcachedev /dev/sdc /dev/sdb    &lt;/p&gt;
&lt;p&gt;cachedev cachedev, ssd_devname /dev/sdc, disk_devname /dev/sdb cache mode WRITE_BACK&lt;/p&gt;
&lt;p&gt;block_size 8, md_block_size 8, cache_size 0&lt;/p&gt;
&lt;p&gt;Flashcache metadata will use 1192MB of your 24016MB main memory</div></p>
<p>可以用3种方式：</p>
<p><br><div class="msgheader"><div class="right"><span style="CURSOR: pointer" class="copybut"><u>复制代码</u></span></div>代码如下:</div><div class="msgborder" id="phpcode164">&lt;/p&gt;
&lt;p&gt;Writethrough - safest, all writes are cached to ssd but also written to disk&lt;/p&gt;
&lt;p&gt;immediately.If your ssd has slower write performance than your disk (likely&lt;/p&gt;
&lt;p&gt;for early generation SSDs purchased in 2008-2010), this may limit your system&lt;/p&gt;
&lt;p&gt;write performance.All disk reads are cached (tunable). </div></p>
<p>数据同时写到ssd和普通硬盘</p>
<p>&nbsp;<br><div class="msgheader"><div class="right"><span style="CURSOR: pointer" class="copybut"><u>复制代码</u></span></div>代码如下:</div><div class="msgborder" id="phpcode165">&lt;/p&gt;
&lt;p&gt;Writearound - again, very safe, writes are not written to ssd but directly to&lt;/p&gt;
&lt;p&gt;disk.Disk blocks will only be cached after they are read.All disk reads&lt;/p&gt;
&lt;p&gt;are cached (tunable).</div></p>
<p>数据同时绕过ssd，直接写到普通硬盘</p>
<p><br><div class="msgheader"><div class="right"><span style="CURSOR: pointer" class="copybut"><u>复制代码</u></span></div>代码如下:</div><div class="msgborder" id="phpcode166">Writeback - fastest but less safe.Writes only go to the ssd initially, and&lt;/p&gt;
&lt;p&gt;based on various policies are written to disk later.All disk reads are&lt;/p&gt;
&lt;p&gt;cached (tunable). </div></p>
<p>数据先写到ssd，随后写到普通硬盘</p>
<p>查看</p>
<p>&nbsp;flashstat</p>
<p>======================================================================================================</p>
<p>Flashstat: a tool for flashcache status per second</p>
<p>Author&nbsp;&nbsp; : NinGoo(seaman.ning@gmail.com)</p>
<p>Version&nbsp; : 0.3</p>
<p>======================================================================================================</p>
<p>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; SSD Device:&nbsp;&nbsp; /dev/sdc&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Disk Device:&nbsp;&nbsp; /dev/sdb&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Cache Mode: WRITE_BACK</p>
<p>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Capacity:&nbsp;&nbsp;&nbsp; 303998M&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Block Size:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 4K&nbsp;&nbsp;&nbsp;&nbsp; Meta Block Size:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 4096b</p>
<p>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Total Blocks:&nbsp;&nbsp; 77823488&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Cached Blocks:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 9&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Cached Percent:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 0</p>
<p>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Set Numbers:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 512&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Dirty Blocks:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Dirty Percent:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 0</p>
<p>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; cache_all:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; reclaim_policy:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; FIFO&nbsp;&nbsp;&nbsp; dirty_thresh_pct:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 20</p>
<p>&nbsp;&nbsp; max_clean_ios_set:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2 max_clean_ios_total:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 4&nbsp;&nbsp;&nbsp;&nbsp; skip_seq_thresh:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 0K</p>
<p>======================================================================================================</p>
<p>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; time&nbsp; read/s write/s diskr/s diskw/s&nbsp; ssdr/s&nbsp; ssdw/s uread/s uwrit/s metaw/s clean/s&nbsp; repl/s wrepl/s&nbsp;&nbsp; hit%&nbsp; whit% dwhit%</p>
<p>03-13 14:51:20&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 0&nbsp;&nbsp; 0|10&nbsp;&nbsp;&nbsp; 0|0&nbsp;&nbsp;&nbsp; 0|0&nbsp;</p>
<p><br><div class="msgheader"><div class="right"><span style="CURSOR: pointer" class="copybut"><u>复制代码</u></span></div>代码如下:</div><div class="msgborder" id="phpcode167">fdisk -l /dev/mapper/cachedev</div>&nbsp; 可以看到一个设备</p>
<p>&nbsp;</p>
<p>Disk /dev/mapper/cachedev: 146.8 GB, 146778685440 bytes</p>
<p>255 heads, 63 sectors/track, 17844 cylinders</p>
<p>Units = cylinders of 16065 * 512 = 8225280 bytes</p>
<p>Sector size (logical/physical): 512 bytes / 512 bytes</p>
<p>I/O size (minimum/optimal): 512 bytes / 512 bytes</p>
<p>Disk identifier: 0x0002da0a</p>
<p>&nbsp;</p>
<p>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Device Boot&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Start&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; End&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Blocks&nbsp;&nbsp; Id&nbsp; System</p>
<p>&nbsp;</p>
<p>使用：</p>
<p>&nbsp;</p>
<p>可以当一个块设备来使用，划分vg lv</p>
<p>&nbsp;<br><div class="msgheader"><div class="right"><span style="CURSOR: pointer" class="copybut"><u>复制代码</u></span></div>代码如下:</div><div class="msgborder" id="phpcode168">&lt;/p&gt;
&lt;p&gt; pvcreate /dev/mapper/cachedev&lt;/p&gt;
&lt;p&gt;Physical volume "/dev/mapper/cachedev" successfully created&lt;/p&gt;
&lt;p&gt; vgcreate ssdcachetest /dev/mapper/cachedev&lt;/p&gt;
&lt;p&gt;Volume group "ssdcachetest" successfully created&lt;/p&gt;
&lt;p&gt; lvcreate -L 100G -n ssdcache_test_lv1 ssdcachetest&lt;/p&gt;
&lt;p&gt;Logical volume "ssdcache_test_lv1" created</div></p>
<p>删除</p>
<p><br><div class="msgheader"><div class="right"><span style="CURSOR: pointer" class="copybut"><u>复制代码</u></span></div>代码如下:</div><div class="msgborder" id="phpcode169">dmsetupremove cachedev&lt;/p&gt;
&lt;p&gt;flashcache_destroy /dev/sdc</div><br /></p>
頁: [1]
查看完整版本: 在Linux系统中将SSD当块设备缓存的方法