MemcacheQ安装及使用方法
<p><strong>一. 安装</strong><br>
MemcacheQ 是一个简单的分布式队列服务，它的运行依赖于BerkeleyDB 和 libevent，所以需要先安装BerkeleyDB和libevent.<br>
Berkeley DB 4.7 or later<br>
Download from <http://www.oracle.com/database/berkeley-db/db/index.html><br>
How to install BerkeleyDB:</p>
<div class="jb51code">
<div>
<div class="syntaxhighlighterbash" id="highlighter_851503">
<div class="toolbar">
<span>?</span>
</div>
<table border="0" cellpadding="0" cellspacing="0"><tbody><tr>
<td class="gutter">
<div class="line number1 index0 alt2">
1</div>
<div class="line number2 index1 alt1">
2</div>
<div class="line number3 index2 alt2">
3</div>
<div class="line number4 index3 alt1">
4</div>
<div class="line number5 index4 alt2">
5</div>
<div class="line number6 index5 alt1">
6</div>
</td>
<td class="code">
<div class="container">
<div class="line number1 index0 alt2">
<code class="bash plain">$</code><code class="bash functions">tar</code> <code class="bash plain">-xvzf db-5.3.21.</code><code class="bash functions">tar</code><code class="bash plain">.gz</code>
</div>
<div class="line number2 index1 alt1">
<code class="bash plain">$</code><code class="bash functions">cd</code> <code class="bash plain">db-5.3.21/</code>
</div>
<div class="line number3 index2 alt2">
<code class="bash plain">$</code><code class="bash functions">cd</code> <code class="bash plain">build_unix/</code>
</div>
<div class="line number4 index3 alt1">
<code class="bash plain">$..</code><code class="bash plain">/dist/configure</code>
</div>
<div class="line number5 index4 alt2">
<code class="bash plain">$</code><code class="bash functions">make</code>
</div>
<div class="line number6 index5 alt1">
<code class="bash plain">$</code><code class="bash functions">make</code> <code class="bash functions">install</code>
</div>
</div>
</td>
</tr></tbody></table>
</div>
</div>
<div class="codetool" id="codetool">
<div class="code_n">
<textarea></textarea>
</div>
</div>
</div>
<p>
安装BerkeleyDB时，可以手动指定安装路径:</p>
<p>
../dist/configure --prefix=/usr/local/berkeleydb</p>
<p>
不指定的话，默认安装在:/usr/local/BerkeleyDB.5.3</p>
<p>
libevent 1.4.x or later</p>
<p>
先检查libevent 是否已经安装:</p>
<p>
#rpm -qa|grep libevent<br>
libevent-devel-2.0.10-2.fc15.x86_64<br>
libevent-2.0.10-2.fc15.x86_64<br>
libevent-2.0.10-2.fc15.i686</p>
<p>
或者：</p>
<p>
<code>ls -al /usr/lib |grep libevent</code></p>
<p>
如果还没有安装：</p>
<p>
Download from <http://monkey.org/~provos/libevent/><br>
How to install libevent:</p>
<div class="jb51code">
<div>
<div class="syntaxhighlighterbash" id="highlighter_436482">
<div class="toolbar">
<span>?</span>
</div>
<table border="0" cellpadding="0" cellspacing="0"><tbody><tr>
<td class="gutter">
<div class="line number1 index0 alt2">
1</div>
<div class="line number2 index1 alt1">
2</div>
<div class="line number3 index2 alt2">
3</div>
<div class="line number4 index3 alt1">
4</div>
<div class="line number5 index4 alt2">
5</div>
</td>
<td class="code">
<div class="container">
<div class="line number1 index0 alt2">
<code class="bash plain">$</code><code class="bash functions">tar</code> <code class="bash plain">-xvzf libevent-1.4.x-stable.</code><code class="bash functions">tar</code><code class="bash plain">.gz</code>
</div>
<div class="line number2 index1 alt1">
<code class="bash plain">$</code><code class="bash functions">cd</code> <code class="bash plain">libevent-1.4.x-stable</code>
</div>
<div class="line number3 index2 alt2">
<code class="bash plain">$.</code><code class="bash plain">/configure</code>
</div>
<div class="line number4 index3 alt1">
<code class="bash plain">$</code><code class="bash functions">make</code>
</div>
<div class="line number5 index4 alt2">
<code class="bash plain">$</code><code class="bash functions">make</code> <code class="bash functions">install</code>
</div>
</div>
</td>
</tr></tbody></table>
</div>
</div>
<div class="codetool" id="codetool">
<div class="code_n">
<textarea></textarea>
</div>
</div>
</div>
<p>
安装libevent时，可以手动指定安装路径:</p>
<p>
./configure --prefix=/usr/local/libevent</p>
<p>
不指定的话，默认安装在:/usr/lib64(64位系统)或者/usr/lib(32位系统)</p>
<p>
<strong>memcacheQ</strong></p>
<p>
下载软件包：http://code.google.com/p/memcacheq/downloads/list<br>
解压缩，cd进目录</p>
<div class="jb51code">
<div>
<div class="syntaxhighlighterbash" id="highlighter_615480">
<div class="toolbar">
<span>?</span>
</div>
<table border="0" cellpadding="0" cellspacing="0"><tbody><tr>
<td class="gutter">
<div class="line number1 index0 alt2">
1</div>
<div class="line number2 index1 alt1">
2</div>
<div class="line number3 index2 alt2">
3</div>
</td>
<td class="code">
<div class="container">
<div class="line number1 index0 alt2">
<code class="bash plain">.</code><code class="bash plain">/configure</code> <code class="bash plain">--</code><code class="bash functions">enable</code><code class="bash plain">-threads</code>
</div>
<div class="line number2 index1 alt1">
<code class="bash functions">make</code>
</div>
<div class="line number3 index2 alt2">
<code class="bash functions">make</code> <code class="bash functions">install</code>
</div>
</div>
</td>
</tr></tbody></table>
</div>
</div>
<div class="codetool" id="codetool">
<div class="code_n">
<textarea></textarea>
</div>
</div>
</div>
<p>
configure 时，如果libevent 不是安装在默认目录，需--with--libevent=/usr/local/libevent指定libevent的安装目录<br>
若没有将</p>
<p>
/usr/local/lib<br>
/usr/local/BerkeleyDB.5.3/lib</p>
<p>
添加进/etc/ld.so.conf 并运行 /sbin/ldconfig 则需--with-bdb=/usr/local/BerkeleyDB.5.3 指定berkeleyDb库的路径</p>
<p>
<strong>二.使用</strong></p>
<p>
启动memcacheQ<br>
使用memcacheq -h 的命令来查看命令行选项<br>
启动memcacheq：memcacheq -d -u nobody -r -H /tmp/memcacheq -N -R -v -L 1024 -B 1024 > /tmp/mq_error.log 2>&1<br>
启动时需-u 参数，指定运行memcacheQ的用户，且指定的用户必须有数据文件的读写权限,如这里的/tmp/memcacheQ目录,否则不能启动</p>
<p>
命令行使用memcacheQ<br>
telnet 127.0.0.1 22202<br>
Trying 127.0.0.1…<br>
Connected to 127.0.0.1.<br>
Escape character is ‘^]'.</p>
<p>
只有两个命令可以在命令行下使用memcacheQ<br>
写对列：<br>
set <queue name> <flags> 0 <message_len>\r\n<br>
<put your message body here>\r\n<br>
STORED\r\n<br>
取出队列：<br>
get <queue name>\r\n<br>
VALUE <queue name> <flags> <message_len>\r\n<br>
<your message body will come here>\r\n<br>
END\r\n<br>
与memcache协议基本一致，只是把key name换成queue name，而且在set的命令中，忽略了expire_time的参数。mq的数据存储是存在berkeleyDB中，做了持久化存储，没有内存的过期时间。<br>
示例：<br>
telnet 127.0.0.1 22202<br>
Trying 127.0.0.1…<br>
Connected to 127.0.0.1.<br>
Escape character is ‘^]'.<br>
set q4 0 0 5<br>
hello<br>
STORED<br>
set q4 0 0 5<br>
world<br>
STORED<br>
stats queue<br>
STAT q4 2/0<br>
END<br>
get q4<br>
VALUE q4 0 5<br>
hello<br>
END<br>
stats queue<br>
STAT q4 2/1<br>
END</p>
<p>
<strong>三.安装使用过程中可能出现的错误</strong></p>
<p>
1.编译出现错误：checking for library containing db_create... no<br>
configure: error: cannot find libdb.so in /usr/local/BerkeleyDB.5.3/lib<br>
需要修改 configure 中的BerkeleyDB中的预编译参数vim configure找到bdbdir="/usr/local/berkeleydb"改为<br>
bdbdir="/usr/local/BerkeleyDB.5.3"再次编译</p>
<p>
2.configure: error: cannot find libdb.so in /usr/local/BerkeleyDB.5.3/lib<br>
出现此错误的原因在于没有安装BerkyleyDb,安装即可</p>
<p>
3./usr/local/memcacheq/bin/memcachq -h<br>
运行报：<br>
memcacheq: error while loading shared libraries: libdb-5.3.so: cannot open shared object file: No such file or directory<br>
解决方法：ln -s /usr/local/BerkeleyDB.5.3/lib/libdb-5.3.so /usr/lib/libdb-5.3.so<br>
注：在64位操作系统中，需执行<br>
ln -s /usr/local/BerkeleyDB.5.3/lib/libdb-5.3.so /usr/lib64/libdb-5.3.so</p>
<p>
<strong>使用memcacheq做异步队列，做个简单的生产者-消费者模型。生产者将数据写入mq中，消费者异步去队列中去取数据，进而进一步的消费处理数据。</strong></p>
<div class="jb51code">
<div>
<div class="syntaxhighlighterpy" id="highlighter_886183">
<div class="toolbar">
<span>?</span>
</div>
<table border="0" cellpadding="0" cellspacing="0"><tbody><tr>
<td class="gutter">
<div class="line number1 index0 alt2">
1</div>
<div class="line number2 index1 alt1">
2</div>
<div class="line number3 index2 alt2">
3</div>
<div class="line number4 index3 alt1">
4</div>
<div class="line number5 index4 alt2">
5</div>
<div class="line number6 index5 alt1">
6</div>
<div class="line number7 index6 alt2">
7</div>
<div class="line number8 index7 alt1">
8</div>
<div class="line number9 index8 alt2">
9</div>
<div class="line number10 index9 alt1">
10</div>
<div class="line number11 index10 alt2">
11</div>
<div class="line number12 index11 alt1">
12</div>
<div class="line number13 index12 alt2">
13</div>
<div class="line number14 index13 alt1">
14</div>
<div class="line number15 index14 alt2">
15</div>
<div class="line number16 index15 alt1">
16</div>
<div class="line number17 index16 alt2">
17</div>
<div class="line number18 index17 alt1">
18</div>
<div class="line number19 index18 alt2">
19</div>
<div class="line number20 index19 alt1">
20</div>
<div class="line number21 index20 alt2">
21</div>
<div class="line number22 index21 alt1">
22</div>
<div class="line number23 index22 alt2">
23</div>
<div class="line number24 index23 alt1">
24</div>
<div class="line number25 index24 alt2">
25</div>
<div class="line number26 index25 alt1">
26</div>
<div class="line number27 index26 alt2">
27</div>
<div class="line number28 index27 alt1">
28</div>
<div class="line number29 index28 alt2">
29</div>
<div class="line number30 index29 alt1">
30</div>
<div class="line number31 index30 alt2">
31</div>
<div class="line number32 index31 alt1">
32</div>
<div class="line number33 index32 alt2">
33</div>
<div class="line number34 index33 alt1">
34</div>
<div class="line number35 index34 alt2">
35</div>
<div class="line number36 index35 alt1">
36</div>
<div class="line number37 index36 alt2">
37</div>
<div class="line number38 index37 alt1">
38</div>
<div class="line number39 index38 alt2">
39</div>
<div class="line number40 index39 alt1">
40</div>
</td>
<td class="code">
<div class="container">
<div class="line number1 index0 alt2">
<code class="py comments">#!/usr/bin/env python</code>
</div>
<div class="line number2 index1 alt1">
<code class="py comments">#-*- coding:utf8 -*-</code>
</div>
<div class="line number3 index2 alt2">
<code class="py spaces"> </code>
</div>
<div class="line number4 index3 alt1">
<code class="py keyword">import</code> <code class="py plain">sys</code>
</div>
<div class="line number5 index4 alt2">
<code class="py keyword">import</code> <code class="py plain">time</code>
</div>
<div class="line number6 index5 alt1">
<code class="py keyword">import</code> <code class="py plain">random</code>
</div>
<div class="line number7 index6 alt2">
<code class="py spaces"> </code>
</div>
<div class="line number8 index7 alt1">
<code class="py keyword">import</code> <code class="py plain">memcache</code>
</div>
<div class="line number9 index8 alt2">
<code class="py spaces"> </code>
</div>
<div class="line number10 index9 alt1">
<code class="py plain">mc </code><code class="py keyword">=</code> <code class="py plain">memcache.Client([</code><code class="py string">"%s:%s"</code><code class="py keyword">%</code><code class="py plain">(</code><code class="py string">"127.0.0.1"</code><code class="py plain">, </code><code class="py string">"22202"</code><code class="py plain">)])</code>
</div>
<div class="line number11 index10 alt2">
<code class="py plain">queue_name </code><code class="py keyword">=</code> <code class="py string">"q1"</code>
</div>
<div class="line number12 index11 alt1">
<code class="py keyword">def</code> <code class="py plain">putter():</code>
</div>
<div class="line number13 index12 alt2">
<code class="py spaces"> </code><code class="py plain">count </code><code class="py keyword">=</code> <code class="py value">0</code>
</div>
<div class="line number14 index13 alt1">
<code class="py spaces"> </code><code class="py keyword">while</code> <code class="py color1">True</code><code class="py plain">:</code>
</div>
<div class="line number15 index14 alt2">
<code class="py spaces"> </code><code class="py plain">data </code><code class="py keyword">=</code> <code class="py string">"hello%d"</code><code class="py keyword">%</code><code class="py plain">(count)</code>
</div>
<div class="line number16 index15 alt1">
<code class="py spaces"> </code><code class="py plain">mc.</code><code class="py functions">set</code><code class="py plain">(queue_name, data)</code>
</div>
<div class="line number17 index16 alt2">
<code class="py spaces"> </code><code class="py keyword">print</code> <code class="py string">"put "</code><code class="py plain">, data</code>
</div>
<div class="line number18 index17 alt1">
<code class="py spaces"> </code><code class="py plain">count </code><code class="py keyword">+</code><code class="py keyword">=</code> <code class="py value">1</code>
</div>
<div class="line number19 index18 alt2">
<code class="py spaces"> </code><code class="py plain">time.sleep(random.randint(</code><code class="py value">1</code><code class="py plain">, </code><code class="py value">10</code><code class="py plain">))</code>
</div>
<div class="line number20 index19 alt1">
<code class="py spaces"> </code>
</div>
<div class="line number21 index20 alt2">
<code class="py keyword">def</code> <code class="py plain">process_data(data):</code>
</div>
<div class="line number22 index21 alt1">
<code class="py spaces"> </code><code class="py functions">print</code> <code class="py string">"processing data :"</code><code class="py plain">, data</code>
</div>
<div class="line number23 index22 alt2">
<code class="py spaces"> </code>
</div>
<div class="line number24 index23 alt1">
<code class="py keyword">def</code> <code class="py plain">getter():</code>
</div>
<div class="line number25 index24 alt2">
<code class="py spaces"> </code><code class="py keyword">while</code> <code class="py color1">True</code><code class="py plain">:</code>
</div>
<div class="line number26 index25 alt1">
<code class="py spaces"> </code><code class="py plain">data </code><code class="py keyword">=</code> <code class="py plain">mc.get(queue_name)</code>
</div>
<div class="line number27 index26 alt2">
<code class="py spaces"> </code><code class="py keyword">if</code> <code class="py plain">data:</code>
</div>
<div class="line number28 index27 alt1">
<code class="py spaces"> </code><code class="py plain">process_data(data)</code>
</div>
<div class="line number29 index28 alt2">
<code class="py spaces"> </code><code class="py keyword">else</code><code class="py plain">:</code>
</div>
<div class="line number30 index29 alt1">
<code class="py spaces"> </code><code class="py keyword">print</code> <code class="py string">"no message, sleep for a while ..."</code>
</div>
<div class="line number31 index30 alt2">
<code class="py spaces"> </code><code class="py plain">time.sleep(</code><code class="py value">30</code><code class="py plain">)</code>
</div>
<div class="line number32 index31 alt1">
<code class="py spaces"> </code>
</div>
<div class="line number33 index32 alt2">
<code class="py keyword">if</code> <code class="py plain">__name__ </code><code class="py keyword">=</code><code class="py keyword">=</code> <code class="py string">"__main__"</code><code class="py plain">:</code>
</div>
<div class="line number34 index33 alt1">
<code class="py spaces"> </code><code class="py keyword">if</code> <code class="py functions">len</code><code class="py plain">(sys.argv) !</code><code class="py keyword">=</code> <code class="py value">2</code><code class="py plain">:</code>
</div>
<div class="line number35 index34 alt2">
<code class="py spaces"> </code><code class="py functions">print</code> <code class="py string">"Wrong arg numbers"</code>
</div>
<div class="line number36 index35 alt1">
<code class="py spaces"> </code><code class="py keyword">else</code><code class="py plain">:</code>
</div>
<div class="line number37 index36 alt2">
<code class="py spaces"> </code><code class="py plain">cmd </code><code class="py keyword">=</code> <code class="py plain">sys.argv[</code><code class="py value">1</code><code class="py plain">]</code>
</div>
<div class="line number38 index37 alt1">
<code class="py spaces"> </code><code class="py keyword">if</code> <code class="py plain">cmd </code><code class="py keyword">=</code><code class="py keyword">=</code> <code class="py string">"put"</code><code class="py plain">: putter()</code>
</div>
<div class="line number39 index38 alt2">
<code class="py spaces"> </code><code class="py keyword">elif</code> <code class="py plain">cmd </code><code class="py keyword">=</code><code class="py keyword">=</code> <code class="py string">"get"</code><code class="py plain">: getter()</code>
</div>
<div class="line number40 index39 alt1">
<code class="py spaces"> </code><code class="py keyword">else</code><code class="py plain">: </code><code class="py functions">print</code> <code class="py string">"wrong cmd"</code>
</div>
</div>
</td>
</tr></tbody></table>
</div>
</div>
<div class="codetool" id="codetool">
<div class="code_n">
<textarea></textarea>
</div>
</div>
</div>
<p>
在使用时，开两个终端模拟两个进程，在一个终端中运行</p>
<p>
<code>python mqdemo.py put</code></p>
<p>
来模拟生产者；另一个终端中运行</p>
<p>
<code>python mqdemo.py get</code></p>
<p>
模拟消费者。</p>
頁:
[1]