python - pyspider使用redis分布式

问题描述

在部署pyspider分布式时,尝试连接redis集群配置项为： 'message_queue':'redis://127.0.0.1:6380/db'在启动时：报如下错误[E 170221 09:40:39 tornado_fetcher:529] MOVED 12068 127.0.0.1:6380

Traceback (most recent call last): File '/usr/lib/python2.6/site-packages/pyspider/fetcher/tornado_fetcher.py', line 515, in queue_loop if self.outqueue.full(): File '/usr/lib/python2.6/site-packages/pyspider/message_queue/redis_queue.py', line 50, in full if self.maxsize and self.qsize() >= self.maxsize: File '/usr/lib/python2.6/site-packages/pyspider/message_queue/redis_queue.py', line 40, in qsize self.last_qsize = self.redis.llen(self.name) File '/usr/lib/python2.6/site-packages/redis/client.py', line 1219, in llen return self.execute_command(’LLEN’, name) File '/usr/lib/python2.6/site-packages/redis/client.py', line 573, in execute_command return self.parse_response(connection, command_name, **options) File '/usr/lib/python2.6/site-packages/redis/client.py', line 585, in parse_response response = connection.read_response() File '/usr/lib/python2.6/site-packages/redis/connection.py', line 582, in read_response raise responseResponseError: MOVED 12068 127.0.0.1:6380

经查找，得到：因为redis由单节点变为集群，而python的redis连接包暂时还不支持redis集群连接方式，需要更换连接包（redis-py-cluster）。

在安装完redis-py-cluster之后，依旧报上面的错误。

问题解答

链接 redis 使用的库是写在代码里面的，不是你安装了就会切换的。就算你要改代码，也要确认 API 是兼容的。