Python aiohttp raise RuntimeError(‘Event loop is closed‘)
<h2 id="问题描述">问题描述</h2><p>aiohttp 的 getting started 入门案例是这样写的</p>
<pre><code class="language-python">import aiohttp
import asyncio
async def main():
async with aiohttp.ClientSession() as session:
async with session.get('https://python.org') as response:
print("Status:", response.status)
print("Content-type:", response.headers['content-type'])
html = await response.text()
print("Body:", html[:15], "...")
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
</code></pre>
<p>运行结果为</p>
<pre><code>Status: 200
Content-type: text/html; charset=utf-8
Body: <!doctype html> ...
</code></pre>
<p>看上去没问题，但是在 Python3.7 后对 asyncio 进行了改进，可以直接调用 <code>asyncio.run()</code> 执行协程程序，而不需管底层 API 如事件循环 loop 的操作，所以上述代码的</p>
<pre><code class="language-python">loop = asyncio.get_event_loop()
loop.run_until_complete(main())
</code></pre>
<p>可以直接替换为</p>
<pre><code class="language-python">asyncio.run()
</code></pre>
<p>Linux 和 Mac 上这样运行是没问题的，但是在 Windows 上运行会报如下错误</p>
<pre><code>Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x000001B1FFE978B0>
Traceback (most recent call last):
File "D:\DevTools\Python\lib\asyncio\proactor_events.py", line 116, in __del__
self.close()
File "D:\DevTools\Python\lib\asyncio\proactor_events.py", line 108, in close
self._loop.call_soon(self._call_connection_lost, None)
File "D:\DevTools\Python\lib\asyncio\base_events.py", line 746, in call_soon
self._check_closed()
File "D:\DevTools\Python\lib\asyncio\base_events.py", line 510, in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
</code></pre>
<h2 id="原因分析">原因分析</h2>
<p>像 aiohttp 这类第三方协程库都是依赖于标准库 asyncio 的，而 asyncio 对 Windows 的支持本来就不好。Python3.8 后默认 Windows 系统上的事件循环采用 <code>ProactorEventLoop</code> （仅用于 Windows ）这篇文档描述了其在 Windows 下的缺陷：https://docs.python.org/zh-cn/3/library/asyncio-platforms.html#windows 👈</p>
<p>引发异常的函数是 <code>_ProactorBasePipeTransport.__del__</code> ，所以 aiohttp 铁定使用了 _ProactorBasePipeTransport，并且在程序退出释放内存时自动调用了其<code>__del__</code> 方法</p>
<p><img src="https://img2023.cnblogs.com/blog/2119256/202308/2119256-20230811091109387-384859993.png" alt="在这里插入图片描述" loading="lazy"></p>
<p><img src="https://img2023.cnblogs.com/blog/2119256/202308/2119256-20230811091109362-136867926.png" alt="在这里插入图片描述" loading="lazy"></p>
<p><img src="https://img2023.cnblogs.com/blog/2119256/202308/2119256-20230811091109338-1440932043.png" alt="在这里插入图片描述" loading="lazy"></p>
<p>就是上述一串连环反应最终抛出了 RuntimeError: Event loop is closed</p>
<p>一般的协程程序是不会使用 _ProactorBasePipeTransport 的，所以下面的代码还是可以正常的运行</p>
<pre><code class="language-python">import asyncio
async def main():
print('Hello...')
await asyncio.sleep(1)
print('...World')
asyncio.run(main())
</code></pre>
<p>我特意写了个装饰器来验证这一点：</p>
<pre><code class="language-python">import asyncio
from asyncio.proactor_events import _ProactorBasePipeTransport
from functools import wraps
def explicit_call(func):
@wraps(func)
def wrappers(self, *args, **kwargs):
print('call _ProactorBasePipeTransport.__del__')
return func(self, *args, **kwargs)
return wrappers
_ProactorBasePipeTransport.__del__ = explicit_call(_ProactorBasePipeTransport.__del__)
async def main():
print('Hello...')
await asyncio.sleep(1)
print('...World')
asyncio.run(main())
</code></pre>
<p>正常执行，没有使用 _ProactorBasePipeTransport</p>
<pre><code>Hello...
...World
</code></pre>
<pre><code class="language-python">import asyncio
from asyncio.proactor_events import _ProactorBasePipeTransport
from functools import wraps
import aiohttp
async def main():
async with aiohttp.ClientSession() as session:
async with session.get('https://www.baidu.com') as response:
print("Status:", response.status)
print("Content-type:", response.headers['content-type'])
html = await response.text()
print("Body:", html[:15], "...")
def explicit_call(func):
@wraps(func)
def wrappers(self, *args, **kwargs):
print('call _ProactorBasePipeTransport.__del__')
return func(self, *args, **kwargs)
return wrappers
_ProactorBasePipeTransport.__del__ = explicit_call(_ProactorBasePipeTransport.__del__)
asyncio.run(main())
</code></pre>
<p>先打印 <code>call _ProactorBasePipeTransport.__del__</code> 然后报错，说明使用了 _ProactorBasePipeTransport</p>
<pre><code>Status: 200
Content-type: text/html
Body: <html>
...
call _ProactorBasePipeTransport.__del__
Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x000001B11653EF70>
Traceback (most recent call last):
File "D:\Codes\SpiderProject\BingImageSpider\demo.py", line 16, in wrappers
return func(self, *args, **kwargs)
File "D:\DevTools\Python\lib\asyncio\proactor_events.py", line 116, in __del__
self.close()
File "D:\DevTools\Python\lib\asyncio\proactor_events.py", line 108, in close
self._loop.call_soon(self._call_connection_lost, None)
File "D:\DevTools\Python\lib\asyncio\base_events.py", line 746, in call_soon
self._check_closed()
File "D:\DevTools\Python\lib\asyncio\base_events.py", line 510, in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
进程已结束，退出代码为 0
</code></pre>
<h2 id="解决方案">解决方案</h2>
<p>如果执意要在 Windows 下继续开发，有这几个方案可以选择</p>
<h3 id="1-不要使用-run-函数">1. 不要使用 run 函数</h3>
<p>既然 _ProactorBasePipeTransport 会在程序结束后自动关闭事件循环，那就不要用 run 函数了，用官网的例子，乖乖使用 loop 吧</p>
<pre><code class="language-python">loop = asyncio.get_event_loop()
loop.run_until_complete(main())
</code></pre>
<h3 id="2-替换事件循环">2. 替换事件循环</h3>
<p>在调用 run 函数前，替换默认的 ProactorEventLoop 为 SelectorEventLoop</p>
<pre><code class="language-python">asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
asyncio.run(main())
</code></pre>
<p>但是 SelectorEventLoop 是有一些缺点的，比如不支持子进程等</p>
<h3 id="3-忽略异常">3. 忽略异常</h3>
<p>这是 Github 上一个外国大佬的方法，在不改变源码的前提下，使用装饰器忽略掉异常</p>
<pre><code class="language-python">import asyncio
from asyncio.proactor_events import _ProactorBasePipeTransport
from functools import wraps
import aiohttp
async def main():
async with aiohttp.ClientSession() as session:
async with session.get('https://www.baidu.com') as response:
print("Status:", response.status)
print("Content-type:", response.headers['content-type'])
html = await response.text()
print("Body:", html[:15], "...")
def silence_event_loop_closed(func):
@wraps(func)
def wrapper(self, *args, **kwargs):
try:
return func(self, *args, **kwargs)
except RuntimeError as e:
if str(e) != 'Event loop is closed':
raise
return wrapper
_ProactorBasePipeTransport.__del__ = silence_event_loop_closed(_ProactorBasePipeTransport.__del__)
asyncio.run(main())
</code></pre>
<p>更详细的信息可以在这个 issue 上找到：https://github.com/aio-libs/aiohttp/issues/4324 👈</p>
<blockquote>
<p>相关链接：</p>
<ul>
<li>https://github.com/aio-libs/aiohttp/issues/4324</li>
<li>https://stackoverflow.com/questions/45600579/asyncio-event-loop-is-closed-when-getting-loop</li>
<li>https://docs.python.org/zh-cn/3/library/asyncio-platforms.html#windows</li>
<li>https://bugs.python.org/issue39232</li>
</ul>
</blockquote>
<hr>
<p>喜欢我的文章的话，欢迎<code>关注</code>👇<code>点赞</code>👇<code>评论</code>👇<code>收藏</code>👇 谢谢支持！！！</p><br><br>
来源：https://www.cnblogs.com/james-wangx/p/16111485.html
頁:
[1]