记一次 .NET 某低代码开发框架 内存暴涨分析
<h2 id="一背景">一：背景</h2><h3 id="1-讲故事">1. 讲故事</h3>
<p>微信里有一位朋友找到我，说他们公司的程序存在内存暴涨问题，自己分析了下没有找到原因，让我看下怎么回事？由于大家都有dump分析基础，所以交流互通上还是很顺利的，接下来就是上dump分析啦。</p>
<h2 id="二内存暴涨分析">二：内存暴涨分析</h2>
<h3 id="1-为什么会内存暴涨">1. 为什么会内存暴涨</h3>
<p>先还是老套路，用 <code>!address -summary</code> 观察下内存分布情况，输出如下：</p>
<pre><code class="language-C#">
0:000> !address -summary
--- Usage Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal
Free 363 7dfd`e87c7000 ( 125.992 TB) 98.43%
<unknown> 9276 201`e5858000 ( 2.007 TB)99.96% 1.57%
Heap 65 0`2547f000 ( 596.496 MB) 0.03% 0.00%
Image 1855 0`09d35000 ( 157.207 MB) 0.01% 0.00%
Stack 93 0`02c00000 (44.000 MB) 0.00% 0.00%
Other 9 0`001de000 ( 1.867 MB) 0.00% 0.00%
TEB 31 0`0003e000 ( 248.000 kB) 0.00% 0.00%
PEB 1 0`00001000 ( 4.000 kB) 0.00% 0.00%
--- State Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal
MEM_FREE 363 7dfd`e87c7000 ( 125.992 TB) 98.43%
MEM_RESERVE 690 201`2b6d4000 ( 2.005 TB)99.82% 1.57%
MEM_COMMIT 10640 0`ec155000 ( 3.689 GB) 0.18% 0.00%
</code></pre>
<p>从卦中可以看到,总计 <code>3.6G</code> 的总提交内存，看样子都落到了 <code>Unk</code> 区域，最好是托管层吃掉了，否则就麻烦了，接下来使用 <code>!dumpheap -stat</code> 观察，输出如下：</p>
<pre><code class="language-C#">
0:000> !dumpheap -stat
Statistics:
MT Count TotalSize Class Name
...
0179c7715cb01,847,901 451,265,880 Free
7ffc6e0a2888 2 536,870,960 System.WeakReference<Microsoft.Extensions.DependencyInjection.ServiceProvider>[]
7ffc6e0a2260 60,873,978 1,460,975,472 System.WeakReference<Microsoft.Extensions.DependencyInjection.ServiceProvider>
Total 63,333,893 objects, 2,494,520,292 bytes
</code></pre>
<p>从卦中可以看到程序中有 <code>6087w</code> 个弱引用，接下来使用 <code>!dumpheap -mt 7ffc6e0a2260</code> 观察下列表详情，然后用 <code>!gcroot</code> 观察其引用根，参考如下：</p>
<pre><code class="language-C#">
0:000> !dumpheap -mt 7ffc6e0a2260
Address MT Size
017988001000 7ffc6e0a2260 24
017988001018 7ffc6e0a2260 24
017988001030 7ffc6e0a2260 24
017988001048 7ffc6e0a2260 24
017988001060 7ffc6e0a2260 24
017988001078 7ffc6e0a2260 24
017988001090 7ffc6e0a2260 24
0179880010a8 7ffc6e0a2260 24
...
017a405f1020 7ffc6e0a2260 24
0:000> !gcroot 0179880010a8
Caching GC roots, this may take a while.
Subsequent runs of this command will be faster.
</code></pre>
<p>等了20多分钟都没有出来结果，可能 6kw 的根纵横交错让windbg不堪重负，没有就没撤了，使用内存搜索法寻找上级所属对象。这里就选择 <code>017a405f1020</code> 对象来开刀。</p>
<pre><code class="language-C#">0:000> !dumpobj /d 17a405f1020
Name: System.WeakReference`1[][]
MethodTable: 00007ffc6e0a2888
EEClass: 00007ffc6dbeb4f8
Tracked Type: false
Size: 536870936(0x20000018) bytes
Array: Rank 1, Number of elements 67108864, Type CLASS (Print Array)
Fields:
None
0:000> s-q 0 L?0xffffffffffffffff 17a405f1020
00000179`c95861d00000017a`405f1020 03a0dcfa`03a0dcfa
0:000> !lno 0000017a`405f1020
Before: 017a405f1000 32 (0x20) Free
Current: 017a405f1020 24 (0x18) System.WeakReference<Microsoft.Extensions.DependencyInjection.ServiceProvider>[]
Error Detected: Object 17a405f1020 has a bad member at offset 12054c00: ???
Could not find object after 17a405f1020
Heap local consistency not confirmed.
0:000> !lno 00000179`c95861d0
Before: 0179c95861c8 32 (0x20) System.Collections.Generic.List<System.WeakReference<Microsoft.Extensions.DependencyInjection.ServiceProvider>>
Next: 0179c95861e8 24 (0x18) System.WeakReference<Microsoft.Extensions.DependencyInjection.ServiceProvider>[]
Heap local consistency confirmed.
0:000> !dumpobj /d 179c95861c8
Name: System.Collections.Generic.List`1[], System.Private.CoreLib]]
MethodTable: 00007ffc6e0a2340
EEClass: 00007ffc6dce0000
Tracked Type: false
Size: 32(0x20) bytes
File: D:\xxx\A_api\System.Private.CoreLib.dll
Fields:
MT Field Offset Type VT Attr Value Name
00007ffc6de328f0400209f 8 System.__Canon[]0 instance 0000017a405f1020 _items
00007ffc6dc894b040020a0 10 System.Int321 instance 60873978 _size
00007ffc6dc894b040020a1 14 System.Int321 instance 60873978 _version
00007ffc6de328f040020a2 8 System.__Canon[]0 static dynamic statics NYI s_emptyArray
0:000> s-q 0 L?0xffffffffffffffff 179c95861c8
00000179`c77571d800000179`c95861c8 00000000`00000000
00000179`c95861b800000179`c95861c8 0800004e`00000000
0:000> !lno 00000179`c77571d8
Failed to find the segment of the managed heap where the object 179c77571d8 resides
0:000> !lno 00000179`c95861b8
Before: 0179c9586108 192 (0xc0) Microsoft.Extensions.DependencyInjection.DependencyInjectionEventSource
Next: 0179c95861c8 32 (0x20) System.Collections.Generic.List<System.WeakReference<Microsoft.Extensions.DependencyInjection.ServiceProvider>>
Heap local consistency confirmed.
</code></pre>
<p><img src="https://img2024.cnblogs.com/blog/214741/202603/214741-20260309102851386-581358220.png" alt="" loading="lazy"></p>
<p>根据卦中的图和输出，终于找到了原来是 <code>DependencyInjectionEventSource._providers</code> 承担了所有，接下来的关注点就来到了 <code>DependencyInjectionEventSource</code>。</p>
<h3 id="2-xxxeventsource-是什么">2. xxxEventSource 是什么</h3>
<p>从名字上看和 ETW 事件有关，接下来用 <code>!eeversion</code> 观察 .net 版本，寻找其对应的C#源代码。</p>
<pre><code class="language-C#">
0:000> !eeversion
6.0.3624.51421 free
6,0,3624,51421 @Commit: f1dd57165bfd91875761329ac3a8b17f6606ad18
Workstation mode
SOS Version: 9.0.13.2701 retail build
</code></pre>
<p><img src="https://img2024.cnblogs.com/blog/214741/202603/214741-20260309103749390-1935374313.png" alt="" loading="lazy"></p>
<p>从上面的源代码看，其实也看不出来个所以，毕竟底层的架构我不熟悉，本着我不是第一个吃螃蟹的人，所以拿关键词在网上索一下，果然 stephentoub 大佬在去年4月份就发现了这个问题，在 .net10 中做了修复，看描述是一个优化级的bug，官方链接：https://github.com/dotnet/runtime/issues/114599 截图如下：</p>
<p><img src="https://img2024.cnblogs.com/blog/214741/202603/214741-20260309104008672-2005324636.png" alt="" loading="lazy"></p>
<p><img src="https://img2024.cnblogs.com/blog/214741/202603/214741-20260309103357229-35928813.png" alt="" loading="lazy"></p>
<p>修改后的代码如下，果然加了很多的业务逻辑来处理。</p>
<pre><code class="language-C#">
public void ServiceProviderBuilt(ServiceProvider provider)
{
lock (_providers)
{
int providersCount = _providers.Count;
if (providersCount > 0 &&
(_survivingProvidersCount is int spc ? (uint)providersCount >= 2 * (uint)spc : providersCount == _providers.Capacity))
{
_providers.RemoveAll(static p => !p.TryGetTarget(out _));
_survivingProvidersCount = _providers.Count;
}
_providers.Add(new WeakReference<ServiceProvider>(provider));
}
WriteServiceProviderBuilt(provider);
}
</code></pre>
<p>从官方描述来看，就是有人创建了 scope，但后续没有调用 dispose 方法来及时释放，导致框架中的 WeakReference 引用滞留，引发内存暴涨，可以说两者都有责任吧。</p>
<p>解决办法很简单，两种方式：</p>
<ol>
<li>检查代码里写 BuildServiceProvider 的地方没有即时的 Dispose。</li>
<li>升级到 .NET10 ，这是最简单粗暴的方法。</li>
</ol>
<p>把结论告诉朋友后，朋友终于在2天后给我反馈了好消息，好心情溢于言表！</p>
<p><img src="https://img2024.cnblogs.com/blog/214741/202603/214741-20260309104826540-387161016.png" alt="" loading="lazy"></p>
<h2 id="三总结">三：总结</h2>
<p>dump之旅是一个修理工不断自我修炼的过程，必须学会在绝望中寻找希望的能力。</p>
<img src="https://images.cnblogs.com/cnblogs_com/huangxincheng/345039/o_210929020104最新消息优惠促销公众号关注二维码.jpg" width="700" height="300" alt="图片名称" align="center"><br><br>
来源：https://www.cnblogs.com/huangxincheng/p/19688949
頁:
[1]