一、背景
1. 讲故事
上个月中旬,星球里的一位朋友在微信找我,说他的程序跑着跑着内存会不断的缓慢增长并无法释放,寻求如何解决 ?
得,看样子星球还得好好弄!!!不管怎么说,先上 windbg 说话。
二、Windbg 分析
1. 经验推理
从朋友的截图看,有大量的 8216 字节的 byte[],这表示什么呢?追随本系列的朋友应该知道,有一篇 某三甲医院 的内存暴涨的dump中,也同样有此 size= (8216-24=8192) 的 byte[] 数组, 他的问题是 Oracle 中读取某大字段时sdk里的 OraBuf 出了问题,换句话说,这肯定又是底层或者第三方库中的池对象搞出来的东西,接下来从 托管堆 看起。
2. 查看托管堆 0:000>!dumpheap-stat Statistics : 00007ffe107248f048370715478624System.Threading.PreAllocatedOverlapped 00007ffe1079c16048374415479808System.Threading.ThreadPoolBoundHandle 00007ffe1079cff848370123217648System.Threading._IOCompletionCallback 00007ffe106e7a9048370423217792Microsoft.Win32.SafeHandles.SafeFileHandle 00007ffe1079b08848370330956992System.IO.FileSystemWatcher+AsyncReadState 00007ffe1079ceb048370734826904System.Threading.OverlappedData 00007ffe1079ccb048370734826904System.Threading.ThreadPoolBoundHandleOverlapped 0000016c646510802456521473128080 Free 00007ffe105abf304881723977571092System.Byte[]
扫完托管堆,卧槽 ,byte[] 没吸引到我,反而被 System.IO.FileSystemWatcher+AsyncReadState 吸引到了,毕竟被 System.IO.FileSystemWatcher 折腾多次了,它已经深深打入了我的脑海。。。毕竟让程序卡死,让句柄爆高的都是它。。。这一回八成又是它惹的祸,看样子还是有很多程序员栽在这里哈。
为做到严谨,我还是从最大的 System.Byte[] 入手,按size对它进行分组再按totalsize降序,丑陋的脚本我就不发了,直接上脚本的输出结果。
!dumpheap-mt00007ffe105abf30 size =8216, count =483703,totalsize=3790M size =8232, count =302,totalsize=2M size =65560, count =6,totalsize=0M size =131096, count =2,totalsize=0M size =4120, count =11,totalsize=0M size =56, count =301,totalsize=0M size =88, count =186,totalsize=0M size =848, count =16,totalsize=0M size =152, count =85,totalsize=0M size =46, count =242,totalsize=0M size =279, count =38,totalsize=0M !dumpheap-mt00007ffe105abf30- min 0n8216- max 0n8216-short 0000016c664277f0 0000016c66432a48 0000016c6648ef88 0000016c6649daa8 0000016c6649fb00 0000016c664a8b90 ...从输出结果看,size=8216 的 byte[] 有 48w 个,然后脚本也列出了一些 8216 大小的 address 地址,接下来用 !gcroot 看下这些地址的引用。
0:000>!gcroot0000016c664277f0 HandleTable: 0000016C65FC28C0(asyncpinnedhandle) ->0000016C6628DEB0System.Threading.OverlappedData ->0000016C664277F0System.Byte[] Found1 unique roots(run '!gcroot-all' to see all roots). 0:000>!gcroot0000016c667c80d0 HandleTable: 0000016C65FB7920(asyncpinnedhandle) ->0000016C663260F8System.Threading.OverlappedData ->0000016C667C80D0System.Byte[]从输出中可以看到这些 byte[] 都是 async pinned,也就是当异步IO回来的时候需要给 byte[] 填充的存储空间,接下来我们看看如何通过 OverlappedData 找到源码中定义为 8192 大小的 byte[] 地方。
如果你了解 FileSystemWatcher ,反向查找链大概是这样的 OverlappedData -> ThreadPoolBoundHandleOverlapped -> System.IO.FileSystemWatcher+AsyncReadState -> Buffer[], 这中间涉及到 ThreadPool 和 SafeHandle 的绑定。
0:000>!do0000016C663260F8 Name :System.Threading.OverlappedData MethodTable:00007ffe1079ceb0 EEClass:00007ffe107ac8d0 Size :72(0x48)bytes File:C:\ProgramFiles\dotnet\shared\Microsoft .NET Core.App\5.0.10\System.Private.CoreLib.dll Fields: MTFieldOffsetTypeVTAttrValue Name 00007ffe106e3c0840009ce8System.IAsyncResult0instance0000000000000000_asyncResult 00007ffe104a0c6840009cf10System.Object0instance0000016c66326140_callback 00007ffe1079cb6040009d018...eading.Overlapped0instance0000016c663260b0_overlapped 00007ffe104a0c6840009d120System.Object0instance0000016c667c80d0_userObject 00007ffe104af50840009d228PTR0instance00000171728f66e0_pNativeOverlapped 00007ffe104aee6040009d330System.IntPtr1instance0000000000000000_eventHandle 00007ffe104ab25840009d438System.Int321instance0_offsetLow 00007ffe104ab25840009d53cSystem.Int321instance0_offsetHigh 0:000>!do0000016c663260b0 Name :System.Threading.ThreadPoolBoundHandleOverlapped MethodTable:00007ffe1079ccb0 EEClass:00007ffe107ac858 Size :72(0x48)bytes File:C:\ProgramFiles\dotnet\shared\Microsoft.NETCore.App\5.0.10\System.Private.CoreLib.dll Fields: MTFieldOffsetTypeVTAttrValue Name 00007ffe1079ceb040009d68...ng.OverlappedData0instance0000016c663260f8_overlappedData 00007ffe1079b81840009c010...ompletionCallback0instance0000016f661ab8a0_userCallback 00007ffe104a0c6840009c118System.Object0instance0000016c667ca0e8_userState 00007ffe107248f040009c220...locatedOverlapped0instance0000016c66326090_preAllocated 00007ffe104af50840009c330PTR0instance00000171728f66e0_nativeOverlapped 00007ffe1079c16040009c428...adPoolBoundHandle0instance0000000000000000_boundHandle 00007ffe104a723840009c538System.Boolean1instance0_completed 00007ffe1079b81840009bf738...ompletionCallback0 static 0000016f661ab990s_completionCallback 0:000>!do0000016c667ca0e8 Name :System.IO.FileSystemWatcher+AsyncReadState MethodTable:00007ffe1079b088 EEClass:00007ffe107a9dc0 Size :64(0x40)bytes File:C:\ProgramFiles\dotnet\shared\Microsoft.NETCore.App\5.0.10\System.IO.FileSystem.Watcher.dll Fields: MTFieldOffsetTypeVTAttrValue Name 00007ffe104ab258400002b30System.Int321instance1 k__BackingField 00007ffe105abf30400002c8System.Byte[]0instance0000016c667c80d0
上面的
有了这些原理之后,接下来就可以问朋友是否有对 appsettings 设置了 reloadonchange=true 的情况,朋友找了下代码,写法大概如下:
public objectGetxxxFlag() { stringvalue=AppConfig.GetConfig( "appsettings.json" ).GetValue( "xxxx" , "0" ); return new { state=200, data=value }; } public classAppConfig { public static AppConfigGetConfig(stringsettingfile= "appsettings.json" ) { return newAppConfig(settingfile); } } public classAppConfig { privateAppConfig(stringsettingfile) { _config=newConfigurationBuilder().AddJsonFile(settingfile,optional: true ,reloadOnChange: true ).Build(); _settingfile=settingfile; } }从源码逻辑看,我猜测朋友将 GetConfig 方法标记成 static 后就以为是单例化了,再次调用不会重复 new AppConfig(settingfile),所以问题就出在这里。
不过有意思的是,前面二篇的 FileSystemWatcher 都会造成程序卡死,那这一篇为啥没有呢?恰好他没有在程序根目录中放日志文件,不然的话。。。,可万万没想到逃过了卡死却没逃过一个 watcher 默认 8byte 空间的灵魂拷问。。。
三、总结
总的来说,设置 reloadOnChange: true 一定要慎重, 可能它会造成你的程序卡死,句柄泄漏, 内存泄漏 等等!!!
原文链接:https://mp.weixin.qq.com/s/k0fBF772iE8whAGDd08hsw
dy("nrwz");
查看更多关于记一次 .NET 某风控管理系统内存泄漏分析的详细内容...