Optimize CINN cache key #37786
Optimize CINN cache key #37786
Conversation
|
Thanks for your contribution! |
… optimize_cinn_cache_key
… optimize_cinn_cache_key
… optimize_cinn_cache_key
| @@ -29,55 +29,69 @@ namespace paddle { | |||
| namespace framework { | |||
| namespace paddle2cinn { | |||
|
|
|||
| using GraphHashProto = CinnCacheKey::GraphHashProto; | |||
There was a problem hiding this comment.
这里Proto后缀中文应该翻译成什么？我一开始以为是个protobuf类
There was a problem hiding this comment.
proto表示原型，这里的意思是函数原型
There was a problem hiding this comment.
你这里像设计模式中的策略模式，可以考虑是否换成strategy为后缀，否则容易和protobuf混淆，不过这不重要，看你喜好。
There was a problem hiding this comment.
Done，已修改GraphHashProto为GraphHashStrategy~
| std::unordered_map<CinnCacheKey, std::unique_ptr<CinnCompiledObject>, | ||
| std::unordered_map<CinnCacheKeyByAddress, CinnCompiledObject*, | ||
| CinnCacheKey::Hash> | ||
| cache_; | ||
| cache_by_address_; | ||
| std::unordered_map<CinnCacheKeyByStructure, | ||
| std::unique_ptr<CinnCompiledObject>, CinnCacheKey::Hash> | ||
| cache_by_struct_; |
There was a problem hiding this comment.
第二层cache是不是没用了？没命中第一层cache却命中第二层的情况是存在两个不同子图结构、所有变量名、shape、arch相同，但其中变量名应该没法一致
There was a problem hiding this comment.
感觉还是有用的吧，比如说万一图被clone了导致图结构虽然一模一样，但地址变了？
There was a problem hiding this comment.
现有场景CinnCompiler的compile只从cinn_launch_op调用，它是用compilation_key先从CinnCompiler拿出Graph，compilation_key和存储的Graph都是在子图划分pass跑完就不变，所以应该不会出现你说的情况。不过也可以保留吧，看后续是否可以把变量名去掉，做到结构一致时也能命中第二层cache
… optimize_cinn_cache_key
… optimize_cinn_cache_key
PR types
Performance optimization
PR changes
OPs
Describe
Optimize cinn_launch_op compile's cache key generates. Test at 32G V100 bs=32, after optimizing, the cost of cinn_cache_key construct and get can reduce from 5ms to 1ms. The IPS would improve from 241 images/sec to 245 images/sec, improving about 1.6%.
Before Optimizing:
After Optimizing:
优化方法
在原有根据图结构来判断是否是相同图外，增加一个根据图地址来判断是否是相同图的功能。
CinnCacheKey基类，其中包含大部分key所需的必要函数，构造函数增加graph_hash函数用于获取图的has值。CinnCacheKey派生出CinnCacheKeyByStructure类，设置graph_hash返回图结构的hash值，该类即为原有的根据图结构来判断是否是相同图。CinnCacheKey派生出CinnCacheKeyByAddress类，设置graph_hash返回图地址的hash值，该类即为新增的根据图地址来判断是否是相同图。Compile设置两层检查，第一层检查图地址是否相同，第二层检查图结构是否相同，图地址检查相比图结构检查速度快很多。因此相比原有单层检查图结构起到了性能优化的作用。