Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pika startup time is too long 启动时间过长 #3031

Open
chenbt-hz opened this issue Feb 28, 2025 · 4 comments
Open

pika startup time is too long 启动时间过长 #3031

chenbt-hz opened this issue Feb 28, 2025 · 4 comments
Labels
☢️ Bug Something isn't working

Comments

@chenbt-hz
Copy link
Collaborator

Is this a regression?

Yes

Description

215g数据启动时间大约需要15分钟
相同配置使用老版本就没有这个问题。

暂时没定位到具体原因

Please provide a link to a minimal reproduction of the bug

No response

Screenshots or videos

No response

Please provide the version you discovered this bug in (check about page for version information)

Version: 4.0.1-alpha以后的版本

Anything else?

No response

@chenbt-hz chenbt-hz added the ☢️ Bug Something isn't working label Feb 28, 2025
@wangshao1
Copy link
Collaborator

有一个max-total-wal-size的参数用来设置单个rocksdb的wal的最大值。
使用复杂数据类型,元信息和data信息存放在不同Columnfamily,元信息数据量小,可能一直没有flush,导致的wal就会攒的比较多。
后续pika这边需要支持并发地open rocksdb,现在顺序open时间也长。

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


There is a max-total-wal-size parameter to set the maximum value of wal for a single rocksdb.
Using complex data types, meta information and data information are stored in different Columnfamily. The amount of meta information data is small, and there may be no flush, which will cause more wals to be saved.
In the future, PIKA needs to support concurrent open rocksdb, and now the order of opening is longer.

@chenbt-hz
Copy link
Collaborator Author

有一个max-total-wal-size的参数用来设置单个rocksdb的wal的最大值。 使用复杂数据类型,元信息和data信息存放在不同Columnfamily,元信息数据量小,可能一直没有flush,导致的wal就会攒的比较多。 后续pika这边需要支持并发地open rocksdb,现在顺序open时间也长。

这个配置就是默认的1G+6db。并且相同配置我换个老版本的包启动就挺快的

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


There is a max-total-wal-size parameter to set the maximum value of wal for a single rocksdb. Using complex data types, meta information and data information are stored in different Columnfamily. The amount of meta information data is small, and there may be no flush, which will cause more wals to be saved. In the future, PIKA needs to support concurrent open rocksdb, and now the order of opening is longer.

This configuration is the default 1G+6db. And I will start with the same configuration quickly if I change the old version of the package.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
☢️ Bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants