Automatically compact DB #435
Conversation
cmd/opera/launcher/db-transform.go
Outdated
| @@ -264,7 +264,7 @@ func transformComponent(datadir string, dbTypes, tmpDbTypes map[multidb.TypeName | |||
| return err | |||
| } | |||
| toMove[dbLocatorOf(e.New)] = true | |||
| newDB = batched.Wrap(newDB) | |||
| newDB = batched.Wrap(autocompact.Wrap(autocompact.Wrap(newDB, 1*opt.GiB), 16*opt.GiB)) | |||
There was a problem hiding this comment.
Can you explain why we always need to double wrap with difference limit like this?
There was a problem hiding this comment.
So that we it would automatically compact every 1 GB of added AND every 16 GB of inserted data
utils/dbutil/autocompact/store.go
Outdated
| } | ||
|
|
||
| func (s *Batch) Put(key []byte, value []byte) error { | ||
| s.written += uint64(len(key) + len(value) + 64) |
There was a problem hiding this comment.
Can you explain, please, why you add up 64 here?
There was a problem hiding this comment.
Storage overheads, related to adding/deleting a record, wouldn't be only proportional to length of key and value. E.g. if one adds 10 records with length of 2, it will be more expensive than 1 record with length 20
Now, 64 wasn't really calculated but is rather a guesstimation
|
Added constraints for |
uprendis
force-pushed
the
feature/autocompact
branch
from
2c68189 to
c2cc618
Compare
It would be great to add some test coverage as some complexity has been introduced |
Fix #433