Can't sync the lotus node on calibnet

[shmilytsui]

Hello!
I run the lotus daemon on the calibnet and prepared two hosts(hostA and hostB), but there were errors when synchronizing data.

HostA was directly synchronized without importing the snapshot, and hostB was synchronized after exporting the snapshot, hostA run "lotus daemon wait" and "lotus daemon status", there is nothing output，and the sycn state of the hostB is error latest and it never output "done".

So,how can I sync lotus node on calibnet?

Thank You!

[shmilytsui]

HostA:

root@daemon0-137test:/shared/lotus# lotus version
Daemon: 1.23.0+calibnet+git.d1d4b35ad.dirty+api1.5.0
Local: lotus version 1.23.0+calibnet+git.d1d4b35ad.dirty

---

root@daemon0-137test:/shared/lotus# lotus sync status
sync status:
worker 1798:
Base: [bafy2bzacebqu6skghkgfqgcodf6j7bomgag426zi6wyyvsu5em4tqlkgrjwn4 bafy2bzacecmefcbszv2ucih55pvckrxomgi4heded2zoamfhotb52liiayv7o bafy2bzaceae4thukyzie4gyaecdgvwisvkvrpdyyndu2zukjpxnhbtunpxtls bafy2bzacea3nplir4rewhs2iu3o7wdt622hsb72by3xkzs57y5a7pen5phofc]
Target: [bafy2bzacebehxtqtq3ordlg2clsv75o3zhrkra6gxf6luafbda7ybdijrqylc] (604544)
Height diff: 142304
Stage: error
Height: 0
Elapsed: 141.984µs
Error: chain linked to block marked previously as bad ([bafy2bzacebehxtqtq3ordlg2clsv75o3zhrkra6gxf6luafbda7ybdijrqylc], bafy2bzaceabw4udwho2gnpg5f4ougkbq2bqist65ta6qta4lwi7eh6rffvd3w) (reason: change contained bafy2bzaceabkar7y5cypr36jw2fhswps4koj532jvlltwxioriqhtefqocbfu caused by: [bafy2bzacedmg7owf3vwrokab65e4twekd5tubwajysocdcnb4z5ghit3kcg3e] 1 error occurred:
* parent state root did not match computed state (bafy2bzaceaosthwndev2ruiuacdowuj2nyf6f22cptvqw6vn3nehg32vgdpda != bafy2bzacec77ppj3tpme4pkpgdjvbhok63ym2oj2y2q3ufmyi2yb5phfdj3ne):
github.com/filecoin-project/lotus/chain/consensus.CommonBlkChecks.func3
/opt/src/lotus/chain/consensus/common.go:118

)
worker 1799:
Base: [bafy2bzacebqu6skghkgfqgcodf6j7bomgag426zi6wyyvsu5em4tqlkgrjwn4 bafy2bzacecmefcbszv2ucih55pvckrxomgi4heded2zoamfhotb52liiayv7o bafy2bzaceae4thukyzie4gyaecdgvwisvkvrpdyyndu2zukjpxnhbtunpxtls bafy2bzacea3nplir4rewhs2iu3o7wdt622hsb72by3xkzs57y5a7pen5phofc]
Target: [bafy2bzacecag6ztihlmafoyqc47b6o3vnlddolbmv4xekocms3njnp5zcwg26] (604544)
Height diff: 142304
Stage: error
Height: 0
Elapsed: 102.072µs
Error: chain linked to block marked previously as bad ([bafy2bzacecag6ztihlmafoyqc47b6o3vnlddolbmv4xekocms3njnp5zcwg26], bafy2bzaceabw4udwho2gnpg5f4ougkbq2bqist65ta6qta4lwi7eh6rffvd3w) (reason: change contained bafy2bzaceabkar7y5cypr36jw2fhswps4koj532jvlltwxioriqhtefqocbfu caused by: [bafy2bzacedmg7owf3vwrokab65e4twekd5tubwajysocdcnb4z5ghit3kcg3e] 1 error occurred:
* parent state root did not match computed state (bafy2bzaceaosthwndev2ruiuacdowuj2nyf6f22cptvqw6vn3nehg32vgdpda != bafy2bzacec77ppj3tpme4pkpgdjvbhok63ym2oj2y2q3ufmyi2yb5phfdj3ne):
github.com/filecoin-project/lotus/chain/consensus.CommonBlkChecks.func3
/opt/src/lotus/chain/consensus/common.go:118

)
worker 1800:
Base: [bafy2bzacebqu6skghkgfqgcodf6j7bomgag426zi6wyyvsu5em4tqlkgrjwn4 bafy2bzacecmefcbszv2ucih55pvckrxomgi4heded2zoamfhotb52liiayv7o bafy2bzaceae4thukyzie4gyaecdgvwisvkvrpdyyndu2zukjpxnhbtunpxtls bafy2bzacea3nplir4rewhs2iu3o7wdt622hsb72by3xkzs57y5a7pen5phofc]
Target: [bafy2bzacedhnwc5yk4tx3fvnvq5dqycyqyh45vird3flj3n2vbskeajo5vlt6] (604545)
Height diff: 142305
Stage: error
Height: 0
Elapsed: 138.663µs
Error: chain linked to block marked previously as bad ([bafy2bzacedhnwc5yk4tx3fvnvq5dqycyqyh45vird3flj3n2vbskeajo5vlt6], bafy2bzacecag6ztihlmafoyqc47b6o3vnlddolbmv4xekocms3njnp5zcwg26) (reason: linked to bafy2bzaceabw4udwho2gnpg5f4ougkbq2bqist65ta6qta4lwi7eh6rffvd3w caused by: [bafy2bzacedmg7owf3vwrokab65e4twekd5tubwajysocdcnb4z5ghit3kcg3e] 1 error occurred:
* parent state root did not match computed state (bafy2bzaceaosthwndev2ruiuacdowuj2nyf6f22cptvqw6vn3nehg32vgdpda != bafy2bzacec77ppj3tpme4pkpgdjvbhok63ym2oj2y2q3ufmyi2yb5phfdj3ne):
github.com/filecoin-project/lotus/chain/consensus.CommonBlkChecks.func3
/opt/src/lotus/chain/consensus/common.go:118

)

---

daemon0-137test:/shared/lotus# lotus sync wait
Worker: 1804; Base: 462240; Target: 604549 (diff: 142309)
State: error; Current Epoch: 604549; Todo: 0
Validated 18 messages (0 per second)

some logs like this:

2023-05-31T00:09:00.165+0800 INFO chain chain/sync_manager.go:233 selected sync target: [bafy2bzaceaszrq32zpzpeplzcqbtzfs5aic3oeo2u2f2dpf4deb6qtp7grxcq]
2023-05-31T00:09:00.165+0800 INFO chain chain/sync_manager.go:315 worker 1807 syncing in [bafy2bzaceaszrq32zpzpeplzcqbtzfs5aic3oeo2u2f2dpf4deb6qtp7grxcq]
2023-05-31T00:09:00.801+0800 INFO chain chain/sync.go:790 Got blocks: 604551 500
2023-05-31T00:09:00.801+0800 INFO chain chain/sync_manager.go:323 worker 1807 done; took 635.974214ms
2023-05-31T00:09:00.801+0800 ERROR chain chain/sync_manager.go:254 error during sync in [bafy2bzaceaszrq32zpzpeplzcqbtzfs5aic3oeo2u2f2dpf4deb6qtp7grxcq]: collectChain failed: chain contained block marked previously as bad ([bafy2bzaceaszrq32zpzpeplzcqbtzfs5aic3oeo2u2f2dpf4deb6qtp7grxcq], bafy2bzacebe5s6c6rqmzlypmy2o6nuvkivb5qcduibyl2xkdb3wfgqikamb5w) (reason: linked to bafy2bzacedhnwc5yk4tx3fvnvq5dqycyqyh45vird3flj3n2vbskeajo5vlt6 caused by: [bafy2bzacedmg7owf3vwrokab65e4twekd5tubwajysocdcnb4z5ghit3kcg3e] 1 error occurred:
* parent state root did not match computed state (bafy2bzaceaosthwndev2ruiuacdowuj2nyf6f22cptvqw6vn3nehg32vgdpda != bafy2bzacec77ppj3tpme4pkpgdjvbhok63ym2oj2y2q3ufmyi2yb5phfdj3ne):
github.com/filecoin-project/lotus/chain/consensus.CommonBlkChecks.func3
/opt/src/lotus/chain/consensus/common.go:118

)

---

root@daemon0-137test:/shared/lotus# du -sh /shared/lotus
45G /shared/lotus

[shmilytsui]

hostB:

root@daemon0-142:/data/lotus# lotus -version
lotus version 1.23.0+calibnet+git.d1d4b35ad.dirty

---

root@daemon0-142:/data/lotus# lotus sync status
sync status:

---

root@daemon0-142:/data/lotus# lotus sync wait

//there is nothing output

---

some error logs like this:

2023-05-31T00:14:00.925+0800 ERROR statetree state/statetree.go:294 failed to load state tree: failed to load hamt node: ipld: could not find bafy2bzacecd4ss7uhgd2oocl73howy2p4lmxtecnzgl5rsce3gic2st5wkjfu
2023-05-31T00:14:00.930+0800 ERROR statetree state/statetree.go:294 failed to load state tree: failed to load hamt node: ipld: could not find bafy2bzacecd4ss7uhgd2oocl73howy2p4lmxtecnzgl5rsce3gic2st5wkjfu
2023-05-31T00:14:00.933+0800 ERROR statetree state/statetree.go:294 failed to load state tree: failed to load hamt node: ipld: could not find bafy2bzacecd4ss7uhgd2oocl73howy2p4lmxtecnzgl5rsce3gic2st5wkjfu
2023-05-31T00:14:00.942+0800 ERROR statetree state/statetree.go:294 failed to load state tree: failed to load hamt node: ipld: could not find bafy2bzacecd4ss7uhgd2oocl73howy2p4lmxtecnzgl5rsce3gic2st5wkjfu

[shmilytsui]

Well, that's about all of the some infomation of hostA and hostB.

So what can I do now? Please help me!

[TippyFlitsUK]

Both of your builds are dirty @shmilytsui! Is this intentional?

Can you also please provide the link for the snapshot that you downloaded?

I can confirm that I am not seeing any issues syncing Calib on my own setup.

[shmilytsui]

Yes! The link is : https://www.mediafire.com/file/gquphc7qw0ffzdk/lotus_cali_snapshot_2022_05_18_high_959844.car.tar.gz
I download the lotus_cali_snapshot_2022_05_18_high_959844.car.tar.gz manually. Just like this : wget https://download855.mediafire.com/z0fel1w7q4sgJicq0EtMhzFaUoYgtHnNBki1hlUqA9l81TYMxOWn3kzhZfyNoq5k_WyVcpZaEPlGQFegPkLZzCi7NXyvBAZOcMFth07ORHRoTGRTzkNGhhAbAzIpo4U3UE_U-kFI1EyJYkWIksLZ1B3dJUpVJMZ2f1AgXP01yIw/gquphc7qw0ffzdk/lotus_cali_snapshot_2022_05_18_high_959844.car.tar.gz

[shmilytsui]

The link is from https://lotus.filecoin.io/tutorials/lotus/build-with-lotus-api/

[TippyFlitsUK]

Thank you @shmilytsui 🙏

Please try using the following snapshot download link instead - https://snapshots.calibrationnet.filops.net/minimal/latest.zst

Can you also please confirm if your dirty build is deliberate?

[shmilytsui]

This issue has been resolved, possibly because I used an old snapshot file.
Now I run "lotus daemon --import-snapshot /608400_2023_06_01T00_13_00Z.car" again and it done.
Thank you very mush!