Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Api called for an unknown Block: Header was not found in the database #670

Open
anignatev opened this issue Apr 6, 2023 · 9 comments
Open

Api called for an unknown Block: Header was not found in the database #670

anignatev opened this issue Apr 6, 2023 · 9 comments
Assignees
@tdimitrov

Comments

@anignatev
Copy link

After updating from v0.9.40 to v0.9.41, A bug has been detected that causes the node to fail and disable validation.

Role of the node: validator.
Startup type: run compiled binary
Version: v0.9.41
OS: FreeBSD 13.1-RELEASE-p3
Kernel: 13.1-RELEASE-p3
MemTotal: 64 GB
SwapTotal: 64 GB
CPU Type: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz
CPU Threads: 12

Command-line options:

/usr/local/bin/polkadot --database paritydb --validator --name NAME --chain=kusama \
 --keystore-path KEYSTORE_PATH --base-path BASE_PATH \
 --telemetry-url 'wss://telemetry-backend.w3f.community/submit 1

Logs (repeated many times):

Apr  6 00:27:36 ks polkadot[38477]: 2023-04-06 00:27:36 âM-^\¨ Imported #17354782 (0x303eâM-^@¦178a)    
Apr  6 00:27:37 ks polkadot[38477]: 2023-04-06 00:27:37 âM-^\¨ Imported #17354782 (0x43a4âM-^@¦1778)    
Apr  6 00:27:40 ks polkadot[38477]: 2023-04-06 00:27:40 ðM-^_M-^R¤ Idle (40 peers), best: #17354782 (0x303eâM-^@¦178a), finalized #17354780 (0x08b3âM-^@¦6045), â¬M-^G 2.7MiB/s â¬M-^F 2.1MiB/s    
Apr  6 00:27:42 ks polkadot[38477]: 2023-04-06 00:27:42 âM-^\¨ Imported #17354783 (0x5afdâM-^@¦e99b)    
Apr  6 00:27:45 ks polkadot[38477]: 2023-04-06 00:27:45 ðM-^_M-^R¤ Idle (40 peers), best: #17354783 (0x5afdâM-^@¦e99b), finalized #17354780 (0x08b3âM-^@¦6045), â¬M-^G 1.6MiB/s â¬M-^F 1.4MiB/s    
Apr  6 00:27:48 ks polkadot[38477]: 2023-04-06 00:27:48 âM-^\¨ Imported #17354784 (0x69bfâM-^@¦57cc)    
Apr  6 00:27:50 ks polkadot[38477]: 2023-04-06 00:27:50 ðM-^_M-^R¤ Idle (40 peers), best: #17354784 (0x69bfâM-^@¦57cc), finalized #17354781 (0xce9bâM-^@¦19f5), â¬M-^G 1.9MiB/s â¬M-^F 1.6MiB/s    
Apr  6 00:27:55 ks polkadot[38477]: 2023-04-06 00:27:55 ðM-^_M-^R¤ Idle (40 peers), best: #17354784 (0x69bfâM-^@¦57cc), finalized #17354782 (0x303eâM-^@¦178a), â¬M-^G 791.6kiB/s â¬M-^F 795.7kiB/s    
Apr  6 00:27:55 ks polkadot[38477]: 2023-04-06 00:27:55 ðM-^_M-^Q¶ New epoch 29380 launching at block 0xbdffâM-^@¦01bc (block slot 280121679 >= start slot 280121679).    
Apr  6 00:27:55 ks polkadot[38477]: 2023-04-06 00:27:55 ðM-^_M-^Q¶ Next epoch starts at slot 280122279    
Apr  6 00:28:00 ks polkadot[38477]: 2023-04-06 00:28:00 ðM-^_M-^R¤ Idle (40 peers), best: #17354784 (0x69bfâM-^@¦57cc), finalized #17354782 (0x303eâM-^@¦178a), â¬M-^G 1.7MiB/s â¬M-^F 469.8kiB/s    
Apr  6 00:28:05 ks polkadot[38477]: 2023-04-06 00:28:05 ðM-^_M-^R¤ Idle (40 peers), best: #17354784 (0x69bfâM-^@¦57cc), finalized #17354782 (0x303eâM-^@¦178a), â¬M-^G 1.4MiB/s â¬M-^F 128.1kiB/s    
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:07 ks polkadot[38477]: 2023-04-06 00:28:07 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 âM-^\¨ Imported #17354785 (0xbdffâM-^@¦01bc)    
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 Advanced session window for approvals update=Advanced { prev_window_start: 29374, prev_window_end: 29379, new_window_start: 29375, new_window_end: 29380 }
Apr  6 00:28:08 ks polkadot[38477]: 2023-04-06 00:28:08 âM-^\¨ Imported #17354786 (0xc1dbâM-^@¦d79b)    
Apr  6 00:28:10 ks polkadot[38477]: 2023-04-06 00:28:10 âM-^\¨ Imported #17354787 (0x9ea8âM-^@¦b9c0)    
Apr  6 00:28:10 ks polkadot[38477]: 2023-04-06 00:28:10 ðM-^_M-^R¤ Idle (40 peers), best: #17354787 (0x9ea8âM-^@¦b9c0), finalized #17354784 (0x69bfâM-^@¦57cc), â¬M-^G 2.5MiB/s â¬M-^F 2.0MiB/s    
Apr  6 00:28:12 ks polkadot[38477]: 2023-04-06 00:28:12 ðM-^_M-^Q´ Applying authority set change scheduled at block #17354785    
Apr  6 00:28:12 ks polkadot[38477]: 2023-04-06 00:28:12 ðM-^_M-^Q´ Applying GRANDPA set change to new set with 1000 authorities    
Apr  6 00:28:12 ks polkadot[38477]: 2023-04-06 00:28:12 ðM-^_M-^Q´ Imported justification for block #17354785 that triggers command Changing authorities, signaling voter.    
Apr  6 00:28:13 ks polkadot[38477]: 2023-04-06 00:28:13 âM-^\¨ Imported #17354788 (0x0c01âM-^@¦f9a0)    
Apr  6 00:28:13 ks polkadot[38477]: 2023-04-06 00:28:13 error=RedundantMessage(PeerId("12D3KooWDKEMrjsEAnnoRtJ4bt3f955LcrbFBaxW7MKAUesVodCe"))
Apr  6 00:28:15 ks polkadot[38477]: 2023-04-06 00:28:15 ðM-^_M-^R¤ Idle (40 peers), best: #17354788 (0x0c01âM-^@¦f9a0), finalized #17354785 (0xbdffâM-^@¦01bc), â¬M-^G 1.2MiB/s â¬M-^F 789.4kiB/s    
Apr  6 00:28:18 ks polkadot[38477]: 2023-04-06 00:28:18 Got a bad assignment from peer hash=0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc peer_id=PeerId("12D3KooWQkGqtb31mViJ2xPw6TLTJrf5u7fPKiJUF45HS4Ge79EQ") error=Unknown block: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:18 ks polkadot[38477]: 2023-04-06 00:28:18 Chain between (0xbdffâM-^@¦01bc, 17354785) and 17354782 not fully known. Forcing vote on 17354782 unknown_number=17354785 unknown_hash=0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:18 ks polkadot[38477]: 2023-04-06 00:28:18 Got a bad assignment from peer hash=0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc peer_id=PeerId("12D3KooWQkGqtb31mViJ2xPw6TLTJrf5u7fPKiJUF45HS4Ge79EQ") error=Unknown block: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:18 ks syslogd: last message repeated 53 times
Apr  6 00:28:18 ks polkadot[38477]: 2023-04-06 00:28:18 Got a bad approval from peer peer_id=PeerId("12D3KooWQkGqtb31mViJ2xPw6TLTJrf5u7fPKiJUF45HS4Ge79EQ") error=Unknown block: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:18 ks syslogd: last message repeated 33 times
Apr  6 00:28:18 ks polkadot[38477]: 2023-04-06 00:28:18 Got a bad assignment from peer hash=0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc peer_id=PeerId("12D3KooWCb2bC1g6BwGNiinmYUpsdVn45XANFoBywJWHarcFMzQu") error=Unknown block: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:18 ks syslogd: last message repeated 47 times
Apr  6 00:28:18 ks polkadot[38477]: 2023-04-06 00:28:18 Got a bad assignment from peer hash=0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc peer_id=PeerId("12D3KooWC9cSD4hyMwSiNiKy4WxwiFgd1NjCEuyD3z4rT6zkFxmn") error=Unknown block: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:18 ks syslogd: last message repeated 53 times
Apr  6 00:28:18 ks polkadot[38477]: 2023-04-06 00:28:18 Got a bad assignment from peer hash=0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc peer_id=PeerId("12D3KooWKmBd7YzvK91Aswxw4BTsAxFY25iHJJqfRyfhuZs3fPYG") error=Unknown block: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:18 ks syslogd: last message repeated 73 times
Apr  6 00:28:18 ks polkadot[38477]: 2023-04-06 00:28:18 Got a bad assignment from peer hash=0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc peer_id=PeerId("12D3KooWB2aEhTNda47aVYNyShcuANLo9VufHVR3xU3CBak79kP5") error=Unknown block: 0xbdff57f10ef41eee5733fdabc38eedea25d466c5d8cabfc1d47e1bd0eec801bc
Apr  6 00:28:18 ks syslogd: last message repeated 35 times
Apr  6 00:28:18 ks polkadot[38477]: 2023-04-06 00:28:18 New dispute initiated for candidate. candidate_hash=0x84a0ace3aa62cc6465fd218d7f861bf6859380b814003cb9b2b6d908e6581cc9 session=29380 traceID=176292369552951078083921839210843479030
Apr  6 00:28:18 ks polkadot[38477]: 2023-04-06 00:28:18 Dispute on candidate concluded with 'valid' result candidate_hash=0x84a0ace3aa62cc6465fd218d7f861bf6859380b814003cb9b2b6d908e6581cc9 session=29380 traceID=176292369552951078083921839210843479030
Apr  6 00:28:18 ks polkadot[38477]: 2023-04-06 00:28:18 âM-^\¨ Imported #17354789 (0xd210âM-^@¦06bc)    
Apr  6 00:28:19 ks polkadot[38477]: 2023-04-06 00:28:19 âM-^\¨ Imported #17354789 (0x0057âM-^@¦9c66)    
Apr  6 00:28:20 ks polkadot[38477]: 2023-04-06 00:28:19 New dispute initiated for candidate. candidate_hash=0x0ed605edf0c78226dac63f91425a6fe074e89928b2bc640e2fdc44e8fe7708f4 session=29380 traceID=19720463732355402691768487457878339552
Apr  6 00:28:20 ks polkadot[38477]: 2023-04-06 00:28:20 ðM-^_M-^R¤ Idle (40 peers), best: #17354789 (0xd210âM-^@¦06bc), finalized #17354786 (0xc1dbâM-^@¦d79b), â¬M-^G 2.3MiB/s â¬M-^F 3.1MiB/s    
Apr  6 00:28:23 ks polkadot[38477]: 2023-04-06 00:28:23 Dispute on candidate concluded with 'valid' result candidate_hash=0x0ed605edf0c78226dac63f91425a6fe074e89928b2bc640e2fdc44e8fe7708f4 session=29380 traceID=19720463732355402691768487457878339552
Apr  6 00:28:25 ks polkadot[38477]: 2023-04-06 00:28:25 âM-^\¨ Imported #17354790 (0xdba9âM-^@¦80e7)    
Apr  6 00:28:25 ks polkadot[38477]: 2023-04-06 00:28:25 ðM-^_M-^R¤ Idle (40 peers), best: #17354790 (0xdba9âM-^@¦80e7), finalized #17354787 (0x9ea8âM-^@¦b9c0), â¬M-^G 1.9MiB/s â¬M-^F 2.5MiB/s    
Apr  6 00:28:30 ks polkadot[38477]: 2023-04-06 00:28:30 âM-^\¨ Imported #17354791 (0x8fd0âM-^@¦b8d1)    
Apr  6 00:28:30 ks polkadot[38477]: 2023-04-06 00:28:30 ðM-^_M-^R¤ Idle (40 peers), best: #17354791 (0x8fd0âM-^@¦b8d1), finalized #17354788 (0x0c01âM-^@¦f9a0), â¬M-^G 1.4MiB/s â¬M-^F 1010.1kiB/s    
Apr  6 00:28:31 ks polkadot[38477]: 2023-04-06 00:28:31 New dispute initiated for candidate. candidate_hash=0x0eed375f399d4fbe7ef1a48b0183030572bde2e6397e12725c1bded9800da86f session=29380 traceID=19840889373496039560077374599073039109
Apr  6 00:28:35 ks polkadot[38477]: 2023-04-06 00:28:35 ðM-^_M-^R¤ Idle (40 peers), best: #17354791 (0x8fd0âM-^@¦b8d1), finalized #17354789 (0xd210âM-^@¦06bc), â¬M-^G 2.4MiB/s â¬M-^F 1.8MiB/s    
Apr  6 00:28:35 ks polkadot[38477]: 2023-04-06 00:28:35 Dispute on candidate concluded with 'valid' result candidate_hash=0x0eed375f399d4fbe7ef1a48b0183030572bde2e6397e12725c1bded9800da86f session=29380 traceID=19840889373496039560077374599073039109
Apr  6 00:28:36 ks polkadot[38477]: 2023-04-06 00:28:36 âM-^\¨ Imported #17354792 (0x9bcdâM-^@¦a0fa)    
Apr  6 00:28:40 ks polkadot[38477]: 2023-04-06 00:28:40 ðM-^_M-^R¤ Idle (40 peers), best: #17354792 (0x9bcdâM-^@¦a0fa), finalized #17354789 (0xd210âM-^@¦06bc), â¬M-^G 968.1kiB/s â¬M-^F 1003.2kiB/s    
Apr  6 00:28:43 ks polkadot[38477]: 2023-04-06 00:28:42 âM-^\¨ Imported #17354793 (0x6afdâM-^@¦1d83)    
Apr  6 00:28:45 ks polkadot[38477]: 2023-04-06 00:28:45 ðM-^_M-^R¤ Idle (40 peers), best: #17354793 (0x6afdâM-^@¦1d83), finalized #17354791 (0x8fd0âM-^@¦b8d1), â¬M-^G 2.3MiB/s â¬M-^F 1.8MiB/s
@bkchr
Copy link
Member

bkchr commented Apr 6, 2023

@tdimitrov you are working on this or?

@tdimitrov
Copy link
Contributor

Yes, this is the fix: paritytech/polkadot#7013
It seems to solve the problem on our test network.

I've described details about the problem in paritytech/polkadot#7009

@tdimitrov
Copy link
Contributor

tdimitrov commented Apr 6, 2023 •
edited
Loading

Just noticed that this happens during sync. On our test network it occurred when scraping dispute votes from the runtime.

@anignatev could you please confirm if this happens during sync or while the node is operating?

@anignatev
Copy link
Author

@tdimitrov As you can see from the log above, the node was in the operating status.

@ciscohite
Copy link

Our nodes also keep falling out of sync and it happens atleast twice a day. We had same issue with v0.9.40 but downgrading to v0.9.39-1 resolved it. Now its back with v0.9.41 upgrade. Below are some logs - ubuntu@kusama-1:$ journalctl -u ksm -f -n 1000 | grep assignment
Apr 07 04:52:05 kusama-1 polkadot[182337]: 2023-04-07 04:52:05 Got a bad assignment from peer hash=0xe64a425c5f7245cd9f19b9777f746a4253857407899cf937239fea4b3e9cd458 peer_id=PeerId("12D3KooWFnjzhEzBKhyvgocs2iYfMPjpjsMeuUaZMwuSvsY1Egu8") error=Invalid cert: ValidatorIndex(110), reason: VRFDelayOutputMismatch
Apr 07 04:52:05 kusama-1 polkadot[182337]: 2023-04-07 04:52:05 Got a bad assignment from peer hash=0x9e11cdd3cc62ef11157421f08c6e96e046aac8a58d8f034928005362c0c53ad5 peer_id=PeerId("12D3KooWFnjzhEzBKhyvgocs2iYfMPjpjsMeuUaZMwuSvsY1Egu8") error=Invalid cert: ValidatorIndex(110), reason: VRFModuloOutputMismatch
Apr 07 04:52:05 kusama-1 polkadot[182337]: 2023-04-07 04:52:05 Got a bad assignment from peer hash=0xfdc2abcaff0d9d8abe5d6e600dfcaa49fee8125fd3a05696d946955c73eb7d39 peer_id=PeerId("12D3KooWFnjzhEzBKhyvgocs2iYfMPjpjsMeuUaZMwuSvsY1Egu8") error=Invalid cert: ValidatorIndex(110), reason: VRFModuloOutputMismatch
^C
ubuntu@kusama-1:$ journalctl -u ksm -f -n 1000 | grep RuntimeRequest
^C
ubuntu@kusama-1:~$ journalctl -u ksm -f -n 1000 | grep dispute
Apr 07 04:58:16 kusama-1 polkadot[182337]: 2023-04-07 04:58:15 New dispute initiated for candidate. candidate_hash=0x853f3715c146b7fb95f1f159435fea0a539aa959ed158364efe7c32b081df297 session=29411 traceID=177115555397617219415124599148277393930
Apr 07 04:58:16 kusama-1 polkadot[182337]: 2023-04-07 04:58:16 New dispute initiated for candidate. candidate_hash=0x27eca57e6b30889dece5a1af4df0c5c7789fe256a75285d36ff3f13f28ad91c3 session=29411 traceID=53068630507732587331784302702632158663
Apr 07 04:58:16 kusama-1 polkadot[182337]: 2023-04-07 04:58:16 New dispute initiated for candidate. candidate_hash=0xb6ecb1c042046773efd508e67f5babd9996cdf4dce900576bbcc6156f31939ff session=29411 traceID=243148482510207292167136204762400926681
Apr 07 04:58:17 kusama-1 polkadot[182337]: 2023-04-07 04:58:17 New dispute initiated for candidate. candidate_hash=0x9579497917d9600f04873440a7c9e6f5b76bb607c8656ab3a6d194f9dda4f227 session=29411 traceID=198684729501724855756602353659239065333
Apr 07 04:58:17 kusama-1 polkadot[182337]: 2023-04-07 04:58:17 New dispute initiated for candidate. candidate_hash=0x4d4e581628e248bee3e0e245f4ccf892776eb130571d0a835c8d821c743564d7 session=29411 traceID=102757341438121896589206479364571330706
Apr 07 04:58:17 kusama-1 polkadot[182337]: 2023-04-07 04:58:17 New dispute initiated for candidate. candidate_hash=0xbc5f1fbb66131d7aaa7494a2a4ab2593a182ffad4b3b1b004500b18ab179a34e session=29411 traceID=250388775011079675903603657528027325843
Apr 07 04:58:17 kusama-1 polkadot[182337]: 2023-04-07 04:58:17 New dispute initiated for candidate. candidate_hash=0x459ce38b436451973ce3f74fd83d30b18152070a4d0d0f7676bf7b96789d0686 session=29411 traceID=92531345159642020293865278593080045745
Apr 07 04:58:18 kusama-1 polkadot[182337]: 2023-04-07 04:58:18 New dispute initiated for candidate. candidate_hash=0x07661c20e5579e227c118f91d6ed1ecd0b249383b71fcffa8511e3c5127b362b session=29411 traceID=9834780763813056797095603542074138317
Apr 07 04:58:19 kusama-1 polkadot[182337]: 2023-04-07 04:58:19 New dispute initiated for candidate. candidate_hash=0x6919b40f0fe25a111b60740140be87dc8c9d1cf12e981f227a336c495540d877 session=29411 traceID=139702399005946550071358272077062637532
Apr 07 04:58:19 kusama-1 polkadot[182337]: 2023-04-07 04:58:19 New dispute initiated for candidate. candidate_hash=0x9e0a7dc95436ba0c70d0ed0de896b624f1eb476cba82cd03c8d0adecdb099746 session=29411 traceID=210072497554726079429929846554567816740
Apr 07 04:58:20 kusama-1 polkadot[182337]: 2023-04-07 04:58:20 New dispute initiated for candidate. candidate_hash=0x5c616991f8ea2f9abc48f86c3c0646d6ba324c71e8b10f62c27a6d8d78b0369a session=29411 traceID=122794769625617482463915222731407640278
Apr 07 04:58:20 kusama-1 polkadot[182337]: 2023-04-07 04:58:20 New dispute initiated for candidate. candidate_hash=0x6558e858135c7349259a3c41ae7c64fba86acfa110f1f8e5324f2970a56cbffe session=29411 traceID=134713662194925897220872754717907510523
Apr 07 04:58:21 kusama-1 polkadot[182337]: 2023-04-07 04:58:21 New dispute initiated for candidate. candidate_hash=0xfcb8cc7c82c831d5860b12b38744bef06eb1a40cf2690122a46bf3c02f353113 session=29411 traceID=335924985036095576740424104823952293616
Apr 07 04:58:21 kusama-1 polkadot[182337]: 2023-04-07 04:58:21 New dispute initiated for candidate. candidate_hash=0x80747852f1b7c617247ab46c7150fea736a4e5ffdf94cbb5f96a4310a8d176ee session=29411 traceID=170745930356729092354124369927922056871
Apr 07 04:58:22 kusama-1 polkadot[182337]: 2023-04-07 04:58:22 New dispute initiated for candidate. candidate_hash=0xf4dab64c2ccb2449d6d7641bede2c04999a290799e918b8a807da57b249815e6 session=29411 traceID=325467249120431204840462992166856343625
Apr 07 04:58:22 kusama-1 polkadot[182337]: 2023-04-07 04:58:22 New dispute initiated for candidate. candidate_hash=0x1c2ca7a08677da52b4db6ea7a5bdd2227aa94c6e6e73f4be52efe097c5fb8d95 session=29411 traceID=37450244824278873458369545005128602146
Apr 07 04:58:23 kusama-1 polkadot[182337]: 2023-04-07 04:58:22 New dispute initiated for candidate. candidate_hash=0x565766d40e02dc39aa8b836fa33f0fd087226da3da37c969da76c956229101f2 session=29411 traceID=114767423070681568541350137083302186960
Apr 07 04:58:23 kusama-1 polkadot[182337]: 2023-04-07 04:58:23 New dispute initiated for candidate. candidate_hash=0xdfc04192975f6080735cc961068c5a4091c54930e96dd959612fd4693d434d62 session=29411 traceID=297416094027658431155962507467057551936
Apr 07 04:58:23 kusama-1 polkadot[182337]: 2023-04-07 04:58:23 New dispute initiated for candidate. candidate_hash=0xe9f7d45df05721e902126cbfe6f9abfaf4cf00820e8c557140785ecde8156583 session=29411 traceID=310996927655380327831311245069388459002
Apr 07 04:58:24 kusama-1 polkadot[182337]: 2023-04-07 04:58:24 New dispute initiated for candidate. candidate_hash=0x6c2599ba956a496d6c7e1eaa4e238ae200f090078a907076ffa3de170b6a8d05 session=29411 traceID=143751856519886048606134086373998824162
Apr 07 04:58:25 kusama-1 polkadot[182337]: 2023-04-07 04:58:25 New dispute initiated for candidate. candidate_hash=0x6c51f3f24bcfea2bfb4767b9f78b082f82e5f78de66580c0a4eaafc9184514d5 session=29411 traceID=143982147412523980345739499467942070319
Apr 07 04:58:29 kusama-1 polkadot[182337]: 2023-04-07 04:58:29 New dispute initiated for candidate. candidate_hash=0x02f581ed2d533bb5d082c8efd030077ae0dc0c2b188af1d6a0f9b91a90cf291d session=29411 traceID=3933203943851699719426785790340892538
Apr 07 04:59:13 kusama-1 polkadot[182337]: 2023-04-07 04:59:13 New dispute initiated for candidate. candidate_hash=0x9a81ae4e2f11f9094b88dcfb4a38a65e09a7a841e77b7b0ea127cd6351a2bc48 session=29411 traceID=205374452979263271935793261680390612574
Apr 07 04:59:49 kusama-1 polkadot[182337]: 2023-04-07 04:59:49 New dispute initiated for candidate. candidate_hash=0x23893e5a78b2f4b9bceb1db00d1cf9f0cce85972b680b52580c36dbd4b23155b session=29411 traceID=47235589199375925282233886265850001904
^C

@sandreim sandreim mentioned this issue Apr 12, 2023
Closed
@anignatev
Copy link
Author

Unfortunately, the error is hapend on version 0.9.39-1:

Configuration and hardware is the same as in my first post, logs:

Apr 19 16:36:03 ksm polkadot[4012]: 2023-04-19 16:36:03 This chain is not in any way
Apr 19 16:36:03 ksm polkadot[4012]: 2023-04-19 16:36:03       endorsed by the
Apr 19 16:36:03 ksm polkadot[4012]: 2023-04-19 16:36:03      KUSAMA FOUNDATION
Apr 19 16:36:03 ksm polkadot[4012]: 2023-04-19 16:36:03 ----------------------------
Apr 19 16:36:03 ksm polkadot[4012]: 2023-04-19 16:36:03 Parity Polkadot
Apr 19 16:36:03 ksm polkadot[4012]: 2023-04-19 16:36:03 .M-^\M-^L..M-^O  version 0.9.39-1-298b4aefe1b
Apr 19 16:36:03 ksm polkadot[4012]: 2023-04-19 16:36:03 .M-^]...M-^O  by Parity Technologies <admin@parity.io>, 2017-2023
Apr 19 16:36:03 ksm polkadot[4012]: 2023-04-19 16:36:03 .M-^_M-^SM-^K Chain specification: Kusama
Apr 19 16:36:03 ksm polkadot[4012]: 2023-04-19 16:36:03 .M-^_M-^O.  Node name: infsysgroup
Apr 19 16:36:03 ksm polkadot[4012]: 2023-04-19 16:36:03 .M-^_M-^Q. Role: AUTHORITY
Apr 19 16:36:03 ksm polkadot[4012]: 2023-04-19 16:36:03 .M-^_M-^R. Database: ParityDb at /mnt/ksm_base/chains/ksmcc3/paritydb/full
Apr 19 16:36:03 ksm polkadot[4012]: 2023-04-19 16:36:03 .M-^[M-^S  Native runtime: kusama-9390 (parity-kusama-0.tx20.au2)
Apr 19 16:36:56 ksm polkadot[4012]: 2023-04-19 16:36:56 .M-^_M-^O.  Local node identity is: 12D3KooWJryJLfeM1eJGhGgLggxQebUaBKRTgmTooANwteC2YpZQ
Apr 19 16:36:56 ksm polkadot[4012]: 2023-04-19 16:36:56 Can't listen on /ip6/::/tcp/30333 because: Other(Custom { kind: Other, error: Other(A(A(A(A(B(Transport(Os { code: 43, kind: Uncategorized, message:
 "Protocol not supported" }))))))) })
Apr 19 16:37:08 ksm polkadot[4012]: 2023-04-19 16:37:08 .M-^_M-^R. Operating system: freebsd
Apr 19 16:37:08 ksm polkadot[4012]: 2023-04-19 16:37:08 .M-^_M-^R. CPU architecture: x86_64
Apr 19 16:37:08 ksm polkadot[4012]: 2023-04-19 16:37:08 .M-^_M-^S. Highest known block at #17549644
Apr 19 16:37:08 ksm polkadot[4012]: 2023-04-19 16:37:08 .M-^@...M-^O Prometheus exporter started at 10.55.0.49:9615
Apr 19 16:37:08 ksm polkadot[4012]: 2023-04-19 16:37:08 Running JSON-RPC HTTP server: addr=10.55.0.49:9933, allowed origins=["http://localhost:*", "http://127.0.0.1:*", "https://localhost:*", "https://127
.0.0.1:*", "https://polkadot.js.org"]
Apr 19 16:37:08 ksm polkadot[4012]: 2023-04-19 16:37:08 Running JSON-RPC WS server: addr=10.55.0.49:9944, allowed origins=["http://localhost:*", "http://127.0.0.1:*", "https://localhost:*", "https://127.0
.0.1:*", "https://polkadot.js.org"]
Apr 19 16:37:08 ksm polkadot[4012]: 2023-04-19 16:37:08 .M-^_M-^OM-^A CPU score: 775.09 MiBs
Apr 19 16:37:08 ksm polkadot[4012]: 2023-04-19 16:37:08 .M-^_M-^OM-^A Memory score: 4.63 GiBs
Apr 19 16:37:08 ksm polkadot[4012]: 2023-04-19 16:37:08 .M-^_M-^OM-^A Disk score (seq. writes): 3.92 GiBs
Apr 19 16:37:08 ksm polkadot[4012]: 2023-04-19 16:37:08 .M-^_M-^OM-^A Disk score (rand. writes): 637.32 MiBs
Apr 19 16:37:08 ksm polkadot[4012]: 2023-04-19 16:37:08 .M-^Z...M-^O  The hardware does not meet the minimal requirements for role 'Authority' find out more at:
Apr 19 16:37:08 ksm polkadot[4012]: https://wiki.polkadot.network/docs/maintain-guides-how-to-validate-polkadot#reference-hardware
Apr 19 16:37:09 ksm polkadot[4012]: 2023-04-19 16:37:09 .M-^_M-^Q. Starting BABE Authorship worker
Apr 19 16:37:10 ksm polkadot[4012]: 2023-04-19 16:37:09 dropping instance 10.55.0.49
Apr 19 16:37:10 ksm polkadot[4012]: 2023-04-19 16:37:10 .M-^_M-^TM-^M Discovered new external address for our node: /ip4/62.77.130.13/tcp/30333/p2p/12D3KooWJryJLfeM1eJGhGgLggxQebUaBKRTgmTooANwteC2YpZQ
Apr 19 16:37:11 ksm polkadot[4012]: 2023-04-19 16:37:11 .M-^_M-^RM-^T The bootnode you want to connect to at `/dns/kusama-bootnode.polkadotters.com/tcp/30333/p2p/12D3KooWLxZmPqzC1itd2hCDLX2Ai8x8ArHbSrMF7w
dbkg2CaEej` provided a different peer ID `12D3KooWHB5rTeNkQdXNJ9ynvGz8Lpnmsctt7Tvp7mrYv6bcwbPG` than the one you expect `12D3KooWLxZmPqzC1itd2hCDLX2Ai8x8ArHbSrMF7wdbkg2CaEej`.
Apr 19 16:37:11 ksm syslogd: last message repeated 2 times
Apr 19 16:37:11 ksm polkadot[4012]: 2023-04-19 16:37:11 Could not retrieve session info from rolling session window session=29706
Apr 19 16:37:11 ksm syslogd: last message repeated 1 times
Apr 19 16:37:12 ksm polkadot[4012]: 2023-04-19 16:37:12 .M-^_M-^RM-^T The bootnode you want to connect to at `/dns/kusama-bootnode.polkadotters.com/tcp/30333/p2p/12D3KooWLxZmPqzC1itd2hCDLX2Ai8x8ArHbSrMF7w
dbkg2CaEej` provided a different peer ID `12D3KooWHB5rTeNkQdXNJ9ynvGz8Lpnmsctt7Tvp7mrYv6bcwbPG` than the one you expect `12D3KooWLxZmPqzC1itd2hCDLX2Ai8x8ArHbSrMF7wdbkg2CaEej`.
Apr 19 16:37:12 ksm syslogd: last message repeated 2 times
Apr 19 16:37:13 ksm polkadot[4012]: 2023-04-19 16:37:13 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0xe4743cff3de1f81293e14fd542d5968610b66
ce116ec539a275925e5473dfca0
Apr 19 16:37:13 ksm polkadot[4012]: 2023-04-19 16:37:13 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr 19 16:37:13 ksm polkadot[4012]: 2023-04-19 16:37:13 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0x793b46f8d54c36fd9137ddef3633c0c5f3c41
9ed9dfa0e4b7a8abc789b08fd72
Apr 19 16:37:13 ksm polkadot[4012]: 2023-04-19 16:37:13 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr 19 16:37:13 ksm polkadot[4012]: 2023-04-19 16:37:13 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0x793b46f8d54c36fd9137ddef3633c0c5f3c41
9ed9dfa0e4b7a8abc789b08fd72
Apr 19 16:37:13 ksm polkadot[4012]: 2023-04-19 16:37:13 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr 19 16:37:13 ksm polkadot[4012]: 2023-04-19 16:37:13 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0x793b46f8d54c36fd9137ddef3633c0c5f3c41
9ed9dfa0e4b7a8abc789b08fd72
Apr 19 16:37:13 ksm polkadot[4012]: 2023-04-19 16:37:13 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr 19 16:37:13 ksm polkadot[4012]: 2023-04-19 16:37:13 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0x793b46f8d54c36fd9137ddef3633c0c5f3c41
9ed9dfa0e4b7a8abc789b08fd72
Apr 19 16:37:13 ksm polkadot[4012]: 2023-04-19 16:37:13 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr 19 16:37:13 ksm polkadot[4012]: 2023-04-19 16:37:13 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0x793b46f8d54c36fd9137ddef3633c0c5f3c41
9ed9dfa0e4b7a8abc789b08fd72
Apr 19 16:37:13 ksm polkadot[4012]: 2023-04-19 16:37:13 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr 19 16:37:13 ksm polkadot[4012]: 2023-04-19 16:37:13 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0x793b46f8d54c36fd9137ddef3633c0c5f3c41
9ed9dfa0e4b7a8abc789b08fd72
Apr 19 16:37:13 ksm polkadot[4012]: 2023-04-19 16:37:13 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))
Apr 19 16:37:13 ksm polkadot[4012]: 2023-04-19 16:37:13 cannot query the runtime API version: Api called for an unknown Block: Header was not found in the database: 0x793b46f8d54c36fd9137ddef3633c0c5f3c41
9ed9dfa0e4b7a8abc789b08fd72
Apr 19 16:37:13 ksm polkadot[4012]: 2023-04-19 16:37:13 error=Runtime(RuntimeRequest(NotSupported { runtime_api_name: "session_info" }))

@tdimitrov
Copy link
Contributor

This problem is addressed by paritytech/polkadot#6993 and paritytech/polkadot#7013

They will be included in the next release.

@deep-ink-ventures deep-ink-ventures mentioned this issue Jun 8, 2023
Merged
6 tasks
@slumber
Copy link
Contributor

slumber commented Jun 16, 2023

Still happens on Kusama as of 0.9.43 @tdimitrov

@eskimor
Copy link
Member

eskimor commented Jun 16, 2023 •
edited
Loading

Seems to happen only in dispute-distribution. What could be the reason is that we are out of sync and just cannot find the block. I would expect a not found error or something not a "not supported" in that case.

We are using the relay parent of the disputed block to fetch the session info. If session info was not yet cached (session boundary) this would fail, if we had that block not yet imported.

If this is the reason the warning is harmless as the sender will just retry until we have seen the block in question or have fetched the session info by other means. We should then make the error correct (block is missing) and demote to debug.

Session change around the time of the incident just confirmed by @slumber .

@Sophia-Gold Sophia-Gold transferred this issue from paritytech/polkadot Aug 24, 2023
claravanstaden added a commit to Snowfork/polkadot-sdk that referenced this issue Dec 8, 2023
@claravanstaden
Tests Refactor (paritytech#670)
c4a0974 
* Move tests to one file.

* Progress on test refactor

* Fixed tests

* Finish test cleanup.

* Tests refactor
helin6 pushed a commit to boolnetwork/polkadot-sdk that referenced this issue Feb 5, 2024
@dependabot
Bump actions/setup-node from 1 to 3 (paritytech#670)
e8069d5 
Bumps [actions/setup-node](/~https://github.com/actions/setup-node) from 1 to 3.
- [Release notes](/~https://github.com/actions/setup-node/releases)
- [Commits](actions/setup-node@v1...v3)

---
updated-dependencies:
- dependency-name: actions/setup-node
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tdimitrov tdimitrov

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@tdimitrov @eskimor @ciscohite @bkchr @anignatev @slumber