Latest subquery changes #42

Douglasacost · 2023-11-18T19:46:50Z

Added compositeIndexes
Improved the uniques creation flow
Removed Allocation events from mapping
Started indexing sponsorship calls

docker-compose.yml

mappingAllocationHandler files

Sponsorship

aliXsed · 2023-11-19T23:33:16Z

package.json

    "@polkadot/api": "^10",
+    "@subql/cli": "^4.2.1",


Was there and issue to use "latest" here?

On another note, I would say using an actual version could be a better choice than "latest". So you may want to use the version uniformly for all of "subql" dependencies.

Yes, had some issues with the package manager Yarn

Indeed, I already had issues related to it, that time only with @subql/cli

docker-compose.yml

project.yaml

aliXsed · 2023-11-19T23:41:24Z

project.yaml

-            method: batch
+            module: sponsorship
+            method: createPot
+        - handler: handleSponsorshipRegisterUsersCall


Usually the EventHandlers are a better choice for indexing compared with the CallHandlers. The reason is when the call is nested for example inside a Utility batch, the subquery used to struggle to trace it. Is it still the case?

True @aliXsed, we had issues with the batch in the past since it handle tons of data (e.x allocations)

Will update this on testnet, for use events instead of calls

Will prepare a new PR when the sponsorship events got updated on mainnet, meanwhile pot indexing is desactivted

schema.graphql

aliXsed · 2023-11-20T01:46:44Z

schema.graphql

@@ -24,7 +24,7 @@ type Item @entity {
  owner: String
 }

-type BalanceTransfer @entity {
+type BalanceTransfer @entity @compositeIndexes(fields: [["from", "to", "amount"], ["from", "to", "timestamp"], ["from", "to"]]) {


This composite indexing doesn't seem to be able to accelerate queries such as from not equal to allocations to dao or company reserve. Correct me if I'm wrong. I'm not against keeping it if it's making our other usual queries faster. However to satisfy https://linear.app/nodle/issue/CHA-137 you may need to create a new entity such as

type TransferToTreasury @entity { id : ID! from: String! @index to: String! @index # This field is still needed because we have a number of treasury accounts and not just one timestamp: Float blockNumber: Int }

The other option is you add another field to BalanceTransfer such as nonRewardToTreasury: Boolean @index to make it clear which entities are "transfers to one of our treasuries but not coming from the allocations".

@aliXsed could you give me all the treasury accounts, or it is a constant in the chain?

Company Reserve: 4jbtsgNhpGB2vH7xTjpZVzZLy7W4sFyxjvD45x7X1m6BSiGx
International Reserve: 4jbtsgNhpGB2voF5dZzKQ2tphWLjV48HSkfQwmWqNn3qa4rv
USA Reserve: 4jbtsgNhpGB2voKv8rRSJYTAohbnHE5oVZ4DejZBEQVT5o86
DAO Reserve: 4jbtsgNhpGB2NtbdLN3xkB2urHo5JboKkSAjcBM4s3SzQdUt
Allocations: 4jbtsgNhpGAzdEGrKRb7g8Mq4ToNUpBVxeye942tWfG3gcYi

schema.graphql

aliXsed · 2023-11-20T01:53:44Z

schema.graphql

+  feeQuota: BigInt!
+  reserveQuota: BigInt!


Quota should be able to show two fields: limit and balance.

Currently is showing the available feeQuota & reserveQuota

available = limit - balance

we'd want to know both the used quota and the available quota about the users. So perhaps it's worth adding both.

aliXsed · 2023-11-20T01:54:15Z

schema.graphql

+  feeQuota: BigInt!
+  reserveQuota: BigInt!


Quota should be able to show two fields: limit and balance.

Currently is showing the available feeQuota & reserveQuota

available = limit - balance

I imagine we'd want to know both the used quota and the available quota about the users. So perhaps it's worth adding both.

Update to registry Limit and Balance for feeQuota and reserveQuota

schema.graphql

aliXsed · 2023-11-21T04:14:36Z

@Douglasacost why do we have npm package lock instead of yarn for this project?

Douglasacost · 2023-11-21T18:11:21Z

@Douglasacost why do we have npm package lock instead of yarn for this project?

Subquery team recommend use NPM, that's why I started using it

mappingBalancesHandlers.ts

Douglasacost · 2023-11-26T23:57:31Z

@aliXsed let me know to deploy a Staged node

aliXsed · 2024-01-22T23:04:27Z

src/mappings/mappingBalancesHandlers.ts

+    '4jbtsgNhpGAzdEGrKRb7g8Mq4ToNUpBVxeye942tWfG3gcYi',
+    '4jByf7kvkZ7hGYwGMYhjFYHoLec3zNZ3EKD86PARZDcfnnkD',
+    '1qnJN7FViy3HZaxZK9tGAA71zxHSBeUweirKqCaox4t8GT7',
+    '4hyBs59AiVKw4jZ851hmEVzpvxFATxwjvTDi67ziwb4vYDCX',
+    '4j5pigNy7LAX1dSGZQ7Tc2oms9dzZSEtwQJukSgQ9gAsQ9Fx',


Can you remind where these addresses come from? you can also put a comment in front of each to remain here.

aliXsed · 2024-01-22T23:09:28Z

src/mappings/mappingBalancesHandlers.ts

+    '4j5pigNy7LAX1dSGZQ7Tc2oms9dzZSEtwQJukSgQ9gAsQ9Fx',
+]
+
+const ALLOCATION_ACCOUNT = '4jbtsgNhpGAzdEGrKRb7g8Mq4ToNUpBVxeye942tWfG3gcYi'


This account has become part of both Rewards and Treasury accounts. What is the logic for that?

aliXsed · 2024-01-22T23:20:23Z

src/mappings/mappingBalancesHandlers.ts

+    if (isReward) {
+        result.push(new Rewards(
+            `${event.block.block.header.number.toNumber()}-${event.idx}`,
+            '',
+            ''
+        ))
    }
-
-    const amount = event.event.data[2];
-    let record = new BalanceTransfer(`${event.block.block.header.number.toNumber()}-${event.idx}`, '', '');
-    record.blockNumber = event.block.block.header.number.toBigInt();
-    record.from = from.toString();
-    record.to = to.toString();
-    record.amount =  (amount as Balance).toBigInt();
-    if (event.extrinsic) {
-        record.txHash = event.extrinsic.extrinsic.hash.toString();
-        record.timestamp = event.extrinsic.block.timestamp.getTime();
-        record.success = checkIfExtrinsicExecuteSuccess(event.extrinsic)
+
+    if (isTreasury) {
+        const entity = new TransferToTreasury(
+            `${event.block.block.header.number.toNumber()}-${event.idx}`,
+            '',
+            ''
+        )
+
+        entity.isAllocation = from.toString() === ALLOCATION_ACCOUNT
+
+        result.push(entity)
+    }
+
+    if (!isReward && !isTreasury) {


I suggest a switch-case or a complete if-else structure for the logic here to make sure we actually separate:

Treasury Commission from Rewards

Rewards

Refund to Treasury

User to User Transfers

into their own dedicated tables with no overlap. As a result, the return from this function should also change from an array of entities to a single entity

…nto feat/composed-index

… entity

Douglasacost added 11 commits November 8, 2023 09:51

update docker-compose

cc71349

use npm

48c642e

Add .prettierrc configuration file

78f2a8a

Update log level and disable historical in

455dbbb

docker-compose.yml

Update dependencies and remove unused code

54af4ad

Refactor collection and item creation logic

64f8390

Delete allocation handler and

9ed9c27

mappingAllocationHandler files

Fix checkIfExtrinsicExecuteSuccess function call

ea76cfd

Fix bug in login functionality

202f1b2

Update mappings and entities for Pot and

06e0985

Sponsorship

Add isReward field to BalanceTransfer model

c33ad74

Douglasacost requested a review from aliXsed November 18, 2023 22:44

aliXsed reviewed Nov 19, 2023

View reviewed changes

docker-compose.yml Show resolved Hide resolved

aliXsed reviewed Nov 19, 2023

View reviewed changes

project.yaml Show resolved Hide resolved

aliXsed reviewed Nov 19, 2023

View reviewed changes

project.yaml Show resolved Hide resolved

aliXsed reviewed Nov 19, 2023

View reviewed changes