From c65ac373414c3b0f6b700e872f25589071343f26 Mon Sep 17 00:00:00 2001 From: dzhelezov Date: Mon, 3 May 2021 14:33:58 +0300 Subject: [PATCH] chore(docs): [skip-ci] docs for queries, pagination, sorting (#4) (#374) * docs(docs): pagination, queries, sorting * docs(docs): update announcing-hydra-v3.md Co-authored-by: Metin Demir --- README.md | 14 +- SUMMARY.md | 13 +- announcing-hydra-v3.md | 16 +- docs/entity-relationship.md | 140 ---------------- docs/graphql-entity-relationships.md | 2 + docs/mappings/README.md | 2 +- docs/paginate-query-results.md | 183 ++++++++++++++++++++ docs/queries.md | 242 +++++++++++++++++++++++++++ docs/quick-start.md | 10 +- docs/sort-query-results.md | 37 ++++ 10 files changed, 495 insertions(+), 164 deletions(-) delete mode 100644 docs/entity-relationship.md create mode 100644 docs/graphql-entity-relationships.md create mode 100644 docs/paginate-query-results.md create mode 100644 docs/queries.md create mode 100644 docs/sort-query-results.md diff --git a/README.md b/README.md index 5824995fb..6f225909e 100644 --- a/README.md +++ b/README.md @@ -34,11 +34,11 @@ and answer the prompts. It will generate a sample project and README with setup The monorepo contains the following sub-packages: -* [Hydra CLI](packages/hydra-cli/README.md): Codegen tools to set up and run a Hydra pipeline -* [Hydra Indexer](packages/hydra-indexer/README.md): Hydra indexer for ingesting raw events and extrinsics -* [Hydra Indexer Gateway](packages/hydra-indexer-gateway/README.md): GraphQL interface for the Indexer -* [Hydra Processor](packages/hydra-processor/README.md): Processing part of the pipeline for transforming events into rich business-level objects -* [Hydra Typegen](packages/hydra-typegen/README.md): A tool for generating typesafe typescript classes for events and extrinsics from the runtime metadata. No more manual deserialization of the event data. -* [Sample Project](packages/sample/README.md): A quickstart Hydra project set up against a publicly available Kusama indexer. Good starting point. -* [Docs](https://app.gitbook.com/@dzhelezov/s/hydra/): In-depth documentation covering the Hydra pipeline and API features, such as full-text search, pagination, extensive filtering and a rich GraphQL dialect defining your schema! +* [Hydra CLI](./packages/hydra-cli/README.md): Codegen tools to set up and run a Hydra pipeline +* [Hydra Indexer](./packages/hydra-indexer/README.md): Hydra indexer for ingesting raw events and extrinsics +* [Hydra Indexer Gateway](./packages/hydra-indexer-gateway/README.md): GraphQL interface for the Indexer +* [Hydra Processor](./packages/hydra-processor/README.md): Processing part of the pipeline for transforming events into rich business-level objects +* [Hydra Typegen](./packages/hydra-typegen/README.md): A tool for generating typesafe typescript classes for events and extrinsics from the runtime metadata. No more manual deserialization of the event data. +* [Sample Project](./packages/sample/README.md): A quickstart Hydra project set up against a publicly available Kusama indexer. Good starting point. +* [Docs](./docs/README.md): In-depth documentation covering the Hydra pipeline and API features, such as full-text search, pagination, extensive filtering and a rich GraphQL dialect defining your schema! diff --git a/SUMMARY.md b/SUMMARY.md index 463cb8927..5dd8383f8 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -1,8 +1,16 @@ # Table of contents * [Hydra](README.md) +* [Hydra CLI](packages/hydra-cli/README.md) +* [Hydra Indexer](packages/hydra-indexer/README.md) +* [Hydra Indexer Gateway](packages/hydra-indexer-gateway/README.md) +* [Hydra Processor](packages/hydra-processor/README.md) +* [Hydra Typegen](packages/hydra-typegen/README.md) * [Overview](docs/README.md) * [Query Node Manifest](docs/manifest-spec.md) + * [Query Node Queries](docs/queries.md) + * [Pagination](docs/paginate-query-results.md) + * [Sorting](docs/sort-query-results.md) * [Mappings](docs/mappings/README.md) * [DatabaseManager](docs/mappings/databasemanager.md) * [SubstrateEvent](docs/mappings/substrateevent.md) @@ -15,10 +23,9 @@ * [Full-text queries](docs/schema-spec/full-text-queries.md) * [Entity Relationships](docs/schema-spec/entity-relationship.md) * [Install Hydra](docs/install-hydra.md) - * [Query Node schema](docs/query-node-schema.md) * [Tutorial](docs/quick-start.md) - * [GraphQL Entity Relationships](docs/entity-relationship.md) + * [GraphQL Entity Relationships](docs/graphql-entity-relationships.md) * [Architecture](docs/architecture.md) * [Migration to Hydra v2](migration-to-hydra-v2.md) -* [Announcing Hydra v3](announcing-hydra-v3.md) +* [What's new in Hydra v3](announcing-hydra-v3.md) diff --git a/announcing-hydra-v3.md b/announcing-hydra-v3.md index 0a217c533..dce525c5e 100644 --- a/announcing-hydra-v3.md +++ b/announcing-hydra-v3.md @@ -1,15 +1,17 @@ -# Announcing Hydra v3 +# What's new in Hydra v3 Hydra v3 is already in the works. Here are the main features planned: * Support for filtering by relations -* Define block ranges for any mapping, i.e. specify that your mapping X should be run only from block 5 to block 7 -* Import all model files from a single library. Instead of the cumbersome +* Support for adding relations to variant types +* Ordering by multiple fields +* Multiple filters in the same query \(`AND` and `OR`\) +* Filters for mapping handlers: specify heights and runtime spec version range +* Import all model files from a single library. Instead of the cumbersome ```typescript -import { MyEntity1 } from '../generated/graphql-server/my-entity2/my-entity2.model' -import { MyEntity2 } from '../generated/graphql-server/my-entity2/my-entity2.model' - +import { MyEntity1 } from '../generated/graphql-server/my-entity2/my-entity2.model' +import { MyEntity2 } from '../generated/graphql-server/my-entity2/my-entity2.model' ``` write @@ -18,5 +20,3 @@ write import { MyEntity1, MyEntity2 } from '../generated/graphql-server/model' ``` - - diff --git a/docs/entity-relationship.md b/docs/entity-relationship.md deleted file mode 100644 index ff7fafdab..000000000 --- a/docs/entity-relationship.md +++ /dev/null @@ -1,140 +0,0 @@ -# GraphQL Entity Relationships - -### One-To-One \(1:1\) Relationships - -In One-To-One relation, one entity instance is related to only one instance of another entity. One side of the relationship should always derive. - -```graphql -type User @entity { - name: String! - profile: Profile! @derivedFrom(field: "user") -} - -type Profile @entity { - avatar: String! - user: User! -} -``` - -Database tables: - -```text - user -| Column | Type -----------|------- -| id | character varying -| name | character varying -``` - -```text - profile -| Column | Type -----------|------- -| id | character varying -| avatar | character varying -| userId | character varying FOREIGN KEY UNIQUE CONSTRAINT -``` - -### One-To-Many \(1:n\) Relationships - -In One-To-Many relation, one entity instance is related to multiple instance of the other entity. - -```graphql -type User @entity { - name: String -} - -type Post @entity { - title: String - author: User! -} -``` - -Database table for the `Post` entity: - -```text - post -| Column | Type -----------|------- -| id | character varying -| avatar | character varying -| authorId | character varying FOREIGN KEY -``` - -The only difference between `1:1` and `1:n` is the unique constraint that `1:1` has. - -### Many-To-Many \(n:n\) Relationships - -Many-To-Many is a relationship where one entity instance is related to many instance of other entity and vice-versa. In this relationship one side of the relation must derive. - -```graphql -type User @entity { - name: String - books: [Book!] @derivedFrom(field: "authors") -} - -type Book @entity { - title: String - authors: [User!] -} -``` - -A junction table is created for n:n relationship. - -Database tables: - -```text - book -| Column | Type -----------|------- -| id | character varying -| title | character varying -``` - -```text - book_user -| Column | Type -----------|------- -| book_id | character varying -| user_id | character varying -``` - -### Reverse Lookups - -Defining reverse lookups on an entity allows you to query other side of the relation. Use `@derivedFrom` directive to add reverse lookup to an entity. - -**Example** If we want to access a user's `posts` from the user entity we should add a derived field to `User` entity: - -```graphql -type User @entity { - name: String - posts: [Post!] @derivedField(field: "author") -} - -type Post @entity { - title: String - author: User! -} -``` - -## Relationships In Mappings - -Each GraphQL entity has a corresponding typeorm entity and we use these entities to perform CRUD operations. - -**Example** - -We will create a new post for an existing user: - -```typescript -export async function handleNewPost(db: DB, event: SubstrateEvent) { - const { userId, title } = event.params; - const user = await db.get(User, { where: { id: userId } }); - - const newPost = new Post(); - newPost.title = title; - newPost.author = user; - - db.save(newPost); -} -``` - diff --git a/docs/graphql-entity-relationships.md b/docs/graphql-entity-relationships.md new file mode 100644 index 000000000..b3247f9fd --- /dev/null +++ b/docs/graphql-entity-relationships.md @@ -0,0 +1,2 @@ +# GraphQL Entity Relationships + diff --git a/docs/mappings/README.md b/docs/mappings/README.md index 7b6c7013e..ecae2122f 100644 --- a/docs/mappings/README.md +++ b/docs/mappings/README.md @@ -21,7 +21,7 @@ mappings: handler: timestampCall(DatabaseManager, Timestamp.SetCall) ``` -Each handler may have an arbitrary number of arguments, and at most one argument of type `DatabaseManager` and `SubstrateEvent` +Each handler may have an arbitrary number of arguments, and at most one argument of type `DatabaseManager` and `SubstrateEvent` Let us look at the sample mapping generated by the scaffolder diff --git a/docs/paginate-query-results.md b/docs/paginate-query-results.md new file mode 100644 index 000000000..6eb56087b --- /dev/null +++ b/docs/paginate-query-results.md @@ -0,0 +1,183 @@ +--- +description: Paginate query results +--- + +# Pagination + +## The `limit` & `offset` arguments + +The operators limit and offset are used for pagination. + +`limit` specifies the number of entities to retain from the result set and `offset` determines which slice to retain from the results. + +Default value for `limit` is `50` and `offset` is `0`. + +**Limit results** + +Example: Fetch the first 5 channels: + +```graphql +query { + channels(limit: 5) { + id + handle + } +} +``` + +**Limit results from an offset** + +Example: Fetch 5 channels from the list of all channels, starting with the 6th one: + +```graphql +query { + channels(limit: 5, offset: 5) { + id + handle + } +} +``` + +## Cursor based pagination + +Cursors are used to traverse across entities of an entity set. They work by returning a pointer to a specific entity which can then be used to fetch the next batch of entities. + +For cursor based pagination for every entity in the input schema a query is generated with the `Connection` pattern. + +Example: Fetch a list of videos where `isExplicit` is true and get their count. Then limit the number of videos to return. + +```graphql +query { + videosConnection(where: { isExplicit_eq: true }) { + totalCount + edges { + node { + id + title + } + } + } +} +``` + +**`first` `last` operators** + +The `first` operator is used to fetch specified number of entities from the beginning and `last` is vice versa. + +Example: Fetch first 5 videos and last 5 videos: + +```graphql +query Query1 { + videosConnection(first: 5) { + edges { + node { + id + title + } + } + } +} + +query Query1 { + videosConnection(last: 5) { + edges { + node { + id + title + } + } + } +} +``` + +**PageInfo object** + +`PageInfo` returns the cursor, page information and object has following fields: + +```javascript +pageInfo { + startCursor + endCursor + hasNextPage + hasPreviousPage +} +``` + +**`before` and `after` operators** + +Example: Fetch a first 10 channels order by `createdAt` and then fetch the next 10 channels: + +```graphql +query FirstBatchQ { + channelsConnection(first: 10, orderBy: createdAt_ASC) { + pageInfo { + endCursor + hasNextPage + } + edges { + node { + id + handle + createdAt + } + } + } +} + +query SecondBatchQ { + channelsConnection(after: , orderBy: createdAt_ASC) { + pageInfo { + endCursor + hasNextPage + } + edges { + node { + id + handle + createdAt + } + } + } +} +``` + +Example: Fetch a last 10 channels order by `createdAt` and then fetch the previous 10 channels: + +```graphql +query FirstBatchQ { + channelsConnection(last: 10, orderBy: createdAt_ASC) { + pageInfo { + endCursor + hasNextPage + } + edges { + node { + id + handle + createdAt + } + } + } +} + +query SecondBatchQ { + channelsConnection(before: , orderBy: createdAt_ASC) { + pageInfo { + endCursor + hasNextPage + } + edges { + node { + id + handle + createdAt + } + } + } +} +``` + +**Important Note on orderBy** + +The field entities are ordered by should be fetch `node { }` otherwise the returned result wouldn't be ordered correctly. + diff --git a/docs/queries.md b/docs/queries.md new file mode 100644 index 000000000..44f382ee4 --- /dev/null +++ b/docs/queries.md @@ -0,0 +1,242 @@ +--- +description: GraphQL queries are used to fetch data from the server. +--- + +# Query Node Queries + +## Introduction + +Hydra cli tooling auto-generates queries as part of the GraphQL schema from your input schema. It generates a range of possible queries and operators that also work with relationships defined in your input schema. + +All entities of the input schema tracked by the cli \(re-generation is required when any change happens to the input schema\) can be queried over the GraphQL endpoint. + +## Exploring queries + +You can explore the entire schema and the available queries using the GraphiQL interface by running your graphql-server or looking at the graphql-server/generated/schema.graphql file. + +Let’s take a look at the different queries you can run using the GraphQL server. We’ll use examples based on a typical channel/video schema for reference. + +* Simple entity queries +* Relation entity queries +* Filter query results / search queries +* Sort query results +* Paginate query results + +### Simple entity queries + +You can fetch a single entity or multiple entities of the same type using a simple entity query. + +**Fetch list of entities** + +Example: Fetch a list of channels: + +```graphql +query { + channels { + id + handle + } +} +``` + +**Fetch an entity using its unique fields** + +Example: Fetch a channel using by unique id: + +```graphql +query Query1 { + channelByUniqueInput(where: { id: "1" }) { + id + handle + } +} + +query Query2 { + channelByUniqueInput(where: { handle: "Joy Channel" }) { + id + handle + } +} +``` + +### Relation entity queries + +Please look at the cross-filters documantation. + +### Filter query results / search queries + +**The `where` argument** + +You can use the `where` argument in your queries to filter results based on some field’s values. You can even use multiple filters in the same where clause using the `AND` or the `OR` operators. + +For example, to fetch data for `Joy Channel`: + +```graphql +query { + channels(where: { handle_eq: "Joy Channel" }) { + id + handle + } +} +``` + +#### Comparison operators + +**Supported Scalar Types** + +Hydra supports following scalar types: + +* String +* Int +* Float +* BigInt +* Boolean +* Bytes +* DateTime + +**Equality Operators \(`_eq`\)** + +`_eq` is supported by all the scalar types + +The following are examples of using this operator on different types: + +1. Fetch a list of videos where `title` is "Bitcoin" +2. Fetch a list of videos where `isExplicit` is "true" +3. Fetch a list of videos `publishedOn` "2021-01-05" + +```graphql +query Query1 { + videos(where: { title_eq: "Bitcoin" }) { + id + title + } +} + +query Query2 { + videos(where: { isExplicit_eq: true }) { + id + title + } +} + +query Query3 { + videos(where: { publishedOn_eq: "2021-01-05" }) { + id + title + } +} +``` + +**Greater than or less than operators \(`gt`, `lt`, `gte`, `lte`\)** + +The `_gt` \(greater than\), `_lt` \(less than\), `_gte` \(greater than or equal to\), `_lte` \(less than or equal to\) operators are available on `Int, BigInt, Float, DataTime` types. + +The following are examples of using these operators on different types: + +1. Fetch a list of videos published before "2021-01-05" +2. Fetch a list of channels before block "999" + +```graphql +query Query1 { + videos(where: { publishedOn_gte: "2021-01-05" }) { + id + title + } +} + +query Query2 { + channels(where: { block_lte: "999" }) { + id + handle + } +} +``` + +**List based search operators \(`_in`\)** + +`_in` operator is available on all scalar types except `DataTime`. + +The following are examples of using this operators on different types: + +1. Fetch a list of videos titled with "For Children" or "For Kids" +2. Fetch a list of channels created in block 1, 2 or 3 + +```graphql +query Query1 { + videos(where: { title_in: ["For Children", "For Kids"] }) { + id + title + } +} + +query Query2 { + channels(where: { block_in: [1, 2, 3] }) { + id + handle + } +} +``` + +**Text search or pattern matching operators \(`_contains`, `_startsWith`, `_endsWith`\)** + +The `_contains`, `_startsWith`, `_endsWith` operators are used for pattern matching on string fields. + +Example: + +```graphql +query Query1 { + videos(where: { title_contains: "Bitcoin" }) { + id + title + } +} + +query Query2 { + videos(where: { title_endsWith: "cryptocurrency" }) { + id + title + } +} +``` + +#### Using multiple filters in the same query \(`AND`, `OR`\) + +You can group multiple parameters in the same where argument using the `AND` or the `OR` operators to filter results based on more than one criteria. + +Example `AND`: + +Fetch a list of videos published in a specific time-frame: + +```graphql +query { + videos( + where: { + AND: [ + { publishedOn_gte: "2021-01-05" } + { publishedOn_lte: "2020-01-05" } + ] + } + ) { + id + title + publishedOn + } +} +``` + +Example `OR`: + +Fetch a list of videos isExplicit "true" or published after "2021-01-05": + +```graphql +query { + videos( + where: { OR: [{ publishedOn_gte: "2021-01-05" }, { isExplicit_eq: true }] } + ) { + id + title + isExplicit + } +} +``` + diff --git a/docs/quick-start.md b/docs/quick-start.md index c81a860c4..56aced2e3 100644 --- a/docs/quick-start.md +++ b/docs/quick-start.md @@ -48,9 +48,9 @@ Under the fold, `yarn booststrap` creates a folder `generated/graphql-server` wi ## 3. Typegen for events and extrinsics - List the events and extrinsics to be used by the mappings and generated type-safe classes using the typegen tool. One can define in a separate yml file or modify the `typegen` section in `manifest.yml` +List the events and extrinsics to be used by the mappings and generated type-safe classes using the typegen tool. One can define in a separate yml file or modify the `typegen` section in `manifest.yml` -Typegen fetches the metadata from the chain from the block with a given hash \(or from the top block if no hash is provided\) +Typegen fetches the metadata from the chain from the block with a given hash \(or from the top block if no hash is provided\) ```yaml typegen: @@ -71,7 +71,7 @@ typegen: ## 4. Mappings and the manifest file -Modify the default mappings in the mappings folder and make sure all the mapping functions are exported. Define the mappings in the `mappings` section +Modify the default mappings in the mappings folder and make sure all the mapping functions are exported. Define the mappings in the `mappings` section ```yaml mappings: @@ -89,12 +89,11 @@ mappings: # extrinsic to handle - extrinsic: timestamp.set handler: timestampCall(DatabaseManager, Timestamp.SetCall) - ``` ## 5. Dockerize -Among other things, the scaffolder generates a `docker` folder with Dockerfiles. +Among other things, the scaffolder generates a `docker` folder with Dockerfiles. First, build the builder image: @@ -130,3 +129,4 @@ $ docker-compose up * Describe your own [schema](schema-spec/) in `schema.graphql` * Write your indexer [mappings](mappings/) * Push your Hydra indexer and GraphQL Docker images to [Docker Hub](https://hub.docker.com/) and deploy + diff --git a/docs/sort-query-results.md b/docs/sort-query-results.md new file mode 100644 index 000000000..19c981758 --- /dev/null +++ b/docs/sort-query-results.md @@ -0,0 +1,37 @@ +--- +description: Results from your query can be sorted by using the orderBy argument. +--- + +# Sorting + +The sort order \(ascending vs. descending\) is set by specifying the asc or desc enum value for the column name in the `orderBy` input object, e.g. `title_DESC`. + +The `orderBy` argument takes an array of field to allow sorting by multiple columns. + +**Sorting entities** + +Example: Fetch a list of videos sorted by their titles in an ascending order: + +```graphql +query { + videos(orderBy: [title_ASC]) { + id + title + } +} +``` + +**Sorting entities by multiple fields** + +Example: Fetch a list of videos that is sorted by their titles \(ascending\) and then on their published date \(descending\): + +```graphql +query { + videos(orderBy: [title_ASC, publishedOn_DESC]) { + id + title + publishedOn + } +} +``` +