Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

changing usage of bridges to connectors #701

Merged
merged 48 commits into from
Feb 3, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
5c5dc3b
Update imodel-bridges.md
timlawrence-bentley Jan 4, 2021
138dbdf
Merge branch 'master' into patch-1
mergify[bot] Jan 27, 2021
dd53d06
Merge branch 'master' into patch-1
mergify[bot] Jan 27, 2021
9a14c9f
Merge branch 'master' into patch-1
mergify[bot] Jan 28, 2021
ed0ad91
Merge branch 'master' into patch-1
mergify[bot] Jan 28, 2021
e7dabf5
Merge branch 'master' into patch-1
mergify[bot] Jan 28, 2021
b5415d9
Update categories.md
timlawrence-bentley Jan 28, 2021
17870c5
Update federated-digital-twins.md
timlawrence-bentley Jan 28, 2021
7248ce3
Update schema-customization.md
timlawrence-bentley Jan 28, 2021
dbb754d
Update Roadmap.md
timlawrence-bentley Jan 28, 2021
933b504
Update App.md
timlawrence-bentley Jan 28, 2021
6f215ad
Update Channel.md
timlawrence-bentley Jan 28, 2021
f34511b
Update Capabilities.md
timlawrence-bentley Jan 28, 2021
ad2bd6d
Update WriteABridge.md
timlawrence-bentley Jan 28, 2021
7acbc5b
Merge branch 'master' into patch-1
mergify[bot] Jan 29, 2021
2b271e0
Merge branch 'master' into patch-1
mergify[bot] Jan 29, 2021
6d33465
Merge branch 'master' into patch-1
mergify[bot] Jan 29, 2021
f6cdd44
Merge branch 'master' into patch-1
mergify[bot] Jan 29, 2021
2405c1b
Merge branch 'master' into patch-1
mergify[bot] Jan 29, 2021
602bed5
Merge branch 'master' into patch-1
mergify[bot] Jan 30, 2021
88ebc65
Merge branch 'master' into patch-1
mergify[bot] Jan 30, 2021
097f999
Merge branch 'master' into patch-1
mergify[bot] Jan 31, 2021
f08a3aa
Merge branch 'master' into patch-1
mergify[bot] Jan 31, 2021
7f685cd
Merge branch 'master' into patch-1
mergify[bot] Feb 1, 2021
c87eea0
Merge branch 'master' into patch-1
mergify[bot] Feb 1, 2021
49931a5
Merge branch 'master' into patch-1
mergify[bot] Feb 1, 2021
4cf64a2
Merge branch 'master' into patch-1
mergify[bot] Feb 1, 2021
f9f26d6
Merge branch 'master' into patch-1
mergify[bot] Feb 1, 2021
48c5c56
Rename WriteABridge.md to WriteAConnector.md
timlawrence-bentley Feb 1, 2021
4bd3e86
Merge branch 'master' into patch-1
mergify[bot] Feb 1, 2021
3654ac3
Update schema-customization.md
timlawrence-bentley Feb 1, 2021
8113874
Merge branch 'master' into patch-1
mergify[bot] Feb 1, 2021
c7dc992
Update federated-digital-twins.md
timlawrence-bentley Feb 2, 2021
a175499
Update information-hierarchy.md
timlawrence-bentley Feb 2, 2021
cf2c0b2
Update App.md
timlawrence-bentley Feb 2, 2021
2b41926
Update Capabilities.md
timlawrence-bentley Feb 2, 2021
dcfd5df
Update index.md
timlawrence-bentley Feb 2, 2021
1f2a0dc
Update create-test-imodel-itwin-sync.md
timlawrence-bentley Feb 2, 2021
5eaa2c9
Update create-test-imodel-offline.md
timlawrence-bentley Feb 2, 2021
2d15a07
Update WriteAConnector.md
timlawrence-bentley Feb 2, 2021
d402a07
Merge branch 'master' into patch-1
pmconne Feb 2, 2021
5227652
Merge branch 'master' into patch-1
mergify[bot] Feb 2, 2021
30b831b
Merge branch 'master' into patch-1
mergify[bot] Feb 2, 2021
fcd215c
Merge branch 'master' into patch-1
mergify[bot] Feb 2, 2021
dfaea3c
Rename imodel-bridges.md to imodel-connectors.md
timlawrence-bentley Feb 2, 2021
b0f6bf7
Merge branch 'master' into patch-1
mergify[bot] Feb 3, 2021
941cc13
Merge branch 'master' into patch-1
mergify[bot] Feb 3, 2021
d0d4d58
Merge branch 'master' into patch-1
mergify[bot] Feb 3, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/bis/intro/categories.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,11 +87,11 @@ DrawingCategories are similar to drawing layer/level standards and are ultimatel

<!-- TODO: Elaborate on how user can (or will be able to) control drawing standards in the future. They won't be manually changing GeometricElement2dIsInCategory? -->

## iModel Bridges and Categories
## iModel Connectors and Categories

Each iModel Bridge job should create a `DefinitionModel` for its Categories. That way each bridge can have its own set of Categories without risk of name collision with other jobs.
Each iModel Connector job should create a `DefinitionModel` for its Categories. That way each connector can have its own set of Categories without risk of name collision with other jobs.

iModel Bridges should respect and use the standard SpatialCategories defined by the Domains.
iModel Connectors should respect and use the standard SpatialCategories defined by the Domains.

---
| Next: [Schema Customization](./schema-customization.md)
Expand Down
2 changes: 1 addition & 1 deletion docs/bis/intro/federated-digital-twins.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Another distinguishing characteristic of a Digital Twin is that it is not applic

## Aligned

A Digital Twin should be a cohesive digital replica against which to write services and apps. BIS embodies standard taxonomy, data structure, and relationships for those services and apps. By migrating data from proprietary repositories into an iModel (using [iModel Bridges](../../learning/imodel-bridges.md)) we can bring “dark data” from proprietary, closed CAD, BIM, and GIS formats in an open and extensible format.
A Digital Twin should be a cohesive digital replica against which to write services and apps. BIS embodies standard taxonomy, data structure, and relationships for those services and apps. By migrating data from proprietary repositories into an iModel (using [iModel Connectors](../../learning/imodel-connectors.md)) we can bring “dark data” from proprietary, closed CAD, BIM, and GIS formats in an open and extensible format.

## Federated

Expand Down
6 changes: 3 additions & 3 deletions docs/bis/intro/information-hierarchy.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,12 +53,12 @@ The top of the information hierarchy is strictly controlled and is very similar

Two examples of repository organizations are described below. It should be noted that a single BIS repository may have multiple uses. When that occurs each use (often corresponding to an application) adds the hierarchy; the resulting hierarchy is similar to a union of the uses' hierarchies.

### iModel Bridge Repository Organization
### iModel Connector Repository Organization

TODO: show organization for an iModel created by both:

* One bridge with two source files
* Another bridge with one source file
* One connector with two source files
* Another connector with one source file

### Editing Application Repository Organization

Expand Down
6 changes: 3 additions & 3 deletions docs/bis/intro/schema-customization.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ While BIS models a significant portion of the world of built infrastructure, and

- Customizations and extensions to existing BIS concepts (often per-project or per-company)
- Vendor-specific information
- Transfer of information from other infrastructure-related databases (e.g. via iModel bridges).
- Transfer of information from other infrastructure-related databases (e.g. via iModel connectors).
- Modeling of concepts that do not currently exist in BIS.
- The need to add unstructured information to Elements.

Expand Down Expand Up @@ -53,9 +53,9 @@ When BigCo's pump catalog is distributed, it will include:
QUESTION: *IN PRACTICE, DOES THIS MEAN THAT IF I WANT TO CHANGE A PUMP FROM BIGCO TO LITTLECO, THEN I NEED TO CHANGE THE CLASS OF PUMP AS WELL (IE - I NEED TO DELETE AND RECREATED THE PUMP)?*
-->

## Data Imported from Other Databases (including via iModel Bridges)
## Data Imported from Other Databases (including via iModel Connectors)

The technology (often iModel bridges) that converts data from other databases into BIS data will usually need to convert from the class structure in the native database to a BIS class structure. It is rare that a BIS schema will work "out of the box". Typically a dynamic schema will need to be defined by the converter and then the data converted from the native DB into instance of the new dynamic schema's classes (likely along with some instances of standard BIS classes).
The technology (often iModel connectors) that converts data from other databases into BIS data will usually need to convert from the class structure in the native database to a BIS class structure. It is rare that a BIS schema will work "out of the box". Typically a dynamic schema will need to be defined by the converter and then the data converted from the native DB into instance of the new dynamic schema's classes (likely along with some instances of standard BIS classes).

## Dynamic Schema Minor Change Considerations

Expand Down
6 changes: 3 additions & 3 deletions docs/changehistory/Roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ Volunteers for help on any or all of these projects are welcome. In particular,
### ET/IT/OT Integration

- Provide working examples and templates for interfacing between engineering content in iModels, enterprise systems, and realtime IOT sensors, cameras, controllers, devices, and processors.
- Create tools for augmenting data synchronized with bridges from engineering design tools with IOT-link elements.
- Create tools for augmenting data synchronized with connectors from engineering design tools with IOT-link elements.
- Enhance BIS schemas for common IOT patterns and query engines.

### Agent Deployment
Expand Down Expand Up @@ -82,9 +82,9 @@ Volunteers for help on any or all of these projects are welcome. In particular,

- Create examples and templates illustrating usage of iModel-to-iModel transformations and synchronization.

### iModel.js based bridge framework
### iModel.js based connector framework

- Support for creating multi-process bridges using iModel.js backends. One process links with source application api to read the source application files. It then communicates with another iModel.js process via RPC to update the iModel.
- Support for creating multi-process connectors using iModel.js backends. One process links with source application api to read the source application files. It then communicates with another iModel.js process via RPC to update the iModel.

### iModel Editing applications

Expand Down
12 changes: 6 additions & 6 deletions docs/learning/App.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,12 @@

From the same JavaScript codebase, it is possible to create:

- [Backend Agents and Services](#agents-and-services) that process iModels and respond to events from iModelHub
- [Interactive Apps](#interactive-apps) that have a GUI and access iModel content. Several kinds of apps are supported:
- [Web Apps](#web-apps) that run in web browsers and communicate with backend code running in Web servers
- [Desktop Apps](#desktop-apps) that run on personal computers
- [Mobile Apps](#mobile-apps) that run on tablets and phones
- [Bridges](../learning/WriteABridge.md)
* [Backend Agents and Services](#agents-and-services) that process iModels and respond to events from iModelHub
* [Interactive Apps](#interactive-apps) that have a GUI and access iModel content. Several kinds of apps are supported:
* [Web Apps](#web-apps) that run in web browsers and communicate with backend code running in Web servers
* [Desktop Apps](#desktop-apps) that run on personal computers
* [Mobile Apps](#mobile-apps) that run on tablets and phones
* [Connectors](../learning/WriteAConnector.md)

## Agents and Services

Expand Down
2 changes: 1 addition & 1 deletion docs/learning/Capabilities.md
Original file line number Diff line number Diff line change
Expand Up @@ -193,7 +193,7 @@ Load new functionality to a running instance of an application in a web browser.

Create iModels from data from external BIM/CAD/GIS/etc. applications.

[iModel Bridges](.\imodel-bridges.md) read data from external formats and *bridge* it into an iModel. They create ChangeSets that are sent to iModelHub.
[iModel Connectors](.\imodel-connectors.md) read data from external formats and *connect* it into an iModel. They create ChangeSets that are sent to iModelHub.

## Application and User Settings

Expand Down
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# Write an iModel Bridge
# Write an iModel Connector

As explained in the [overview](../learning/imodel-bridges.md), a "bridge" is a program that:
As explained in the [overview](../learning/imodel-connectors.md), a "connector" is a program that:

1. Reads information from a data source,
2. Aligns the source data with the BIS schema and preferably a domain schema, and
3. Writes BIS data to an iModel.

Specifically, a bridge must:
Specifically, a connector must:

* [Open a local briefcase copy](./backend/IModelDb.md) of the iModel that is to be updated.
* Import or Update Schema
Expand All @@ -15,21 +15,21 @@ Specifically, a bridge must:
* Convert Changed Data
* Connect to the data source.
* Detect changes to the source data.
* [Transform](../learning/imodel-bridges.md#data-alignment) the new or changed source data into the target BIS schema.
* [Transform](../learning/imodel-connectors.md#data-alignment) the new or changed source data into the target BIS schema.
* Write the resulting BIS data to the local briefcase.
* Remove BIS data corresponding to deleted source data.
* Obtain required [Locks and Codes](./backend/ConcurrencyControl.md) from the iModel server and/or code server.
* [Push](./backend/IModelDbReadwrite.md#pushing-changes-to-imodelhub) changes to the iModel server.

## Writing a bridge
## Writing a connector

A bridge would import the following packages:
A connector would import the following packages:

``` ts
[[include:Bridge.imports.example-code]]
```

When the bridge runs for the very first time, it would look like the following. This example revolves around the fictitious "RobotWorld" schema. RobotWorld consists of Robots and Barriers. The details of RobotWorld and its schema are not important. The steps, such as importing a schema, reserving codes, pushing changesets, creating definition models, etc., are the important points to see in the example code.
When the connector runs for the very first time, it would look like the following. This example revolves around the fictitious "RobotWorld" schema. RobotWorld consists of Robots and Barriers. The details of RobotWorld and its schema are not important. The steps, such as importing a schema, reserving codes, pushing changesets, creating definition models, etc., are the important points to see in the example code.

``` ts
[[include:Bridge.firstTime.example-code]]
Expand Down Expand Up @@ -62,28 +62,28 @@ Here is a simple example of a fictitious source data format and the logic to con

## Detecting and pushing changes

Rather than starting over when the source data changes, a bridge should be able to detect and convert only the changes. That makes for compact, meaningful ChangeSets, which are added to the iModel's
Rather than starting over when the source data changes, a connector should be able to detect and convert only the changes. That makes for compact, meaningful ChangeSets, which are added to the iModel's
[timeline](../learning/IModelHub/index.md#the-timeline-of-changes-to-an-imodel).

In the case of source data that was previously converted and has changed, the bridge should update the data in the iModel that were the results of the previous conversion. In the case of source data that was previously converted and has been deleted in the source, the bridge should delete the results of the previous conversion. Source data that has been added should be inserted.
In the case of source data that was previously converted and has changed, the connector should update the data in the iModel that were the results of the previous conversion. In the case of source data that was previously converted and has been deleted in the source, the connector should delete the results of the previous conversion. Source data that has been added should be inserted.

To do incremental updates, a bridge must do Id mapping and change-detection.
To do incremental updates, a connector must do Id mapping and change-detection.

### Id mapping

Id mapping is a way of looking up the data in the iModel that corresponds to a given piece of source data.

If the source data has stable, unique IDs, then Id mapping could be straightforward. The bridge just needs to record the source -> BIS Id mappings somewhere. If the source data IDs are GUIDs, then the bridge can assign them to the federationGuid property value of the BIS elements that it creates. That way, the mappings will be directly recorded in the iModel itself.
If the source data has stable, unique IDs, then Id mapping could be straightforward. The connector just needs to record the source -> BIS Id mappings somewhere. If the source data IDs are GUIDs, then the connector can assign them to the federationGuid property value of the BIS elements that it creates. That way, the mappings will be directly recorded in the iModel itself.

If the source data does not have stable, unique IDs, then the bridge will have to use some other means of identifying pieces of source data in a stable way. A cryptographic hash of the source data itself can work as a stable Id -- that is, it can be used to identify data that has not changed.
If the source data does not have stable, unique IDs, then the connector will have to use some other means of identifying pieces of source data in a stable way. A cryptographic hash of the source data itself can work as a stable Id -- that is, it can be used to identify data that has not changed.

### Change-detection

Change-detection is a way of detecting changes in the source data.

If the source data is timestamped in some way, then the change-detection logic should be easy. The bridge just has to save the highest timestamp at the end of the conversion and then look for source data with later timestamps the next time it runs.
If the source data is timestamped in some way, then the change-detection logic should be easy. The connector just has to save the highest timestamp at the end of the conversion and then look for source data with later timestamps the next time it runs.

If timestamps are not available, then the bridge will have to use some other means of recording and then comparing the state of the source data from run to run. If conversion is cheap, then the source data can be be converted again and the results compared to the previous results, as stored in the iModel. Or, a cryptographic hash of the source data may be used to represent the source data. The hash could be stored along with the mappings and used to detect changes.
If timestamps are not available, then the connector will have to use some other means of recording and then comparing the state of the source data from run to run. If conversion is cheap, then the source data can be be converted again and the results compared to the previous results, as stored in the iModel. Or, a cryptographic hash of the source data may be used to represent the source data. The hash could be stored along with the mappings and used to detect changes.

A basic change-detection algorithm is:

Expand Down
4 changes: 2 additions & 2 deletions docs/learning/backend/Channel.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

A "channel" is a tree of elements. The root of the tree is called the channel "root" element. The tree includes the root and all of its child elements and sub-models, recursively. Channels do not nest.

To be a channel root, an element must have a [ChannelRootAspect]($backend). (Legacy iModel bridges/connectors mark their channel roots with a special JSON property.)
To be a channel root, an element must have a [ChannelRootAspect]($backend). (Legacy iModel connectors mark their channel roots with a special JSON property.)

To help visualize what a channel is, imagine an iModel with the following breakdown:

Expand Down Expand Up @@ -70,7 +70,7 @@ The rules of locking are slightly different for channels:

## Connectors and Channels

A connector always works in a channel. A connector does not create channels or change the channel. A supervisor calls some connector methods in the repository channel and others in the bridge's own private channel. The channel is locked by the supervisor.
A connector always works in a channel. A connector does not create channels or change the channel. A supervisor calls some connector methods in the repository channel and others in the connector's own private channel. The channel is locked by the supervisor.

## Non-Connector Apps and Channels

Expand Down
Loading