From 3ec331e1da7905f609b6286533fcd53ccc9a908b Mon Sep 17 00:00:00 2001 From: Hongchao Deng Date: Mon, 18 May 2020 11:53:59 -0700 Subject: [PATCH 1/3] design: resource dependency in OAM Signed-off-by: Hongchao Deng --- ...er-trait-workload-interaction-mechanism.md | 0 doc/design/resource-dependency.md | 315 ++++++++++++++++++ 2 files changed, 315 insertions(+) rename {design => doc/design}/one-pager-trait-workload-interaction-mechanism.md (100%) create mode 100644 doc/design/resource-dependency.md diff --git a/design/one-pager-trait-workload-interaction-mechanism.md b/doc/design/one-pager-trait-workload-interaction-mechanism.md similarity index 100% rename from design/one-pager-trait-workload-interaction-mechanism.md rename to doc/design/one-pager-trait-workload-interaction-mechanism.md diff --git a/doc/design/resource-dependency.md b/doc/design/resource-dependency.md new file mode 100644 index 00000000..f3829dcf --- /dev/null +++ b/doc/design/resource-dependency.md @@ -0,0 +1,315 @@ +# Resource Dependency in OAM + +- Owner: Hongchao Deng (@hongchaodeng) +- Date: 05/17/2020 + +## Background + +Creating one application service or resource often relies on other services or resources to be ready. +This is a well-known pattern called dependency. + +In OAM, we talk about dependency in two different contexts: + +1. **Inter ApplicationConfiguration**: + There is dependency workflow between multiple ApplicationConfigurations. This usually can be achieved + via external workflow engine such as Tekton/Argo Workflow. +2. **Intra ApplicationConfiguration**: + There is dependency between Components and Traits within the same ApplicationConfiguration. + Since they are applied at the same time, external workflow engine couldn't do anything to control their ordering. + It is the OAM runtime that needs to satisfy this requirement. + +In this proposal, we are handling the second case, *Intra ApplicationConfiguration* dependency. + +## Goals + +This proposal includes the following goals: + +1. **Describe ordering for k8s-style resources**. + One key part of depedency is ordering, i.e. creating resources one after another. + For example, resource A could not be created unless resource B has been ready (we will describe the meaning of ready below). + We want to apply this to k8s-style resource. + Since both components and traits are k8s-style resources, as long as we achieve this goal it should satisfy OAM requirements too. +2. **Describe field data passing between resources**. + If we think dependency as ordering, it is still not enough. We also need to handle field data passing. + For example, resource A's field "spec.connSecret" relies on resource B's field "status.connSecret". + This means the field data of one resource is passed from another field of other resource and needs to wait the field data to be ready. + In fact, ordering in the first goal can be modeled as data dependency as well except that the receiver doesn't use the data. +3. **OAM native experience**. + OAM has application concepts like Components, Traits, etc. + The solution will provide OAM native experience built on top of the generic solution. For example, modeling as traits. + +In the following we will first go through the use cases and then propose a solution to satisfy them. + + +## Use Cases + +### 1. Web app and database + +An applicationConfiguration consists of two components: web and db (database). +Web component should not be created until db component is created, +including public endpoint and connection secret to be made ready. + +### 2. ML app workflow + +An ML applicationConfiguration from [Hypercycle ML](http://www.4paradigm.com/product/hypercycleml) +has three components: + +- preprocess +- training +- serving + +They have strict ordering: + +``` +preprocess -> training -> serving +``` + +Each component has to wait for data to be finished processing by the previous stage. +Today, ML frameworks including Tensorflow do not handle such dependency issue. +They rely on external tools (e.g. Argo Workflow) to handle it. +If we want to model them as Components in the same ApplicationConfiguration, the external tools could not help in this case. + +### 3. Service-binding Trait + +Service binding is an OAM trait that we develop to bind user specified input such as env or file path +to specific data sources such as secrets or events. +More background introduction can be found in this [slides](https://docs.google.com/presentation/d/1PseN_8_zZH8clWZJzP8tuRMpz51SxQb-mKeG1f4QVZQ/edit?usp=sharing). + +A service binding trait needs to be applied before creating the component. +Moreover, this kind of trait ordering is quite general and applies to many other traits we have seen. + +### 4. NSQ deployment + +An [NSQ](/~https://github.com/nsqio/nsq) cluster is composed with three components: nsqd, nsqlookup and nsqadmin. +To set up a `nsq` cluster, one needs to first make sure that the `nsqd` component is up, then start and make sure that the `nsqlookup` component is up, and run the `nsqadmin` component at last. +nsqlookup consumes the result of nsqd component instance, and nsqadmin consumes the result of nsqlookup component instance. +If we couldn't describe the dependencies order between the components, the component couldn't run properly. + +This is similar to [kubernetes#65502](/~https://github.com/kubernetes/kubernetes/issues/65502) , the issue describes dependencies between containers on the same Pod, while we are discussing dependencies between components. + + +## Proposal + +The overall idea is to have each resource object specify data inputs coming from other resources' fields. +The OAM runtime will build the dependency graph accordingly and only create resources once the data inputs are ready. +To achieve this, we propose: + +1. Add the following `DataInputs` into each resource. + + ```go + type DataInputs []DataInput + + type DataInput struct { + // FromDataOutput specifies the name of the DataOutput object in the same namespace. + FromDataOutput string `json:"fromDataOutput"` + + // ToFieldPaths specifies an array of fields within this resource object + // that will be overwritten by the value of given DataOutput. The type of the + // parameter (e.g. int, string) is inferred from the type of these fields; + // All fields must be of the same type. Fields are specified as JSON field + // paths without a leading dot, for example 'spec.replicas'. + // If empty, it means this input is not used at all. + // This is allowed for resource ordering purpose. + ToFieldPaths []string `json:"toFieldPaths"` + } + ``` + + For example, a web application may specify input of mysql secret in yaml style: + + ```yaml + datainputs: + - toFieldPaths: ["spec.connSecret"] + fromDataOutput: mysql-conn-secret # check below DataOutput of the same name + ``` + + The above DataInputs will be json-marshaled and put into an annotation key `core.oam.dev/datainputs` of my-app Deployment: + + ```yaml + kind: Deployment + metadata: + name: my-app + annotations: + "core.oam.dev/datainputs": "[{\"toFieldPaths\": [\"spec.connSecret\"],\"fromDataOutput\": \"mysql-conn-secret\"}]" + ``` + + Note that my-app won't be created since it has DataInputs dependency to satisfy. + +2. Add a new CRD `DataOutput` with the following definition: + + ```go + type DataOutput struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec DataOutputSpec `json:"spec,omitempty"` + Status DataOutputStatus `json:"status,omitempty"` + } + + type DataOutputSpec struct { + // Resource specifies the reference of specific resource, which points to a unique resource. + Resource ResourceReference `json:"resource"` + + // FieldPath specifies the field path of a given k8s-style resource. + FieldPath string `json:"fieldPath"` + } + + type ResourceReference struct { + APIVersion string `json:"apiVersion"` + + Kind string `json:"kind"` + + // Name indicates the name of the object in the same namespace. + Name string `json:"name,omitempty"` + } + + type DataOutputStatus struct { + // Ready indicates whether the field data of the source resource is ready. + Ready bool `json:"ready"` + } + ``` + + For example, a mysql connection secret DataOutput coming from RDSInstance my-rds would look like: + + ```yaml + apiVersion: core.oam.dev/v1alpha2 + kind: DataOutput + metadata: + name: mysql-conn-secret + spec: + resource: + apiVersion: alibaba.oam.crossplane.io/v1alpha1 + kind: RDSInstance + name: my-rds + fieldPath: "status.connSecret" + ``` + + In this case, the OAM runtime will automatically build a dependency between the `my-app` and `my-rds`. + + It will wait until `my-rds` object's field `status.connSecret` has value, and marks the DataOutput status ready: + + ```yaml + apiVersion: alibaba.oam.crossplane.io/v1alpha1 + kind: RDSInstance + metadata: + name: my-rds + spec: + ... + status: + connSecret: mysql-conn # OAM runtime will wait for this field to have value + --- + apiVersion: core.oam.dev/v1alpha2 + kind: DataOutput + metadata: + name: mysql-conn-secret + ... + status: + ready: true # once the above connSecret has value this will be marked true + ``` + + At this point, the dependency has been satisfied. + The OAM runtime will create `my-app` object with corresponding values passed to its fields. + + ```yaml + kind: Deployment + metadata: + name: my-app + .. + spec: + connSecret: mysql-conn + ``` + +3. In OAM Component we put a special syntax sugar in `Parameters` fields: + as long asthe name of the Parameter matches the name of the DataOutput, it becomes a DataInput. + + For example, the following parameter mysql-conn-secret in Component matches the name of DataOutput trait + in the same ApplicationConfiguration. + + ```yaml + kind: Component + metadata: + name: my-app + spec: + workload: + kind: Deployment + metadata: + name: my-app + parameters: + - name: mysql-conn-secret # this matches the name of DataOutput + fieldPaths: ["spec.connSecret"] + + --- + # In ApplicationConfiguration + components: + - componentName: my-app + traits: + - trait: + apiVersion: core.oam.dev/v1alpha2 + kind: DataOutput + metadata: + name: mysql-conn-secret # this matches the name of a component's parameter + spec: + resource: + apiVersion: alibaba.oam.crossplane.io/v1alpha1 + kind: RDSInstance + name: my-rds + fieldPath: "status.connSecret" + ``` + + In this case, the OAM runtime will build a dependency from my-app to my-rds, + wait until the field data ready, and create the workload patched with DataInputs annotation. + + ```yaml + kind: Deployment + metadata: + name: my-app + annotations: + "core.oam.dev/datainputs": "[{\"toFieldPaths\": [\"spec.connSecret\"],\"fromDataOutput\": \"mysql-conn-secret\"}]" + ``` + +### Limitations + +All the resources in the dependency graph should be created and managed by OAM runtime. + +## Solution examples + +In this section we are providing more solution examples for dependency use cases. + +1. Service binding trait: + + ApplicationConfiguration (partial): + + ```yaml + components: + - componentName: my-app + traits: + - trait: + apiVersion: core.oam.dev/v1alpha1 + kind: ServiceBinding + metadata: + name: my-secret-binding + spec: + from: + secret: + name: my-secret + to: + env: true + - trait: + kind: DataOutput + resource: + apiVersion: core.oam.dev/v1alpha1 + kind: ServiceBinding + name: my-secret-binding + fieldPath: "status.ready" + ``` + + Component: + + ```yaml + kind: Component + metadata: + name: my-app + spec: + parameters: + - name: my-secret-binding + fieldPaths: [] # This parameter is for ordering purpose only + ``` From 20e743cc87c1c6a3f27efd3627d1ef74fd60f6f9 Mon Sep 17 00:00:00 2001 From: Hongchao Deng Date: Mon, 18 May 2020 21:22:02 -0700 Subject: [PATCH 2/3] fix comments Signed-off-by: Hongchao Deng --- ...er-trait-workload-interaction-mechanism.md | 0 design/resource-dependency.md | 323 ++++++++++++++++++ doc/design/resource-dependency.md | 315 ----------------- 3 files changed, 323 insertions(+), 315 deletions(-) rename {doc/design => design}/one-pager-trait-workload-interaction-mechanism.md (100%) create mode 100644 design/resource-dependency.md delete mode 100644 doc/design/resource-dependency.md diff --git a/doc/design/one-pager-trait-workload-interaction-mechanism.md b/design/one-pager-trait-workload-interaction-mechanism.md similarity index 100% rename from doc/design/one-pager-trait-workload-interaction-mechanism.md rename to design/one-pager-trait-workload-interaction-mechanism.md diff --git a/design/resource-dependency.md b/design/resource-dependency.md new file mode 100644 index 00000000..0508a822 --- /dev/null +++ b/design/resource-dependency.md @@ -0,0 +1,323 @@ +# Resource Dependency in OAM + +- Owner: Hongchao Deng (@hongchaodeng) +- Date: 05/17/2020 +- Status: Draft + +## Background + +Creating one application service or resource often relies on other services or resources to be ready. +This is a well-known pattern called dependency. + +In OAM, we talk about dependency in two different contexts: + +1. **Inter ApplicationConfiguration**: + There is dependency workflow between multiple ApplicationConfigurations. This usually can be achieved + via external workflow engine such as Tekton/Argo Workflow. +2. **Intra ApplicationConfiguration**: + There is dependency between Components and Traits within the same ApplicationConfiguration. + Since they are applied at the same time, external workflow engine couldn't do anything to control their ordering. + It is the OAM runtime that needs to satisfy this requirement. + +In this proposal, we are handling the second case, *Intra ApplicationConfiguration* dependency. + +## Goals + +This proposal includes the following goals: + +1. **Describe ordering for k8s-style resources**. + The ordering between resources means creating one resource after another. + For example, resource A could not be created until resource B has been created and provided outputs. + +2. **Describe field data passing between resources**. + It is still not enough to solve only ordering problem. We also need to handle field data passing. + For example, resource A's field "spec.connSecret" relies on resource B's field "status.connSecret". + This means the field data of one resource is passed from another field of other resource and needs to wait the field data to be ready. + In fact, ordering in the first goal can be modeled as data dependency as well except that the receiver doesn't use the data. + +3. **OAM native experience**. + OAM has application concepts like Components, Traits, etc. + The solution should provide OAM native experience and written as native fields. + But OAM resources are k8s resources too. + The core dependency engine should be generic enough to extend to more OAM resources. + +In the following, we will go through the use cases first and then propose a solution to satisfy them. + + +## Use Cases + +### 1. Web app and database + +An applicationConfiguration consists of two components: web and db (database). +Web component should not be created until db component is created, +including public endpoint and connection secret to be made ready. + +### 2. ML app workflow + +An ML applicationConfiguration from [Hypercycle ML](http://www.4paradigm.com/product/hypercycleml) +has three components: + +- preprocess +- training +- serving + +They have strict ordering: + +``` +preprocess -> training -> serving +``` + +Each component has to wait for data to be finished processing by the previous stage. +Today, ML frameworks including Tensorflow do not handle such dependency issue. +They rely on external tools (e.g. Argo Workflow) to handle it. +If we want to model them as Components in the same ApplicationConfiguration, the external tools could not help in this case. + +### 3. Service-binding Trait + +Service binding is an OAM trait that we develop to bind user specified input such as env or file path +to specific data sources such as secrets or events. +More background introduction can be found in this [slides](https://docs.google.com/presentation/d/1PseN_8_zZH8clWZJzP8tuRMpz51SxQb-mKeG1f4QVZQ/edit?usp=sharing). + +A service binding trait needs to be applied before creating the component. +Moreover, this kind of trait ordering is quite general and applies to many other traits we have seen. + +### 4. NSQ deployment + +An [NSQ](/~https://github.com/nsqio/nsq) cluster is composed with three components: nsqd, nsqlookup and nsqadmin. +To set up a `nsq` cluster, one needs to first make sure that the `nsqd` component is up, then start and make sure that the `nsqlookup` component is up, and run the `nsqadmin` component at last. +nsqlookup consumes the result of nsqd component instance, and nsqadmin consumes the result of nsqlookup component instance. +If we couldn't describe the dependencies order between the components, the component couldn't run properly. + +This is similar to [kubernetes#65502](/~https://github.com/kubernetes/kubernetes/issues/65502) , the issue describes dependencies between containers on the same Pod, while we are discussing dependencies between components. + +### 5. etcd cluster backup + +An etcd cluster would be created via EtcdCluster CR, and automatic periodic etcd backup would be made via EtcdBackup CR. +In OAM, we model EtcdCluster as Component and EtcdBackup as Trait. +However, the EtcdBackup depends on the connection secret specified in `status.connSecret` field of EtcdCluster. +In other words, the EtcdBackup trait depends on EtcdCluster component. + +## Proposal + +The overall idea is to have each resource specify data inputs coming from data outputs (i.e. other resources' fields). +The OAM runtime will build the dependency graph accordingly and only create dependent resources once the data outputs are ready. +To achieve this, we propose: + + +1. Add a field `dataOutputs` in Component configuration to specify a data output source: + + ```yaml + dataOutputs: + - name: + fieldPath: + ``` + + For example, a my-rds database that outputs connection secret could be specified as: + + ```yaml + kind: Component + metadata: + name: my-rds + spec: + workload: + ... # connection secret name will be written to 'status.connSecret' + + --- + kind: ApplicationConfiguration + spec: + components: + - componentName: my-rds + dataOutputs: + - name: mysql-conn-secret + fieldPath: status.connSecret + ``` + +2. Add a field `dataInputs` in Component configuration to specify a data input: + + ```yaml + dataInputs: + - valueFrom: + dataOutputName: + toFieldPaths: + ``` + + For example, web app's dependency on RDS's database secret could be specified as: + + ```yaml + kind: ApplicationConfiguration + spec: + components: + - componentName: my-rds + dataOutputs: + - name: mysql-conn-secret + fieldPath: status.connSecret + + - componentName: my-web + dataInputs: + - valueFrom: + # Specify the dependency on a data output. + # The component's workload won't be created until the data output has returned value to fill workload's fieldPaths. + dataOutputName: mysql-conn-secret + toFieldPaths: + - "spec.connSecret" + + ``` + + In this case, the OAM runtime will automatically build a dependency from `my-web` to `my-rds`. + + It will wait until `my-rds` object's field `status.connSecret` has value: + + ```yaml + apiVersion: example.io/v1alpha1 + kind: RDSInstance + metadata: + name: my-rds + status: + connSecret: mysql-conn # OAM runtime will wait for this field to have value + ``` + + Once the dependency has been satisfied, the OAM runtime will create `my-web` workload with + the corresponding field paths patched with values from data outputs. + + ```yaml + kind: MyWorkload + metadata: + name: my-web + spec: + connSecret: mysql-conn # patched by OAM runtime + ``` + +3. Similary, add fields `dataOutputs` and `dataInputs` in Trait configuration to specify the dependency on data output objects. + + For example, we have an etcd backup trait that relies on the data output from an etcd cluster resource. + We can specify it as: + + ```yaml + kind: ApplicationConfiguration + spec: + components: + - componentName: my-etcd-cluster + dataoutputs: + - name: etcd-conn-secret + fieldpath: "status.connSecret" + + traits: + - trait: + apiVersion: example.io/v1alpha1 + kind: EtcdBackup + metadata: + name: my-etcd-backup + dataInputs: + - valueFrom: # An EtcdBackup trait has dependency on some DataOutput to fill its fields. + dataOutputName: etcd-conn-secret + toFieldPaths: + - spec.connSecret + ``` + + In this case, the EtcdBackup CR creation will be hold until EtcdCluster CR has outputed value in `status.connSecret` + and passed to EtcdBackup CR field `spec.connSecret`. + +### Limitations + +1. All the resources in the dependency graph should be created and managed by OAM runtime. +2. This proposal is designed only for creating resources. After resources have been created, subsequent data output updates will not trigger any actions. + +### Generic core engine consideration + +The design of resource dependency is a pretty generic engine that works for any k8s resources. +Even though we use OAM native fields like `dataInputs` or `dataOutputs` fields, those are just representation layer. +The underlying dependency engine along with the DataInput and DataOutput model is the core implementation, +which should be generic enough to convert or adopt future concepts or fields. + + +## Impact to existing system + +This proposal is additional to existing OAM runtime. It would not affect any existing deployments. + +If dataOutputName in a data input doesn't match any name of the data outputs, it would fail the deployment. + +### Spec upgrade path consideration + +Initially the feature will be experimental. It might change iteratively. +Once that happens, we strive to keep it upgradeable between two releases. +Here we discuss an idea to upgrade if that happens. + +The core dependency engine in OAM runtime is a generic implementation. +We would map APIs to underlying DataInput/DataOutput objects. + +It's inevitable that an intermediate phase where both old and new APIs co-exist. +But they don't both exist in the same ApplicationConfiguration. +Only one could exist and we don't need to handle any conflicts. +In that case, We could add an webhook to translate the old to the new API. + +## Solution examples + +In this section we are providing more solution examples for dependency use cases. + +1. Web app and database. + + ```yaml + kind: Component + metadata: + name: my-rds + spec: + workload: + kind: RDSInstance + metadata: + name: my-rds + --- + + kind: Component + metadata: + name: my-web + spec: + workload: + kind: MyWorkload + metadata: + name: my-web + --- + + kind: ApplicationConfiguration + spec: + components: + - componentName: my-web + dataInputs: + - valueFrom: + dataOutputName: mysql-conn-secret + toFieldPaths: + - "spec.connSecret" + - componentName: my-rds + dataOutputs: + - name: mysql-conn-secret + fieldPath: "status.connSecret" + ``` + +2. Service binding trait. + + ApplicationConfiguration : + + ```yaml + kind: ApplicationConfiguration + spec: + components: + - componentName: my-web + dataInputs: + - fieldPaths: [] # This data input is for ordering purpose only + valueFrom: + dataOutputName: my-secret-binding + traits: + - trait: + apiVersion: core.oam.dev/v1alpha1 + kind: ServiceBinding + metadata: + name: my-secret-binding + spec: + from: + secret: + name: my-secret + to: + env: true + dataOutputs: + - name: my-secret-binding + fieldPath: status.ready + ``` diff --git a/doc/design/resource-dependency.md b/doc/design/resource-dependency.md deleted file mode 100644 index f3829dcf..00000000 --- a/doc/design/resource-dependency.md +++ /dev/null @@ -1,315 +0,0 @@ -# Resource Dependency in OAM - -- Owner: Hongchao Deng (@hongchaodeng) -- Date: 05/17/2020 - -## Background - -Creating one application service or resource often relies on other services or resources to be ready. -This is a well-known pattern called dependency. - -In OAM, we talk about dependency in two different contexts: - -1. **Inter ApplicationConfiguration**: - There is dependency workflow between multiple ApplicationConfigurations. This usually can be achieved - via external workflow engine such as Tekton/Argo Workflow. -2. **Intra ApplicationConfiguration**: - There is dependency between Components and Traits within the same ApplicationConfiguration. - Since they are applied at the same time, external workflow engine couldn't do anything to control their ordering. - It is the OAM runtime that needs to satisfy this requirement. - -In this proposal, we are handling the second case, *Intra ApplicationConfiguration* dependency. - -## Goals - -This proposal includes the following goals: - -1. **Describe ordering for k8s-style resources**. - One key part of depedency is ordering, i.e. creating resources one after another. - For example, resource A could not be created unless resource B has been ready (we will describe the meaning of ready below). - We want to apply this to k8s-style resource. - Since both components and traits are k8s-style resources, as long as we achieve this goal it should satisfy OAM requirements too. -2. **Describe field data passing between resources**. - If we think dependency as ordering, it is still not enough. We also need to handle field data passing. - For example, resource A's field "spec.connSecret" relies on resource B's field "status.connSecret". - This means the field data of one resource is passed from another field of other resource and needs to wait the field data to be ready. - In fact, ordering in the first goal can be modeled as data dependency as well except that the receiver doesn't use the data. -3. **OAM native experience**. - OAM has application concepts like Components, Traits, etc. - The solution will provide OAM native experience built on top of the generic solution. For example, modeling as traits. - -In the following we will first go through the use cases and then propose a solution to satisfy them. - - -## Use Cases - -### 1. Web app and database - -An applicationConfiguration consists of two components: web and db (database). -Web component should not be created until db component is created, -including public endpoint and connection secret to be made ready. - -### 2. ML app workflow - -An ML applicationConfiguration from [Hypercycle ML](http://www.4paradigm.com/product/hypercycleml) -has three components: - -- preprocess -- training -- serving - -They have strict ordering: - -``` -preprocess -> training -> serving -``` - -Each component has to wait for data to be finished processing by the previous stage. -Today, ML frameworks including Tensorflow do not handle such dependency issue. -They rely on external tools (e.g. Argo Workflow) to handle it. -If we want to model them as Components in the same ApplicationConfiguration, the external tools could not help in this case. - -### 3. Service-binding Trait - -Service binding is an OAM trait that we develop to bind user specified input such as env or file path -to specific data sources such as secrets or events. -More background introduction can be found in this [slides](https://docs.google.com/presentation/d/1PseN_8_zZH8clWZJzP8tuRMpz51SxQb-mKeG1f4QVZQ/edit?usp=sharing). - -A service binding trait needs to be applied before creating the component. -Moreover, this kind of trait ordering is quite general and applies to many other traits we have seen. - -### 4. NSQ deployment - -An [NSQ](/~https://github.com/nsqio/nsq) cluster is composed with three components: nsqd, nsqlookup and nsqadmin. -To set up a `nsq` cluster, one needs to first make sure that the `nsqd` component is up, then start and make sure that the `nsqlookup` component is up, and run the `nsqadmin` component at last. -nsqlookup consumes the result of nsqd component instance, and nsqadmin consumes the result of nsqlookup component instance. -If we couldn't describe the dependencies order between the components, the component couldn't run properly. - -This is similar to [kubernetes#65502](/~https://github.com/kubernetes/kubernetes/issues/65502) , the issue describes dependencies between containers on the same Pod, while we are discussing dependencies between components. - - -## Proposal - -The overall idea is to have each resource object specify data inputs coming from other resources' fields. -The OAM runtime will build the dependency graph accordingly and only create resources once the data inputs are ready. -To achieve this, we propose: - -1. Add the following `DataInputs` into each resource. - - ```go - type DataInputs []DataInput - - type DataInput struct { - // FromDataOutput specifies the name of the DataOutput object in the same namespace. - FromDataOutput string `json:"fromDataOutput"` - - // ToFieldPaths specifies an array of fields within this resource object - // that will be overwritten by the value of given DataOutput. The type of the - // parameter (e.g. int, string) is inferred from the type of these fields; - // All fields must be of the same type. Fields are specified as JSON field - // paths without a leading dot, for example 'spec.replicas'. - // If empty, it means this input is not used at all. - // This is allowed for resource ordering purpose. - ToFieldPaths []string `json:"toFieldPaths"` - } - ``` - - For example, a web application may specify input of mysql secret in yaml style: - - ```yaml - datainputs: - - toFieldPaths: ["spec.connSecret"] - fromDataOutput: mysql-conn-secret # check below DataOutput of the same name - ``` - - The above DataInputs will be json-marshaled and put into an annotation key `core.oam.dev/datainputs` of my-app Deployment: - - ```yaml - kind: Deployment - metadata: - name: my-app - annotations: - "core.oam.dev/datainputs": "[{\"toFieldPaths\": [\"spec.connSecret\"],\"fromDataOutput\": \"mysql-conn-secret\"}]" - ``` - - Note that my-app won't be created since it has DataInputs dependency to satisfy. - -2. Add a new CRD `DataOutput` with the following definition: - - ```go - type DataOutput struct { - metav1.TypeMeta `json:",inline"` - metav1.ObjectMeta `json:"metadata,omitempty"` - - Spec DataOutputSpec `json:"spec,omitempty"` - Status DataOutputStatus `json:"status,omitempty"` - } - - type DataOutputSpec struct { - // Resource specifies the reference of specific resource, which points to a unique resource. - Resource ResourceReference `json:"resource"` - - // FieldPath specifies the field path of a given k8s-style resource. - FieldPath string `json:"fieldPath"` - } - - type ResourceReference struct { - APIVersion string `json:"apiVersion"` - - Kind string `json:"kind"` - - // Name indicates the name of the object in the same namespace. - Name string `json:"name,omitempty"` - } - - type DataOutputStatus struct { - // Ready indicates whether the field data of the source resource is ready. - Ready bool `json:"ready"` - } - ``` - - For example, a mysql connection secret DataOutput coming from RDSInstance my-rds would look like: - - ```yaml - apiVersion: core.oam.dev/v1alpha2 - kind: DataOutput - metadata: - name: mysql-conn-secret - spec: - resource: - apiVersion: alibaba.oam.crossplane.io/v1alpha1 - kind: RDSInstance - name: my-rds - fieldPath: "status.connSecret" - ``` - - In this case, the OAM runtime will automatically build a dependency between the `my-app` and `my-rds`. - - It will wait until `my-rds` object's field `status.connSecret` has value, and marks the DataOutput status ready: - - ```yaml - apiVersion: alibaba.oam.crossplane.io/v1alpha1 - kind: RDSInstance - metadata: - name: my-rds - spec: - ... - status: - connSecret: mysql-conn # OAM runtime will wait for this field to have value - --- - apiVersion: core.oam.dev/v1alpha2 - kind: DataOutput - metadata: - name: mysql-conn-secret - ... - status: - ready: true # once the above connSecret has value this will be marked true - ``` - - At this point, the dependency has been satisfied. - The OAM runtime will create `my-app` object with corresponding values passed to its fields. - - ```yaml - kind: Deployment - metadata: - name: my-app - .. - spec: - connSecret: mysql-conn - ``` - -3. In OAM Component we put a special syntax sugar in `Parameters` fields: - as long asthe name of the Parameter matches the name of the DataOutput, it becomes a DataInput. - - For example, the following parameter mysql-conn-secret in Component matches the name of DataOutput trait - in the same ApplicationConfiguration. - - ```yaml - kind: Component - metadata: - name: my-app - spec: - workload: - kind: Deployment - metadata: - name: my-app - parameters: - - name: mysql-conn-secret # this matches the name of DataOutput - fieldPaths: ["spec.connSecret"] - - --- - # In ApplicationConfiguration - components: - - componentName: my-app - traits: - - trait: - apiVersion: core.oam.dev/v1alpha2 - kind: DataOutput - metadata: - name: mysql-conn-secret # this matches the name of a component's parameter - spec: - resource: - apiVersion: alibaba.oam.crossplane.io/v1alpha1 - kind: RDSInstance - name: my-rds - fieldPath: "status.connSecret" - ``` - - In this case, the OAM runtime will build a dependency from my-app to my-rds, - wait until the field data ready, and create the workload patched with DataInputs annotation. - - ```yaml - kind: Deployment - metadata: - name: my-app - annotations: - "core.oam.dev/datainputs": "[{\"toFieldPaths\": [\"spec.connSecret\"],\"fromDataOutput\": \"mysql-conn-secret\"}]" - ``` - -### Limitations - -All the resources in the dependency graph should be created and managed by OAM runtime. - -## Solution examples - -In this section we are providing more solution examples for dependency use cases. - -1. Service binding trait: - - ApplicationConfiguration (partial): - - ```yaml - components: - - componentName: my-app - traits: - - trait: - apiVersion: core.oam.dev/v1alpha1 - kind: ServiceBinding - metadata: - name: my-secret-binding - spec: - from: - secret: - name: my-secret - to: - env: true - - trait: - kind: DataOutput - resource: - apiVersion: core.oam.dev/v1alpha1 - kind: ServiceBinding - name: my-secret-binding - fieldPath: "status.ready" - ``` - - Component: - - ```yaml - kind: Component - metadata: - name: my-app - spec: - parameters: - - name: my-secret-binding - fieldPaths: [] # This parameter is for ordering purpose only - ``` From 7fc6aff69735cb7b9a5b3da4aca8a5844d92a9ca Mon Sep 17 00:00:00 2001 From: Hongchao Deng Date: Fri, 12 Jun 2020 21:24:24 -0700 Subject: [PATCH 3/3] add ingress example Signed-off-by: Hongchao Deng --- design/resource-dependency.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/design/resource-dependency.md b/design/resource-dependency.md index 0508a822..0401d4da 100644 --- a/design/resource-dependency.md +++ b/design/resource-dependency.md @@ -97,6 +97,12 @@ In OAM, we model EtcdCluster as Component and EtcdBackup as Trait. However, the EtcdBackup depends on the connection secret specified in `status.connSecret` field of EtcdCluster. In other words, the EtcdBackup trait depends on EtcdCluster component. +### 6. Ingress after component + +We have an ingress implementation that requires the microservice component must be setup before applying ingress entry. +In this case, the ingress trait should only be created after the component is ready. + + ## Proposal The overall idea is to have each resource specify data inputs coming from data outputs (i.e. other resources' fields). @@ -292,7 +298,7 @@ In this section we are providing more solution examples for dependency use cases fieldPath: "status.connSecret" ``` -2. Service binding trait. +2. Service binding trait must start before component. ApplicationConfiguration : @@ -321,3 +327,4 @@ In this section we are providing more solution examples for dependency use cases - name: my-secret-binding fieldPath: status.ready ``` +