DAOSGCP-83 and DAOSGCP-84 Automatic format storage with pool and cont…

…ainer creation that supports ACLs (#40)

* DAOSGCP-83 and DAOSGCP-84 Automatic format storage with pool and container creation that supports ACLs

Signed-off-by: Łukasz Sitkiewicz <lukasz.sitkiewicz@intel.com>

Co-authored-by: Mark A. Olson <mark.a.olson@intel.com>

terraform/examples/daos_cluster/README.md

@@ -272,7 +272,7 @@ No resources.
 | <a name="input_server_daos_disk_count"></a> [server\_daos\_disk\_count](#input\_server\_daos\_disk\_count) | Number of local ssd's to use | `number` | `16` | no |
 | <a name="input_server_daos_disk_type"></a> [server\_daos\_disk\_type](#input\_server\_daos\_disk\_type) | Daos disk type to use. For now only suported one is local-ssd | `string` | `"local-ssd"` | no |
 | <a name="input_server_daos_scm_size"></a> [server\_daos\_scm\_size](#input\_server\_daos\_scm\_size) | scm\_size | `number` | `200` | no |
-| <a name="input_server_gvnic"></a> [server\_gvnic](#input\_server\_gvnic) | Use Google Virtual NIC (gVNIC) network interface on DAOS servers | `bool` | `false` | no |
+| <a name="input_server_gvnic"></a> [server\_gvnic](#input\_server\_gvnic) | Use Google Virtual NIC (gVNIC) network interface | `bool` | `false` | no |
 | <a name="input_server_instance_base_name"></a> [server\_instance\_base\_name](#input\_server\_instance\_base\_name) | MIG instance base names to use | `string` | `"daos-server"` | no |
 | <a name="input_server_labels"></a> [server\_labels](#input\_server\_labels) | Set of key/value label pairs to assign to daos-server instances | `any` | `{}` | no |
 | <a name="input_server_machine_type"></a> [server\_machine\_type](#input\_server\_machine\_type) | GCP machine type. ie. e2-medium | `string` | `"n2-custom-36-215040"` | no |
@@ -282,7 +282,7 @@ No resources.
 | <a name="input_server_os_disk_type"></a> [server\_os\_disk\_type](#input\_server\_os\_disk\_type) | OS disk type ie. pd-ssd, pd-standard | `string` | `"pd-ssd"` | no |
 | <a name="input_server_os_family"></a> [server\_os\_family](#input\_server\_os\_family) | OS GCP image family | `string` | `"daos-server-centos-7"` | no |
 | <a name="input_server_os_project"></a> [server\_os\_project](#input\_server\_os\_project) | OS GCP image project name. Defaults to project\_id if null. | `string` | `null` | no |
-| <a name="input_server_pools"></a> [server\_pools](#input\_server\_pools) | If provided, this module will generate a script to create a list of pools. pool attributes have to be specified in a format acceptable by [dmg](https://docs.daos.io/v2.0/admin/pool_operations/) and daos. | <pre>list(object({<br>    pool_name  = string<br>    pool_size  = string<br>    containers = list(string)<br>    })<br>  )</pre> | `[]` | no |
+| <a name="input_server_pools"></a> [server\_pools](#input\_server\_pools) | List of pools and containers to be created | <pre>list(object({<br>    name       = string<br>    size       = string<br>    tier_ratio = number<br>    acls       = list(string)<br>    properties = map(string)<br>    containers = list(object({<br>      name            = string<br>      type            = string<br>      acls            = list(string)<br>      properties      = map(string)<br>      user_attributes = map(any)<br>    }))<br>  }))</pre> | `[]` | no |
 | <a name="input_server_preemptible"></a> [server\_preemptible](#input\_server\_preemptible) | If preemptible instances | `string` | `false` | no |
 | <a name="input_server_service_account"></a> [server\_service\_account](#input\_server\_service\_account) | Service account to attach to the instance. See https://www.terraform.io/docs/providers/google/r/compute_instance_template.html#service_account. | <pre>object({<br>    email  = string,<br>    scopes = set(string)<br>  })</pre> | <pre>{<br>  "email": null,<br>  "scopes": [<br>    "https://www.googleapis.com/auth/devstorage.read_only",<br>    "https://www.googleapis.com/auth/logging.write",<br>    "https://www.googleapis.com/auth/monitoring.write",<br>    "https://www.googleapis.com/auth/servicecontrol",<br>    "https://www.googleapis.com/auth/service.management.readonly",<br>    "https://www.googleapis.com/auth/trace.append",<br>    "https://www.googleapis.com/auth/cloud-platform"<br>  ]<br>}</pre> | no |
 | <a name="input_server_template_name"></a> [server\_template\_name](#input\_server\_template\_name) | MIG template name | `string` | `"daos-server"` | no |
@@ -292,5 +292,7 @@ No resources.
 
 ## Outputs
 
-No outputs.
+| Name | Description |
+|------|-------------|
+| <a name="output_daos_pools"></a> [daos\_pools](#output\_daos\_pools) | Specification of pools and containers to create |
 <!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

terraform/examples/daos_cluster/module.json

@@ -163,7 +163,7 @@
     {
       "name": "server_gvnic",
       "type": "bool",
-      "description": "Use Google Virtual NIC (gVNIC) network interface on DAOS servers",
+      "description": "Use Google Virtual NIC (gVNIC) network interface",
       "default": false,
       "required": false
     },
@@ -232,8 +232,8 @@
     },
     {
       "name": "server_pools",
-      "type": "list(object({\n    pool_name  = string\n    pool_size  = string\n    containers = list(string)\n    })\n  )",
-      "description": "If provided, this module will generate a script to create a list of pools. pool attributes have to be specified in a format acceptable by [dmg](https://docs.daos.io/v2.0/admin/pool_operations/) and daos.",
+      "type": "list(object({\n    name       = string\n    size       = string\n    tier_ratio = number\n    acls       = list(string)\n    properties = map(string)\n    containers = list(object({\n      name            = string\n      type            = string\n      acls            = list(string)\n      properties      = map(string)\n      user_attributes = map(any)\n    }))\n  }))",
+      "description": "List of pools and containers to be created",
       "default": [],
       "required": false
     },
@@ -305,7 +305,12 @@
       "description": null
     }
   ],
-  "outputs": [],
+  "outputs": [
+    {
+      "name": "daos_pools",
+      "description": "Specification of pools and containers to create"
+    }
+  ],
   "providers": [],
   "requirements": [
     {

terraform/examples/daos_cluster/terraform.tfvars.perf.example

@@ -25,12 +25,37 @@ server_daos_scm_size       = 45
 # server_os_project          = "<project_id>"
 # server_os_disk_type        = "pd-ssd"
 
-# pools = [
-#  {
-#    pool_name = "test_pool"
-#    pool_size = "1TB"
-#    containers = [ "test_container1", "test_container2" ]
-#  }
+# server_pools = [
+#   {
+#     name       = "pool1",
+#     size       = "1TB",
+#     tier_ratio = 6
+#     acls = [
+#       "A::OWNER@:rwdtTaAo",
+#       "A:G:GROUP@:rwtT",
+#       "A::EVERYONE@:r",
+#     ],
+#     properties = {
+#       reclaim = "lazy"
+#     },
+#     containers = [
+#       {
+#         name = "container1",
+#         type = "POSIX",
+#         acls = [
+#           "A::OWNER@:rwdtTaAo",
+#           "A:G:GROUP@:rwtT",
+#           "A::EVERYONE@:rwtT"
+#         ],
+#         properties = {
+#           rf = 0
+#         },
+#         user_attributes = {
+#           automount = true
+#         }
+#       }
+#     ]
+#   }
 # ]
 
 # client_number_of_instances = 16

terraform/examples/daos_cluster/terraform.tfvars.tco.example

@@ -25,12 +25,37 @@ server_number_of_instances = 4
 # server_os_project          = "<project_id>"
 # server_os_disk_type        = "pd-ssd"
 
-# pools = [
-#  {
-#    pool_name = "test_pool"
-#    pool_size = "1TB"
-#    containers = [ "test_container1", "test_container2" ]
-#  }
+# server_pools = [
+#   {
+#     name       = "pool1",
+#     size       = "1TB",
+#     tier_ratio = 6
+#     acls = [
+#       "A::OWNER@:rwdtTaAo",
+#       "A:G:GROUP@:rwtT",
+#       "A::EVERYONE@:r",
+#     ],
+#     properties = {
+#       reclaim = "lazy"
+#     },
+#     containers = [
+#       {
+#         name = "container1",
+#         type = "POSIX",
+#         acls = [
+#           "A::OWNER@:rwdtTaAo",
+#           "A:G:GROUP@:rwtT",
+#           "A::EVERYONE@:rwtT"
+#         ],
+#         properties = {
+#           rf = 0
+#         },
+#         user_attributes = {
+#           automount = true
+#         }
+#       }
+#     ]
+#   }
 # ]
 
 # client_number_of_instances = 16

terraform/examples/daos_cluster/variables.tf

@@ -152,7 +152,6 @@ variable "server_preemptible" {
   type        = string
 }
 
-
 variable "server_daos_scm_size" {
   description = "scm_size"
   default     = 200
@@ -166,20 +165,28 @@ variable "server_daos_crt_timeout" {
 }
 
 variable "server_gvnic" {
-  description = "Use Google Virtual NIC (gVNIC) network interface on DAOS servers"
+  description = "Use Google Virtual NIC (gVNIC) network interface"
   default     = false
   type        = bool
 }
 
 variable "server_pools" {
-  description = "If provided, this module will generate a script to create a list of pools. pool attributes have to be specified in a format acceptable by [dmg](https://docs.daos.io/v2.0/admin/pool_operations/) and daos."
+  description = "List of pools and containers to be created"
   default     = []
   type = list(object({
-    pool_name  = string
-    pool_size  = string
-    containers = list(string)
-    })
-  )
+    name       = string
+    size       = string
+    tier_ratio = number
+    acls       = list(string)
+    properties = map(any)
+    containers = list(object({
+      name            = string
+      type            = string
+      acls            = list(string)
+      properties      = map(any)
+      user_attributes = map(any)
+    }))
+  }))
 }
 
 variable "client_labels" {

terraform/examples/io500/start.sh

@@ -522,7 +522,7 @@ To run the IO500 benchmark:
    ./login
 
 2. Run IO500
-   ~/run_io500-sc21.sh
+   ./run_io500-sc21.sh
 
 EOF
 }

terraform/examples/only_daos_server/README.md

@@ -157,7 +157,7 @@ No resources.
 | <a name="input_server_os_disk_type"></a> [server\_os\_disk\_type](#input\_server\_os\_disk\_type) | OS disk type ie. pd-ssd, pd-standard | `string` | `"pd-ssd"` | no |
 | <a name="input_server_os_family"></a> [server\_os\_family](#input\_server\_os\_family) | OS GCP image family | `string` | `"daos-server-centos-7"` | no |
 | <a name="input_server_os_project"></a> [server\_os\_project](#input\_server\_os\_project) | OS GCP image project name. Defaults to project\_id if null. | `string` | `null` | no |
-| <a name="input_server_pools"></a> [server\_pools](#input\_server\_pools) | If provided, this module will generate a script to create a list of pools. pool attributes have to be specified in a format acceptable by [dmg](https://docs.daos.io/v2.0/admin/pool_operations/) and daos. | <pre>list(object({<br>    pool_name  = string<br>    pool_size  = string<br>    containers = list(string)<br>    })<br>  )</pre> | `[]` | no |
+| <a name="input_server_pools"></a> [server\_pools](#input\_server\_pools) | List of pools and containers to be created | <pre>list(object({<br>    name       = string<br>    size       = string<br>    tier_ratio = number<br>    acls       = list(string)<br>    properties = map(string)<br>    containers = list(object({<br>      name            = string<br>      type            = string<br>      acls            = list(string)<br>      properties      = map(string)<br>      user_attributes = map(any)<br>    }))<br>  }))</pre> | `[]` | no |
 | <a name="input_server_preemptible"></a> [server\_preemptible](#input\_server\_preemptible) | If preemptible instances | `string` | `false` | no |
 | <a name="input_server_service_account"></a> [server\_service\_account](#input\_server\_service\_account) | Service account to attach to the instance. See https://www.terraform.io/docs/providers/google/r/compute_instance_template.html#service_account. | <pre>object({<br>    email  = string,<br>    scopes = set(string)<br>  })</pre> | <pre>{<br>  "email": null,<br>  "scopes": [<br>    "https://www.googleapis.com/auth/devstorage.read_only",<br>    "https://www.googleapis.com/auth/logging.write",<br>    "https://www.googleapis.com/auth/monitoring.write",<br>    "https://www.googleapis.com/auth/servicecontrol",<br>    "https://www.googleapis.com/auth/service.management.readonly",<br>    "https://www.googleapis.com/auth/trace.append",<br>    "https://www.googleapis.com/auth/cloud-platform"<br>  ]<br>}</pre> | no |
 | <a name="input_server_template_name"></a> [server\_template\_name](#input\_server\_template\_name) | MIG template name | `string` | `"daos-server"` | no |

terraform/examples/only_daos_server/module.json

@@ -130,8 +130,8 @@
     },
     {
       "name": "server_pools",
-      "type": "list(object({\n    pool_name  = string\n    pool_size  = string\n    containers = list(string)\n    })\n  )",
-      "description": "If provided, this module will generate a script to create a list of pools. pool attributes have to be specified in a format acceptable by [dmg](https://docs.daos.io/v2.0/admin/pool_operations/) and daos.",
+      "type": "list(object({\n    name       = string\n    size       = string\n    tier_ratio = number\n    acls       = list(string)\n    properties = map(string)\n    containers = list(object({\n      name            = string\n      type            = string\n      acls            = list(string)\n      properties      = map(string)\n      user_attributes = map(any)\n    }))\n  }))",
+      "description": "List of pools and containers to be created",
       "default": [],
       "required": false
     },

terraform/examples/only_daos_server/terraform.tfvars.perf.example

@@ -24,10 +24,35 @@ server_labels              = {
 # server_os_project          = "<project_id>"
 # server_os_disk_type        = "pd-ssd"
 
-# pools = [
-#  {
-#    pool_name = "test_pool"
-#    pool_size = "1TB"
-#    containers = [ "test_container1", "test_container2" ]
-#  }
+# server_pools = [
+#   {
+#     name       = "pool1",
+#     size       = "1TB",
+#     tier_ratio = 6
+#     acls = [
+#       "A::OWNER@:rwdtTaAo",
+#       "A:G:GROUP@:rwtT",
+#       "A::EVERYONE@:r",
+#     ],
+#     properties = {
+#       reclaim = "lazy"
+#     },
+#     containers = [
+#       {
+#         name = "container1",
+#         type = "POSIX",
+#         acls = [
+#           "A::OWNER@:rwdtTaAo",
+#           "A:G:GROUP@:rwtT",
+#           "A::EVERYONE@:rwtT"
+#         ],
+#         properties = {
+#           rf = 0
+#         },
+#         user_attributes = {
+#           automount = true
+#         }
+#       }
+#     ]
+#   }
 # ]

terraform/examples/only_daos_server/terraform.tfvars.tco.example

@@ -24,10 +24,35 @@ server_labels              = {
 # server_os_project          = "<project_id>"
 # server_os_disk_type        = "pd-ssd"
 
-# pools = [
-#  {
-#    pool_name = "test_pool"
-#    pool_size = "1TB"
-#    containers = [ "test_container1", "test_container2" ]
-#  }
+# server_pools = [
+#   {
+#     name       = "pool1",
+#     size       = "1TB",
+#     tier_ratio = 6
+#     acls = [
+#       "A::OWNER@:rwdtTaAo",
+#       "A:G:GROUP@:rwtT",
+#       "A::EVERYONE@:r",
+#     ],
+#     properties = {
+#       reclaim = "lazy"
+#     },
+#     containers = [
+#       {
+#         name = "container1",
+#         type = "POSIX",
+#         acls = [
+#           "A::OWNER@:rwdtTaAo",
+#           "A:G:GROUP@:rwtT",
+#           "A::EVERYONE@:rwtT"
+#         ],
+#         properties = {
+#           rf = 0
+#         },
+#         user_attributes = {
+#           automount = true
+#         }
+#       }
+#     ]
+#   }
 # ]

terraform/examples/only_daos_server/variables.tf

@@ -152,14 +152,22 @@ variable "server_preemptible" {
 }
 
 variable "server_pools" {
-  description = "If provided, this module will generate a script to create a list of pools. pool attributes have to be specified in a format acceptable by [dmg](https://docs.daos.io/v2.0/admin/pool_operations/) and daos."
+  description = "List of pools and containers to be created"
   default     = []
   type = list(object({
-    pool_name  = string
-    pool_size  = string
-    containers = list(string)
-    })
-  )
+    name       = string
+    size       = string
+    tier_ratio = number
+    acls       = list(string)
+    properties = map(any)
+    containers = list(object({
+      name            = string
+      type            = string
+      acls            = list(string)
+      properties      = map(any)
+      user_attributes = map(any)
+    }))
+  }))
 }
 
 variable "server_daos_scm_size" {

terraform/modules/daos_client/README.md

@@ -55,8 +55,7 @@ No modules.
 
 | Name | Description | Type | Default | Required |
 |------|-------------|------|---------|:--------:|
-| <a name="input_allow_insecure"></a> [allow\_insecure](#input\_allow\_insecure) | Sets the allow\_insecure setting in the transport\_config section of the daos\_*.yml files | `bool` | `false` | no |
-| <a name="input_certs_install_script_content"></a> [certs\_install\_script\_content](#input\_certs\_install\_script\_content) | Script to install certificates for DAOS secure transport | `string` | n/a | yes |
+| <a name="input_certs_install_content"></a> [certs\_install\_content](#input\_certs\_install\_content) | DAOS certificate installation code to be included in the client startup script | `string` | n/a | yes |
 | <a name="input_daos_agent_yml"></a> [daos\_agent\_yml](#input\_daos\_agent\_yml) | YAML to configure the daos agent. | `string` | n/a | yes |
 | <a name="input_daos_control_yml"></a> [daos\_control\_yml](#input\_daos\_control\_yml) | YAML configuring DAOS control. | `string` | n/a | yes |
 | <a name="input_gvnic"></a> [gvnic](#input\_gvnic) | Use Google Virtual NIC (gVNIC) network interface | `bool` | `false` | no |

terraform/modules/daos_client/main.tf

@@ -18,9 +18,9 @@ locals {
   os_project         = var.os_project != null ? var.os_project : var.project_id
   subnetwork_project = var.subnetwork_project != null ? var.subnetwork_project : var.project_id
   # Google Virtual NIC (gVNIC) network interface
-  nic_type                     = var.gvnic ? "GVNIC" : "VIRTIO_NET"
-  total_egress_bandwidth_tier  = var.gvnic ? "TIER_1" : "DEFAULT"
-  certs_install_content        = var.certs_install_content
+  nic_type                    = var.gvnic ? "GVNIC" : "VIRTIO_NET"
+  total_egress_bandwidth_tier = var.gvnic ? "TIER_1" : "DEFAULT"
+  certs_install_content       = var.certs_install_content
 
   startup_script = templatefile(
     "${path.module}/templates/daos_startup_script.tftpl",
@@ -100,11 +100,11 @@ resource "google_compute_per_instance_config" "named_instances" {
   name                   = format("%s-%04d", var.instance_base_name, sum([count.index, 1]))
   preserved_state {
     metadata = {
-      inst_type                    = "daos-client"
-      enable-oslogin               = "true"
-      daos_control_yaml_content    = var.daos_control_yml
-      daos_agent_yaml_content      = var.daos_agent_yml
-      startup-script               = local.startup_script
+      inst_type                 = "daos-client"
+      enable-oslogin            = "true"
+      daos_control_yaml_content = var.daos_control_yml
+      daos_agent_yaml_content   = var.daos_agent_yml
+      startup-script            = local.startup_script
       # Adding a reference to the instance template used causes the stateful instance to update
       # if the instance template changes. Otherwise there is no explicit dependency and template
       # changes may not occur on the stateful instance

terraform/modules/daos_client/module.json

@@ -3,16 +3,9 @@
   "footer": "",
   "inputs": [
     {
-      "name": "allow_insecure",
-      "type": "bool",
-      "description": "Sets the allow_insecure setting in the transport_config section of the daos_*.yml files",
-      "default": false,
-      "required": false
-    },
-    {
-      "name": "certs_install_script_content",
+      "name": "certs_install_content",
       "type": "string",
-      "description": "Script to install certificates for DAOS secure transport",
+      "description": "DAOS certificate installation code to be included in the client startup script",
       "default": null,
       "required": true
     },