Skip to content

Commit

Permalink
Added support for load_props, added missing load_xml tests
Browse files Browse the repository at this point in the history
  • Loading branch information
mikefarah committed Feb 15, 2022
1 parent e1b5e90 commit 75fffc1
Show file tree
Hide file tree
Showing 6 changed files with 125 additions and 6 deletions.
1 change: 1 addition & 0 deletions examples/small.properties
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
this.is = a properties file
1 change: 1 addition & 0 deletions examples/small.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
<this>is some xml</this>
22 changes: 20 additions & 2 deletions pkg/yqlib/doc/operators/headers/load.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,32 @@
# Load

The `load`/`strload` operator allows you to load in content from another file referenced in your yaml document.
The load operators allows you to load in content from another file.

Note that you can use string operators like `+` and `sub` to modify the value in the yaml file to a path that exists in your system.

Use `strload` to load text based content as a string block, and `load` to interpret the file as yaml.
You can load files of the following supported types:

|Format | Load Operator |
| --- | --- |
| Yaml | load |
| XML | load_xml |
| Properties | load_props |
| Plain String | load_str |

Lets say there is a file `../../examples/thing.yml`:

```yaml
a: apple is included
b: cool
```
and a file `small.xml`:

```xml
<this>is some xml</this>
```

and `small.properties`:

```properties
this.is = a properties file
```
77 changes: 74 additions & 3 deletions pkg/yqlib/doc/operators/load.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,35 @@
# Load

The `load`/`strload` operator allows you to load in content from another file referenced in your yaml document.
The load operators allows you to load in content from another file.

Note that you can use string operators like `+` and `sub` to modify the value in the yaml file to a path that exists in your system.

Use `strload` to load text based content as a string block, and `load` to interpret the file as yaml.
You can load files of the following supported types:

|Format | Load Operator |
| --- | --- |
| Yaml | load |
| XML | load_xml |
| Properties | load_props |
| Plain String | load_str |

Lets say there is a file `../../examples/thing.yml`:

```yaml
a: apple is included
b: cool
```
and a file `small.xml`:

```xml
<this>is some xml</this>
```

and `small.properties`:

```properties
this.is = a properties file
```

{% hint style="warning" %}
Note that versions prior to 4.18 require the 'eval/e' command to be specified.&#x20;
Expand Down Expand Up @@ -89,7 +107,7 @@ something:
```
then
```bash
yq '.something |= strload("../../examples/" + .file)' sample.yml
yq '.something |= load_str("../../examples/" + .file)' sample.yml
```
will output
```yaml
Expand All @@ -98,3 +116,56 @@ something: |-
b: cool.
```

## Load from XML
Given a sample.yml file of:
```yaml
cool: things
```
then
```bash
yq '.more_stuff = load_xml("../../examples/small.xml")' sample.yml
```
will output
```yaml
cool: things
more_stuff:
this: is some xml
```

## Load from Properties
Given a sample.yml file of:
```yaml
cool: things
```
then
```bash
yq '.more_stuff = load_props("../../examples/small.properties")' sample.yml
```
will output
```yaml
cool: things
more_stuff:
this:
is: a properties file
```

## Merge from properties
This can be used as a convenient way to update a yaml document

Given a sample.yml file of:
```yaml
this:
is: from yaml
cool: ay
```
then
```bash
yq '. *= load_props("../../examples/small.properties")' sample.yml
```
will output
```yaml
this:
is: a properties file
cool: ay
```

3 changes: 3 additions & 0 deletions pkg/yqlib/expression_tokeniser.go
Original file line number Diff line number Diff line change
Expand Up @@ -380,6 +380,9 @@ func initLexer() (*lex.Lexer, error) {
lexer.Add([]byte(`load_xml`), opTokenWithPrefs(loadOpType, nil, loadPrefs{loadAsString: false, decoder: NewXMLDecoder(XMLPreferences.AttributePrefix, XMLPreferences.ContentName)}))
lexer.Add([]byte(`loadxml`), opTokenWithPrefs(loadOpType, nil, loadPrefs{loadAsString: false, decoder: NewXMLDecoder(XMLPreferences.AttributePrefix, XMLPreferences.ContentName)}))

lexer.Add([]byte(`load_props`), opTokenWithPrefs(loadOpType, nil, loadPrefs{loadAsString: false, decoder: NewPropertiesDecoder()}))
lexer.Add([]byte(`loadprops`), opTokenWithPrefs(loadOpType, nil, loadPrefs{loadAsString: false, decoder: NewPropertiesDecoder()}))

lexer.Add([]byte(`strload`), opTokenWithPrefs(loadOpType, nil, loadPrefs{loadAsString: true}))
lexer.Add([]byte(`load_str`), opTokenWithPrefs(loadOpType, nil, loadPrefs{loadAsString: true}))
lexer.Add([]byte(`loadstr`), opTokenWithPrefs(loadOpType, nil, loadPrefs{loadAsString: true}))
Expand Down
27 changes: 26 additions & 1 deletion pkg/yqlib/operator_load_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -35,11 +35,36 @@ var loadScenarios = []expressionScenario{
description: "Replace node with referenced file as string",
subdescription: "This will work for any text based file",
document: `{something: {file: "thing.yml"}}`,
expression: `.something |= strload("../../examples/" + .file)`,
expression: `.something |= load_str("../../examples/" + .file)`,
expected: []string{
"D0, P[], (doc)::{something: \"a: apple is included\\nb: cool.\"}\n",
},
},
{
description: "Load from XML",
document: "cool: things",
expression: `.more_stuff = load_xml("../../examples/small.xml")`,
expected: []string{
"D0, P[], (doc)::cool: things\nmore_stuff:\n this: is some xml\n",
},
},
{
description: "Load from Properties",
document: "cool: things",
expression: `.more_stuff = load_props("../../examples/small.properties")`,
expected: []string{
"D0, P[], (doc)::cool: things\nmore_stuff:\n this:\n is: a properties file\n",
},
},
{
description: "Merge from properties",
subdescription: "This can be used as a convenient way to update a yaml document",
document: "this:\n is: from yaml\n cool: ay\n",
expression: `. *= load_props("../../examples/small.properties")`,
expected: []string{
"D0, P[], (!!map)::this:\n is: a properties file\n cool: ay\n",
},
},
}

func TestLoadScenarios(t *testing.T) {
Expand Down

0 comments on commit 75fffc1

Please sign in to comment.