Skip to content

Commit

Permalink
[influxdb] allow defining measurement name in item metadata and code …
Browse files Browse the repository at this point in the history 
…improvements (#71)

* metadata improvements

- Add option for using metadata value as measurement name
- Code improvements

Also-By: Johannes Ott <info@johannes-ott.net>
Signed-Off-By: Jan N. Klug <jan.n.klug@rub.de>

* refactoring

Signed-off-by: Jan N. Klug <jan.n.klug@rub.de>

* use checked exception

Signed-off-by: Jan N. Klug <jan.n.klug@rub.de>

* fixes

Signed-off-by: Jan N. Klug <jan.n.klug@rub.de>

* fix

Signed-off-by: Jan N. Klug <jan.n.klug@rub.de>

* Upgrade dependencies to fix issue with closing influx connections

Signed-off-by: Jan N. Klug <jan.n.klug@rub.de>
  • Loading branch information
@J-N-K
J-N-K committed Apr 18, 2021
1 parent 2621bbf commit fa517f2
Show file tree
Hide file tree
Showing 21 changed files with 621 additions and 508 deletions.
91 changes: 70 additions & 21 deletions bundles/org.smarthomej.persistence.influxdb/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,12 @@ There also are nice tools on the web for visualizing InfluxDB time series, such


- This service allows you to persist and query states using the time series database.
- The states of an item are persisted in *measurements* points with names equal to the name of the item, or the alias, if one is provided. In both variants, a *tag* named "item" is added, containing the item name.
All values are stored in a *field* called "value" using the following types:
- **float** for DecimalType and QuantityType
- **integer** for `OnOffType` and `OpenClosedType` (values are stored using 0 or 1) and `DateTimeType` (milliseconds since 1970-01-01T00:00:00Z)
- **string** for the rest of types
- The states of an item are persisted in _measurements_ with names equal to the name of the item, its alias or from some metadata (depending on the configuration).
In all variants, a tag named "item" is added, containing the item name.
All values are stored in a _field_ called "value" using the following types:
- **float** for DecimalType and QuantityType
- **integer** for `OnOffType` and `OpenClosedType` (values are stored using 0 or 1) and `DateTimeType` (milliseconds since 1970-01-01T00:00:00Z)
- **string** for the rest of types

- If configured, extra tags for item category, label or type can be added fore each point.

Expand All @@ -31,25 +32,73 @@ Some example entries for an item with the name "speedtest" without any further c

## Prerequisites

First of all you have to setup and run an InfluxDB 1.X or 2.X server.
This is very easy and you will find good documentation on it on the
[InfluxDB web site for 2.X version](https://v2.docs.influxdata.com/v2.0/get-started/) and [InfluxDB web site for 1.X version](https://docs.influxdata.com/influxdb/v1.7/).
First you have to install and run an InfluxDB 1.X or 2.X server.
This is very easy, and you will find good documentation on it on the [InfluxDB web site for 2.X version](https://v2.docs.influxdata.com/v2.0/get-started/) and [InfluxDB web site for 1.X version](https://docs.influxdata.com/influxdb/v1.7/).

## Configuration

This service can be configured in the file `services/influxdb.cfg`.
| Property | Default | Required | Description |
| --------------- | --------------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
| version | V1 | No | InfluxDB database version V1 for 1.X and V2 for 2.x |
| url | http://127.0.0.1:8086 | No | database URL |
| user | openhab | No | name of the database user, e.g. `openhab` |
| password | | No (\*) | password of the database user you choose |
| token | | No (\*) | token to authenticate the database (only for V2) [Intructions about how to create one](https://v2.docs.influxdata.com/v2.0/security/tokens/create-token/) |
| db | openhab | No | name of the database for V1 and name of the organization for V2 |
| retentionPolicy | autogen | No | name of the retention policy for V1 and name of the bucket for V2 |

| Property | Default | Required | Description |
|------------------------------------|-------------------------|----------|------------------------------------------|
| version | V1 | No | InfluxDB database version V1 for 1.X and V2 for 2.x|
| url | http://127.0.0.1:8086 | No | database URL |
| user | openhab | No | name of the database user, e.g. `openhab`|
| password | | No(*) | password of the database user you choose |
| token | | No(*) | token to authenticate the database (only for V2) [Intructions about how to create one](https://v2.docs.influxdata.com/v2.0/security/tokens/create-token/) |
| db | openhab | No | name of the database for V1 and name of the organization for V2 |
| retentionPolicy | autogen | No | name of the retention policy for V1 and name of the bucket for V2 |

(*) For 1.X version you must provide user and password, for 2.X you can use user and password or a token. That means
that if you use all default values at minimum you must provide a password or a token.
(*) For 1.X version you must provide user and password, for 2.X you can use user and password or a token.
That means that if you use all default values at minimum you must provide a password or a token.

All item- and event-related configuration is defined in the file `persistence/influxdb.persist`.


### Additional configuration for customized storage options in InfluxDB

By default, the plugin writes the data to a `measurement` name equals to the `item's name` and adds a tag with key item and value `item's name` as well.
You can customize that behavior and use a single measurement for several items using item metadata.

#### Measurement name by Item Metadata

By setting the `influxdb` metadata key you can change the name of the measurement by setting the desired name as metadata value.
You can also add additional tags for structuring your data.
For example, you can add a floor tag to all sensors to filter all sensors from the first floor or combine all temperature sensors into one measurement.

The item configuration will look like this:

```
Group:Number:AVG gTempSensors
Number:Temperature tempLivingRoom (gTempSensors) { influxdb="temperature" [floor="groundfloor"] }
Number:Temperature tempKitchen (gTempSensors) { influxdb="temperature" [floor="groundfloor"] }
Number:Temperature tempBedRoom (gTempSensors) { influxdb="temperature" [floor="firstfloor"] }
Number:Temperature tempBath (gTempSensors) { influxdb="temperature" [floor="firstfloor"] }
```

You can also set the `influxdb` metadata using the UI.
From each item configuration screen do:

`Metadata` → `Add Metadata` → `Enter Custom Namespace` → Enter `influxdb` as namespace name → And enter your desired item name in value field. i.e.:

value: temperature
config: {}

This will end up with one measurement named temperature and four different series inside:

```
temperature,item=tempLivingRoom,floor=groundfloor
temperature,item=tempKitchen,floor=groundfloor
temperature,item=tempBedRoom,floor=firstfloor
temperature,item=tempBath,floor=firstfloor
```

You can now easily select all temperatures of the `firstfloor` or the average temperature of the `groundfloor`.

#### Extended automatic tagging

Besides the metadata tags, there are additional configuration parameters to activate different automatic tags generation.

| Property | Default | Required | Description |
| -------------- | ------- | -------- | ---------------------------------------------------------------------------------------------------- |
| addCategoryTag | false | no | Should the category of the item be included as tag "category"? If no category is set, "n/a" is used. |
| addTypeTag | false | no | Should the item type be included as tag "type"? |
| addLabelTag | false | no | Should the item label be included as tag "label"? If no label is set, "n/a" is used. |
97 changes: 49 additions & 48 deletions bundles/org.smarthomej.persistence.influxdb/pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,82 +16,92 @@

<properties>
<bnd.importpackage>
!javax.annotation;!android.*,!com.android.*,!com.google.appengine.*,!dalvik.system,!kotlin.*,!kotlinx.*,!org.conscrypt,!sun.security.ssl,!org.apache.harmony.*,!org.apache.http.*,!rx.*,!org.msgpack.*
!javax.annotation.*;!android.*,!com.android.*,!com.google.appengine.*,!dalvik.system,!kotlin.*,!kotlinx.*,!org.conscrypt,!sun.security.ssl,!org.apache.harmony.*,!org.apache.http.*,!rx.*,!org.msgpack.*
</bnd.importpackage>
<okhttp3.version>3.14.9</okhttp3.version>
<retrofit.version>2.7.2</retrofit.version>
<influx2.version>1.15.0</influx2.version>
<influx1.version>2.21</influx1.version>
</properties>

<dependencies>
<!-- START InfluxDB 2.0 -->
<!-- START influxdb-client-java -->
<dependency>
<groupId>com.influxdb</groupId>
<artifactId>influxdb-client-java</artifactId>
<version>1.6.0</version>
<version>${influx2.version}</version>
</dependency>
<dependency>
<groupId>com.influxdb</groupId>
<artifactId>influxdb-client-core</artifactId>
<version>${influx2.version}</version>
</dependency>
<dependency>
<groupId>com.influxdb</groupId>
<version>1.6.0</version>
<artifactId>flux-dsl</artifactId>
<version>${influx2.version}</version>
</dependency>

<dependency>
<artifactId>converter-gson</artifactId>
<groupId>com.squareup.retrofit2</groupId>
<version>2.5.0</version>
<artifactId>converter-gson</artifactId>
<version>${retrofit.version}</version>
</dependency>
<dependency>
<groupId>com.squareup.retrofit2</groupId>
<artifactId>converter-scalars</artifactId>
<version>${retrofit.version}</version>
</dependency>
<dependency>
<artifactId>retrofit</artifactId>
<groupId>com.squareup.retrofit2</groupId>
<version>2.5.0</version>
<version>${retrofit.version}</version>
</dependency>

<dependency>
<groupId>com.squareup.okhttp3</groupId>
<artifactId>okhttp</artifactId>
<version>${okhttp3.version}</version>
</dependency>
<dependency>
<groupId>com.squareup.okhttp3</groupId>
<artifactId>logging-interceptor</artifactId>
<version>${okhttp3.version}</version>
</dependency>
<dependency>
<artifactId>gson</artifactId>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.8.5</version>
</dependency>
<dependency>
<artifactId>gson-fire</artifactId>
<groupId>io.gsonfire</groupId>
<version>1.8.0</version>
<artifactId>gson-fire</artifactId>
<version>1.8.4</version>
</dependency>
<dependency>
<artifactId>okio</artifactId>
<groupId>com.squareup.okio</groupId>
<version>1.17.3</version>
<artifactId>okio</artifactId>
<version>1.17.2</version>
</dependency>
<dependency>
<artifactId>commons-csv</artifactId>
<groupId>org.apache.commons</groupId>
<version>1.6</version>
<artifactId>commons-csv</artifactId>
<version>1.8</version>
</dependency>
<dependency>
<artifactId>json</artifactId>
<groupId>org.json</groupId>
<version>20180813</version>
</dependency>
<dependency>
<artifactId>okhttp</artifactId>
<groupId>com.squareup.okhttp3</groupId>
<version>3.14.4</version>
</dependency>
<dependency>
<artifactId>retrofit</artifactId>
<groupId>com.squareup.retrofit2</groupId>
<version>2.6.2</version>
</dependency>
<dependency>
<artifactId>jsr305</artifactId>
<groupId>com.google.code.findbugs</groupId>
<version>3.0.2</version>
</dependency>
<dependency>
<artifactId>logging-interceptor</artifactId>
<groupId>com.squareup.okhttp3</groupId>
<version>3.14.4</version>
<version>20200518</version>
</dependency>
<!-- <dependency> -->
<!-- <artifactId>jsr305</artifactId> -->
<!-- <groupId>com.google.code.findbugs</groupId> -->
<!-- <version>3.0.2</version> -->
<!-- </dependency> -->
<dependency>
<artifactId>rxjava</artifactId>
<groupId>io.reactivex.rxjava2</groupId>
<version>2.2.17</version>
<version>2.2.19</version>
</dependency>
<dependency>
<artifactId>reactive-streams</artifactId>
Expand All @@ -101,29 +111,20 @@
<dependency>
<artifactId>swagger-annotations</artifactId>
<groupId>io.swagger</groupId>
<version>1.5.22</version>
</dependency>
<!--END influxdb-client-java -->


<dependency>
<groupId>com.influxdb</groupId>
<artifactId>flux-dsl</artifactId>
<version>1.6.0</version>
<version>1.6.1</version>
</dependency>

<!--END InfluxDB 2.0 -->

<!--START InfluxDB 1.0 -->
<dependency>
<groupId>org.influxdb</groupId>
<artifactId>influxdb-java</artifactId>
<version>2.17</version>
<version>${influx1.version}</version>
</dependency>
<dependency>
<groupId>com.squareup.retrofit2</groupId>
<artifactId>converter-moshi</artifactId>
<version>2.6.2</version>
<version>${retrofit.version}</version>
</dependency>
<dependency>
<groupId>com.squareup.moshi</groupId>
Expand Down
Loading

0 comments on commit fa517f2

Please sign in to comment.