Skip to content

Commit

Permalink
Merge branch '7.0' into 7.1
Browse files Browse the repository at this point in the history
  • Loading branch information
at055612 committed Oct 30, 2023
2 parents a62368c + 4e6cebd commit 8d4c975
Show file tree
Hide file tree
Showing 2 changed files with 7 additions and 3 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,6 @@ rectangle "Stream nnn" #Silver {
}
rectangle "Meta" #SeaGreen {
}
rectangle "Context" #SlateBlue {
}
}
}

Expand Down
8 changes: 7 additions & 1 deletion content/en/docs/user-guide/concepts/streams.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ Streams can either be created when data is directly POSTed in to Stroom or durin
When data is directly POSTed to Stroom the content of the POST will be stored as one Stream.
With proxy aggregation multiple files in the proxy repository will/can be aggregated together into a single Stream.


## Anatomy of a Stream

A Stream is made up of a number of parts of which the raw or cooked data is just one.
Expand Down Expand Up @@ -40,16 +41,19 @@ Segmented means that the data has been demarcated into segments or records.

This is the actual data of the stream, e.g. the XML events, raw CSV, JSON, etc.


#### Context

This is additional contextual data that can be sent with the data.
Context data can be used for reference data lookups.


#### Meta Data

This is the data about the Stream (e.g. the feed name, receipt time, user agent, etc.).
This meta data either comes from the HTTP headers when the data was POSTed to Stroom or is added by Stroom or Stroom-Proxy on receipt/processing.


### Non-Segmented Stream

The following is a representation of a non-segmented stream with three parts, each with Meta Data and Context child streams.
Expand All @@ -64,16 +68,18 @@ The Stream may be single-part or multi-part depending on how it is received.
If it is the product of proxy aggregation then it is likely to be multi-part.
Each part will have its own context and meta data child streams, if applicable.


### Segmented Stream

The following is a representation of a segmented stream that contains three records (i.e events) and has Meta Data and Context child streams.
The following is a representation of a segmented stream that contains three records (i.e events) and the Meta Data.

{{< image "user-guide/concepts/streams/segmented-stream.puml.svg" >}}Segmented Stream{{< /image >}}

Cooked Events and Reference data are forms of segmented data.
The raw data has been parsed and split into records/events and the resulting data is stored in a way that allows Stroom to know where each record/event starts/ends.
These streams only have a single part.


### Error Stream

{{< image "user-guide/concepts/streams/error-stream.puml.svg" >}}Error Stream{{< /image >}}
Expand Down

0 comments on commit 8d4c975

Please sign in to comment.