Processing HTML, AsciiDoc and Markdown with the File Repository

[slu-it]

As a developer I want to be able to use files (e.g. HTML, AsciiDoc, Markdown) as the source of documentation for LivingDoc.

Background:

With LivingDoc 2 we want to give our users the option to use any kind of documentation source (Confluence, Jira, HTML files etc.). For this we define the concept of repositories. A repository is an abstraction over any kind of datastore (be it remote or local). Currently we plan at least two repositories for the release of version 2.0:

1. File Repository
2. Confluence Repository

This story covers the implementation of a File Repository. This repository will handle files checked in with the source code. Currently planned formats are: HTML, AsciiDoc and Markdown. Implementing a Confluence repository first would take much longer and does not give us any advantages for proofing LivingDoc 2’s quality.

Scope:

This story covers two things which need implementing:

1. The Document class. This is an abstraction over all possible documentation input. Any source documentation needs to be transformed / parsed into this data structure. It is also the basis for further processing of the documentation as examples.
2. The FileRepository class. This is the implementation of DocumentRepository for the file system.

There is a base implementation of this in PR #75.

Requirements for the Document implementation:

• Any textual information which is not an example is still available for processing.
• can represent DecisionTable as well as Scenario examples
• The data structure and its parser are extensible (there will be changes and additions to the structure in the future.
• The Document class contains all the information of the source document, but not necessarily all formatting. Line breaks and paragraphs in texts should be kept, but colours and other mark-up does not have to be considered.

Requirements for the FileRepository implementation:

• Provide fun getDocument(documentIdentifier: String): Document? for document lookup
• Support for different DocumentFormat (HTML, Markdown, AsciiDoc)
• Root directory and DocumentFormat can be specified in the repository configuration

[slu-it]

@BalazsAtWork @Alexander-Miller This is your story :) At least to get you started. I'll add some more acceptance criteria next week and maybe some examples.
@Alexander-Miller Feel free to correct any spelling errors you find ;)

[bitterblue]

Feels pretty epic to me :)
Why not split the LocalRepository into different repositories depending on format?

[slu-it]

@bitterblue lets workshop this on friday

[bitterblue]

We discussed today that DocumentRepository should be responsible for access to a data store (Confluence, local file system) and provide a stream of content to a DocumentFormat instance (HtmlDocumentFormat, MarkdownDocumentFormat, ...). The responsibility of DocumentFormat is to parse a content stream into our object model of a Document.

[Alexander-Miller]

Now that we have working parser implementations, and for the sake of getting closer to running an end-to-end test, let's have a look a the existing class hierarchy. I do believe there are questions that need to be answered and improvements and refactorings that can be made.

At the top there's the DocumentRepository which may or may not return a Document. Questions:

• Why is the returned Document nullable?
• What is the documentIndentifier that needs to be supplied?

The concrete implementations of the DocumentRepository have a DocumentFormat which knows how to provide a Document (non-nullable), given that they receive an appropriate InputStream. Who is responsible for supplying that InputStream?

I would also like to propose some refactoring. As I see it there's 2 questions that need answering that I think are badly answered with the current types: where is the Document and what type of Document is it? I propose we introduce interface(s) for both. The former, let's call it DocumentLocation could know how to supply an InputStream, the latter is already available as the DocumentFormat. A repository would then no longer need subtypes but could simply be instantiated with appropriate references.