Version incompatibility with main OpenSearch

[ashwin-pc]

Describe the bug

Right now if you clone the latest version of OpenSearch and OpenSearch Dashboards and try to run it, you get the following error

This version of OpenSearch Dashboards (v1.2.0) is incompatible with the following OpenSearch nodes in your cluster: v2.0.0 @ 127.0.0.1:9200 (127.0.0.1)

This is because the latest commit on OpenSearch's main branch is already on 2.0 and OpenSearch Dashboards is 1.2. There is no compatible version of OpenSearch for OpenSearch Dashboards 1.2 unless you change its branch to 1.x and run that version of OpenSearch. This is not ideal since you want to be developing against either the latest version of OpenSearch or the latest released version (which is 1.1 in this case). Right now both these versions throw a version mismatch error and Dashboards fails to start up.

There is also no mention in the Developer docs on how to get around this issue.

To Reproduce
Steps to reproduce the behavior:

1. Clone main OpenSearch, build, and run it
2. Clone main OpenSearch Dashboards, install dependencies, and run it
3. See the console output for OpenSearch Dashboards when run

Expected behavior
Any one of the following options could be a valid expected behavior:

• Latest commit in OpenSearch Dashboards main should always be compatible with the latest released version of OpenSearch
• Last commit in OpenSearch Dashboards main should always run with the latest commit in OpenSearch main
• Developer docs should explain what to do in this situation, such as describe how to disable version check

OpenSearch Version
1.1 and 2.0

Dashboards Version
1.2

[tmarkley]

Thanks for bringing this issue up Ashwin! I don't think this is necessarily a bug, so I've changed the label. I think your first suggestion is the direction we want to go, the second is a little more tricky (albeit probably something we could make work) because of the different pace of changes in the main branch for each repository, and the third is a good immediate way to alleviate the pain.

This is not ideal since you want to be developing against either the latest version of OpenSearch or the latest released version

These are two distinct scenarios, but we maintainers typically just pull down the 1.x OpenSearch branch, or in this case fetch the v1.2.0 artifact, and run that.

I recognize that there will be situations where developers need to test the latest OpenSearch changes, but I'm not sure if there is a solution for guaranteeing that the latest commit on each repo's main branch is compatible. As mentioned above, versions will not always match perfectly.

Ideas for next steps:

1. Bump the version in the OpenSearch Dashboards main branch to 2.0 and add the required version checks.
2. Add specific instructions to the developer guide for each scenario - running the latest OpenSearch changes, the latest release, etc.

[kavilla]

To add on too we might need a version compatibility matrix with future plans.

Right now I'm assuming OpenSearch 2.x is compatible with OpenSearch Dashboards 1.x (no guarantees) so feel free to add to your opensearch_dashboards.yml the following:

opensearch.ignoreVersionMismatch: true

But yeah I agree with Tommy it probably be best to just let people know.

[ashwin-pc]

That sounds perfectly fine. I was able to get around the issue using @kavilla's suggestion but I wasn't sure if that would always be true.

If the solution for now is to let people know and use the ignore flag when such an issue happens, I'd suggest adding that to the developer docs and opensearch_dashboards.yml's comments. I was only able to find it because because I knew such a config existed and looked through the code to find it. It wasn't documented anywhere and isn't in the comment template for opensearch_dashboards.yml either.