[MDS] Supporting OpenSearch plugin APIs in data source client.

[zengyan-amazon]

Is your feature request related to a problem? Please describe.

Today in OpenSearch Dashboards data source plugin, it provides dataSource.opensearch.getClient(dataSourceId) function for plugins to get OpenSearch client instance and subsequently make calls to OpenSearch.

Right now the OpenSearch client are plain client object which are only aware of pre-defined OpenSearch APIs defined by all module and plugins in core OpenSearch repo, other APIs provided by plugins in separate repo like sql are not supported out of the box. We need to support those plugin APIs so that corresponding Dashboards plugin can easily integrate with multi data source feature.

Describe the solution you'd like

We need to provide an function in data source plugin to allow other plugins to register their OpenSearch API definition, so that data source plugin can make those APIs available when data source plugin creating OpenSearch client.

Describe alternatives you've considered
An alternative is let plugins to get OpenSearch client from request context in request handler as it today, then allow them to extend the OpenSearch client to be aware of the plugin APIs. This is not ideal as it put too much work to other OpenSearch Dashboards plugins.

[kavilla]

+1

Brain dump ahead:

This might not be completely related but is there a reason why clients like the OpenSearch JS OpenSearch Dashboards extends didn't add the APIs from other plugins. I'm sure other folks in the community would love if the OpenSearch clients had these APIs defined. It would clean up the code a great deal within OpenSearch Dashboards plugins. I also see plugins essentially re-write the same lines of code when they have to cross interact.

cc: @wbeckler ^?

Even more unrelated to the issue here but more tied to MDS. Has there been consideration into converting some of the heavy lifting of OpenSearch Dashboards into OpenSearch [issue]. Might be too large of a project but I'd imagine it might solve some of the challenges presented by MDS when some of the OSD APIs are already available in OpenSearch.

My final thought would be it would be nice if this was apart of SEMVER SDK like:
/~https://github.com/opensearch-project/opensearch-dashboards-sdk-js/pull/47/files

[BionIT]

Happy to work on the issue

[seraphjiang]

Thanks @zengyan-amazon @kavilla for the ideas, and @BionIT for volunteer on the task. Let's quick come up use case to help scoping and immediate task.

maybe start from Observability? @joshuali925 @anirudha

[wbeckler]

The clients team has been working on generators based on API specs. If a plugin has a spec in Smithy or OpenAPI, then the js client could generate the relevant methods to call the plugin API correctly. Today, however, the plugins haven't yet produced those specs.