Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable the implementation of Etcd during the build of ShardingSphere Proxy Native #28194

Merged
merged 1 commit into from
Aug 21, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions distribution/proxy-native/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,12 @@
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-proxy-bootstrap</artifactId>
<version>${project.version}</version>
<exclusions>
<exclusion>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-cluster-mode-repository-etcd</artifactId>
</exclusion>
</exclusions>
</dependency>

<dependency>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@ weight = 2

## 注意事项

- ShardingSphere Proxy 尚未准备好与 GraalVM Native Image 集成。
Proxy 的 Native Image 产物在 /~https://github.com/apache/shardingsphere/pkgs/container/shardingsphere-proxy-native 存在每夜构建。
假设存在包含`server.yaml``conf` 文件夹为 `./custom/conf`,你可通过如下的 `docker-compose.yml` 文件进行测试。
- ShardingSphere Proxy 尚未准备好与 GraalVM Native Image 集成。 Proxy 的 Native Image 产物在
/~https://github.com/apache/shardingsphere/pkgs/container/shardingsphere-proxy-native 存在每夜构建。假设存在包
`server.yaml``conf` 文件夹为 `./custom/conf`,你可通过如下的 `docker-compose.yml` 文件进行测试。

```yaml
version: "3.8"
Expand Down Expand Up @@ -44,6 +44,9 @@ services:
- 本节假定处于 Linux(amd64,aarch64), MacOS(amd64)或 Windows(amd64)环境。
如果你位于 MacOS(aarch64/M1) 环境,你需要关注尚未关闭的 /~https://github.com/oracle/graal/issues/2666 。

- `org.apache.shardingsphere:shardingsphere-cluster-mode-repository-etcd`
/~https://github.com/micronaut-projects/micronaut-gcp/issues/532 影响,不可使用。

## 前提条件

1. 根据 https://www.graalvm.org/downloads/ 要求安装和配置 JDK 17 对应的 `GraalVM Community Edition` 或 `Oracle GraalVM`
Expand Down Expand Up @@ -148,16 +151,14 @@ services:

# 可观察性

- 针对 GraalVM Native Image 形态的 ShardingSphere
Proxy,其提供的可观察性的能力与 https://shardingsphere.apache.org/document/current/cn/user-manual/shardingsphere-proxy/observability/
并不一致。
- 针对 GraalVM Native Image 形态的 ShardingSphere Proxy,其提供的可观察性的能力与
https://shardingsphere.apache.org/document/current/cn/user-manual/shardingsphere-proxy/observability/ 并不一致。

- 你可以使用 https://www.graalvm.org/latest/tools/ 提供的一系列命令行工具或可视化工具观察 GraalVM Native Image
的内部行为,并根据其要求使用 VSCode 完成调试工作。
如果你正在使用 IntelliJ IDEA 并且希望调试生成的 GraalVM Native
Image,你可以关注 https://blog.jetbrains.com/idea/2022/06/intellij-idea-2022-2-eap-5/#Experimental_GraalVM_Native_Debugger_for_Java
及其后继。如果你使用的不是 Linux,则无法对 GraalVM Native Image 进行
Debug,请关注尚未关闭的 /~https://github.com/oracle/graal/issues/5648 。
- 你可以使用 https://www.graalvm.org/latest/tools/ 提供的一系列命令行工具或可视化工具观察 GraalVM Native Image 的内部行为,
并根据其要求使用 VSCode 完成调试工作。如果你正在使用 IntelliJ IDEA 并且希望调试生成的 GraalVM Native Image,你可以关注
https://blog.jetbrains.com/idea/2022/06/intellij-idea-2022-2-eap-5/#Experimental_GraalVM_Native_Debugger_for_Java
及其后继。如果你使用的不是 Linux,则无法对 GraalVM Native Image 进行 Debug,请关注尚未关闭的
/~https://github.com/oracle/graal/issues/5648 。

- 对于使用 `ShardingSphere Agent` 等 APM Java Agent 的情形, GraalVM 的 `native-image` 组件尚未完全支持在构建 Native
Image 时使用 javaagent,你需要关注尚未关闭的 /~https://github.com/oracle/graal/issues/1065。
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ corresponding `Docker Image` through the `native-image` component of `GraalVM`.

## Notice

- ShardingSphere Proxy is not yet ready to integrate with GraalVM Native Image.
Proxy's Native Image artifacts are built nightly at /~https://github.com/apache/shardingsphere/pkgs/container/shardingsphere-proxy-native .
- ShardingSphere Proxy is not yet ready to integrate with GraalVM Native Image. Proxy's Native Image artifacts are
built nightly at /~https://github.com/apache/shardingsphere/pkgs/container/shardingsphere-proxy-native .
Assuming there is a `conf` folder containing `server.yaml` as `./custom/conf`, you can test it with the
following `docker-compose.yml` file.

Expand All @@ -27,35 +27,39 @@ services:
- "3307:3307"
````

- If you find that the build process has missing GraalVM Reachability Metadata,
a new issue should be opened at /~https://github.com/oracle/graalvm-reachability-metadata,
and submit a PR containing GraalVM Reachability Metadata missing from ShardingSphere itself or dependent third-party libraries.
- If you find that the build process has missing GraalVM Reachability Metadata, a new issue should be opened
at /~https://github.com/oracle/graalvm-reachability-metadata, and submit a PR containing GraalVM Reachability Metadata
missing from ShardingSphere itself or dependent third-party libraries.

- The master branch of ShardingSphere is not yet ready to handle unit tests in Native Image,
you always need to build GraalVM Native Image in the process,
Plus `-DskipNativeTests` or `-DskipTests` parameter specific to `GraalVM Native Build Tools` to skip unit tests in
Native Image.

- The following algorithm classes are not available under GraalVM Native Image due to /~https://github.com/oracle/graal/issues/5522 involved.
- The following algorithm classes are not available under GraalVM Native Image due
to /~https://github.com/oracle/graal/issues/5522 involved.
- `org.apache.shardingsphere.sharding.algorithm.sharding.inline.InlineShardingAlgorithm`
- `org.apache.shardingsphere.sharding.algorithm.sharding.inline.ComplexInlineShardingAlgorithm`
- `org.apache.shardingsphere.sharding.algorithm.sharding.hint.HintInlineShardingAlgorithm`

- At this stage, ShardingSphere Proxy in the form of GraalVM Native Image does not support the use of `Row Value Expressions`
with Groovy syntax, which first results in the `actualDataNodes` property of the `Sharding` feature being only
configurable using a pure list, such as `ds_0.t_order_0, ds_0.t_order_1` or `ds_0.t_user_0, ds_15.t_user_1023`. This
issue is tracked in /~https://github.com/oracle/graal/issues/5522 .
- At this stage, ShardingSphere Proxy in the form of GraalVM Native Image does not support the use
of `Row Value Expressions` with Groovy syntax, which first results in the `actualDataNodes` property of the `Sharding`
feature being only configurable using a pure list, such as `ds_0.t_order_0, ds_0.t_order_1`
or `ds_0.t_user_0, ds_15.t_user_1023`. This issue is tracked in /~https://github.com/oracle/graal/issues/5522 .

- This section assumes a Linux (amd64, aarch64), MacOS (amd64) or Windows (amd64) environment.
If you are on MacOS (aarch64/M1) environment, you need to follow /~https://github.com/oracle/graal/issues/2666 which is
not closed yet.

- 'org.apache.shardingsphere:shardingsphere-cluster-mode-repository-etcd' is affected by
/~https://github.com/micronaut-projects/micronaut-gcp/issues/532 and cannot be used.

## Premise

1. Install and configure `GraalVM Community Edition` or `Oracle GraalVM` for JDK 17 according
to https://www.graalvm.org/downloads/. Or use `SDKMAN!`. If you wish to use `Oracle GraalVM`
with [GraalVM Free Terms and Conditions license](https://www.oracle.com/downloads/licenses/graal-free-license.html),
the following command should be changed to `sdk install java 17.0.8-graal`
the following command should be changed to `sdk install java 17.0.8-graal`.

```shell
sdk install java 17.0.8-graalce
Expand Down Expand Up @@ -118,12 +122,11 @@ sdk install java 17.0.8-graalce
./mvnw -am -pl distribution/proxy-native -B -T1C -Prelease.native -DskipTests clean package
```

3. To start Native Image through the command line, you need to bring 4 parameters.
The first parameter is the port used by ShardingSphere Proxy, the second parameter is the `/conf` folder
containing `server.yaml` written by you, the third parameter is the Address of the bound port, and the fourth parameter is
Force Start, if it is true, it will ensure that ShardingSphere Proxy Native can start normally no matter whether it
is connected or not.
Assuming the folder `./custom/conf` already exists, the example is
3. To start Native Image through the command line, you need to bring 4 parameters. The first parameter is the `Port`
used by ShardingSphere Proxy, the second parameter is the `/conf` folder containing `server.yaml` written by you, the
third parameter is the `Address` of the bound port, and the fourth parameter is `Force Start`, if it is true, it will
ensure that ShardingSphere Proxy Native can start normally no matter whether it is connected or not. Assuming the
folder `./custom/conf` already exists, the example is

```bash
./apache-shardingsphere-proxy-native 3307 ./custom/conf "0.0.0.0" false
Expand Down Expand Up @@ -151,13 +154,14 @@ services:
- "3307:3307"
```

- If you don't make any changes to the Git Source, the commands mentioned above will use `oraclelinux:9-slim` as the Base Docker Image.
But if you want to use a smaller Docker Image like `busybox:glic`, `gcr.io/distroless/base` or `scratch` as the Base Docker Image, you need according
- If you don't make any changes to the Git Source, the commands mentioned above will use `oraclelinux:9-slim` as the
Base Docker Image. But if you want to use a smaller Docker Image like `busybox:glic`, `gcr.io/distroless/base` or
`scratch` as the Base Docker Image, you need according
to https://www.graalvm.org/latest/reference-manual/native-image/guides/build-static-executables/,
add operations such as `-H:+StaticExecutableWithDynamicLibC` to `jvmArgs` as the `native profile` of `pom.xml`.
Also note that some 3rd-party dependencies will require more system libraries such as `libdl` to be installed in
the `Dockerfile`.
So make sure to tune `distribution/proxy-native` according to your usage `pom.xml` and `Dockerfile` below.
the `Dockerfile`. So make sure to tune `distribution/proxy-native` according to your usage `pom.xml` and `Dockerfile`
below.

# Observability

Expand All @@ -167,8 +171,8 @@ services:

- You can observe GraalVM Native Image using a series of command line tools or visualization tools available
at https://www.graalvm.org/latest/tools/, and use VSCode to debug it according to its requirements.
If you are using IntelliJ IDEA and want to debug the generated GraalVM Native Image, You can
follow https://blog.jetbrains.com/idea/2022/06/intellij-idea-2022-2-eap-5/#Experimental_GraalVM_Native_Debugger_for_Java
If you are using IntelliJ IDEA and want to debug the generated GraalVM Native Image, You can follow
https://blog.jetbrains.com/idea/2022/06/intellij-idea-2022-2-eap-5/#Experimental_GraalVM_Native_Debugger_for_Java
and its successors. If you are not using Linux, you cannot debug GraalVM Native Image, please pay attention
to /~https://github.com/oracle/graal/issues/5648 which has not been closed yet.

Expand Down