Skip to content

Commit

Permalink
Update README
Browse files Browse the repository at this point in the history
  • Loading branch information
Fanoid committed Nov 30, 2020
1 parent 7d05a7c commit 1cf6749
Show file tree
Hide file tree
Showing 2 changed files with 40 additions and 45 deletions.
42 changes: 20 additions & 22 deletions README.en-US.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,55 +30,53 @@ Welcome everyone to join the Alink open source user group to communicate.
---------


About package names and versions:
#### About package names and versions:
- PyAlink provides different Python packages for Flink versions that Alink supports:
package `pyalink` always maintains Alink Python API against the latest Flink version, which is 1.11,
while `pyalink-flink-***` support old-version Flink, which are `pyalink-flink-1.10` and `pyalink-flink-1.9` for now.
- The version of python packages always follows Alink Java version, like `1.2.0`.
- The version of python packages always follows Alink Java version, like `1.3.0`.

Installation steps:
#### Installation steps:

1. Make sure the version of python3 on your computer is 3.6 or 3.7.
2. Make sure Java 8 is installed on your computer.
3. Use pip to install:
`pip install pyalink`, `pip install pyalink-flink-1.10` or `pip install pyalink-flink-1.9`.


Potential issues:
#### Potential issues:

1. `pyalink` and/or `pyalink-flink-***` can not be installed at the same time. Multiple versions are not allowed.
If `pyalink` or `pyalink-flink-***` was/were installed, please use `pip uninstall pyalink` or `pip uninstall pyalink-flink-***` to remove them.

2. If `pip install` is slow of failed, refer to [this article](https://segmentfault.com/a/1190000006111096) to change the pip source, or use the following download links:
- Flink 1.11:[Link](https://alink-release.oss-cn-beijing.aliyuncs.com/v1.2.0/pyalink-1.2.0-py3-none-any.whl) (MD5: 8a38e8009712afcbf5ecdb297db3c8ac)
- Flink 1.10:[Link](https://alink-release.oss-cn-beijing.aliyuncs.com/v1.2.0/pyalink_flink_1.10-1.2.0-py3-none-any.whl) (MD5: 8a38e8009712afcbf5ecdb297db3c8ac)
- Flink 1.9: [Link](https://alink-release.oss-cn-beijing.aliyuncs.com/v1.2.0/pyalink_flink_1.9-1.2.0-py3-none-any.whl) (MD5: 2f25eec02a692661c0f0f3a89e5c2f0c)
- Flink 1.11:[Link](https://alink-release.oss-cn-beijing.aliyuncs.com/v1.3.0/pyalink-1.3.0-py3-none-any.whl) (MD5: 1e5fb63c798a4aafe4461505521ac79a)
- Flink 1.10:[Link](https://alink-release.oss-cn-beijing.aliyuncs.com/v1.3.0/pyalink_flink_1.10-1.3.0-py3-none-any.whl) (MD5: f0d35a4c3500db0e52c390ed1ab830c5)
- Flink 1.9: [Link](https://alink-release.oss-cn-beijing.aliyuncs.com/v1.3.0/pyalink_flink_1.9-1.3.0-py3-none-any.whl) (MD5: 3bfbef09e5d5147d2db2aeba785f3ba6)
3. If multiple version of Python exist, you may need to use a special version of `pip`, like `pip3`;
If Anaconda is used, the command should be run in Anaconda prompt.


Download file system and Hive dependency jar files:
#### Download file system and Catalog dependency jar files:

After PyAlink installed, you can run ```download_pyalink_dep_jars``` to download dependency jars for file system and Hive.

If there is an error that could not find the command, you can run the python command ```python3 -c 'from pyalink.alink.download_pyalink_dep_jars import main;main()'``` directly.
(If there is an error that could not find the command, you can run the python command ```python3 -c 'from pyalink.alink.download_pyalink_dep_jars import main;main()'``` directly.)

After executed the command, you'll see a prompt asking you about the dependencies and their versions to be downloaded.
The following dependencies and their versions of jars are supported:

- OSS:1.10.0
- Hadoop:2.4.1, 2.6.5, 2.7.5, 2.8.3
- Hive:2-0-v0.1, 2-1-v0.1, 2-2-v0.1, 2-3-v0.1, 3-1-v0.1
- OSS:3.4.1
- Hadoop:2.8.3
- Hive:2.3.4
- MySQL: 5.1.27
- Derby: 10.6.1.0
- SQLite: 3.19.3

These jars will be installed to the ```lib``` folder of PyAlink.
These jars will be installed to the ```lib/plugins``` folder of PyAlink.
Note that these command require the access for the folder.

You can also add the argument ```-d``` when executing the command, i.e. ```download_pyalink_dep_jars -d```.
It will install the dependency jars with default versions:

- OSS:1.10.0
- Hadoop:2.8.3
- Hive:2-3-v0.1
It will install all dependency jars.

### Start using:
-------
Expand Down Expand Up @@ -156,7 +154,7 @@ pipeline.fit(data).transform(data).print();
<dependency>
<groupId>com.alibaba.alink</groupId>
<artifactId>alink_core_flink-1.11_2.11</artifactId>
<version>1.2.0</version>
<version>1.3.0</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
Expand All @@ -175,7 +173,7 @@ pipeline.fit(data).transform(data).print();
<dependency>
<groupId>com.alibaba.alink</groupId>
<artifactId>alink_core_flink-1.10_2.11</artifactId>
<version>1.2.0</version>
<version>1.3.0</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
Expand All @@ -195,7 +193,7 @@ pipeline.fit(data).transform(data).print();
<dependency>
<groupId>com.alibaba.alink</groupId>
<artifactId>alink_core_flink-1.9_2.11</artifactId>
<version>1.2.0</version>
<version>1.3.0</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
Expand Down
43 changes: 20 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,48 +28,45 @@
### 使用前准备:
---------

包名和版本说明:
#### 包名和版本说明:

- PyAlink 根据 Alink 所支持的 Flink 版本提供不同的 Python 包:
其中,`pyalink` 包对应为 Alink 所支持的最新 Flink 版本,当前为 1.11,而 `pyalink-flink-***` 为旧版本的 Flink 版本,当前提供 `pyalink-flink-1.10``pyalink-flink-1.9`
- Python 包的版本号与 Alink 的版本号一致,例如`1.2.0`
- Python 包的版本号与 Alink 的版本号一致,例如`1.3.0`

安装步骤:
####安装步骤:
1. 确保使用环境中有Python3,版本限于 3.6 和 3.7。
2. 确保使用环境中安装有 Java 8。
3. 使用 pip 命令进行安装:
`pip install pyalink``pip install pyalink-flink-1.10` 或者 `pip install pyalink-flink-1.9`

安装注意事项:
#### 安装注意事项:

1. `pyalink``pyalink-flink-***` 不能同时安装,也不能与旧版本同时安装。
如果之前安装过 `pyalink` 或者 `pyalink-flink-***`,请使用`pip uninstall pyalink` 或者 `pip uninstall pyalink-flink-***` 卸载之前的版本。
2. 出现`pip`安装缓慢或不成功的情况,可以参考[这篇文章](https://segmentfault.com/a/1190000006111096)修改pip源,或者直接使用下面的链接下载 whl 包,然后使用 `pip` 安装:
- Flink 1.11:[链接](https://alink-release.oss-cn-beijing.aliyuncs.com/v1.2.0/pyalink-1.2.0-py3-none-any.whl) (MD5: 8a38e8009712afcbf5ecdb297db3c8ac)
- Flink 1.10:[链接](https://alink-release.oss-cn-beijing.aliyuncs.com/v1.2.0/pyalink_flink_1.10-1.2.0-py3-none-any.whl) (MD5: 8a38e8009712afcbf5ecdb297db3c8ac)
- Flink 1.9: [链接](https://alink-release.oss-cn-beijing.aliyuncs.com/v1.2.0/pyalink_flink_1.9-1.2.0-py3-none-any.whl) (MD5: 2f25eec02a692661c0f0f3a89e5c2f0c)
- Flink 1.11:[链接](https://alink-release.oss-cn-beijing.aliyuncs.com/v1.3.0/pyalink-1.3.0-py3-none-any.whl) (MD5: 1e5fb63c798a4aafe4461505521ac79a)
- Flink 1.10:[链接](https://alink-release.oss-cn-beijing.aliyuncs.com/v1.3.0/pyalink_flink_1.10-1.3.0-py3-none-any.whl) (MD5: f0d35a4c3500db0e52c390ed1ab830c5)
- Flink 1.9: [链接](https://alink-release.oss-cn-beijing.aliyuncs.com/v1.3.0/pyalink_flink_1.9-1.3.0-py3-none-any.whl) (MD5: 3bfbef09e5d5147d2db2aeba785f3ba6)
3. 如果有多个版本的 Python,可能需要使用特定版本的 `pip`,比如 `pip3`;如果使用 Anaconda,则需要在 Anaconda 命令行中进行安装。

安装文件系统和 Hive 依赖 jar 包:
#### 下载安装文件系统或 Catalog 依赖 jar 包

安装 PyAlink 之后,可以直接运行 ```download_pyalink_dep_jars``` 命令,下载支持文件系统功能所需要的 jar 包。

如果提示找不到这个命令,可以尝试直接运行脚本: ```python3 -c 'from pyalink.alink.download_pyalink_dep_jars import main;main()'```
(如果提示找不到这个命令,可以尝试直接运行脚本: ```python3 -c 'from pyalink.alink.download_pyalink_dep_jars import main;main()'```。)

运行这个命令后,将提问是否安装某种文件系统对应的 jar 包,并选择合适的版本。 当前支持的文件系统包括:

- OSS:1.10.0
- Hadoop:2.4.1, 2.6.5, 2.7.5, 2.8.3
- Hive:2-0-v0.1, 2-1-v0.1, 2-2-v0.1, 2-3-v0.1, 3-1-v0.1

这些 jar 包将被下载到 PyAlink 安装路径的 ```lib``` 目录下。
注意,这里要求运行命令时有 PyAlink 安装目录的权限。
- OSS:3.4.1
- Hadoop:2.8.3
- Hive:2.3.4
- MySQL: 5.1.27
- Derby: 10.6.1.0
- SQLite: 3.19.3

运行命令时,也可以增加参数:```download_pyalink_dep_jars -d```,将下载默认的 jar 包:
这些 jar 包将被下载到 PyAlink 安装路径的 ```lib/plugins``` 目录下,所以要求运行命令时有 PyAlink 安装目录的权限。

- OSS:1.10.0
- Hadoop:2.8.3
- Hive:2-3-v0.1
运行命令时,也可以增加参数:```download_pyalink_dep_jars -d```,将自动下载所有的 jar 包。

### 开始使用:
-------
Expand Down Expand Up @@ -142,7 +139,7 @@ pipeline.fit(data).transform(data).print();
<dependency>
<groupId>com.alibaba.alink</groupId>
<artifactId>alink_core_flink-1.11_2.11</artifactId>
<version>1.2.0</version>
<version>1.3.0</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
Expand All @@ -161,7 +158,7 @@ pipeline.fit(data).transform(data).print();
<dependency>
<groupId>com.alibaba.alink</groupId>
<artifactId>alink_core_flink-1.10_2.11</artifactId>
<version>1.2.0</version>
<version>1.3.0</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
Expand All @@ -181,7 +178,7 @@ pipeline.fit(data).transform(data).print();
<dependency>
<groupId>com.alibaba.alink</groupId>
<artifactId>alink_core_flink-1.9_2.11</artifactId>
<version>1.2.0</version>
<version>1.3.0</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
Expand Down

0 comments on commit 1cf6749

Please sign in to comment.