Supports serving for PPMiniLM #1620

LiuChiachi · 2022-01-21T07:37:09Z

PR types

New features

PR changes

Models

Description

Supports Serving for PPMiniLM

This PR could run if these PRs are merged:

Paddle: Make var of STRINGS dtype able to get lod_level property Paddle#39077
Serving: Supports prediction for string list input Serving#1598
Serving: Supports string input export Serving#1597

add copyright for serving reorganize

ZeyuChen · 2022-01-22T06:43:14Z

examples/model_compression/pp-minilm/deploy/serving/web_service.py

+
+class PPMiniLMOp(Op):
+    def init_op(self):
+        import paddlenlp as ppnlp


from paddlenlp.transformers import PPMiniLMTokenizer

Thanks. Done.

…nto add-ppminilm-serving

ZeyuChen · 2022-01-25T08:40:57Z

examples/model_compression/pp-minilm/deploy/serving/README.md

@@ -0,0 +1,82 @@
+# PP-MiniLM 使用 Paddle Serving API 进行推理


PP-MiniLM 使用 Paddle Serving 进行服务化部署

Paddle Serving不是个API，也不是推理了，是服务化部署

收到，感谢指出，已修改

ZeyuChen · 2022-01-26T08:21:33Z

examples/model_compression/pp-minilm/README.md

@@ -394,6 +403,12 @@ cd ..



+<a name="使用PaddleServing预测"></a>
+
+### 使用 Paddle Serving 进行预测


使用 Paddle Serving 进行服务化部署

感谢，已修改:)

ZeyuChen · 2022-01-26T08:21:51Z

examples/model_compression/pp-minilm/README.md


-### 预测
+### 使用 Paddle Inference 进行预测


使用 Paddle Inference 进行推理部署

ZeyuChen · 2022-01-26T08:22:50Z

examples/model_compression/pp-minilm/deploy/serving/README.md

+| ppminilm.pdiparams      | 模型权重文件，供推理时加载使用            |
+| ppminilm.pdmodel        | 模型结构文件，供推理时加载使用            |
+
+假设这 2 个文件已生成，其中模型是集成了 FasterTokenizer 算子的模型，并放在在目录 `$MODEL_DIR` 下


其中模型是集成了 FasterTokenizer 算子的模型
这个前提需要让用户感知吗？

不需要，已删除。默认导出的模型是带 FasterTokenizer 算子的模型，所以不需要强调。之后lite那边再单独写一下目前只支持不带 FasterTokenizer 算子。

ZeyuChen · 2022-01-26T08:24:49Z

examples/model_compression/pp-minilm/deploy/serving/README.md

+
+使用 Paddle Serving 需要在服务器端安装相关模块，需要 v0.8.0 之后的版本：
+```shell
+pip install paddle-serving-app paddle-serving-client paddle-serving-server paddlepaddle


为什么还要引导安装paddlepaddle呢？这个地方是否会跟paddlenlp本身已安装的paddle冲突呢？

感谢指出，已经修改

ZeyuChen · 2022-01-26T08:25:40Z

examples/model_compression/pp-minilm/deploy/serving/README.md

+在启动预测之前，需要按照自己的情况修改 config 文件中的配置，主要需要修改的配置释义如下：
+
+- `rpc_port` : rpc端口。
+- `device_type` : 0 代表 cpu, 1 代表 gpu, 2 代表 tensorRT, 3 代表 arm cpu, 4 代表 kunlun xpu。


CPU GPU TensorRT Arm CPU， Kunlun XPU
注意术语的标准写法。

感谢，已经修改

ZeyuChen · 2022-01-26T08:27:13Z

examples/model_compression/pp-minilm/deploy/serving/rpc_client.py

@@ -0,0 +1,35 @@
+# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.


这个文件是干什么呢的？我看文档没有对RPC client进行介绍
那现在是默认只使用websevice是吗？

web service起服务，然后rpc client发请求，两个需要配套使用。

ZeyuChen · 2022-01-26T08:28:22Z

examples/model_compression/pp-minilm/deploy/serving/README.md

+
+```
+
+## 启动 client 进行推理


Serving的逻辑，启动客户端不是为了推理，是为了获取服务端的结果
这些标题从技术角度看不严谨。

感谢指出，已经修改成“启动 client 发起推理请求“

ZeyuChen · 2022-01-26T08:28:48Z

examples/model_compression/pp-minilm/README.md

            * [环境要求](#环境要求)
            * [运行方式](#运行方式)
            * [性能测试](#性能测试)
+        * [使用 Paddle Serving 预测](#使用PaddleServing预测)


部署和预测，是两码事。

…nto add-ppminilm-serving

tianxin1860

LGTM

supports serving

f5f25fc

add copyright for serving reorganize

LiuChiachi changed the title ~~Supports serving~~ Supports serving for PPMiniLM Jan 21, 2022

fix dir

6f45423

LiuChiachi marked this pull request as ready for review January 21, 2022 08:37

ZeyuChen reviewed Jan 22, 2022

View reviewed changes

LiuChiachi added 2 commits January 24, 2022 12:09

add readme

b598e8a

Merge branch 'develop' of /~https://github.com/PaddlePaddle/PaddleNLP i…

28ff12c

…nto add-ppminilm-serving

LiuChiachi requested a review from ZeyuChen January 24, 2022 12:24

LiuChiachi added 2 commits January 24, 2022 13:36

update readme

2d25f73

update serving readme

52d7817

LiuChiachi requested a review from guoshengCS January 25, 2022 05:45

LiuChiachi assigned ZeyuChen and guoshengCS Jan 26, 2022

ZeyuChen reviewed Jan 26, 2022

View reviewed changes

LiuChiachi added 4 commits January 29, 2022 06:41

fix some expression in serving readme

95f8f85

Merge branch 'develop' of /~https://github.com/PaddlePaddle/PaddleNLP i…

5e02121

…nto add-ppminilm-serving

fix serving readme

b373677

Merge branch 'develop' into add-ppminilm-serving

9af4945

tianxin1860 approved these changes Feb 17, 2022

View reviewed changes

Merge branch 'develop' into add-ppminilm-serving

bb01c0e

LiuChiachi merged commit 9ac1714 into PaddlePaddle:develop Feb 17, 2022

LiuChiachi mentioned this pull request Mar 17, 2022

PaddleNLP 2.2.5 Release Note Candidate #1772

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Supports serving for PPMiniLM #1620

Supports serving for PPMiniLM #1620

LiuChiachi commented Jan 21, 2022 •

edited

Loading

ZeyuChen Jan 22, 2022

LiuChiachi Jan 24, 2022

ZeyuChen Jan 25, 2022

LiuChiachi Jan 29, 2022

ZeyuChen Jan 26, 2022

LiuChiachi Jan 29, 2022

ZeyuChen Jan 26, 2022

ZeyuChen Jan 26, 2022

LiuChiachi Jan 29, 2022

ZeyuChen Jan 26, 2022

LiuChiachi Jan 29, 2022

ZeyuChen Jan 26, 2022

LiuChiachi Jan 29, 2022

ZeyuChen Jan 26, 2022

LiuChiachi Jan 29, 2022

ZeyuChen Jan 26, 2022

LiuChiachi Jan 29, 2022

ZeyuChen Jan 26, 2022

tianxin1860 left a comment

		@@ -0,0 +1,82 @@
		# PP-MiniLM 使用 Paddle Serving API 进行推理

		@@ -394,6 +403,12 @@ cd ..



		<a name="使用PaddleServing预测"></a>

		### 使用 Paddle Serving 进行预测

		@@ -0,0 +1,35 @@
		# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.

Supports serving for PPMiniLM #1620

Supports serving for PPMiniLM #1620

Conversation

LiuChiachi commented Jan 21, 2022 • edited Loading

PR types

PR changes

Description

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

PP-MiniLM 使用 Paddle Serving 进行服务化部署

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tianxin1860 left a comment

Choose a reason for hiding this comment

LiuChiachi commented Jan 21, 2022 •

edited

Loading