Design Doc: Session #3993

helinwang · 2017-09-10T22:13:31Z

Here is better for review.

typhoonzero · 2017-09-11T02:21:16Z

doc/design/session.md

+
+## Abstract
+
+This design doc proposes to have an object called *Session* which


Is these design came out from #3811 (comment)?

If so, you probably need to add descriptions like "session is able to distinguish running a graph locally or remotely, using CPU or GPU, using one device or more"

Sorry for the late reply. Great point! Added "The Session is able to distinguish running a graph locally or remotely, using CPU only or using one or more GPUs."

typhoonzero · 2017-09-11T02:27:02Z

doc/design/session.md

+## Background
+
+A computation graph is executed in an environment which contains the
+[scope](./scope.md) and other states. PaddlePaddle used to only have


Sorry, what do you mean by "other states"?

scope, device, context etc.

I have the same question with @typhoonzero , @jacquesqiao do you mean the Session contains runtime resources?
e.g. already allocated memory in scope, occupied device, etc?

Sorry for the late reply. @dzhwinter The environment contains runtime resources. The session is a "owner" of these runtime resources.

typhoonzero · 2017-09-11T02:33:59Z

doc/design/session.md

+a = paddle.constant(1.0)
+b = paddle.constant(2.0)
+c = a + b
+sess = paddle.session()


Some thoughts of mine:

An NN training job must contain: graph(containing sub nets like init_net, forward_net, backward_net, opt_net); scope containing tensors as parameters; hyper parameters(learning_rate, batch_size, etc.); settings(cluster or not, devices, quotas, node ip address etc.), so here may be something like:
sess = paddle.session(graph, scope_list, settings)
# or
sess = paddle.remote_session(graph, scop_list, cluster_settings)

states can be stored all in some "scope" by creating scope for storing tensor for forward and backward, and for storing hyper parameters changing states.

Agree with

sess = paddle.session(graph, scope_list, settings) # or sess = paddle.remote_session(graph, scop_list, cluster_settings)

states can be stored all in some "scope" by creating scope for storing tensor for forward and backward, and for storing hyper parameters changing states.

Yes the variable states are stored in "scope". One session means one scope (just added into the doc).

Superjomn · 2017-09-11T04:18:58Z

doc/design/session.md

+## Background
+
+A computation graph is executed in an environment which contains the
+[scope](./scope.md) and other states. PaddlePaddle used to only have


need to specify one session contains one scope?

Sorry for the late reply. Added "This indicates different sessions have different scopes.".

wangkuiyi

Thanks for this PR! But after reading it, still I couldn't tell what must be included in a Session. Though it seems that this information should appear in the first paragraph of this document?

wangkuiyi · 2017-09-11T04:23:12Z

doc/design/session.md

+
+## Background
+
+A computation graph is executed in an environment which contains the


Try not to use passive voice.

A computation graph runs in an environment

Sorry for the late reply! Done.

wangkuiyi · 2017-09-11T04:23:51Z

doc/design/session.md

+## Background
+
+A computation graph is executed in an environment which contains the
+[scope](./scope.md) and other states. PaddlePaddle used to only have


"used to only have" => "used only to have" or "used to have only". Indeed, here we are describing the current statues, so it should be

The current design has an implicit session ...

Thanks! Done.

wangkuiyi · 2017-09-11T04:25:24Z

doc/design/session.md

+
+## Session
+
+Session is an object that owns all runtime states such as scope,


Session is an object ==> A session is an object

Thanks! Done.

wangkuiyi · 2017-09-11T04:27:40Z

doc/design/session.md

+[scope](./scope.md) and other states. PaddlePaddle used to only have
+an implicit global session on which `paddle.eval()` is executed.
+
+This has the limitation that the user can not create two independent


Here is a broken logic. It claimed but didn't explain why users cannot have two environments. The second sentence is to claim that it is necessary to have two environments.

From the text above, readers would ask it seems that what we need is two Scope instances, but not a new class Session.

Thanks! Explained why user cannot have two environments, and also changed wording so that reader will know we need a new class Session.

putcn · 2017-09-11T05:32:09Z

doc/design/session.md

+label = reader.column(1)
+fc1 = paddle.op.fc(image, size=256, act="sigmoid")
+fc2 = paddle.op.fc(fc1, size=10, act="softmax")
+cost = paddle.op.cross_entropy(fc2)


should this line be cost = paddle.op.cross_entropy(fc2, label) ?

Thanks! Done.

QiJune · 2017-09-11T05:57:30Z

It's better to give a C++ or Python class definition of Session.

dzhwinter · 2017-09-11T22:43:53Z

doc/design/session.md

+## Background
+
+A computation graph is executed in an environment which contains the
+[scope](./scope.md) and other states. PaddlePaddle used to only have


I have the same question with @typhoonzero , @jacquesqiao do you mean the Session contains runtime resources?
e.g. already allocated memory in scope, occupied device, etc?

dzhwinter · 2017-09-11T22:54:12Z

doc/design/session.md

+
+## Abstract
+
+This design doc proposes to have an object called *Session* which


Maybe the Session try to solve the problem of

unify the runtime sources management between a local machine and distributed environment.

Replace the global scope with a named concept, which holds resources explicitly.

is it correct ?

Yes.
Updated, please review:)

typhoonzero · 2017-09-26T02:09:40Z

doc/design/session.md

+
+## Session
+
+A session is an object that owns all runtime states such as scope,


Should define what a session owns exactly, like:

The "graph" to run locally or remotely.

Exactly one scope, containing all tensor variables for all the subnets.

Settings and hyper-parameters.

Thanks! Updated (the session owns the scope), please review.

The session owns a single global scope, but a scope can have sub-scope, so I did not specify one scope.
The session does not own the graph, the graph is what gets evaluated with session.
The session does not own settings and hyper-parameters, the session could be created from settings and hyper-parameters.

helinwang · 2017-09-28T01:20:40Z

@QiJune

It's better to give a C++ or Python class definition of Session.

Added Python interface, please review.

typhoonzero · 2017-09-28T01:32:13Z

doc/design/refactor/session.md

+
+Evaluates the target Operations or Variables in `targets`.
+
+- *targets*: the evaluation targets. Can be a single Operation or


Should targets be an instance of "Block"(graph)?

Targets will be a OP the output of a OP (Var). The "Block" (ProgramDesc to be exact) will be inferred by eval.
To make the relationship more clear, I have updated the PR, please take a look.

typhoonzero · 2017-09-28T01:33:59Z

doc/design/refactor/session.md

+
+  The computation graph is implicitly inferred from the targets.
+
+- *feed_dict*: a dictionary that contains the tensors which overrides


Is this the input data?

Yes, but not only the input data, can override any edge as well. E.g.,:

a = pd.constant(1.0, name="a") b = pd.constant(2.0) c = pd.mul(a,b) sess.eval(targets=c, feed_dict={"a":3.0}) # returns 6.0

I have added the above example into the design doc.

typhoonzero · 2017-09-28T01:34:34Z

doc/design/refactor/session.md

+close()
+```
+
+Closes the session. Calling this method releases the scope.


Need save() and load() also to do checkpointing.

save and load will be OPs, so user will need to run something like sess.eval(targets=save) or sess.eval(targets=load).

Similar to TF (which treats save and load as OPs), we can add syntax sugar, wrap the saving and loading model into something like:

tf.reset_default_graph() # Create some variables. v1 = tf.get_variable("v1", [3], initializer = tf.zeros_initializer) v2 = tf.get_variable("v2", [5], initializer = tf.zeros_initializer) # Add ops to save and restore only `v2` using the name "v2" saver = tf.train.Saver({"v2": v2}) # Use the saver object normally after that. with tf.Session() as sess: # Initialize v1 since the saver will not. v1.initializer.run() saver.restore(sess, "/tmp/model.ckpt") print("v1 : %s" % v1.eval()) print("v2 : %s" % v2.eval())

I think the syntax sugar will not be in the scope of this design doc (maybe more suited for Python API design doc).

typhoonzero · 2017-09-28T01:42:31Z

doc/design/refactor/session.md

+Creates a new session. One session owns one scope, so creating
+multiple sessions will create different scopes.
+
+- *gpu_ids*: a single `int` or a list of `int` of the GPU IDs to be


Setting up devices can take advantage of paddle v1 design, which default set gpu_ids to all of the available GPUs.

Can we use a string to specify devices, because there may be other devices than GPU, like FPGA, what TF does is /job:worker/task:1/gpu:0 also can be /job:worker/task:1/fpga:0

Great idea! Changed gpu_ids to devices.

I thinking what's the difference between Session and Executor. The Executor has a Run interface to execute a ProgramDesc created in compile-time. DeviceContext(CPUDeviceContext/CUDADeviceContext) is created and managed by Executor.
And Maybe Executor is a data member of Session. Session will get a target and some device ids. The target is parsed to get a ProgramDesc. Then, the ProgramDesc and device ids are passed to Executor. Executor will created a DeviceContextManeger according to device ids. At last, the ProgramDesc will be executed in specific hardwares.

@QiJune Great! That is a clear logic. We could add one more step ProgramOptimizer (currently called Converter) between Session and Executor. Please see this graph for more detail: /~https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/refactor/distributed_architecture.md#local-training-architecture (In the graph, "PaddlePaddle runtime" means the Executor. Btw, there are many names of the same thing, we need to decide on the naming).

After the offline discussion with @helinwang , I believe Session is

a manager of the resources, which is a more high-level concept than DeviceContextManager. It owns the resources.

The job handler.

it's will construct a new graph according to the targets user given.

Interact with remote cluster or local machine, fetch/feed tensor, start/stop running a graph....

The close interface allows user release the resource it owns.

The Executor is a runtime concept, which has nothing to do with Session.

it's will construct a new graph according to the targets user given.

The session will not "construct a new graph", it will send the graph to the converter, and the converter will construct a new graph.

dzhwinter

After the offline discussion with @helinwang , I believe Session is

a manager of the resources, which is a more high-level concept than DeviceContextManager. It owns the resources.
The job handler.
- it's will construct a new graph according to the targets user given.
- Interact with remote cluster or local machine, fetch/feed tensor, start/stop running a graph....
- The close interface allows user release the resource it owns.

The Executor is a runtime concept, which has nothing to do with Session.

helinwang · 2017-09-29T00:31:31Z

@dzhwinter

it's will construct a new graph according to the targets user given.

The session will not "construct a new graph", it will send the graph to the converter, and the converter will construct a new graph.

dzhwinter

Do we need to implement this module before the 0.11.0?

dzhwinter · 2017-10-04T19:02:37Z

doc/design/refactor/session.md

+  a = pd.constant(1.0, name="a")
+  b = pd.constant(2.0)
+  c = pd.mul(a,b)
+  sess.eval(targets=c, feed_dict={"a":3.0}) # returns 6.0


constant can be changed value looks weird. Maybe name them with variable is better?

Thanks! Done.

dzhwinter · 2017-10-04T19:07:10Z

doc/design/refactor/session.md

+)
+```
+
+Creates a new session. One session owns one scope, so creating


This statement is confusing. One session owns at least one scope, namely, global scope in one single session, right?
Or you mean that one session will have exactly one scope?

Yes, the global scope. Done.

dzhwinter · 2017-10-04T19:13:34Z

doc/design/refactor/session.md

+Creates a new session. One session owns one scope, so creating
+multiple sessions will create different scopes.
+
+- *devices*: a single `string` or a list of `string` of device names,


Maybe we really need a SessionConfig here? I mean, if one session cannot fully utilize the GPU resource, then another session may also own the same GPU.

In my view, we need to submit the config for this round of session run call. If it is a local run call, we provide the local config, vice versa, we provide the cluster config. Even more, if it is an inference run call, we may provide another config, which is totally different from those ones above.
We can leave these complexities to be solved in the future, but we need to figure the concept clear.

if one session cannot fully utilize the GPU resource, then another session may also own the same GPU.

Devices only means the devices that the session uses, multiple sessions can use the same device. I have added "Multiple sessions can use the same device." to clear up this point.

If it is a local run call, we provide the local config, vice versa, we provide the cluster config

Local session is created by paddle.session, remote session is created by paddle.remote_session.

if it is an inference run call, we may provide another config

In my view, inferencing is just user specifying a inference target, which is no different than training (specifying a training target). They should use the same kind of session. The layers below session should do the optimization (e.g., based on batch size) transparent to which session is used.

dzhwinter · 2017-10-04T21:00:28Z

We had discussed this module for a long-term and had reached an agreement, so it's better to merge it. For anyone who has any questions or different view, we can have an offline meeting. :)

dzhwinter

LGTM!

helinwang requested review from Superjomn, reyoung, wangkuiyi, jacquesqiao and typhoonzero September 10, 2017 22:13

helinwang force-pushed the session branch 2 times, most recently from 9de0cb3 to b411ae0 Compare September 10, 2017 22:17

typhoonzero reviewed Sep 11, 2017

View reviewed changes

Superjomn reviewed Sep 11, 2017

View reviewed changes

wangkuiyi reviewed Sep 11, 2017

View reviewed changes

putcn reviewed Sep 11, 2017

View reviewed changes

dzhwinter reviewed Sep 11, 2017

View reviewed changes

helinwang force-pushed the session branch 2 times, most recently from 5960fd2 to 526c9eb Compare September 25, 2017 23:09

typhoonzero reviewed Sep 26, 2017

View reviewed changes

helinwang added 2 commits September 27, 2017 17:28

Design Doc: Session

1c71005

fix according to comments

94dfd86

helinwang force-pushed the session branch from 526c9eb to 3691a7c Compare September 28, 2017 01:17

Update Session design doc

f24b5df

helinwang force-pushed the session branch from 3691a7c to f24b5df Compare September 28, 2017 01:23

typhoonzero reviewed Sep 28, 2017

View reviewed changes

helinwang force-pushed the session branch from 93049cd to 140b291 Compare September 28, 2017 22:21

update according to comments

757c76b

helinwang force-pushed the session branch from 140b291 to 757c76b Compare September 28, 2017 22:24

dzhwinter reviewed Sep 29, 2017

View reviewed changes

tonyyang-svail mentioned this pull request Oct 2, 2017

Add Executor #4524

Closed

dzhwinter reviewed Oct 4, 2017

View reviewed changes

fix according to comments

a9e298b

dzhwinter approved these changes Oct 4, 2017

View reviewed changes

helinwang merged commit 15b35f9 into PaddlePaddle:develop Oct 4, 2017

helinwang deleted the session branch October 4, 2017 21:34


		## Abstract

		This design doc proposes to have an object called Session which


		## Background

		A computation graph is executed in an environment which contains the


		## Session

		Session is an object that owns all runtime states such as scope,


		## Session

		A session is an object that owns all runtime states such as scope,


		Evaluates the target Operations or Variables in `targets`.

		- targets: the evaluation targets. Can be a single Operation or


		The computation graph is implicitly inferred from the targets.

		- feed_dict: a dictionary that contains the tensors which overrides

Design Doc: Session #3993

Design Doc: Session #3993

Conversation

helinwang commented Sep 10, 2017 • edited Loading

Choose a reason for hiding this comment

helinwang Sep 25, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wangkuiyi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

putcn Sep 11, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

QiJune commented Sep 11, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

helinwang Sep 28, 2017 • edited Loading

Choose a reason for hiding this comment

helinwang commented Sep 28, 2017

Choose a reason for hiding this comment

helinwang Sep 28, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

helinwang Sep 28, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

helinwang Sep 28, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

helinwang Sep 28, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dzhwinter Sep 28, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dzhwinter left a comment

Choose a reason for hiding this comment

helinwang commented Sep 29, 2017

dzhwinter left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dzhwinter commented Oct 4, 2017

dzhwinter left a comment

Choose a reason for hiding this comment

helinwang commented Sep 10, 2017 •

edited

Loading

helinwang Sep 25, 2017 •

edited

Loading

putcn Sep 11, 2017 •

edited

Loading

helinwang Sep 28, 2017 •

edited

Loading

helinwang Sep 28, 2017 •

edited

Loading

helinwang Sep 28, 2017 •

edited

Loading

helinwang Sep 28, 2017 •

edited

Loading

helinwang Sep 28, 2017 •

edited

Loading

dzhwinter Sep 28, 2017 •

edited

Loading