Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactorize framework/*.proto #3320

Closed
wangkuiyi opened this issue Aug 7, 2017 · 3 comments
Closed

Refactorize framework/*.proto #3320

wangkuiyi opened this issue Aug 7, 2017 · 3 comments
Assignees

Comments

@wangkuiyi
Copy link
Collaborator

Our CMakeLists file contains

proto_library(op_desc SRCS op_desc.proto DEPS attribute_proto)
cc_test(op_proto_test SRCS op_proto_test.cc DEPS op_proto protobuf)
cc_test(op_desc_test SRCS op_desc_test.cc DEPS op_desc protobuf)

cc_library(attribute SRCS attribute.cc DEPS op_desc op_proto)

cc_library(operator SRCS operator.cc DEPS op_desc device_context tensor scope attribute)
cc_test(operator_test SRCS operator_test.cc DEPS operator op_registry)

cc_library(grad_op_builder SRCS grad_op_builder.cc DEPS op_proto operator)
cc_library(op_registry SRCS op_registry.cc DEPS op_desc grad_op_builder)
cc_test(op_registry_test SRCS op_registry_test.cc DEPS op_registry)
cc_test(grad_op_builder_test SRCS grad_op_builder_test.cc DEPS grad_op_builder op_registry add_op)

py_proto_compile(framework_py_proto SRCS attribute.proto op_proto.proto op_desc.proto)

We can see that operator depends on op_desc and attribute, and the latter depends on op_proto as well. As this dependency propagates in following rules, any target that depends on op_desc also depends on op_desc. So we can merged these two proto files together.

@wangkuiyi
Copy link
Collaborator Author

Also both op_desc and op_proto depend on attribute_proto, these three proto files should be merged.

@wangkuiyi
Copy link
Collaborator Author

The current op_proto.proto file contains the following field in message VarProto

message VarProto {
  // Is that input/output could be a list or not.
  // If so, that Op should write a attributed named `input_format` or
  // `output_format`.
  //
  // e.g.
  //   If the op is a fc op, the inputs are `X`, `W`, `b`. The `X` and `W`
  //   could be multiple, so the multiple of `X` and `W` is True, and OpDesc
  //   will hold a attribute of them.
  //
  //   The Op desc of same fc could be
  //   {
  //      "type": "fc",
  //      "input": ["X1", "X2", "W1", "W2", "b"],
  //      "output": "fc.out",
  //      "attrs" : {
  //        "input_format": [0, 2, 4, 5]
  //      }
  //   }
  //
  optional bool multiple = 3 [ default = false ];

I don't understand the meaning of multiple by reading the comment and had to ask @Canpio .

This design is really hacky and far from readable.

I would propose the introduction of

message VarDesc {
  string name = 1;
  int dup = 2 [default = 0];
};

@wangkuiyi wangkuiyi changed the title Merge op_desc.proto and op_proto.proto Refactorize framework/*.proto Aug 8, 2017
@jacquesqiao
Copy link
Member

VarDesc is better than attr of temporary_index

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants