Describe problems for functor

[reyoung]

Part one of Functor Design Doc. It describes what problems should be resolved.

Related issue #3894

[wangkuiyi]

Functions or Functors

#include <algorithm>
#include <iostream>

template <typename T>
T tanh(T x) {
  return static_cast<T>(1);
}

template <typename T, typename S>
T sigmoid(T x, S s) {
  return (tanh(x) + 1) * s;
}

template <typename Place>
class SigmodOpKernel /*: public framework::OpKernel*/ {
template <typename T, typename S>
class Sigmoid {
 public:
  Sigmoid(S scale) : scale_(scale) {}
  /*__host__ __device__*/ T operator(T x) const { return sigmoid(x, scale_); }
 private:
  const S scale_;
}
};

template <>
class SigmodOpKernel<CPUPlace> : public framework::OpKernel {};
 public:
  void Compute(ctx) {
    std::transform(
        ctx.Input<Tensor>("X")->data(),
        ctx.Input<Tensor>("X")->data() + product(ctx.Input<Tensor>("X")->dims()),
        ctx.Output<Tensor>("Out")->mutable_data(),
        Sigmoid(ctx->Attr<float>("scale")));
  }
};

// template <>
// class SigmodOpKernel<GPUPlace> : public framework::OpKernel {};
//  public:
//   void Compute(ctx) {
//     thrust::transform(
//         ctx.Input<Tensor>("X")->data(),
//         ctx.Input<Tensor>("X")->data() + product(ctx.Input<Tensor>("X")->dims()),
//         ctx.Output<Tensor>("Out")->mutable_data(),
//         Sigmoid(ctx->Attr<float>("scale")));
//   }
// };

[wangkuiyi]

Design Doc: Functions

Following the overall design, this document is about the functions and how they compose into operators.

Basically, to the ease of definitions, functions operate on scalar values. But most operators operate on Tensors. For example:

[QiJune]

Dose this doc only focus on element-wise operators or try to propose a general solution?
std::transform is not suitable for all operations.

[reyoung]

Closed since #3901