Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How can we load/produce data into paddle core in our new design #2917

Closed
QiJune opened this issue Jul 17, 2017 · 1 comment
Closed

How can we load/produce data into paddle core in our new design #2917

QiJune opened this issue Jul 17, 2017 · 1 comment
Assignees

Comments

@QiJune
Copy link
Member

QiJune commented Jul 17, 2017

There are mainly two kinds of data Paddle core have to handle, training data and model parameters.
And There are commonly two ways to put them into Paddle core. The first one is to load them from a data source(Python Numpy object will be a good choice). And the second one is to produce them in Paddle, which users just set a few attributes to give a data distribution, such as zero/one/constant/rand.

In GPU training process, we have to make copy in loading ways(load to CPU first, and then copy to GPU). But, we want to produce specific data distribution in GPU directly(It's more efficient and copy is no need).

So, an efficient and easy-to-use solution have to be put forward.

@jacquesqiao
Copy link
Member

I think the first step is use numpy array

@QiJune QiJune mentioned this issue Aug 16, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants