Convert core to use rollup for bundling.

[nsthorat]

Before I move forward with this change on the other repos I'd like feedback here.

This change does the following:

• Convert to rollup for bundling. Remove browserify.
• Use "extend: true" to extend a global "tf" object. There is now no "tfc".
• Ship es6 in a dist-es6 folder as well as es5.
• Remove models. They don't build anymore nor are they useful. We'll keep the KNN image classifier around and not unit test it just so we have the code. I'll remove this at some point in the future.

This PR goes along with:
tensorflow/tfjs-layers#191
tensorflow/tfjs#301

I tested with yalc across all packages, and eventually all the way to the mobilenet example in tfjs-example and everything seems to work. Using the bundles I've also used the code on the homepage to train a simple layers model (tested both union and including layers and core independently).

Hosted bundles here:
https://storage.googleapis.com/dljs-test/rollup/tf.js
https://storage.googleapis.com/dljs-test/rollup/tf.min.js
https://storage.googleapis.com/dljs-test/rollup/tf-core.js
https://storage.googleapis.com/dljs-test/rollup/tf-core.min.js
https://storage.googleapis.com/dljs-test/rollup/tf-layers.js
https://storage.googleapis.com/dljs-test/rollup/tf-layers.min.js

This change is 

[dsmilkov]

Really nice work. Few comments, the biggest one is to export both es5 and es6 code to npm. (see comments below)

---

Reviewed 39 of 40 files at r1.
Review status: 39 of 40 files reviewed at latest revision, all discussions resolved, some commit checks failed.

---

package.json, line 6 at r1 (raw file):

  "description": "Hardware-accelerated JavaScript library for machine intelligence",
  "private": false,
  "main": "dist/index.js",

Since we now also export es6, add the "module" and "jsnext:main" fields with the value of "dist/index" (no extension). These fields are used when you export es6 js, while the "main" field is reserved for es5. See /~https://github.com/d3/d3/blob/master/package.json#L21

Also we should produce both es5 and es6 (see other comment in tsconfig.json). We should export es6 in dist-es6 folder, while es5 in the current dist folder. Then "main" will be "dist/index.js", while "module" will be "dist-es6/index".

---

package.json, line 28 at r1 (raw file):

    "karma-typescript": "~3.0.8",
    "rimraf": "~2.6.2",
    "rollup": "^0.58.2",

use ~

---

package.json, line 33 at r1 (raw file):

    "rollup-plugin-typescript2": "0.13.0",
    "shelljs": "~0.7.8",
    "tsify": "~3.0.1",

remove tsify

---

rollup.config.js, line 1 at r1 (raw file):

import node from "rollup-plugin-node-resolve";

add license

---

tsconfig.json, line 3 at r1 (raw file):

{
  "compilerOptions": {
    "module": "ES2015",

I think we need to export both es5 and es6, otherwise we require all node users to use typescript or babel in the toolchain. To confirm my worry, do the following exercise:

Build the npm package as tgz file and start a new node project with @tensorflow/tfjs-core as a dep. Write a sample.js program in es5 (no typescript, no babel) that require('@tensorflow/tfjs-core) and does matmul. Then run node sample.js. I think it will fail because we only export es6. This forces every user to have either babel or typescript in their dev toolchain, which is fairly strict and non-standard. E.g. d3 exports both es6 and es5 builds (see "main: dist/d3.node.js" vs "module: index.js" in package.json) /~https://github.com/d3/d3/blob/master/package.json#L18

---

scripts/build-npm.sh, line 23 at r1 (raw file):

tsc --sourceMap false
rollup -c

double check that tf-core.js has all the ops (tf.matMul for example - last time I tried rollup, it failed to export things that were exported in ops.ts)

---

scripts/build_and_lint_all, line 33 at r1 (raw file):

shell.echo('=======================================================');
shell.echo('Building', cwd);
run('yarn', cwd);

Since there are no more subfolders, remove this script entirely and call yarn && yarn build && yarn lint directly in .travis.yml

---

Comments from Reviewable

[nsthorat]

Review status: 35 of 43 files reviewed at latest revision, 7 unresolved discussions, some commit checks pending.

---

package.json, line 6 at r1 (raw file):

Previously, dsmilkov (Daniel Smilkov) wrote…

Since we now also export es6, add the "module" and "jsnext:main" fields with the value of "dist/index" (no extension). These fields are used when you export es6 js, while the "main" field is reserved for es5. See /~https://github.com/d3/d3/blob/master/package.json#L21

Also we should produce both es5 and es6 (see other comment in tsconfig.json). We should export es6 in dist-es6 folder, while es5 in the current dist folder. Then "main" will be "dist/index.js", while "module" will be "dist-es6/index".

Removed ES6 entirely.

---

package.json, line 28 at r1 (raw file):

Previously, dsmilkov (Daniel Smilkov) wrote…

use ~

Done.

---

package.json, line 33 at r1 (raw file):

Previously, dsmilkov (Daniel Smilkov) wrote…

remove tsify

Done.

---

tsconfig.json, line 3 at r1 (raw file):

Previously, dsmilkov (Daniel Smilkov) wrote…

I think we need to export both es5 and es6, otherwise we require all node users to use typescript or babel in the toolchain. To confirm my worry, do the following exercise:

Build the npm package as tgz file and start a new node project with @tensorflow/tfjs-core as a dep. Write a sample.js program in es5 (no typescript, no babel) that require('@tensorflow/tfjs-core) and does matmul. Then run node sample.js. I think it will fail because we only export es6. This forces every user to have either babel or typescript in their dev toolchain, which is fairly strict and non-standard. E.g. d3 exports both es6 and es5 builds (see "main: dist/d3.node.js" vs "module: index.js" in package.json) /~https://github.com/d3/d3/blob/master/package.json#L18

Now just exporting ES5. Unfortunately rollup needs to use ES2015 in tsconfig as a target, and doesn't allow you to specify a tsconfig. This means this tsconfig relates specifically to rollup, and the tsconfig-package.json refers to the one which we will use to generate the ES5 build.

And just to respond to you, what I did was link this package from layers and run all their unit tests.

---

scripts/build-npm.sh, line 23 at r1 (raw file):

Previously, dsmilkov (Daniel Smilkov) wrote…

double check that tf-core.js has all the ops (tf.matMul for example - last time I tried rollup, it failed to export things that were exported in ops.ts)

Yep double checked that.

---

scripts/build_and_lint_all, line 33 at r1 (raw file):

Previously, dsmilkov (Daniel Smilkov) wrote…

Since there are no more subfolders, remove this script entirely and call yarn && yarn build && yarn lint directly in .travis.yml

Done.

---

Comments from Reviewable

[caisq]

cc myself.

[nsthorat]

Review status: 33 of 43 files reviewed at latest revision, 7 unresolved discussions, some commit checks pending.

---

package.json, line 6 at r1 (raw file):

Previously, nsthorat (Nikhil Thorat) wrote…

Removed ES6 entirely.

I added ES6 back beacuse rollup chokes when dependents are using commonjs modules (from union package I couldn't bundle this). It's now shipping to dist-es6 and I added a "jsnext:main" and "module" field. Rollup now nicely bundles everything.

---

Comments from Reviewable

[nsthorat]

Review status: 33 of 43 files reviewed at latest revision, 7 unresolved discussions, all commit checks successful.

---

rollup.config.js, line 1 at r1 (raw file):

Previously, dsmilkov (Daniel Smilkov) wrote…

add license

Done.

---

tsconfig.json, line 3 at r1 (raw file):

Previously, nsthorat (Nikhil Thorat) wrote…

Now just exporting ES5. Unfortunately rollup needs to use ES2015 in tsconfig as a target, and doesn't allow you to specify a tsconfig. This means this tsconfig relates specifically to rollup, and the tsconfig-package.json refers to the one which we will use to generate the ES5 build.

And just to respond to you, what I did was link this package from layers and run all their unit tests.

Shipping es6 again. Default tsconfig is es6, tsconfig-es5 is for the es5 build.

---

Comments from Reviewable