[core, wasm] Modularize FromPixels and make it work for the wasm backend.

[dsmilkov]

Remove backend.fromPixels(). The user-facing tf.browser.fromPixels() will call the current backend's FromPixels kernel IF it is registered. Else it will fallback to a platform-agnostic and backend-agnostic logic in a centralized place.

The cpu, wasm and node backends do not need to have FromPixels kernel registered since the centralized logic in tf.browser.fromPixels() will work.

The desired outcome of this is that now tf.browser.fromPixels() just works with the WASM backend without any additional code.

FEATURE

---

This change is 

[annxingyuan]

Reviewable status: complete! 1 of 1 approvals obtained (waiting on @annxingyuan and @dsmilkov)

---

tfjs-core/src/ops/browser.ts, line 64 at r2 (raw file):

  }
  const isPixelData = (pixels as PixelData).data instanceof Uint8Array;
  const isImageData =

Should we wrap these checks in if / else statements so we can stop checking once we find a type match?

---

tfjs-core/src/ops/browser.ts, line 68 at r2 (raw file):

  const isVideo = typeof (HTMLVideoElement) !== 'undefined' &&
      pixels instanceof HTMLVideoElement;
  const isImage = typeof (HTMLImageElement) !== 'undefined' &&

Seems like we prefer the "HTMLImageElement != null" style elsewhere, as opposed to checking for typeof? Or maybe that doesn't work in node?

---

tfjs-core/src/ops/browser.ts, line 95 at r2 (raw file):

  const kernel = getKernel('FromPixels', ENGINE.backendName);
  if (kernel != null) {
    return ENGINE.runKernel('FromPixels', {pixels} as {}, {numChannels}) as

Does it make sense to have the fromPixels kernel take pixel data directly, so the pixel type checking and the rendering --> pixels processing can live just in this file?

---

tfjs-core/src/ops/browser.ts, line 136 at r2 (raw file):

  }
  const outShape: [number, number, number] = [height, width, numChannels];
  return tensor3d(values, outShape, 'int32');

Awesome cleanup!

[dsmilkov]

Reviewable status: complete! 1 of 1 approvals obtained (waiting on @annxingyuan and @dsmilkov)

---

tfjs-core/src/ops/browser.ts, line 68 at r2 (raw file):

Previously, annxingyuan (Ann Yuan) wrote…

Seems like we prefer the "HTMLImageElement != null" style elsewhere, as opposed to checking for typeof? Or maybe that doesn't work in node?

can't do that since it will throw Uncaught ReferenceError: HTMLImageElement is not defined`. typeof() is the way to test if things are defined.

---

tfjs-core/src/ops/browser.ts, line 95 at r2 (raw file):

Previously, annxingyuan (Ann Yuan) wrote…

Does it make sense to have the fromPixels kernel take pixel data directly, so the pixel type checking and the rendering --> pixels processing can live just in this file?

WebGL backend wants the original pixels object, since it does different things to it depending on the type, so it's best to give the backend the original arguments.

---

tfjs-core/src/ops/browser.ts, line 136 at r2 (raw file):

Previously, annxingyuan (Ann Yuan) wrote…

Awesome cleanup!

Thanks!

[dsmilkov]

Reviewable status: complete! 1 of 1 approvals obtained (waiting on @annxingyuan)

---

tfjs-core/src/ops/browser.ts, line 64 at r2 (raw file):

Previously, annxingyuan (Ann Yuan) wrote…

Should we wrap these checks in if / else statements so we can stop checking once we find a type match?

Good idea. Done.

[tafsiri]

Reviewed 6 of 11 files at r1, 4 of 7 files at r2, 1 of 1 files at r3.
Reviewable status: complete! 2 of 1 approvals obtained (waiting on @annxingyuan and @tafsiri)