Question about writing my OPS

[DavidGOrtega]

Hi,

I'm writing my own ops for TF. I have some doubts and maybe someone can explain them to me.
I have read the docs however I had to figure out the rest looking at the code.

There are four backends:

• fallback (js)
• webGL (glsl/gles)
• webassembly (C++)
• webGPU (C++ alike code)

I'm interested on webGL and webassembly, my doubts are:

1. Would be amazing if I can write my Layers in any language (js, webGL, webassembly). At the end everything is going to run in the browser so like in other DL frameworks I can use python OPs or GPU/CPU with the penalty of uploading/downloading the data, in webDNN would be very nice if I can just use a layer that runs in vanilla JS. That way operations that are done just once like resize could be reused.
2. Could be possible to write webGL code in plain JS? I mean, there are many libraries that has those operations I need actually written in glsl. Would be very helpful if I just can call plain JS instead. Is that possible?
3. During the installation I saw that webassembly backend relies on Eigen. How can I access EIgen?

[milhidaka]

1. Currently webdnn does not support mixing backends in one model.

You may divide the model in three graphs

(WebGL part 1) -> (Original part) -> (WebGL part 2)

Then, you can convert two WebGL part separately and write original operator in pure js (not using webdnn). WebDNN can load multiple graphs in one browser session.

2. Not possible. gpu.js is a solution to run js code on GPU, but combination to webdnn is not straightforward.

3. see latest guide. https://mil-tokyo.github.io/webdnn/docs/tutorial/setup.html#installing-emscripten-and-eigen

[DavidGOrtega]

@milhidaka thanks for reply.

My third question is more how do I access eigen in my webassembly OPs. I have properly installed them.
I'm wondering if webDNN exposes the operations in Eigen.

register_elementwise_kernel_webassembly(TransposeOperatorExample, "y = x.transpose(); ")

[milhidaka]

Eigen is not seamlessly integrated with webdnn's array.

You have to manually wrap raw point as Eigen matrix with specifying shape.

This is the (only) working example:

webdnn/src/graph_transpiler/webdnn/backend/webassembly/kernels/tensordot.py

Line 62 in d5d949e

Eigen::Map<Eigen::Matrix<float, Eigen::Dynamic, Eigen::Dynamic, Eigen::%%A_MAJOR%%> > a_mat(A, %%LOAD_BUFFER(M)%%, %%LOAD_BUFFER(K)%%);

[DavidGOrtega]

@milhidaka

1. Not possible. gpu.js is a solution to run js code on GPU, but combination to webdnn is not straightforward.

I'm thinking in creating a replacement of fallback backend replacing all the ops with gpu.js and giving access to the api, that way we would have access to js with gpu right?