improve system node identification: use port-group prop

[x37v]

{
  "FULL_PATH": "/rnbo/jack/info/ports/properties",
  "DESCRIPTION": "Ports and a list of their properties",
  "CONTENTS": {
    "system:capture_1": {
      "FULL_PATH": "/rnbo/jack/info/ports/properties/system:capture_1",
      "TYPE": "s",
      "VALUE": "{\"http://jackaudio.org/metadata/port-group\":\"rnbo-graph-user-io\",\"physical\":true,\"terminal\":true}",
      "ACCESS": 3,
      "CLIPMODE": "none",
      "DESCRIPTION": "JSON key/value object indicating Jack properties for this port"
    },
    "system:capture_2": {
      "FULL_PATH": "/rnbo/jack/info/ports/properties/system:capture_2",
      "TYPE": "s",
      "VALUE": "{\"http://jackaudio.org/metadata/port-group\":\"rnbo-graph-user-io\",\"physical\":true,\"terminal\":true}",
      "ACCESS": 3,
      "CLIPMODE": "none",
      "DESCRIPTION": "JSON key/value object indicating Jack properties for this port"
    },
    "system:playback_1": {
      "FULL_PATH": "/rnbo/jack/info/ports/properties/system:playback_1",
      "TYPE": "s",
      "VALUE": "{\"http://jackaudio.org/metadata/port-group\":\"rnbo-graph-user-io\",\"physical\":true,\"terminal\":true}",
      "ACCESS": 3,
      "CLIPMODE": "none",
      "DESCRIPTION": "JSON key/value object indicating Jack properties for this port"
    },
    "system:playback_2": {
      "FULL_PATH": "/rnbo/jack/info/ports/properties/system:playback_2",
      "TYPE": "s",
      "VALUE": "{\"http://jackaudio.org/metadata/port-group\":\"rnbo-graph-user-io\",\"physical\":true,\"terminal\":true}",
      "ACCESS": 3,
      "CLIPMODE": "none",
      "DESCRIPTION": "JSON key/value object indicating Jack properties for this port"
    },
    "snd-0:out1": {
      "FULL_PATH": "/rnbo/jack/info/ports/properties/snd-0:out1",
      "TYPE": "s",
      "VALUE": "{\"http://jackaudio.org/metadata/pretty-name\":\"left\"}",
      "ACCESS": 3,
      "CLIPMODE": "none",
      "DESCRIPTION": "JSON key/value object indicating Jack properties for this port"
    },
    "snd-0:out2": {
      "FULL_PATH": "/rnbo/jack/info/ports/properties/snd-0:out2",
      "TYPE": "s",
      "VALUE": "{\"http://jackaudio.org/metadata/pretty-name\":\"right\"}",
      "ACCESS": 3,
      "CLIPMODE": "none",
      "DESCRIPTION": "JSON key/value object indicating Jack properties for this port"
    },
    "snd-1:out1": {
      "FULL_PATH": "/rnbo/jack/info/ports/properties/snd-1:out1",
      "TYPE": "s",
      "VALUE": "{\"http://jackaudio.org/metadata/pretty-name\":\"left\"}",
      "ACCESS": 3,
      "CLIPMODE": "none",
      "DESCRIPTION": "JSON key/value object indicating Jack properties for this port"
    },
    "snd-1:out2": {
      "FULL_PATH": "/rnbo/jack/info/ports/properties/snd-1:out2",
      "TYPE": "s",
      "VALUE": "{\"http://jackaudio.org/metadata/pretty-name\":\"right\"}",
      "ACCESS": 3,
      "CLIPMODE": "none",
      "DESCRIPTION": "JSON key/value object indicating Jack properties for this port"
    },
    "snd-2:out1": {
      "FULL_PATH": "/rnbo/jack/info/ports/properties/snd-2:out1",
      "TYPE": "s",
      "VALUE": "{\"http://jackaudio.org/metadata/pretty-name\":\"left\"}",
      "ACCESS": 3,
      "CLIPMODE": "none",
      "DESCRIPTION": "JSON key/value object indicating Jack properties for this port"
    },
    "snd-2:out2": {
      "FULL_PATH": "/rnbo/jack/info/ports/properties/snd-2:out2",
      "TYPE": "s",
      "VALUE": "{\"http://jackaudio.org/metadata/pretty-name\":\"right\"}",
      "ACCESS": 3,
      "CLIPMODE": "none",
      "DESCRIPTION": "JSON key/value object indicating Jack properties for this port"
    },
    "snd-3:out1": {
      "FULL_PATH": "/rnbo/jack/info/ports/properties/snd-3:out1",
      "TYPE": "s",
      "VALUE": "{\"http://jackaudio.org/metadata/pretty-name\":\"left\"}",
      "ACCESS": 3,
      "CLIPMODE": "none",
      "DESCRIPTION": "JSON key/value object indicating Jack properties for this port"
    },
    "snd-3:out2": {
      "FULL_PATH": "/rnbo/jack/info/ports/properties/snd-3:out2",
      "TYPE": "s",
      "VALUE": "{\"http://jackaudio.org/metadata/pretty-name\":\"right\"}",
      "ACCESS": 3,
      "CLIPMODE": "none",
      "DESCRIPTION": "JSON key/value object indicating Jack properties for this port"
    }
  }
}

[fde31]

OK when looking at this closer I think we have an opportunity here to make this potentially easier and simpler and am curious what you think. Currently the frontend has to construct "virtual" nodes using the port info. When utilizing PortGroups and properties as outlined above the already fragile concept gets even more complex, partially bc we have "input" and "output" system nodes, but also bc not all control nodes seem to be assigned to a PortGroup.

Happy to chat this through in a call but I wonder if it would be beneficial here to introduce the rule that all non-instance ports are part of a group. This way we can use the PortGroup as a reliable source for node management in the graph instead of dynamically creating "virtual nodes".

The current code also splits system I/O into input and output nodes. How would you feel about introducing two distinct PortGroups for that instead of a single rnbo-graph-user-io?

[fde31]

Summing up the discussion yesterday it would be great if:

• /rnbo/jack/info/ports/properties has an entry for every port in the graph (incl. what we currently consider ports of PatcherInstance and Control nodes
• within the properties a designated key would identify the node by id a port belongs to rather than relying upon the UI to build a somewhat implicit, virtual state of the Jack graph
• we currently break system I/O into 2 designated input and output nodes in the UI, if rather than relying upon string matching this could already be happening in the state of the runner that would simplify things a lot

The idea here is basically to get a more representative state of the nodes and ports into the OSCQuery data to reliable represent the state of the graph, nodes and ports on the frontend. In the current state the frontend code does a lot of string matching, creates something I'd describe as "virtual" nodes (system in and out) and therefore is quite cumbersome and error-prone. It also doesn't easily allow for mapping what is returned as JSON from the OSCQuery endpoint to a Graph by just looking at the data.

In your proposal above you are introducing the concept of using port-group (http://jackaudio.org/metadata/port-group) to organize ports in groups, which could be thought of as the node id. If we stick to this I'd propose we move to something like:

{
  "FULL_PATH": "/rnbo/jack/info/ports/properties",
  "DESCRIPTION": "Ports and a list of their properties",
  "CONTENTS": {
    "<port_id>": {
      "FULL_PATH": "/rnbo/jack/info/ports/properties/<port_id>",
      "TYPE": "s",
      "VALUE": "{\"http://jackaudio.org/metadata/port-group\":\"<node_id>\"}",
      "ACCESS": 3,
      "CLIPMODE": "none",
      "DESCRIPTION": "JSON key/value object indicating Jack properties for this port"
    }
}

Where /rnbo/jack/info/ports/properties has an entry for every <port_id> listed in the sinks and sources defined in /rnbo/jack/info/ports/audio and /rnbo/jack/info/ports/midi. Additionally

• system I/O would be split into an input and output group, eg. rnbo-graph-user-input and rnbo-graph-user-output
• instance ports are grouped as well, eg. by using instance's jackName
• control (aka non-instance and non-system) nodes will also have a group associated with it reflected in the port properties

We'd be interpreting the http://jackaudio.org/metadata/port-group concept as node IDs basically. I'm not 100% certain if there is something that would conflict with Jack's interpretation of the port-group so I'd refer to your judgement. If it does, I assume we can always introduce a more custom property identifier for this?

The only caveat would be that using a group rnbo-hidden would effectively mean that any hidden port is not really associated with any node other than a single hidden one. Maybe that is fine or alternatively we could just store that info in an additional property for a port?

If that were in place the frontend can basically construct and manage the the Jack Graph (nodes, ports and connections) entirely decoupled from the RNBO instance state and manage it in a much simpler way by storing:

• a list of ports (incl. a flag whether it's shown or hidden)
• a list of nodes
• a relationship of nodes and ports
• use the node id (aka port-group) to manage any meta / position data

Here one more complete example:

{
  "FULL_PATH": "/rnbo/jack/info/ports/properties",
  "DESCRIPTION": "Ports and a list of their properties",
  "CONTENTS": {
    "system:capture_1": {
      "FULL_PATH": "/rnbo/jack/info/ports/properties/system:capture_1",
      "TYPE": "s",
      "VALUE": "{\"http://jackaudio.org/metadata/port-group\":\"rnbo-graph-user-input\",\"physical\":true,\"terminal\":true}",
      "ACCESS": 3,
      "CLIPMODE": "none",
      "DESCRIPTION": "JSON key/value object indicating Jack properties for this port"
    },
    "system:capture_2": {
      "FULL_PATH": "/rnbo/jack/info/ports/properties/system:capture_2",
      "TYPE": "s",
      "VALUE": "{\"http://jackaudio.org/metadata/port-group\":\"rnbo-graph-user-input\",\"physical\":true,\"terminal\":true}",
      "ACCESS": 3,
      "CLIPMODE": "none",
      "DESCRIPTION": "JSON key/value object indicating Jack properties for this port"
    },
    "system:playback_1": {
      "FULL_PATH": "/rnbo/jack/info/ports/properties/system:playback_1",
      "TYPE": "s",
      "VALUE": "{\"http://jackaudio.org/metadata/port-group\":\"rnbo-graph-user-output\",\"physical\":true,\"terminal\":true}",
      "ACCESS": 3,
      "CLIPMODE": "none",
      "DESCRIPTION": "JSON key/value object indicating Jack properties for this port"
    },
    "system:playback_2": {
      "FULL_PATH": "/rnbo/jack/info/ports/properties/system:playback_2",
      "TYPE": "s",
      "VALUE": "{\"http://jackaudio.org/metadata/port-group\":\"rnbo-graph-user-output\",\"physical\":true,\"terminal\":true}",
      "ACCESS": 3,
      "CLIPMODE": "none",
      "DESCRIPTION": "JSON key/value object indicating Jack properties for this port"
    },
      "rnbo-control:midiin": {
      "FULL_PATH": "/rnbo/jack/info/ports/properties/rnbo-control:midiin",
      "TYPE": "s",
      "VALUE": "{\"http://jackaudio.org/metadata/port-group\":\"rnbo-control\"}",
      "ACCESS": 3,
      "CLIPMODE": "none",
      "DESCRIPTION": "JSON key/value object indicating Jack properties for this port"
    },
    "system:playback_2": {
      "FULL_PATH": "/rnbo/jack/info/ports/properties/system:playback_2",
      "TYPE": "s",
      "VALUE": "{\"http://jackaudio.org/metadata/port-group\":\"rnbo-graph-user-output\",\"physical\":true,\"terminal\":true}",
      "ACCESS": 3,
      "CLIPMODE": "none",
      "DESCRIPTION": "JSON key/value object indicating Jack properties for this port"
    },
    "feedback-synth-0:out1": {
      "FULL_PATH": "/rnbo/jack/info/ports/properties/feedback-synth-0:out1",
      "TYPE": "s",
      "VALUE": "{\"http://jackaudio.org/metadata/port-group\":\"feedback-synth-0\"}",
      "ACCESS": 3,
      "CLIPMODE": "none",
      "DESCRIPTION": "JSON key/value object indicating Jack properties for this port"
    },
    "feedback-synth-0:out1": {
      "FULL_PATH": "/rnbo/jack/info/ports/properties/feedback-synth-0:out2",
      "TYPE": "s",
      "VALUE": "{\"http://jackaudio.org/metadata/port-group\":\"feedback-synth-0\"}",
      "ACCESS": 3,
      "CLIPMODE": "none",
      "DESCRIPTION": "JSON key/value object indicating Jack properties for this port"
    }
}

The frontend would be able to figure out if a port-group is related to a PatcherInstance by checking if the port-group value is present in the list of instances and link out correctly. However, if we want this to be even more predictable and determinist we could also carry a property that identifies the instance id specifically alongside some form of type (similar to how physical and terminal are present), eg. rnbo-instance: true (or some more generic enum value) and rnbo-instance-id: <id>.

If that seems desirable an instance port properties entry could look like:

{
  "http://jackaudio.org/metadata/port-group" : "feedback-synth-0",
  "rnbo-instance": true,
  "rnbo-instance-id": 0
}