RFC: notation for invocation-wide core.shm names?

[javierguerragiraldez]

I've been heavily using the new core.shm facility to easily share FFI-based objects and there's a couple of improvements I'd really like, but want to discuss them first.

In the usual single-threaded architecture, the 'pid-local' syntaxes, like /freelists/struct_packet_# are quite handy to get a global structure that doesn't conflict with other snabb instances running at the same time, and also are easy to cleanup afterwards (something like for d in /var/run/snabb/*; do if [! -e /proc/$(basename $d) ]; then rm -r $d; fi; done).

When extending to a multiprocessing architecture, two new needs appear: "foreign" and "invocation-wide" objects.

Foreign objects
Many objects are process-specific,(i.e, "one packet freelist per process" => /freelists/struct_packet_# => /var/run/snabb/<pid>/freelists/struct_packet_#); but some could be "one per process, but sometimes read by another one".

For example, a simple statistics counter: there's one for each process, so it's fast to update with each processed packet. With a name like /stats/packets, resolving to /var/run/snabb/<pid>/stats/packets. But to get the aggregate statistics, the 'main' process has to read the counter of the 'working' processes. For that, I've added an optional id argument to shm.resove() and shm.map() that defaults to the current pid. When omitted, the behaviour is the same as usual: shm.map('/stats/packets', 'struct stats') maps an object for the current process, but makes it easy for the main process to ask shm.map('/stats/packets', 'struct stats', true, pid) for a readonly map of the stats object of the given pid.

Invocation-wide objects
There are some objects that are truly global (i.e. the DMA pool) or that don't belong to a single process, but are specified by name (i.e. an interprocess link) unfortunately, the "fully qualified" variant is not enough, since it would be the same for all SnabbSwitch invocations. Either from one run to the next (making for very weird and non-funny failures by inheriting the 'just before crash' state of the previous test) or between simultaneous but independent runs.

The easiest way to identify program invocations is the process group id, which is set by the shell when executing jobs, and can be read with S.getpgid() from ljsyscall.

Currently, I'm just using the optional id argument discussed above, like shm.map('/dma_pool', heap_t, false, S.getpgid()), but it would be much nicer to have a name syntax for that and simply use shm.map('#/dma_pool', heap_t) or something like that.

The biggest obstacle right now is settling in a passable syntax. So far, the only one I've been able to come up looks like:

foo/bar => /var/run/snabb/<pid>/<path>/foo/bar
/foo/bar => /var/run/snabb/<pid>/foo/bar
#/foo/bar => /var/run/snabb/<pgid>/foo/bar
//foo/bar => /var/run/snabb/foo/bar

Am I wrong in thinking it's too ugly? Using :/foo/bar looks a little better but can be harder to spot with some fonts.

What about ditching the idea of "scope defining prefixes" and just replacing # with <pid> and $ with <pgid> ? then:

foo/bar => /var/run/snabb/<pid>/<path>/foo/bar
#/foo/bar => /var/run/snabb/<pid>/foo/bar
$/foo/bar => /var/run/snabb/<pgid>/foo/bar
//foo/bar => /var/run/snabb/foo/bar

... but also ...

foo/#/bar => /var/run/snabb/<path>/foo/<pid>/bar
foo/$-bar => /var/run/snabb/foo/<pid>-bar

is that too much useless flexibility?

[lukego]

So I suppose that we have separate questions of syntax and semantics here, i.e. what different kinds of scope should exist and how should we specify the right one.

For syntax one idea would be to specify the scope with separate functions map_global(), map_process(), map_local(), etc. That would be an alternative to encoding scope into the object name.

For semantics it seems worthwhile to support only a minimum number of simple kinds of objects. Do we really want to have global objects that are not owned by any one process? How are we going to synchronize access to them? Specifically, would it be simpler for each process to allocate memory from a private DMA pool instead of sharing that?

How would we solve these problems if Snabb processes were each started separately as peers instead of by forking children from a common parent? (Could we support both models i.e. forked children and links between separate processes? Could we make our implementation simpler in the process e.g. by minimizing shared state that will have to be synchronized?)

[eugeneia]

@javierguerragiraldez You can use snabb gc to cleanup stale runtime files: #558

Personally I feel adding new special syntax is overkill for this situation. We now have:

foo/bar => /var/run/snabb/<pid>/<path>/foo/bar  (for ourself)
/foo/bar => /var/run/snabb/<pid>/foo/bar
//foo/bar => /var/run/snabb/foo/bar             (for outside ourself)

We can use "//"..pid.."/path" to reference another processes resource, and depending on layout, define a function that builds a Group PID path for us, without having to support that specific layout in core.shm.

[eugeneia]

I am closing this because there is currently no activity. Please reopen it when necessary.