[WIP] Finalize firewalld port forwarding support

[mheon]

There are two major changes here.

Firstly, this adds proper support for port forwarding from localhost via a new policy accepting traffic from HOST. This is the last bit we were missing from the original port-forwarding implementation.

Secondly, this fixes a bug where we generated incorrect rules when port-forwarding from a single IP. Instead of doing standard port-forwarding rules, those need rich rules. This was reported as #881.

There are also some small code cleanups in how we handle setting up and tearing down port forwarding. It's still rather ugly, but at least a little better than it was before.

Fixes #881

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mheon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [mheon]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[mheon]

Tagging WIP as I'm still working on some bugs, seemingly related to firewalld 2.0

[mheon]

Hm. Tests aren't even running firewalld, despite fw_driver=firewalld being set.

[mheon]

Localhost port forwarding: still broken, being worked on.
Remote port forwarding: should be fully working after latest push.

[tgerczei]

@mheon might I ask whether this is still being worked on?

[mheon]

It is, though there were problems on the firewalld side that needed to be resolved before I went ahead. Those should be fixed, so I'm hoping to get back to this in January.

[mheon]

@Luap99 After some further work, I have this working, with some serious caveats.

• We cannot hit access forwarded ports locally except via localhost due to a missing bit of firewalld functionality.
• Isolation driver had to be yanked out. Had to add a new firewalld policy to deal with localhost forwarding. Problem: every time we added a zone for isolation, that policy would have to be edited, and policy changes require a firewalld reload. Which wipes out all our non-permanent rules, breaking forwarding to all containers every time we want to add a container in a new zone.

Problem 2 I think we can ship with, I'll just add an error if isolation is requested. Problem 1 prevents a lot of testing we could do in CI. I can sort of work around this until firewalld adds the functionality we need by querying firewalld to make sure the right rules are in place but that doesn't seem like a complete solution.