When setting up an additional IPv4 subnet on a dedicated Hetzner server I learned that Firewalld is terrible at handling IP forwarding.
- Until now, Firewalld has been bad at [intra zone forwarding](Firewalld project blog post).
- A new feature solves the problem:
firewall-cmd --zone=example --add-forward
- The feature is unavailable in versions <= 0.8.2. Migrate to CentOS Stream and upgrade to v0.9.3 where it is available.
When assigning subnet addresses to virtualised hosts Hetzner recommend in their documentation that you set the host up as a 'brouter' where the host machine forwards all packets to the subnet. In fact there is no choice, because if you us a traditional bridge exposing virtual hosts interfaces and mac addresses you will get a warning from Hetzner support. Forwarding packets is quite straightforward and well discussed elsewhere.
The unusal thing I discovered is that Firewalld is not very good at forward and output filtering (by the project's own admission). When I enabled Firewalld logging I found that while I set the IP forwarding up correctly, Firewalld was dropping the forwarded packets.
No amount of direct or rich rules in Firewalld solved this issue and manipulating the netfilter tables seemed unsustainable.
Then I read a Firewalld project blog post discussing this issue of "Intra Zone Forwarding" and learned there was a solution:
firewall-cmd --zone=example --add-forward. The only challenge is this feature was introduced after v0.8.2 (the latest available in CentOS 8). This was resolved by migrating to CentOS 8 Stream where v0.9.3 is available.