Use pf's anchor to load the rules for nat and port forwarding

[jesa7955]

Now vagrant-bhyve loads pf rules by modifying /etc/pf.conf which is not very a clean way. Sometimes our rules may have a conflict order with those users added before using vagrant-bhyve which will cause a error when restarting PF. We was planing to use pfctl -a to load those rules as an anchor so that we can manage them without touching user's /etc/pf.conf. I used commands like these:

pfctl -a vagrant_<id> -f /path/to/vagrant.pf.conf
service pf onerestart

PF restarted correctly and I can see those rules have been loaded into the anchor with pfctl -a vagrant_<id> -s all, But none of those rules have an effect. Did I miss any step after load rules as an anchor?

[churchers]

without touch user's /etc/pf.conf

My understanding is that the main ruleset needs an anchor {anchorname} command, which tells it to process any rules in the anchor. The idea being that you can add an anchor in a specific place in the main ruleset, then easily add/remove rules at that location by modifying the anchor's ruleset. From the PF docs:

When PF comes across an anchor rule in the main ruleset, it will evaluate the rules contained within the anchor point as it evaluates rules in the main ruleset.

Of course this ends up with the same initial problem. If you add an anchor rule to the end of the main ruleset, it's no different than just adding the original nat rules there. Really you need the user to add the anchor statement to their ruleset in the correct place for their configuration, or we need to find a more clever way of enabling nat.

[jesa7955]

Thank you for your comment and your great vm-bhyve project, it helped me a lot. :)
After reading PF documents again and doing a little testes, I think you are right. It seems that we should add nat-anchor {anchorname} to make the rules like nat on em0 inet from 172.16.0.0/24 to any -> (em0) work. But it will meet the same problem as using include statement. About another way to enable NAT, will switching to another firewall like IPFW be an option?

[kusuriya]

If it is just for NAT could just using NATD(8) be an option? This would also end up getting you port forwarding a bit easier since it would be just adding the natd config rules for it instead of trying to munge the users firewall to get it to work.

[fabianfreyer]

I think adding anchors in the right places shouldn't be the problem. I'd go with the following to not violate POLA (which IMHO the current situation of just prepending to /etc/pf.conf does, by creating a probably invalid /etc/pf.conf):

• if /etc/pf.conf does not exist, create a default /etc/pf.conf with something like the following, load it and notify the user:

    nat-anchor "vagrant/*"
    rdr-anchor "vagrant/*"
    anchor "vagrant/*"
  

• if /etc/pf.conf does exist, check the output of pfctl -sA whether the vagrant anchor exists. If it doesn't throw an error and instruct the user to add the above anchors to their pf.conf.

• Add pf rulesets for bridge interfaces using pfctl -a vagrant/bridgeN -f /usr/local/etc/pf.bridgeN.conf

• Add pf rulesets for tap interfaces accordingly in the vagrant/vm_name anchor.

• On shutdown of the VM, flush the rulesets for that vm with pfctl -a vagrant/vm_name -F rules

Unfortunately, I don't know enough ruby to put this into a PR.