I wanted to keep concepts or topics separated, so I am doing this as another post. I'm still here watching my pork cook, and playing with my DEGRADED FreeBSD box. So one of the things I like to do with my Linux KVM box is run a few servers for various little tasks on my home network. But I like some separation there too. For example maybe a web server, or a mail server needs to be reachable from the public, but a backend database does not. While this can be handled via a firewall, I like the networks to be separated as well.
Being newer to the BSD world, I wasn't sure how easy this was to do with jails. I had always just seen people bind jails to the same network as the host. The solution that seems to work is very simple. In your rc.conf file, you just need to setup cloned interfaces:
cloned_interfaces="lo1 lo2" ifconfig_lo1="inet 192.168.77.1 netmask 255.255.255.0" ifconfig_lo2="inet 192.168.78.1 netmask 255.255.255.0"
Then restart/reload network interfaces. Those are not my correct IPs, but it doesn't really matter since it's all internal anyways.
For jails I decided to try iocage. So to test this I did:
iocage create tag=test1 ip4_addr="lo1|192.168.77.2" iocage create tag=test2 ip4_addr="lo2|192.168.78.2" iocage start test1 iocage start test2
That took a little to build the jail, but once it came up I could ping both without issue locally. But now for remote ping. I tried that but it failed (which I expected). This is because my firewall/router had no idea how to get to these subnets or even that they exist. Well that machine is now OpenBSD, so the fix was really easy. I opened my hostname.interface file, added:
!route add -net 192.168.77.0/24 192.168.76.5 !route add -net 192.168.78.0/24 192.168.76.5
reload that and I was good to go. Now it was reachable from my other networks (at least as far as the firewall configuration allowed).
The part that puzzles me a little is that FreeBSD didn't require any forwarding setup or gateway mode enabled to allow this. But that's OK for now. I'm just playing around. In theory now though I could let some hosts get to the 77 subnet and some to the 78, but block them from reaching each other, or other subnets. It just seems like firewall configuration will be easier this way.