Firewall questions

Wed Sep 7 16:42:15 PDT 2022

Hi,

A couple of firewall questions.  I'll start with something easy.  If I'm ssh'd into my router and I restart the firewall (I'm using firewall3 and iptables), then on one router I get dropped from my shell after a timeout... but on another router, I stay connected.

In both cases, I'll be doing this on adjacent hosts (i.e. on the same subnet), and inside the firewall.

What would cause the firewall to not persist my connection state when restarting?

2nd question: how do I trace a packet flowing through my firewall?  I have two firewalls, in to different locations, connected by an IPsec tunnel.  Location A has the public IP address of my site.  Location B has a dynamic (but routable) address on a different ISP. Unfortunately, my servers are on a sandbox at Location B and therefore can't egress via this address (the local firewall's WAN address), so I do policy routing over the IPsec tunnel (via xfrm tunnels) to have their traffic egress (and ingress, via redirects) via Location A.

Things mostly work: HTTP, HTTPS, SMTP, IMAPS, Submission all work fine.

But port forwarding Ssh to my servers doesn't.

I've used tcpdump to see the ssh traffic arriving on the xfrm0 tunnel interface, but it then gets silently dropped and never appears on the terminal ethernet interface.  No idea why.  And no log messages get generated about it being DROPped or REJECTed.

So I need to trace it through the rules to see if it's implicitly hitting a chain with a DROP or REJECT policy.

I tried something like:

iptables -t raw -D PREROUTING -s 72.104.76.181/32 -d 192.168.8.0/24 -p tcp -m tcp --dport 22 -j TRACE

And get traces like:

Sep  6 22:25:07 OpenWrt3 kernel: [526070.718061] TRACE: raw:PREROUTING:rule:3 IN=xfrm0 OUT= MAC= SRC=72.104.76.181 DST=192.168.8.12 LEN=48 TOS=0x00 PREC=0x00 TTL=45 ID=0 DF PROTO=TCP SPT=53442 DPT=22 SEQ=3707460739 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (0204056C04020000) 
Sep  6 22:25:07 OpenWrt3 kernel: [526070.739550] TRACE: raw:PREROUTING:rule:7 IN=xfrm0 OUT= MAC= SRC=72.104.76.181 DST=192.168.8.12 LEN=48 TOS=0x00 PREC=0x00 TTL=45 ID=0 DF PROTO=TCP SPT=53442 DPT=22 SEQ=3707460739 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (0204056C04020000) 
Sep  6 22:25:07 OpenWrt3 kernel: [526070.761691] TRACE: raw:zone_vpn_helper:rule:12 IN=xfrm0 OUT= MAC= SRC=72.104.76.181 DST=192.168.8.12 LEN=48 TOS=0x00 PREC=0x00 TTL=45 ID=0 DF PROTO=TCP SPT=53442 DPT=22 SEQ=3707460739 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (0204056C04020000) 
Sep  6 22:25:07 OpenWrt3 kernel: [526070.783690] TRACE: raw:PREROUTING:rule:8 IN=xfrm0 OUT= MAC= SRC=72.104.76.181 DST=192.168.8.12 LEN=48 TOS=0x00 PREC=0x00 TTL=45 ID=0 DF PROTO=TCP SPT=53442 DPT=22 SEQ=3707460739 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (0204056C04020000) 
Sep  6 22:25:07 OpenWrt3 kernel: [526070.805671] TRACE: nat:PREROUTING:rule:1 IN=xfrm0 OUT= MAC= SRC=72.104.76.181 DST=192.168.8.12 LEN=48 TOS=0x00 PREC=0x00 TTL=45 ID=0 DF PROTO=TCP SPT=53442 DPT=22 SEQ=3707460739 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (0204056C04020000) 
Sep  6 22:25:07 OpenWrt3 kernel: [526070.827115] TRACE: nat:prerouting_rule:rule:1 IN=xfrm0 OUT= MAC= SRC=72.104.76.181 DST=192.168.8.12 LEN=48 TOS=0x00 PREC=0x00 TTL=45 ID=0 DF PROTO=TCP SPT=53442 DPT=22 SEQ=3707460739 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (0204056C04020000) 
Sep  6 22:25:07 OpenWrt3 kernel: [526070.848978] TRACE: nat:PREROUTING:rule:6 IN=xfrm0 OUT= MAC= SRC=72.104.76.181 DST=192.168.8.12 LEN=48 TOS=0x00 PREC=0x00 TTL=45 ID=0 DF PROTO=TCP SPT=53442 DPT=22 SEQ=3707460739 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (0204056C04020000) 
Sep  6 22:25:07 OpenWrt3 kernel: [526070.870409] TRACE: nat:zone_vpn_prerouting:rule:1 IN=xfrm0 OUT= MAC= SRC=72.104.76.181 DST=192.168.8.12 LEN=48 TOS=0x00 PREC=0x00 TTL=45 ID=0 DF PROTO=TCP SPT=53442 DPT=22 SEQ=3707460739 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (0204056C04020000) 
Sep  6 22:25:07 OpenWrt3 kernel: [526070.892639] TRACE: nat:prerouting_vpn_rule:rule:1 IN=xfrm0 OUT= MAC= SRC=72.104.76.181 DST=192.168.8.12 LEN=48 TOS=0x00 PREC=0x00 TTL=45 ID=0 DF PROTO=TCP SPT=53442 DPT=22 SEQ=3707460739 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (0204056C04020000) 
Sep  6 22:25:07 OpenWrt3 kernel: [526070.915444] TRACE: nat:zone_vpn_prerouting:rule:2 IN=xfrm0 OUT= MAC= SRC=72.104.76.181 DST=192.168.8.12 LEN=48 TOS=0x00 PREC=0x00 TTL=45 ID=0 DF PROTO=TCP SPT=53442 DPT=22 SEQ=3707460739 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (0204056C04020000) 
Sep  6 22:25:07 OpenWrt3 kernel: [526070.937704] TRACE: nat:PREROUTING:rule:7 IN=xfrm0 OUT= MAC= SRC=72.104.76.181 DST=192.168.8.12 LEN=48 TOS=0x00 PREC=0x00 TTL=45 ID=0 DF PROTO=TCP SPT=53442 DPT=22 SEQ=3707460739 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (0204056C04020000) 

But then nothing more.  What am I missing?

Thanks,

-Philip