[Swan] Interworking with Cisco VTI model (any-to-any tunnel, selected routing)

[Sébastien Lefevre]

Hello everybody,

I'm having some trouble trying to configure a vpn against a Cisco ASA appliance that is using the Cisco Virtual Tunnel Interface (VTI) configuration style, which is apparently the recommended configuration model for Cisco for some years now.
The problem is that, in this mode, the Cisco endpoint only advertises "any-to-any" in its traffic selectors (IKEv2) or equivalent payloads in IKEv1, and won't accept a libreswan-emitted proposal that does not contain these 0.0.0.0/0 <-> 0.0.0.0/0 traffic selectors either.
On libreswan side, it won't accept Cisco proposals until I configure my leftsubnet to 0.0.0.0/0 as well as rightsubnet to 0.0.0.0/0.
Of course, in that case, the whole traffic is routed to this Cisco endpoint, which is not acceptable, as I'm using libreswan as a VPN concentrator, also connected to other VPNs (other Cisco in older crypto-map style or other libre/open/strongswan and racoon implementations).

We're using the libreswan version of our distrib (CentOS7) whose kernel does not support KLIPS by default, so I'm currently stuck with NETKEY.

I'm focusing my tests on IKEv1, though a similar problem occurs on IKEv2.

My connection:
conn ciscovti
type=passthrough
left=myendpoint
# this is my "target" leftsubnet I would like to route through this tunnel
# leftsubnet=10.246.89.31/255.255.255.255
# But this is what I need to set for IKE proposals to be accepted on both sides
leftsubnet=0.0.0.0/0
right=ciscoendpoint
rightsubnet=0.0.0.0/0
ike=aes256-sha1
phase2=esp
pfs=no
phase2alg=aes192-sha
authby=secret
# This connection is started manually as it can break the routing
auto=ignore
ikelifetime=8h
salifetime=1h
remote_peer_type=cisco
# Lowest priority possible - because we set a policy any-to-any without tunnel at prio 60000
priority=65535
leftupdown="/root/updown.sh"
# I tried a dpdaction=restart to see if in that case the updown script was called on SA nego, bu this is not the case apparently
dpdaction=restart
dpddelay=30
dpdtimeout=120
# Forced reqid so that we can control SA/SPD entries policies manually including in case of a SA refresh
reqid=16000


My understanding of the problem:

(1) Upon successful IKE phase 1 negotiation (or is that phase 2?), libreswan sets netkey policies matching the configured and accepted leftsubnet/rightsubnet:

ip xfrm policy add src 0.0.0.0/0 dst 0.0.0.0/0 priority 65535 dir in tmpl src myendpoint dst ciscoendpoint mode tunnel proto esp reqid 16000.
ip xfrm policy add src 0.0.0.0/0 dst 0.0.0.0/0 priority 65535 dir out tmpl dst myendpoint src ciscoendpoint mode tunnel proto esp reqid 16000.
ip xfrm policy add src 0.0.0.0/0 dst 0.0.0.0/0 priority 65535 dir fwd tmpl dst myendpoint src ciscoendpoint mode tunnel proto esp reqid 16000.

As soon as these rules are set, I lose the connectivity to my VPN concentrator, unless I set, prior to the negotiation (i.e. before ipsec auto --add ciscovti for instance), "masking" policy rules such as:

ip xfrm policy add src 0.0.0.0/1 dst 0.0.0.0/1 priority 60000 dir {in,out,fwd}
ip xfrm policy add src 128.0.0.0/1 dst 128.0.0.0/1 priority 60000 dir {in,out,fwd}

(using split networks because rules matching 0.0.0.0/0 directly collide with the ones installed by libreswan and are replaced)

If these rules are set, then I can still get maintain a ssh connectivity to my VPN concentrator. However, at this stage, external traffic that is supposed to be forwarded by my concentrator, including the one coming from other tunnels terminated on the concentrator, is routed through this Cisco tunnel (?!).

(2) Several seconds after, librewan calls my custom updown script.

(3) In this custom updown.sh scripts, on route-client, I remove the 0.0.0.0/0 rules added in (1) and add some narrower rules to select the actual traffic to route through this tunnel. This works pretty well and from then only selected traffic is routed through this tunnel.

(4) However at the next SA negotiation (i.e. every hour), libreswan re-install a policy that seems to be:
ip xfrm policy add src 0.0.0.0/0 dst 0.0.0.0/0 priority 65535 dir out tmpl dst myendpoint src ciscoendpoint mode tunnel proto esp reqid 16000

And this time my updown.sh script is not called. So after 1h, incoming forwardable traffic is forwarded through this cisco tunnel again. So unless I poll for this policy change and delete it as soon as I detect it, the concentrator is basically broken.


So, my questions:

(a) Is there a recommended approach to deal with this Cisco VTI model? According to our partner operating this Cisco endpoint, this is a widely used configuration. I might have missed something as I couldn't find a lot of posts relating a problem with it.
(b) Is there a way to completely disable policy management done by libreswan, making it focus on SA/keys negotiations only? (I tried mode=passthrough but policies are still set by it)

Thank you very much,

-seb

[Paul Wouters]

Post by Sébastien Lefevre
Hello everybody,
I'm having some trouble trying to configure a vpn against a Cisco ASA appliance that is using the Cisco Virtual Tunnel Interface (VTI) configuration style, which is apparently the recommended configuration model for Cisco for some years now.
The problem is that, in this mode, the Cisco endpoint only advertises "any-to-any" in its traffic selectors (IKEv2) or equivalent payloads in IKEv1, and won't accept a libreswan-emitted proposal that does not contain these 0.0.0.0/0 <-> 0.0.0.0/0 traffic selectors either.
On libreswan side, it won't accept Cisco proposals until I configure my leftsubnet to 0.0.0.0/0 as well as rightsubnet to 0.0.0.0/0.
Of course, in that case, the whole traffic is routed to this Cisco endpoint, which is not acceptable, as I'm using libreswan as a VPN concentrator, also connected to other VPNs (other Cisco in older crypto-map style or other libre/open/strongswan and racoon implementations).
We're using the libreswan version of our distrib (CentOS7) whose kernel does not support KLIPS by default, so I'm currently stuck with NETKEY.

That is what is called a "routed based VPN" (versus a "policy based
VPN"). It is, in my very opiniated view, a lazy and insecure way of
setting up VPNs.

Post by Sébastien Lefevre
(a) Is there a recommended approach to deal with this Cisco VTI model? According to our partner operating this Cisco endpoint, this is a widely used configuration. I might have missed something as I couldn't find a lot of posts relating a problem with it.
(b) Is there a way to completely disable policy management done by libreswan, making it focus on SA/keys negotiations only? (I tried mode=passthrough but policies are still set by it)

While we just added support for marking, which could help you a bit, we
are still working on properly supporting Linux VTI. I envision that in
the case of VTI, we would setup the above policy, but only that what is
routed into the ip_vtiXX device, would actually get into the IPsec SA.

If you use marking (grab the libreswan git master from github), you can
select a mark number and then only mark the traffic with the right
source/dest you really want to send over the tunnel. We should probably
add a new option for that to list a comma seperated list of CIDRs that
would automatically get MARKed for you if set.

the marking iptables rules also need the reqid, so easiest is if you
add to your conn:

mark=7
reqid=7

then run something like this (assuming you only wanted to give the
remote Cisco the traffic of 10.246.89.31/32):


MARK=7
REQID=7
iptables -t mangle -A PREROUTING -i eth0 -s 10.246.89.31/32 -m policy --dir in --reqid $REQID -j MARK --set-xmark $MARK
iptables -t mangle -A PREROUTING -i eth0 -s 10.246.89.31/32 -m policy --dir in --reqid $REQID -j CONNMARK --save-mark

Paul
ps. we just found a bug - you might need to use REQID=6 (eg -1)

[Sébastien Lefevre]

Hi Paul,

Thanks for all of this, it worked perfectly.

Post by Sébastien Lefevre

Post by Sébastien Lefevre
Hello everybody,
I'm having some trouble trying to configure a vpn against a Cisco ASA

appliance that is using the Cisco Virtual Tunnel Interface (VTI) configuration
style, which is apparently the recommended configuration model for Cisco
for some years now.

Post by Sébastien Lefevre
The problem is that, in this mode, the Cisco endpoint only advertises "any-

to-any" in its traffic selectors (IKEv2) or equivalent payloads in IKEv1, and
won't accept a libreswan-emitted proposal that does not contain these
0.0.0.0/0 <-> 0.0.0.0/0 traffic selectors either.

Post by Sébastien Lefevre
On libreswan side, it won't accept Cisco proposals until I configure my

leftsubnet to 0.0.0.0/0 as well as rightsubnet to 0.0.0.0/0.

Post by Sébastien Lefevre
Of course, in that case, the whole traffic is routed to this Cisco endpoint,

which is not acceptable, as I'm using libreswan as a VPN concentrator, also
connected to other VPNs (other Cisco in older crypto-map style or other
libre/open/strongswan and racoon implementations).

Post by Sébastien Lefevre
We're using the libreswan version of our distrib (CentOS7) whose kernel

does not support KLIPS by default, so I'm currently stuck with NETKEY.
That is what is called a "routed based VPN" (versus a "policy based
VPN"). It is, in my very opiniated view, a lazy and insecure way of
setting up VPNs.

I have the feeling that Cisco is pushing this approach for scalability and performance reasons: they are better at manipulating large routing tables than SPD entries or SAs, so it's simpler for them to implement a single policy any-to-any. But anyway, this might be off topic, so sorry.

Post by Sébastien Lefevre

Post by Sébastien Lefevre
(a) Is there a recommended approach to deal with this Cisco VTI model?

According to our partner operating this Cisco endpoint, this is a widely used
configuration. I might have missed something as I couldn't find a lot of posts
relating a problem with it.

Post by Sébastien Lefevre
(b) Is there a way to completely disable policy management done by

libreswan, making it focus on SA/keys negotiations only? (I tried
mode=passthrough but policies are still set by it)
While we just added support for marking, which could help you a bit, we
are still working on properly supporting Linux VTI. I envision that in
the case of VTI, we would setup the above policy, but only that what is
routed into the ip_vtiXX device, would actually get into the IPsec SA.
If you use marking (grab the libreswan git master from github), you can
select a mark number and then only mark the traffic with the right
source/dest you really want to send over the tunnel. We should probably
add a new option for that to list a comma seperated list of CIDRs that
would automatically get MARKed for you if set.

That's a good idea indeed to simplify some setups.

Post by Sébastien Lefevre
the marking iptables rules also need the reqid, so easiest is if you
mark=7
reqid=7

Small remarks here:
- The connection's mark parameter expects a string, so this is mark="7" to make it work. I don't know if this is on purpose.
- That was enough to make my workaround based on a updown script work: it can safely set policies without involving mangling/marking, the any-to-any policies set by libreswan being just unused and not matching any packets anymore. That's much less elegant than the complete solution you proposed, but it works.
- When stopping the tunnel via an "ipsec auto --delete/down conn", the dir out policy remains in place and is not automatically removed by libreswan, while the dir in and dir fwd are (got a ERROR: netlink XFRM_MSG_DELPOLICY response for flow eroute_connection delete included errno 2: No such file or directory, but only on --delete).

Post by Sébastien Lefevre
then run something like this (assuming you only wanted to give the
MARK=7
REQID=7
iptables -t mangle -A PREROUTING -i eth0 -s 10.246.89.31/32 -m policy --dir in
--reqid $REQID -j MARK --set-xmark $MARK
iptables -t mangle -A PREROUTING -i eth0 -s 10.246.89.31/32 -m policy --dir in
--reqid $REQID -j CONNMARK --save-mark

I tried this approach instead of setting explicit policies, but didn't manage to get it work, probably due to my poor knowledge of iptables and mangle/nat interactions (10.246.89.31/32 is the published source IP for SNATted systems SNATed on the host running libreswan).
Anyway, the initial problem has now a solution and we can connect to multiple Cisco VTI-based tunnels if needed. Thank you!

Cheers,
-seb

[Paul Wouters]

Post by Sébastien Lefevre
Thanks for all of this, it worked perfectly.

Awesome!

Post by Sébastien Lefevre

Post by Paul Wouters
If you use marking (grab the libreswan git master from github), you can
select a mark number and then only mark the traffic with the right
source/dest you really want to send over the tunnel. We should probably
add a new option for that to list a comma seperated list of CIDRs that
would automatically get MARKed for you if set.

That's a good idea indeed to simplify some setups.

Yes. We will not put that in the 3.16 release yet but surely it will go
into 3.17.

Post by Sébastien Lefevre

Post by Paul Wouters
the marking iptables rules also need the reqid, so easiest is if you
mark=7
reqid=7

- The connection's mark parameter expects a string, so this is mark="7" to make it work. I don't know if this is on purpose.

That's an odd bug in the parser it seems. The type value of mark is
kt_string because it also accepts a mask, eg: mark=7/0xffffffff. But
if you only put in a number, for some reason it does not think that is
valid text. Maybe Hugh (CC: added) can share some light on that?

Post by Sébastien Lefevre
- That was enough to make my workaround based on a updown script work: it can safely set policies without involving mangling/marking, the any-to-any policies set by libreswan being just unused and not matching any packets anymore. That's much less elegant than the complete solution you proposed, but it works.

I guess since you "hardcode" the network ranges, you didn't need to use
marking right? You just used marking to ensure all default unmarked
packets would not hit the policy? But you must have marked something
in updown to make it match the policy?

Post by Sébastien Lefevre
- When stopping the tunnel via an "ipsec auto --delete/down conn", the dir out policy remains in place and is not automatically removed by libreswan, while the dir in and dir fwd are (got a ERROR: netlink XFRM_MSG_DELPOLICY response for flow eroute_connection delete included errno 2: No such file or directory, but only on --delete).

Ahh, we might not have added the mark bits to the delete command. We'll
look at fixing that before 3.15.

Post by Sébastien Lefevre

Post by Paul Wouters
then run something like this (assuming you only wanted to give the
MARK=7
REQID=7
iptables -t mangle -A PREROUTING -i eth0 -s 10.246.89.31/32 -m policy --dir in
--reqid $REQID -j MARK --set-xmark $MARK
iptables -t mangle -A PREROUTING -i eth0 -s 10.246.89.31/32 -m policy --dir in
--reqid $REQID -j CONNMARK --save-mark

I tried this approach instead of setting explicit policies, but didn't manage to get it work, probably due to my poor knowledge of iptables and mangle/nat interactions (10.246.89.31/32 is the published source IP for SNATted systems SNATed on the host running libreswan).
Anyway, the initial problem has now a solution and we can connect to multiple Cisco VTI-based tunnels if needed. Thank you!

could you share how you actually are ensuring the right packets go into
the right tunnels if you dont use iptables mangling in updown?

Paul

[Sébastien Lefevre]

Post by Sébastien Lefevre

Post by Sébastien Lefevre
- That was enough to make my workaround based on a updown script

work: it can safely set policies without involving mangling/marking, the any-
to-any policies set by libreswan being just unused and not matching any
packets anymore. That's much less elegant than the complete solution you
proposed, but it works.
I guess since you "hardcode" the network ranges, you didn't need to use
marking right? You just used marking to ensure all default unmarked packets
would not hit the policy? But you must have marked something in updown to
make it match the policy?

Marking is indeed useful here to ensure that default packets are not matching the policy, which was my initial problem.
In my updown, instead of marking packets with mangle, I still use my old approach of setting narrower policies:

--
Updown script:

# We want to route LOCAL_IP <-> any, not any <-> any, through the tunnel

LOCAL_IP=10.246.89.31

# Update the SPD to use <local-ip>/ <-> /any through the tunnel
function set_specific_rules() {
log "Adding specific-ip-to-any SPD entries..."
ip xfrm policy add src $LOCAL_IP/32 dst 0.0.0.0/0 dir out tmpl src $PLUTO_ME dst $PLUTO_PEER proto esp reqid $PLUTO_SA_REQID mode tunnel priority 2000
log "Adding dir out result: $?"
ip xfrm policy add dst $LOCAL_IP/32 src 0.0.0.0/0 dir in tmpl dst $PLUTO_ME src $PLUTO_PEER proto esp reqid $PLUTO_SA_REQID mode tunnel priority 2000
log "Adding dir in result: $?"
ip xfrm policy add dst $LOCAL_IP/32 src 0.0.0.0/0 dir fwd tmpl dst $PLUTO_ME src $PLUTO_PEER proto esp reqid $PLUTO_SA_REQID mode tunnel priority 2000
log "Adding dir fwd result: $?"
}

function unset_specific_rules() {
log "Removing customized specific-ip-to-any SPD entries..."
ip xfrm policy del src $LOCAL_IP/32 dst 0.0.0.0/0 dir out
log "Removal for dir out result: $?"
ip xfrm policy del src 0.0.0.0/0 dst $LOCAL_IP/32 dir in
log "Removal for dir in result: $?"
ip xfrm policy del src 0.0.0.0/0 dst $LOCAL_IP/32 dir fwd
log "Removal for dir fwd result: $?"
}

# This script is called with a verb:
# up-client, then prepare-client, then route-client.
# Based on tests, even on up-client netkeys rules are set already (i.e. SPD updated).
# However because this looks prettier, we will update the rules in route-client only.
case $PLUTO_VERB in
route-client)
set_specific_rules
;;
unroute-client)
unset_specific_rules
;;
*)
log "$0 called with verb $PLUTO_VERB"
;;
Esac

--

As a matter of fact, I don't need to set these rules in a updown script, they could have been set once for all outside a connection.
However "ipsec (re)start" resets all set policies, so that's still a convenient place to do it.

Post by Sébastien Lefevre
could you share how you actually are ensuring the right packets go into the
right tunnels if you dont use iptables mangling in updown?

See above.
In our VPN concentrator model, we expose a single private IP address specific to each tunnel. So based on this specific IP, we can select the tunnel to use.
This approach does not block packets if the VPN is not up (they'll be routed through the default gateway), so this is still far from perfect.

Thanks,
-seb

[Paul Wouters]

Post by Sébastien Lefevre
Marking is indeed useful here to ensure that default packets are not matching the policy, which was my initial problem.

Ahh I see. Of course manually adding/removing XFRM rules is not
recommended because pluto is not aware of those and it keeps its
on list of what it thinks is in the kernel.

Post by Sébastien Lefevre
In our VPN concentrator model, we expose a single private IP address specific to each tunnel. So based on this specific IP, we can select the tunnel to use.
This approach does not block packets if the VPN is not up (they'll be routed through the default gateway), so this is still far from perfect.

Right.

So your use case would be fixed with leftpolicynets=a.b.c.d/32 and rightpolicynets=0.0.0.0/0

Paul

[Sébastien Lefevre]

Post by Paul Wouters
So your use case would be fixed with leftpolicynets=a.b.c.d/32 and
rightpolicynets=0.0.0.0/0

Just to make sure: are you talking of future options or are they already available? That sounds *exactly* like what I was looking for...
(I can't find them in the current codebase, so maybe I didn't waste your time after all :)

-seb

[Paul Wouters]

Post by Sébastien Lefevre

Post by Paul Wouters
So your use case would be fixed with leftpolicynets=a.b.c.d/32 and
rightpolicynets=0.0.0.0/0

Just to make sure: are you talking of future options or are they already available? That sounds *exactly* like what I was looking for...
(I can't find them in the current codebase, so maybe I didn't waste your time after all :)

Future options, sorry. although it should not be too hard to implement.
It just needs to take a string and pass that to the updown script,
which can then run the iptables rules required. Possibly we need
an option to generate a unique mark so people don't have to specify
the mark manually.

Paul

[Sébastien Lefevre]

That's perfect.

Looking forward such a future release, in the meantime we still have options to support this Cisco VTI model.

Again, thank you very much for your prompt suggestions and advices.

Cheers,
-seb

Post by Sébastien Lefevre

Post by Sébastien Lefevre

Post by Paul Wouters
So your use case would be fixed with leftpolicynets=a.b.c.d/32 and
rightpolicynets=0.0.0.0/0

Just to make sure: are you talking of future options or are they already

available? That sounds *exactly* like what I was looking for...

Post by Sébastien Lefevre
(I can't find them in the current codebase, so maybe I didn't waste
your time after all :)

Future options, sorry. although it should not be too hard to implement.
It just needs to take a string and pass that to the updown script, which can
then run the iptables rules required. Possibly we need an option to generate
a unique mark so people don't have to specify the mark manually.
Paul

[D. Hugh Redelmeier]

| From: Paul Wouters <***@nohats.ca>

| On Thu, 3 Dec 2015, Sébastien Lefevre wrote:
| > - The connection's mark parameter expects a string, so this is mark="7" to
| > make it work. I don't know if this is on purpose.
|
| That's an odd bug in the parser it seems. The type value of mark is
| kt_string because it also accepts a mask, eg: mark=7/0xffffffff. But
| if you only put in a number, for some reason it does not think that is
| valid text. Maybe Hugh (CC: added) can share some light on that?

This is a manifestation of the simplistic parser. It thinks that
strings and numbers are distinct. mark, since it needs to handle
complicated values, requires strings. And a number isn't a string.

There are many things that could and should be improved in the parser.
This one seems easy to live with so I suggest fixing the manual so
that users don't get surprised.