[Swan] "cannot install eroute" when second client connected from behind the same NAT

[j***@use.startmail.com]

Configured L2TP using slightly simplified instructions from https://blog.ls20.com/ipsec-l2tp-vpn-auto-setup-for-ubuntu-12-04-on-amazon-ec2/
(RHEL version https://gist.github.com/hwdsl2/e9a78a50e300d12ae195 )
I used latest libreswan-3.13-1.el6.i686 from epel, my own firewall rules and shorter sysctl list:

net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.all.rp_filter = 0

Configured two users as suggested in https://gist.github.com/hwdsl2/123b886f29f4c689f531

First user connects fine, but second times out, with "cannot install eroute". Here is a fragment from log file:

Jul 26 14:16:25 localhost pluto[4299]: "vpnpsk"[8] <client external IP> #27: responding to Quick Mode proposal {msgid:ebbfa25f}
Jul 26 14:16:25 localhost pluto[4299]: "vpnpsk"[8] <client external IP> #27: us: <server IP>/32===<server IP><<server IP>>:17/1701
Jul 26 14:16:25 localhost pluto[4299]: "vpnpsk"[8] <client external IP> #27: them: <client external IP>[<client internal IP>]:17/0
Jul 26 14:16:25 localhost pluto[4299]: "vpnpsk"[8] <client external IP> #27: cannot install eroute -- it is in use for "vpnpsk"[6] <client external IP> #6

I saw similar subject in archives (https://lists.libreswan.org/pipermail/swan/2014/001001.html) but it seems to be a slightly different case.
Is this an ipsec limitation or error in configuration?

Thanks,
Josh.

[Paul Wouters]

Post by j***@use.startmail.com
Configured L2TP using slightly simplified instructions from https://blog.ls20.com/ipsec-l2tp-vpn-auto-setup-for-ubuntu-12-04-on-amazon-ec2/
(RHEL version https://gist.github.com/hwdsl2/e9a78a50e300d12ae195 )
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.all.rp_filter = 0
Configured two users as suggested in https://gist.github.com/hwdsl2/123b886f29f4c689f531

This is not currently supported with NETKEY. You can get passed the
"eroute is in use" by adding overlapip=yes (I believe we removed the
stack restriction on that) but you still need some iptables rules
based on the reqid to ensure these two flows will work properly.

(We'd gladly receive patches for this :)

Paul

[j***@use.startmail.com]

Thanks for overlapip=yes suggestion, however, would you mind to let me know what "reqid" is?

Does https://libreswan.org/wiki/SAref_code sample have anything to do with this eroute problem?

In general, logs show that server sees real (behind NAT) client IP address and can, theoretically, construct unique eroute.

Also, there are several VPN providers which offer L2TP. Do you know if they have any NAT related limitations?

Post by Paul Wouters

Post by j***@use.startmail.com
First user connects fine, but second times out, with "cannot install

This is not currently supported with NETKEY. You can get passed the
"eroute is in use" by adding overlapip=yes (I believe we removed the
stack restriction on that) but you still need some iptables rules
based on the reqid to ensure these two flows will work properly.
(We'd gladly receive patches for this :)

Josh

[Steve Leung]

I have the same problem here. While doing some searches on Google, looks
like strongswan has a "connmark" plugin (
https://wiki.strongswan.org/projects/strongswan/wiki/Connmark) for this,
they are using a similar idea as Paul suggested I think, but they are
matching the spi instead. However in this way I think pluto will need to be
updated as well so "ip xfrm" will xfrm packets by src/dst and the mark
defined in iptables.

Still studying.. any pointer is appreciated :)


Best regards,
Steve

Post by j***@use.startmail.com
Thanks for overlapip=yes suggestion, however, would you mind to let me
know what "reqid" is?
Does https://libreswan.org/wiki/SAref_code sample have anything to do
with this eroute problem?
In general, logs show that server sees real (behind NAT) client IP address
and can, theoretically, construct unique eroute.
Also, there are several VPN providers which offer L2TP. Do you know if
they have any NAT related limitations?

Post by Paul Wouters

Post by j***@use.startmail.com
First user connects fine, but second times out, with "cannot install

This is not currently supported with NETKEY. You can get passed the
"eroute is in use" by adding overlapip=yes (I believe we removed the
stack restriction on that) but you still need some iptables rules
based on the reqid to ensure these two flows will work properly.
(We'd gladly receive patches for this :)

Josh
_______________________________________________
Swan mailing list
https://lists.libreswan.org/mailman/listinfo/swan

[Paul Wouters]

I have the same problem here. While doing some searches on Google, looks like strongswan has a "connmark"
plugin (https://wiki.strongswan.org/projects/strongswan/wiki/Connmark) for this, they are using a similar
idea as Paul suggested I think, but they are matching the spi instead. However in this way I think pluto
will need to be updated as well so "ip xfrm" will xfrm packets by src/dst and the mark defined in iptables.
Still studying.. any pointer is appreciated :)

We currently don't expose the SPI numbers to the updown scripts, although
we do expose the reqid. SPIs is something we can add if people want to use
it for connmark. It seems both spi and reqid are supposed with iptables:

http://ipset.netfilter.org/iptables-extensions.man.html

Apart from exposing the SPIs, we would not need to make any changes to
pluto. This is why we use the updown scripts, to give people to freedom
to do things on a per-sa basis. We could change the updown script to
detect NAT+transport mode and automatically insert the right iptables
rules when we see this happening. That would be my preference over a
new keyword.

Paul

[Steve Leung]

Thank you Paul, I'm wondering if this idea can be applied to NETKEY, I
guess in this case pluto will need to be updated as well? so that adding
new SA will include "mark", and then updown script can insert iptables rule
in the mangle table to set connmark according to different SPI.

Best regards,
Steve

Post by Steve Leung
I have the same problem here. While doing some searches on Google, looks

Post by Steve Leung
like strongswan has a "connmark"
plugin (https://wiki.strongswan.org/projects/strongswan/wiki/Connmark)
for this, they are using a similar
idea as Paul suggested I think, but they are matching the spi instead.
However in this way I think pluto
will need to be updated as well so "ip xfrm" will xfrm packets by src/dst
and the mark defined in iptables.
Still studying.. any pointer is appreciated :)

We currently don't expose the SPI numbers to the updown scripts, although
we do expose the reqid. SPIs is something we can add if people want to use
http://ipset.netfilter.org/iptables-extensions.man.html
Apart from exposing the SPIs, we would not need to make any changes to
pluto. This is why we use the updown scripts, to give people to freedom
to do things on a per-sa basis. We could change the updown script to
detect NAT+transport mode and automatically insert the right iptables
rules when we see this happening. That would be my preference over a
new keyword.
Paul

[j***@use.startmail.com]

I don't know how it is done but softether vpn server accepts at least two L2TP connections from behind the same NAT/subnet and traffic flows to/from both devices.

Post by Steve Leung
Thank you Paul, I'm wondering if this idea can be applied to NETKEY, I
guess in this case pluto will need to be updated as well? so that adding
new SA will include "mark", and then updown script can insert iptables rule
in the mangle table to set connmark according to different SPI.
Best regards,
Steve

Post by Steve Leung
I have the same problem here. While doing some searches on Google, looks

Post by Steve Leung
like strongswan has a "connmark"
plugin (https://wiki.strongswan.org/projects/strongswan/wiki/Connmark)
for this, they are using a similar
idea as Paul suggested I think, but they are matching the spi instead.
However in this way I think pluto
will need to be updated as well so "ip xfrm" will xfrm packets by src/dst
and the mark defined in iptables.
Still studying.. any pointer is appreciated :)

We currently don't expose the SPI numbers to the updown scripts, although
we do expose the reqid. SPIs is something we can add if people want to use
http://ipset.netfilter.org/iptables-extensions.man.html
Apart from exposing the SPIs, we would not need to make any changes to
pluto. This is why we use the updown scripts, to give people to freedom
to do things on a per-sa basis. We could change the updown script to
detect NAT+transport mode and automatically insert the right iptables
rules when we see this happening. That would be my preference over a
new keyword.
Paul

[j***@use.startmail.com]

Adding overlapip=yes allows second client connection but then both clients timeout and disconnect.

What iptables rules are needed? Are there any samples?

Regards,
Josh.

Post by Paul Wouters
This is not currently supported with NETKEY. You can get passed the
"eroute is in use" by adding overlapip=yes (I believe we removed the
stack restriction on that) but you still need some iptables rules
based on the reqid to ensure these two flows will work properly.