[Swan] vxlan support

[antonio]

hi, did anyone configured vxlan with libreswan? my idea is to create a tunel ipsec and then send all the vxlan traffic trought the tunnel. i guess that would be something like l2tp/ipsec, but i must create ip xfrm rules to re-direct the traffic, no? apreciate any sugestion on how to do it. regards, antonio

[Paul Wouters]

Post by antonio
did anyone configured vxlan with libreswan?
my idea is to create a tunel ipsec  and then send all the vxlan traffic trought the tunnel. i guess that would be
something like l2tp/ipsec, but i must create ip xfrm rules to re-direct the traffic, no?
apreciate any sugestion on how to do it.

I dont know vxlan, but assuming it is some IP based encapsulation, you
should be able to setup a tunnel that only allows the encapsulated
protocol (using leftprotoport= and rightprotoport=)

Paul

[Paul Wouters]

vxlan tunnels an L2 frame over udp. (rfc 7348)

Ahh. I see.

are you planning on applying ipsec to the vxlan'ed frame?
If yes, you'd have to set up your swan tunnel config for something like
leftprotoport=udp/4789
and
rightprotoport=udp/4789
(you'd need 2 tunnels per peering pair)

Why two? Are both peers using an ephemeral souce port? If it is port
4789 to port 4789, wouldn't one tunnel be enough?

Paul

[António Silva]

Thanks for the reply.


My idea is to have traffic between vxlan encrypted:

host1/vxlan1
      |  x  |
      |  x  |
 ipsec tunel
      |  x  |
      |  x  |
host2/vxlan1


Do i still need to connect to tunnels?

I'm trying to configure it now..

Post by Paul Wouters
Why two? Are both peers using an ephemeral souce port? If it is port
4789 to port 4789, wouldn't one tunnel be enough?

I'm assuming that the local host is both sends (to other node's
udp port 4789) and receives (on udp port 4789 from other peers)
vxlan packets, and that we want ipsec for both directions.
Depends on what Antonio is trying to achieve, I suppose.
--Sowmini

[António Silva]

I try to set the leftprotoport / rightprotoport=udp/4789 , i can ping
the ip on boxB going trough the vxlan but the traffic is not
encrypted... is not going to the ipsec tunnel... on side is behind NAT
but the ipsec tunnel stablish fine.


Sowmini, you suggest using two tunnels, how should they be?

With this configuration i should have on side, no?




there is my configuration:


BoxA
# configure vxlan
ip link add vxlan0 type vxlan id 1 dstport 4789
ip addr add 192.168.10.1/24 dev vxlan0
bridge fdb add to 00:00:00:00:00:00 dst 10.10.100.10 dev vxlan0
ip l set dev vxlan0 up

ipsec.conf:
conn boxA-nat
    rightsubnet=vhost:%priv
    also=boxA

conn boxA
    pfs=no
    type=transport
    auto=start
    phase2=esp
    sha2-truncbug=yes
    authby=secret
    keyingtries=3
    ikelifetime=8h
    salifetime=1h
    leftid=10.10.100.100
    left=%defaultroute
    leftprotoport=udp/4789
    ikev2=never
    right=10.10.100.10
    rightid=10.10.100.106
    rightprotoport=udp/4789
    dpddelay=30
    dpdtimeout=60
    dpdaction=hold


BoxB:
# configure vxlan
ip link add vxlan0 type vxlan id 1 dstport 4789
ip addr add 192.168.10.2/24 dev vxlan0
bridge fdb append to 00:00:00:00:00:00 dst 10.10.100.100 dev vxlan0
ip l set dev vxlan0 up


ipsec.conf:

conn boxB-nat
    rightsubnet=vhost:%priv
    also=boxB

conn boxB
    pfs=no
    type=transport
    auto=add
    phase2=esp
    sha2-truncbug=yes
    authby=secret
    keyingtries=3
    ikelifetime=8h
    salifetime=1h
    left=10.10.100.10
    leftprotoport=udp/4789
    ikev2=never
    leftid=10.10.100.10
    right=10.10.100.100
    rightid=10.10.100.100
    rightprotoport=udp/4789
    dpddelay=30
    dpdtimeout=60
    dpdaction=hold



Logs from boxB

Jan 23 23:39:46 a2 pluto[27131]: "boxB-nat"[1] 10.10.100.100 #1:
responding to Main Mode from unknown peer 10.10.100.100
Jan 23 23:39:46 a2 pluto[27131]: "boxB-nat"[1] 10.10.100.100 #1:
WARNING: connection boxB-nat PSK length of 7 bytes is too short for
sha2_256 PRF in FIPS mode (16 bytes required)
Jan 23 23:39:46 a2 pluto[27131]: "boxB-nat"[1] 10.10.100.100 #1:
transition from state STATE_MAIN_R0 to state STATE_MAIN_R1
Jan 23 23:39:46 a2 pluto[27131]: "boxB-nat"[1] 10.10.100.100 #1:
STATE_MAIN_R1: sent MR1, expecting MI2
Jan 23 23:39:46 a2 pluto[27131]: "boxB-nat"[1] 10.10.100.100 #1:
transition from state STATE_MAIN_R1 to state STATE_MAIN_R2
Jan 23 23:39:46 a2 pluto[27131]: "boxB-nat"[1] 10.10.100.100 #1:
STATE_MAIN_R2: sent MR2, expecting MI3
Jan 23 23:39:46 a2 pluto[27131]: "boxB-nat"[1] 10.10.100.100 #1: Peer ID
is ID_IPV4_ADDR: '10.10.100.100'
Jan 23 23:39:46 a2 pluto[27131]: "boxB-nat"[1] 10.10.100.100 #1:
transition from state STATE_MAIN_R2 to state STATE_MAIN_R3
Jan 23 23:39:46 a2 pluto[27131]: "boxB-nat"[1] 10.10.100.100 #1: new NAT
mapping for #1, was 10.10.100.100:500, now 10.10.100.100:9816
Jan 23 23:39:46 a2 pluto[27131]: "boxB-nat"[1] 10.10.100.100 #1:
STATE_MAIN_R3: sent MR3, ISAKMP SA established {auth=PRESHARED_KEY
cipher=aes_256 integ=sha2_256 group=MODP2048}
Jan 23 23:39:47 a2 pluto[27131]: "boxB-nat"[1] 10.10.100.100 #1: the
peer proposed: 10.10.100.10/32:17/4789 -> 192.168.1.97/32:17/4789
Jan 23 23:39:47 a2 pluto[27131]: "boxB-nat"[1] 10.10.100.100 #1:
NAT-Traversal: received 2 NAT-OA. Using first, ignoring others
Jan 23 23:39:47 a2 pluto[27131]: "boxB-nat"[1] 10.10.100.100 #2:
responding to Quick Mode proposal {msgid:db11192c}
Jan 23 23:39:47 a2 pluto[27131]: "boxB-nat"[1] 10.10.100.100 #2: us:
10.10.100.10<10.10.100.10>:17/4789
Jan 23 23:39:47 a2 pluto[27131]: "boxB-nat"[1] 10.10.100.100 #2: them:
10.10.100.100<10.10.100.100>:17/4789===192.168.1.97/32
Jan 23 23:39:47 a2 pluto[27131]: "boxB-nat"[1] 10.10.100.100 #2:
transition from state STATE_QUICK_R0 to state STATE_QUICK_R1
Jan 23 23:39:47 a2 pluto[27131]: "boxB-nat"[1] 10.10.100.100 #2:
STATE_QUICK_R1: sent QR1, inbound IPsec SA installed, expecting QI2
transport mode {ESP/NAT=>0x6e4fb7e7 <0xd041d2f6 xfrm=AES_128-HMAC_SHA1
NATOA=192.168.1.97 NATD=10.10.100.100:9816 DPD=active}
Jan 23 23:39:47 a2 pluto[27131]: "boxB-nat"[1] 10.10.100.100 #2:
transition from state STATE_QUICK_R1 to state STATE_QUICK_R2
Jan 23 23:39:47 a2 pluto[27131]: "boxB-nat"[1] 10.10.100.100 #2:
STATE_QUICK_R2: IPsec SA established transport mode {ESP/NAT=>0x6e4fb7e7
<0xd041d2f6 xfrm=AES_128-HMAC_SHA1 NATOA=192.168.1.97
NATD=10.10.100.100:9816 DPD=active}



[23:45:14][a2][~]# ip xfrm s
src 10.10.100.100 dst 10.10.100.10
    proto esp spi 0xd041d2f6 reqid 16397 mode transport
    replay-window 32
    auth-trunc hmac(sha1) 0xfdda97ee7fde32a70e7e1ce5b01920188219991b 96
    enc cbc(aes) 0xf51066695322609bca3f4c5bbcb62a3e
    encap type espinudp sport 9816 dport 4500 addr 0.0.0.0
    anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000
    sel src 10.10.100.100/32 dst 10.10.100.10/32 proto udp sport 4789
dport 4789
src 10.10.100.10 dst 10.10.100.100
    proto esp spi 0x6e4fb7e7 reqid 16397 mode transport
    replay-window 32
    auth-trunc hmac(sha1) 0xdbdeb0f206d448b2f9b5d1935261bd349668b500 96
    enc cbc(aes) 0x1f0e7b47308b63712770662021cce883
    encap type espinudp sport 4500 dport 9816 addr 0.0.0.0
    anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000
    sel src 10.10.100.10/32 dst 10.10.100.100/32 proto udp sport 4789
dport 4789


[23:47:32][a2][~]# ip xfrm p
src 10.10.100.10/32 dst 10.10.100.100/32 proto udp sport 4789 dport 4789
    dir out priority 1952 ptype main
    tmpl src 0.0.0.0 dst 0.0.0.0
        proto esp reqid 16397 mode transport
src 10.10.100.100/32 dst 10.10.100.10/32 proto udp sport 4789 dport 4789
    dir in priority 1952 ptype main
    tmpl src 0.0.0.0 dst 0.0.0.0
        proto esp reqid 16397 mode transport

Post by António Silva
Thanks for the reply.
host1/vxlan1
      |  x  |
      |  x  |
 ipsec tunel
      |  x  |
      |  x  |
host2/vxlan1
Do i still need to connect to tunnels?
I'm trying to configure it now..

Post by Paul Wouters
Why two? Are both peers using an ephemeral souce port? If it is port
4789 to port 4789, wouldn't one tunnel be enough?

I'm assuming that the local host is both sends (to other node's
udp port 4789) and receives (on udp port 4789 from other peers)
vxlan packets, and that we want ipsec for both directions.
Depends on what Antonio is trying to achieve, I suppose.
--Sowmini

[Sowmini Varadhan]

I try to set the leftprotoport / rightprotoport=udp/4789 , i can ping the ip
on boxB going trough the vxlan but the traffic is not encrypted... is not
going to the ipsec tunnel... on side is behind NAT but the ipsec tunnel
stablish fine.

Paul is the expert here..

Sowmini, you suggest using two tunnels, how should they be?

By setting up the 2 tunnel config files as above (for leftprotport and
rightprotoport).

With this configuration i should have on side, no?

Again, Paul's the expert here.

But this (leftprotoport, rightprotoport) is what I do for my experiments
with, e.g. iperf.

[Paul Wouters]

I try to set the leftprotoport / rightprotoport=udp/4789 , i can ping the ip
on boxB going trough the vxlan but the traffic is not encrypted..

Well yes, ping does not use udp port 4789 :)

Sowmini, you suggest using two tunnels, how should they be?
conn boxA

[...]

    leftprotoport=udp/4789
    rightprotoport=udp/4789

I think you want:

conn boxA-out
[...]
leftprotoport=udp
rightprotoport=udp/4789

conn boxA-in
[...]
leftprotoport=udp/4789
rightprotoport=udp

That covers two flows, any ephemeral port to remote udp 4789
and any ephemeral port from remote to local udp 4789

Paul

[antonio]

Hi,

Thanks Paul, this work defined but i think i found a issue, the
left/right protocol are not respected... and so the tunnels are partial
up and i cannot send vxlan traffic through the vpn.


My current conf is (boxA and boxB - reverted left/right params):

conn ipsec9convxlanout
        also=ipsec9convxlan
        leftprotoport=17/0
        rightprotoport=17/4789
        auto=start

conn ipsec9convxlanin
        also=ipsec9convxlan
        leftprotoport=17/4789
        rightprotoport=17/0
        auto=start

conn ipsec9convxlan
        type=transport
        leftrsasigkey=%cert
        leftcert=LabVxLANandDemoVxLAN
        rightrsasigkey=%cert
        leftid=@LabVxLAN
        left=192.168.1.108
        right=20.20.10.4
        rightid=@DemoVxLAN
        dpddelay=30
        dpdtimeout=60
        dpdaction=restart


My left side is behind NAT and i cannot force port 500 or 4500 to
libreswan box, so i end up with partial tunnels up.

left both conns are up:

ipsec whack --trafficstatus
006 #3: "ipsec1convxlanin", type=ESP, add_time=1526136419, inBytes=0,
outBytes=0, id='@LabVxLAN'
006 #2: "ipsec1convxlanout", type=ESP, add_time=1526136418, inBytes=0,
outBytes=0, id='@LabVxLAN'


right side is wrong:
ipsec whack --trafficstatus
006 #6: "ipsec9convxlanin", type=ESP, add_time=1526136418, inBytes=0,
outBytes=0, id='@DemoVxLAN'
006 #7: "ipsec9convxlanin", type=ESP, add_time=0, inBytes=0, outBytes=0,
id='@DemoVxLAN'

/Expected:ipsec whack --trafficstatus
006 #6: "ipsec9convxlanin", type=ESP, add_time=1526136418, inBytes=0,
outBytes=0, id='@DemoVxLAN'
006 #7: "ipsec9convxlanout", type=ESP, add_time=//1526136418, inBytes=0,
outBytes=0, id='@DemoVxLAN'/


When connecting the ipsec1convxlanout from left side it detects the
connection as ipsec9convxlanin....

If i can "dial out" from the right to the left side (removing the nat
issue), all is ok.


Can i do this with my current configuration? Or i should defined two
different connections (different ids)?

Post by Paul Wouters

Post by António Silva
I try to set the leftprotoport / rightprotoport=udp/4789 , i can ping
the ip on boxB going trough the vxlan but the traffic is not encrypted..

Well yes, ping does not use udp port 4789 :)

Post by António Silva
Sowmini, you suggest using two tunnels, how should they be?
conn boxA

[...]

Post by António Silva
    leftprotoport=udp/4789
    rightprotoport=udp/4789

conn boxA-out
    [...]
    leftprotoport=udp
    rightprotoport=udp/4789
conn boxA-in
    [...]
    leftprotoport=udp/4789
    rightprotoport=udp
That covers two flows, any ephemeral port to remote udp 4789
and any ephemeral port from remote to local udp 4789
Paul

[Paul Wouters]

Thanks Paul, this work defined but i think i found a issue, the left/right protocol are not respected... and
so the tunnels are partial up and i cannot send vxlan traffic through the vpn.
conn ipsec9convxlanout
        also=ipsec9convxlan
        leftprotoport=17/0
        rightprotoport=17/4789
        auto=start
conn ipsec9convxlanin
        also=ipsec9convxlan
        leftprotoport=17/4789
        rightprotoport=17/0
        auto=start
conn ipsec9convxlan
        type=transport
        leftrsasigkey=%cert
        leftcert=LabVxLANandDemoVxLAN
        rightrsasigkey=%cert
        left=192.168.1.108
        right=20.20.10.4
        dpddelay=30
        dpdtimeout=60
        dpdaction=restart

That should work.

My left side is behind NAT and i cannot force port 500 or 4500 to libreswan box, so i end up with partial
tunnels up.

The NAT should not matter as long as one end is not behind NAT (or
behind a port forward)

ipsec whack --trafficstatus

Although no traffic has ever matched these tunnels as the byte counters
are all zero.

ipsec whack --trafficstatus

That's odd. you should check the logs what happened. It looks like one
might have replaced the other.

When connecting the  ipsec1convxlanout from left side it detects the connection as ipsec9convxlanin....

During the IKE negotiation, pluto cannot yet tell which of the two will
match. It is perfectly normal for it to "switch" from one conn to the
other once it learns the phase2/ipsec selectors.

Can i do this with my current configuration? Or i should defined two different connections (different ids)?

You can do that but it should not be needed.

Paul

[antonio]

Hi Paul,

Attach the full log in boxB when connecting the remote endpoint (boxA).
I've dynamic ip from BoxA, so i change the conf to %any in the right
parameter.

Also from boxB to boxA i cannot start the connection because of
firewall/NAT issue.


From the log i see.

When remote is dialing convxlanout:

May 14 13:29:44.205380: | peer client is 10.136.31.107
May 14 13:29:44.205421: | peer client protocol/port is 17/0
May 14 13:29:44.205441: | our client is 90.175.130.18
May 14 13:29:44.205464: | our client protocol/port is 17/4789
May 14 13:29:44.205490: "ipsec9convxlanin"[1] 84.79.21.145 #1: the peer
proposed: 90.175.130.18/32:17/4789 -> 10.136.31.107/32:17/0


When remote is dialing convxlanin:

May 14 13:29:44.217021: | peer client is 10.136.31.107
May 14 13:29:44.217041: | peer client protocol/port is 17/4789
May 14 13:29:44.217060: | our client is 90.175.130.18
May 14 13:29:44.217080: | our client protocol/port is 17/0
May 14 13:29:44.217104: "ipsec9convxlanin"[1] 84.79.21.145 #1: the peer
proposed: 90.175.130.18/32:17/4789 -> 10.136.31.107/32:17/0

Here should it print:

May 14 13:29:44.217104: "ipsec9convxlanin"[1] 84.79.21.145 #1: the peer
proposed: 90.175.130.18/32:17/*0* -> 10.136.31.107/32:17/*4789*



As for the counters to 0, that's another issue...but i think is related
to the fact i'm not able to establish the convxlanout in the boxB.

Although i see the policy rules:

src 10.136.31.107/32 dst 90.175.130.18/32 proto udp dport 4789
    dir out priority 2016 ptype main
    tmpl src 0.0.0.0 dst 0.0.0.0
        proto esp reqid 16393 mode transport


and the traffic is well generated:

14:05:42.664684 Out fa:16:31:3b:f6:45 ethertype IPv4 (0x0800), length
112: (tos 0x0, ttl 64, id 64205, offset 0, flags [none], proto UDP (17),
length 96)
    10.136.31.107.40204 > 90.175.130.18.4789: VXLAN, flags [I] (0x08),
vni 20


No traffic is encrypted.

Post by Paul Wouters

Post by antonio
Thanks Paul, this work defined but i think i found a issue, the
left/right protocol are not respected... and
so the tunnels are partial up and i cannot send vxlan traffic through the vpn.
conn ipsec9convxlanout
        also=ipsec9convxlan
        leftprotoport=17/0
        rightprotoport=17/4789
        auto=start
conn ipsec9convxlanin
        also=ipsec9convxlan
        leftprotoport=17/4789
        rightprotoport=17/0
        auto=start
conn ipsec9convxlan
        type=transport
        leftrsasigkey=%cert
        leftcert=LabVxLANandDemoVxLAN
        rightrsasigkey=%cert
        left=192.168.1.108
        right=20.20.10.4
        dpddelay=30
        dpdtimeout=60
        dpdaction=restart

That should work.

Post by antonio
My left side is behind NAT and i cannot force port 500 or 4500 to
libreswan box, so i end up with partial
tunnels up.

The NAT should not matter as long as one end is not behind NAT (or
behind a port forward)

Post by antonio
ipsec whack --trafficstatus
006 #3: "ipsec1convxlanin", type=ESP, add_time=1526136419, inBytes=0,
006 #2: "ipsec1convxlanout", type=ESP, add_time=1526136418,

Although no traffic has ever matched these tunnels as the byte counters
are all zero.

Post by antonio
ipsec whack --trafficstatus
006 #6: "ipsec9convxlanin", type=ESP, add_time=1526136418, inBytes=0,
006 #7: "ipsec9convxlanin", type=ESP, add_time=0, inBytes=0,

That's odd. you should check the logs what happened. It looks like one
might have replaced the other.

Post by antonio
When connecting the  ipsec1convxlanout from left side it detects the
connection as ipsec9convxlanin....

During the IKE negotiation, pluto cannot yet tell which of the two will
match. It is perfectly normal for it to "switch" from one conn to the
other once it learns the phase2/ipsec selectors.

Post by antonio
Can i do this with my current configuration? Or i should defined two
different connections (different ids)?

You can do that but it should not be needed.
Paul