How to get LXD containers get IP from the LAN with routed network

I am new to lxc and I am trying to setup a routed profile for an lxc container in order to make the container reachable from the host as well as other servers on the local network over a wifi interface.

I am following the steps in this page and although the profile created successfully, when I add it to the lxc container and restart the container, no IP address is ever assigned to the container.

Where would I need to look for logs to get an idea of where the setup is going wrong?

I am trying this on Ubuntu 20.04 using lxc version 4.0.2.

Thanks in advance,
PM

Simos

I am on Ubuntu 20.04 system with my Wifi Interface on Host = wlp3s0

whose IP is 192.168.1.81

I tried your Profile but changed the Parent to wlp3s0

= = = = = = =

config:
user.network-config: |
version: 2
ethernets:
eth0:
addresses:
– 192.168.1.200/32
nameservers:
addresses:
– 8.8.8.8
search: []
routes:
– to: 0.0.0.0/0
via: 169.254.0.1
description: Default LXD profile
devices:
eth0:
ipv4.address: 192.168.1.200
nictype: routed
parent: wlp3s0
type: nic
name: routed
used_by:

= = = = = = =

Created a test Ubuntu 20.04 container naamed “test”

$ lxc launch ubuntu:18.04 test –profile default –profile routed_200

$ lxc list
shows Container test has IP address 192.168.1.200

From HOST I can PING container’s IP 192.168.1.200

From Container TEST I can Ping my Host: 192.168.1.81

From Container TEST I can Ping my Router: 192.168.1.1

From my Container I CANNOT Ping http://www.google.com or any other Internet site or IP.

Any ideas?

Thanks
Brian Mullan

Hello @simos,

updated the profiles to include on-link: true as you suggested and I can confirm that Ubuntu 20.04 containers using routed profiles are now networked as expected.

Cheers,
PM

@Ponder Muse does your host use wifi or eth ?

Hi @bmullan, only just reading this now. My LXC containers are all networked over wifi.

I tried with routed network it worked but was not able to internet from container (like ping google.com) but other hosts are reachable from the container

My host machine is not a VM, it is a bare metal box and it is running with ubuntu 20.04

~ lxc list
+------+---------+----------------------+------+-----------+-----------+
| NAME |  STATE  |         IPV4         | IPV6 |   TYPE    | SNAPSHOTS |
+------+---------+----------------------+------+-----------+-----------+
| u1   | RUNNING | 192.168.1.200 (eth0) |      | CONTAINER | 0         |
+------+---------+----------------------+------+-----------+-----------+

~ lxc exec u1 bash
root@u1:~# ping 192.168.1.36 
PING 192.168.1.36 (192.168.1.36) 56(84) bytes of data.
64 bytes from 192.168.1.36: icmp_seq=1 ttl=64 time=0.116 ms
64 bytes from 192.168.1.36: icmp_seq=2 ttl=64 time=0.069 ms
--- 192.168.1.36 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1027ms
rtt min/avg/max/mdev = 0.069/0.092/0.116/0.023 ms

root@u1:~# ping google.com
ping: google.com: Temporary failure in name resolution

My profile config

lxc profile show routed
config:
  user.network-config: |
    version: 2
    ethernets:
        eth0:
            addresses:
            - 192.168.1.200/24
            nameservers:
                addresses:
                - 8.8.8.8
                search: []
            routes:
            -   to: 0.0.0.0/0
                via: 169.254.0.1
                on-link: true
description: Default LXD profile
devices:
  eth0:
    ipv4.address: 192.168.1.200
    nictype: routed
    parent: wlp3s0
    type: nic
name: routed
used_by:
- /1.0/instances/u1

lxc exec u1 bash
root@u1:~# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0@if28: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether 9a:0e:27:a5:33:d9 brd ff:ff:ff:ff:ff:ff link-netnsid 0
root@u1:~#

root@u1:~# ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
64 bytes from 1.1.1.1: icmp_seq=1 ttl=56 time=1001 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=56 time=102 ms
64 bytes from 1.1.1.1: icmp_seq=3 ttl=56 time=23.5 ms
64 bytes from 1.1.1.1: icmp_seq=4 ttl=56 time=46.3 ms
64 bytes from 1.1.1.1: icmp_seq=5 ttl=56 time=70.8 ms

I am using ubuntu:20.04 container, Let me try other steps explained above, tx Simos once again.

config:
  user.network-config: |
    version: 2
    ethernets:
        eth0:
            addresses:
            - 192.168.100.200/32
            nameservers:
                addresses:
                - 8.8.8.8
                search: []
            routes:
            -   to: 0.0.0.0/0
                via: 169.254.0.1
                on-link: true
description: Default LXD profile
devices:
  eth0:
    ipv4.address: 192.168.100.200
    nictype: routed
    parent: wlan0
    type: nic
name: routed_192.168.100.200
used_by:


lxc launch ubuntu:20.04 myrouted --profile default --profile routed_192.168.100.200
Creating myrouted
Starting myrouted
Error: Error setting up reverse path filter: Failed adding reverse path filter rules for instance device "myrouted.eth0" (inet): Failed apply nftables config: Failed to run: nft 
table inet lxd {
chain prert.myrouted.eth0 {
    type filter hook prerouting priority -300; policy accept;
    iif "vethab605f49" fib saddr . iif oif missing drop
}
}

Error: Could not process rule: No such file or directory

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Try <code>lxc info --show-log local:myrouted</code> for more info

lxc info --show-log local:myrouted
Name: myrouted
Location: none
Remote: unix://
Architecture: aarch64
Created: 2021/05/03 21:36 UTC
Status: Stopped
Type: container
Profiles: default, routed_192.168.100.200

Log:

lxc myrouted 20210503213622.158 WARN     cgfsng - cgroups/cgfsng.c:mkdir_eexist_on_last:1129 - File exists - Failed to create directory "/sys/fs/cgroup/cpuset//lxc.monitor.myrouted"
lxc myrouted 20210503213622.167 WARN     cgfsng - cgroups/cgfsng.c:mkdir_eexist_on_last:1129 - File exists - Failed to create directory "/sys/fs/cgroup/cpuset//lxc.payload.myrouted"
lxc myrouted 20210503213622.182 WARN     cgfsng - cgroups/cgfsng.c:fchowmodat:1550 - No such file or directory - Failed to fchownat(17, memory.oom.group, 1000000000, 0, AT_EMPTY_PATH | AT_SYMLINK_NOFOLLOW )
lxc myrouted 20210503213623.711 ERROR    network - network.c:lxc_ovs_delete_port:2353 - Failed to delete "vethab605f49" from openvswitch bridge "wlan0": ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory
lxc myrouted 20210503213623.711 WARN     network - network.c:lxc_delete_network_priv:3236 - Failed to remove port "vethab605f49" from openvswitch bridge "wlan0"

Can you help me with this problem? Your ipvlan tutorial works fine, but I have this problem with routed network 🙁

For some reason I can’t see my comment, so I’ll post it again. If this turns out to be a duplicate, you can delete it.

Comment:

Oh, sorry, the main interface is actually wlxXXXXXXXXXXXX (I copied the logs to a shared file, where I accidentally replaced it with wlan0).
The profile I provided above actually contains the wlx… interface, not wlan0.
However, with regard to IP, everything is correct: in my local network, devices have addresses starting with 192.168.100.xxx
DHCP on my router works in the range 192.168.100.20 – 192.168.100.199 .

However, if it matters, my system also has a wlan0 interface, but it is disabled (state DOWN). wlan0 is a WiFi module built into the device.
The wlx… interface on my system is an external USB device.

The host device itself is a Raspberry Pi 4 Model B (8 GB).
The operating system installed on it is Ubuntu 21.04 Server (64 bit).

$ snap list
Name    Version   Rev    Tracking         Publisher   Notes
core18  20210309  2002   latest/stable    canonical   base
lxd     4.13      20227  latest/stable/   canonical   -
snapd   2.49.2    11584  latest/stable    canonical   snapd

Hi, thank you for the wonderful guides. I am having a hard time figuring out what option to use when configuring the networking for my lxd container.

My host machine has 2 physical network adapters: an ethernet and separate wireless one. My isp also provides two separate public IP addresses. My host is already set up in such a way that if I connect with the eth connection I get one IP, and if I connect with the wifi I get the other one.

I have an lxd container already set up and working perfectly with macvlan using your guide and the ethernet adapter.

I want a second container that uses the wifi adapter to get the second ip address. I tried using the bridge networking but it seems that I can’t because “Device does not allow enslaving to a bridge.”

Is routed the way to go? What about the “physical” nic type?

Thanks!