How to use the LXD Proxy Device to map ports between the host and the containers

LXD supports proxy devices, which is a way to proxy connections between the host and containers. This includes TCP, UDP and Unix socket connections, in any combination between each other, in any direction. For example, when someone connects to your host on port 80 (http), then this connection can be proxied to a container using a proxy device. In that way, you can isolate your Web server into a LXD container. By using a TCP proxy device, you do not need to use iptables instead.

There are 3³=9 combinations for connections between TCP, UDP and Unix sockets, as follows. Yes, you can proxy, for example, a TCP connection to a Unix socket!

  1. TCP to TCP, for example, to expose a container’s service to the Internet.
  2. TCP to UDP
  3. TCP to Unix socket
  4. UDP to UDP
  5. UDP to TCP
  6. UDP to Unix socket
  7. Unix socket to Unix socket, for example, to share the host’s X11 socket to a container. Or, to make available a host’s Unix socket into the container.
  8. Unix socket to TCP
  9. Unix socket to UDP

Earlier I wrote that you can make a connection in any direction. For example, you can expose the host’s Unix socket for X11 into the container so that the container can run X11 applications and have them appear on the host’s X11 server. Or, in the other way round, you can make available LXD’s Unix socket at the host to a container so that you can manage LXD from inside a container.

Note that LXD 3.0.x only supports TCP to TCP proxy devices. Support for UDP and Unix sockets was added in later versions.

Launching a container and setting up a Web server

Let’s launch a container, install a Web server, and, then expose the Web server to the local network (or the Internet, if you are using a VPS/Internet server).

First, launch the container.

$ lxc launch ubuntu:18.04 mycontainer
Creating mycontainer
Starting mycontainer

We get a shell into the container, update the package list and install nginx. Finally, verify that nginx is running.

ubuntu@mycontainer:~$ sudo apt update
ubuntu@mycontainer:~$ sudo apt install -y nginx
ubuntu@mycontainer:~$ curl http://localhost
...
 Welcome to nginx! 

Exposing the Web server of a container to the Internet

We logout to the host and verify that there is no Web server already running on port 80. If port 80 is not available on your host, change it to something else, like 8000. Finally, we create the TCP to TCP LXD Proxy Device.

ubuntu@mycontainer:~$ logout
$ lxc config device add mycontainer myport80 proxy listen=tcp:0.0.0.0:80 connect=tcp:127.0.0.1:80
Device myport80 added to mycontainer

The command that creates the proxy device is made of the following components.

  1. lxc config device add, we configure to have a device added,
  2. mycontainer, to the container mycontainer,
  3. myport80, with name myport80,
  4. proxy, a proxy device, we are adding a LXD Proxy Device.
  5. listen=tcp:0.0.0.0:80, we listen (on the host by default) on all network interfaces on TCP port 80.
  6. connect=tcp:127.0.0.1:80, we connect (to the container by default) to the existing TCP port 80 on localhost, which is our nginx.

Note that previously you would specify hostnames when you were creating LXD Proxy Devices. This is no longer supported (has security implications), therefore you get an error if you specify a hostname such as localhost. This post was primarily written because the top Google result on proxy devices is an old Reddit read-only post that suggests to use localhost.

Let’s test that the Web server in the container is accessible on the host. We can use both localhost (or 127.0.0.1) on the host to access the website of the container. We can also use the public IP address of the host (in this case, the LAN IP address) to access the container.

$ curl http://localhost
...
 Welcome to nginx! 
...
$ curl http://192.168.1.100
...
 Welcome to nginx! 
...

Other features of the proxy devices

By default, a proxy device exposes an existing service in the container to the host. If we need to expose an existing service on the host to a container, we would add the parameter bind=container to the proxy device command.

You can expose a single webserver to a port on the host. But how do you expose many web servers in containers to the host? You can use a reverse proxy that goes in front of the containers. To retain the remote IP address of the clients visiting the Web servers, you can add the proxy_protocol=true to enable support for the PROXY protocol. Note that you also need to enable the PROXY protocol on the reverse proxy.

Permanent link to this article: https://blog.simos.info/how-to-use-the-lxd-proxy-device-to-map-ports-between-the-host-and-the-containers/

3 comments

    • Yoann on September 7, 2019 at 09:06
    • Reply

    Hi, thanks for your post about lxc/d.
    Proxy device seems great, but in the use case you describe there is a problem, because in the access log of ngix container every requests seems to came from 127.0.0.1 instead of the real IP of the client.
    Yoann

    1. Seeing 127.0.0.1 in the logs instead of the real IP of the client is a typical issue with any kind of proxies.

      To bypass this issue, you use the PROXY protocol,
      https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt

      In practice, in LXD, you add the proxy_protocol=true when you create the proxy device. That is what is only required from LXD (LXD 3.16 or newer).

      On the receiving end (your Web server), you need to activate support for the PROXY protocol.
      Depending on your Web server, there are instructions on how to do this.
      If you use either nginx or Apache, have a look at https://www.linode.com/docs/applications/containers/beginners-guide-to-lxd-reverse-proxy/
      Note that in your case, the trusted IP address is “127.0.0.1”, to be used in your configuration on the Web server.

        • Yoann on September 11, 2019 at 17:26

        THANKS A LOT, it was the missing piece of my configuration (multiple LXD container for different versions of php/mysql/postgresql).
        I’m really HAPPY.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: