LXD supports proxy devices, which is a way to proxy connections between the host and containers. This includes TCP, UDP and Unix socket connections. For example, when someone connects to your host on port 80 (http), then this connection can be proxied to a container using a proxy device. In that way, you can isolate your Web server into a LXD container. By using a TCP proxy device, you do not need to use
Earlier I wrote that you can make a connection in any direction. For example, you can expose the host’s Unix socket for X11 into the container so that the container can run X11 applications and have them appear on the host’s X11 server. Or, in the other way round, you can make available LXD’s Unix socket at the host to a container so that you can manage LXD from inside a container.
Note that LXD 3.0.x only supports TCP to TCP proxy devices. Support for UDP and Unix sockets was added in later versions.
Launching a container and setting up a Web server
Let’s launch a container, install a Web server, and, then expose the Web server to the local network (or the Internet, if you are using a VPS/Internet server).
First, launch the container.
$ lxc launch ubuntu:18.04 mycontainer Creating mycontainer Starting mycontainer
We get a shell into the container, update the package list and install
nginx. Finally, verify that
nginx is running.
ubuntu@mycontainer:~$ sudo apt update ubuntu@mycontainer:~$ sudo apt install -y nginx ubuntu@mycontainer:~$ curl http://localhost ... Welcome to nginx!
Exposing the Web server of a container to the Internet
We logout to the host and verify that there is no Web server already running on port 80. If port 80 is not available on your host, change it to something else, like 8000. Finally, we create the TCP to TCP LXD Proxy Device.
ubuntu@mycontainer:~$ logout $ lxc config device add mycontainer myport80 proxy listen=tcp:0.0.0.0:80 connect=tcp:127.0.0.1:80 Device myport80 added to mycontainer
The command that creates the proxy device is made of the following components.
lxc config device add, we configure to have a device added,
mycontainer, to the container mycontainer,
myport80, with name myport80,
proxy, a proxy device, we are adding a LXD Proxy Device.
listen=tcp:0.0.0.0:80, we listen (on the host by default) on all network interfaces on TCP port 80.
connect=tcp:127.0.0.1:80, we connect (to the container by default) to the existing TCP port 80 on localhost, which is our
Note that previously you would specify hostnames when you were creating LXD Proxy Devices. This is no longer supported (has security implications), therefore you get an error if you specify a hostname such as
localhost. This post was primarily written because the top Google result on proxy devices is an old Reddit read-only post that suggests to use
Let’s test that the Web server in the container is accessible on the host. We can use both
127.0.0.1) on the host to access the website of the container. We can also use the public IP address of the host (in this case, the LAN IP address) to access the container.
$ curl http://localhost ... Welcome to nginx! ... $ curl http://192.168.1.100 ... Welcome to nginx! ...
Other features of the proxy devices
By default, a proxy device exposes an existing service in the container to the host. If we need to expose an existing service on the host to a container, we would add the parameter
bind=container to the proxy device command.
You can expose a single webserver to a port on the host. But how do you expose many web servers in containers to the host? You can use a reverse proxy that goes in front of the containers. To retain the remote IP address of the clients visiting the Web servers, you can add the
proxy_protocol=true to enable support for the PROXY protocol. Note that you also need to enable the PROXY protocol on the reverse proxy.
Hi, thanks for your post about lxc/d.
Proxy device seems great, but in the use case you describe there is a problem, because in the access log of ngix container every requests seems to came from 127.0.0.1 instead of the real IP of the client.
Seeing 127.0.0.1 in the logs instead of the real IP of the client is a typical issue with any kind of proxies.
To bypass this issue, you use the PROXY protocol,
In practice, in LXD, you add the
proxy_protocol=truewhen you create the proxy device. That is what is only required from LXD (LXD 3.16 or newer).
On the receiving end (your Web server), you need to activate support for the PROXY protocol.
Depending on your Web server, there are instructions on how to do this.
If you use either nginx or Apache, have a look at https://www.linode.com/docs/applications/containers/beginners-guide-to-lxd-reverse-proxy/
Note that in your case, the trusted IP address is “127.0.0.1”, to be used in your configuration on the Web server.
THANKS A LOT, it was the missing piece of my configuration (multiple LXD container for different versions of php/mysql/postgresql).
I’m really HAPPY.
compared to use traditional iptables NAT, what’s the benefits of proxy?
If you use
iptables, you would need to
iptablescommand which is not trivial
ufw. When you apply firewall rules, they may clear your own
iptablesrule needs updating.
The LXD proxy does not have these issues. You do not specify the IP address of the container because LXD already knows it. When you start the container, the LXD proxy is automatically set up.
The benefit of
iptablesis that of performance.
iptablescreates a rule for the Linux kernel to take care off, while the LXD proxy is a process that provides the service. You would consider such benefit if you are running a very fast network. In other words, if you where to consider the issue of performance, you would have already created a benchmark to compare the two and found the sweet spot of traffic that the LXD proxy can sufficiently serve. If you actually do this, please report back! 🙂
In cluster mode (lxd 3 or 4) is possible to expose the container port to all the cluster nodes. So I do not need to know in which host the container is running. Example: container expose port 22 as port 2222, and I have 2 cluster nodes node1 and node2, I can connect either with ssh node1 -p 2222 or ssh node2 -p 2222 to the same container. Thanks
In a cluster, a container is running on only one of the hosts. Therefore, it is not possible to expose the container port to the other hosts of the cluster.
What you can do though, is setup a reverse proxy on the rest of the hosts of the cluster, and have that reverse proxy direct the traffic to the container in question.
I am using debian as my distro of choice which does not yet support lxd natively. Do you know if similar functionality exists in lxc containers?
I suppose you mean lxc as in LXC in https://blog.simos.info/comparison-between-lxc-and-lxd/
LXC does not have proxy devices, which means you have to use
See the package
iptables-persistentfor persistence of your rules.
You can install the snap package of LXD in Debian. According to https://snapcraft.io/lxd (see Stats section), LXD is the most popular distribution after Ubuntu.
Yes, I do mean LXC. I was trying to setup a legacy printer in a debian jessie LXC container, but am having trouble and thought maybe a proxy would help. I also tried an LXD container, but I am unable to get cups to install. Ah well, I’ll keep trying.
I assume that this legacy printer only comes with some obscure .deb package that can be installed in some very old version of Ubuntu?
Here is what I suggest:
Thank you so much for all of your articles. I really love it and helped me a lot.
Using this tutorial I was able to implement this to connect to my multiple LXD containers using SSH. I can connect from my local computer to those LXD containers. So this is perfect
But using this tutorial, if I setup multiple containers to access the nginx from the internet, how can I make this work if I have 5 LXD’s running the same nginx with port 80?
I following this tutorial and this is working just fine: https://www.digitalocean.com/community/tutorials/how-to-host-multiple-web-sites-with-nginx-and-haproxy-using-lxd-on-ubuntu-16-04
This domain: http://lxd1.causingdesigns.net/
That is currently connected to one of my LXC using hafproxy.
How can I connect to nginx from the internet using proxy device?
When you need to use SSH to connect to each individual LXD containers, you configure to use separate ports for SSH, such as 22122, 22222, 22322, 22422 and so on. This is because there is no easy way for your server to know from before which server you intended to connect to. There are some workarounds with bastion hosts, but it’s simpler with separate ports. I suppose you use separate ports as well?
In the case of the HTTP/HTTPS protocol, the protocol allows to peek inside the packets and figure out which website was intended to connect to. The software that does this task, is the reverse proxy, such as HAProxy and even nginx in reverse-proxy mode.
My DigitalOcean tutorial shows how to configure HAProxy to perform this reverse-proxy task. That is, you configure the host to direct any connections for ports 80 and 443 to the reverse-proxy container, then inside the reverse-proxy container (HAProxy, nginx or other) you configure to identify which website is supposed to connect to, and redirect to the appropriate container.
Perhaps nginx as a reverse-proxy is easier to use. Have a look at this tutorial, https://www.linode.com/docs/applications/containers/beginners-guide-to-lxd-reverse-proxy/
I need to warn everyone. Never forward port 25 or at least take extra precautions. If you do your mailserver will be an open relay because messages from 127.0.0.1 are trusted.
You are the man. Now I have three containers running happily. Thanks.