I love it. It’s like a cross between virtual box and docker. You get a container that spins up fast but behaves more like a vm. You can install services, you get an ip address, etc.
There are a few differences because lxc runs along side the reast of host system rather than the daemonized container service that Docker does.
From the host you can access kernel related controls within the target system. You can see the processes running, perform tuning on them, etc while also having the same kernel level control inside the target. This also means you can have better control over security bu setting group policies, apparmor profiles and system aware firewall rules because you aren’t running your target in a black box.
Their purposes are very different. If you are running a single process for a single purpose you use Docker. When you want yo run a system for a specific service you run lxc. Can you do the opposite within each type? Yep. But that’s not what they are designed for. Can you run a full blown email service with imap and pop, a web server for a webmail client and antivirus services inside a docker container…of course. But all the tuning and configuration is done at the container level which means that we assume all installs and replication must be the same. In lxc i can install the same system but if we want to tweak max memory usage or niceness of a given service you can do that globally or target a specific container while on docker youd have to go to each container to do that work.
I used to use LXC maybe 5 years ago but I’ve since replaced everything with docker/compose. The main difference between LXC and Docker is that LXC is meant to be more like a Virtual Machine than a container. LXC containers run their own instance of systemd and can run multiple processes easily. Docker is meant to run a single process although people sometimes do hacks with supervisord or s6 overlay to run multiple processes.
At the time LXC didn’t really have a concept of images like Docker, it was just base images like Ubuntu 18.04 or Debian 9 and you’d shell in the container and install your stuff.
LXD is a tool built on top of LXC, confusingly enough the LXD client is called lxc… It’s higher level and might have the ability to use images, not sure, I never felt the need to learn it.
I’ve always used lxc and only recently tried docker.
I really cant wrap my head around all the crazy shit docker alters on your network settings like rewriting a bunch of firewall rules without telling you
Not sure if i was doing something wrong but that was my experience with docker
There are scripts for making a jail around single apps but yeah I typically don’t use them that way. Lxc I very often install an app I want to test out and toss once I want to dedicate compile time to it.
Yeah, I’d want a jail dockerfile system too, I just usually do them manually. Still, a way to run dockerfiles to build jails would be epic if you could make it work.
I used gentoo for a decade, I just can’t afford the downtime if my workstation goes down, so it’s debian with lxc workspaces for a while, but gentoo actually runs well under lxc.
Mostly every app expects its own distro, either debian or centos, few actually are agnostic, so getting them to run on gentoo was always more of a challenge than on raw debian/Ubuntu.
I’m actually the opposite. Run gentoo as my host and toss up a debian lxc if needed. Worst case scenario im running just the kernel and everything else from a container (actually how i typically run when rebuilding a system from start).
I’ve never run into a situation where an app “couldn’t” run in Gentoo. It’s just that I’ve had cases where an app is build for a 8 year old LTS of debian with such old dependencies it wouldn’t be worth my time building them all when i can just pull up a container with that super old build. The nice thing is that all the vulnerabilities that old Debian had is now in a container and less of a target.
I swear i must be lucky cuz i do often hear of gentpo fatigue but I’ve been running it since the project started and never had issues outside the things they legitimately broke.
This is the same issue I have. I much prefer to manage my own firewall policies and having to make those play nicely with Docker was a huge pain in the ass in most cases. I’d rather use snaps than Docker for stuff that requires a daemon and regular updates, and Snaps have plenty of issues as well
Same, cause I can’t pass through my video card or my coral with the motherboard I have. So frigate runs in an lxc. When I move jellyfin over to that box, it will be an lxc too for the same reason
I don’t have anything to compare it to, but I wanted local object detection and it’s kind of the only option. Configuration had a learning curve. There’s somewhat limited playback options, but they cover the basics. It doesn’t transcode playback so on mobile, the 4k playback buffers. I could make a smaller stream to record but I wish it was dynamic on playback. The object detection works well enough, but there are a lot of false positives. They’re using the default models for now, but are working on frigate.plus models. The pricing structure for that is wack though. I bitch about photoprism’s subscription features, and frigate plus also wants $5/month for the rest of your life or the model stops working. I’d be happy to pay a one-time fee to unlock but I refuse to pay subscription for a product that is self-hosted.
Well hold on, LXD is a subset of LXC, that is LXC is at the heart of LXD but LXD brings with it a RESTful API written in Go to control LXC. Canonical doesn’t own LXC, IBM wrote LXC.
LXD and LXC became really intertwined once Docker and CoreOS Containers dropped LXC and went their own way. Basically leaving LXD as the sole claim to fame for LXC. What Incus is doing is basically providing a RESTful API on top of LXC, pretty much the exact same way LXD does exactly that as well.
In fact given Canonical’s Google-lite approach to dropping projects like they’re hot and the maintainers that are heading to Incus, Incus is less fragmentation and more migration.
the initial set of maintainers for Incus will include Christian Brauner, Serge Hallyn, Stéphane Graber and Tycho Andersen
I mean that pretty much is the bulk of people that know how this software works inside and out. I just don’t see Canonical (inventor of the MIR Display Server) devoting the resources to keeping up with LXD when a good bit of mind-share just moved over to Incus.
This is just more of the same that’s helping Canonical become less leader in the deb based distros and more just a player. Add in their wonderful call to double down on snaps and you’ve got a 1-2 combo they’ve dealt to themselves. Canonical just did the MySQL vs MariaDB to LXD. Like MySQL is still useful, but MariaDB left MySQL in terms of features and functions in the dust long ago. You use MySQL today because of name recognition. You use MariaDB when you actually need a database with actual features.
And the likelihood the exact same thing happens with LXD just jumped an order of magnitude by seeing who just signed on to Incus.
EDIT: And Incus has replaced LXD on the linuxcontainers.org page already. Ooof. I wouldn’t want to be Canonical at the moment.
I use it all the time, similar to how I use jails on my FreeBSD systems. Basically when I need to compartmentalize an app I launch a new instance of Alpine and install the app.
As an example I have a container that has my VPN software and a browser that I know is a clean room.
I run Gentoo as my main distro and sometimes a package is distributed only as a deb with very specific version dependencies I can’t build. So I spin up a base Debian container and install the app. If it’s X11 I can launch it into my current session and if it’s console then I can always mount my home directory as a network share.
Yeah, on bsd jails are basically shared because of zfs, I should use more alpine, but more complex applications often need something closer to debian, and my alpine fu isn’t very good yet.
We really, REALLY need a dockerfile for lxc so you call lxc-build and it pulls and compiles/configures everything for you automatically.
Sadly my daily driver is getting pretty old and slow so i typically dont put big distros on lxc. Maybe NixOS can he configured to be super slim. New weekend project.
Does anyone actually use LXD? I never could figure out the deal with this.
I love it. It’s like a cross between virtual box and docker. You get a container that spins up fast but behaves more like a vm. You can install services, you get an ip address, etc.
But you can do all that in docker? Heck I have full GNOME installs with novnc in docker.
There are a few differences because lxc runs along side the reast of host system rather than the daemonized container service that Docker does.
From the host you can access kernel related controls within the target system. You can see the processes running, perform tuning on them, etc while also having the same kernel level control inside the target. This also means you can have better control over security bu setting group policies, apparmor profiles and system aware firewall rules because you aren’t running your target in a black box.
Their purposes are very different. If you are running a single process for a single purpose you use Docker. When you want yo run a system for a specific service you run lxc. Can you do the opposite within each type? Yep. But that’s not what they are designed for. Can you run a full blown email service with imap and pop, a web server for a webmail client and antivirus services inside a docker container…of course. But all the tuning and configuration is done at the container level which means that we assume all installs and replication must be the same. In lxc i can install the same system but if we want to tweak max memory usage or niceness of a given service you can do that globally or target a specific container while on docker youd have to go to each container to do that work.
I used to use LXC maybe 5 years ago but I’ve since replaced everything with docker/compose. The main difference between LXC and Docker is that LXC is meant to be more like a Virtual Machine than a container. LXC containers run their own instance of systemd and can run multiple processes easily. Docker is meant to run a single process although people sometimes do hacks with supervisord or s6 overlay to run multiple processes.
At the time LXC didn’t really have a concept of images like Docker, it was just base images like Ubuntu 18.04 or Debian 9 and you’d shell in the container and install your stuff.
LXD is a tool built on top of LXC, confusingly enough the LXD client is called
lxc… It’s higher level and might have the ability to use images, not sure, I never felt the need to learn it.I’ve always used lxc and only recently tried docker.
I really cant wrap my head around all the crazy shit docker alters on your network settings like rewriting a bunch of firewall rules without telling you
Not sure if i was doing something wrong but that was my experience with docker
Docker is spaghetti-ware, they try to control everything, which ironically makes me Isolate my dockers in a vm.
Ok, i’m glad my solution to the problem (run docker in an lxc container) isn’t as harebrained as i thought
Other people are doing the same
Haven’t done that, but honestly I’m thinking that’s my next workflow.
That is kind of the expected setup. Either a vm or a dedicated system. You let docker do its thing and it should work.
I run lxc because i want contained systems I control. That just means I have to do the work too.
Same, I love lxc like I love jails, you craft beautiful systems that are isolated and clean.
I wouldn’t make a disposable jail, but I make disposable lxcs, lxcs are like temporary distros for me.
There are scripts for making a jail around single apps but yeah I typically don’t use them that way. Lxc I very often install an app I want to test out and toss once I want to dedicate compile time to it.
Yeah, I’d want a jail dockerfile system too, I just usually do them manually. Still, a way to run dockerfiles to build jails would be epic if you could make it work.
I used gentoo for a decade, I just can’t afford the downtime if my workstation goes down, so it’s debian with lxc workspaces for a while, but gentoo actually runs well under lxc.
Mostly every app expects its own distro, either debian or centos, few actually are agnostic, so getting them to run on gentoo was always more of a challenge than on raw debian/Ubuntu.
I’m actually the opposite. Run gentoo as my host and toss up a debian lxc if needed. Worst case scenario im running just the kernel and everything else from a container (actually how i typically run when rebuilding a system from start).
I’ve never run into a situation where an app “couldn’t” run in Gentoo. It’s just that I’ve had cases where an app is build for a 8 year old LTS of debian with such old dependencies it wouldn’t be worth my time building them all when i can just pull up a container with that super old build. The nice thing is that all the vulnerabilities that old Debian had is now in a container and less of a target.
I swear i must be lucky cuz i do often hear of gentpo fatigue but I’ve been running it since the project started and never had issues outside the things they legitimately broke.
This is the same issue I have. I much prefer to manage my own firewall policies and having to make those play nicely with Docker was a huge pain in the ass in most cases. I’d rather use snaps than Docker for stuff that requires a daemon and regular updates, and Snaps have plenty of issues as well
Yeah I use it through proxmox but it doesn’t make much difference to me. It’s practically a lower-overhead VM as far as I’m concerned
Same, cause I can’t pass through my video card or my coral with the motherboard I have. So frigate runs in an lxc. When I move jellyfin over to that box, it will be an lxc too for the same reason
Frigate the NVR? What’re your thoughts on it? Been looking at NVRs for a while and hadn’t seen that one
I don’t have anything to compare it to, but I wanted local object detection and it’s kind of the only option. Configuration had a learning curve. There’s somewhat limited playback options, but they cover the basics. It doesn’t transcode playback so on mobile, the 4k playback buffers. I could make a smaller stream to record but I wish it was dynamic on playback. The object detection works well enough, but there are a lot of false positives. They’re using the default models for now, but are working on frigate.plus models. The pricing structure for that is wack though. I bitch about photoprism’s subscription features, and frigate plus also wants $5/month for the rest of your life or the model stops working. I’d be happy to pay a one-time fee to unlock but I refuse to pay subscription for a product that is self-hosted.
I don’t know anyone using it personally but I have seen lots of folks here and Reddit that use LXC through Proxmox, I had the same thought though.
Well hold on, LXD is a subset of LXC, that is LXC is at the heart of LXD but LXD brings with it a RESTful API written in Go to control LXC. Canonical doesn’t own LXC, IBM wrote LXC.
LXD and LXC became really intertwined once Docker and CoreOS Containers dropped LXC and went their own way. Basically leaving LXD as the sole claim to fame for LXC. What Incus is doing is basically providing a RESTful API on top of LXC, pretty much the exact same way LXD does exactly that as well.
In fact given Canonical’s Google-lite approach to dropping projects like they’re hot and the maintainers that are heading to Incus, Incus is less fragmentation and more migration.
I mean that pretty much is the bulk of people that know how this software works inside and out. I just don’t see Canonical (inventor of the MIR Display Server) devoting the resources to keeping up with LXD when a good bit of mind-share just moved over to Incus.
This is just more of the same that’s helping Canonical become less leader in the deb based distros and more just a player. Add in their wonderful call to double down on snaps and you’ve got a 1-2 combo they’ve dealt to themselves. Canonical just did the MySQL vs MariaDB to LXD. Like MySQL is still useful, but MariaDB left MySQL in terms of features and functions in the dust long ago. You use MySQL today because of name recognition. You use MariaDB when you actually need a database with actual features.
And the likelihood the exact same thing happens with LXD just jumped an order of magnitude by seeing who just signed on to Incus.
EDIT: And Incus has replaced LXD on the linuxcontainers.org page already. Ooof. I wouldn’t want to be Canonical at the moment.
Thank you for finally explaining lxd.
I actually might use the python api, I didn’t see a point for it otherwise.
Exactly that, I have a few lxd containers on my proxmox host along with traditional vms, also have docker running inside a lxd vs a vm
I use it all the time, similar to how I use jails on my FreeBSD systems. Basically when I need to compartmentalize an app I launch a new instance of Alpine and install the app.
As an example I have a container that has my VPN software and a browser that I know is a clean room.
I run Gentoo as my main distro and sometimes a package is distributed only as a deb with very specific version dependencies I can’t build. So I spin up a base Debian container and install the app. If it’s X11 I can launch it into my current session and if it’s console then I can always mount my home directory as a network share.
Use lxc same way, works well, used lxd that way once or twice but with a decent lxc script it worked that way.
Agreed on jails, lxc finally brought that functionality to linux.
With the addition of Alpine Linux containers are now barely bigger than the application itself.
Yeah, on bsd jails are basically shared because of zfs, I should use more alpine, but more complex applications often need something closer to debian, and my alpine fu isn’t very good yet.
We really, REALLY need a dockerfile for lxc so you call lxc-build and it pulls and compiles/configures everything for you automatically.
Thats the dream.
I want to look into NixOS. They basically have a one file config and install for your entire system. Wonder how well it works in lxc.
I wouldn’t want Nixos as my base, but yeah, it sounds like the ultimate lxc base.
Sadly my daily driver is getting pretty old and slow so i typically dont put big distros on lxc. Maybe NixOS can he configured to be super slim. New weekend project.