@tig @dick_turpin can often be better packaged than some of the shit RPMs or bonkers PHP / Python deployments I've seen over the years. The technology is not a limiter of the quality, just like it never was before.

@tig @dick_turpin but having my prod servers auto-update themselves with security fixes every few days, in a totally automated way is game changing in many ways.

@intrbiz @dick_turpin I completely agree and I usually have complete trust in my upstream but it isn't going to wash with the change control board :P Also I know what you are saying it just can't be used in every deployment :) and this is kinda the centos problem, they really fucked up a lot of people's environment that are working in the redhat world with redhat standards for paths etc if the dev licence was a sensible or practical thing then they would have used it but centos was less hassle

@tig @dick_turpin companies change control boards will eventually have to come into the 21st century. I've never seen change control actually help. Far better is to build real feedback loops and a focus on quality, rather than gating things based on bullshit.

@intrbiz @dick_turpin there is that but if I am called out in the middle of the night I want to know the platform is where I left it... A classic example is the famous left-pad npm issue

@tig @dick_turpin in a world of immutable containers the left-pad problem happens in your CI build pipeline. Great thing with openSUSE MicroOS is it will automatically reboot into the previous snapshot on healthcheck fails.

The biggest issue I had was a bootloader fuckup, which was not fun, but only an issue on my Raspberry Pis and not any of my VMs.

Problems will happen, which is why approaches like canary and staggered deployments are needed.

@tig @dick_turpin but upgrade small and often is usually far less error prone, since people understand what is changing more.

@intrbiz @dick_turpin which is kinda what you do if you have to go through change control. I am not saying one side is better as I have spent hours trying to get simple stuff through CAB but what I was getting at that for those of us in the trenches we can't always dictate what we are going to use, also yes left-pad should have been picked up by CI but what about a fishy kernel update? or a change to shells or default behavior those would have been standard updates

@tig @dick_turpin if those in the trenches don't fight for other options, it won't ever change.

Staggering updates and actually using test environments should help spot kernel issues before it's too much of an issue.

Shell issues, would be part of a container build.

@intrbiz @dick_turpin up until recently those test environments could be spun up using centos vms which would have been etc... but if you are staggering upgrades the you are no longer having your rolling server updates automagically happening. I believe it can be done just don't have the time here but would love a demo :)

@intrbiz @dick_turpin the shell issue was the ubuntu bash/dash change I was thinking of :)

@tig @dick_turpin sure, but it would only cause an issue inside the container and thus be caught in your CI pipeline.

@intrbiz @dick_turpin I am in awe of your perfect CI pipeline that catches everything. :) Any jobs going? :)

Some of us are having to take care of somewhat older software :)

@tig @dick_turpin my pipeline is far from perfect, but build container and then run it, catches a lot of issues.

I saw a talk the other year of a team who put a CI/CD pipeline around an ancient VMS app which they ran in openVMS. It was pretty crazy but showed what can be done. Getting that done let them actually develope and evolve the ancient code, rather than just be scared of touching it.

@intrbiz @tig @dick_turpin openSUSE Aeon is working great for me. Updates do come in large amounts sometimes like today's 1014 packages.

My ArchlinuxARM server running its systemd-nspawn containers for services has now clocked-up over two years of hassle-free service.

Coming from staged-release distributions and VMs to rolling distributions and containers has been a positive experience for me.

@intrbiz

RH used to do that; they had "Auto-Update" until the -y flag started killing systems and they realised that maybe it wasn't such a good idea.

@tig

@dick_turpin @intrbiz @tig I think the rise of immutable OSes and flatpak can enable the use of leading-edge (not bleeding-edge) distribution in the enterprise.

This is brought about by the separation of OS and Applications. As every seat runs the same, immutable, OS image they can be well tested and QC'd.

flatpaks allow you to curate older application versions on any version of the underlying OS so permitting updates to suit business decisions. The only imposition is that the underlying runtimes need to maintain backwards compatibility in their updates (at least to some sensible timeframe) before an application update is mandated.

I think there is a case for an underlying OS to permit upgrading of the hardware it runs on independently of the applications running on it. Being able to run a current kernel and desktop environment give you the flexibility to do that and an immutable OS is central to it.

Classic Enterprise Linux distributions will, I believe, be replaced by these new paradigms and they will make support efforts easier.

@neil @dick_turpin @intrbiz I think I am thinking about this more from the server side which is where EL is more prevalent rather than desktop use where perhaps you do want/need leading edge versions for hardware support etc. Most corporate desktop linux deployments seem to be Ubuntu based not EL. TBH I am not that keen on flatpak style installs as it takes up extra space and breaks the whole integrated package management and ease of updates sanely.