About Software Updates in Big Sur



Reading Time: 7 minutes

If you work with Apple in some capacity, you know that they’re not very likely to admit mistakes. I’m not aware of Apple publishing postmortems after outages or providing details about known issues. So it’s up to the developer and admin communities at large to help each other learn about outages and potential causes. With the release of macOS 11.1 this week I’ve been debugging a new issue with the Mac software update process, one which will affect most enterprise users. I decided I should write about it and let everyone know what the issue is, workarounds, and how to avoid it. I’ve also been paying close attention to some recent changes to the software update mechanism, so I’ll try to mention them below as well.

A macOS 11 bug prevents upgrades to the next version

After macOS 11.1 was released earlier this week, many users started reporting that they were not able to see the software update. Others reported that they saw it, but were not able to download it. I personally experienced this issue when 11.1 was in beta and filed a case with Apple about it, but then moved on to other problems. When the final release came out this week, there were widespread reports of it on the MacAdmins Slack. Reading through system logs, I as well as other admins were able to find what appears to be the root cause. Under certain conditions, macOS 11.0.1 and macOS 11.1 hosts are requesting the update server send the 11.0.1 update, instead of requesting the next available one. The server rejects this update as it’s already either installed or older. This somehow corrupts the state of the software update process, and the update is no longer visible as an option in System Preferences.

Workarounds

Can Apple fix this bug without manual intervention?

Apple is well aware this is a problem now, so I am confident the issue will be addressed in 11.2. Unfortunately, it is a client-side issue affecting both 11.0.1 and 11.1 clients. So there’s not much Apple can do to provide a fix. One potential solution I see is for Apple to detect the wrong request and instead of rejecting the download, offer the right file archive instead. But this is a complex system and it’s unclear if the server-side changes alone are enough.

Something else Apple could try is to side load a patch through another software update. There’s background configuration and malware removal tools that are likely capable of fixing the issue on the system. But it’s an ugly hack and one I’d personally stay clear of even if the option was available.

In my opinion, the most likely outcome is that the bug will be fixed in 11.2, but clients that have already upgraded to Big Sur (or any M1 macs you might’ve bought) will have to work around the problem themselves. If we’re lucky Apple will publish a support article and that will be that.

What else you need to know

Big Sur has changed the software update mechanism entirely, especially on Apple Silicon macs. It’s a long time coming and Apple spoke about some of this at WWDC in the MDM and IT sessions, so it shouldn’t be entirely surprising. But a lot was left unsaid for us to discover on our own.

I work with the MDM protocol a lot day-to-day and have been testing software updates for a while. I was even optimistic about what it would look like in Big Sur back in June. Apple had promised it’s an entirely new implementation, closer to what is available on iOS and that everything would work better than before. But we’re not off to a good start, and all the concerns I had for several years now are back. The design of OS updates in MDM is brittle, requiring multiple remote procedure calls to accomplish something that was previously done by a few lines of shell scripting. And that would be bad on its own, but the bad design is coupled with a buggy implementation; there are many known issues, besides the one I described above. Unless something drastically changes, we’re likely to see many months, if not years of software update bugs that are entirely out of our control.

Apple was never particularly great at building systems for the enterprise. But, until recently, the Unix components were available for software developers and system administrators to work with. The end result ended up being that if you made an investment within your organization, macOS was a great experience for end-users. It took a lot of work, but it was all achievable. Today, Apple is no better at developing systems that are required in the enterprise. But the ball is entirely in their court. Apple still makes great tools for consumers, and there will be a demand from employees to provide them with Apple gear. But, if Apple can’t start acting on our collective feedback, the experience of using a Mac in the workplace will quickly become unbearable to most.