How NOT to do Patch Management for IT companies
This week saw Microsoft Patch Tuesday, the second Tuesday of each month where Microsoft release security patches for their Operating Systems and Applications.
A number of IT companies have since mentioned that they’d observed peculiar behaviour with Windows 7 PC’s since they had been updated, where after booting up, Windows 7 failed to load the Desktop.
While this type of update, one which seems to cause more problems than it fixes, is now rare – it causes gnashing of teeth from clients, who perceive no benefit from such patches and see them as a obstruction to getting on with their work, and naturally it causes pain for IT providers who have to deal with irritated clients.
Let’s turn off Windows Update!
But the way in which some IT companies choose to deal with this challenge scares me. Their thinking goes along the lines of “If Windows Updates are causing my client, and me, a problem then I’ll turn off Windows Updates”.
As the former owner of an MSP I’ve seen examples of this myself when winning a new client away from an incumbent IT provider. PC’s and Servers haven’t been patched for months, if not years. Widely known vulnerabilities aren’t secured. Operating Systems and Applications lag Service Pack’s behind currently supported minimums.
This approach to using the virtual Sledgehammer to crack a Walnut isn’t restricted to patching alone.
- That Line-of-Business Application doesn’t work. It requires write access to a folder on the C:\ drive, but I’m not sure which folder. I’ll grant Full access to the C:\ drive.
- This application needs a port opening on the Firewall. We’ve opened the required port, but it still doesn’t seem to be working. Let’s just put the machine in question in a DMZ.
- This employee needs access to this network folder. I can’t remember which Active Directory group I should add that user to to gain access, so I’ll give the employee’s user account full access to the folder directly.
In each case, instead of providing a best practice granular solution, the IT company is going for the quick fix. Sure, it gets the job done – but at what cost in the long-term?
So how to do Patch Management properly?
If you’re a Managed Service Provider, then you really, really need to be using an Remote Management and Monitoring (RMM) tool. For the purposes of this blog, I’m going to choose GFI Max – I used it in my own MSP, it’s easy to deploy, it’s cost effective and it does patch management very, very well. But you could also consider using any of the excellent RMM tools on the market.
With GFI Max, you deploy an agent to each workstation or server. Once deployed, you have full control over the patches you can deploy to that machine – and not just Microsoft patches, but patches from numerous 3rd Party companies too.
But this isn’t just an “on/off” switch for patches – it allows you to be much more granular in your approach. For instance, instead of automatically deploying patches you can be alerted to the fact they are available, and make a decision on whether to install those patches on a selective basis.
(BTW – GFI Max doesn’t just cover patching, it also highlights vulnerabilities. So you’re constantly being alerted to new weaknesses and how to resolve them).
If you decide to automatically install updates, then you can do so on a more granular level. For instance you could choose to install Critical updates, but not Moderate or Low level updates.
Naturally you can choose a schedule of when to install updates, and whether to reboot once those updates are applied.
Which leads us to the common question that most MSP’s are comfortable with some PC’s being rebooted automatically, but not others – for example, Servers or Mission Critical PC’s.
This is where the granular approach in GFI Max really pays dividends. You could choose to automatically reboot Workstations, but not servers. Or you can tell certain workstations to reboot, but mission critical workstations not to. The benefit of being granular in the approach is that you can deal with Workstations, Servers, Client Sites and even individual machines in a very specific manner – all the while being centrally maintained.
This granularity allows you to make choices over how you tackle patch management, not as a “one size fits all”, not as “Patches are on” or “Patches are off” – but in a sensible, intelligent way.
What if I’m a Break/Fix Provider?
If you are an aspiring Managed Service Provider, but most of your clients are still working with you on a Break/Fix basis, then Patch Management is the ideal way to help that client dip their toe into the Managed Services waters.
Let’s take the smallest of clients. One PC, works from home. You speak once a month or so when they need assistance. By installing GFI Max (or your choice of RMM tool) you can offer a secure Patch Management service. Cost to you – likely less than £1. Bill to client? Your choice. £5, £10/month?
Friends Don’t Let Friends Use WSUS!
Of course, the hard-core technicians amongst us will scoff at paying a RMM tool vendor to use their software. “You can do all this stuff for free!”.
You certainly can. You can leave Windows Update turned on automatically for free – but remember the start of this post? It doesn’t always work out for the best.
You can also use WSUS – Windows Software Update Services. It’s a free tool from Microsoft and a centralised way of managing patches. But there’s a phrase that many of us in the MSP market use – “Friends Don’t Let Friends Use WSUS”.
WSUS is fine for an internal IT department managing a single clients infrastructure. But it’s unwieldy, it’s time consuming and it’s noisy. Multiply the time you spend managing one WSUS installation by a dozen, or ten dozen clients – and as an MSP you’re wasting a lot of time and profit.
Conclusion
Turning off Windows Update as a way to solve your Patch Management headaches isn’t the answer.
You can use free tools such as WSUS to do Patch Management, but not effectively across multiple sites.
Or you can install an RMM tool and begin to realise that a Managed Service Provider looks to automate and centrally manage like Patch Management – knowing that their time is better spent undertaking tasks that clients perceive to add real value to the relationship.
Comments
9 thoughts on How NOT to do Patch Management for IT companies
DAVE MONTGOMERY
15TH MARCH 2012 08:58:32
Another great article Richard.
RICHARD TUBB
15TH MARCH 2012 09:04:18
Thanks Dave - appreciate the kind feedback!
CRAIG SHARP
9TH MAY 2012 20:44:26
Great article again, but as always, it's important to make sure that an MSP is not hiding behind the PAtch Management of the Support Ticket and actually visiting the clients, pressing the flesh and talking. Customer Contact is King, which is also the title of a recent Blog I wrote ) shows how I think it's important to keep a personal connection with the client while also ensuring these important patch's are maintained thorough your regular service offering.
JEREMY CURTIS
21ST SEPTEMBER 2012 12:03:10
Good article & I agree with what you say Richard and I have implemented this for some clients where we use GFI Max as our RMM. However, we find that the updates with EULAs including the all important Service Packs are not applied by GFI Max (and their support say - that is the way it is). I have also found this to be the case with at least one other RMM (Labtech). So if the most important updates are not rolled out by the RMM and you still need to use WSUS then why not use WSUS all the time? Also, GFI has a nice feature to update non Microsoft products too such as Flash, Firefox etc. But it only applies security patches not major updates. I have found that ninite does an excellent job of this. What do you think to providing patching / upgrades for non Microsoft products?
RICHARD TUBB
21ST SEPTEMBER 2012 12:20:24
Thanks for the kind words Jeremy! Have you considered using GFI Max's scripting features to roll out those problematic updates? Typically, updates have a "Silent" option that bypasses EULA agreements and the like. Could this be a solution?
JEREMY CURTIS
25TH SEPTEMBER 2012 08:10:15
Hi Richard, Yes it could be a solution if it all worked smoothly. I've written the scripts to install silently as you say. The issue with GFI is that getting a large file onto the local PC is a challenge. There is no inbuilt feature to do this so you have to download it to the local PC via FTP which is OK but does not always work and if it fails it restarts etc. The bottom line is that by the time you do this, test the scripts and deploy and monitor them the gains you make are mostly wiped out. I tried labtech for a while which was much better than GFI at scripting. It had the same problem with the EULAs but would deal with the file download internally and then reliably run the script. The problem with labtech was its monitoring was not up to scratch compared to GFI. So we've remained with GFI as it is very low management compared to other RMMs and remained with WSUS for server based environments and use GFI patch management for non-server setups which are typically newer cloud based solutions. Jeremy
DOMINIC TUTTY
20TH DECEMBER 2012 01:53:37
I stumbled across this looking for best practice in using GFI Patch Management. I have XP computers with GFI patching enabled and it is doing what it should. What I am stumped on is why the windows update area is still showing options for auto, notify ect. I would of thought that if it becomes managed these options would be greyed out? Users are still getting notifications saying the updates are off and therefore is a security problem. In any case I have forwarded issue to GFI to see what they say.
MARK PETRIE
20TH DECEMBER 2012 11:31:34
Hi Everyone, At GFI, we are constantly looking to improve our Patch Management solution by adding more functionality and support for more vendors and products. Reading feedback like this about how you use it on a day to day basis, really does help us so thank you Richard and all. We have recently updated GFI MAX RemoteManagement to use the latest version of GFI LANGuard 2012 to add support for non-security updates. At our recent customer conference series, many customers requested that we disable Windows Updates when Patch Management is enabled on a workstation and this is something we are looking to add in 2013. We would also like to give you greater control over major version upgrades and solve the EULA problem - although as this varies from vendor to vendor (in some instances the EULA acceptance overrides the silent install) so is somewhat harder to solve. There will be a number of releases in early 2013 to improve the reliability, functionality and coverage of Patch Management in GFI MAX RemoteManagement. Mark.
RICHARD TUBB
20TH DECEMBER 2012 11:35:58
Mark - thank-you for taking the time to answer Dominic's query. It's good of you!