Surely you must be joking …

It takes all types of people to run a corporation, but only certain individuals can tap into the ActiveEngine and put things into overdrive. This post is not about a great new design pattern that solved a problem or fit neatly into a methodology. More time for that in another post.

This is about the onerous task of updating 200 desktop computers with a technology that barely passes for worst practices from 1997. Yep, ten years of garbage code and retarded customizations, compounded by bad design and awful practices, 10 servers to update and 200 desktops to uninstall the old version and update with next version, the operative word being next.

For those not of the techie world, this can be a technician’s nightmare. This complexity is increased when the software manufacturer pretends that they themselves have wisdom greater than the rest of software industry,shun standard practice, and create some unique yet “better” software update process. Processes that can not be controlled, that when followed would result in 200 computers updating automatically at once, and sentencing a corporate network to a speedy death.

Can you see the challenge? The task, create an update process that circumvents the software manufacturers, is automated with measurable outcomes, saves time, and executes successfully. The risk, if you fail the company can’t invoice or collect the next month end close. Oh, and by the way, the sales force uses a pseudo workflow engine based on this mess as well. Better not disappoint that bunch of patient delicate artisans.

A team of heroes was assembled, people who had the strength of will to overcome the repeated failures and obstacles that such a nightmare offered. A team that turned on the ActiveEngine and fought like hell. For the most part, automation of software processes could be accomplished easily enough, but for a software push to operate successfully presupposes that all target desktops are in a state to be updated, and this is where the nightmare truly began. Picture this:

• 43 desktop computers could not be connected to remotely, meaning that files could not be copied from a central source using automation. So updates had to be processed by a technician logging directly into the machine.
• 15 desktop computers continued to execute update processes that prevent the crucial update from completing successfully.
• 27 computes had corrupted Windows Registries, meaning that network administrator ids could not install new or remove old software, period. This could be repaired but the duration of this task took 10 -15 minutes. If the repair process failed the computer could be rendered useless.

This last issue was the deal breaker, as it indicated that a computer was no longer under control. Scary thought. You see, you wouldn’t find out that the Windows registry was corrupted until 2/3 of the install was completed. The figure above was ascertained after the entire set of 200 computers were updated. The prospect of failure heading into this mess was down right terrifying as what was not known could not be known.

Immediately diagnostics were performed on the registry. For non technical folks, this means that files containing more than 120,000 lines of code and security settings had to be examined. On a lucky break the starting point of the issue was ascertained, but this starting point was narrowed down to c. 20,000 lines. More comparisons to functioning registries were attempted, but the problem remained that comparisons were limited to eyeballing lines of registry keys. The support folks groaned, but offered their standard answer of “we’ll have to compare this manually”.

Unacceptable. That’s acquiescing, not thinking.

The heroes researched and found SubinACL, a tool that creates text file of all registry keys and ACL’s for those keys. The problem still existed that there was no way to compare text files from machine to machine. Not until the ActiveEngine kicked in and agile minds woke up, and a process to index the files – assign the same value for the same registry entry across all files – was created. Next, the files were imported into Access where the registry keys could be queried based on their ACL characteristics, eliminating any need for reading lines of registry dumps. 683 culprit lines were identified, 13 of these were unique. A text file with the new ACL’s was created and SubInACL was executed with this file to apply to the security settings for every machine. Yes, one day’s worth of programming eliminated weeks of work. The beauty of the brute force method was that if the settings were already correct, SubInACL would not apply the changes.

The code for this process will be discussed in a later post. It was a quick and dirty, actually really dirty, C# console app. Extensible enough to make quick changes and now subject to a re-write in hopes to create a flexible text file parser that will accept user defined fields that can be applied to the parsing process on the fly.