In a large corporate environment, all changes must be carefully tested and deployed. In this video, you’ll learn about the change control process and how organizations can manage constant technology changes.
<< Previous Video: Documentation Best Practices Next: Disaster Recovery >>
Change management, or what some people call change control, is literally controlling how much change occurs in your organization. In any size company, you’ll have changes that need to occur all the time. They may be related to application upgrades, a security patch, an update to a switch configuration, or a new routing interface needs to be installed. Those changes need to be managed so there’s no downtime, nobody’s confused, and everybody understands exactly when these changes are going to occur.
Change management is a process that takes you all the way from the beginning of when a change needs to be made until the change is finally made and tested. You’ll start with the process of understanding what change needs to occur. From there you’ll perform an analysis to understand the risks involved in making this change, and then you’ll create a plan for implementing the change in the organization.
Of course, those changes will need to be approved and then presented to a change control committee or board for their approval as well. There needs to be a plan B or back out plan if that particular change does not work as expected. And once that change is in place and everything is working as you expect, then you’ll need to document exactly what those changes were.
Let’s start with the beginning, where we need to make a change to something in the infrastructure. This might be an operating system update or a series of patches, it might be a change to a switch or router configuration. But we need to understand exactly what that change entails. This may affect a single router or switch at a site, or you may be making changes to every single workstation that’s on the network.
One single change that seems to be minor can have a dramatic effect on the organization. If you make a change to a switch, it could affect an entire application or series of applications from working properly. Maybe you’re affecting the type of connectivity you have out to the internet, you might limit what people are able to do from remote locations, or your external customers may have problems connecting back to your site, just because of that single change you happen to make on a router or a switch.
You also need to understand how long this particular change will take. On a simple router upgrade, it might be a matter of minutes. For something that’s being pushed out to every workstation at the site, it may be a number of hours. So you should understand exactly what the end users are going to experience when they try to use the network or the resources during this time-frame.
From the perspective of the organization being able to continue with its primary goals and objectives, we need to understand the risk of making this change. Is this change relatively low risk, or could there be significant risk associated with making this type of change? You need to understand exactly what the potential is for making this change.
Let’s say that you’re making a bug fix to an application or an operating system. Once you implement the change, you may find that the bug fix doesn’t actually fix the problem that you’re having. Or even worse, the fix actually breaks something else, which is even more of a problem. Or once you install the update, the operating system is suddenly failing on that device or the data itself becomes corrupted. All of these are significant issues that can be a very minor problem to a very significant problem for your organization.
We also have to think about the risk involved in not proceeding with this particular change. This is especially important when dealing with security patches because not implementing the change could open up your organization to an attack. It may be that applications are required to be updated during a certain time-frame and not making those updates may cause those applications to stop working. And it may be that third party services are expecting this change to be in place, and if they aren’t, you may have unexpected downtime to those services.
Before making the change you need to document every step it takes to perform this change from the beginning to the end. This is a technical document that will be presented to technical people so they know exactly what’s going to happen with a switch, a router, an operating system, or any other component that you’ll be making this change to.
By documenting this process, other people can have some insight into what this change really entails, and they may be able to find conflicts between what you’re planning to do and what other things may be happening with that operating system. Making one change to the operating system may change a feature of an application that you weren’t previously aware of, but by providing this plan to the application team, they can see what might be affected by the change you’ve planned.
Given what you know about the change and what it could affect, you can then decide how you would schedule this. Would this be something that would occur during the night or early morning hours? Would this have to occur on the weekend? Or do you need to wait for a very large period of downtime, such as a holiday, to be able to implement this change?
We usually have very good reasons for making these changes, and those reasons are usually to make sure that our customers, or the end users, are able to continue using their applications and work normally without any downtime. So of course, before we make any of these changes, we want to be sure that they are aware of the change, what the impact might be, and what the time-frames are. So we want to be sure to get their approval before continuing with the change.
The end users will have a better idea of how this time-frame is going to affect them. There may be certain times during a quarter or during a month where it’s very important that there is no downtime. So they may adjust these schedules based upon what your requirements are for making this change. Hopefully, this end user acceptance is going to be a formality. Ideally, you’ve been working with the end users through this entire process to document the change, the time-frames, and to make sure that they understand exactly what’s going to happen when this change is implemented.
Once you have the sign up from the end users, it’s time to present the change to the change control committee. This is a group that generally meets every week to examine all of the changes that are being proposed, and to be able to identify how important certain changes might be to the organization. This also gives everyone in the room a chance to see if there may be conflicts between different changes that may be occurring, and there may be a need to stretch those changes out over a much longer time-frame.
This also might be based on the priorities of the company. If we know that a busy season is coming up, certain changes may be pushed through earlier rather than later. It’s the job of the change control committee to properly schedule all of these changes and know exactly what’s going on. The change control committee is that final step before you’re given approval and can proceed with the actual change.
Even with all of the planning and testing that you do prior to making this change, there always seems to be occasions where something doesn’t go as planned. And in those scenarios you need to have some way to revert back to the previous configuration so that everything was exactly the way it was before you started making the change.
This philosophy of planning for the worst possible scenario but hoping for the best is one that will give you a way to revert back to that previous configuration. However, it’s not always as simple as it sounds. There may be certain changes that are very involved, and reverting back to a previous configuration is just as involved, if not more difficult, in some scenarios. You should always have a backup plan and you should always have backups. In fact, it may be that the backup process is the first step that you created when you wrote out the plan for making this change.
After the change has been applied, you need to make sure that everyone knows what has changed in the organization. There may be documentation at the help desk that you can update showing the new version numbers or the new server names of what was changed. It’s also useful if you can go back in time to see what the changes have been over time. A year from now you may not remember what those changes were, and having some type of documentation can help you understand how you got to where you are today from where you were a year ago.
And with his documentation of what these changes are, we can start looking at what the differences are in performance. Perhaps the application has faster response time, or that uptime happens to be better, and the way you’re able to track all of this information is by referencing what changed during this change control process.