Friday, August 10, 2012

incident prevention and incident response

I had wanted to write about this for quite some time, but had never gotten the chance to type it out. In IT systems support/management, part of the maintenance phase is incident response and prevention. It's always in that sequence. The prevention bit normally comes as a mitigating measure after the incident has happened.

There was a JC class outing I had in 1999 where a group of us went to The Heeren, and wanted to take a group photo at the atrium area in front of HMV (it was HMV then). As it was quite late at night, 9 pm, there wasn't many people around to help us take a photo, so we approached an idle-looking security guard who was standing outside HMV. He said no, I asked him why, he said it's his job to guard the gates - those magnetic gates at HMV that sound of alarms when someone tries to take an unpaid merchandise out. We looked for another person to take the photo for us.

In what happened in a blink of an eye, while we were gathering for a pose, the alarm sounded, the camera flash went off, and a guy was pinned down by the security guard whom we approached earlier, just beside the magnetic gate. Everyone gasped. Another security guard came over, helped to restrain the guy (a young malay boy by the way), and took the guy into a room. Before we could even move out of our formation, that same security guard was back at his original patrol position, and we didn't even notice when he moved!

After that incident, I had a very different perspective towards incident response. Hence I have a different approach to incident response for the systems I support/manage, I have hawk eyes on incident prevention. However, with resource optimisation, and optimising optimised optimisation, incident prevention is almost a non-existent scope of work. You are hired as the security guard to guard the gates, to control what goes in and out of your system, but instead of watching the door, it's likely that you are re-arranging a CD, or processing a customer's payment, or receiving a delivery, or helping a customer look for a soundtrack he wants, anything except standing at the gate, watching. As a result, any thief (incident), almost always escapes your eyes (avoid getting prevented), and becomes an incident (theft case), which you will then need to respond. You then still have to stop re-arranging the CD, or stop processing the payment, or stop receiving the delivery, or stop helping the customer look for his soundtrack, and chase after the thief, in vain.

You then activate the other teams to help you trace that guy, and as you are on it, your boss will ask you to provide a regular status update on the situation, and it's likely that you won't be able to trace the guy (don't know cause of incident), don't know what he took (because it's not RFID, just magnetic strip due to cost cutting measures), and your boss still insists you trace the CD, so you end up having to do an ad hoc stock inventory, tally it against your point-of-sales system for the day (because the IT system doesn't have a report to handle such a scenario), and finally as a mitigating measure, you state that you need to be more vigilent in future, and it's weird if you say that, so your boss helps you say that.

That's why some people look idle, always idle (due to sampling frequency), and are deemed idle. While you are posing for a photo, he would have stopped an incident from happening, and returned to his idle position, without you noticing.

No comments:

Post a Comment