The ideal world

R&D, Support Service and the IT Ops collaborate
in the same room, on the same incident.

Investigation gets easy and fast.
Root cause gets identified in no time.
Resolution is just a matter of hours.

The reality

R&D, Support Service and the IT Ops reside in their own space.
People work in silos.

And multiple support levels co-exist.

How can we ensure a perfect collaboration?

The production

Production environment is an excellent stronghold.

Expected for security reasons. No discussion.

You do not let strangers accessing it easily, even colleagues.

So data access is under strict control : procedures must be followed.
Every investigation step takes time and has a cost.

The geography

All actors are not always in the same geographical region.
Servers hosted in the US, support in India, R&D in Europe.

Communication gets slower because different time zones.
Actors get sometimes their answer only the next day.

Differences of culture can also affect the communication.

Time and distance should not impact the business.
We must abstract from that.

 

The means

IT operators, support people and developers do not think the same way.
They basically do not speak the same language.

Tools at their disposal are really different :
 – Production tools are monitoring oriented, very expensive.
 – Customer support have often home made investigation tools.
 – R&D tools are powerful but not adapted for production environments.

Issue resolution is currently relying again on human talents : the ones able to abstract the communication and the ones able to take advantage of the various tools. Rare people. We must do more.

The organizations

Different companies can be involved : barriers get in place.
It means different internal process flows and company cultures.

Team rotation is also a reality in support and production.
And resource turnover, especially in developing nations, cannot be ignored on medium term.

“Sorry but your support contact has unfortunately left the company.
Yes I know, he was very knowledgeable on the product.”
This frequent situation is clearly lost investment.
Consider also the investment required for the replacement. 
A persistent incident resolution solution must come into play.

The Ping Pong game

And finally are the interactions, called the ping pong game.

Successive round trips, taking time to access partial information,
hoping R&D will get the right ball to catch the issue.

Some people do play the watch, asking useless questions
while waiting for knowledgeable people to appear.

At the end of the day, the end user expects to get the critical issue resolved.
If not, what will be the cost and impact ?
Will your product or service be considered as reliable ?
Will competitors take advantage ?
Resolution must clearly be under control on all aspects.

Multiple failure factors are probably active in your organization

Incident impacts and related support costs
can be strongly reduced

How to make the incident management fast and efficient ?