Tuesday, September 18, 2012

Risk Assessment in New Software Product Development Projects




All new Software product development plans involve risks. These risks must be handled and managed using project management techniques.  From a project planning perspective, risks is classified into high risk, medium risk, or low risk, and appropriate risk mitigation plans has to be developed. Project risk management is a complex subject and has several nuances. However for simplicity of understanding, this article will deal with risk assessment from new product development perspective.

At the high level, the risk for any new product development project can be classified into:

1. Probability of the product not delivered on time.
2. Probability of the product not developed within budget
3. Probability of the product not having all the required features or not working as intended.

The job of management is to assess the risks and develop risk mitigation strategy.  With good management and product development process, all risks can be assessed early in development process and eliminated by proper allocation of resources.

A good approach for risk assessment includes:

1. Establishing a set of risk assessment criteria derived from both the organization's experience and industry wide data.

2. Consensus among all the parties involved - i.e., Product manager, marketing, finance, engineering, QA, &  project manager etc. The entire team agrees on the various risks involved in the project.

Based on this assessment, the project risk is usually classified into High(> 30% probability), Medium (> 20% probability) or Low risk ( < 10% probability).

It is important to establish a company wide database of risk criteria for your software systems development projects. In large organizations, office of program management maintains this and is primarily responsible for risk assessment.  The risk assessment logic can also be applied at each individual task level, sub-task level, etc. Furthermore, as the project unfolds, this logic can be applied on a periodic or event-driven basis as a part of your overall risk management approach.

Once the risk assessments are completed, the product team: Product manager, marketing, finance, engineering, QA, &  project manager etc., is responsible to developing the risk mitigation team.

Often times the risk mitigation plan would involve adding resources to the area that has high risks. If additional resources does not mitigate the risk, then in some cases certain requirements may have to be modified.

Risk Criteria

The main outcome of risk assessment exercise is to identify all the risks involved in the project and based on the set of risks, the project is classified into High, Medium or Low risk project.

1. High Risk Projects 

Projects that have more than two of the following risks identified, then such product development projects are usually classified as high risk project:

In case of new product development which is developing a "new-to-world" or "new-to-company" type of products - the risks are always high due to:

1. Unique product
2. Lack of clear requirements documentation.
3. Inexperienced development staff
4. No clear QA & test plans
5. Unknown customer use cases
6. Financial constraints
7. Technical challenges such unavailable technology or tools, etc.

Even projects that have high levels of subcontractor labor - with at least 50% of total development labor should be treated as high risk projects - due to less management oversight, lack of labor flexibility, and over dependence on subcontractor.

The project must be classified as "high-risk" if the product is a mission critical product - i.e., the failure of the product leads to death or injury or very large financial losses.  For example failure of Control system products can lead to death or injury during testing, Financial trading software that can lead to big losses (like the one that brought down Knight Capital.  Knight Capital lost $440 million in 45 minutes of trade.

Such high risk products must have extensive beta testing programs before they can be released.

Even projects that are just new revisions of an existing product can become high risk project if:

1. Development or QA resources are not fully committed.
2. Product requirements are not fully defined.
3. Project funding is not secured - i.e., management team is not in full agreement for the project.
4. There is no slack or buffers in the project schedule.

Once a project is identified as a "high-risk" project, full commitment from all stake holders is required for risk mitigation. Without total commitment from all the stake holders the new product development project is doomed to fail. In such a case, the project must be kept in abeyance till all stake holders agree and give their full commitment.  

High risk projects demand a very close monitoring of project and project risks. High risk projects require a greater commitment of resources - perhaps in form of reserve pool of resources being identified and kept ready for rapid deployment if needed.

High risk projects also require high levels of testing, Quality assurance and on-site validation. For high risk projects - about 65-75% of the total efforts are allocated to testing & quality assurance.

Risk mitigation plans for such high risk projects are almost always never complete, and with the risk materializes, the management team will have to tailor/modify risk mitigation plans.

2. Medium-risk projects 

Medium risk projects are usually the average software products that are:
1. Another revision of a current product.
2. Does not play a mission critical role at customer site
3. Customer has no critical dependency on the product
4. The functionality of the product is well understood.
5. Developer has some flexibility on release dates
6. Product has few fall back options: reduce functionality or release a patch after the release etc.
7. There is no over dependence on subcontracted labor (i.e., subcontracted labor is less than 50%)
8. Most of the project efforts are in product development - i.e., to add new features.
9. Failure of the product at customer site results in a negative publicity for the product or/& company.

Routine product enhancements can also become a medium risk project if:

1. The development or QA staff is inexperienced
2. Product testing procedures or resources are inadequate for the new features
3. There is little slack or buffer in project schedule
4. Requirements are not fully understood by the development team
5. There is uncertainty in product requirements - either the PRD is not fully developed.
6. There is dependency on external partners or vendors
7. External Market conditions are uncertain - i.,e possible recession etc.

Medium risk projects have to be monitored with pre-approved risk mitigation plans. These risk mitigation plans must be exercised immediately when needed to avoid  project failures.

3. Low-risk Projects

Projects that are not classified into high risk or medium risk projects are by default low-risk projects. In Low risk projects:

1. Product requirements are fully understood.
2. Have experienced development staff.
3. Have flexible schedules
4. Product failures do not cause any losses & can be easily rectified with a patch releases.

In low risk projects, most  (> 80%) of the resources are allocated for product development and has no allocated reserves. Usually teams/resources for low risk projects are identified as reserve resources for high risk projects, and such resources can be quickly redeployed for high risk projects - if there is a need.

Risk Mitigation planning

In most of the cases, risk mitigation plans translates to identification of additional resources that can be used when needed. In reality, most organizations are stretched for resources and often will have nothing to spare for a high risk project. In such cases, Management is running on hope & luck - to see the project through. Failures in such situation will be disastrous - often at an enormous cost. Many startups die because they cannot afford resources for risk mitigation. Even in large companies, projects get canceled when there are no resources for risk mitigation.

As a prudent management planning, companies today are using the option of killing the product development projects are part of their risk mitigation strategy. In mission critical projects, it may be prudent to kill the product if the risks become too high.

Closing Thoughts

It is important for all stake holders to be involved in the risk assessment process & achieve consensus on project risks. Risk assessment starts with the product definition stage, right at the time of developing the product requirements and then working as a team to develop plans for risk mitigation.

All stake holders must have full visibility to risk identification, risk assessment and risk mitigation plans. This leads to better understanding of the risks associated with the new product development projects and thus allocate the right resources needed for successful new product development.

Risk mitigation plans are almost always never complete, mainly because, resource estimating is, at best, educated guesswork. So when a project risk materializes, the entire management and project teams will have to work to mitigate it. This often needs heroic efforts from few individuals.

Product managers must be prudent to kill the project when necessary. Many a times it is better to kill a project than throw more resources at a doomed project. This decision of killing a project is more of an art than science.

No comments: