Topic Definitions #
Methods for Evaluating Ideas in Project
Introduction to Idea Evaluation
During the planning phase, project managers and teams assess how each idea impacts the project outcome, evaluate risks and resource needs, and decide the optimal execution order. As a result, they create a prioritization system that maximizes outcomes while optimizing time, budget, and effort.
Expert Evaluation
What is Expert Evaluation?
Expert evaluation refers to the process of analyzing how promising, realistic, or useful an idea or project may be. Specialists assess technical feasibility, economic viability, risks, and expected outcomes. Their conclusions stem from hands-on experience, industry insight, and an ability to forecast potential results.Why are Expert Evaluations Important?
This method provides valuable insights when teams lack measurable data. Experts help fill information gaps and offer well-informed, qualitative judgments. Although such assessments can be subjective, they often prove accurate due to professional experience and deep contextual understanding. In situations with limited data or complex choices, expert evaluations speed up decision-making and guide teams toward success. Combining expert input with quantitative analysis ensures a balanced and comprehensive assessment.
Scope
Feature
Project Idea Evaluation Methods #
Goals:
- Identify which ideas to include in the project;
- Determine the most valuable ideas and evaluate
- their potential impact;
- Allocate resources efficiently across tasks;
- Reduce risks by thoroughly analyzing alternatives.
Let’s now explore some popular evaluation methods that help achieve these goals.
Impact and Effort Evaluation #
Teams can evaluate ideas using two key factors: impact and effort evaluation.
Impact
This measures the extent to which an idea influences project success.
Effort
This evaluates the resources required, including time, money, and workforce.
Ideas generally fall into four main categories:
-
Easy Wins
High impact, low effort. These tasks deliver quick benefits with minimal investment.
-
Big Bets
High impact, high effort. While requiring extensive planning, these can lead to substantial rewards.
-
Incremental
Low mpact, low effort. These tasks contribute positively but without major impact.
-
Money Pit
Low impact, high effort. Such tasks often offer little value and should be avoided.
Additionally, intermediate options may offer moderate effort and reasonable impact. Teams can pursue these if resources are available. This method enables project teams to identify ideas that provide the greatest benefit at the lowest cost.
Voting #
A democratic decision-making process is often used for idea evaluation. In this approach, team members select and vote on the ideas they consider most promising for implementation. This method proves especially effective when multiple stakeholders or experts participate in the discussion.
Voting Process:
- Review the proposed ideas.
- Select and vote for the ideas you consider most important.
- Ideas with the most votes proceed for further development.
The image showcases different voting methods can apply to prioritize ideas effectively:
-
Ranked Choice Voting
Participants rank ideas by preference, creating a priority list.
-
Points Allocation
Each person distributes a set number of points to various ideas.
-
Pairwise Comparison
Ideas are compared in pairs to establish overall priorities.
-
Dot Voting
Participants assign a limited number of dots (votes) to the ideas they support.
-
MaxDiff Analysis
Participants identify the most and least preferred ideas, forming a priority gradient.
Using these methods allows teams to evaluate and prioritize ideas in an objective and collaborative manner.
Impact Mapping #
This approach clarifies factors that influence project success and guides teams to focus on actual business needs. Impact Mapping helps visualize how business goals align with specific actions and outcomes.
The image above shows the following steps to apply the Impact Mapping method:
“Why?” — Define the Goal:
- The first step in the process is to clearly define the business goal the project is aiming to achieve. It’s crucial to answer the question “Why?” — why is this goal important to the business, what problem does it solve, and how will it contribute to success?
“Who?” — Identify Key Actors:
- Next, it’s essential to identify “Who?” — the key actors who can influence the achievement of the goal. These could be both external and internal stakeholders, such as users, customers, or employees. Each actor affects the project in different ways, and this impact must be considered.
“How?” — Analyze the Impact:
- The following step is to answer the question “How?” , — how can the behavior or actions of the actors influence the business goals? It’s important to identify both the positive and negative impacts of each action on goal achievement. The diagram shows how actors’ actions are linked to different levels of impacts.
“What?” — Determine the Deliverables:
- Finally, you need to answer the question “What?” — what results or key project elements (such as epics, feautures, or user stories) will be delivered if the goals are achieved. This stage involves defining the specific product elements needed to realize the goal.
Expected Monetary Value (EMV) #
The Expected Monetary Value (EMV) method evaluates different execution scenarios by applying probabilties and monetary estimates. This approach helps teams assess the potential costs or benefits of implementing an idea while considering possible risks and their likelihood.
Formula for calculating Expected Monetary Value (EMV):
Where:
- P {\scriptscriptstyle 1} is the probability of each outcome occuring (from 0 to 1);
- V {\scriptscriptstyle 1} is the monetary value or benefit of each outcome.
To calculate EMV, teams multiply the probability of each outcome by its corresponding monetary value and sum the results. This formula helps estimate the expected financial impact of an idea, considering both potential benefits and associated risks.
Teams often represent the EMV method using a decision tree, which follows these steps:
-
1. Define the Decision (Decision Node):
The first step is to define the decision that needs to be made. In the example shown, it’s “Should we build a new plant or renovate the existing one?”.
-
2. Define Scenarios and Probabilities (Probability Node):
Next, for each decision option, possible scenarios are outlined. For example, a “strong demand” scenario (with a 60% probability) or a “weak demand” scenario (with a 40% probability). Each scenario has a likelihood associated with it.
-
3. Calculate Costs for Each Scenario:
For each scenario, the potential cost or benefit is calculated. In the example, building a new plant would result in an $80M profit under strong demand or a $30M loss under weak demand.
-
4. Calculate EMV (Expected Monetary Value):
Ideas with low impact and high effort. These tasks are not worth the investment and typically should not be pursued.
Where:
- P {\scriptscriptstyle 1} is the probability of each outcome occuring (from 0 to 1);
- V {\scriptscriptstyle 1} is the monetary value or benefit of each outcome.
In the example, the EMV for building a new plant is $36M, while for renovating the existing plant, it’s $46M.
-
5. Choose the Best Option (End of Branch):
The final step is to choose the option with the highest EMV. In this case, renovating the plant is more favorable, as its EMV ($46M) is higher than that of building a new plant ($36M).
The EMV method, combined with decision trees, is a powerful tool for risk analysis and making optimal decisions under uncertainty.
Weighted Shortest Job First (WSJF) #
Teams prioritize ideas, features or tasks using the Weighted Shortest Job First (WSJF) method. This approach evaluates ideas, features or tasks based on business value, time criticality, and their potential to reduce risks or create opportunities. By comparing these values to the task size, teams establish a priority queue for execution.
WSJF formula:
Where:
Business Value:
How important the task is to the business or users.
Time Criticality:
How urgent it is to complete the task soon.
Risk Reduction or Opportunity Enablement:
How much the task helps reduce risks or opens new opportunities.
Job Size:
The estimated complexity or effort required to complete the task.
Where:
- B V – Business Value: How important the task is to the business or users.
- T C – Time Criticality: How urgent it is to complete the task soon.
- R or O – Risk Reduction or Opportunity Enablement: How much the task helps reduce risks or opens new opportunities.
- Job Size: The estimated complexity or effort required to complete the task.
Steps for Applying WSJF:
1. Define Task Metrics:
Evaluate each feature or task based on metrics such as business value, time criticality, and risk reduction or opportunity enablement. In the table provided, these values are scored from 1 (low) to 10 (high).
2. Assess the Task Size:
Evaluate the complexity or size of the task (Job Size) on a similar scale. A higher score means more effort or time is needed to complete the task.
3. Calculate the Priority:
For each task, calculate the priority by dividing the sum of business value, time criticality, and risk reduction by the task size.
For example, for “Feature A”:
WSJF = {10 + 1 + 1 \above{1pt} 1} = 10And for “Feature D”:
WSJF = {1 + 8 + 10 \above{1pt} 8} = 1,254. Rank Tasks by Priority:
Rank the tasks by their WSJF score: the higher the value, the higher the priority. In the example, Feature A gets the highest priority with a score of 10.
This method helps focus on tasks that deliver the highest value in the shortest possible time, with minimal effort.
RICE (Reach, Impact, Confidence, Effort) #
Teams use the RICE framework to prioritize ideas, features or tasks based on four key factors: Reach, Impact, Confidence, and Effort. This method ensures that the most valuable tasks receive priority while considering the cost of implementation.
RICE Formula:
Where:
Reach:
How many people or segments will be impacted by this feature or task? This factor is usually represented as a number of users or affected entities over a defined period.
Impact:
How strongly will this feature or task affect the user or the business? This is usually rated on a scale from 1 (minimal impact) to 5 (massive impact).
Confidence:
How confident are we in the accuracy of the reach and impact estimates? This is expressed as a percentage.
Effort:
How many resources (measured in person-months, for example) are required to complete the task? A higher score indicates a larger effort.
The RICE score helps project managers focus on tasks that provide the most value with the least effort.
Steps to Apply the RICE Method Using the Example in the Image:
1. Define the Feature or Task:
List the features or projects that need to be prioritized. In the image, the list includes:
- Biometric Authentication
- Stripe Pay Integration
- Shopify Ecommerce Cart
- API Dashboard
2. Evaluate Each Feature Across the Four RICE Factors:
Reach:
- Estimate how many users or stakeholders the feature will affect. In the image, Biometric Authentication has a reach of 81 users, while API Dashboard affects 43 users.
Impact:
- Rate the impact on a scale from 1 to 5. In the example, Biometric Authentication has a maximum impact score of 5, while Stripe Pay Integration has a slightly lower impact score of 3.
Confidence:
- Provide a confidence percentage to show how sure you are about the accuracy of your reach and estimates. Biometric Authentication has 75% confidence, whereas Shopify Ecommerce Cart has 50%.
Effort:
- Rate the effort required to implement the feature on a scale of 1 to 5. A lower score indicates less effort. In the image, Biometric Authentication has a relatively low effort score of 2, while API Dashboard has a higher effort score of 3.
3. Calculate the RICE Score:
- Apply the formula for each feature:
- For example, for Biometric Authentication, the calculation would be:
- For Stripe Pay Integration:
4. Rank the Features by RICE Score:
- After calculating the RICE scores, rank the features from highest to lowest. In the image, Biometric Authentication ranks the highest with a RICE score of 152, indicating that it should be the top priority, followed by Stripe Pay Integration (35), Shopify Ecommerce Cart (29), and API Dashboard (14).
The RICE method allows you to focus on the most important functions, ensuring the most efficient use of resources. This method allows you to balance the impact of an idea and its implementation, taking into account resource costs.
MoSCoW (Must, Should, Could, Won’t) #
The MoSCoW method is based on the principle of clearly regulating priorities. Its main idea is to help the team decide what must remain in the project scope, what can be added as secondary, what could become a nice-to-have bonus if resources allow, and what should be consciously excluded. This approach helps avoid vague requirements, focus on key results, and communicate expectations transparently among all stakeholders.
In fact, MoSCoW is a tool that eliminates uncertainty: we clearly define what we are creating, what is postponed “for later,” and what remains in question or outside the scope. Such an approach creates a balance between business ambitions and resource limitations, resulting in a realistic roadmap.
Step 1: Simple Task Distribution Using MoSCoW
The image below illustrates a simple task distribution using the MoSCoW methodology, without the use of buffers. Each task is categorized as follows:
-
Must Have:
These tasks occupy no more than 60% of the total effort, and they are essential for achieving the project goal.
-
Should Have:
Tasks that should be completed but are not critical, typically occupying around 20% of the effort.
-
Could Have:
Additional tasks that can be completed if there are resources available, taking up around 20% of the effort.
-
Won’t Have:
Tasks that will not be included in this cycle.
This approach clearly allocates time and resources based on task importance, with no mention of buffers.
Step 2: Detailing the Approach with Buffers
The image below provides a more complex workflow, showing how buffers are used during the Planning Phase and Execution Phase:
In this phase, 60% of the resources are allocated to “Must Have” tasks.
- External organizations or individuals who provide materials, equipment, or services for the project.
In the execution phase, “Must Have” tasks remain the top priority, and the project cannot be completed without them.
The remaining tasks can be completed if time and resources allow, utilizing the buffer.
This flowchart demonstrates the importance of having a buffer to manage unexpected delays or additional tasks.
Step 3: Example of Feature Allocation Using MoSCoW
The image below presents a matrix where project features are distributed according to the MoSCoW categories.
-
Must Have:
Mobile app integration and API for third-party integrations are critical tasks that must be completed.
-
Should Have:
User profile customization is important but not critical to project success.
-
Could Have:
Multi-language support can be added if resources are available.
-
Won’t Have:
The dark mode feature will not be implemented in this cycle.
This approach enables the team to concentrate on the most critical tasks, ensuring efficient allocation of effort and resources based on priority.
By applying various evaluation and prioritization methods, project teams make well-informed decisions that consider impact, effort, risks, and long-term value. Selecting the most suitable methods based on project specifics, goals, and available resources is essential for success.
Conclusion #
Selecting the right ideas for implementation is a cornerstone of successful project management, and the diverse evaluation methods presented—such as Impact and Effort, Voting, Impact Mapping, EMV, WSJF, RICE, and MoSCoW—offer project teams a powerful toolkit for making well-informed decisions. Each method provides a unique lens through which to assess ideas, enabling teams to balance business impact, urgency, risk, resource allocation, and stakeholder alignment.
Whether it’s identifying low-effort, high-impact “easy wins,” calculating financial risks through EMV, or prioritizing based on confidence and reach using RICE, these frameworks allow for both strategic clarity and operational precision. Moreover, applying structured prioritization methods like WSJF or MoSCoW ensures that critical tasks receive the focus they deserve, while less essential ideas are appropriately deferred or excluded.
Ultimately, the thoughtful use of these evaluation techniques empowers teams to maximize value, reduce waste, and ensure that every decision contributes meaningfully to project success. The key lies not in using every method, but in choosing and combining those best suited to the project’s context, goals, and constraints.
