September 02nd, 2018

How to Evaluate Technology
According to Potential Impact

...consider how impactful the result would be, if it were achieved. What would it do? What would it change? Does it help solve a big problem? Does it put a dent in a major challenge?"

While How to Focus, Categorize, and Integrate Your Urban Technology covered key topics, it did not get into the details of evaluating technology product types for potential impact. How to Combine Mapping and Results for Holistic Organizations states, “The impact your organization makes is the combination of the proven results across the three levels – functional, cross-functional, and outcome-based. Each level is important in its own right, but it is the combination of all three that holds real power for exponential impact.” Further, potential impact is the likelihood of successfully moving from anticipated results (AR) to proven results (PR). Review the second link above and How to Map for Outcomes for an explanation of functional, cross-functional, and outcome-based levels as well as for AR and PR. This is a framework you’ll use to evaluate potential impact.  
 
Pinpoint each technology’s anticipated result
First, alone or with colleagues, build out a table that lists the technology product types under discussion in the left column. For each technology, decide where each one best fits - functional, cross-functional, or outcome-based. In some cases, a technology that is multi-faceted could belong to more than one of these types. In such cases, describe the detail according to the appropriate column. Explain the anticipated result in each box, as you currently understand it. This will evolve over time as you learn more.   

Evaluate
While there are many ways to evaluate and prioritize, I’ll share a fairly simple, subjective approach for you to consider using. To evaluate, use three subjective criteria for scoring including 1) ease of implementation, 2) potential impact, and 3) organizational focus. Ease of implementation encompasses a multiple of subfactors such as cost, internal/external support, staff skills, set up and maintenance, and others. It could vary depending on the potential for partnerships in cases where implementation would benefit from external collaboration. Potential impact is the likelihood of successfully moving from anticipated results (AR) to proven results (PR). Take a look at the anticipated result table you created. Whether the AR is functional, cross-functional, or outcome-based, consider how impactful the result would be, if it were achieved. What would it do? What would it change? Does it help solve a big problem? Does it put a dent in a major challenge?

Then, consider the likelihood of the AR becoming a PR. Sometimes, we have a great technology idea, but there are reasons it is highly unlikely it will get off the ground. Some barriers are surmountable, and others are not. As a professional or group of professionals, it is up to you to understand such implications. If a technology with an AR is very impactful, but with a low likelihood of success, it should probably either be removed or placed on a low priority. The last criterion is organizational focus. This pertains to the organization’s core delivery areas, but it could also relate to its core competencies. This may be formally stated in the organization’s mission, goals, and tactics, or it may be a fuzzier concept worthy of more thought. Once you are done, you’ll end up with a table to summarize the evaluation. You could have actual numeric scores or “low, medium, and high” categories. 

Prioritize
From here, you will prioritize the technology product types in terms of low, medium, and high. There are a few rules of thumb to follow here. If a technology is “low” across all 3 criteria, it should be removed or put on a low priority.  If a technology is “high” across all 3 criteria, it should be put on a high priority.  If a technology is “low” on potential impact, it should be removed or put on a low priority. If a technology is “low” on organizational focus, it should also likely be removed or put on a low priority. If it is “high” on impact, regardless of the score for the other 2 criteria, it may deserve to be on a high or medium priority, unless it is also “low” on organizational focus, rendering it infeasible. And the list goes on until you have each technology associated with a low, medium, or high priority. After this step is complete, you can build the technologies into other planning efforts for budgeting and phasing. Keep in mind, this is a highly iterative process. This is just a first pass at bringing together a large and complex amount of decision-making information. You’ll continue to refine these steps as you move forward. Some of them may even be left grey while research is conducted to have a realistic idea of how the technology would perform or to understand its value to your mission.
 
Ecosystem and network effects
After you finish an initial pass at evaluation and prioritization, consider that some of the technology product types may be mutually reinforcing. What that means is that some might be stronger together than separate. Take a look at the integration diagram you created through How to Focus, Categorize, and Integrate Your Urban Technology. Are there connections between some of the technologies that warrant a note on your prioritization table that they should stick together? Perhaps two highly connected technologies are on different prioritization levels and should be brought onto the same one and thought of as a complementary pair. Think of how you can translate insights you gained from the integration diagram into your prioritization table. Learn a few new tools in How to Apply Strategic Planning Techniques to Manage Future Technology Unknowns.