A

Possible Metrics for Assessment of Program and Project Accomplishments and Plans

PROGRAM AND PROJECT ACCOMPLISHMENTS

• Did the accomplishments represent significant scientific advances?
• What are major consequences for the science if the project succeeds?
• Is the potential, long-term Army application of the research significant?
• Is the research novel, leading the field in an important area, and does it have the appropriate level of risk and payoff?
• Was related research being sponsored by other major players adequately summarized in terms of approach and/or goals? Were there areas of duplication?
• When comparisons are made, are the comparison groups well defined?
• How do the accomplishments map to the stated program goals and Army Research Office (ARO) strategic plans?
• Do the accomplishments reflect productivity and ingenuity on the part of the performers?
• What portion of the accomplishments is attributable to the ARO funding?
• What is the network of contacts involved in the project?
• Were there appropriate examples of significant transitions, or anticipated transitions of research, to follow on applied research or exploratory development either within industry or within an Army or Department of Defense (DoD) laboratory?
• Are there any high-priority missed opportunities/areas relating to the program or project?
• Was the speed of knowledge acceleration or transfer appropriate?
• What are the number, type, and caliber of awards and recognition related to the program or project?
• Has follow-up funding been awarded for the project or program?

PROGRAM PLANNING

• How, specifically, does the project address one or more critical challenges that the Army of the Future will face?
• Should research funding topically align with the current or the anticipated future focus of a division/program?
• Is there a clear and cogent strategy regarding how each of the program managers’ major objectives are likely to make substantial and unique progress in advancing scientific frontiers of their discipline?
• Is there some reasonable basis (e.g., incipient breakthrough, new understanding, or novel theory) to believe that the scientific objectives might be met?
• Have the highest priority objectives been selected?

• If not, what is the potential that the accomplishments will lead to significant scientific advances?
• What mixture of collaboration (within ARO and with outside teams/agencies) and competition is best? What percentage of projects should be unique to ARO?
• If higher priority areas are identified, what projects or programs should be reduced or eliminated to accommodate the new areas?
• Are anticipated transitions within the Army likely to be greater than transitions to the commercial sector or other government agencies—for example, the Naval Research Laboratory (NRL) or Office of Naval Research (ONR)?
• Should funding portfolios be homogeneous or heterogeneous? Should funding of divergent or convergent works be preferred?
• What diversified portfolio of short-/medium-/long-term projects; low-/high-risk projects; and collaborative versus ARO-only projects is best, and why?
• What project type (e.g., preliminary exploration, longer term research, workshop, or Multidisciplinary University Research Initiative [MURI]) is best, and why?
• What ratio of faculty, staff, and students is best for a given topic and type of project, and why?
• What principal investigator (PI) turnover rate is best?
• How closely should program managers work with principal investigators? Are research collaborations between both required for a given project?
• What diversity ratios (e.g., gender, age, ethnicity) are best for a given project or program, and why?
• What percentage of a given project or program should be undertaken by postdoctoral researchers and graduate students?
• Are the most qualified applicants selected, independent of geospatial distribution?
• What composition of researchers, practitioners, and entrepreneurs is best to ensure high-quality research results that can be translated into products?
• What are the difficult aspects of a proposed project, and what are the major technical risks involved?