Standard publication seeks to validate the two objectives indirectly, arguing that the editor and reviewers of the publication must be satisfied that the claims of novelty and ownership are true, and that the significance is high enough to meet the journal’s standards. There is obvious justification for this view, and so standard publication is an acceptable, albeit indirect, means of assessing impact. But it can be challenged on two counts. First, the same rationale can be applied to conference proceedings provided they are as carefully reviewed as the prestige conferences are in the computer science and engineering field. Second the measure of the impact is embodied in the quality of the publication, ie if the publication’s standards are high then the significance is presumed to be high. Not all papers in high quality publications are of great significance, and high quality papers can appear in lower quality venues. Publication’s indirect approach to assessing impact implies that it is useful, but not definitive.

The primary direct means of assessing impact—to document items (a) and (b) above—is by letters of evaluation from peers. Peers understand the contribution as well as its significance. Though some institutions demand that peer letter writers be selected to maximize the peer’s stature in the field, eg membership in the National Academy, a more rational basis should be used.