The Real Software Crisis: Repeatability as a Core Value.

The authors argue that where appropriate, research papers in computer programming should be accompanied by software artifacts that let others test the repeatability of researchers' claims. The authors summarize results of an artifact evaluation process begun on an experimental basis at conferences of the Association of Computing Machinery, and provide commentary the mechanics of evaluating artifacts, who the evaluators should be, and the benefits of the evaluation process. Consideration is also given to whether artifacts should be published.