State of the Practice for Lattice Boltzmann Method Software.

We analyze the state of software development practice for Lattice Boltzmann solvers by quantitatively and qualitatively measuring and comparing 24 software packages for 10 software qualities (installability, correctness/verifiability, reliability, robustness, usability, maintainability, reusability, understandability, visibility/transparency and reproducibility). Our reproducible analysis method employs a measurement template (containing 108 measures that are manually and automatically extracted from the project repository) and developer interviews (consisting of 20 questions). After measuring, we rank the software using the Analytic Hierarchy Process. Our ranking is roughly consistent with GitHub stars ranking, suggesting at least a partial correlation between the use of best practices and popularity. We find the state of the practice to be healthy with 67% of the measured packages ranking in the top two for at least two qualities, the majority of LBM generated artifacts corresponding to general recommendations from research software developers, common use of version control (67% of packages) and the adoption of a quasi-agile development process. Areas of best practice to potentially improve include adoption of continuous integration, API documentation and enforcement of programming style guides. We interviewed four developers to gain insight into their current pain points. Identified challenges include lack of development time, lack of funding, and difficulty with ensuring correctness. Developers are addressing these pain points by designing for change, circumventing the oracle problem and prioritizing documentation and usability. For future improvements we suggest the following: employing linters, conducting rigorous peer reviews, writing and submitting more papers on software, growing the number of contributors by following current recommendations for open source projects, and augmenting the theory manuals to include more requirements specification relevant information. [ABSTRACT FROM AUTHOR]