A systematic mapping study method for software architecture research with LLM-assisted quality assessment (2026)

MethodsX

Authors

Andrei Olar, Oskar Picus, L. Cernau, Laura Dioșan, Simona Motogna

Abstract

This article describes the protocol used to conduct a systematic mapping study (SMS) of software architecture. The presence of architectural and design instruction in primary studies cannot be determined through search and superficial review alone. A principled quality assessment must be conducted to identify studies that are both relevant and of high quality. LLM assistance offers practical means of performing such assessments in a timely and efficient manner. Therefore, our contribution is threefold: • Guidance for the following SMS steps: tooling setup, database search, deduplication and snowballing, primary study selection, quality assessment, and data extraction. • The rationale and methodology to perform quality assessment on primary studies regarding software architecture. • A method of using large language models (LLMs) together with human evaluators is presented. The method is supported by bespoke open-source software.

Citation

@Inproceedings{Olar2026ASM,
 author = {Andrei Olar and Oskar Picus and L. Cernau and Laura Dioșan and Simona Motogna},
 booktitle = {MethodsX},
 title = {A systematic mapping study method for software architecture research with LLM-assisted quality assessment},
 year = {2026}
}