Augmenting Enterprise Architecture With Large Language Models: A Single-Case Empirical Evaluation Of The TOGAF Preliminary Phase In PropTech
TL;DR
This research affirms the necessity of a human-in-the-loop paradigm in the EA discipline within this specific context and suggests that LLMs function effectively as augmentation accelerators rather than human replacements, thereby shifting the enterprise architect's focus from manual drafting to analytical validation and strategic contextualization.
Abstract
Designing Enterprise Architecture (EA) artifacts, particularly during the initiation phase such as the TOGAF Preliminary Phase, is a crucial foundation for business and technology alignment. However, this formulation process often creates an operational bottleneck as it demands high precision, involves heavy administrative documentation, and consumes a significant amount of time. This single-case study aims to evaluate the effectiveness of utilizing a Large Language Model (LLM)—specifically Gemini 3.1 Pro with a structured Prompt Engineering technique—as a solution to accelerate the design of these documents. Through a case study on the development of Smart Estate Management capabilities within the PropTech ecosystem of PT XYZ, the AI-generated EA artifacts were evaluated by five senior IT experts. The evaluation methodology employed an Expert Judgement approach (Likert scale) cross-validated with in-depth qualitative interviews. The findings indicate a strong agreement among the participating experts that the LLM output possesses a high degree of clarity and structural consistency (median score 5.00/5.00). Furthermore, stakeholders perceived that the validated use of AI could provide an estimated time savings of approximately 80% for initial drafting. Nevertheless, the practicality of document implementation was rated neutral (median score 3.00/5.00) because the AI outputs were found to be inherently generic and unable to accommodate tacit knowledge, legacy infrastructure, and company-specific compliance constraints. Ultimately, this research affirms the necessity of a human-in-the-loop paradigm in the EA discipline within this specific context. It suggests that LLMs function effectively as augmentation accelerators rather than human replacements, thereby shifting the enterprise architect's focus from manual drafting to analytical validation and strategic contextualization.