Test Smells through the Lens of the Dart Community: A Mixed-Methods Study on Grey Literature and Developer Perceptions

Abstract

Automated testing is a cornerstone of modern software quality, especially in rapidly growing ecosystems like Dart and Flutter. However, poor design choices in tests, known as test smells, can compromise maintainability and reliability. While test smells are widely studied in Java and C#, their prevalence and developer perceptions in the Dart ecosystem remain largely unexplored. This work aims to investigate how Dart developers perceive bad testing practices and identify which test smell patterns emerge from real-world community discussions.We conducted a mixed-methods study on grey literature (specifically Stack Overflow Q&A posts), analyzing 307 selected discussions that yielded 432 individual test smell occurrences. Applying Pareto Analysis, we find that a small subset of seven smells (the "Vital Few") accounts for approximately 80% of all occurrences, with Magic Number Test, Unknown Test, and Assertion Roulette being the most prevalent. Furthermore, we identify and formalize the Expected Resolution Omission (ERO) smell, a Dart-specific asynchronous testing anti-pattern that causes silent false positives or flaky tests. Our findings highlight critical pain points in the Dart testing community and provide a formally defined catalog to support developers and tool builders in improving test quality.