This thesis investigates whether machine learning can automatically learn what normal data looks like and reliably flag deviations, serving as a low-cost complement to rule-based checks at the point where data first enters the pipeline, and whether AutoML-based anomaly detection is a practical complement to rule-based checks in data pipelines.
A formal model that assists practitioners and designers to construct resilient, adaptable, and budget friendly data pipelines, which are in line with the current data needs is delivered.
Madhurima Kommuru· International Journal of Dat...· 0 citations
Enterprise data platforms run tens of thousands of scheduled jobs every day to feed operational, financial, and regulatory reporting. When these jobs fail, the failure is usually found late, after a business user notices a missing report. Worse, one upstream failure floods the operations team with hundreds of independe...
Gowtham Reddy Pappula· International journal of com...· 0 citations
The novel method of co-evolution labeling for predictive test optimization is introduced, deriving test relevance from tests and code changing together in the version history, which nearly matches the failure detection capabilities of failure-based models, while being more resistant to label noise and requiring no test...
Maximilian Jungwirth, RaphaelN ̈ommer, Andreas Stahlbauer et al.· 0 citations
Data is a critical component of modern decision-making systems; system malfunctions (e.g., performance degradation and module failure) can often be traced back to a mismatch between the properties of the data and the assumptions of the system modules that process the data. For example, with the increasing use of open-s...
Jahid Hasan, Stanley Jiang, T. Singh et al.· Proceedings of the VLDB Endo...· 0 citations
The increasing complexity and scale of modern cyber-attacks demand intelligent and computationally efficient Intrusion Detection Systems (IDS). However, designing effective data pre-processing pipelines traditionally involves substantial trial-and-error effort and repeated evaluation of alternative configurations. For...
Bao-Phong Nguyen, G. Pham, Thai-Duong Do et al.· Journal of Supercomputing· 0 citations
Traditional data reduction pipelines are run on local or remote machines and depend on user interaction for execution. This approach can lead to reproducibility and discoverability issues, as processing decisions made by the user go undocumented and are made in a single instance leading to difficulty in reproduction. O...
James Coulson, Nuria P. F. Lorente, Brent Miszalski et al.· Astronomical Telescopes + In...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
