AdaStop, a new framework developed by researchers including Bonan Shen, addresses the challenges of deep neural network (DNN) testing. Submitted on July 6, 2026, this innovative approach focuses on optimizing the labeling budget while maximizing fault discovery.
Understanding Cost-Aware Early Stopping in DNN Testing
Traditional methods for testing DNNs often rely on a fixed labeling budget, which can lead to either missed failures or excessive costs. AdaStop transforms this process by framing testing as a cost-benefit decision where the cost of labeling an input is weighed against the value of discovering a fault. This allows for more efficient use of resources.
The framework introduces a novel approach to estimate the marginal fault discovery rate during testing, enabling it to halt labeling when the estimated rate drops below a specific threshold defined as τ = c/v, where c is the cost and v is the value of discovering a fault.
Experimental Results and Effectiveness
In various experiments conducted across different datasets, architectures, and selection strategies, AdaStop demonstrated impressive results. The framework was able to uncover 65% to 84% of faults using only 9% to 31% of the labeling budget. This significant reduction in resource expenditure showcases the potential of AdaStop in real-world applications.
