An Infinite Problem Space – ACIS

While a number of software defects are the result of simple human errors that escape either scrutiny or test cases, the vast majority of defects found in ACIS come out of geometric cases that were not - and likely could not, be practically anticipated. If you think about the problem of testing geometric cases thoroughly and start enumerating them, it quickly becomes obvious that the number of possible test scenarios are near infinite as the input space (combinations of topology and geometry) is nearly infinite. 

This problem is not unique to ACIS. All the modern boundary representation geometry kernels face the same challenge. And due to the commercial nature of the modelers (they have to handle whatever customers ask them to handle) simplifying the problem set by restricting input and operations is not an option. A very large portion of defects in modeling kernels arises from their handling of non-manifold topology and operations on mixed mode topologies, as these cases are much harder to check for correctness. As modeling operations can fail even while producing some kind of output, simple code coverage metrics are insufficient. A test can run through the code but if the test itself is not able to check all outputs for validity, not uncommon, a defect can escape even with the code path being exercised. 

Because all test cases cannot be practically enumerated, testing becomes a creative but heuristic endeavor. Developers have to become very creative in generating test cases, and checking for valid results. Simple checks of mass properties are insufficient, face counts, gaps, sliver faces, edge directions, surface normals, should be examined.

While it is impossible to guarantee correctness for all inputs and all operators on the input the good news is that ACIS has so many applications using it for so long that the vast majority of cases have been tested, in this case with real world inputs. As noted in John Sloan's post "Evolutionary Biology and Software Engineering" - you can think of these huge code bases as small eco systems that evolve under different environments. The richer and more demanding the environment the better evolved the organisms in it. 

While never perfect, after 15 years of being bombarded with millions of input combinations, ACIS has become very evolved indeed.