“Refers to the relationship between the software input change and output change.” Metamorphic relations are a great way of extending the test coverage of your ML application. That’s how metamorphic relations can contribute to solving the oracle problem.
But because you know that the label for the rotated image is still cancer, you don’t need to. You could then send this rotated image to be re-annotated to solve the test oracle problem. Take the image that you already have and rotate it. This means that only a part of relevant conditions can be tested for, leading to insufficient test coverage. To thoroughly test the system under different types of conditions (e.g., the color of the sample, type of microscope used to take the image, zoom level, etc.), more data and labels than can be realistically collected are needed. The problem is that these images are scarce to begin with and the ones that you do have will be expensive to annotate or collect. You can send these images to histopathologists, who can play the role of the test oracle by adding a label to each sample image. This is solved by having the image annotated. The test oracle problem presents itself because you have some input data, but you don’t know the label (cancer/no cancer). The output = cancer or no cancer diagnosis. The input = images of histopathology samples.
Imagine that you are building an ML system that is used as a diagnostic tool for cancer histopathology. Let’s look at an example from medical imaging. It refers to determining the correct test output for a given test input. The test oracle problem is not specific to ML, and it is well known from traditional software. Here we discuss how they can be used to extend the test coverage of your ML (machine learning) system beyond what can be achieved through normal data collection. In this part of our testing series, we’ll look at metamorphic relations – a technique used to multiply your available data and labels.
0 kommentar(er)
