Evaluating Interactive 2D Visualization as a Sample Selection Strategy for Biomedical Time-Series Data Annotation

Dimensions overall score 4.0

• The 2D visualization (2DV) sample selection method performed best when aggregating labels across annotators for all classification tasks.result 95%
  Evidencepartial

  Across all classification tasks, 2DV performed best when aggregating labels across annotators.

  Implicationpartial

  This is a primary finding explicitly stated in the abstract and supported by the overall results.

  Verificationpartial

  partial

• In infant motility assessment (IMA), 2DV was most effective at capturing rare classes.result 90%
  Evidencepartial

  In IMA, 2DV most effectively captured rare classes

  Implicationpartial

  This is a specific result mentioned in the abstract regarding the performance of 2DV in a particular domain.

  Verificationpartial

  partial

• The 2DV method exhibited greater annotator-to-annotator label distribution variability in IMA, which decreased classification performance when models were trained on individual annotators' labels.limitation 90%
  Evidencepartial

  but also exhibited greater annotator-to-annotator label distribution variability resulting from the limited annotation budget, decreasing classification performance when models were trained on individual annotators' labels

  Implicationpartial

  This is a specific limitation and its consequence for the 2DV method in IMA, clearly stated in the abstract.

  Verificationpartial

  partial

• In cases where models were trained on individual annotators' labels in IMA, the farthest-first traversal (FAFT) method excelled.result 90%
  Evidencepartial

  in these cases, FAFT excelled.

  Implicationpartial

  This is a comparative result highlighting the strength of FAFT in a specific scenario where 2DV struggled.

  Verificationpartial

  partial

• For speech emotion recognition (SER), 2DV outperformed other methods among expert annotators and matched their performance for non-experts in the individual-annotator setting.result 90%
  Evidencepartial

  For SER, 2DV outperformed the other methods among expert annotators and matched their performance for non-experts in the individual-annotator setting.

  Implicationpartial

  This is a specific result detailing the performance of 2DV across different annotator expertise levels in SER.

  Verificationpartial

  partial

• Random sampling (RND) was the safest choice when annotator count or annotator expertise was uncertain, whereas 2DV had the highest risk due to its greater label distribution variability.result 95%
  Evidencepartial

  A failure risk analysis revealed that RND was the safest choice when annotator count or annotator expertise was uncertain, whereas 2DV had the highest risk due to its greater label distribution variability.

  Implicationpartial

  This is a direct conclusion from the failure risk analysis, clearly stated in the abstract.

  Verificationpartial

  partial

• The 2DV method made the annotation task more interesting and enjoyable for annotators.result 90%
  Evidencepartial

  Furthermore, post-experiment interviews indicated that 2DV made the annotation task more interesting and enjoyable.

  Implicationpartial

  This is a qualitative finding from post-experiment interviews, indicating a benefit of the 2DV method.

  Verificationpartial

  partial

• The 2DV-based sampling strategy is promising for biomedical time-series data annotation, especially when the annotation budget is not highly constrained.result 95%
  Evidencepartial

  Overall, 2DV-based sampling appears promising for biomedical time-series data annotation, particularly when the annotation budget is not highly constrained.

  Implicationpartial

  This is the overarching conclusion of the study, summarizing the potential of the 2DV method.

  Verificationpartial

  partial

Competitive landscape

A novel interactive visualization tool improves the accuracy and efficiency of annotating complex biomedical time-series data.