------------------------------------------------------------------------------------------------------------------------
Vector testing report
Params:
    conf: "conf/embeddings_validation_baselines_unsupervised.hocon"

Collected 30 files with 0 errors

------------------------------------------------------------------------------------------------------------------------
Metric: "auroc"
                           scores_valid                                                                                                                                                                                             
                                   mean  t_pm t_int_l t_int_h   std                           values t_f_stat t_f_alpha t_t_stat t_t_alpha t_delta t_delta_pm t_delta_l t_delta_h t_delta_pp t_delta_pm_pp t_delta_l_pp t_delta_h_pp
model_name feature_name                                                                                                                                                                                                             
lgbm       baseline               0.876 0.012   0.863   0.888 0.010  [0.891 0.879 0.866 0.871 0.871]    1.000     0.500    0.000     0.500   0.000      0.014    -0.014     0.014      0.000         1.645       -1.645        1.645
           cpc_embeddings         0.851 0.009   0.842   0.860 0.008  [0.858 0.859 0.847 0.851 0.840]    1.694     0.689   -4.402     0.001  -0.025      0.013    -0.037    -0.012     -2.801         1.467       -4.268       -1.334
           mles_embeddings        0.876 0.008   0.868   0.884 0.006  [0.884 0.881 0.869 0.874 0.873]    2.503     0.802    0.118     0.545   0.001      0.012    -0.011     0.013      0.070         1.376       -1.306        1.447
           nsp_embeddings         0.823 0.015   0.808   0.837 0.012  [0.828 0.834 0.812 0.832 0.808]    0.707     0.373   -7.682     0.000  -0.053      0.016    -0.069    -0.037     -6.022         1.808       -7.829       -4.214
           rtd_embeddings         0.851 0.007   0.844   0.858 0.006  [0.849 0.858 0.851 0.855 0.843]    2.906     0.837   -4.780     0.001  -0.024      0.012    -0.036    -0.013     -2.796         1.349       -4.145       -1.447
           sop_embeddings         0.748 0.016   0.732   0.763 0.013  [0.732 0.754 0.738 0.763 0.752]    0.620     0.327  -17.871     0.000  -0.128      0.016    -0.144    -0.111    -14.576         1.881      -16.457      -12.695
                           scores_test                                                                                                                                                                                             
                                  mean  t_pm t_int_l t_int_h   std                           values t_f_stat t_f_alpha t_t_stat t_t_alpha t_delta t_delta_pm t_delta_l t_delta_h t_delta_pp t_delta_pm_pp t_delta_l_pp t_delta_h_pp
model_name feature_name                                                                                                                                                                                                            
lgbm       baseline              0.875 0.003   0.872   0.878 0.003  [0.875 0.876 0.878 0.871 0.875]    1.000     0.500    0.000     0.500   0.000      0.004    -0.004     0.004      0.000         0.418       -0.418        0.418
           cpc_embeddings        0.845 0.003   0.842   0.848 0.002  [0.843 0.844 0.845 0.844 0.849]    1.155     0.554  -19.767     0.000  -0.030      0.004    -0.034    -0.027     -3.464         0.404       -3.869       -3.060
           mles_embeddings       0.890 0.004   0.886   0.894 0.003  [0.891 0.891 0.888 0.886 0.894]    0.654     0.345    8.115     1.000   0.014      0.004     0.010     0.019      1.656         0.471        1.185        2.126
           nsp_embeddings        0.837 0.005   0.832   0.841 0.004  [0.837 0.839 0.830 0.840 0.837]    0.459     0.234  -19.302     0.000  -0.039      0.005    -0.043    -0.034     -4.416         0.528       -4.944       -3.889
           rtd_embeddings        0.860 0.004   0.856   0.864 0.003  [0.857 0.859 0.858 0.861 0.865]    0.687     0.362   -8.696     0.000  -0.015      0.004    -0.019    -0.011     -1.748         0.464       -2.212       -1.285
           sop_embeddings        0.749 0.011   0.738   0.760 0.009  [0.741 0.738 0.760 0.753 0.754]    0.076     0.014  -29.776     0.000  -0.126      0.010    -0.136    -0.116    -14.413         1.116      -15.529      -13.297

------------------------------------------------------------------------------------------------------------------------
Metric: "accuracy"
                           scores_valid                                                                                                                                                                                             
                                   mean  t_pm t_int_l t_int_h   std                           values t_f_stat t_f_alpha t_t_stat t_t_alpha t_delta t_delta_pm t_delta_l t_delta_h t_delta_pp t_delta_pm_pp t_delta_l_pp t_delta_h_pp
model_name feature_name                                                                                                                                                                                                             
lgbm       baseline               0.792 0.014   0.778   0.806 0.011  [0.811 0.791 0.787 0.790 0.781]    1.000     0.500    0.000     0.500   0.000      0.016    -0.016     0.016      0.000         2.064       -2.064        2.064
           cpc_embeddings         0.767 0.010   0.756   0.777 0.008  [0.778 0.765 0.765 0.771 0.755]    1.782     0.705   -4.054     0.002  -0.025      0.014    -0.040    -0.011     -3.206         1.824       -5.030       -1.383
           mles_embeddings        0.793 0.012   0.781   0.805 0.009  [0.806 0.798 0.783 0.787 0.790]    1.462     0.639    0.142     0.555   0.001      0.015    -0.014     0.016      0.117         1.894       -1.777        2.011
           nsp_embeddings         0.747 0.020   0.728   0.767 0.016  [0.757 0.756 0.734 0.763 0.727]    0.493     0.255   -5.110     0.000  -0.045      0.020    -0.065    -0.024     -5.628         2.540       -8.168       -3.088
           rtd_embeddings         0.776 0.011   0.765   0.786 0.008  [0.772 0.779 0.771 0.788 0.767]    1.754     0.700   -2.632     0.015  -0.017      0.014    -0.031    -0.002     -2.088         1.829       -3.916       -0.259
           sop_embeddings         0.686 0.013   0.673   0.699 0.011  [0.680 0.685 0.679 0.704 0.683]    1.132     0.547  -15.420     0.000  -0.106      0.016    -0.122    -0.090    -13.393         2.003      -15.396      -11.391
                           scores_test                                                                                                                                                                                             
                                  mean  t_pm t_int_l t_int_h   std                           values t_f_stat t_f_alpha t_t_stat t_t_alpha t_delta t_delta_pm t_delta_l t_delta_h t_delta_pp t_delta_pm_pp t_delta_l_pp t_delta_h_pp
model_name feature_name                                                                                                                                                                                                            
lgbm       baseline              0.786 0.006   0.780   0.792 0.004  [0.790 0.782 0.790 0.781 0.786]    1.000     0.500    0.000     0.500   0.000      0.007    -0.007     0.007      0.000         0.832       -0.832        0.832
           cpc_embeddings        0.763 0.002   0.761   0.764 0.001  [0.764 0.763 0.763 0.761 0.762]   10.923     0.980  -11.132     0.000  -0.023      0.005    -0.028    -0.018     -2.969         0.615       -3.584       -2.354
           mles_embeddings       0.804 0.007   0.797   0.811 0.006  [0.795 0.805 0.804 0.805 0.811]    0.654     0.346    5.598     1.000   0.018      0.007     0.011     0.025      2.272         0.936        1.336        3.208
           nsp_embeddings        0.755 0.009   0.746   0.763 0.007  [0.762 0.751 0.745 0.755 0.761]    0.424     0.213   -8.483     0.000  -0.031      0.008    -0.040    -0.023     -3.968         1.079       -5.047       -2.890
           rtd_embeddings        0.766 0.010   0.756   0.775 0.008  [0.764 0.758 0.758 0.775 0.773]    0.329     0.153   -5.017     0.001  -0.020      0.009    -0.030    -0.011     -2.575         1.183       -3.758       -1.392
           sop_embeddings        0.688 0.010   0.678   0.698 0.008  [0.689 0.674 0.692 0.690 0.694]    0.311     0.142  -23.807     0.000  -0.098      0.010    -0.108    -0.089    -12.481         1.209      -13.690      -11.272

------------------------------------------------------------------------------------------------------------------------
End of report.     Current time: 2020-10-02 07:24:08
------------------------------------------------------------------------------------------------------------------------
