[BUG] Precision Loss 🤷‍♀️

[serkor1]

issue

All classification functions are passed through the cmatrix()-function. But for foo.cmatrix()-functions where a weighted confusion is passed, produces different results from weighted.foo()-functions.

reprex

# 1) generate vector
# for values
matrix_method <- weighted_method <- reference <- numeric()

# 2) simulate sample
# sizes
i <- 1
for (N in c(1e2, 1e3, 1e4, 1e4, 1e6)) {

  # 2.1) generate classes
  actual    <- factor(sample(letters[1:3], N, replace = TRUE))
  predicted <- factor(sample(letters[1:3], N, replace = TRUE))
  weights   <- runif(length(actual))

  # 2.2) generate accuracy
  # from each method
  matrix_method[i] <- SLmetrics::accuracy(
    SLmetrics::cmatrix(
      actual    = actual,
      predicted = predicted,
      w         = weights
    )
  )

  weighted_method[i] <- SLmetrics::weighted.accuracy(
    actual    = actual,
    predicted = predicted,
    w         = weights
  )
  
  reference[i] <- weighted.mean(
    x = as.numeric(actual) == as.numeric(predicted),
    w = weights
  )

  i <<- i + 1
}

# 3) display results
data.frame(
  reference,
  matrix_method,
  weighted_method
)
#>   reference matrix_method weighted_method
#> 1 0.3943053     0.3829787       0.3943053
#> 2 0.3465705     0.3486056       0.3465705
#> 3 0.3388643     0.3389222       0.3388643
#> 4 0.3350625     0.3349970       0.3350625
#> 5 0.3342705     0.3342708       0.3342705

^{Created on 2024-12-16 with reprex v2.1.1}

source of bug

The output does converge, which suggest that its a floating-point precision issue. This also explains why all functions suddenly failed unit-tests. One possible source is as follows,

/~https://github.com/serkor1/SLmetrics/blob/f86d934b03910e8e385128563afb490d448a2199/src/classification_Helpers.h#L28C1-L41C2

The matrix is getting passed as IntegerMatrix and converted to Eigen::MatrixXd.

fix

One possibility is to 🤷, another is to 😱 or change the following code:

/~https://github.com/serkor1/SLmetrics/blob/f86d934b03910e8e385128563afb490d448a2199/src/classification_Helpers.h#L28C1-L41C2

To

// 1) matrix template
// without micro-agument
template <typename... Args>
Rcpp::NumericVector classification_base(
    const Rcpp::NumericMatrix& matrix,
    const classification& foo,
    Args&&... args)
{
    // 0) Convert matrix to Eigen format
    Eigen::MatrixXd eigen_matrix = Rcpp::as<Eigen::MatrixXd>(matrix);


    // 1) Forward the additional arguments to foo.compute
    return foo.compute(eigen_matrix, std::forward<Args>(args)...);
}

This goes for all .cpp-files too! 😥

[serkor1]

Fixed

# 1) generate vector
# for values
matrix_method <- weighted_method <- reference <- numeric()

# 2) simulate sample
# sizes
i <- 1
for (N in c(1e2, 1e3, 1e4, 1e4, 1e6)) {

  # 2.1) generate classes
  actual    <- factor(sample(letters[1:3], N, replace = TRUE))
  predicted <- factor(sample(letters[1:3], N, replace = TRUE))
  weights   <- runif(length(actual))

  # 2.2) generate accuracy
  # from each method
  matrix_method[i] <- SLmetrics::accuracy(
    SLmetrics::cmatrix(
      actual    = actual,
      predicted = predicted,
      w         = weights
    )
  )

  weighted_method[i] <- SLmetrics::weighted.accuracy(
    actual    = actual,
    predicted = predicted,
    w         = weights
  )
  
  reference[i] <- weighted.mean(
    x = as.numeric(actual) == as.numeric(predicted),
    w = weights
  )

  i <<- i + 1
}

# 3) display results
data.frame(
  reference,
  matrix_method,
  weighted_method
)
#>   reference matrix_method weighted_method
#> 1 0.3695044     0.3695044       0.3695044
#> 2 0.3241264     0.3241264       0.3241264
#> 3 0.3379269     0.3379269       0.3379269
#> 4 0.3362808     0.3362808       0.3362808
#> 5 0.3327261     0.3327261       0.3327261

^{Created on 2024-12-16 with reprex v2.1.1}

Changing IntegerMatrix to NumericMatrix fixed it!