{SLmetrics} Version 0.3-0 🚀

[serkor1]

Note

See NEWS or commit history for detailed changes.

📚 What?

🚀 New features

This update introduces four new features. These are described below,

Cross-Entropy Loss (PR #34): Weighted and unweighted cross-entropy loss. The function can be used as follows,

# 1) define classes and
# observed classes (actual)
classes <- c("Class A", "Class B")

actual   <- factor(
  c("Class A", "Class B", "Class A"), 
  levels = classes

)

# 2) define probabilites
# and construct response_matrix
response <- c(
  0.2, 0.8, 
  0.8, 0.2, 
  0.7, 0.3
)

response_matrix <- matrix(
  response,
  nrow = 3,
  ncol = 2,
  byrow = TRUE
)

colnames(response_matrix) <- classes

response_matrix
#>      Class A Class B
#> [1,]     0.2     0.8
#> [2,]     0.8     0.2
#> [3,]     0.7     0.3

# 3) calculate entropy
SLmetrics::entropy(
  actual,
  response_matrix
)
#> [1] 1.19185

Relative Root Mean Squared Error (Commit 5521b5b):

The function normalizes the Root Mean Squared Error by a factor. There is no official way of normalizing it - and in {SLmetrics} the RMSE can be normalized using three options; mean-, range- and IQR-normalization. It can be used as follows,

# 1) define values
actual <- rnorm(1e3)
predicted <- actual + rnorm(1e3)

# 2) calculate Relative Root Mean Squared Error
cat(
  "Mean Relative Root Mean Squared Error", SLmetrics::rrmse(
    actual        = actual,
    predicted     = predicted,
    normalization = 0
  ),
  "Range Relative Root Mean Squared Error", SLmetrics::rrmse(
    actual        = actual,
    predicted     = predicted,
    normalization = 1
  ),
  "IQR Relative Root Mean Squared Error", SLmetrics::rrmse(
    actual        = actual,
    predicted     = predicted,
    normalization = 2
  ),
  sep = "\n"
)

#> Mean Relative Root Mean Squared Error
#> 2751.381
#> Range Relative Root Mean Squared Error
#> 0.1564043
#> IQR Relative Root Mean Squared Error
#> 0.7323898

Weighted Receiver Operator Characteristics and Precision-Recall Curves (PR #31):

These functions returns the weighted version of TPR, FPR and precision, recalll in weighted.ROC() and weighted.prROC() respectively. The weighted.ROC()-function¹ can be used as follows,

actual    <- factor(sample(c("Class 1", "Class 2"), size = 1e6, replace = TRUE, prob = c(0.7, 0.3)))
response  <- ifelse(actual == "Class 1", rbeta(sum(actual == "Class 1"), 2, 5), rbeta(sum(actual == "Class 2"), 5, 2))
w         <- ifelse(actual == "Class 1", runif(sum(actual == "Class 1"), 0.5, 1.5), runif(sum(actual == "Class 2"), 1, **2))

# Plot
plot(SLmetrics::weighted.ROC(actual, response, w))

⚠️ Breaking Changes

• Weighted Confusion Matix: The w-argument in cmatrix() has been
  removed in favor of the more verbose weighted confusion matrix call
  weighted.cmatrix()-function. See below,

Prior to version 0.3-0 the weighted confusion matrix were a part of
the cmatrix()-function and were called as follows,

SLmetrics::cmatrix(
    actual    = actual,
    predicted = predicted,
    w         = weights
)

This solution, although simple, were inconsistent with the remaining
implementation of weighted metrics in {SLmetrics}. To regain consistency
and simplicity the weighted confusion matrix are now retrieved as
follows,

# 1) define factors
actual    <- factor(sample(letters[1:3], 100, replace = TRUE))
predicted <- factor(sample(letters[1:3], 100, replace = TRUE))
weights   <- runif(length(actual))

# 2) without weights
SLmetrics::cmatrix(
    actual    = actual,
    predicted = predicted
)

#>    a  b  c
#> a  7  8 18
#> b  6 13 15
#> c 15 14  4

# 2) with weights
SLmetrics::weighted.cmatrix(
    actual    = actual,
    predicted = predicted,
    w         = weights
)

#>          a        b        c
#> a 3.627355 4.443065 7.164199
#> b 3.506631 5.426818 8.358687
#> c 6.615661 6.390454 2.233511

🐛 Bug-fixes

• Return named vectors: The classification metrics when
  micro == NULL were not returning named vectors. This has been fixed.

Footnotes

1. The syntax is the same for weighted.prROC() ↩