Calculates common error metrics of fitted binary and multi-categorical response models. Available measures include: the brier score, logloss and misclassification error.
erroR(model, type = c("brier", "logloss", "misclass"), thresh = 5e-1)a model object or data.frame of observed and predicted values.
The following class of objects can be directly passed to the erroR
function: glm(), vglm(), serp(), polr(), clm(), mlogit() and multinom(). Other
Other class of objects require providing a data.frame of observed and predicted
values.
specifies the type of error metrics
resets the default misclassification threshold
a numeric vector of the realized error value.
a character vector of error type.
a numeric vector of the misclassification threshold.
require(serp)
set.seed(1)
n <- 100
y <- factor(rbinom(n, 1, 0.3))
x <- rnorm(n)
#p <- runif(n)
m1 <- glm(y ~ x, family = binomial())
erroR(m1, type = "brier")
#>
#> Brier Score:
#> 0.2175822
erroR(m1, type = "logloss")
#>
#> LogLoss:
#> 0.6268293
erroR(m1, type = "misclass")
#>
#> Misclassification Error:
#> 0.32
erroR(m1, type = "misclass", thresh=0.3)
#>
#> Misclassification Error:
#> 0.68
# using data.frame
df <- data.frame(y, fitted(m1))
erroR(df, type = "brier")
#>
#> Brier Score:
#> 0.2175822
m2 <- serp(rating ~ temp + contact, slope = "parallel", link = "logit",
data = wine)
erroR(m2, type = "brier")
#>
#> Brier Score:
#> 0.6658793
erroR(m2, type = "logloss")
#>
#> LogLoss:
#> 1.201277
erroR(m2, type = "misclass")
#>
#> Misclassification Error:
#> 0.7083333