Skip to contents

Benchmark soilKey classifiers against BDsolos national reference labels

Source: R/benchmark-bdsolos.R
benchmark_bdsolos.Rd

Runs classify_wrb2022, classify_sibcs, and classify_usda on each PedonRecord loaded from a BDsolos CSV via load_bdsolos_csv, then compares each predicted classification against the corresponding BDsolos reference label (reference_sibcs, reference_wrb, reference_st) and reports per-system accuracy, per-class recall, and a confusion matrix.

Usage

benchmark_bdsolos(
  pedons,
  systems = c("wrb2022", "sibcs", "usda"),
  sibcs_level = c("order", "subordem"),
  max_n = NULL,
  verbose = TRUE,
  sampling_type = "Perfil completo"
)

Arguments

pedons

A list of PedonRecord objects, typically produced by load_bdsolos_csv.

systems

Character vector. Any subset of c("wrb2022", "sibcs", "usda"). Default runs all three.

sibcs_level

One of "order" (default) or "subordem". Forwarded to normalise_febr_sibcs.

max_n

Optional integer; cap classification at the first max_n pedons. NULL (default) classifies every pedon.

verbose

If TRUE (default), emits cli progress messages.

sampling_type

The kinds of sampling point the figures of each system are computed on, as site$sampling_type words them. Default "Perfil completo". NULL counts every record, as the function did until v0.9.241. Ignored when no pedon carries the field.

Value

A list with elements:

  • per_system – named list (one entry per requested system) of list(accuracy, n_compared, n_correct, n_errors, confusion, per_class, by_sampling_type, sampling_type) (or list(accuracy = NA_real_, message) when no reference labels were present). by_sampling_type is a data frame with one row per kind of point and one for all the records (n_compared, n_correct, accuracy), or NULL when no pedon carries the field; sampling_type is the kinds the main figures were computed on (NULL for all).

  • coverage – named list of list(n_with_ref, n_total, pct) per system.

  • config – named list capturing n_pedons, systems, sibcs_level, soilKey_version, timestamp.

Reference label coverage

BDsolos densely populates reference_sibcs (~82 of the v0.9.59 audit) but sparsely populates reference_wrb and reference_st (UF-dependent; ~5 states). The function always reports the per-system label coverage ($coverage) so the caller can judge how representative each accuracy figure is.

Comparison level

SiBCS comparison is at level = "order" by default, which converts the BDsolos all-caps Portuguese label (e.g. "ARGISSOLO VERMELHO Tb EUTROFICO ...") to the soilKey plural Title Case form ("Argissolos") via normalise_febr_sibcs. Set sibcs_level = "subordem" to compare the first two SiBCS tokens (Ordem + Subordem).

WRB and USDA comparisons are at the Reference Soil Group / Order level: normalise_febr_wrb() strips qualifier parens and pluralises the bare RSG ("Xanthic Ferralsol" -> "Ferralsols"); normalise_febr_usda() maps the suffix of the last subgroup token to the USDA Order ("Typic Haplorthox" -> "Oxisols").

Errors and missing-label handling

Pedons without a reference label for a given system are silently excluded from THAT system's comparison (but still classified for the other two systems). If a system has zero pedons with a reference label, the corresponding $per_system entry has accuracy = NA_real_ and message = "no_reference_labels". Classifier errors are caught per-pedon and recorded in n_errors; they do not abort the run.

Kinds of sampling point

The export marks each record as "Perfil completo", "Perfil parcial" or "Amostra extra" (site$sampling_type, read by load_bdsolos_csv since v0.9.242). An extra sample is not a described profile: in most cases it is two sampled layers, and the thickness a diagnostic horizon is asked for cannot be read from it. The figures of each system (accuracy, n_compared, n_correct, confusion, per_class) are therefore those of the complete profiles by default, and by_sampling_type gives the same counts for every kind and for all the records together. Pedons that carry no site$sampling_type (another source, a hand-made list) are all counted, as before.

See also

load_bdsolos_csv, benchmark_lucas_2018, classify_all, normalise_febr_sibcs, normalise_febr_wrb, normalise_febr_usda.

Examples

if (FALSE) { # \dontrun{
# Single UF -- typical SiBCS-dense slice
peds <- load_bdsolos_csv("RJ.csv")
bench <- benchmark_bdsolos(peds, systems = c("sibcs", "wrb2022", "usda"))
bench$coverage      # how many pedons had each reference label
bench$per_system$sibcs$accuracy
bench$per_system$sibcs$confusion

# Subordem level
bench2 <- benchmark_bdsolos(peds, systems = "sibcs",
                               sibcs_level = "subordem")
} # }