Package 'OptimalBinningWoE'

Categorical-Only Algorithms

Description

Internal function returning algorithm identifiers that support only categorical features.

Usage

.categorical_only_algorithms()

Value

A character vector of categorical-only algorithm names.

---

Numerical-Only Algorithms

Description

Internal function returning algorithm identifiers that support only numerical features.

Usage

.numerical_only_algorithms()

Value

A character vector of numerical-only algorithm names.

---

Universal Algorithms

Description

Internal function returning algorithm identifiers that support both numerical and categorical features.

Usage

.universal_algorithms()

Value

A character vector of universal algorithm names.

---

Valid Binning Algorithms

Description

Internal function returning the vector of all valid algorithm identifiers supported by the OptimalBinningWoE package. Used for validation and parameter definition.

Usage

.valid_algorithms()

Value

A character vector of valid algorithm names including "auto".

---

Apply the Optimal Binning Transformation

Description

Applies the learned binning and WoE transformation to new data. This method is called by bake and should not be invoked directly.

Usage

## S3 method for class 'step_obwoe'
bake(object, new_data, ...)

Arguments

object	A trained step_obwoe object.
new_data	A tibble or data frame to transform.
...	Additional arguments (currently unused).

Value

A tibble with transformed columns according to the output parameter.

---

Control Parameters for Optimal Binning Algorithms

Description

Constructs a validated list of control parameters for the obwoe master interface. These parameters govern the behavior of all supported binning algorithms, including convergence criteria, minimum bin sizes, and optimization limits.

Usage

control.obwoe(
  bin_cutoff = 0.05,
  max_n_prebins = 20,
  convergence_threshold = 1e-06,
  max_iterations = 1000,
  bin_separator = "%;%",
  verbose = FALSE,
  ...
)

Arguments

bin_cutoff	Numeric value in $(0, 1)$ specifying the minimum proportion of total observations that a bin must contain. Bins with fewer observations are merged with adjacent bins. Serves as a regularization mechanism to prevent overfitting and ensure statistical stability of WoE estimates. Recommended range: 0.02 to 0.10 for numerical features. For categorical features with many levels the default can cost predictive power: compare lower values on held-out data (see Details). Default is 0.05 (5%).
max_n_prebins	Integer specifying the maximum number of initial bins created before optimization. For high-cardinality categorical features, categories with similar event rates are pre-merged until this limit is reached. Higher values preserve more granularity but increase computational cost. Typical range: 10 to 50. Default is 20.
convergence_threshold	Numeric value specifying the tolerance for algorithm convergence. Iteration stops when the absolute change in Information Value between successive iterations falls below this threshold: $|IV_{t} - IV_{t-1}| < \epsilon$. Smaller values yield more precise solutions at higher computational cost. Typical range: $10^{-4}$ to $10^{-8}$. Default is $10^{-6}$.
max_iterations	Integer specifying the maximum number of optimization iterations. Prevents infinite loops in degenerate cases. If the algorithm does not converge within this limit, it returns the best solution found. Typical range: 100 to 10000. Default is 1000.
bin_separator	Character string used to concatenate category names when multiple categories are merged into a single bin. Should be a string unlikely to appear in actual category names. Default is "%;%".
verbose	Logical indicating whether to print progress messages during feature processing. Useful for debugging or monitoring long-running jobs. Default is FALSE.
...	Additional named parameters reserved for algorithm-specific extensions. Currently unused but included for forward compatibility.

Details

Parameter Impact on Results

bin_cutoff: Lower values allow smaller bins, which may capture subtle patterns but risk unstable WoE estimates. The variance of WoE estimates increases as $1/n_i$ where $n_i$ is the bin size. For bins with fewer than ~30 observations, consider using Laplace or Bayesian smoothing (applied automatically by most algorithms).

bin_cutoff and high-cardinality categorical features: the default of 0.05 is not neutral for a categorical feature with many levels. In jedi, the default engine, levels that hold less than bin_cutoff of the rows are pooled into one bin before the optimization, whatever their event rates (the min_bins most frequent rare levels keep bins of their own). Other engines treat rare levels in their own ways, for example by merging a rare level with the level of closest event rate. The pooled bin can be very heterogeneous, and it costs IV, in-sample as well as on new data.

In a census-income case study (UCI Census-Income (KDD) data, doi:10.24432/C5N30T; target income above 50,000, 143,531 adults used to fit, IV measured on the held-out file, jedi with min_bins = 2 and max_bins = 6), 11 of the 16 education levels were pooled into one bin, putting "Less than 1st grade" (0.1% events) together with "Doctorate degree" (52%). Held-out IV for education was 0.67 at bin_cutoff = 0.05 and 1.27 at 0.005; for the 47-level detailed occupation it was 0.97 and 1.64.

The default is unchanged because another cutoff moves fitted cut points. For a categorical feature with many levels, fit at a few cutoffs (for example 0.05, 0.02 and 0.005) and compare the IV on held-out data. In-sample IV cannot warn you off a cutoff that is too low, because smaller bins never lower it. Stop lowering the cutoff once the held-out IV no longer improves: bins of fewer than ~30 observations give unstable WoE estimates.

max_n_prebins: Critical for categorical features with many levels. If a feature has 100 categories, setting max_n_prebins = 20 will pre-merge similar categories into 20 groups before optimization.

For numerical features this parameter is a modelling decision, not an implementation detail, and the default of 20 is not neutral. Pre-binning runs before any algorithm sees the data, so a heavy-tailed variable whose signal sits in the upper tail can have that tail smeared across a single quantile cell and lost before optimization begins. The effect is silent: the fit reports converged = TRUE and no warning.

Benchmarks on two open datasets (76,020 x 369 and 590,540 x 454, five-fold held-out IV, min_bins and max_bins held fixed) show the size of it. Raising max_n_prebins from 20 to 200 moved the median held-out IV of heavy-tailed numerical predictors by +129% and +39% respectively, while leaving discrete and categorical predictors untouched. The bin count barely moved, so this is the placement of cut points changing, not extra splits.

It moves results in both directions, which is why the default is left alone. On one of the two benchmarks, raising it improved the portfolio but cost 15% of held-out IV on the twenty-five strongest predictors – exactly the ones a scorecard would keep – while inflating IV on weak predictors no model would select. No cheap rule separated the two cases: gating on the tail ratio worked cleanly on one dataset and captured almost nothing on the other, and for roughly half of all predictors the default of 20 was already the best value tested.

The practical advice is therefore per-variable rather than global: leave the default, and tune it against held-out data for continuous predictors with a long tail. Tuning on in-sample IV will always favour more resolution and is not evidence of anything.

convergence_threshold: Trade-off between precision and speed. For exploratory analysis, $10^{-4}$ is sufficient. For production models requiring reproducibility, use $10^{-8}$ or smaller.

Value

An S3 object of class "obwoe_control" containing all specified parameters. This object is validated and can be passed directly to obwoe.

See Also

obwoe for the main binning interface.

Examples

# Default control parameters
ctrl_default <- control.obwoe()
print(ctrl_default)

# Conservative settings for production
ctrl_production <- control.obwoe(
  bin_cutoff = 0.03,
  max_n_prebins = 30,
  convergence_threshold = 1e-8,
  max_iterations = 5000
)

# Aggressive settings for exploration
ctrl_explore <- control.obwoe(
  bin_cutoff = 0.01,
  max_n_prebins = 50,
  convergence_threshold = 1e-4,
  max_iterations = 500
)

---

Control Parameters for the Scorecard Pipeline

Description

Settings for the stages of obwoe_scorecard that have no home in control.obwoe or obwoe_select.

Usage

control.obwoe_scorecard(
  corr_cutoff = 0.7,
  corr_method = "pearson",
  pdo = 20,
  score_ref = 600,
  odds_ref = 50,
  direction = "higher_is_safer",
  n_groups = 10L,
  na_woe = 0,
  drop_negative = TRUE,
  max_abs_coef = 15,
  min_events_per_variable = 20,
  engine_fallback = FALSE,
  overwrite = TRUE,
  digits = 6L
)

Arguments

corr_cutoff	Numeric in $(0, 1]$. Absolute correlation in the WoE space at or above which two variables are treated as redundant. Default 0.70. Set to 1 to disable pruning.
corr_method	Character string passed to obcorr. Default "pearson".
pdo, score_ref, odds_ref, direction	Passed to obwoe_scale.
n_groups	Integer. Number of score bands in the gains and stability tables. Default 10.
na_woe	Numeric. WoE given to a value that falls in no fitted bin — an unseen category, or a missing value where the binner built no missing bin. Default 0, which is the population-average contribution.
drop_negative	Logical. Refit without any variable whose coefficient on WoE comes out negative? Default TRUE. A negative coefficient means the variable is fighting the rest of the model, and a scorecard with one cannot be signed off.
max_abs_coef	Numeric. Coefficients above this in absolute value are flagged as separation. Default 15.
min_events_per_variable	Numeric. Events per variable in the model below which the fit is flagged as thin. Default 20, the usual rule of thumb.
engine_fallback	Logical. If the requested engine's package is not installed, fall back to "glm" with a warning instead of failing? Default FALSE.
overwrite	Logical. Overwrite file if it exists? Default TRUE.
digits	Integer. Rounding used in the workbook. Default 6.

Value

An object of class "obwoe_scorecard_control".

See Also

obwoe_scorecard

Examples

control.obwoe_scorecard(pdo = 25, score_ref = 700, corr_cutoff = 0.8)

---

Apply Optimal Weight of Evidence (WoE) to a Categorical Feature

Description

Transforms a categorical feature into its corresponding Weight of Evidence (WoE) values using pre-computed binning results from an optimal binning algorithm (e.g., ob_categorical_cm).

Usage

ob_apply_woe_cat(
  obresults,
  feature,
  bin_separator = "%;%",
  missing_values = c("NA", "Missing", "")
)

Arguments

obresults	List output from an optimal binning function for categorical variables. Must contain elements bin (character vector of bin labels) and woe (numeric vector of WoE values). Bins may represent individual categories or merged groups separated by bin_separator.
feature	Character or factor vector of categorical values to be transformed. Automatically coerced to character if provided as factor.
bin_separator	Character string used to separate multiple categories within a single bin label (default: "%;%"). For example, a bin "A%;%B%;%C" contains categories A, B, and C.
missing_values	Character vector specifying which values should be treated as missing (default: c("NA", "Missing", "")). NA inputs and these values (after trimming surrounding whitespace) are scored with the missing-value bin: the first bin of obresults whose categories include one of them (the categorical binners store NA as the category "NA"), which is also the bin obwoe_sql and obwoe_apply use for missing values. A value that is itself a category of a bin (e.g. "NA") is scored with that bin. Without a missing-value bin they are scored as "Special" with WoE NA.

Details

This function is typically used in a two-step workflow:

1. Train binning on training data: bins <- ob_categorical_cm(feature_train, target_train)

2. Apply WoE to new data: woe_test <- ob_apply_woe_cat(bins, feature_test)

Bin labels are split on bin_separator and each value of feature is matched against the resulting categories exactly, byte for byte, as obwoe_apply and obwoe_sql do. A category that appears in more than one bin label is scored with the first of those bins, as in the CASE expression generated by obwoe_sql. A value that only matches once leading and trailing whitespace is removed is still accepted, and a category whose own name contains bin_separator is recognised as a run of consecutive components of a bin label. Each distinct value is resolved once, so scoring takes $O(n)$ hash lookups whatever the number of categories.

Value

A data frame with one row per element of feature and the columns feature (the input), bin (the label of the matched bin, or "Special"), woe (the bin's WoE; NA for "Special"), idbin (the bin's id; the number of bins plus one for "Special") and ismissing (1 for missing values and for categories not seen in obresults, 0 otherwise). Unseen categories are scored as "Special" without a warning.

Examples

# Mock data
train_data <- data.frame(
  category = c("A", "B", "A", "C", "B", "A"),
  default = c(0, 1, 0, 1, 0, 0)
)
test_data <- data.frame(
  category = c("A", "C", "B")
)

# Train binning on training set
train_bins <- ob_categorical_cm(
  feature = train_data$category,
  target = train_data$default
)

# Apply to test set
test_woe <- ob_apply_woe_cat(
  obresults = train_bins,
  feature = test_data$category
)

# Handle custom missing indicators
test_woe <- ob_apply_woe_cat(
  obresults = train_bins,
  feature = test_data$category,
  missing_values = c("NA", "Unknown", "N/A", "")
)

---

Apply Optimal Weight of Evidence (WoE) to a Numerical Feature

Description

Transforms a numerical feature into its corresponding Weight of Evidence (WoE) values using pre-computed binning results from an optimal binning algorithm (e.g., ob_numerical_mdlp, ob_numerical_mob).

Usage

ob_apply_woe_num(
  obresults,
  feature,
  include_upper_bound = TRUE,
  missing_values = c(-999)
)

Arguments

obresults	List output from an optimal binning function for numerical variables. Must contain elements cutpoints (numeric vector of bin boundaries) and woe (numeric vector of WoE values). The number of WoE values should equal length(cutpoints) + 1.
feature	Numeric vector of values to be transformed. Automatically coerced to numeric if provided in another type.
include_upper_bound	Logical flag controlling interval boundary behavior (default: TRUE): TRUE: Intervals are (lower, upper] (right-closed). FALSE: Intervals are [lower, upper) (left-closed). This must match the convention used during binning.
missing_values	Numeric vector of values to be treated as missing (default: c(-999)). These values are assigned the WoE of the special missing bin if it exists in obresults, or NA otherwise.

Details

This function is typically used in a two-step workflow:

1. Train binning on training data: bins <- ob_numerical_mdlp(feature_train, target_train)

2. Apply WoE to new data: woe_test <- ob_apply_woe_num(bins, feature_test)

Bin Assignment Logic: For k cutpoints $c_1 < c_2 < \cdots < c_k$, values are assigned as:

• Bin 1: $x \le c_1$ (if include_upper_bound = TRUE)

• Bin i: $c_{i-1} < x \le c_i$ for $i = 2, \ldots, k$

• Bin k+1: $x > c_k$

Handling of Edge Cases:

• obresults$woe and obresults$id may carry length(cutpoints) + 2 entries when the fit has a dedicated missing-value bin (e.g. ob_numerical_udt on data with NA): that bin is the one labelled "NA" or "Missing" in obresults$bin (the last one when no label says so), and missing values are scored with it.

• With no cut points at all (a single bin) every non-missing value gets the WoE of that bin.

• Each value is placed with one binary search over the cut points, $O(n \log k)$ in total.

• Inf and -Inf are assigned to the last and first bins, respectively.

• Values exactly equal to cutpoints follow the include_upper_bound convention.

Value

A data frame with one row per element of feature and the columns feature, bin (interval label, or "Special"), woe, idbin and ismissing. Values outside the range of cutpoints are assigned to the first or last bin. NA/NaN values and values listed in missing_values get ismissing = 1 and the WoE of the missing-value bin when obresults has one (see Details), or bin = "Special" and woe = NA otherwise.

See Also

ob_numerical_mdlp for MDLP binning, ob_numerical_mob for monotonic binning, ob_apply_woe_cat for applying WoE to categorical features.

Examples

# Mock data
train_data <- data.frame(
  income = c(50000, 75000, 30000, 45000, 80000, 60000),
  default = c(0, 0, 1, 1, 0, 0)
)
test_data <- data.frame(
  income = c(55000, 35000, 90000)
)

# Train binning on training set
train_bins <- ob_numerical_mdlp(
  feature = train_data$income,
  target = train_data$default
)

# Apply to test set
test_woe <- ob_apply_woe_num(
  obresults = train_bins,
  feature = test_data$income
)

# Handle custom missing indicators (e.g., -999, -1)
test_woe <- ob_apply_woe_num(
  obresults = train_bins,
  feature = test_data$income,
  missing_values = c(-999, -1, -9999)
)

# Use left-closed intervals (match scikit-learn convention)
test_woe <- ob_apply_woe_num(
  obresults = train_bins,
  feature = test_data$income,
  include_upper_bound = FALSE
)

---

Optimal Binning for Categorical Variables using Enhanced ChiMerge Algorithm

Description

Performs supervised discretization of categorical variables using an enhanced implementation of the ChiMerge algorithm (Kerber, 1992) with optional Chi2 extension (Liu & Setiono, 1995). This method optimally groups categorical levels based on their relationship with a binary target variable to maximize predictive power while maintaining statistical significance.

Usage

ob_categorical_cm(
  feature,
  target,
  min_bins = 3,
  max_bins = 5,
  bin_cutoff = 0.05,
  max_n_prebins = 20,
  bin_separator = "%;%",
  convergence_threshold = 1e-06,
  max_iterations = 1000,
  chi_merge_threshold = 0.05,
  use_chi2_algorithm = FALSE
)

Arguments

feature	A character vector or factor representing the categorical predictor variable to be binned.
target	An integer vector of binary outcomes (0/1) corresponding to each observation in feature.
min_bins	Integer. Minimum number of bins to produce. Must be >= 2. Defaults to 3.
max_bins	Integer. Maximum number of bins to produce. Must be >= min_bins. Defaults to 5.
bin_cutoff	Numeric. Threshold for treating categories as rare. Categories with frequency < bin_cutoff will be merged with their most similar neighbors. Value must be in (0, 1). Defaults to 0.05.
max_n_prebins	Integer. Maximum number of initial pre-bins before merging. Controls computational complexity. Must be >= 2. Defaults to 20.
bin_separator	String. Separator used when combining multiple categories into a single bin label. Defaults to "%;%". A warning is issued when a category name contains it, since such labels cannot be split back into categories.
convergence_threshold	Numeric. Convergence tolerance for iterative merging process. Smaller values require stricter convergence. Must be > 0. Defaults to 1e-6.
max_iterations	Integer. Maximum iterations for the merging algorithm. Prevents infinite loops. Must be > 0. Defaults to 1000.
chi_merge_threshold	Numeric. Significance level $\alpha$ of the chi-square test used as the ChiMerge stopping rule: adjacent bins keep being merged while their chi-square statistic (1 degree of freedom, Yates' continuity correction) is below the critical value $\chi^2_{1,1-\alpha}$ (3.841 for $\alpha = 0.05$), i.e. while they do not differ significantly. Lower values merge more and create fewer bins. Value must be in (0, 1). Defaults to 0.05.
use_chi2_algorithm	Logical. If TRUE, uses the Chi2 variant which performs multi-pass merging with decreasing significance thresholds. Defaults to FALSE.

Details

The algorithm implements two main approaches:

1. Standard ChiMerge: Iteratively merges adjacent bins with lowest chi-square statistics until all remaining pairs are statistically distinguishable at the specified significance level.

2. Chi2 Algorithm (when use_chi2_algorithm = TRUE): Performs multiple passes with decreasing significance thresholds (0.5 → 0.001), creating more robust binning structures particularly for noisy data.

Key features include:

• Rare category handling through pre-merging

• Monotonicity enforcement of Weight of Evidence

• Numerical stability with underflow protection

• Efficient chi-square caching for performance

• Comprehensive input validation and error handling

Information Value interpretation:

• < 0.02: Predictive power not useful

• 0.02-0.1: Weak predictive power

• 0.1-0.3: Medium predictive power

• 0.3-0.5: Strong predictive power

• > 0.5: Suspiciously high (potential overfitting)

Value

A list containing binning results with the following components:

• id: Integer vector of bin identifiers (1:n_bins)

• bin: Character vector of bin labels (merged category names)

• woe: Numeric vector of Weight of Evidence for each bin

• iv: Numeric vector of Information Value contribution per bin

• count: Integer vector of total observations per bin

• count_pos: Integer vector of positive cases per bin

• count_neg: Integer vector of negative cases per bin

• converged: Logical indicating if algorithm converged

• iterations: Integer count of algorithm iterations performed

• algorithm: Character string identifying algorithm used

• warnings: Character vector of any warnings encountered

• metadata: List with additional diagnostic information:

  • total_iv: Total Information Value of the binned variable

  • n_bins: Final number of bins produced

  • unique_categories: Number of unique input categories

  • total_obs: Total number of observations processed

  • execution_time_ms: Processing time in milliseconds

  • monotonic: Direction of WoE monotonicity ("increasing"/"decreasing")

Author(s)

Developed as part of the OptimalBinningWoE package

References

Kerber, R. (1992). ChiMerge: Discretization of numeric attributes. In Proceedings of the Tenth National Conference on Artificial Intelligence (pp. 123-128).

Liu, B., & Setiono, R. (1995). Chi2: Feature selection and discretization of numeric attributes. In Proceedings of the Seventh IEEE International Conference on Tools with Artificial Intelligence (pp. 372-377).

Examples

# Example 1: Basic usage with synthetic data
set.seed(123)
n <- 1000
categories <- c("A", "B", "C", "D", "E", "F", "G", "H")
feature <- sample(categories, n, replace = TRUE, prob = c(
  0.2, 0.15, 0.15,
  0.1, 0.1, 0.1,
  0.1, 0.1
))
# Create target with some association to categories
probs <- c(0.3, 0.4, 0.5, 0.6, 0.7, 0.75, 0.8, 0.85) # increasing probability
target <- sapply(seq_along(feature), function(i) {
  cat_idx <- which(categories == feature[i])
  rbinom(1, 1, probs[cat_idx])
})

result <- ob_categorical_cm(feature, target)
print(result[c("bin", "woe", "iv", "count")])

# View metadata
print(paste("Total IV:", round(result$metadata$total_iv, 3)))
print(paste("Algorithm converged:", result$converged))

# Example 2: Using Chi2 algorithm for more conservative binning
result_chi2 <- ob_categorical_cm(feature, target,
  use_chi2_algorithm = TRUE,
  max_bins = 6
)

# Compare number of bins
cat("Standard ChiMerge bins:", result$metadata$n_bins, "\n")
cat("Chi2 algorithm bins:", result_chi2$metadata$n_bins, "\n")

---

Optimal Binning for Categorical Variables using Divergence Measures

Description

Performs supervised discretization of categorical variables using a divergence-based hierarchical merging algorithm. This implementation supports multiple information-theoretic and metric divergence measures as described by Zeng (2013), enabling flexible optimization of binning structures for credit scoring and binary classification tasks.

Usage

ob_categorical_dmiv(
  feature,
  target,
  min_bins = 3,
  max_bins = 5,
  bin_cutoff = 0.05,
  max_n_prebins = 20,
  bin_separator = "%;%",
  convergence_threshold = 1e-06,
  max_iterations = 1000,
  bin_method = "woe1",
  divergence_method = "l2"
)

Arguments

feature	A character vector or factor representing the categorical predictor variable to be binned. Missing values are automatically converted to the category "NA".
target	An integer vector of binary outcomes (0/1) corresponding to each observation in feature. Missing values are not permitted.
min_bins	Integer. Minimum number of bins to produce. Must be >= 2. If the final number of bins after merging falls below this threshold, the algorithm will attempt to split bins. Defaults to 3.
max_bins	Integer. Maximum number of bins to produce. Must be >= min_bins. The algorithm performs hierarchical merging until this constraint is satisfied. Defaults to 5.
bin_cutoff	Numeric. Frequency threshold for rare category handling. Categories with relative frequency below this value are candidates for pre-binning. Must be in (0, 1). Defaults to 0.05.
max_n_prebins	Integer. Maximum number of initial bins before the main merging phase. When unique categories exceed this limit, rare categories are pre-merged into an "other" bin. Must be >= 2. Defaults to 20.
bin_separator	Character string used to concatenate category names when multiple categories are merged into a single bin. Defaults to "%;%". A warning is issued when a category name contains it, since such labels cannot be split back into categories.
convergence_threshold	Numeric. Convergence tolerance for the iterative merging process. When the change in minimum divergence between iterations falls below this threshold, the fit is recorded as converged. It does not stop the merging: max_bins is a hard constraint, so merging continues by the same criterion until the bin count meets it. Must be > 0. Defaults to 1e-6.
max_iterations	Integer. Iteration budget for the merge phase. Must be > 0. Defaults to 1000. When it is used up before the convergence tolerance is met, converged is FALSE; merging still continues until max_bins is satisfied, since max_bins is a hard constraint.
bin_method	Character string specifying the Weight of Evidence calculation method. Must be one of: "woe" Traditional WoE: $\ln\left(\frac{p_i/P}{n_i/N}\right)$ "woe1" Smoothed WoE (Zeng): $\ln\left(\frac{g_i + 0.5}{b_i + 0.5}\right)$ The smoothed variant provides numerical stability for sparse bins. Defaults to "woe1".
divergence_method	Character string specifying the divergence measure used for determining bin similarity. Must be one of: "he" Hellinger Distance: $\sum(\sqrt{p_i} - \sqrt{n_i})^2$ "kl" Symmetrized Kullback-Leibler Divergence "klj" Jeffreys J-Divergence: $(p-n)\ln(p/n)$ "tr" Triangular Discrimination: $(p-n)^2/(p+n)$ "sc" Symmetric Chi-Square: $(p-n)^2(p+n)/(pn)$ "js" Jensen-Shannon Divergence "l1" L1 Metric (Manhattan Distance): $|p-n|$ "l2" L2 Metric (Euclidean Distance): $\sqrt{\sum(p-n)^2}$ "ln" L-infinity Metric (Chebyshev Distance): $\max|p-n|$ Defaults to "l2".

Details

The algorithm implements a hierarchical agglomerative approach where bins are iteratively merged based on minimum pairwise divergence until the max_bins constraint is satisfied or convergence is achieved.

Algorithm Workflow:

1. Input validation and frequency computation

2. Pre-binning of rare categories (if unique categories > max_n_prebins)

3. Initialization of pairwise divergence matrix

4. Iterative merging of most similar bin pairs

5. Splitting of heterogeneous bins (if bins < min_bins)

6. Final metric computation and WoE-based sorting

Divergence Measure Selection: The choice of divergence measure affects the binning structure:

• Information-theoretic measures ("kl", "js", "klj"): Emphasize distributional differences; sensitive to rare events

• Metric measures ("l1", "l2", "ln"): Provide geometric interpretation; robust to outliers

• Chi-square family ("sc", "tr"): Balance between information content and robustness

• Hellinger distance ("he"): Bounded measure; suitable for probability distributions

Pre-binning Strategy: When the number of unique categories exceeds max_n_prebins, categories with fewer than 5 observations are pooled into one bin to control computational complexity. The pooled bin is labelled with the categories it contains, like any other bin, so it can be mapped when the binning is applied. Pre-binning is skipped when it would leave fewer than min_bins bins.

Value

A list containing the binning results with the following components:

id

Integer vector of bin identifiers (1-indexed)

bin

Character vector of bin labels (merged category names)

woe

Numeric vector of Weight of Evidence values per bin

divergence

Numeric vector of divergence contribution per bin

count

Integer vector of total observations per bin

count_pos

Integer vector of positive cases (target=1) per bin

count_neg

Integer vector of negative cases (target=0) per bin

converged

Logical indicating algorithm convergence

iterations

Integer count of merge operations performed

total_divergence

Numeric total divergence of the binning solution

bin_method

Character string of WoE method used

divergence_method

Character string of divergence measure used

References

Zeng, G. (2013). Metric Divergence Measures and Information Value in Credit Scoring. Journal of Mathematics, 2013, Article ID 848271. doi:10.1155/2013/848271

Kullback, S., & Leibler, R. A. (1951). On Information and Sufficiency. The Annals of Mathematical Statistics, 22(1), 79-86.

Lin, J. (1991). Divergence Measures Based on the Shannon Entropy. IEEE Transactions on Information Theory, 37(1), 145-151.

See Also

ob_categorical_cm for ChiMerge-based categorical binning

Examples

# Example 1: Basic usage with synthetic credit data
set.seed(42)
n <- 1000

# Simulate occupation categories with varying default rates
occupations <- c(
  "Engineer", "Doctor", "Teacher", "Sales",
  "Manager", "Clerk", "Other"
)
default_probs <- c(0.05, 0.03, 0.08, 0.15, 0.07, 0.12, 0.20)

feature <- sample(occupations, n,
  replace = TRUE,
  prob = c(0.15, 0.10, 0.20, 0.18, 0.12, 0.15, 0.10)
)
target <- sapply(feature, function(x) {
  rbinom(1, 1, default_probs[which(occupations == x)])
})

# Apply optimal binning with L2 divergence
result <- ob_categorical_dmiv(feature, target,
  min_bins = 2,
  max_bins = 4,
  divergence_method = "l2"
)

# Examine binning results
print(data.frame(
  bin = result$bin,
  woe = round(result$woe, 3),
  count = result$count,
  event_rate = round(result$count_pos / result$count, 3)
))

# Example 2: Comparing divergence methods
result_js <- ob_categorical_dmiv(feature, target,
  divergence_method = "js",
  max_bins = 4
)
result_kl <- ob_categorical_dmiv(feature, target,
  divergence_method = "kl",
  max_bins = 4
)

cat("Jensen-Shannon bins:", length(result_js$bin), "\n")
cat("Kullback-Leibler bins:", length(result_kl$bin), "\n")

# Example 3: High cardinality feature with pre-binning
set.seed(123)
postal_codes <- paste0("ZIP_", sprintf("%03d", 1:50))
feature_high_card <- sample(postal_codes, 2000, replace = TRUE)
target_high_card <- rbinom(2000, 1, 0.1)

result_prebin <- ob_categorical_dmiv(
  feature_high_card,
  target_high_card,
  max_n_prebins = 15,
  max_bins = 5
)

cat("Final bins after pre-binning:", length(result_prebin$bin), "\n")
cat("Algorithm converged:", result_prebin$converged, "\n")

---

Optimal Binning for Categorical Variables using Dynamic Programming

Description

Performs supervised discretization of categorical variables using a dynamic programming algorithm with optional monotonicity constraints. This method maximizes the total Information Value (IV) while ensuring optimal bin formation that respects user-defined constraints on bin count and frequency. The algorithm guarantees global optimality through dynamic programming.

Usage

ob_categorical_dp(
  feature,
  target,
  min_bins = 3,
  max_bins = 5,
  bin_cutoff = 0.05,
  max_n_prebins = 20,
  convergence_threshold = 1e-06,
  max_iterations = 1000,
  bin_separator = "%;%",
  monotonic_trend = "auto"
)

Arguments

feature	A character vector or factor representing the categorical predictor variable to be binned. Missing values are automatically converted to the category "NA".
target	An integer vector of binary outcomes (0/1) corresponding to each observation in feature. Missing values are not permitted.
min_bins	Integer. Minimum number of bins to produce. Must be >= 2. The algorithm searches for solutions within [min_bins, max_bins] that maximize total IV. Defaults to 3.
max_bins	Integer. Maximum number of bins to produce. Must be >= min_bins. Defines the upper bound of the search space for the optimal solution. Defaults to 5.
bin_cutoff	Numeric. Minimum proportion of total observations required for a category to remain separate. Categories below this threshold are merged with similar categories. Must be in (0, 1). Defaults to 0.05.
max_n_prebins	Integer. Maximum number of initial bins before dynamic programming optimization. Controls computational complexity. Must be >= 2. Defaults to 20.
convergence_threshold	Numeric. Convergence tolerance for the iterative dynamic programming updates. Smaller values require stricter convergence. Must be > 0. Defaults to 1e-6.
max_iterations	Integer. Maximum number of dynamic programming iterations. Prevents excessive computation in edge cases. Must be > 0. Defaults to 1000.
bin_separator	Character string used to concatenate category names when multiple categories are merged into a single bin. Defaults to "%;%". A warning is issued when a category name contains it, since such labels cannot be split back into categories.
monotonic_trend	Character string specifying monotonicity constraint for Weight of Evidence. Must be one of: "auto" Automatically determine trend direction (default) "ascending" Enforce increasing WoE across bins "descending" Enforce decreasing WoE across bins "none" No monotonicity constraint Categories are ordered by event rate (reversed for "descending") before the DP, so every bin partition it considers is monotone. Defaults to "auto".

Details

This implementation uses dynamic programming to find the globally optimal binning solution that maximizes total Information Value subject to constraints.

Algorithm Workflow:

1. Input validation and data preprocessing

2. Rare category merging (frequencies below bin_cutoff)

3. Pre-binning limitation (if categories exceed max_n_prebins)

4. Category sorting by event rate

5. Dynamic programming table initialization

6. Iterative DP optimization with optional monotonicity constraints

7. Backtracking to construct optimal bins

8. Final metric computation

Dynamic Programming Formulation:

Let $DP[i][k]$ represent the maximum total IV achievable using the first $i$ categories partitioned into $k$ bins. The recurrence relation is:

$DP[i][k] = \max_{j<i} \{DP[j][k-1] + IV(j+1, i)\}$

where $IV(j+1, i)$ is the Information Value of a bin containing categories from $j+1$ to $i$. Because the categories are sorted by event rate, the pooled event rate – and hence the WoE – of consecutive bins is monotone for every partition, so no transition needs to be excluded.

Computational Complexity:

• Time: $O(n^2 \cdot k)$ where $n$ = pre-bins and $k$ = max_bins (a single exact pass)

• Space: $O(n \cdot k)$ for DP tables

Advantages over Heuristic Methods:

• Guarantees global optimality (within constraint space)

• Explicit monotonicity enforcement

• Deterministic and reproducible results

• Efficient caching mechanism for bin statistics

Value

A list containing the binning results with the following components:

id

Integer vector of bin identifiers (1-indexed)

bin

Character vector of bin labels (merged category names)

woe

Numeric vector of Weight of Evidence values per bin

iv

Numeric vector of Information Value contribution per bin

count

Integer vector of total observations per bin

count_pos

Integer vector of positive cases (target=1) per bin

count_neg

Integer vector of negative cases (target=0) per bin

event_rate

Numeric vector of event rates per bin

total_iv

Numeric total Information Value of the binning solution

converged

Logical indicating if the DP algorithm converged

iterations

Integer count of DP iterations performed

execution_time_ms

Numeric execution time in milliseconds

References

Navas-Palencia, G. (2022). Optimal Binning: Mathematical Programming Formulation. arXiv preprint arXiv:2001.08025.

Bellman, R. (1954). The theory of dynamic programming. Bulletin of the American Mathematical Society, 60(6), 503-515.

Siddiqi, N. (2017). Intelligent Credit Scoring: Building and Implementing Better Credit Risk Scorecards (2nd ed.). Wiley.

Thomas, L. C., Edelman, D. B., & Crook, J. N. (2017). Credit Scoring and Its Applications (2nd ed.). SIAM.

See Also

ob_categorical_cm for ChiMerge-based binning, ob_categorical_dmiv for divergence measure-based binning

Examples

# Example 1: Basic usage with monotonic WoE enforcement
set.seed(123)
n_obs <- 1000

# Simulate education levels with increasing default risk
education <- c("High School", "Associate", "Bachelor", "Master", "PhD")
default_probs <- c(0.20, 0.15, 0.10, 0.06, 0.03)

cat_feature <- sample(education, n_obs,
  replace = TRUE,
  prob = c(0.30, 0.25, 0.25, 0.15, 0.05)
)
bin_target <- sapply(cat_feature, function(x) {
  rbinom(1, 1, default_probs[which(education == x)])
})

# Apply DP binning with ascending monotonicity
result_dp <- ob_categorical_dp(
  cat_feature,
  bin_target,
  min_bins = 2,
  max_bins = 4,
  monotonic_trend = "ascending"
)

# Display results
print(data.frame(
  Bin = result_dp$bin,
  WoE = round(result_dp$woe, 3),
  IV = round(result_dp$iv, 4),
  Count = result_dp$count,
  EventRate = round(result_dp$event_rate, 3)
))

cat("Total IV:", round(result_dp$total_iv, 4), "\n")
cat("Converged:", result_dp$converged, "\n")

# Example 2: Comparing monotonicity constraints
result_dp_asc <- ob_categorical_dp(
  cat_feature, bin_target,
  max_bins = 3,
  monotonic_trend = "ascending"
)

result_dp_none <- ob_categorical_dp(
  cat_feature, bin_target,
  max_bins = 3,
  monotonic_trend = "none"
)

cat("\nWith monotonicity:\n")
cat("  Bins:", length(result_dp_asc$bin), "\n")
cat("  Total IV:", round(result_dp_asc$total_iv, 4), "\n")

cat("\nWithout monotonicity:\n")
cat("  Bins:", length(result_dp_none$bin), "\n")
cat("  Total IV:", round(result_dp_none$total_iv, 4), "\n")

# Example 3: High cardinality with pre-binning
set.seed(456)
n_obs_large <- 5000

# Simulate customer segments (high cardinality)
segments <- paste0("Segment_", LETTERS[1:20])
segment_probs <- runif(20, 0.01, 0.20)

cat_feature_hc <- sample(segments, n_obs_large, replace = TRUE)
bin_target_hc <- rbinom(
  n_obs_large, 1,
  segment_probs[match(cat_feature_hc, segments)]
)

result_dp_hc <- ob_categorical_dp(
  cat_feature_hc,
  bin_target_hc,
  min_bins = 3,
  max_bins = 5,
  bin_cutoff = 0.03,
  max_n_prebins = 10
)

cat("\nHigh cardinality example:\n")
cat("  Original categories:", length(unique(cat_feature_hc)), "\n")
cat("  Final bins:", length(result_dp_hc$bin), "\n")
cat("  Execution time:", result_dp_hc$execution_time_ms, "ms\n")

# Example 4: Handling missing values
set.seed(789)
cat_feature_na <- cat_feature
cat_feature_na[sample(n_obs, 50)] <- NA # Introduce 5% missing

result_dp_na <- ob_categorical_dp(
  cat_feature_na,
  bin_target,
  min_bins = 2,
  max_bins = 4
)

# Check if NA was treated as a category
na_bin <- grep("NA", result_dp_na$bin, value = TRUE)
if (length(na_bin) > 0) {
  cat("\nNA handling:\n")
  cat("  Bin containing NA:", na_bin, "\n")
}

---

Optimal Binning for Categorical Variables using Fisher's Exact Test

Description

Performs supervised discretization of categorical variables using Fisher's Exact Test as the similarity criterion for hierarchical bin merging. This method iteratively merges the most statistically similar bins (highest p-value) while enforcing Weight of Evidence monotonicity, providing a statistically rigorous approach to optimal binning.

Usage

ob_categorical_fetb(
  feature,
  target,
  min_bins = 3,
  max_bins = 5,
  bin_cutoff = 0.05,
  max_n_prebins = 20,
  convergence_threshold = 1e-06,
  max_iterations = 1000,
  bin_separator = "%;%"
)

Arguments

feature	A character vector or factor representing the categorical predictor variable to be binned. Missing values are automatically converted to the category "NA".
target	An integer vector of binary outcomes (0/1) corresponding to each observation in feature. Missing values are not permitted.
min_bins	Integer. Minimum number of bins to produce. Must be >= 2. The algorithm will not merge below this threshold. Defaults to 3.
max_bins	Integer. Maximum number of bins to produce. Must be >= min_bins. The algorithm merges bins until this constraint is satisfied. Defaults to 5.
bin_cutoff	Numeric. Minimum proportion of total observations required for a category to avoid being classified as rare. Rare categories are pre-merged before the main algorithm. Must be in (0, 1). Defaults to 0.05.
max_n_prebins	Integer. Maximum number of initial bins before the merging phase. Controls computational complexity for high-cardinality features. Must be >= 2. Defaults to 20.
convergence_threshold	Numeric. Convergence tolerance based on Information Value change between iterations. Algorithm stops when $|\Delta IV| <$ convergence_threshold. Must be > 0. Defaults to 1e-6.
max_iterations	Integer. Iteration budget for the merge phase. Must be > 0. Defaults to 1000. When it is used up before the convergence tolerance is met, converged is FALSE; merging still continues until max_bins is satisfied, since max_bins is a hard constraint.
bin_separator	Character string used to concatenate category names when multiple categories are merged into a single bin. Defaults to "%;%". A warning is issued when a category name contains it, since such labels cannot be split back into categories.

Details

This algorithm employs Fisher's Exact Test to quantify statistical similarity between bins based on their 2×2 contingency tables. Unlike chi-square based methods, Fisher's test provides exact p-values without relying on asymptotic approximations, making it particularly suitable for small sample sizes.

Algorithm Workflow:

1. Data preprocessing and frequency computation

2. Rare category identification and pre-merging (frequencies < bin_cutoff); if pooling all rare categories would leave fewer than min_bins bins, they are pooled in event-rate order into groups of at least bin_cutoff

3. Initial bin creation (one category per bin)

4. Iterative merging phase:

   • Compute Fisher's Exact Test p-values for all adjacent bin pairs

   • Merge the pair with the highest p-value (most similar)

   • Enforce WoE monotonicity after each merge

   • Check convergence based on IV change

5. Final monotonicity enforcement

Fisher's Exact Test:

For two bins with contingency table:

	Bin 1	Bin 2
Positives	$a$	$c$
Negatives	$b$	$d$

Under the null hypothesis of identical event rates, with all margins fixed, the number of positives in Bin 1 is hypergeometric, and the probability of a table with $x$ positives in Bin 1 is

$P(x) = \frac{\binom{a+b}{x}\binom{c+d}{a+c-x}}{\binom{n}{a+c}}, \quad n = a + b + c + d.$

The two-sided p-value is the total probability of all tables with the same margins that are no more likely than the observed one, $p = \sum_{x:\,P(x) \le P(a)} P(x)$ (with the relative tolerance $10^{-7}$ used by fisher.test), which is the value fisher.test() reports. Higher p-values indicate greater similarity (less evidence against the null hypothesis of identical distributions). Monotonicity repairs stop at min_bins.

Key Features:

• Exact inference: No asymptotic approximations required

• Small sample robustness: Valid for any sample size

• Automatic monotonicity: WoE ordering enforced after each merge

• Efficient caching: Log-factorial and p-value caching for speed

• Rare category handling: Pre-merging prevents sparse bins

Computational Complexity:

• Time: $O(k^2 \cdot m)$ where $k$ = initial bins, $m$ = iterations

• Space: $O(k + n_{max})$ for bins and factorial cache

Value

A list containing the binning results with the following components:

id

Integer vector of bin identifiers (1-indexed)

bin

Character vector of bin labels (merged category names)

woe

Numeric vector of Weight of Evidence values per bin

iv

Numeric vector of Information Value contribution per bin

count

Integer vector of total observations per bin

count_pos

Integer vector of positive cases (target=1) per bin

count_neg

Integer vector of negative cases (target=0) per bin

converged

Logical indicating algorithm convergence

iterations

Integer count of merge operations performed

References

Fisher, R. A. (1922). On the interpretation of chi-squared from contingency tables, and the calculation of P. Journal of the Royal Statistical Society, 85(1), 87-94. doi:10.2307/2340521

Agresti, A. (2013). Categorical Data Analysis (3rd ed.). Wiley.

Mehta, C. R., & Patel, N. R. (1983). A network algorithm for performing Fisher's exact test in r×c contingency tables. Journal of the American Statistical Association, 78(382), 427-434.

Zeng, G. (2014). A necessary condition for a good binning algorithm in credit scoring. Applied Mathematical Sciences, 8(65), 3229-3242.

See Also

ob_categorical_cm for ChiMerge-based binning, ob_categorical_dp for dynamic programming approach, ob_categorical_dmiv for divergence measure-based binning

Examples

# Example 1: Basic usage with Fisher's Exact Test
set.seed(42)
n_obs <- 800

# Simulate customer segments with different risk profiles
segments <- c("Premium", "Standard", "Basic", "Budget", "Economy")
risk_rates <- c(0.05, 0.10, 0.15, 0.22, 0.30)

cat_feature <- sample(segments, n_obs,
  replace = TRUE,
  prob = c(0.15, 0.25, 0.30, 0.20, 0.10)
)
bin_target <- sapply(cat_feature, function(x) {
  rbinom(1, 1, risk_rates[which(segments == x)])
})

# Apply Fisher's Exact Test binning
result_fetb <- ob_categorical_fetb(
  cat_feature,
  bin_target,
  min_bins = 2,
  max_bins = 4
)

# Display results
print(data.frame(
  Bin = result_fetb$bin,
  WoE = round(result_fetb$woe, 3),
  IV = round(result_fetb$iv, 4),
  Count = result_fetb$count,
  EventRate = round(result_fetb$count_pos / result_fetb$count, 3)
))

cat("\nAlgorithm converged:", result_fetb$converged, "\n")
cat("Iterations performed:", result_fetb$iterations, "\n")

# Example 2: Comparing with ChiMerge method
result_cm <- ob_categorical_cm(
  cat_feature,
  bin_target,
  min_bins = 2,
  max_bins = 4
)

cat("\nFisher's Exact Test:\n")
cat("  Final bins:", length(result_fetb$bin), "\n")
cat("  Total IV:", round(sum(result_fetb$iv), 4), "\n")

cat("\nChiMerge:\n")
cat("  Final bins:", length(result_cm$bin), "\n")
cat("  Total IV:", round(sum(result_cm$iv), 4), "\n")

# Example 3: Small sample size (Fisher's advantage)
set.seed(123)
n_obs_small <- 150

# Small sample with sparse categories
occupation <- c(
  "Doctor", "Lawyer", "Teacher", "Engineer",
  "Sales", "Manager"
)

cat_feature_small <- sample(occupation, n_obs_small,
  replace = TRUE,
  prob = c(0.10, 0.10, 0.20, 0.25, 0.20, 0.15)
)
bin_target_small <- rbinom(n_obs_small, 1, 0.12)

result_fetb_small <- ob_categorical_fetb(
  cat_feature_small,
  bin_target_small,
  min_bins = 2,
  max_bins = 3,
  bin_cutoff = 0.03 # Allow smaller bins for small sample
)

cat("\nSmall sample binning:\n")
cat("  Observations:", n_obs_small, "\n")
cat("  Original categories:", length(unique(cat_feature_small)), "\n")
cat("  Final bins:", length(result_fetb_small$bin), "\n")
cat("  Converged:", result_fetb_small$converged, "\n")

# Example 4: High cardinality with rare categories
set.seed(789)
n_obs_hc <- 2000

# Simulate product codes (high cardinality)
product_codes <- paste0("PROD_", sprintf("%03d", 1:30))

cat_feature_hc <- sample(product_codes, n_obs_hc,
  replace = TRUE,
  prob = c(
    rep(0.05, 10), rep(0.02, 10),
    rep(0.01, 10)
  )
)
bin_target_hc <- rbinom(n_obs_hc, 1, 0.08)

result_fetb_hc <- ob_categorical_fetb(
  cat_feature_hc,
  bin_target_hc,
  min_bins = 3,
  max_bins = 6,
  bin_cutoff = 0.02,
  max_n_prebins = 15
)

cat("\nHigh cardinality example:\n")
cat("  Original categories:", length(unique(cat_feature_hc)), "\n")
cat("  Final bins:", length(result_fetb_hc$bin), "\n")
cat("  Iterations:", result_fetb_hc$iterations, "\n")

# Check for rare category merging
for (i in seq_along(result_fetb_hc$bin)) {
  n_merged <- length(strsplit(result_fetb_hc$bin[i], "%;%")[[1]])
  if (n_merged > 1) {
    cat("  Bin", i, "contains", n_merged, "merged categories\n")
  }
}

# Example 5: Missing value handling
set.seed(456)
cat_feature_na <- cat_feature
na_indices <- sample(n_obs, 40) # 5% missing
cat_feature_na[na_indices] <- NA

result_fetb_na <- ob_categorical_fetb(
  cat_feature_na,
  bin_target,
  min_bins = 2,
  max_bins = 4
)

# Check NA treatment
na_bin_idx <- grep("NA", result_fetb_na$bin)
if (length(na_bin_idx) > 0) {
  cat("\nMissing value handling:\n")
  cat("  NA bin:", result_fetb_na$bin[na_bin_idx], "\n")
  cat("  NA count:", result_fetb_na$count[na_bin_idx], "\n")
  cat("  NA WoE:", round(result_fetb_na$woe[na_bin_idx], 3), "\n")
}

---

Optimal Binning for Categorical Variables using Greedy Merge Algorithm

Description

Performs supervised discretization of categorical variables using a greedy bottom-up merging strategy that iteratively combines bins to maximize total Information Value (IV). This approach uses Bayesian smoothing for numerical stability and employs adaptive monotonicity constraints, providing a fast approximation to optimal binning suitable for high-cardinality features.

Usage

ob_categorical_gmb(
  feature,
  target,
  min_bins = 3,
  max_bins = 5,
  bin_cutoff = 0.05,
  max_n_prebins = 20,
  bin_separator = "%;%",
  convergence_threshold = 1e-06,
  max_iterations = 1000
)

Arguments

feature	A character vector or factor representing the categorical predictor variable to be binned. Missing values are automatically converted to the category "NA".
target	An integer vector of binary outcomes (0/1) corresponding to each observation in feature. Missing values are not permitted.
min_bins	Integer. Minimum number of bins to produce. Must be >= 2. Merging stops when this threshold is reached. Defaults to 3.
max_bins	Integer. Maximum number of bins to produce. Must be >= min_bins. The algorithm terminates when bins are reduced to this number. Defaults to 5.
bin_cutoff	Numeric. Minimum proportion of total observations required for a category to remain separate during initialization. Categories below this threshold are pre-merged. Must be in (0, 1). Defaults to 0.05.
max_n_prebins	Integer. Maximum number of initial bins before the greedy merging phase. Controls computational complexity. Must be >= min_bins. Defaults to 20.
bin_separator	Character string used to concatenate category names when multiple categories are merged into a single bin. Defaults to "%;%". A warning is issued when a category name contains it, since such labels cannot be split back into categories.
convergence_threshold	Numeric. Convergence tolerance for IV change between iterations. Algorithm stops when $|\Delta IV| <$ convergence_threshold. Must be > 0. Defaults to 1e-6.
max_iterations	Integer. Maximum number of merge operations allowed. Prevents excessive computation. Must be > 0. Defaults to 1000.

Details

The Greedy Merge Binning (GMB) algorithm employs a bottom-up approach where bins are iteratively merged based on maximum IV improvement. Unlike exact optimization methods (e.g., dynamic programming), GMB provides approximate solutions with significantly reduced computational cost.

Algorithm Workflow:

1. Input validation and preprocessing

2. Initial bin creation (one category per bin)

3. Rare category merging (frequencies < bin_cutoff); if this would leave fewer than min_bins bins, rare neighbours are pooled only until each pool reaches bin_cutoff

4. Pre-bin limitation (if bins > max_n_prebins)

5. Greedy merging phase:

   • Evaluate IV for all possible adjacent bin merges

   • Select merge that maximizes total IV

   • Apply tie-breaking rules for similar merges

   • Check convergence criteria

6. Adaptive monotonicity enforcement

7. Final metric computation

Bayesian Smoothing:

To prevent numerical instability with sparse bins, WoE is calculated using Bayesian smoothing:

$WoE_i = \ln\left(\frac{p_i + \alpha_p}{n_i + \alpha_n}\right)$

where $\alpha_p$ and $\alpha_n$ are prior pseudocounts proportional to the overall event rate. This regularization ensures stable WoE estimates even for bins with zero events.

Greedy Selection Criterion:

At each iteration, the algorithm evaluates the IV gain for merging adjacent bins $i$ and $j$:

$\Delta IV_{i,j} = IV_{merged}(i,j) - (IV_i + IV_j)$

The pair with maximum $\Delta IV$ is merged. Early stopping occurs if $\Delta IV > 0.05 \cdot IV_{current}$ (5% improvement threshold).

Tie Handling:

When multiple merges yield similar IV gains (within 10× convergence threshold), the algorithm prefers merges that produce more balanced bins, breaking ties based on size difference:

$balance\_score = |count_i - count_j|$

Computational Complexity:

• Time: $O(k^2 \cdot m)$ where $k$ = bins, $m$ = iterations

• Space: $O(k^2)$ for IV cache (optional)

• Typical runtime: 10-100× faster than exact methods for $k > 20$

Advantages:

• Fast execution for high-cardinality features

• Incremental IV caching for efficiency

• Bayesian smoothing prevents overfitting

• Adaptive monotonicity with gradient relaxation

• Handles imbalanced datasets robustly

Limitations:

• Approximate solution (not guaranteed global optimum)

• Greedy nature may miss better non-adjacent merges

• Sensitive to initialization order

Value

A list containing the binning results with the following components:

id

Integer vector of bin identifiers (1-indexed)

bin

Character vector of bin labels (merged category names)

woe

Numeric vector of Weight of Evidence values per bin

iv

Numeric vector of Information Value contribution per bin

count

Integer vector of total observations per bin

count_pos

Integer vector of positive cases (target=1) per bin

count_neg

Integer vector of negative cases (target=0) per bin

total_iv

Numeric total Information Value of the binning solution

converged

Logical indicating algorithm convergence

iterations

Integer count of merge operations performed

References

Navas-Palencia, G. (2020). Optimal binning: mathematical programming formulation and solution approach. Expert Systems with Applications, 158, 113508. doi:10.1016/j.eswa.2020.113508

Good, I. J. (1965). The Estimation of Probabilities: An Essay on Modern Bayesian Methods. MIT Press.

Zeng, G. (2014). A necessary condition for a good binning algorithm in credit scoring. Applied Mathematical Sciences, 8(65), 3229-3242.

Mironchyk, P., & Tchistiakov, V. (2017). Monotone optimal binning algorithm for credit risk modeling. SSRN Electronic Journal. doi:10.2139/ssrn.2978774

See Also

ob_categorical_dp for exact optimization via dynamic programming, ob_categorical_cm for ChiMerge-based binning, ob_categorical_fetb for Fisher's Exact Test binning

Examples

# Example 1: Basic greedy merge binning
set.seed(123)
n_obs <- 1500

# Simulate customer types with varying risk
customer_types <- c(
  "Premium", "Gold", "Silver", "Bronze",
  "Basic", "Trial"
)
risk_probs <- c(0.02, 0.05, 0.10, 0.15, 0.22, 0.35)

cat_feature <- sample(customer_types, n_obs,
  replace = TRUE,
  prob = c(0.10, 0.15, 0.25, 0.25, 0.15, 0.10)
)
bin_target <- sapply(cat_feature, function(x) {
  rbinom(1, 1, risk_probs[which(customer_types == x)])
})

# Apply greedy merge binning
result_gmb <- ob_categorical_gmb(
  cat_feature,
  bin_target,
  min_bins = 3,
  max_bins = 4
)

# Display results
print(data.frame(
  Bin = result_gmb$bin,
  WoE = round(result_gmb$woe, 3),
  IV = round(result_gmb$iv, 4),
  Count = result_gmb$count,
  EventRate = round(result_gmb$count_pos / result_gmb$count, 3)
))

cat("\nTotal IV:", round(result_gmb$total_iv, 4), "\n")
cat("Converged:", result_gmb$converged, "\n")
cat("Iterations:", result_gmb$iterations, "\n")

# Example 2: Comparing speed with exact methods
set.seed(456)
n_obs <- 3000

# High cardinality feature
regions <- paste0("Region_", sprintf("%02d", 1:25))
cat_feature_hc <- sample(regions, n_obs, replace = TRUE)
bin_target_hc <- rbinom(n_obs, 1, 0.12)

# Greedy approach (fast)
time_gmb <- system.time({
  result_gmb_hc <- ob_categorical_gmb(
    cat_feature_hc,
    bin_target_hc,
    min_bins = 3,
    max_bins = 6,
    max_n_prebins = 20
  )
})

# Dynamic programming (exact but slower)
time_dp <- system.time({
  result_dp_hc <- ob_categorical_dp(
    cat_feature_hc,
    bin_target_hc,
    min_bins = 3,
    max_bins = 6,
    max_n_prebins = 20
  )
})

cat("\nPerformance comparison (high cardinality):\n")
cat("  GMB time:", round(time_gmb[3], 3), "seconds\n")
cat("  DP time:", round(time_dp[3], 3), "seconds\n")
cat("  Speedup:", round(time_dp[3] / time_gmb[3], 1), "x\n")
cat("\n  GMB IV:", round(result_gmb_hc$total_iv, 4), "\n")
cat("  DP IV:", round(result_dp_hc$total_iv, 4), "\n")

# Example 3: Convergence behavior
set.seed(789)
n_obs_conv <- 1000

# Feature with natural groupings
education <- c("PhD", "Master", "Bachelor", "HighSchool", "NoHighSchool")
cat_feature_conv <- sample(education, n_obs_conv,
  replace = TRUE,
  prob = c(0.05, 0.15, 0.35, 0.30, 0.15)
)
bin_target_conv <- sapply(cat_feature_conv, function(x) {
  probs <- c(0.02, 0.05, 0.08, 0.15, 0.25)
  rbinom(1, 1, probs[which(education == x)])
})

# Test different convergence thresholds
thresholds <- c(1e-3, 1e-6, 1e-9)

for (thresh in thresholds) {
  result_conv <- ob_categorical_gmb(
    cat_feature_conv,
    bin_target_conv,
    min_bins = 2,
    max_bins = 4,
    convergence_threshold = thresh
  )

  cat(sprintf(
    "\nThreshold %.0e: %d iterations, converged=%s\n",
    thresh, result_conv$iterations, result_conv$converged
  ))
}

# Example 4: Handling rare categories
set.seed(321)
n_obs_rare <- 2000

# Simulate with many rare categories
products <- c(paste0("Common_", 1:5), paste0("Rare_", 1:15))
product_probs <- c(rep(0.15, 5), rep(0.01, 15))

cat_feature_rare <- sample(products, n_obs_rare,
  replace = TRUE,
  prob = product_probs
)
bin_target_rare <- rbinom(n_obs_rare, 1, 0.10)

result_gmb_rare <- ob_categorical_gmb(
  cat_feature_rare,
  bin_target_rare,
  min_bins = 3,
  max_bins = 5,
  bin_cutoff = 0.03 # Aggressive rare category merging
)

cat("\nRare category handling:\n")
cat("  Original categories:", length(unique(cat_feature_rare)), "\n")
cat("  Final bins:", length(result_gmb_rare$bin), "\n")

# Count merged rare categories
for (i in seq_along(result_gmb_rare$bin)) {
  n_merged <- length(strsplit(result_gmb_rare$bin[i], "%;%")[[1]])
  if (n_merged > 1) {
    cat(sprintf("  Bin %d: %d categories merged\n", i, n_merged))
  }
}

# Example 5: Imbalanced dataset robustness
set.seed(555)
n_obs_imb <- 1200

# Highly imbalanced target (2% event rate)
cat_feature_imb <- sample(c("A", "B", "C", "D", "E"),
  n_obs_imb,
  replace = TRUE
)
bin_target_imb <- rbinom(n_obs_imb, 1, 0.02)

result_gmb_imb <- ob_categorical_gmb(
  cat_feature_imb,
  bin_target_imb,
  min_bins = 2,
  max_bins = 3
)

cat("\nImbalanced dataset:\n")
cat("  Event rate:", round(mean(bin_target_imb), 4), "\n")
cat("  Total events:", sum(bin_target_imb), "\n")
cat("  Bins created:", length(result_gmb_imb$bin), "\n")
cat("  WoE range:", sprintf(
  "[%.2f, %.2f]\n",
  min(result_gmb_imb$woe),
  max(result_gmb_imb$woe)
))

---

Optimal Binning for Categorical Variables using Information Value Dynamic Programming

Description

Performs supervised discretization of categorical variables using a dynamic programming algorithm specifically designed to maximize total Information Value (IV). This implementation employs Bayesian smoothing for numerical stability, maintains monotonic Weight of Evidence constraints, and uses efficient caching strategies for optimal performance with high-cardinality features.

Usage

ob_categorical_ivb(
  feature,
  target,
  min_bins = 3,
  max_bins = 5,
  bin_cutoff = 0.05,
  max_n_prebins = 20,
  bin_separator = "%;%",
  convergence_threshold = 1e-06,
  max_iterations = 1000
)

Arguments

feature	A character vector or factor representing the categorical predictor variable to be binned. Missing values are automatically converted to the category "NA".
target	An integer vector of binary outcomes (0/1) corresponding to each observation in feature. Missing values are not permitted.
min_bins	Integer. Minimum number of bins to produce. Must be >= 2. The algorithm searches for solutions within [min_bins, max_bins] that maximize total IV. Defaults to 3.
max_bins	Integer. Maximum number of bins to produce. Must be >= min_bins. Defines the upper bound of the search space. Defaults to 5.
bin_cutoff	Numeric. Minimum proportion of total observations required for a category to remain separate. Categories below this threshold are pre-merged before the optimization phase. Must be in (0, 1). Defaults to 0.05.
max_n_prebins	Integer. Maximum number of initial bins before dynamic programming optimization. Controls computational complexity for high-cardinality features. Must be >= 2. Defaults to 20.
bin_separator	Character string used to concatenate category names when multiple categories are merged into a single bin. Defaults to "%;%". A warning is issued when a category name contains it, since such labels cannot be split back into categories.
convergence_threshold	Numeric. Convergence tolerance for the iterative optimization process based on IV change. Algorithm stops when $|\Delta IV| <$ convergence_threshold. Must be > 0. Defaults to 1e-6.
max_iterations	Integer. Maximum number of optimization iterations. Prevents excessive computation. Must be > 0. Defaults to 1000.

Details

The Information Value Binning (IVB) algorithm uses dynamic programming to find the globally optimal binning solution that maximizes total IV subject to constraints on bin count and monotonicity.

Algorithm Workflow:

1. Input validation and preprocessing

2. Single-pass category counting and statistics computation

3. Rare category pre-merging (frequencies < bin_cutoff); if pooling all rare categories would leave fewer than min_bins bins, they are pooled in event-rate order into groups of at least bin_cutoff

4. Pre-bin limitation (if categories > max_n_prebins)

5. Category sorting by event rate

6. Cumulative statistics cache initialization

7. Dynamic programming table computation:

   • State: $DP[i][k]$ = max IV using first $i$ categories in $k$ bins

   • Transition: $DP[i][k] = \max_j \{DP[j][k-1] + IV(j+1, i)\}$

8. Backtracking to reconstruct optimal bins

9. Adaptive monotonicity enforcement

10. Final metric computation with Bayesian smoothing

Dynamic Programming Formulation:

Let $DP[i][k]$ represent the maximum total IV achievable using the first $i$ categories (sorted by event rate) partitioned into $k$ bins.

Recurrence relation:

$DP[i][k] = \max_{k-1 \leq j < i} \{DP[j][k-1] + IV(j+1, i)\}$

Base case:

$DP[i][1] = IV(1, i) \quad \forall i$

where $IV(j+1, i)$ is the Information Value of a bin containing categories from $j+1$ to $i$.

Bayesian Smoothing:

To prevent numerical instability and overfitting with sparse bins, WoE and IV are calculated using Bayesian smoothing with pseudocounts:

$p'_i = \frac{n_{i,pos} + \alpha_p}{N_{pos} + \alpha_{total}}$

$n'_i = \frac{n_{i,neg} + \alpha_n}{N_{neg} + \alpha_{total}}$

where $\alpha_p$ and $\alpha_n$ are prior pseudocounts proportional to the overall event rate, and $\alpha_{total} = 1.0$ (prior strength).

$WoE_i = \ln\left(\frac{p'_i}{n'_i}\right)$

$IV_i = (p'_i - n'_i) \times WoE_i$

Adaptive Monotonicity Enforcement:

After finding the optimal bins, the algorithm enforces WoE monotonicity by:

1. Computing average WoE gap: $\bar{\Delta} = \frac{1}{k-1}\sum_{i=1}^{k-1}|WoE_{i+1} - WoE_i|$

2. Setting adaptive threshold: $\tau = \min(\epsilon, 0.01\bar{\Delta})$

3. Identifying worst violation: $i^* = \arg\max_i \{WoE_i - WoE_{i+1}\}$

4. Evaluating forward and backward merges by IV retention

5. Selecting merge direction that maximizes total IV

Computational Complexity:

• Time: $O(k^2 \cdot n)$ where $k$ = max_bins, $n$ = categories

• Space: $O(k \cdot n)$ for DP tables and cumulative caches

• IV calculations are $O(1)$ due to cumulative statistics caching

Advantages over Alternative Methods:

• Global optimality: Guaranteed maximum IV (within constraint space)

• Bayesian regularization: Robust to sparse bins and class imbalance

• Efficient caching: Cumulative stats and IV memoization

• Adaptive monotonicity: Context-aware threshold for enforcement

Comparison with Related Methods:

• vs DP (general): IVB specifically optimizes IV; general DP more flexible

• vs GMB: IVB guarantees optimality; GMB is faster but approximate

• vs ChiMerge: IVB uses IV criterion; ChiMerge uses chi-square

Value

A list containing the binning results with the following components:

id

Integer vector of bin identifiers (1-indexed)

bin

Character vector of bin labels (merged category names)

woe

Numeric vector of Weight of Evidence values per bin

iv

Numeric vector of Information Value contribution per bin

count

Integer vector of total observations per bin

count_pos

Integer vector of positive cases (target=1) per bin

count_neg

Integer vector of negative cases (target=0) per bin

total_iv

Numeric total Information Value of the binning solution

converged

Logical indicating algorithm convergence

iterations

Integer count of optimization iterations performed

References

Navas-Palencia, G. (2020). Optimal binning: mathematical programming formulation and solution approach. Expert Systems with Applications, 158, 113508. doi:10.1016/j.eswa.2020.113508

Bellman, R. (1957). Dynamic Programming. Princeton University Press.

Siddiqi, N. (2017). Intelligent Credit Scoring: Building and Implementing Better Credit Risk Scorecards (2nd ed.). Wiley.

Good, I. J. (1965). The Estimation of Probabilities: An Essay on Modern Bayesian Methods. MIT Press.

Anderson, R. (2007). The Credit Scoring Toolkit: Theory and Practice for Retail Credit Risk Management and Decision Automation. Oxford University Press.

See Also

ob_categorical_dp for general dynamic programming binning, ob_categorical_gmb for greedy merge approximation, ob_categorical_cm for ChiMerge-based binning

Examples

# Example 1: Basic IV optimization with Bayesian smoothing
set.seed(42)
n_obs <- 1200

# Simulate industry sectors with varying default risk
industries <- c(
  "Technology", "Healthcare", "Finance", "Manufacturing",
  "Retail", "Energy"
)
default_rates <- c(0.03, 0.05, 0.08, 0.12, 0.18, 0.25)

cat_feature <- sample(industries, n_obs,
  replace = TRUE,
  prob = c(0.20, 0.18, 0.22, 0.18, 0.12, 0.10)
)
bin_target <- sapply(cat_feature, function(x) {
  rbinom(1, 1, default_rates[which(industries == x)])
})

# Apply IVB optimization
result_ivb <- ob_categorical_ivb(
  cat_feature,
  bin_target,
  min_bins = 3,
  max_bins = 4
)

# Display results
print(data.frame(
  Bin = result_ivb$bin,
  WoE = round(result_ivb$woe, 3),
  IV = round(result_ivb$iv, 4),
  Count = result_ivb$count,
  EventRate = round(result_ivb$count_pos / result_ivb$count, 3)
))

cat("\nTotal IV (maximized):", round(result_ivb$total_iv, 4), "\n")
cat("Converged:", result_ivb$converged, "\n")
cat("Iterations:", result_ivb$iterations, "\n")

# Example 2: Comparing IV optimization with other methods
set.seed(123)
n_obs_comp <- 1500

regions <- c("North", "South", "East", "West", "Central")
cat_feature_comp <- sample(regions, n_obs_comp, replace = TRUE)
bin_target_comp <- rbinom(n_obs_comp, 1, 0.15)

# IVB (IV-optimized)
result_ivb_comp <- ob_categorical_ivb(
  cat_feature_comp, bin_target_comp,
  min_bins = 2, max_bins = 3
)

# GMB (greedy approximation)
result_gmb_comp <- ob_categorical_gmb(
  cat_feature_comp, bin_target_comp,
  min_bins = 2, max_bins = 3
)

# DP (general optimization)
result_dp_comp <- ob_categorical_dp(
  cat_feature_comp, bin_target_comp,
  min_bins = 2, max_bins = 3
)

cat("\nMethod comparison:\n")
cat("  IVB total IV:", round(result_ivb_comp$total_iv, 4), "\n")
cat("  GMB total IV:", round(result_gmb_comp$total_iv, 4), "\n")
cat("  DP total IV:", round(result_dp_comp$total_iv, 4), "\n")
cat("\nIVB typically achieves highest IV due to explicit optimization\n")

---

Optimal Binning for Categorical Variables using JEDI Algorithm

Description

Performs supervised discretization of categorical variables using the Joint Entropy-Driven Information Maximization (JEDI) algorithm. This advanced method combines information-theoretic optimization with intelligent bin merging strategies, employing Bayesian smoothing for numerical stability and adaptive monotonicity enforcement to produce robust, interpretable binning solutions.

Usage

ob_categorical_jedi(
  feature,
  target,
  min_bins = 3,
  max_bins = 5,
  bin_cutoff = 0.05,
  max_n_prebins = 20,
  bin_separator = "%;%",
  convergence_threshold = 1e-06,
  max_iterations = 1000
)

Arguments

feature	A character vector or factor representing the categorical predictor variable to be binned. Missing values are automatically converted to the category "NA".
target	An integer vector of binary outcomes (0/1) corresponding to each observation in feature. Missing values are not permitted.
min_bins	Integer. Minimum number of bins to produce. Must be >= 2. The algorithm will not merge below this threshold. Defaults to 3.
max_bins	Integer. Maximum number of bins to produce. Must be >= min_bins. The algorithm iteratively merges until this constraint is satisfied. Defaults to 5.
bin_cutoff	Numeric. Minimum proportion of total observations required for a category to remain separate during initialization. Categories below this threshold are pre-merged into an "Others" bin. Must be in (0, 1). Defaults to 0.05.
max_n_prebins	Integer. Maximum number of initial bins before the main optimization phase. Controls computational complexity for high-cardinality features. Must be >= min_bins. Defaults to 20.
bin_separator	Character string used to concatenate category names when multiple categories are merged into a single bin. Defaults to "%;%".
convergence_threshold	Numeric. Convergence tolerance based on Information Value change between iterations. Algorithm stops when $|\Delta IV| <$ convergence_threshold. Must be > 0. Defaults to 1e-6.
max_iterations	Integer. Maximum number of optimization iterations. Prevents infinite loops in edge cases. Must be > 0. Defaults to 1000.

Details

The JEDI (Joint Entropy-Driven Information Maximization) algorithm represents a sophisticated approach to categorical binning that jointly optimizes Information Value while maintaining monotonic Weight of Evidence constraints through intelligent violation detection and repair strategies.

Algorithm Workflow:

1. Input validation and preprocessing

2. Initial bin creation (one category per bin)

3. Rare category merging (frequencies < bin_cutoff)

4. WoE-based monotonic sorting

5. Pre-bin limitation via minimal IV-loss merging

6. Main optimization loop:

   • Monotonicity violation detection (peaks and valleys)

   • Violation severity quantification

   • Intelligent merge selection (minimize IV loss)

   • Convergence monitoring

   • Best solution tracking

7. Final constraint satisfaction (max_bins enforcement)

8. Bayesian-smoothed metric computation

Joint Entropy-Driven Optimization:

Unlike greedy algorithms that optimize locally, JEDI considers the global impact of each merge on total Information Value:

$IV_{total} = \sum_{i=1}^{k} (p_i - n_i) \times \ln\left(\frac{p_i}{n_i}\right)$

For each potential merge of bins $j$ and $j+1$, JEDI evaluates:

$\Delta IV_{j,j+1} = IV_{current} - IV_{merged}(j,j+1)$

The pair with minimum $\Delta IV$ (least information loss) is selected.

Violation Detection and Repair:

JEDI identifies two types of monotonicity violations:

• Peaks: $WoE_i > WoE_{i-1}$ and $WoE_i > WoE_{i+1}$

• Valleys: $WoE_i < WoE_{i-1}$ and $WoE_i < WoE_{i+1}$

For each violation, severity is quantified as:

$severity_i = \max\{|WoE_i - WoE_{i-1}|, |WoE_i - WoE_{i+1}|\}$

The algorithm prioritizes fixing the most severe violation first, evaluating both forward merge $(i, i+1)$ and backward merge $(i-1, i)$ to select the option that minimizes information loss.

Bayesian Smoothing:

To ensure numerical stability with sparse bins, JEDI applies Bayesian smoothing:

$p'_i = \frac{n_{i,pos} + \alpha_p}{N_{pos} + \alpha_{total}}$

$n'_i = \frac{n_{i,neg} + \alpha_n}{N_{neg} + \alpha_{total}}$

where prior pseudocounts are proportional to overall prevalence:

$\alpha_p = \alpha_{total} \times \frac{N_{pos}}{N_{pos} + N_{neg}}$

$\alpha_n = \alpha_{total} - \alpha_p$

with $\alpha_{total} = 0.5$ as the prior strength parameter.

Adaptive Monotonicity Threshold:

Rather than using a fixed threshold, JEDI computes a context-aware tolerance:

$\bar{\Delta} = \frac{1}{k-1}\sum_{i=1}^{k-1}|WoE_{i+1} - WoE_i|$

$\tau = \min(\epsilon, 0.01\bar{\Delta})$

This adaptive approach prevents over-merging when natural WoE gaps are small.

Computational Complexity:

• Time: $O(k^2 \cdot m)$ where $k$ = bins, $m$ = iterations

• Space: $O(k)$; the IV loss of each candidate merge is screened from per-bin IV contributions and confirmed exactly

Key Innovations:

• Joint optimization: Global IV consideration (vs. local greedy)

• Smart violation repair: Severity-based prioritization

• Bidirectional merge evaluation: Forward vs. backward analysis

• Best solution tracking: Retains optimal intermediate states

• Adaptive thresholds: Context-aware monotonicity tolerance

Comparison with Related Methods:

Method	Optimization	Monotonicity	Speed
JEDI	Joint/Global	Adaptive	Medium
IVB	DP (Exact)	Enforced	Slow
GMB	Greedy/Local	Enforced	Fast
ChiMerge	Statistical	Optional	Fast

Value

A list containing the binning results with the following components:

id

Integer vector of bin identifiers (1-indexed)

bin

Character vector of bin labels (merged category names)

woe

Numeric vector of Weight of Evidence values per bin

iv

Numeric vector of Information Value contribution per bin

count

Integer vector of total observations per bin

count_pos

Integer vector of positive cases (target=1) per bin

count_neg

Integer vector of negative cases (target=0) per bin

total_iv

Numeric total Information Value of the binning solution

converged

Logical indicating algorithm convergence

iterations

Integer count of optimization iterations performed

References

Cover, T. M., & Thomas, J. A. (2006). Elements of Information Theory (2nd ed.). Wiley-Interscience. doi:10.1002/047174882X

Kullback, S. (1959). Information Theory and Statistics. Wiley.

Navas-Palencia, G. (2020). Optimal binning: mathematical programming formulation and solution approach. Expert Systems with Applications, 158, 113508. doi:10.1016/j.eswa.2020.113508

Good, I. J. (1965). The Estimation of Probabilities: An Essay on Modern Bayesian Methods. MIT Press.

Zeng, G. (2014). A necessary condition for a good binning algorithm in credit scoring. Applied Mathematical Sciences, 8(65), 3229-3242.

See Also

ob_categorical_ivb for Information Value DP optimization, ob_categorical_gmb for greedy merge binning, ob_categorical_dp for general dynamic programming, ob_categorical_cm for ChiMerge-based binning

Examples

# Example 1: Basic JEDI optimization
set.seed(42)
n_obs <- 1500

# Simulate employment types with risk gradient
employment <- c(
  "Permanent", "Contract", "Temporary", "SelfEmployed",
  "Unemployed", "Student", "Retired"
)
risk_rates <- c(0.03, 0.08, 0.15, 0.12, 0.35, 0.25, 0.10)

cat_feature <- sample(employment, n_obs,
  replace = TRUE,
  prob = c(0.35, 0.20, 0.15, 0.12, 0.08, 0.06, 0.04)
)
bin_target <- sapply(cat_feature, function(x) {
  rbinom(1, 1, risk_rates[which(employment == x)])
})

# Apply JEDI algorithm
result_jedi <- ob_categorical_jedi(
  cat_feature,
  bin_target,
  min_bins = 3,
  max_bins = 5
)

# Display results
print(data.frame(
  Bin = result_jedi$bin,
  WoE = round(result_jedi$woe, 3),
  IV = round(result_jedi$iv, 4),
  Count = result_jedi$count,
  EventRate = round(result_jedi$count_pos / result_jedi$count, 3)
))

cat("\nTotal IV (jointly optimized):", round(result_jedi$total_iv, 4), "\n")
cat("Converged:", result_jedi$converged, "\n")
cat("Iterations:", result_jedi$iterations, "\n")

# Example 2: Method comparison (JEDI vs alternatives)
set.seed(123)
n_obs_comp <- 2000

departments <- c(
  "Sales", "IT", "HR", "Finance", "Operations",
  "Marketing", "Legal", "R&D"
)
cat_feature_comp <- sample(departments, n_obs_comp, replace = TRUE)
bin_target_comp <- rbinom(n_obs_comp, 1, 0.12)

# JEDI (joint optimization)
result_jedi_comp <- ob_categorical_jedi(
  cat_feature_comp, bin_target_comp,
  min_bins = 3, max_bins = 4
)

# IVB (exact DP)
result_ivb_comp <- ob_categorical_ivb(
  cat_feature_comp, bin_target_comp,
  min_bins = 3, max_bins = 4
)

# GMB (greedy)
result_gmb_comp <- ob_categorical_gmb(
  cat_feature_comp, bin_target_comp,
  min_bins = 3, max_bins = 4
)

cat("\nMethod comparison (Total IV):\n")
cat(
  "  JEDI:", round(result_jedi_comp$total_iv, 4),
  "- converged:", result_jedi_comp$converged, "\n"
)
cat(
  "  IVB:", round(result_ivb_comp$total_iv, 4),
  "- converged:", result_ivb_comp$converged, "\n"
)
cat(
  "  GMB:", round(result_gmb_comp$total_iv, 4),
  "- converged:", result_gmb_comp$converged, "\n"
)

# Example 3: Bayesian smoothing with sparse data
set.seed(789)
n_obs_sparse <- 400

# Small sample with rare events
categories <- c("A", "B", "C", "D", "E", "F", "G")
cat_probs <- c(0.25, 0.20, 0.18, 0.15, 0.12, 0.07, 0.03)

cat_feature_sparse <- sample(categories, n_obs_sparse,
  replace = TRUE,
  prob = cat_probs
)
bin_target_sparse <- rbinom(n_obs_sparse, 1, 0.05) # 5% event rate

result_jedi_sparse <- ob_categorical_jedi(
  cat_feature_sparse,
  bin_target_sparse,
  min_bins = 2,
  max_bins = 4,
  bin_cutoff = 0.02
)

cat("\nBayesian smoothing (sparse data):\n")
cat("  Sample size:", n_obs_sparse, "\n")
cat("  Total events:", sum(bin_target_sparse), "\n")
cat("  Event rate:", round(mean(bin_target_sparse), 4), "\n")
cat("  Bins created:", length(result_jedi_sparse$bin), "\n\n")

# Show how smoothing prevents extreme WoE values
for (i in seq_along(result_jedi_sparse$bin)) {
  cat(sprintf(
    "  Bin %d: events=%d/%d, WoE=%.3f (smoothed)\n",
    i,
    result_jedi_sparse$count_pos[i],
    result_jedi_sparse$count[i],
    result_jedi_sparse$woe[i]
  ))
}

# Example 4: Violation detection and repair
set.seed(456)
n_obs_viol <- 1200

# Create feature with non-monotonic risk pattern
risk_categories <- c(
  "VeryLow", "Low", "MediumHigh", "Medium", # Intentional non-monotonic
  "High", "VeryHigh"
)
actual_risks <- c(0.02, 0.05, 0.20, 0.12, 0.25, 0.40) # MediumHigh > Medium

cat_feature_viol <- sample(risk_categories, n_obs_viol, replace = TRUE)
bin_target_viol <- sapply(cat_feature_viol, function(x) {
  rbinom(1, 1, actual_risks[which(risk_categories == x)])
})

result_jedi_viol <- ob_categorical_jedi(
  cat_feature_viol,
  bin_target_viol,
  min_bins = 3,
  max_bins = 5,
  max_iterations = 50
)

cat("\nViolation detection and repair:\n")
cat("  Original categories:", length(unique(cat_feature_viol)), "\n")
cat("  Final bins:", length(result_jedi_viol$bin), "\n")
cat("  Iterations to convergence:", result_jedi_viol$iterations, "\n")
cat("  Monotonicity achieved:", result_jedi_viol$converged, "\n\n")

# Check final WoE monotonicity
woe_diffs <- diff(result_jedi_viol$woe)
cat(
  "  WoE differences between bins:",
  paste(round(woe_diffs, 3), collapse = ", "), "\n"
)
cat("  All positive (monotonic):", all(woe_diffs >= -1e-6), "\n")

# Example 5: High cardinality performance
set.seed(321)
n_obs_hc <- 3000

# Simulate product categories (high cardinality)
products <- paste0("Product_", sprintf("%03d", 1:50))

cat_feature_hc <- sample(products, n_obs_hc, replace = TRUE)
bin_target_hc <- rbinom(n_obs_hc, 1, 0.08)

# Measure JEDI performance
time_jedi_hc <- system.time({
  result_jedi_hc <- ob_categorical_jedi(
    cat_feature_hc,
    bin_target_hc,
    min_bins = 4,
    max_bins = 7,
    max_n_prebins = 20,
    bin_cutoff = 0.02
  )
})

cat("\nHigh cardinality performance:\n")
cat("  Original categories:", length(unique(cat_feature_hc)), "\n")
cat("  Final bins:", length(result_jedi_hc$bin), "\n")
cat("  Execution time:", round(time_jedi_hc[3], 3), "seconds\n")
cat("  Total IV:", round(result_jedi_hc$total_iv, 4), "\n")
cat("  Converged:", result_jedi_hc$converged, "\n")

# Show merged categories
for (i in seq_along(result_jedi_hc$bin)) {
  n_merged <- length(strsplit(result_jedi_hc$bin[i], "%;%")[[1]])
  if (n_merged > 1) {
    cat(sprintf("  Bin %d: %d categories merged\n", i, n_merged))
  }
}

# Example 6: Convergence behavior
set.seed(555)
n_obs_conv <- 1000

education_levels <- c(
  "Elementary", "HighSchool", "Vocational",
  "Bachelor", "Master", "PhD"
)

cat_feature_conv <- sample(education_levels, n_obs_conv,
  replace = TRUE,
  prob = c(0.10, 0.30, 0.20, 0.25, 0.12, 0.03)
)
bin_target_conv <- rbinom(n_obs_conv, 1, 0.15)

# Test different convergence thresholds
thresholds <- c(1e-3, 1e-6, 1e-9)

for (thresh in thresholds) {
  result_conv <- ob_categorical_jedi(
    cat_feature_conv,
    bin_target_conv,
    min_bins = 2,
    max_bins = 4,
    convergence_threshold = thresh,
    max_iterations = 100
  )

  cat(sprintf("\nThreshold %.0e:\n", thresh))
  cat("  Final bins:", length(result_conv$bin), "\n")
  cat("  Total IV:", round(result_conv$total_iv, 4), "\n")
  cat("  Converged:", result_conv$converged, "\n")
  cat("  Iterations:", result_conv$iterations, "\n")
}

# Example 7: Missing value handling
set.seed(999)
cat_feature_na <- cat_feature
na_indices <- sample(n_obs, 75) # 5% missing
cat_feature_na[na_indices] <- NA

result_jedi_na <- ob_categorical_jedi(
  cat_feature_na,
  bin_target,
  min_bins = 3,
  max_bins = 5
)

# Locate NA bin
na_bin_idx <- grep("NA", result_jedi_na$bin)
if (length(na_bin_idx) > 0) {
  cat("\nMissing value treatment:\n")
  cat("  NA bin:", result_jedi_na$bin[na_bin_idx], "\n")
  cat("  NA count:", result_jedi_na$count[na_bin_idx], "\n")
  cat(
    "  NA event rate:",
    round(result_jedi_na$count_pos[na_bin_idx] /
      result_jedi_na$count[na_bin_idx], 3), "\n"
  )
  cat("  NA WoE:", round(result_jedi_na$woe[na_bin_idx], 3), "\n")
  cat("  NA IV contribution:", round(result_jedi_na$iv[na_bin_idx], 4), "\n")
}

---

Optimal Binning for Categorical Variables with Multinomial Target using JEDI-MWoE

Description

Performs supervised discretization of categorical variables for multinomial classification problems using the Joint Entropy-Driven Information Maximization with Multinomial Weight of Evidence (JEDI-MWoE) algorithm. This advanced method extends traditional binning to handle multi-class targets through specialized information-theoretic measures and intelligent optimization strategies.

Usage

ob_categorical_jedi_mwoe(
  feature,
  target,
  min_bins = 3,
  max_bins = 5,
  bin_cutoff = 0.05,
  max_n_prebins = 20,
  bin_separator = "%;%",
  convergence_threshold = 1e-06,
  max_iterations = 1000
)

Arguments

feature	A character vector or factor representing the categorical predictor variable to be binned. Missing values are automatically converted to the special category "NA".
target	An integer vector representing the multinomial outcome variable with consecutive integer classes starting from 0 (e.g., 0, 1, 2, ...). Missing values are not permitted. Must contain at least 2 distinct classes.
min_bins	Integer. Minimum number of bins to produce. Must be >= 1. The algorithm will not merge below this threshold. Defaults to 3.
max_bins	Integer. Maximum number of bins to produce. Must be >= min_bins. The algorithm iteratively merges until this constraint is satisfied. Defaults to 5.
bin_cutoff	Numeric. Minimum proportion of total observations required for a category to remain separate during initialization. Categories below this threshold are pre-merged. Must be in (0, 1). Defaults to 0.05.
max_n_prebins	Integer. Maximum number of initial bins before the main optimization phase. Controls computational complexity for high-cardinality features. Must be >= min_bins. Defaults to 20.
bin_separator	Character string used to concatenate category names when multiple categories are merged into a single bin. Defaults to "%;%".
convergence_threshold	Numeric. Convergence tolerance based on Information Value change between iterations. Algorithm stops when $\max_c |\Delta IV_c| <$ convergence_threshold across all classes. Must be > 0. Defaults to 1e-6.
max_iterations	Integer. Maximum number of optimization iterations. Prevents infinite loops in edge cases. Must be > 0. Defaults to 1000.

Details

The JEDI-MWoE (Joint Entropy-Driven Information Maximization with Multinomial Weight of Evidence) algorithm extends traditional optimal binning to handle multinomial classification problems by computing class-specific information measures and optimizing joint information content across all target classes.

Algorithm Workflow:

1. Input validation and preprocessing (multinomial target verification)

2. Initial bin creation (one category per bin)

3. Rare category merging (frequencies < bin_cutoff)

4. Pre-bin limitation via statistical similarity merging

5. Main optimization loop with alternating strategies:

   • Jensen-Shannon divergence minimization for similar bin detection

   • Adjacent bin merging with minimal information loss

   • Class-wise monotonicity violation detection and repair

   • Convergence monitoring across all classes

6. Final constraint satisfaction (max_bins enforcement)

7. Laplace-smoothed metric computation

Multinomial Weight of Evidence (M-WoE):

For a bin $B$ and class $c$, the Multinomial Weight of Evidence is:

$M\text{-}WoE_{B,c} = \ln\left(\frac{P(c|B) + \alpha}{P(\neg c|B) + \alpha}\right)$

where:

• $P(c|B) = \frac{n_{B,c}}{n_B}$ is the class probability in bin $B$

• $P(\neg c|B) = \frac{\sum_{k \neq c} n_{B,k}}{\sum_{k \neq c} n_k}$ is the combined probability of all other classes in bin $B$

• $\alpha = 0.5$ is the Laplace smoothing parameter

Information Value Extension:

The Information Value for class $c$ in bin $B$ is:

$IV_{B,c} = \left(P(c|B) - P(\neg c|B)\right) \times M\text{-}WoE_{B,c}$

Total IV for class $c$ across all bins:

$IV_c = \sum_{B} |IV_{B,c}|$

Statistical Similarity Measure:

JEDI-MWoE uses Jensen-Shannon divergence to identify similar bins for merging:

$JS(P_B, P_{B'}) = \frac{1}{2} \sum_{c=0}^{C-1} \left[ P(c|B) \ln\frac{P(c|B)}{M(c)} + P(c|B') \ln\frac{P(c|B')}{M(c)} \right]$

where $M(c) = \frac{1}{2}[P(c|B) + P(c|B')]$ is the average distribution.

Class-wise Monotonicity Enforcement:

For each class $c$, the algorithm enforces WoE monotonicity by detecting violations (peaks and valleys) and repairing them through strategic bin merges:

• Peak: $M\text{-}WoE_{i-1,c} < M\text{-}WoE_{i,c} > M\text{-}WoE_{i+1,c}$

• Valley: $M\text{-}WoE_{i-1,c} > M\text{-}WoE_{i,c} < M\text{-}WoE_{i+1,c}$

Violation severity is measured as:

$severity_{i,c} = \max\{|M\text{-}WoE_{i,c} - M\text{-}WoE_{i-1,c}|, |M\text{-}WoE_{i,c} - M\text{-}WoE_{i+1,c}|\}$

Alternating Optimization Strategies:

The algorithm alternates between two merging strategies to balance global similarity and local information preservation:

1. Divergence-based: Merge bins with minimum JS divergence

2. IV-preserving: Merge adjacent bins with minimum information loss

Laplace Smoothing:

To ensure numerical stability and prevent undefined logarithms, all probability estimates are smoothed with a Laplace prior:

$P_{smooth}(c|B) = \frac{n_{B,c} + \alpha}{n_B + \alpha \cdot C}$

where $C$ is the number of classes and $\alpha = 0.5$.

Computational Complexity:

• Time: $O(k^2 \cdot C \cdot m)$ where $k$ = bins, $C$ = classes, $m$ = iterations

• Space: $O(k \cdot C)$; during pre-binning each bin keeps its most similar partner, so a merge rescans only the affected bins

Key Innovations:

• Multinomial extension: Generalizes WoE/IV to multi-class problems

• Joint optimization: Simultaneously optimizes across all classes

• Alternating strategies: Balances global similarity and local preservation

• Class-wise monotonicity: Enforces meaningful ordering for each class

• Statistical similarity: Uses Jensen-Shannon divergence for merging

Comparison with Binary Methods:

Aspect	Binary	Multinomial	Extension
Target Classes	2	C >= 2	One-vs-rest
WoE Definition	$\ln(p/n)$	$\ln(P(c)/P(\neg c))$	Class-specific
IV Aggregation	Sum	Per-class	Vector-valued
Similarity	Chi-square	Jensen-Shannon	Distribution-based
Monotonicity	Global	Per-class	Multi-constraint

Value

A list containing the multinomial binning results with the following components:

id

Integer vector of bin identifiers (1-indexed)

bin

Character vector of bin labels (merged category names)

woe

Numeric matrix of Multinomial Weight of Evidence values with dimensions (bins × classes)

iv

Numeric matrix of Information Value contributions with dimensions (bins × classes)

count

Integer vector of total observations per bin

class_counts

Integer matrix of observations per class per bin with dimensions (bins × classes)

class_rates

Numeric matrix of class proportions per bin with dimensions (bins × classes)

converged

Logical indicating algorithm convergence

iterations

Integer count of optimization iterations performed

n_classes

Integer number of target classes

total_iv

Numeric vector of total Information Value per class

References

Siddiqi, N. (2006). Credit Risk Scorecards: Developing and Implementing Intelligent Credit Scoring. Wiley.

Lin, J. (1991). Divergence measures based on the Shannon entropy. IEEE Transactions on Information Theory, 37(1), 145-151. doi:10.1109/18.61115

Cover, T. M., & Thomas, J. A. (2006). Elements of Information Theory (2nd ed.). Wiley-Interscience. doi:10.1002/047174882X

Navas-Palencia, G. (2020). Optimal binning: mathematical programming formulation and solution approach. Expert Systems with Applications, 158, 113508. doi:10.1016/j.eswa.2020.113508

Good, I. J. (1965). The Estimation of Probabilities: An Essay on Modern Bayesian Methods. MIT Press.

See Also

ob_categorical_jedi for binary target JEDI algorithm, ob_categorical_ivb for binary Information Value DP optimization, ob_categorical_dp for general dynamic programming binning

Examples

# Example 1: Basic multinomial JEDI-MWoE optimization
set.seed(42)
n_obs <- 1500

# Simulate customer segments with 3 risk categories
segments <- c("Premium", "Standard", "Basic", "Economy")
# Class probabilities: 0=LowRisk, 1=MediumRisk, 2=HighRisk
risk_probs <- list(
  Premium = c(0.80, 0.15, 0.05), # Mostly LowRisk
  Standard = c(0.40, 0.40, 0.20), # Balanced
  Basic = c(0.15, 0.35, 0.50), # Mostly HighRisk
  Economy = c(0.05, 0.20, 0.75) # Almost all HighRisk
)

cat_feature <- sample(segments, n_obs,
  replace = TRUE,
  prob = c(0.25, 0.35, 0.25, 0.15)
)

# Generate multinomial target (classes 0, 1, 2)
multinom_target <- sapply(cat_feature, function(segment) {
  probs <- risk_probs[[segment]]
  sample(0:2, 1, prob = probs)
})

# Apply JEDI-MWoE algorithm
result_mwoe <- ob_categorical_jedi_mwoe(
  cat_feature,
  multinom_target,
  min_bins = 2,
  max_bins = 3
)

# Display results
cat("Number of classes:", result_mwoe$n_classes, "\n")
cat("Number of bins:", length(result_mwoe$bin), "\n")
cat("Converged:", result_mwoe$converged, "\n")
cat("Iterations:", result_mwoe$iterations, "\n\n")

# Show bin details
for (i in seq_along(result_mwoe$bin)) {
  cat(sprintf("Bin %d (%s):\n", i, result_mwoe$bin[i]))
  cat("  Total count:", result_mwoe$count[i], "\n")
  cat("  Class counts:", result_mwoe$class_counts[i, ], "\n")
  cat("  Class rates:", round(result_mwoe$class_rates[i, ], 3), "\n")

  # Show WoE and IV for each class
  for (class in 0:(result_mwoe$n_classes - 1)) {
    cat(sprintf(
      "  Class %d: WoE=%.3f, IV=%.4f\n",
      class,
      result_mwoe$woe[i, class + 1], # R is 1-indexed
      result_mwoe$iv[i, class + 1]
    ))
  }
  cat("\n")
}

# Show total IV per class
cat("Total IV per class:\n")
for (class in 0:(result_mwoe$n_classes - 1)) {
  cat(sprintf("  Class %d: %.4f\n", class, result_mwoe$total_iv[class + 1]))
}

# Example 2: High-cardinality multinomial problem
set.seed(123)
n_obs_hc <- 2000

# Simulate product categories with 4 classes
products <- paste0("Product_", LETTERS[1:15])
cat_feature_hc <- sample(products, n_obs_hc, replace = TRUE)

# Generate 4-class target
multinom_target_hc <- sample(0:3, n_obs_hc,
  replace = TRUE,
  prob = c(0.3, 0.25, 0.25, 0.2)
)

result_mwoe_hc <- ob_categorical_jedi_mwoe(
  cat_feature_hc,
  multinom_target_hc,
  min_bins = 3,
  max_bins = 6,
  max_n_prebins = 15,
  bin_cutoff = 0.03
)

cat("\nHigh-cardinality example:\n")
cat("Original categories:", length(unique(cat_feature_hc)), "\n")
cat("Final bins:", length(result_mwoe_hc$bin), "\n")
cat("Classes:", result_mwoe_hc$n_classes, "\n")
cat("Converged:", result_mwoe_hc$converged, "\n\n")

# Show merged categories
for (i in seq_along(result_mwoe_hc$bin)) {
  n_merged <- length(strsplit(result_mwoe_hc$bin[i], "%;%")[[1]])
  if (n_merged > 1) {
    cat(sprintf("Bin %d: %d categories merged\n", i, n_merged))
  }
}

# Example 3: Laplace smoothing demonstration
set.seed(789)
n_obs_smooth <- 500

# Small sample with sparse categories
categories <- c("A", "B", "C", "D", "E")
cat_feature_smooth <- sample(categories, n_obs_smooth,
  replace = TRUE,
  prob = c(0.3, 0.25, 0.2, 0.15, 0.1)
)

# Generate 3-class target with class imbalance
multinom_target_smooth <- sample(0:2, n_obs_smooth,
  replace = TRUE,
  prob = c(0.6, 0.3, 0.1)
) # Class 0 dominant

result_mwoe_smooth <- ob_categorical_jedi_mwoe(
  cat_feature_smooth,
  multinom_target_smooth,
  min_bins = 2,
  max_bins = 4,
  bin_cutoff = 0.02
)

cat("\nLaplace smoothing demonstration:\n")
cat("Sample size:", n_obs_smooth, "\n")
cat("Classes:", result_mwoe_smooth$n_classes, "\n")
cat("Event distribution:", table(multinom_target_smooth), "\n\n")

# Show how smoothing prevents extreme values
for (i in seq_along(result_mwoe_smooth$bin)) {
  cat(sprintf("Bin %d (%s):\n", i, result_mwoe_smooth$bin[i]))
  cat("  Counts per class:", result_mwoe_smooth$class_counts[i, ], "\n")
  cat("  WoE values:", round(result_mwoe_smooth$woe[i, ], 3), "\n")
  cat("  Note: Extreme WoE values prevented by Laplace smoothing\n\n")
}

# Example 4: Class-wise monotonicity
set.seed(456)
n_obs_mono <- 1200

# Feature with predictable class patterns
education <- c("PhD", "Master", "Bachelor", "College", "HighSchool")
# Each education level has a preferred class
preferred_classes <- c(2, 1, 0, 1, 2) # PhD→High(2), Bachelor→Low(0), etc.

cat_feature_mono <- sample(education, n_obs_mono, replace = TRUE)

# Generate target with preferred class bias
multinom_target_mono <- sapply(cat_feature_mono, function(edu) {
  pref_class <- preferred_classes[which(education == edu)]
  # Create probability vector with preference
  probs <- rep(0.1, 3) # Base probability
  probs[pref_class + 1] <- 0.8 # Preferred class gets high probability
  sample(0:2, 1, prob = probs / sum(probs))
})

result_mwoe_mono <- ob_categorical_jedi_mwoe(
  cat_feature_mono,
  multinom_target_mono,
  min_bins = 3,
  max_bins = 5
)

cat("Class-wise monotonicity example:\n")
cat("Education levels:", length(education), "\n")
cat("Final bins:", length(result_mwoe_mono$bin), "\n")
cat("Iterations:", result_mwoe_mono$iterations, "\n\n")

# Check monotonicity for each class
for (class in 0:(result_mwoe_mono$n_classes - 1)) {
  woe_series <- result_mwoe_mono$woe[, class + 1]
  diffs <- diff(woe_series)
  is_mono <- all(diffs >= -1e-6) || all(diffs <= 1e-6)
  cat(sprintf("Class %d WoE monotonic: %s\n", class, is_mono))
  cat(sprintf("  WoE series: %s\n", paste(round(woe_series, 3), collapse = ", ")))
}

# Example 5: Missing value handling
set.seed(321)
cat_feature_na <- cat_feature
na_indices <- sample(n_obs, 75) # 5% missing
cat_feature_na[na_indices] <- NA

result_mwoe_na <- ob_categorical_jedi_mwoe(
  cat_feature_na,
  multinom_target,
  min_bins = 2,
  max_bins = 3
)

# Locate missing value bin
missing_bin_idx <- grep("NA", result_mwoe_na$bin)
if (length(missing_bin_idx) > 0) {
  cat("\nMissing value handling:\n")
  cat("Missing value bin:", result_mwoe_na$bin[missing_bin_idx], "\n")
  cat("Missing value count:", result_mwoe_na$count[missing_bin_idx], "\n")
  cat(
    "Class distribution in missing bin:",
    result_mwoe_na$class_counts[missing_bin_idx, ], "\n"
  )

  # Show class rates for missing bin
  for (class in 0:(result_mwoe_na$n_classes - 1)) {
    cat(sprintf(
      "  Class %d rate: %.3f\n", class,
      result_mwoe_na$class_rates[missing_bin_idx, class + 1]
    ))
  }
}

# Example 6: Convergence behavior
set.seed(555)
n_obs_conv <- 1000

departments <- c("Sales", "IT", "HR", "Finance", "Operations")
cat_feature_conv <- sample(departments, n_obs_conv, replace = TRUE)
multinom_target_conv <- sample(0:2, n_obs_conv, replace = TRUE)

# Test different convergence thresholds
thresholds <- c(1e-3, 1e-6, 1e-9)

for (thresh in thresholds) {
  result_conv <- ob_categorical_jedi_mwoe(
    cat_feature_conv,
    multinom_target_conv,
    min_bins = 2,
    max_bins = 4,
    convergence_threshold = thresh,
    max_iterations = 100
  )

  cat(sprintf("\nThreshold %.0e:\n", thresh))
  cat("  Final bins:", length(result_conv$bin), "\n")
  cat("  Converged:", result_conv$converged, "\n")
  cat("  Iterations:", result_conv$iterations, "\n")

  # Show total IV for each class
  cat("  Total IV per class:")
  for (class in 0:(result_conv$n_classes - 1)) {
    cat(sprintf(" %.4f", result_conv$total_iv[class + 1]))
  }
  cat("\n")
}

---