Machine Learning for Language Research

A logistic model draws a straight boundary between two groups and reports one coefficient per feature. The coefficient's sign says which group a feature favors; its exponential is an odds ratio, the interpretable currency of much applied-linguistics modeling.

An SVM chooses the boundary that maximizes the margin to the nearest points; only those support vectors matter. The C parameter trades margin width against training errors, and the RBF kernel bends the boundary for non-linear structure.

A single tree carves the feature space with axis-aligned splits and overfits easily. A random forest fits many trees on bootstrap resamples and averages them: variance drops and the boundary smooths as trees are added.

Boosting builds the model in rounds: each new depth-2 tree is fitted to the residuals of the current ensemble and added with learning rate η, its leaf weights shrunk by regularization λ. Watch the residuals drain and the held-out error turn when it overfits.

SHAP assigns each feature a fair share of one specific prediction, summing from the model's base rate to the final output. The waterfall below explains one synthetic learner text: which features pushed the prediction up, which pulled it down.

LIME perturbs the input, watches how a black-box model reacts, and fits a small local surrogate whose weights are readable. For text, the perturbations are word deletions, so each word gets a local importance.

What do contextual embeddings actually encode? The same word in different contexts gets different vectors, so word senses separate; whole sentences become single vectors whose cosine similarity tracks paraphrase. All values precomputed offline; no models run on this page.