The Persistence-State Framework: A Comprehensive Tutorial

00 How to Use This Tutorial

This tutorial is designed for three audiences, and each can take a different path through the material.

If you are a researcher who wants to understand the theoretical foundations and design an empirical study, read Parts I and IV carefully, skim Parts II and III, and study the case studies in Part V.

If you are a data scientist who wants to apply the library to institutional data, skim Part I, read Parts II and III carefully, work through Part IV hands-on, and refer to Part V for specific applications.

If you are a student learning computational learning analytics for the first time, read the tutorial in order, run every code example, and complete the exercises in Part VII.

01 Why Stop-Out Is Not Attrition

1.1 A Scene from Practice

Imagine you are an academic advisor at an open university. On your dashboard this morning, three students have been flagged as "at risk of dropout" by the institution's learning analytics system. The flag is triggered by a simple rule: no learning management system (LMS) login for 30 days.

Student A registered four months ago, completed two assignments, then stopped logging in. Last week she requested a two-month extension because of a family emergency. She intends to resume.

Student B registered for the same course, completed one assignment, then stopped logging in. She has not contacted anyone. No extension request on file, no future registration, no advisor notes.

Student C registered, completed two assignments, and formally withdrew eight weeks ago.

The dashboard treats all three as identical. But they are not. Student A needs a supportive check-in and a processed extension. Student B needs substantive outreach. Student C is gone and does not need an intervention at all.

This is the problem the Persistence-State Framework (PSF) addresses. A binary dropout model collapses three fundamentally different trajectories into a single category, and in doing so it overestimates attrition, wastes advising resources, and may harm the very students it intends to help.

1.2 Why the Problem Is Structural, Not Just Technical

It is tempting to treat this as a model-selection problem: "our classifier isn't good enough, let's try a better one." But the issue is deeper. The binary dropout model is not simply inaccurate; it is asking the wrong question.

In a cohort-based program with fixed term schedules, inactivity during a term is genuinely predictive of dropout. In a self-paced program, that assumption breaks down. Students register at any time, study at their own pace, and may legitimately pause for weeks or months. A student who has not logged in for 30 days may be:

• Traveling for work and reading a printed textbook.
• Preparing for a proctored exam and studying offline.
• Waiting for an extension request to be processed.
• Pausing because of illness, caregiving, or a job transition.
• Reconsidering whether to continue.

Only the last case is genuinely about withdrawal. The others are ordinary features of self-paced study. A model that treats them all as dropout is not making a small error; it is systematically misreading the population.

1.3 The Cost of Getting It Wrong

Overestimated attrition. If a program reports a 40% dropout rate, but half of those "dropouts" later return, the true attrition rate is 20%. Institutional policy and accreditation reporting built on the inflated figure will be misleading.

Misdirected interventions. Advisors have limited time. If the system flags 200 students per week and only 60 are genuinely at risk, the advisor's attention is diluted.

Unwanted intrusion. Adult learners have complex lives. A phone call asking "why haven't you logged in?" can feel patronizing when the student is simply studying at their own pace. Trust is fragile, and surveillance erodes it.

Alienation. If students learn that pausing triggers a dropout alert, they may avoid pausing even when they should — by remaining nominally enrolled while making no progress rather than formally stopping out.

1.4 What We Need Instead

The PSF proposes a different way of thinking about persistence. Rather than asking "is this student active?", it asks "what persistence state is this student in, and what is the probability they will return?"

1. Persistence becomes a state, not a binary label. Students move among states over time, and the model tracks their trajectories.
2. Intent becomes a first-class concept. The distinction between "temporarily paused and intending to return" and "paused without intent to return" is modeled explicitly.
3. Uncertainty is represented. Instead of forcing a confident label, the model reports probabilities that can be updated as new observations arrive.
4. Time matters. The model tracks the trajectory of each student, not just their current snapshot.

02 The Five-State Model

2.1 The States

The PSF represents persistence as a dynamic process over five states. These states are mutually exclusive and exhaustive for the purpose of persistence modeling, though a student can transition among them over time.

2.2 The Critical Distinction: S₃ vs. S₄

The whole framework hinges on distinguishing S₃ from S₄. In an inactivity-threshold model, both would be labeled "dropped out." In the PSF, they are distinct.

What distinguishes them is Intent to Return. A student in S₃ intends to resume study; a student in S₄ does not. Intent is latent — it cannot be observed directly — but it can be inferred from observable proxies.

2.3 State Transitions

                    ┌──────────────┐
                    │  Registered  │
                    │      S₁      │
                    └──────┬───────┘
                           │ first activity
                           ▼
                    ┌──────────────┐
              ┌─────│  Progressing │─────┐
              │     │      S₂      │     │ completion
              │     └──────┬───────┘     ▼
              │            │      ┌──────────────┐
              │            │      │  Completed   │
              │            │      │      S₅      │
              │            │      └──────────────┘
              │            │ inactivity
              │            ▼
              │     ┌──────────────┐
              │     │  Stopped-Out │
              │     │      S₃      │
              │     └──────┬───────┘
              │     ┌──────┴──────┐
              │     │ Iₜ=1        │ Iₜ=0
              │     ▼             ▼
              │  ┌──────────────┐ ┌──────────────┐
              └─▶│  Progressing │ │  Withdrawn   │
                 │      S₂      │ │      S₄      │
                 └──────────────┘ └──────────────┘

Three structural features are worth noting. S₁ has limited outgoing transitions. From Registered, a student can only remain Registered or transition to Progressing. S₂ can go anywhere meaningful. From Progressing, a student can remain in S₂, stop out to S₃, withdraw to S₄, or complete to S₅. S₄ and S₅ are absorbing. Once Withdrawn or Completed, the student remains in that state (within the course-level model).

2.4 The Role of Intent

From S₃, the transition probabilities depend on Intent to Return. A student with Iₜ = 1 has a meaningful probability of returning to S₂ and a lower probability of withdrawing to S₄. A student with Iₜ = 0 has essentially no probability of returning and a higher probability of withdrawing.

This is the mechanism by which the framework distinguishes stop-out from attrition.

2.5 Why Five States, Not More or Fewer

A three-state model would collapse Registered and Progressing into a single "Enrolled" state, and would collapse Completed and Withdrawn into a single "Exited" state. But these distinctions matter: a completed student is not at risk, while a withdrawn student is; a registered student who has not engaged is in a different position from a progressing one. Five states is the minimum that preserves these operational distinctions.

A nine-state model with a full engagement taxonomy would add detail not needed for the persistence question. Parsimony matters: fewer states means fewer parameters, easier fitting, and more robust generalization.

03 Intent to Return

3.1 What Intent Is

Intent to Return (Iₜ ∈ {0, 1}) is a latent variable representing a student's commitment to continue their studies. At any time t, a student in S₃ either intends to return (Iₜ = 1) or does not (Iₜ = 0).

Intent is not the same as activity. A student may be inactive but still intend to return. Another may be active but already intend to withdraw. Intent is about commitment, not behavior.

3.2 Why Intent Is Latent

Intent cannot be observed directly. What we can observe are signals that correlate with intent:

• Extension requests. A student who requests an extension is signaling that they plan to continue.
• Tutor contacts. A student who contacts their tutor — even to say "I'm struggling" — is engaging.
• Future registrations. A student who registers for another course while paused is clearly planning to continue.
• Advising communications. A student who emails an advisor about their situation is signaling engagement.
• Submission of partially completed work. Even partial submissions indicate an intent to eventually complete.

3.3 Formalizing Intent

In the model, intent evolves over time. Its dynamics are:

Pr(I_{t+1} = 1 | I_t = 1) = φ₁
Pr(I_{t+1} = 1 | I_t = 0) = φ₀

Here φ₁ is the probability that a student who intends to return at time t still intends to return at time t+1. This is high (close to 1) for most students. φ₀ is the probability that a student who does not intend to return comes to intend to return. This is typically low, though not zero.

3.4 The Role of Intent in Transitions

When Iₜ = 1, a Stopped-Out student has a probability λ_return(t) of returning to S₂, a probability λ_{withdraw|intent}(t) of withdrawing, and a probability of remaining in S₃ given by 1 − λ_return(t) − λ_{withdraw|intent}(t).

When Iₜ = 0, the hazard of return is essentially zero and the hazard of withdrawal is higher.

3.5 Construct Validity

For Intent to Return, the construct is grounded in the persistence literature. Tinto (1993) distinguished between temporary and permanent departure. Bean and Metzner (1985) emphasized the role of external commitments. Kember (1995) emphasized learner autonomy and institutional support. Intent to Return is a parsimonious construct capturing the student's commitment to continue, distinct from their current activity level.

Construct validity can be assessed empirically: if extension requests, tutor contacts, and future registrations all predict return (controlling for activity recency), the construct is validated.

3.6 The Most Important Pre-Modeling Analysis

Before fitting any model, run this analysis:

import numpy as np

# Among students who stopped out, compare return rates
# for those with and without intent proxies
stopped_out = (labels == 2) | (labels == 1)
has_extension = np.array([r.extension_active_after is not None for r in records])

return_rate_with = (labels[stopped_out & has_extension] == 2).mean()
return_rate_without = (labels[stopped_out & ~has_extension] == 2).mean()

print(f"Return rate with extension: {return_rate_with:.3f}")
print(f"Return rate without extension: {return_rate_without:.3f}")

04 Operational Definitions and Validation

4.1 The Ground-Truth Problem

If we use the model to infer withdrawal, and then use inferred withdrawal as the ground truth for training, we have circular reasoning. The PSF avoids this by defining operational labels independently of the model. These labels are derived from administrative records and observation windows, not from the model's inferences.

4.2 Operational Definitions

4.3 The Four Resulting Labels

The censored label is important. A student who registered three months ago and is currently progressing cannot be classified as stop-out or withdrawn because there is not yet enough evidence.

4.4 The Observation Window and Pause Threshold

The observation window W determines how long we wait after a pause before classifying the student as Withdrawn. A short window (e.g., 12 months) produces a stricter definition; a long window (e.g., 36 months) is more permissive and delays classification.

The pause threshold T_pause determines how much inactivity is required before a student is considered Stopped-Out. For AU's learner-paced courses, W = 24 months and T_pause = 60 days are reasonable defaults.

4.5 Sensitivity Analysis

for W in [12, 24, 36]:
    for T_pause in [1, 2, 3]:  # months
        labels_w = generate_operational_labels(
            records, W=W, T_pause=T_pause, observation_end=36,
        )
        print(f"W={W}, T_pause={T_pause}: "
              f"Withdrawn={(labels_w == 1).sum()}, "
              f"Stopped-Out={(labels_w == 2).sum()}")

If the model's performance changes materially across these settings, report the sensitivity, and do not present any single setting as definitive.

05 From Theory to Computation

5.1 The HMM Formulation

The five-state model with latent intent can be implemented as a Hidden Markov Model (HMM). For the PSF, we use six joint states by splitting S₃ into two variants:

5.2 The Joint Likelihood

Pr(Z_{1:T}, I_{1:T}, Y_{1:T}) =
    Pr(Z_1) Pr(I_1)
    ∏_{t=1}^{T-1} Pr(Z_{t+1} | Z_t, I_t) Pr(I_{t+1} | I_t, Z_t)
    ∏_{t=1}^{T}   Pr(Y_t | Z_t, I_t)

Pr(Z₁)Pr(I₁) is the initial state distribution. Pr(Z_t+1 | Z_t, I_t) is the transition model. Pr(I_t+1 | I_t, Z_t) is the intent evolution. Pr(Y_t | Z_t, I_t) is the emission model.

5.3 The Transition Model in Detail

From S₁ (Registered): stay in S₁, or transition to S₂.

From S₂ (Progressing): stay in S₂, stop out to S₃(I=0) or S₃(I=1), withdraw to S₄, or complete to S₅.

From S₃(I=0): stay in S₃(I=0), transition to S₃(I=1) (developing intent), or withdraw to S₄.

From S₃(I=1): stay in S₃(I=1), return to S₂, or withdraw to S₄.

From S₄ and S₅: absorbing.

5.4 The Emission Model

For continuous features, we use a Gaussian emission:

Pr(Y_t = y | Z_t = k) = N(y | μ_k, Σ_k)

In practice, the emission model can be more flexible. For count data, a Poisson emission is more appropriate. For binary data, Bernoulli. For ordinal data, ordered probit. The library currently implements Gaussian emissions, with other families available as extensions.

5.5 Inference

The smoothing posterior γ_t(k) = Pr(Z_t = k | Y_1:T) is computed via the forward-backward algorithm. All computations are done in log space for numerical stability.

5.6 Parameter Estimation

Parameters are estimated using the Expectation-Maximization (EM) algorithm, also known as Baum-Welch for HMMs. EM alternates between an E-step (compute posterior over latent states) and an M-step (update parameters). EM monotonically increases the observed-data log-likelihood but can converge to a local optimum. This motivates multi-restart fitting.

5.7 The Role of Structural Constraints

Forbidden transitions, absorbing states, and symmetry/asymmetry constraints are encoded as a structural mask that is applied at each iteration of EM. They reduce the effective number of free parameters and improve identifiability.

06 Installation and Orientation

6.1 Installing the Library

git clone https://github.com/hongxueharriswang/psf_lib.git
cd psf_lib
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install -e ".[viz,dev]"

6.2 Verifying the Installation

import psf
print(psf.__version__)  # 0.1.0

from psf import PSFModel, simulate_cohort, baum_welch
from psf.states import IDX, JOINT_STATE_LABELS
print("Library loaded successfully.")

6.3 The Package Structure

psf/
├── states.py        # State definitions, structural constraints
├── model.py         # PSFModel class (HMM parameters)
├── inference.py     # Forward-backward, Viterbi, particle filter
├── learning.py      # Baum-Welch (EM) parameter estimation
├── survival.py      # Competing-risk survival models
├── labels.py        # Operational label generation
├── baselines.py     # Baseline models for comparison
├── evaluation.py    # Metrics and model comparison
├── features.py      # Feature extraction
├── simulation.py    # Synthetic data generation
├── viz.py           # Plotting utilities
└── utils.py         # Numerical utilities

6.4 Running the Reference Simulation

python examples/01_simulation_study.py

This runs a complete end-to-end pipeline: builds a true model, simulates a cohort, fits the model via multi-restart EM, evaluates parameter and state recovery, compares against baselines, and generates diagnostic figures in ./figures/.

07 The PSFModel Object

7.1 Creating a Model

from psf import PSFModel

model = PSFModel(n_features=6)
print(model)
# PSFModel(K=6, D=6, n_params=92)

7.2 The Model's Components

model.pi          # Initial state distribution (K,)
model.A           # Transition matrix (K, K)
model.mu          # Emission means (K, D)
model.log_sigma   # Emission log standard deviations (K, D)

7.3 The Structural Constraints

from psf.states import ALLOWED_TRANSITIONS
import pandas as pd
from psf.states import JOINT_STATE_LABELS

print(pd.DataFrame(
    ALLOWED_TRANSITIONS.astype(int),
    index=JOINT_STATE_LABELS,
    columns=JOINT_STATE_LABELS,
))

7.4 The Emission Model

import numpy as np
y = np.random.randn(10, 6)
log_b = model.log_emission_prob(y)
print(log_b.shape)  # (10, 6)

# With missing features
mask = np.ones_like(y, dtype=bool)
mask[0, 2] = False
log_b = model.log_emission_prob(y, mask)

08 Fitting with Baum-Welch

8.1 Basic Fitting

from psf import baum_welch

model = PSFModel(n_features=6)
baum_welch(model, obs, masks, n_iter=30, verbose=True)

8.2 What Happens During Fitting

At each EM iteration: E-step (compute posteriors), M-step (update parameters), constraint application (mask + absorbing states), and log-likelihood check. The per-iteration log-likelihood is recorded as model.em_history_.

8.3 The Importance of Multi-Restart

EM converges to a local optimum. In the reference simulation, the difference between the best and worst restart is approximately 980 log-likelihood units. The best restart achieves AUC 0.998; the worst, 0.685. Always use at least 3–5 restarts.

import numpy as np
from psf import PSFModel, baum_welch, forward_backward

def fit_with_restarts(obs, masks, n_restarts=5, n_iter=30):
    n_features = obs[0].shape[1]
    best_model, best_ll, best_seed = None, -np.inf, -1

    for seed in range(n_restarts):
        rng = np.random.default_rng(seed)
        model = PSFModel(n_features=n_features)
        model.mu = rng.normal(size=model.mu.shape) * 0.5
        model.log_sigma = np.zeros_like(model.log_sigma)
        baum_welch(model, obs, masks, n_iter=n_iter)

        ll = sum(forward_backward(model, o, m)["log_lik"]
                 for o, m in zip(obs, masks))
        if ll > best_ll:
            best_model, best_ll, best_seed = model, ll, seed

    return best_model

8.4 Convergence Diagnostics

from psf import viz
viz.plot_em_convergence(
    {"restart 0": model.em_history_},
    save_path="figures/em_convergence.png",
)

09 Inference

9.1 Forward-Backward

from psf import forward_backward

seq, mask = obs[0], masks[0]
out = forward_backward(model, seq, mask)
gamma = out["gamma"]     # (T, K)
log_lik = out["log_lik"] # scalar

9.2 Collapsing the Posterior

from psf import collapse_posterior
persistence = collapse_posterior(gamma)  # (T, 5)

9.3 Extracting the Intent Posterior

from psf import intent_posterior
intent = intent_posterior(gamma)  # (T,) with NaN outside S₃

9.4 Viterbi Decoding

from psf import viterbi
path = viterbi(model, seq, mask)

9.5 Particle Filtering for Online Tracking

from psf import particle_filter
out = particle_filter(model, seq, mask, n_particles=1000)
filtered = out["filtered_probs"]
ess = out["ess"]

9.6 Return and Withdrawal Probabilities

from psf import return_probabilities, withdrawal_probabilities
p_return   = return_probabilities(model, gamma, horizon=6)
p_withdraw = withdrawal_probabilities(model, gamma, horizon=6)

10 Evaluation

10.1 The Evaluation Pipeline

1. Generate operational labels.
2. Fit the model on a training set and evaluate on a held-out test set.
3. Compute predictions for each test-set student.
4. Compare against baselines using standard metrics.

10.2 Generating Labels

from psf import generate_operational_labels

labels = generate_operational_labels(
    records, W=24, T_pause=2, observation_end=36,
)
print(f"Completed:   {(labels == 0).sum()}")
print(f"Withdrawn:   {(labels == 1).sum()}")
print(f"Stopped-Out: {(labels == 2).sum()}")
print(f"Censored:    {(labels == 3).sum()}")

10.3 Computing Predictions

import numpy as np
from psf import forward_backward
from psf.states import IDX

psf_prob = []
for seq, mask in zip(test_obs, test_masks):
    out = forward_backward(model, seq, mask)
    g = out["gamma"]
    p_terminal = g[-1, IDX.S4] + g[-1, IDX.S3_NO_INTENT]
    psf_prob.append(float(p_terminal))
psf_prob = np.asarray(psf_prob)

10.4 Baselines

from psf import InactivityThresholdClassifier, LogisticDropoutModel, SurvivalWithoutIntent

# Inactivity threshold
inactivity = np.array([seq[-1, 5] for seq in test_obs])
baseline_pred = InactivityThresholdClassifier(threshold=1.0).predict_proba(inactivity)[:, 1]

# Logistic regression
X = np.array([seq.mean(axis=0) for seq in test_obs])
logit = LogisticDropoutModel().fit(X_train, y_train)
baseline_pred = logit.predict_proba(X_test)[:, 1]

# Survival without intent (with label re-encoding)
event_codes = encode_for_survival(labels)
surv = SurvivalWithoutIntent().fit(X_train, train_durations, event_codes[train_idx])

10.5 Metrics

from psf import evaluate_predictions, evaluate_at_positive_rate, compare_models

results = {
    "PSF": evaluate_predictions(y_test, psf_prob),
    "Inactivity": evaluate_predictions(y_test, inactivity_pred),
    "Logistic": evaluate_predictions(y_test, logit_pred),
}
print(compare_models(results).round(3))

# Matched-rate metrics (the operationally meaningful comparison)
prevalence = y_test.mean()
matched = {
    "PSF": evaluate_at_positive_rate(y_test, psf_prob, prevalence),
    "Inactivity": evaluate_at_positive_rate(y_test, inactivity_pred, prevalence),
    "Logistic": evaluate_at_positive_rate(y_test, logit_pred, prevalence),
}
print(compare_models(matched).round(3))

11 The Reference Simulation

11.1 What the Simulation Does

1. Builds a "true" PSF model with known parameters.
2. Simulates a cohort of 400 students over 36 monthly time steps.
3. Fits a new model from scratch using multi-restart EM.
4. Evaluates parameter recovery, state recovery, and predictive performance.
5. Produces thirteen diagnostic figures.

The simulation serves as a machine verification: it confirms the EM algorithm, forward-backward recursions, and parameter estimation all work correctly. It does not validate the framework on real data.

11.2 The True Model

from psf import PSFModel
from psf.states import IDX
import numpy as np

def build_true_model(n_features=6):
    model = PSFModel(n_features=n_features)

    A_logits = np.full((6, 6), -np.inf)
    A_logits[IDX.S1, IDX.S1] = 1.0
    A_logits[IDX.S1, IDX.S2] = 3.0
    A_logits[IDX.S2, IDX.S2] = 3.0
    A_logits[IDX.S2, IDX.S3_NO_INTENT] = 0.0
    A_logits[IDX.S2, IDX.S3_INTENT] = 1.0
    A_logits[IDX.S2, IDX.S4] = -2.0
    A_logits[IDX.S2, IDX.S5] = -0.5
    A_logits[IDX.S3_NO_INTENT, IDX.S3_NO_INTENT] = 2.0
    A_logits[IDX.S3_NO_INTENT, IDX.S3_INTENT] = 0.0
    A_logits[IDX.S3_NO_INTENT, IDX.S4] = 0.0
    A_logits[IDX.S3_INTENT, IDX.S3_INTENT] = 1.0
    A_logits[IDX.S3_INTENT, IDX.S2] = 0.5
    A_logits[IDX.S3_INTENT, IDX.S4] = -1.0
    A_logits[IDX.S4, IDX.S4] = 0.0
    A_logits[IDX.S5, IDX.S5] = 0.0

    model.A_logits = A_logits
    model._refresh_A()

    rng = np.random.default_rng(42)
    model.mu = rng.normal(size=(6, n_features))
    model.mu[IDX.S2] += np.array([1.0, 0.8, 0.0, 0.0, 0.0, -0.5])
    model.mu[IDX.S3_NO_INTENT] += np.array([-1.0, -0.5, 0.0, 0.0, 0.0, 1.0])
    model.mu[IDX.S3_INTENT] += np.array([-0.5, -0.3, 0.5, 0.5, 0.8, 0.8])
    model.mu[IDX.S4] += np.array([-2.0, -1.0, -0.5, -0.5, -0.5, 1.5])
    model.mu[IDX.S5] += np.array([0.5, 1.2, 0.2, 0.0, 0.0, -1.0])
    model.log_sigma = np.log(np.full((6, n_features), 0.7))

    return model

11.3 Simulation and Fitting

from psf import simulate_cohort, forward_backward
data = simulate_cohort(true_model, n_students=400, max_time=36, missing_rate=0.2)
fitted = fit_with_restarts(data["observations"], data["masks"], n_restarts=5, n_iter=30)

11.4 Parameter Recovery

from scipy.optimize import linear_sum_assignment

def align_states(true_model, fitted_model):
    cost_mu = np.linalg.norm(true_model.mu[:, None, :] - fitted_model.mu[None, :, :], axis=2)
    cost_A  = np.linalg.norm(true_model.A[:, None, :]  - fitted_model.A[None, :, :],  axis=2)
    cost = cost_mu + cost_A
    row_ind, col_ind = linear_sum_assignment(cost)
    perm = np.zeros(6, dtype=int)
    perm[col_ind] = row_ind
    return perm

11.5 State Recovery

correct, total = 0, 0
for seq, mask, true_states in zip(
    data["observations"], data["masks"], data["states"]
):
    out = forward_backward(fitted, seq, mask)
    pred = out["gamma"].argmax(axis=1)
    correct += (pred == true_states).sum()
    total += len(true_states)

print(f"Joint-state accuracy: {correct / total:.3f}")
# Reference run: 0.859

12 Reading the Diagnostic Figures

12.1 EM Convergence

Plot the log-likelihood per iteration for each restart. Look for monotone increase in every curve (EM guarantees this), the spread between curves (a wide spread justifies multi-restart), and which curve ends highest (that restart is selected).

12.2 Transition Matrix Comparison

Look for diagonal dominance in the persistence states, correct absorbing behavior for S₄ and S₅, and non-zero S₃(I=0) → S₃(I=1) transition.

12.3 Emission Mean Comparison

Look for vertical alignment between true and fitted bars, and distinct panels for S₃(I=0) and S₃(I=1) on the intent-proxy features.

12.4 State Trajectories

Look for sharp transitions between bands, agreement between the dominant band and the true state, and plausible trajectories.

12.5 Intent Trajectory

This is the central evidence for the framework: if the posterior tracks the true intent, the intent construct is being recovered. Reference run: AUC > 0.95.

12.6 ROC and PR Curves

Reference run: PSF achieves AUC 0.998 and PR-AUC 0.999, substantially above all baselines.

12.7 Matched-Rate Comparison

Reference run: PSF achieves precision 0.987 at the matched rate, versus 0.895 for the inactivity threshold.

13 Data Collection and Ethics

13.1 Institutional Approval

Before accessing student records, obtain IRB (or equivalent) approval. Typical elements: a clear statement of research question and methods, a data management plan, a description of risks and mitigations, and informed consent procedures if applicable.

13.2 Data Streams

13.3 Ethics of Persistence Modeling

Transparency. Students should know that their activity is being used to model their persistence.

Non-punitiveness. The model's outputs should not be used to penalize students.

Bias auditing. Before deployment, check for differential performance across demographic groups.

Consent and opt-out. Where feasible, provide students with the ability to see whether they are being modeled and to opt out.

Human-in-the-loop. Any decision with material consequences for a student should involve an advisor.

13.4 De-identification

Before analysis, replace student IDs with synthetic identifiers, remove or hash fields that could identify individuals, aggregate demographic data into coarse categories, and store the link between synthetic and real IDs separately with access controls.

14 Preparing Observation Sequences

14.1 Choosing a Time Grid

• Daily — finest resolution, requires lots of data, noisy.
• Weekly — good for early warning, moderate data requirements.
• Monthly — robust, good for program evaluation.

14.2 Building the Feature Matrix

from psf import FeatureExtractor

extractor = FeatureExtractor()
obs = np.zeros((T, extractor.n_features))
for t in range(T):
    period_data = student_data[student_data.t == t]
    obs[t] = extractor.transform_timestep(
        activity_count=len(period_data[period_data.type == "login"]),
        submission=int((period_data.type == "submission").any()),
        extension=int((period_data.type == "extension").any()),
        tutor=int((period_data.type == "tutor_contact").any()),
        future_registration=int((period_data.type == "future_registration").any()),
        time_since_last_activity=t - last_active_time,
    )

14.3 Handling Missing Data

mask = np.ones_like(obs, dtype=bool)
mask[t, feature_idx] = False

14.4 Normalizing Features

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()
X_train_flat = np.vstack([o for o in train_obs])
scaler.fit(X_train_flat)

def normalize(obs):
    return [scaler.transform(o) for o in obs]

15 Generating Operational Labels

15.1 Building Student Records

from psf import StudentRecord

record = StudentRecord(
    student_id=student_id,
    registration_times=[reg_month],
    last_activity_time=last_active_month,
    completion_time=completion_month,
    withdrawal_time=withdrawal_month,
    extension_active_after=extension_month,
    future_registration_after=future_reg_month,
)

15.2 Generating Labels

from psf import generate_operational_labels
labels = generate_operational_labels(records, W=24, T_pause=2, observation_end=36)

15.3 Inspecting the Label Distribution

print(f"Completed:   {(labels == 0).sum()} ({(labels == 0).mean():.1%})")
print(f"Withdrawn:   {(labels == 1).sum()} ({(labels == 1).mean():.1%})")
print(f"Stopped-Out: {(labels == 2).sum()} ({(labels == 2).mean():.1%})")
print(f"Censored:    {(labels == 3).sum()} ({(labels == 3).mean():.1%})")

15.4 Splitting the Data

sorted_idx = np.argsort([r.registration_times[0] for r in records])
split = int(0.7 * len(sorted_idx))
train_idx, test_idx = sorted_idx[:split], sorted_idx[split:]

Random splits leak future information into the training set. Always split temporally or by cohort.

16 Fitting on Real Data

16.1 Initialize the Model

from psf import PSFModel
model = PSFModel(n_features=6)

16.2 Fit with Multi-Restart

fitted = fit_with_restarts(train_obs, train_masks, n_restarts=5, n_iter=30)

16.3 Check Convergence

from psf import viz
viz.plot_em_convergence(
    {"restart": fitted.em_history_},
    save_path="figures/em_convergence_real.png",
)

16.4 Save the Fitted Model

import pickle
with open("model.pkl", "wb") as f:
    pickle.dump(fitted, f)

17 Validating the Fit

17.1 Inspect the Fitted Parameters

Look for diagonal dominance in persistence states, absorbing behavior in S₄ and S₅, non-zero S₃(I=0) → S₃(I=1) transition (if the population shows evidence of developing intent), and reasonable S₃(I=1) → S₂ return probability.

17.2 Check Emission Separation

from psf.states import IDX
mu_s3_no  = fitted.mu[IDX.S3_NO_INTENT]
mu_s3_yes = fitted.mu[IDX.S3_INTENT]
sigma = np.exp(fitted.log_sigma).mean(axis=0)
standardized_gap = np.abs(mu_s3_yes - mu_s3_no) / sigma
print("Standardized gap:", standardized_gap)

If the maximum gap is below 0.5, the two variants are not separable, and the intent posterior will be near 0.5 everywhere.

17.3 Evaluate Predictive Performance

from psf import forward_backward, evaluate_predictions, evaluate_at_positive_rate
from psf.states import IDX

y_test = (labels[test_idx] == 1).astype(int)

psf_prob = []
for seq, mask in zip(test_obs, test_masks):
    out = forward_backward(fitted, seq, mask)
    g = out["gamma"]
    psf_prob.append(float(g[-1, IDX.S4] + g[-1, IDX.S3_NO_INTENT]))
psf_prob = np.asarray(psf_prob)

metrics = evaluate_predictions(y_test, psf_prob)
print(f"AUC: {metrics['auc']:.3f}")
print(f"PR-AUC: {metrics['pr_auc']:.3f}")

prevalence = float(y_test.mean())
matched = evaluate_at_positive_rate(y_test, psf_prob, prevalence)
print(f"Precision at matched rate: {matched['precision']:.3f}")

17.4 Interpret the Results

18 Case Study — Early Warning Triage

An open university wants to build an early warning system for advisors with a capacity of 100 students per week. Using AU-style records (2,000 training, 1,000 test, monthly grid, six default features), the PSF is fitted with multi-restart EM and compared to the inactivity-threshold baseline.

psf_rank = np.argsort(-psf_prob)[:100]
inactivity_rank = np.argsort(-inactivity_prob)[:100]

y_test = (labels[test_idx] == 1).astype(int)
print(f"PSF: {y_test[psf_rank].sum()}/100 true withdrawals in top 100")
print(f"Inactivity: {y_test[inactivity_rank].sum()}/100 true withdrawals in top 100")

In a representative run: PSF achieves 87/100 precision, inactivity achieves 72/100. This means 15 fewer wasted interventions per week — a reduction of more than 50% in wasted effort, with no reduction in the number of true withdrawals reached.

19 Case Study — Validating Intent Recovery

A researcher wants to demonstrate construct validity for the latent Intent-to-Return variable. The analysis proceeds in four steps: confirm the two S₃ variants are distinguishable, extract intent posteriors for S₃ timesteps, compute AUC against Viterbi-derived intent labels, and demonstrate the predictive value of intent.

In a representative study: standardized gap of 0.8 on extension, 0.6 on tutor, 1.2 on future_reg. Intent posterior AUC of 0.92. Return prediction PR-AUC of 0.87 for the full model vs. 0.71 for the restricted model.

20 Case Study — Online Tracking

An institution wants to deploy the PSF as a real-time system that updates as students interact with the LMS. The particle filter is the online inference algorithm.

from psf import particle_filter
out = particle_filter(fitted, seq, mask, n_particles=1000)
filtered = out["filtered_probs"]

With 500 particles per student and a monthly time grid, the particle filter processes a new observation in under a millisecond. Best practices: use a fixed number of particles (500–1000), monitor effective sample size, and refresh the model quarterly.

21 Case Study — Program Evaluation

A university wants to report accurate persistence metrics to its Board and accreditation body.

labels = generate_operational_labels(records, W=24, T_pause=2, observation_end=36)
total = len(labels)
print(f"Completed:   {(labels == 0).sum()} ({(labels == 0).mean():.1%})")
print(f"Withdrawn:   {(labels == 1).sum()} ({(labels == 1).mean():.1%})")
print(f"Stopped-Out: {(labels == 2).sum()} ({(labels == 2).mean():.1%})")
print(f"Censored:    {(labels == 3).sum()} ({(labels == 3).mean():.1%})")

The traditional completion-rate metric reports a 40% non-completion rate. The PSF-based breakdown separates: 24% withdrawn (true attrition), 28% stopped out (may return), and 13% censored. The true attrition rate is 24%, not 40%.

22 Case Study — Ablation and Sensitivity

Which components of the PSF are actually contributing to model performance?

The intent variable contributes substantially to return prediction (0.79 vs. 0.51). The full model outperforms the three-state model on withdrawal prediction (0.87 vs. 0.65).

23 Extending Emissions

The default emission model is Gaussian, appropriate for continuous features. For count data, use Poisson; for binary data, Bernoulli; for ordinal data, ordered probit.

import numpy as np
import scipy.special
from psf import PSFModel

class PoissonPSFModel(PSFModel):
    def log_emission_prob(self, observations, mask=None):
        rate = np.exp(self.mu)
        log_b = np.zeros((observations.shape[0], self.n_states))
        for k in range(self.n_states):
            log_b[:, k] = np.sum(
                observations * np.log(rate[k]) - rate[k]
                - scipy.special.gammaln(observations + 1),
                axis=1,
            )
        return log_b

24 Covariate-Aware DBN

The basic PSF assumes transition probabilities are constant across students and time. A Dynamic Bayesian Network extends the HMM by allowing transition probabilities to depend on covariates.

class DBNPSFModel(PSFModel):
    def __init__(self, n_features, n_covariates, **kwargs):
        super().__init__(n_features=n_features, **kwargs)
        self.n_covariates = n_covariates
        self.beta = np.zeros((self.n_states, self.n_states, n_covariates))

    def set_covariates(self, covariates):
        self._covariates = covariates

    def _refresh_A(self):
        if not hasattr(self, "_covariates"):
            super()._refresh_A()
            return
        logits = self.A_logits + np.einsum("ijm,m->ij", self.beta, self._covariates)
        logits = np.where(self.allowed > 0, logits, -np.inf)
        self.A = softmax_rows(logits)

25 Hierarchical Program-Level Models

The basic PSF is a course-level model. Program-level persistence tracks a student's relationship to the program as a whole.

class HierarchicalPSF:
    def __init__(self, n_courses):
        self.course_models = [PSFModel(n_features=6) for _ in range(n_courses)]
        self.program_model = PSFModel(n_features=6)

    def fit(self, course_data, program_data):
        for i, (obs, masks) in enumerate(course_data):
            baum_welch(self.course_models[i], obs, masks)
        program_obs = self._aggregate(course_data)
        baum_welch(self.program_model, program_obs)

26 Deployment and Monitoring

Before deploying the PSF in production, verify: model validated on held-out data, calibration acceptable, subgroup performance audited, ethical review complete, student notification in place, advisor training done, monitoring set up.

Shadow mode. Run the model alongside the existing system for one term without acting on its outputs.

Calibration monitoring. Every month, compute the reliability diagram on the most recent cohort. If expected calibration error exceeds 0.05, refit.

Subgroup auditing. Quarterly, check performance separately for domestic vs. international, full-time vs. part-time, program level, and age bands.

Model refresh. Retrain quarterly or annually as the population and course designs drift.

Ten Exercises

Exercise 1: Understanding the State Space

1. Load the ALLOWED_TRANSITIONS mask from psf.states.
2. Compute the number of allowed transitions out of each state.
3. Identify which transitions are asymmetric.
4. Modify the mask to allow S₁ → S₅. Re-fit and observe how the fit changes.

Exercise 2: Multi-Restart EM

1. Run the reference simulation with a single restart. Record LL and AUC.
2. Run with five restarts. Record best LL and AUC.
3. Run ten more times with different seeds, single restart. Compute mean and std of LL.
4. What fraction of single-restart runs land in the best basin?

Exercise 3: Construct Validity

1. Simulate a cohort under the reference model.
2. Fit the model.
3. Extract the intent posterior for all S₃ timesteps.
4. Compute the AUC of the intent posterior against the true intent labels.
5. Simulate a cohort where S₃(I=0) and S₃(I=1) emissions are identical. Refit and recompute AUC. What happens?

Exercise 4: Handling Missing Data

1. Simulate cohorts with missing_rate = 0.1, 0.2, 0.3, 0.4.
2. Fit the model on each cohort.
3. Plot state recovery accuracy vs. missing rate.
4. At what missing rate does accuracy drop below 0.75?

Exercise 5: Real-Data Preparation

If you have access to institutional data: extract registration, activity, assessment, and administrative records; build StudentRecord objects; generate operational labels with W=24 and T_pause=2; report the label distribution; split temporally. Otherwise, use the simulation as a substitute.

Exercise 6: Ablation Study

1. Fit the full PSF.
2. Fit a restricted PSF with the intent proxies masked out.
3. Fit a three-state PSF (no intent).
4. Compare PR-AUC for withdrawal and return prediction.
5. Which component contributes most?

Exercise 7: Online Tracking

1. Simulate a single trajectory of 50 timesteps.
2. For each t, compute the filtering distribution using the particle filter on the first t observations.
3. Compare to the smoothing distribution from forward-backward on the full sequence.
4. How quickly does filtering converge to smoothing?

Exercise 8: Calibration Analysis

1. Fit the model on training data.
2. Generate predictions on test data.
3. Plot a reliability diagram.
4. Compute the Brier score and expected calibration error.
5. If over- or under-confident, apply isotonic regression to recalibrate.

Exercise 9: Extending Emissions

1. Subclass PSFModel to implement a Bernoulli emission.
2. Test the emission on synthetic binary data.
3. Compare state recovery to the Gaussian version on the same features.

Exercise 10: Design an Empirical Study

Write a research proposal covering: research question, data streams and preparation, operational definitions (W and T_pause), model specification (features and constraints), validation (splits, baselines, metrics), ablation study, sensitivity analysis, and ethical considerations.

Appendix A — Quick Reference

State Reference

Label Reference

Default Hyperparameters

Appendix B — Glossary

• DBN: Dynamic Bayesian Network.
• EM: Expectation-Maximization.
• HMM: Hidden Markov Model.
• Intent to Return (Iₜ): Latent variable capturing commitment to continue.
• Matched-rate evaluation: Comparing models at equal positive-class capacity.
• PR-AUC: Precision-Recall Area Under Curve.
• PSF: Persistence-State Framework.
• Stopped-Out: Temporary interruption with intent to return.
• Withdrawn: Permanent exit with no intent to return.

Appendix C — Further Reading

• Rabiner, L. R. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2), 257–286.
• Murphy, K. P. (2002). Dynamic Bayesian networks: Representation, inference and learning. Doctoral dissertation, UC Berkeley.
• Koller, D., & Friedman, N. (2009). Probabilistic graphical models: Principles and techniques. MIT Press.
• Tinto, V. (1993). Leaving college: Rethinking the causes and cures of student attrition (2nd ed.). University of Chicago Press.
• Bean, J. P., & Metzner, B. S. (1985). A conceptual model of nontraditional undergraduate student attrition. Review of Educational Research, 55(4), 485–540.
• Simpson, O. (2013). Student retention in distance education: Are we failing our students? Open Learning, 28(2), 105–119.