PSF User Guide

01 Introduction

1.1 What This Guide Is For

This guide introduces the Persistence-State Framework (PSF) and provides practical instructions for using the accompanying Python library, psf, to validate the framework against institutional data. It is written for:

• Learning analytics researchers who want to test whether stop-out and attrition can be distinguished computationally in their own institutional context.
• Institutional data scientists who want to implement PSF-based early warning systems or intervention triage pipelines.
• Graduate students who want to replicate or extend the framework as part of a thesis or dissertation.
• Reviewers and methodologists who want to understand the assumptions and mechanics of the framework in detail.

1.2 What the PSF Is (and Is Not)

The PSF is a conceptual and computational framework for modeling persistence in self-paced learning. It provides:

• A formal state-space representation of persistence behavior.
• A latent variable (Intent to Return) that distinguishes temporary stop-out from permanent attrition.
• A set of inference algorithms (HMM, DBN, survival analysis, particle filtering).
• A validation methodology with operational definitions and baselines.
• A set of diagnostic and visualization tools.

The PSF does not provide: pre-trained models (these must be fitted on institutional data); a dashboard or user interface; automated intervention delivery; a guarantee of external validity across institutions.

1.3 When to Use the PSF

Use the PSF when your institution offers self-paced or continuous-enrollment courses; inactivity is a poor proxy for dropout because students regularly pause and return; you need to distinguish temporary stop-out from permanent withdrawal; you want to predict return, not just dropout; and you have access to administrative records (registrations, extensions, tutor contacts, withdrawals).

Do not use the PSF when your program is strictly cohort-based with fixed term schedules; you only have clickstream data and no administrative records; you need a turnkey production system rather than a research framework; or you need causal guarantees — the PSF is an observational model.

1.4 What This Version Adds

• Multi-restart EM fitting, which lifted simulated AUC from 0.685 (single restart) to 0.998 (five restarts) on the reference simulation.
• A visualization module (psf.viz) with thirteen plotting functions.
• A matched-positive-rate evaluator (evaluate_at_positive_rate) for operationally meaningful model comparison.
• A four-tier validation protocol that separates machine verification from empirical validation.
• Parameter-recovery diagnostics including automatic state alignment.
• Corrected label conventions between simulation and survival models.

02 Theoretical Foundations

2.1 The Core Idea

In self-paced learning, the absence of visible activity does not necessarily mean a student has withdrawn. Many students pause for weeks or months and later return. Traditional dropout models treat all inactive students as dropouts, which overestimates attrition, misdirects interventions, and alienates adult learners who intend to return.

The PSF addresses this by representing persistence as a dynamic state rather than a binary outcome, and by introducing a latent Intent to Return variable.

2.2 The Five Persistence States

The critical distinction is between S₃ (transient) and S₄ (absorbing).

2.3 Intent to Return

The latent variable Iₜ ∈ {0,1} captures the student's commitment to continue. It is inferred from proxies such as extension requests, tutor contacts, future course registrations, communication with advising services, and submission of partially completed work.

2.4 The Joint State Space

2.5 What the Model Computes

• P(Return | History) — probability the student returns to Progressing.
• P(Withdraw | History) — probability the student transitions to Withdrawn.
• P(Iₜ = 1 | History) — posterior probability of Intent to Return.
• Time-varying state posteriors — P(Zₜ = k | y₁:ₜ).

2.6 Computational Formalisms

The PSF can be implemented using Hidden Markov Models (HMMs), Dynamic Bayesian Networks (DBNs), survival analysis, and event history analysis. The psf library implements all four.

03 Installation and Setup

3.1 Requirements

• Python ≥ 3.9
• NumPy ≥ 1.22
• SciPy ≥ 1.9
• scikit-learn ≥ 1.1
• pandas ≥ 1.4
• matplotlib ≥ 3.5 (optional, for visualization)

3.2 Installation

Option A: Editable install (recommended for development)

git clone https://github.com/hongxueharriswang/psf_lib.git
cd psf_lib
python -m venv venv
source venv/bin/activate     # On Windows: venv\Scripts\activate
pip install -e ".[viz,dev]"

Option B: From PyPI (once published)

pip install psf-learning

Option C: Minimal install

pip install numpy scipy scikit-learn pandas

3.3 Verify Installation

import psf
print(psf.__version__)  # 0.1.0

from psf import PSFModel, simulate_cohort, baum_welch
from psf import viz
print("PSF library loaded successfully.")

3.4 Run the Reference Simulation

python examples/01_simulation_study.py

Expected output includes EM convergence across five restarts, parameter recovery diagnostics, and a comparison table. Figures are written to ./figures/.

3.5 Package Structure

psf/
├── states.py        # State definitions and structural constraints
├── model.py         # PSFModel (HMM parameters)
├── inference.py     # Forward-backward, Viterbi, particle filter
├── learning.py      # Baum-Welch (EM) parameter estimation
├── survival.py      # Competing-risk discrete-time survival
├── labels.py        # Operational label generation
├── baselines.py     # Baseline models for comparison
├── evaluation.py    # AUC, PR-AUC, Brier, matched-rate evaluation
├── features.py      # Feature extraction
├── simulation.py    # Synthetic data generation
├── viz.py           # Visualization utilities
└── utils.py         # Numerical utilities

04 Data Collection

4.1 Required Data Streams

4.2 Minimum Viable Dataset

• Registration records: student ID, course ID, registration date.
• Activity records: student ID, timestamp, activity type.
• Withdrawal records: student ID, withdrawal date.
• Completion records: student ID, completion date.

4.3 Recommended Additional Data

• Extension request logs with approval status.
• Tutor contact logs with contact type.
• Advising interaction logs.
• Future registration records (even if subsequently dropped).
• Offline indicators (textbook purchases, proctored exam scheduling).

4.4 Data Collection Checklist

• Data are de-identified and ethically approved.
• Each student has a unique identifier.
• Timestamps are in a consistent timezone and format.
• Course start and end dates are recorded.
• Withdrawal and completion records are complete.
• Extension requests are logged.
• Tutor contacts are logged (if available).
• Observation window is at least 24 months for ground-truth labeling.

4.5 Ethical Considerations

• Students should know how persistence is measured.
• Interventions should support rather than surveil.
• Data should be stored securely and de-identified.
• Model outputs should not be used punitively.

05 Data Preparation

5.1 Time Grid

Choose a weekly grid for early warning applications (finer resolution, more data required) or a monthly grid for program evaluation (coarser resolution, more robust). All timestamps must be mapped to grid indices.

5.2 Student Records

from psf import StudentRecord

record = StudentRecord(
    student_id=1001,
    registration_times=[0],
    last_activity_time=8,
    completion_time=None,
    withdrawal_time=None,
    extension_active_after=10,
    future_registration_after=None,
)

5.3 Observation Sequences

Each student's observation sequence is a 2D array of shape (T, D). Default features:

5.4 Missing Data

The PSF handles missingness via a mask array of shape (T, D) where True means observed and False means missing. Missing features are marginalized out of the Gaussian emission. Missing timesteps propagate the state without a state update.

5.5 Complete Data Preparation Example

import numpy as np
from psf import FeatureExtractor, StudentRecord

def prepare_student(raw_df, student_id, time_grid="monthly"):
    student_df = raw_df[raw_df.student_id == student_id].copy()
    student_df["t"] = student_df["timestamp"].dt.month
    T = student_df["t"].max() + 1

    extractor = FeatureExtractor()
    obs = np.zeros((T, extractor.n_features))
    mask = np.zeros((T, extractor.n_features), dtype=bool)

    for t in range(T):
        period = student_df[student_df.t == t]
        activity_count = len(period[period.event_type == "login"])
        submission = int((period.event_type == "submission").any())
        extension = int((period.event_type == "extension").any())
        tutor = int((period.event_type == "tutor_contact").any())
        future_reg = int((period.event_type == "future_registration").any())

        active = student_df[student_df.event_type.isin(["login", "submission"])]
        time_since = t - active.t.max() if len(active) > 0 else 0

        obs[t] = extractor.transform_timestep(
            activity_count, submission, extension, tutor,
            future_reg, max(time_since, 0),
        )
        mask[t] = True

    return obs, mask

5.6 Generating Operational Labels

from psf import generate_operational_labels

labels = generate_operational_labels(
    records,
    W=24,
    T_pause=2,
    observation_end=36,
)

# 0 = Completed
# 1 = Withdrawn (operational attrition)
# 2 = Stopped-Out (operational stop-out)
# 3 = Censored / still enrolled

5.7 The Ground-Truth Definition

5.8 Data Splitting

sorted_idx = np.argsort([r.registration_times[0] for r in records])
split = int(0.7 * len(sorted_idx))
train_idx, test_idx = sorted_idx[:split], sorted_idx[split:]

train_obs = [obs[i] for i in train_idx]
train_masks = [masks[i] for i in train_idx]
test_obs = [obs[i] for i in test_idx]
test_masks = [masks[i] for i in test_idx]

06 Library Architecture

6.1 Component Overview

User Data (registration, activity, assessments, intent proxies)
    │
    ▼
features.FeatureExtractor → (T, D) observation arrays + masks
    │
    ▼
labels.generate_operational_labels → operational labels (0/1/2/3)
    │
    ▼
model.PSFModel → HMM with 6 joint states
    │
    ├──────────────┬──────────────┐
    ▼              ▼
learning.       survival.
baum_welch      CompetingRiskSurvival
    │              │
    ▼              ▼
inference.forward_backward / particle_filter
    → state posteriors, intent posteriors, return probabilities
    │
    ▼
evaluation.evaluate_predictions / compare_models
    → AUC, PR-AUC, Brier, FPR, concordance, matched-rate
    │
    ▼
viz.plot_* → publication-ready figures

6.2 Key Classes

6.3 Key Functions

6.4 Key Visualization Functions

07 Step-by-Step Validation Workflow

Step 1: Prepare Data

import numpy as np
from psf import StudentRecord, FeatureExtractor

records = load_student_records()
obs, masks = load_observation_sequences()

Step 2: Generate Operational Labels

from psf import generate_operational_labels

labels = generate_operational_labels(records, W=24, T_pause=2, observation_end=36)

print(f"Completed:   {(labels == 0).sum()}")
print(f"Withdrawn:   {(labels == 1).sum()}")
print(f"Stopped-Out: {(labels == 2).sum()}")
print(f"Censored:    {(labels == 3).sum()}")

Step 3: Split Data Temporally

sorted_idx = np.argsort([r.registration_times[0] for r in records])
split = int(0.7 * len(sorted_idx))
train_idx, test_idx = sorted_idx[:split], sorted_idx[split:]

Step 4: Fit the PSF with Multi-Restart

from psf import PSFModel, baum_welch, forward_backward

best_model, best_ll, best_seed = None, -np.inf, -1
for seed in range(5):
    rng = np.random.default_rng(seed)
    model = PSFModel(n_features=obs[0].shape[1])
    model.mu = rng.normal(size=model.mu.shape) * 0.5
    model.log_sigma = np.zeros_like(model.log_sigma)

    baum_welch(
        model,
        [obs[i] for i in train_idx],
        [masks[i] for i in train_idx],
        n_iter=30,
    )

    ll = sum(
        forward_backward(model, obs[i], masks[i])["log_lik"]
        for i in train_idx
    )
    print(f"restart {seed}: LL = {ll:.2f}")

    if ll > best_ll:
        best_model, best_ll, best_seed = model, ll, seed

print(f"Selected restart: seed={best_seed}, LL={best_ll:.2f}")

Step 5: Inspect the Fitted Model

import pandas as pd
from psf.states import JOINT_STATE_LABELS

print("Transition matrix (rows = from-state):")
print(pd.DataFrame(
    best_model.A,
    index=JOINT_STATE_LABELS,
    columns=JOINT_STATE_LABELS,
).round(3))

Expected: diagonal dominance, absorbing S4 and S5, and non-zero transitions between S3(I=0) and S3(I=1).

Step 6: Compute State and Intent Posteriors

from psf import forward_backward, collapse_posterior, intent_posterior

i = test_idx[0]
out = forward_backward(best_model, obs[i], masks[i])
gamma = out["gamma"]
persistence = collapse_posterior(gamma)
intent = intent_posterior(gamma)

print("Persistence posteriors (last 5 timesteps):")
print(pd.DataFrame(
    persistence[-5:],
    columns=["Registered", "Progressing", "Stopped-Out", "Withdrawn", "Completed"],
).round(3))

Step 7: Compute Return and Withdrawal Probabilities

from psf import return_probabilities, withdrawal_probabilities

p_return = return_probabilities(best_model, gamma, horizon=6)
p_withdraw = withdrawal_probabilities(best_model, gamma, horizon=6)

print(f"P(Return within 6 months): {p_return[-1, -1]:.3f}")
print(f"P(Withdraw within 6 months): {p_withdraw[-1, -1]:.3f}")

Step 8: Evaluate Against Baselines

from psf import (
    InactivityThresholdClassifier,
    LogisticDropoutModel,
    SurvivalWithoutIntent,
    evaluate_predictions,
    compare_models,
    evaluate_at_positive_rate,
)
from psf.states import IDX

y_test = (labels[test_idx] == 1).astype(int)

psf_prob = []
for i in test_idx:
    out = forward_backward(best_model, obs[i], masks[i])
    g = out["gamma"]
    p_terminal = g[-1, IDX.S4] + g[-1, IDX.S3_NO_INTENT]
    psf_prob.append(float(p_terminal))
psf_prob = np.asarray(psf_prob)

X_test = np.array([obs[i].mean(axis=0) for i in test_idx])
inactivity = np.array([obs[i][-1, 5] for i in test_idx])

results = {}
results["PSF (fitted)"] = evaluate_predictions(y_test, psf_prob)
results["Inactivity threshold"] = evaluate_predictions(
    y_test,
    InactivityThresholdClassifier(threshold=1.0).predict_proba(inactivity)[:, 1],
)
logit = LogisticDropoutModel().fit(X_test, y_test)
results["Logistic regression"] = evaluate_predictions(
    y_test, logit.predict_proba(X_test)[:, 1]
)

# Re-encode labels for survival model
event_for_survival = np.zeros_like(labels)
event_for_survival[labels == 0] = 3
event_for_survival[labels == 1] = 2
event_for_survival[labels == 2] = 0
event_for_survival[labels == 3] = 0

surv = SurvivalWithoutIntent().fit(
    X_test, [records[i].last_activity_time for i in test_idx],
    event_for_survival[test_idx],
)
cif = surv.predict_cumulative_incidence(X_test, horizon=12)
results["Survival (no intent)"] = evaluate_predictions(y_test, cif[:, -1, 1])

table = compare_models(results)
print(table[["auc", "pr_auc", "brier", "f1"]].round(3))

# Matched-rate comparison
prevalence = float(y_test.mean())
matched = {
    name: evaluate_at_positive_rate(y_test, s, prevalence)
    for name, s in {
        "PSF (fitted)": psf_prob,
        "Inactivity threshold": InactivityThresholdClassifier(threshold=1.0).predict_proba(inactivity)[:, 1],
        "Logistic regression": logit.predict_proba(X_test)[:, 1],
        "Survival (no intent)": cif[:, -1, 1],
    }.items()
}
matched_table = compare_models(matched)
print(matched_table[["auc", "pr_auc", "precision", "recall", "f1"]].round(3))

Step 9: Sensitivity Analysis

for W in [12, 24, 36]:
    for T_pause in [1, 2, 3]:
        labels_w = generate_operational_labels(
            records, W=W, T_pause=T_pause, observation_end=36,
        )
        print(f"W={W}, T_pause={T_pause}: "
              f"Withdrawn={(labels_w == 1).sum()}, "
              f"Stopped-Out={(labels_w == 2).sum()}")

08 Understanding the Figures

8.1 The Three Key Figures

1. 06_intent_student_*.png — central evidence for the framework. Shows that the latent Intent-to-Return variable is recoverable from observable proxies.
2. 03_transition_matrices.png — confirms that the fitted transition structure matches the true one after state alignment.
3. 10_matched_rate_metrics.png — the operationally meaningful comparison. At the matched positive rate, precision answers the advisor's question.

8.2 A Note on the Inactivity Baseline

In the reference simulation, the inactivity threshold achieves AUC 0.881, better than chance (0.500). This is because the simulation's label is a deterministic function of the terminal joint state, and the terminal state is directly observable from the final value of the inactivity feature.

09 Worked Examples

Example 1: Simulation Study (Reference)

python examples/01_simulation_study.py

Expected output on a fresh run:

=== PSF Simulation Study ===
[1] Simulating cohort...
[2] Fitting model via Baum-Welch (multi-restart)...
    restart 0: final LL =    -41352.95
    restart 1: final LL =    -40860.32
    restart 2: final LL =    -40857.21  <-- new best
    restart 3: final LL =    -41838.84
    restart 4: final LL =    -41437.30
    selected restart: seed=2, LL=-40857.21
[3] Parameter recovery diagnostic:
[4] Evaluating state-recovery accuracy...
    Posterior-argmax accuracy: 0.859
    Viterbi accuracy:          0.912
[5] Comparing to baselines...
[6] Matched-rate comparison at top 59.8% of students...

Example 2: Validating Intent Recovery

import numpy as np
from sklearn.metrics import roc_auc_score
from psf import (
    PSFModel, simulate_cohort, baum_welch,
    forward_backward, intent_posterior,
)
from psf.states import IDX

true_model = PSFModel(n_features=6)
data = simulate_cohort(true_model, n_students=400, max_time=36)

best_model, best_ll = None, -np.inf
for seed in range(5):
    rng = np.random.default_rng(seed)
    m = PSFModel(n_features=6)
    m.mu = rng.normal(size=m.mu.shape) * 0.5
    m.log_sigma = np.zeros_like(m.log_sigma)
    baum_welch(m, data["observations"], data["masks"], n_iter=30)
    ll = sum(forward_backward(m, o, mk)["log_lik"]
             for o, mk in zip(data["observations"], data["masks"]))
    if ll > best_ll:
        best_model, best_ll = m, ll

all_intent, all_true = [], []
for seq, m, true_states in zip(
    data["observations"], data["masks"], data["states"]
):
    out = forward_backward(best_model, seq, m)
    intent = intent_posterior(out["gamma"])
    true_intent = (true_states == IDX.S3_INTENT).astype(float)
    s3_mask = (true_states == IDX.S3_NO_INTENT) | (true_states == IDX.S3_INTENT)
    all_intent.extend(intent[s3_mask])
    all_true.extend(true_intent[s3_mask])

all_intent = np.nan_to_num(all_intent)
auc = roc_auc_score(all_true, all_intent)
print(f"Intent-to-Return AUC: {auc:.3f}")
# Expected: > 0.95 with multi-restart

Example 3: Handling Missing Data

data = simulate_cohort(true_model, n_students=400, missing_rate=0.4)
# ... fit as above ...
# State recovery should remain reasonable even at 40% missingness

Example 4: Online Tracking with Particle Filter

from psf import particle_filter

student_obs = data["observations"][0]
student_mask = data["masks"][0]

out = particle_filter(best_model, student_obs, student_mask, n_particles=1000)
filtered = out["filtered_probs"]
ess = out["ess"]

print("Filtered state probabilities (last 5 timesteps):")
for t in range(-5, 0):
    row = "  ".join(f"{v:.3f}" for v in filtered[t])
    print(f"  t={len(filtered)+t:3d}  {row}")

Example 5: Running on Institutional Data

def load_data():
    df = pd.read_parquet("data/au_courses.parquet")
    records, obs, masks = [], [], []
    for student_id, group in df.groupby("student_id"):
        records.append(build_student_record(group))
        o, m = prepare_student_observations(group)
        obs.append(o)
        masks.append(m)
    return records, obs, masks

10 The Four-Tier Validation Protocol

Tier 1 — Synthetic Recovery (Implemented)

Question: Does the EM and inference machinery recover the true model when the data are generated by the model?

Status: Verified. Joint-state accuracy 0.859–0.912 with multi-restart, monotone EM convergence, correct parameter recovery after alignment, intent posterior AUC > 0.95.

Limit: This tier cannot detect model misspecification.

Tier 2 — Operational Label Prediction (Implemented)

Question: Does the PSF predict the operational label better than baselines on held-out data?

Design: Temporal split (70/30), leave-one-course-out, leave-one-cohort-out.

Metrics: Primary — PR-AUC. Secondary — matched-positive-rate precision, calibration (Brier, reliability diagram), recall of true returners.

Required ablation: Fit without intent proxies (features 2–4) and with LMS-only (features 0, 5). If the ablation without intent proxies performs nearly as well as the full model, the intent construct is not empirically grounded.

Tier 3 — Decision-Theoretic Evaluation (Proposed)

Question: Would decisions made using the PSF be better than decisions made using baselines?

Design: Retrospective decision analysis. For each student at a fixed decision point, compute the intervention each model would recommend, then evaluate intervention precision, missed returners, and cost-weighted utility.

Tier 4 — Prospective Deployment (Future Work)

Question: Does using the PSF change student outcomes?

Design: Randomized or staggered deployment. Advisors see PSF outputs for a treatment cohort and baseline flags for a control cohort. Outcomes: return rate, time-to-return, completion rate, student satisfaction.

10.1 The Key Pre-Modeling Analysis

stopped_out = (labels == 2) | (labels == 1)
has_extension = np.array([r.extension_active_after is not None for r in records])

return_rate_with = (labels[stopped_out & has_extension] == 2).mean()
return_rate_without = (labels[stopped_out & ~has_extension] == 2).mean()

print(f"Return rate with extension: {return_rate_with:.3f}")
print(f"Return rate without extension: {return_rate_without:.3f}")

If the difference is less than 10 percentage points, the intent proxies are not informative in this population, and the PSF will reduce to a three-state HMM with extra parameters. This is a construct-validity check that must be reported.

11 Interpreting Results

11.1 State Posteriors

• High P(S2 = Progressing): Student is actively engaged.
• High P(S3 = Stopped-Out): Student is inactive but may return.
• High P(S3 with I=1): Student intends to return; light-touch intervention.
• High P(S3 with I=0): Student likely to withdraw; escalate.
• High P(S4 = Withdrawn): Student is effectively gone.

11.2 Intent-to-Return Posteriors

• Rising intent: Student is re-engaging; consider supportive check-in.
• Falling intent: Student is drifting toward withdrawal; consider proactive outreach.
• Stable high intent: Student is committed but paused; leave alone.

11.3 Return and Withdrawal Probabilities

• Early warning: Flag students with P(Withdraw within 3 months) > 0.5.
• Intervention triage: Prioritize students with high withdrawal and low return probability.
• Program evaluation: Aggregate return probabilities to identify bottleneck courses.

11.4 Model Comparison

• PR-AUC: Because withdrawal is rare, PR-AUC is more informative than AUC.
• Matched-rate precision: The operationally meaningful metric.
• Recall of true returners: The fraction of returners not flagged as at-risk.
• Calibration: For intervention decisions, probabilistic calibration matters more than ranking.

11.5 Sensitivity Analysis

Report results under multiple values of W and T_pause. If conclusions change materially, discuss the sensitivity in the paper.

11.6 What Counts as "Good" Performance

12 Troubleshooting

Problem 1: EM Does Not Converge

Symptom: Log-likelihood oscillates or decreases. Solutions: reduce states or features; increase sigma_floor; try multiple random initializations; check for degenerate sequences.

Problem 2: All Students Assigned to One State

Symptom: Posterior is degenerate. Solutions: check emissions distinguishable; initialize emissions from data; increase training size.

Problem 3: Intent AUC Is Near 0.5

Symptom: Intent not recoverable from proxies. Solutions: verify proxies encoded; check they vary across students; run pre-modeling analysis; if no proxies informative, report as a limitation.

Problem 4: Survival Baseline Predicts Zero Positives

Symptom: tp=0, fp=0 for Survival (no intent). Cause: label-convention mismatch. Solution: re-encode labels as shown in Step 8.

Problem 5: Multi-Restart Takes Too Long

Symptom: 5 × 30 × 400 is slow. Solutions: reduce n_restarts to 3 for exploration; use 5–10 for final; reduce n_iter to 20; subsample training students.

Problem 6: Figures Are Not Generated

Symptom: AttributeError: module 'psf.viz' has no attribute .... Solutions: confirm psf/viz.py exists and contains the functions; confirm from . import viz is in psf/__init__.py; clear caches; reinstall.

Problem 7: Overfitting

Symptom: Training log-likelihood much higher than test. Solutions: reduce features; add regularization; use temporal CV; report test-set metrics only.

13 Ethics and Deployment

13.1 Transparency

Students should know: what data are collected, how persistence is measured, how the model is used, and what interventions may follow.

13.2 Non-Punitiveness

PSF outputs should never be used to penalize students, deny services, restrict access, or rank students publicly. They should be used to trigger supportive outreach, allocate advising resources, identify course-design bottlenecks, and inform institutional policy.

13.3 Bias Auditing

Before deployment, check for differential performance across age groups, gender, disability status, international vs. domestic students, and program type.

13.4 Data Security

Store data in encrypted, access-controlled environments. De-identify before analysis whenever possible. Retain data only as long as necessary. Document all processing steps.

13.5 Student Consent

If possible, obtain informed consent for the use of persistence data in predictive modeling. Where consent is not feasible, ensure institutional review and student notification.

13.6 Human-in-the-Loop

The PSF should inform outreach, not replace it. Any decision with material consequences for a student should require advisor review.

13.7 Ongoing Monitoring

• Shadow mode: Run alongside the existing system without acting on outputs for one term.
• Calibration monitoring: Compute reliability diagrams monthly.
• Subgroup audits: Check performance quarterly across demographic groups.

14 Extending the Library

14.1 Adding New Emission Families

from psf.model import PSFModel
import numpy as np
import scipy.special

class PoissonPSFModel(PSFModel):
    def log_emission_prob(self, observations, mask=None):
        rate = np.exp(self.mu)
        log_b = np.zeros((observations.shape[0], self.n_states))
        for k in range(self.n_states):
            log_b[:, k] = np.sum(
                observations * np.log(rate[k]) - rate[k]
                - scipy.special.gammaln(observations + 1),
                axis=1,
            )
        return log_b

14.2 Adding Covariates (DBN)

class DBNPSFModel(PSFModel):
    def set_covariates(self, X_t):
        self.X_t = X_t

    def _refresh_A(self):
        logits = np.einsum("ijm,m->ij", self.X_t, self.beta)
        logits = np.where(self.allowed > 0, logits, -np.inf)
        from psf.utils import softmax_rows
        self.A = softmax_rows(logits)

14.3 Adding Deep Extensions

import torch
import torch.nn as nn

class NeuralEmission(nn.Module):
    def __init__(self, n_states, n_features):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(n_features, 32),
            nn.ReLU(),
            nn.Linear(32, n_states),
        )

    def forward(self, y):
        return torch.log_softmax(self.net(y), dim=-1)

14.4 Program-Level Extension

class HierarchicalPSF:
    def __init__(self, n_courses):
        self.course_models = [PSFModel() for _ in range(n_courses)]
        self.program_model = PSFModel()

    def fit(self, course_data, program_data):
        for i, data in enumerate(course_data):
            baum_welch(self.course_models[i], *data)
        program_obs = self._aggregate(course_data)
        baum_welch(self.program_model, program_obs)

15 Frequently Asked Questions

Q1: Do I need all six features?

No. You can use any subset. The library adapts to whatever n_features you specify. However, Intent-to-Return inference works best with extension, tutor, and registration features.

Q2: How many students do I need?

At least 200 for stable EM. For reliable PR-AUC estimates, 500+ is better. For intent recovery with informative proxies, 1000+ is ideal.

Q3: How long should the observation window be?

At least 24 months for ground-truth labeling. For training, 12–36 months is typical.

Q4: Can I use weekly data instead of monthly?

Yes. The time grid is arbitrary. Weekly data gives finer resolution but requires more data.

Q5: What if I don't have extension requests?

The model still works, but Intent-to-Return will be inferred mostly from registration patterns. Report this as a limitation.

Q6: How do I know if my model is any good?

Compare PR-AUC to the inactivity-threshold baseline. If the PSF does not improve PR-AUC or reduce FPR, the intent proxies may not be informative in your population.

Q7: Why does my single-restart fit perform worse than multi-restart?

EM converges to local optima. On the reference simulation, single-restart AUC was 0.685; multi-restart AUC was 0.998. Always use at least 3–5 restarts and keep the best.

Q8: Can I deploy this in production?

The library is research-grade. For production, add logging, monitoring, and drift detection. Validate regularly. Use shadow mode before live deployment.

Q9: What about GDPR / FERPA?

Consult your institutional legal and privacy offices. De-identify data. Obtain IRB approval.

Q10: How do I cite the library?

Cite the companion manuscript and the repository (see the README for BibTeX).

Q11: Why does the survival baseline predict zero positives?

Label-convention mismatch. See Problem 4 in Section 12.

Q12: Why does the inactivity threshold beat the PSF in the simulation?

Because the simulation's label is a deterministic function of the terminal joint state, and that state is directly observable from the final value of the inactivity feature. The baseline is near-Bayes-optimal by construction. In real data, the label is genuinely latent.

Q13: Can I contribute?

Yes. Open an issue or pull request. Contributions to emissions, covariates, deep extensions, and real-world validation are welcome.

16 References

• Athabasca University. (2003). Completion rates for learner-paced undergraduate courses. Athabasca University.
• Bean, J. P., & Metzner, B. S. (1985). A conceptual model of nontraditional undergraduate student attrition. Review of Educational Research, 55(4), 485–540.
• Kember, D. (1995). Open learning courses for adults: A model of student progress. Educational Technology Publications.
• Murphy, K. P. (2002). Dynamic Bayesian networks: Representation, inference and learning [Doctoral dissertation, University of California, Berkeley].
• Rabiner, L. R. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2), 257–286.
• Simpson, O. (2013). Student retention in distance education: Are we failing our students? Open Learning, 28(2), 105–119.
• Tinto, V. (1993). Leaving college: Rethinking the causes and cures of student attrition (2nd ed.). University of Chicago Press.
• Wang, H. H. (2025). Distinguishing stop-out from attrition in self-paced online learning: A computational persistence framework [Manuscript].

Appendix A — Quick Reference

Minimal Working Example

import numpy as np
from psf import (
    PSFModel, simulate_cohort, baum_welch,
    forward_backward, evaluate_predictions,
)
from psf.states import IDX

true_model = PSFModel(n_features=6)
data = simulate_cohort(true_model, n_students=400, max_time=36)

best_model, best_ll = None, -np.inf
for seed in range(5):
    rng = np.random.default_rng(seed)
    m = PSFModel(n_features=6)
    m.mu = rng.normal(size=m.mu.shape) * 0.5
    m.log_sigma = np.zeros_like(m.log_sigma)
    baum_welch(m, data["observations"], data["masks"], n_iter=30)
    ll = sum(forward_backward(m, o, mk)["log_lik"]
             for o, mk in zip(data["observations"], data["masks"]))
    if ll > best_ll:
        best_model, best_ll = m, ll

y = (data["final_label"] == 1).astype(int)
prob = np.array([
    forward_backward(best_model, o, m)["gamma"][-1, [IDX.S4, IDX.S3_NO_INTENT]].sum()
    for o, m in zip(data["observations"], data["masks"])
])
print(evaluate_predictions(y, prob))

State Reference

Label Reference

Default Hyperparameters

Appendix B — Glossary

• DBN: Dynamic Bayesian Network.
• EM: Expectation-Maximization.
• ESS: Effective Sample Size.
• HMM: Hidden Markov Model.
• Intent to Return (Iₜ): Latent variable capturing commitment to continue.
• Matched-rate evaluation: Comparing models at equal positive-class capacity, mirroring advisor workload constraints.
• PR-AUC: Precision-Recall Area Under Curve.
• PSF: Persistence-State Framework.
• Stopped-Out: Temporary interruption with intent to return.
• Withdrawn: Permanent exit with no intent to return.