skip to content
Bernardo Fernandez
all projects

Machine learning / 2025

PsyPredict

Comparing models also means investigating data, preprocessing and the limits of evaluation.

context
Survey-data experiment
role
Exploratory code and notebooks
state
Exploratory study; incomplete pipeline
stack
Python / scikit-learn / pandas / SHAP
view code

what it is

The notebooks explore a workplace mental-health survey through classifier training and hyperparameter tuning. The target is the treatment response, not a clinical diagnosis or a need for treatment.

the problem

A score must be interpreted alongside its target, split and transformations. Saved outputs allow inspection of the comparison, but the data required to repeat it is not versioned.

the constraints

External data
Notebooks expect archive/survey.csv and intermediate files. The preprocessing notebook contains a saved FileNotFoundError.
Exploratory scope
Survey responses do not establish diagnosis, efficacy or clinical validity.

the architecture

The mental-health notebook workflow.

  1. Survey

    notebooks/eda*

    Inspect columns, missing values and target values.

  2. Preprocessing

    notebooks/02*

    Cleaning, category encoding and split.

  3. Evaluation

    notebooks/03* · 04*

    Random Forest, logistic regression and GridSearchCV.

  4. Explainability

    notebooks/05*

    TreeExplainer; plotting ends in a shape error.

Streamlit, FastAPI and src/ address telco churn. They do not integrate with this mental-health workflow.

technical decisions

  1. Two classifiers on the same split

    why
    The notebook trains Random Forest and logistic regression on the same training and test files.
    instead of
    Evaluating just one classifier.
    the price
    Logistic regression emits a convergence warning and has no scaling. The scores do not isolate algorithm quality.

  2. Keep evaluation in notebooks

    why
    Per-class reports, ROC-AUC and parameter search stay next to the code that produced them.
    instead of
    Publishing only a final score.
    the price
    Saved outputs can become stale. Reproduction requires compatible data and environment.

limits and lessons

Unfinished explainability

The call to shap.summary_plot(shap_values[1], X_sample) ends in a shape AssertionError. The artifact does not demonstrate a working SHAP plot.

Two domains in one repository

Notebooks address mental health while interfaces and Python modules address churn. The dashboard file is empty.

current state

The result is an inspectable exploratory study. Versioned outputs record 0.79158 ROC-AUC for the tuned Random Forest on 252 test records. These values were not reproduced in this audit and do not establish clinical validity.

next questions

  • Separate the churn and mental-health domains.
  • Version the environment, document data provenance and review preprocessing before the split.
  • Re-evaluate the scaled baseline and fix SHAP dimensions.

evidence

notebooks/04_hyperparameter_tuning_mental_health.ipynb

Fitting 5 folds for each of 108 candidates, totalling 540 fits
Best ROC-AUC score (CV): 0.8193298109672427
Test set ROC-AUC: 0.791580008823344
Excerpt from versioned output, inspected on 1 October 2026; not a new execution.

sources and review

Code reviewed on

PsyPredict@06a1661

next: tbook