Skip to content

Explorations + Code

In this space, I explore the intersection between ML technology and people science (i.e., IO Psychology).

Some of the ML code can be found on by clicking on the icons.

Open In Colab Generic badge

Support This Site

Recent Post(s)

🧠🚀 Scaling Expertise

🎙️ GPT-2 Interview Response

🏷️ Zero-shot Classify Big 5 Personality

🧠🚀 Scaling Expertise 🔗

Feb 25, 2021 📖 < 9 min read

TLDR: Snorkel is a fitting framework that promotes SMEs ability to impart their wisdom to scale.

Specifically, we 👇

  • Programmed functions in Python that mapped onto our SME ground truth gold labels
    • Zero-shot predictions for the 35 factors/facets of the Big 5 personality taxonomy
    • TextBlob sentiment
    • Pattern-based heuristics (i.e., keywords)
  • Created a generative model based on accuracies and correlations of our labeling functions
    • Programmatically labeled all of our unlabeled responses
  • Trained a machine learning model on all (previously unlabeled) data
  • Strategy works with guidelines and ethical considerations for assessment center operations

🎙️ GPT-2 Interview Response 🔗

Nov 12, 2020 📖 < 5 min read

It's been a turvy topsy year. I'm looking for someone or something to help lighten my workload. Let's interview the GPT-2 artificial intelligence language model – the gpt-2-simple package – for the job 🤖

TLDR: After learning more about GPT-2 artificial intelligence's working style I am feeling optimistic.

Specifically, we 👇

  • Interviewed GPT-2 and asked it to describe it's work style
  • Evaluated the GPT-2 interview response in terms of the O*NET work styles (WS)
    • Top WS: Concern for Others, Attention to Detail, and Social Orientation
    • Bottom WS: Self Control, Stress Tolerance, and Achievement Effort
  • Used zero-shot classification to evaluate the responses
  • GPT-2 response appears in keeping with the following occupations 👇

🏷️ Zero-shot Classify Big 5 Personality 🔗

Oct 27, 2020 📖 < 9 min read

Zero-shot learning (ZSL) – from the Transformers package 🤗 – is an exciting approach to classify text responses in terms of a label or set of labels not explicitly trained by a model.

TLDR: zero-shot looks like an excellent tool for lower-stakes measurement, but for higher-stakes settings such as evaluating someone for a job we need further evidence.

Specifically, we 👇

  • Classified scenario based text responses in terms of the Big 5 personality traits
  • Psychometric validity evidence of ZSL was pretty encouraging
    • Face validity – ZSL scores passed the eyeball test
    • Convergent validity – ZSL scores were positively related to self-report scores of corresponding traits
  • ZSL was a bit overzealous compared to expert gold standards
    • ZSL did a nice job of classifying agreeable responses as agreeable
    • ZSL struggled to classify only relevant responses as agreeable
    • ZSL maintained an inter-rater agreement/reliability approximately 50% to goal

Support This Site