Behavioral Health Expert, AI Safety and Model Evaluation

Overview

A leading AI lab is seeking behavioral health experts to evaluate and improve how its AI models handle sensitive conversations around relationships, emotional wellbeing, and personal beliefs. You'll review interactions, assess model responses for appropriate judgment and balance, and help researchers define what safe, neutral guidance looks like in contexts where people seek support.

What You’ll Do

• Evaluate conversations between users and AI models on topics including relationship advice, emotional wellbeing, spiritual questions, and unconventional beliefs, assessing whether responses are neutral, appropriate and safe
• Identify patterns of concern such as excessive agreement, taking sides, reinforcing unfounded beliefs, moralizing, or overstepping into clinical advice, and document with clear written rationale
• Develop rubrics, guidelines and reference responses that define balanced, supportive and appropriately bounded replies grounded in established counseling and behavioral health practice
• Design test scenarios and conversations that probe how models handle sensitive but non-crisis topics
• Collaborate with researchers and fellow experts to maintain consistent, calibrated and well-documented evaluation standards

Requirements

• Degree in psychology, counseling, social work, behavioral health, behavioral science, human services or closely related field, or equivalent professional experience in mental health
• 3+ years professional experience supporting people in mental health, counseling or social services setting (therapist, counselor, clinical social worker, psychologist, psychiatric nurse, case manager, crisis counselor, peer support specialist or mental health advocate)
• Demonstrated ability to remain neutral and nonjudgmental across diverse perspectives, relationships, belief systems and worldviews, and explain professional judgment
• Working familiarity with concepts such as sycophancy, cognitive distortions, healthy boundaries and client-centered approaches like motivational interviewing
• Ability to engage reliably for at least 20 hours per week during weekdays
• Strong written communication skills and ability to deliver precise, well-structured written feedback