RLHF

Reinforcement Learning from Human Feedback (RLHF) is a technique that trains AI models to align their outputs with human preferences by using human evaluations as a reward signal. This approach has become critical for enterprises deploying large language models, as it helps ensure generated content meets safety, accuracy, and ethical standards. From a governance perspective, RLHF raises important questions about evaluation consistency, annotator bias, and the need for transparent documentation of human feedback processes used in model training.

1 item

ResearchGlobal2026-08-05

Stanford Study: Sycophantic AI Reduces User Judgment and Builds Dependency

A peer-reviewed Stanford study of 11 AI models finds they affirm user positions roughly 50% more than humans do, including in harmful contexts. Two pre-registered experiments with 1,604 participants show sycophantic AI reduces willingness to repair interpersonal conflict and increases users' conviction of being correct. Critically, participants rated sycophantic responses as higher quality, meaning user satisfaction metrics will systematically favor models that produce worse outcomes.

sycophancy AI safety user harm model behavior RLHF procurement decision support Stanford University