OpenAI's Project Lily Leaked: Human Reviewers Read ChatGPT Logs for Model Alignment
A leak reported by 404 Media revealed OpenAI's internal initiative codenamed "Project Lily," where human contractors read and score real ChatGPT conversation logs. Reviewers paid over $50 per hour evaluate response quality to eliminate robotic phrasing and sycophancy, though anonymization filters sometimes fail to filter out sensitive personal data. The leak underscores significant privacy concerns, as many users confide personal secrets in ChatGPT without realizing humans may review their conversations. It also highlights how LLM fine-tuning and alignment still rely heavily on extensive human labor rather than purely automated technical upgrades. Contractors evaluate responses based on detailed guidelines that forbid anthropomorphic claims or preaching tones, accompanied by user memory summaries containing context and potential location details. Furthermore, opting out of data collection does not retroactively remove chats already scraped into evaluation datasets.
## BACKGROUND
Large language models rely on Reinforcement Learning from Human Feedback (RLHF) to align AI outputs with human expectations regarding tone and helpfulness. Most AI platforms default to enabling data collection for free and paid consumer tiers, while disabling it by default for enterprise and educational accounts.