At GoDaddy, engineers across every team are wiring AI coding tools into their daily workflows. Most have a CLAUDE.md or system prompt full of instructions they assume are helping. But how much of that guidance actually improves the code the model writes, and how much is just noise?
Johnathen Chilcher, Senior Site Reliability Engineer at GoDaddy, set out to answer that with data instead of intuition. What started as 1,458 Python benchmarks has grown into 212,000+ controlled runs across Python, Go, JavaScript, and C#, tested on Claude Haiku, Sonnet, and Opus. The results are counterintuitive: across the original Python runs, no prompt configuration consistently beat an empty prompt, and there was a strong negative correlation between the number of instruction tokens and code quality. In other words, telling Claude how to write good code (restating best practices it already knows) tends to make the output worse, not better.
The post turns that finding into a practical framework. Generic coding advice is noise; what pays rent in a system prompt is knowledge the model can't have from training — your build commands, project structure, team conventions, and current context. It also breaks down where the advice flips by language (chain-of-thought helps Go and C# but hurts Python), how tone matters (encouragement helps, high-pressure urgency backfires), and how to match instructions to the specific model you're using. Chilcher also owns up to earlier advice in the series that the newer data proved wrong.
Read the full post: What an Effective AI Coding Prompt Actually Looks Like
Explore the open-source benchmarking tool: claude-benchmark
