"Build only pull requests" feature disable all builds except for master


I enabled the Only build pull request setting under Advanced Setting, but doing so actually stops all builds except for builds on master (the primary branch of the repo). This is the opposite of what I want–I want to ONLY build commits with a corresponding PR, and ignore all other branches (except for master, which I understand is by default the branch that is always run).