So, the main difference between my 12.04 and 14.04 runs is the version of Qt5. I mentioned that we were installing Qt5 in 12.04 ourselves in each build, and were not doing that anymore in Trusty cause it’s already installed there by default. However, the default version that comes with Trusty is 5.2.1 (https://circleci.com/docs/build-image-trusty/), which is different from the one we’ve been successfully using with Precise (5.4.2). Weird that nobody else has ran into this, right? But apparently it could happen in certain (rare) cases (one ref here: https://gist.github.com/juniorz/4022134).
As a POC, I installed Qt5.5 on my Trusty build and it solved my memory issues. 
Thanks to Eric, Tom, and Lev at Circle support team for helping troubleshooting this!