"We look at these metrics in aggregate as well as through segmented groups to make sure benefits to one user type (e.g., newbies) don't make things worse for experienced users."
Newbies aren't using the search function. They rely on hints, and attach without evaluation of the data. If the metrics are relying on attachment, well then that accounts for why the search function becomes more useless every day.
This mindless attaching in concert with the massive volume of *alternate* data being entered by inexperienced users and then indexed by the system is having an adverse effect on search results, the pages of irrelevant returns is staggering. Something's gotta give.