Finishing #5 in a meaningless poll probably has little to no bearing on how most fans feel on whether he deserves to be fired. Most fans expectations are not centered around where their basketball team finishes in a subjective poll, at least not the primary expectation. Sure, it's nice to have on a resume, but it doesn't close many deals.
Winning (or consistently contending for) conference championships and making deep runs in the NCAAT are what most fans want. That is how they measure success.
Unfortunately, it's now a moot point, since our genius AD went ahead ad gave him an ill-timed extension prior to the start of the tourney.