People doing their best job at comparing HS athletes from different schools and different parts of the country.
For what purpose? College coaches do their own evaluations. The sole purpose of these ratings is to sell memberships to their sites and to make money off getting kids to their camps. Period. Most college coaches could care less what 'stars' an athlete got running around in his shorts at a Nike camp in Chicago.
Do coaches use these 'star' ratings? Who?
Think about it, Were coaches able to evaluate and recruit kids prior to these 'services'? Of course they were, thus they are worthless. What school has improved their recruiting since the advent of these services and their 'stars'? Name one.
They are simply an entertainment venue meant to make money, Period. They don't help anyone recruit. They are for 'fans'. Coaches don't sit around waiting for 'ratings' come out to know who they want to recruit and kids start contacting schools they are interested in long before they are 'rated' and have 'stars'. People need to go read how recruiting actually works.
If they were completely inaccurate, though, no fan would ever pay attention to them at all. You're making them out to be pointless or worthless compared to coaching evaluations. I don't buy it. No one puts the kind of money these recruiting sites do for following and evaluating high school players without some sort of benchmarks for evaluative success. For example, if their rankings from five years ago were evaluated and proven to be completely wrong, that Florida State's recruits were actually way worse than Central Michigan's based on on-field performance, you'd definitely see personnel changes at Rivals or 247Sports. the idea that this is for "entertainment" and therefore requires no effort or expertise at all is ludicrous and even insulting. The one thing recruiting rankings have over specific coaching staffs is the volume of video evidence and evaluation of players from all over the country. No team has the resources on the coaching staff to do what Rivals or 247Sports can do, which is evaluate vastly larger numbers of players.
Coaches specialize in relationships with high school coaches, camps, and specific areas (unless you're Alabama or Ohio State and they can go after the most obviously exceptional high school athletes; their jobs in recruiting are much easier than Iowa's or Central Michigan's). In addition to that, there are scheme considerations for both high school players and coaches. Wisconsin runs a 3-4 so they want different types of defensive ends and outside linebackers than, say, Iowa may want. If a player is so good and versatile it may not matter, but those guys typically wind up with the blue bloods, though not always. So finding guys who also fit team culture in addition to scheme is important for college programs.
Rivals isn't doing that level of analysis and it would be damn hard to do it anyway given that most teams don't have 20-year coaches like Iowa does. Whatever evaluations would have been useful for UCLA under MOra now go out the window. There'd be a period of evaluation of Chip Kelly's first couple of seasons necessary to make any type of worthwhile evaluation. Yes, going back to his days at Oregon provides some insight, but who knows how Kelly makes changes between then and now, plus the fact that the rest of the PAC-12 coaching staffs are very different than his time at Oregon. Hell, Oregon has gone through how many coaches the past ten years? Four? Five? A lot fo change.
So for Rivals the analysis has to be on the players alone rather than specific fits with specific teams and coaching staffs and team cultures. It's on that level that the lower-ranked guys, the 3-stars and 2-stars, are less predictable in terms of how well they'll perform at a given school in a given system. Plus, if a team has a crappy d-line and a linebacker recruit stud goes to that school and he's required to make most of the plays yet the d-line can't occupy blockers the way the scheme demands, the LB's play looks worse than it may be and the numbers by performance may seem to "prove" that the guy was overrated.
So, yeah, the recruiting rankings may not tell the whole story, but it's not because the recruiting sites are just half-assing things believing that making up numbers for players and star rankings is just a "crap shoot." It's because of the level of complexities that cannot be accurately anticipated for each individual because scheme fit, team culture, player's coach ability in relation to specific coaches, and more aren't part of the ranking system. Those are variables even coaching staffs have difficulty projecting. You can't perfectly anticipate a guy like Jewell excelling and never getting hurt and having no one in front of him playing at a high level when he's a young player. Nor can you anticipate a guy like Marcus Coker who winds up being an incredible talent as expected running into off-field problems that cause him to transfer and never again reach that potential. Or Adam Robinson exceeding his recruiting value (even by the coaches at Iowa) and then being cut from the team for smoking marijuana after demonstrating on-field success for two years.
So if recruiting is a crap shoot, it's because life itself is too complicated with too many unknown and unmeasurable variables to lead to perfect accuracy. But it's been proven rather well that the five and high-four star guys have been reliably predictable compared to the lower ranked guys. You might get a diamond in the rough or you might get a total flop with a 2-star or low 3-star. I'll take the five stars every day of the week if you can load up on them. otherwise, yeah, you have to trust the coaches are getting the RIGHT three-star guy for their team compared to other 3-star guys that might have been available and willing to commit.