In training right now as a new football ref. Watching the game, covering the game as a journalist, coaching the game at the youth level and being around it for 30+ years I thought I knew most of the rules but wow, looking at the materials and what's expected I've found a whole new appreciation for this even ahead of my first game.
Depending on role (Back/Side Judge/Linesman, etc.) you have to know your keys, responsibilities, where to lineup, what to watch and that's even before there's a penalty, etc. At least in high school football you have a crew, you are fairly far removed from the fans vs. basketball, etc.