If you look at the timeline, the time between movies is not very consistent with Bond. There are a couple of 4 year gaps, and there is also a 6 year gap in there.
The Bond fan base doesn't just disappear in 5 years and "who the new bond is" makes big news for a reason.
The Bond fan base might eventually go, but now is not that time.
Also I'm not entirely sure that the Bond fan base is comparable to the Marvel, Star Wars or Harry Potter fan bases. I don't necessarily think they feel the need for a large universe. They just want a movie about a British man of action who shoots people and blows things up to take down impossibly well funded international criminal organizations and bangs lots of extremely attractive women in the process.
In fact I might argue that they are more resistant to change than the other film franchises. Imagine the meltdown that would happen if a Bond started liking his Martini's stirred. The meltdown if Bond where gay. The meltdown if he actually banged Moneypenny (This hasn't been done has it, again I've only watched a few of the movies.)
This is even though it's basically acknowledged that James Bond and the 007 title are just things that pass from person to person, they still want each and every James Bond to be pretty much the same guy.
I don't think you can draw the comparison between them and Harry Potter fans.