Like all presidents, Carter was part baby and part bath water. The Camp David Accords were probably his greatest success, and his important role in actually beginning some of defense weapons system development that took off in the 80s is often understated. At the same time, by the time 1980 rolled around, between his management of economic conditions, energy, Iran, and some other matters, the reality is that as a country, we all felt pretty beat up and, frankly, adrift - the 1980 electoral map doesn't lie on that front.
I'm a big believer that we tend to get the president we need at any given time (whether we like them or not), so it's also important not to forget the front end. The late 60s and the first half of the 1970s were a train wreck - politically, morally, economically, culturally, and most importantly, in terms of the collective values that are about the only thing that defines what an American is. So yeah, he also inherited a shit show, and he did a lot to right the ship on those fronts.