I'd say it has more to do with the turnovers and scoring. He may not be the best defender, but he make the plays that stand out to the casual observer.
I'm not sure Berry was in the right situation, though. He was honestly just better than everyone else.
There are flaws in his game for sure and he's not close to the best defender in the country. That said, he would make an impact for any team in the country. One play that stands out is the strip/run in against Oregon. Best special teamer in the country is probably more accurate.