If you look at yesterday and say, "Manning had a 400-yard game and threw TDs, and the Colts were at home, and the Chargers played half the game without Rivers and LT... and San Diego still won," then you get worried.
However, you have to look at the bigger picture.
1.) The Chargers might be playing the WHOLE upcoming game without Rivers and LT. And even if both play, you can guarantee they won't be at full strength. A healthier Rivers in the first half against Indy won't be the same as a hobbling Rivers against the Pats. Take a half-second off Rivers' reaction moves and you're talking the difference between "last minute TD pass" and "strip-sack fumble".
2.) The Chargers were the beneficiary of not one but TWO bad tipped passes near their end zone. That right there was a 6 to 14 point swing in the game. That stuff usually doesn't happen with the Pats.
3.) The Pats have a better receiving corps. (Not just talking about Moss, but the whole package of guys.... Moss, Stallworth, Welker, Gaffney, Watson, Faulk, even Maroney now.)
4.) The Pats have a better head coach.
5.) The Pats have a better QB. (Sorry, I know, the debate rages on, but...)
6.) The Pats don't overlook opponents.
7.) The Pats have just shown they can shut down a running game.