Matthew Yglesias touches on some of the issues raised in Jacob Hacker and Paul Pierson's new book, Off-Center, on why congressional elections are so uncompetitive:
As a general matter, gerrymandering is less important to the declining competitiveness of congressional elections than is generally believed. State boundaries haven't changed at all in decades, but Senate races are becoming less competitive, too. The main problem here is that elections are becoming increasingly expensive. Since the parties have only so much cash to spread around, they tend to target only a few races, leaving most of the country grossly uncompetitive.
Well, I'm for public financing, and gerrymander reform, on principle, but one can't rule out a third factor here: namely, that the parties have plenty of cash to spread around, but they spread it around counter-productively. Here's a recent paper by Jonathan Krasno and Donald Green of Yale:
[P]arty campaign committees generally take charge in identifying the hottest races. They interview candidates, evaluate their chances of victory, and establish funding priorities. Their allies among interest groups and private donors follow their lead, opening their wallets for contenders in targeted races while rebuffing less likely candidates. The results of this targeting process are evident in FEC reports. In 2004, 33 challengers spent over $2 million while nearly 200 spent less than $100,000 .
Doing so leads, Krasno and Green argue, to diminishing returns:
[I]t is hard to peg the exact point at which the returns from campaign spending become so negligible as to be worthless. Still, it is safe to say for the vast majority of candidates that the impact of expenditures beyond $1 million is heavily attenuated. What that means is simple: spending past $1 million gains far fewer votes (and maybe none at all) than does earlier spending.
Seventy challengers in 2004 spent between $100,000 to $500,000, and 19 of them won at least 40 percent of the vote. Boosting their spending by as little as $50,000 or $100,000 would have a discernable effect on their chances, while increasing expenditures by $500,000 in an expensive race would likely have little effect. Parties ignore long shots because viewed individually no single candidate has a particularly good chance of winning. But as a group, long shots are ripe with possibility because of their numbers and because their low spending gives parties a chance to influence their chances. Targeting overlooks many potential winners....
To be clear, I do think gerrymandering and a lack of publicly-financed campaigns stand in the way democracy, but it also seems likely that Democrats aren't doing all they can to stay competitive in the current system.