On Caring Less

I know that David Mitchell is just playing around here, but can someone tell me why so many people really do object to the phrase "I could care less"? It seems to me that the meaning here is obvious. If you say:

I couldn't care less

You're saying it straight. You literally mean that you care so little about something that you couldn't care less about it. But if you say:

I could care less

You're saying it sarcastically. As in, "Oh sure, as if I could possibly care less." Right? Try to imagine a world weary teenager's tone of voice here. So both usages make perfect sense depending on how you say it. Anyone disagree?

UPDATE: Sorry, I failed to be as explicit here as I should have been. My fault. My argument here is that "I could care less" began as a sarcastic version of the phrase, and although sometimes it's still used that way, it's also morphed into being used with standard intonation. So you hear it both ways these days. In other words, just ordinary idiomatic language evolution.