I went with everything should be down by Martin Luther King Day. I usually try to get my stuff away the weekend next after Epiphany. My mother always insisted that everything be put away New Year's Day--though she had started decorating in early December, so by New Year's Day I imagine she was pretty sick of the stuff.
A caveat. If the outside of your house is tastefully decorated in white lights, I have no problem with keeping those lights up and lighted at night until, perhaps, sometime in February, by which time the days are noticeably getting longer. Just a personal taste.