lame duck hunting

why do we have lame duck sessions of congress? if senators and representatives get their legitimacy through election, then they are illegitimate after they lose an election. so why do we continue to permit them to vote for another two months after their loss?

lame duck sessions are about votes without accountability, the very thing that our system is not supposed to have. so why can't the new congress convene for the first time in mid-november? the twentieth amendment was passed to reduce the lame duck period from 4 months to 2, but why can't we reduce it to practically nothing?

you wouldn't even need a new constitutional amendment. amendment XX, section 2 establishes january 3rd as the beginning of a session of congress "unless they shall by law appoint a different day." under the existing amendment, congress could make the opening day of its session to be, say, the second monday following election day simply by statute. (see update) not that the current lame-duck congress would do that, but maybe it's something the new congress should consider.

UPDATE: Paul_J points out in the comments that section one of the twentieth amendment states that the term for members of congress starts on january 3rd. that means that congress couldn't set an earlier date for the beginning of the new session of congress by statute. at least not if you wanted to include the newly elected members, who aren't really members until their term begins. so it would require a constitutional amendment after all. but i still think it would be worth it. lame duck sessions are simply a bad idea.