I worked in the IRS in both low level department (lv5 mostly) and large department (Large business and international). I will tell you this: the IRS has real professionals as well as lazy worms.
In the low level department, even the managers are a group of nonsense makers. Let me give you these details:
-They changed my team management in a sudden, and a manager who didn't know about what my team was doing came and managed us. When there was a decision that must be made, he said “why don't you ask cycgundam (my name-I was the lead)?”
-Managers played, joked, and even flirted with each other. Also other group managers liked to take a seat in my team's business (even the birthday party, which they weren't even related). I guess that they were just too boring and had nothing to do.
-My duty hour was from 4 pm to 1 am. But the IRS system shut down at 00:00. The entire department actually couldn't do anything after 12 am. So there would be chatting, partying, and laughing. The IRS paid us 20 bucks a hour to play with each other for 1 hour every working day.
However, in the LBI department, things change. Employees in this department are sometimes lazy too, but they are real-life professionals. Someone of them come from big 4 or big law firms. They may feel a release and gain work/life balance but their working schedules don't change after transferring to the government agencies. I came with one agent to the field one day. The guy showed up in 7:30 am in the morning and well suited. We went to a large corporation to do tax audit. We dealt with tax professionals from KPMG. I couldn't reveal too many details but I could tell you this: the IRS large business agents are really good. I also had a chance to work with a tax attorney in a tax controversy with a large insurance company (very famous one). During the meeting, the director of that insurance company worked up and charged the IRS for something (weird). That tax attorney sooner figured the situation out and subdued the director in a professional way. Then the director calmed him down and apologized to us.
Let me be clear. Some government agencies have to deal with large corporations, and somethings they are on a different side of a table. The common sense is that to fight a strong enemy, one must be strong. So the higher level (titles, skills, or influence) you stay at, the more you witness about the true power of the IRS. If you think that the IRS agents are lazy, it pretty much tells me that you are not a big role in your company, firm, or profession (no offense).