Well, I guess it depends on a lot of factors; I really shouldn't have generalized! I started as a tax associate in a very small office at a big 4 firm, so maybe that's a unique scenario. So consider the below, but it might not apply to your situation.
In my area, if the recruiters know that you're in tax, the first thing they will do is to recruit you for a higher-level tax position, and it's usually going to be corporate tax. If you're interested in tax-only positions and don't intend to do accounting, you'll still have many opportunities to work in different industries. My coworkers have switched to corporate tax positions in smaller companies or become tax analysts.
However, if you're a tax professional looking for accounting positions, it really depends on the experience you have. This is where I think your options are much wider provided you have relevant audit or FAS109 experience. People in my office who started in audit were more likely to switch to tax, but it's much harder to go from tax to audit, or even get an audit rotation.
CPA license in progress....