“Is it fair that someone with greater credentials and interviewing skills than a minority should lose out at a Big 4 chance simply due to diversity reasons?”
Nope. It happens, and I don't think it's fair. But then again, life isn't. People born in the majority have a built-in advantage that has nothing to do with their talent or skills. So IMHO, it's less of a question of fairness and more of a question of what you can do about it.
The answer is, not much beyond trying to maximize your potential. Because really…what else can you do?
“I feel that this happens a lot, and it specifically has happened to me”
Meh. I've seen it happen, and I've been involved in it personally. But I'd hardly say it's “a lot”. It happens because the firms ARE getting pressure to hire diversity candidates. This is especially true in areas where the diversity population is high. Think NYC, California, Texas, Florida, etc. I'd say it's very likely that the pressure to hire minorities in a predominantly Caucasian city, say Ohio, is a lot less than in Miami, FL.
However, I will say that from my experience, these “diversity” hires are usually pretty bright, and much of the disadvantage is mostly cultural. Hell, this isn't just a race/ethnicity issue either. Take a person of the same ethnicity whose father is a CEO of a midsize or large company. Then compare that same person to someone who is a first generation college student, parents are blue collar, and has little exposure to the business community and its social norms. You'll definitely see a difference in polish, social intelligence, and other subtle qualitative factors.
ANYWAYS….
My main point is that the firms are making a concerted effort to build their minority credentials because they A.) Are getting social pressure when they see a lot of white men in Senior positions, B.) Understand the shifting demographics of the nation, and C.) truly believe there are great candidates of different background waiting to be discovered.
So, TL;DR, The decision to focus on hiring minorities is a business decision which is legal, in their best interests, and as unfair as a lot of aspects in our society that we accept.
My advice is to move on from this and focus on improving things you control. Because if other non-minorities are getting hired by firms, it means you're not good enough (yet).
.