[Anonymous]

Would you please enlighten me on how the scores are curved? If for example, a sim tab is given and a majority of students did not do well (either they don’t understand the wording or it is difficult to do it within the time constraints), then what happens? Does the minority high graders who did do well get low scores to fit within the normal bell curve? Or are these sim tabs not considered in evaluating the score?

I have read the article in JOA describing the scoring process but I need to know more about curve grading. This is more bothersome because of all the changes this year in the sims section. Assuming the examiners haven’t tested all the new sims, what would you do if you find that majority students did not do well on them?

If anybody else has knowledge about this, please feel free to reply.

[kb24]

The AICPA specifically states that they do not curve scores. I haven't seen anything about how they are evaluating the new SIMS except that the change in SIMS is the reason they need to do the additional quality control evaluation that is causing the scores to be issued later for the first three test windows in 2011 than in previous years.

FAR 4/1/11 - 89
AUD 4/15/11 - 85
REG 4/29/11 - 80
BEC 5/13/11 - 85

[Anonymous]

kb24 is right, AICPA denies curving scores…but 90% of us believe they curve the exams to some extent. They may not curve the final overall score, but they are kind of curving it when they try to determine which questions are easy, medium, difficult based on the number of people who got it correct. I mean, they may put a ‘difficult' question in a testlet but then change it to ‘medium' if 99% of people got it correct. None of us really know how they grade the exams, but we have plenty of theories. 🙂

[Minimorty]

I guess I just dont understand the interest in knowing precisely how the exam is curved. Does it have any impact on how you study? Does it have any impact on how you take the test? Just study your butt off, do the best you can, and let the pieces fall as they may. If you pass then move on to the next one and if you fail then study harder the next time.

[Anonymous]

Yes it does impact me. I gave REG in Apr end and now it is June. I am hardly able to study for AUD. I think I am that kind of person who needs closure with one exam to be able to go on with the next. This wait till June end is too hard. I think what you are saying is very practical and the right way but I guess I am not able to get rid of my nightmares about REG. Until the scores come, I am just looking to get some knowledge, some assurance and some distraction. Nothing much. But thanks for saying that:) My main rock of support and motivator is on vacation (my sis) so I am digressing here.

[Minimorty]

Yeah, I think everyone would agree that the wait sucks. But if you dont move on to the next one, you'll be losing over a month of studying for each exam score you are waiting on. You would hate for your first passed exam to drop off because the 18 month window has expired. If you think you are borderline on REG, I suggest moving on to the next one. Try not to get worked up over the scoring process. Its out of your hands.

[Anonymous]

Thanks for the motivation. I will try. Best of luck with your studies too.

[HighMileageDude]

While the exam isn't “Curved,” as in BONUS points awarded to everyone because everyone did poorly, the exam is adjusted/scaled from 0-99, and you can be assured that 40-50% are going to score >75+

Each multiple choice question has different weights assigned to it based on 3 factors defined in great detail (psychometrics) by the AICPA. You can't really spend time thinking about this too much. Generally, You just go into Prometric, give it your all, and hope you come out better than 55% of other testtakers.

[adman0214]

Agreed, you just can't tell. The first testlet is always considered a medium difficulty testlet. Based on your performance you either get another medium, an easy or a hard. Questions answered correct on the hard testlet get more consideratioin than a question answered on the medium, and medium receives more consideration than the easy.

The difficutly is not based on the length of the quesions or how wordy they are, but an average taken per question on the accuracy of CPA exam takers. One person could recive a hard testles and think it is easy while another person gets a medium and thinks it is really hard.

SIMS are pre-determined and do not reflect your MCQ performance. If you get really difficult TBS then you are just unlucky. The hard part is…as I mentioned a hard testlet could be easy and an easy testlet could be hard for the test takers. There really is no true way to estimate how you have performed until the score appears.

Gool luck to everyone!

REG: Passed,(expired 4/30/11), Passed - 87 (2/29/12)
BEC: Passed,(expired 2/29/12), Scheduled May 2012
FAR: 72,74,74(C'MON!), (Schedule 4/20/12)
AUD: Passed

[Trying Again]

I would say that HighMileageDude just hit the proverbial nail on the head. According to the AICPA the exam isn't “curved”, but you can be assured beyond a reasonable doubt that there is a very good reason why a multiple choice and fill-in-the-blank exam takes nearly three weeks to score after the last person takes the exam. They are waiting until everyone takes it so that they can adjust the passing rate to between 40-45%. They have to keep the cash cow mooing somehow.

Anyone can go to guidestar.org (free registration) and see the AICPA tax return. In 2009 the AICPA took in over $22 million in CPA exam fees. The CPA exam is a revenue generating activity for them.

I can go take the GMAT or any other multiple choice and fill-in-the-blank exam and have the score in a matter of days from when I actually took the exam. For some exams you can actually leave the testing center with a score in hand.

Licensed CPA since September 2011

[HighMileageDude]

https://www.nccpaboard.gov/clients/ncboa/public/static/How_the_CPA_Exam_is_Scored.pdf

For weighting of questions, read under FAQ here…. the following are direct quotes from nccpaboard.gov link above:

“5. In college some of my professors gave tests that had questions that were worth

one point and others that were worth two points. If one student got five of the one-

point questions right, he got five points. If another student answered five two-

point questions right, he got 10 points. Is that what you're doing?

Conceptually, yes. But the professor assigned the weights based on judgment. In the

CPA Exam, the weights are determined from candidate response data using Item

Response Theory (IRT). “

“14. What do you mean when you say “statistical characteristics” of test

questions?

There are three statistics used to describe the questions: Difficulty – whether the

question is generally easier or more difficult for candidates, Discrimination – how well the

question differentiates between more able and less able candidates, and Guessing – the

chances of candidates answering the question correctly just by guessing. The statistics

are generated when the questions are administered as pretest questions and used in the

scoring when the questions are operational. The formulas for generating the statistics

and scoring the Exam come from a scoring approach commonly referred to as “Item

Response Theory.” Item Response Theory is currently being used or has recently been

adopted by nearly all of the large licensing examination programs in this country and

also by many of the moderate-sized and smaller examination programs.”

[HighMileageDude]

@Trying Again

My understanding is that they are supposed to have some predetermined idea of what passing is and what it should be, which AICPA defines below as what it takes to “ensure the protection of public interest.” (see below)

But we all know that passing scores for each section range from 41-50ish% every quarter, usually 45-50%.

Heck, the lowest pass rate in years was for Q1 REG 2011 (last quarter) as it was still ~41%!

So it just so happens 40-50+% of testtakers each quarter have what it takes to “ensure the protection of public interest.”

https://www.nccpaboard.gov/clients/ncboa/public/static/How_the_CPA_Exam_is_Scored.pdf

“16. How was the passing score set for each section?

Volunteers who are licensed CPAs with recent experience supervising entry level CPAs

participated in a passing score study. They reviewed test questions and how candidates

performed on those questions in order to make judgments of what test performance is

required to ensure the protection of the public interest.”

Does this mean if I got a super easy question in my 90MCs like what does GAAP stand for and I answered it incorrectly and I got everything else correct that I automatically failed the ENTIRE exam because I DONT HAVE what's “REQUIRED TO ENSURE THE PROTECTION OF THE PUBLIC INTEREST?” 😉

[HighMileageDude]

@kb24, is the new TBS format the only reason for the delay in scoring process? Obviously that's the most drastic change to the 2011 CPA exam (still 90mcs, but 6-7tabs). I thought the delay in scoring was also in part due to introduction of IFRS into the 90Q MCs?

Per the AICPA email we all received:

“Because of the significant changes made to the content and structure of the Uniform CPA Examination in 2011, there have been comparable changes to the scoring process, all of which requires more extensive psychometric analyses of both test questions and candidate performance for the first three windows of this year. Sufficient data needs to be aggregated for the required additional analyses and the data must come from actual, operational exam results. For each testing window we have to acquire a sufficient sample size of exam results in order to perform the required analyses and score the exams properly. This process takes time. Therefore, for the first three testing windows of 2011 scores will only be released after the end of each window.”

I can't tell from reading this if it's just the new SIMs format causing the delay…looks like they have to score 90MCs on a 0-100 scale and the 6-7 TBS on a 0-100 scale and then have to test out the WEIGHTING of that to a 0-99 lol

[kb24]

The following are some quotes I found on the AICPA website regarding the exam. Sorry for the long post, but I thought the info might be of interest to some following this thread.

@HighMileageDude—-The first two quotes only mention the change in TBS as the reason for the delay in score release. While FAR and AUD were affected by the addition of IFRS and ISA to the exam, REG and BEC were not. Also, the examiners are constantly adding new info to the exam as new standards and regulations are passed. My opinion (based solely on my thinking and not on anything the AICPA has issued) is that the addition of international standards is an example of normal additions to the exam over time.

The third quote is about the passing rates. While I agree that it's a little suspicious that the exam passing rates are fairly constant, think about the people taking the exam. I don't know about you, but even my toughest college exams were a piece of cake compared to the CPA exam. Also, aside from one professor who used a few of the Gleim book questions on his exams, not one of my professors ever talked about how difficult the CPA exam is or how well you need to know the material. How many people go into this exam without preparing sufficiently? My guess is that most of us who've taken at least one section don't, but there are thousands of people taking the exam for the first time each quarter. I know that when I bought my first Wiley book before I started studying for the exam I thought about it in terms brushing up on some of the courses I took in my first couple of semesters rather than intense study. There's a post somewhere on this forum that says you have to know the material rather than just be familiar with it in order to pass and that there's a big difference between the two. I now realize that I was just familiar with a lot of what I thought I knew when I graduated.

The fourth quote is from the MDA of the 2009 annual report. Yes, the exam costs us a lot and does generate revenue for the AICPA. However, not all the money we pay goes to the AICPA, and they have expenses associated with that revenue. I'm not defending the AICPA, but I do think we should consider all the facts rather than just part of them when forming an opinion.

I'm as frustrated as anyone by the wait for the scores, and I'm not an apologist for the AICPA. Yes, the difficulty factor is based on judgment which may or may not apply to you. However, my guess is that we'd all agree on the vast majority of relative ratings if we had access to them. Wouldn't you rather that the difficulty was recognized rather than having an obviously easy question count the same as an obviously tough one? There's no way to completely eliminate bias, but the exam uses a testing theory that's been widely recognized and used for several years on other exams such as the GMAT. Every licensed profession has hoops that you have to jump through, and the CPA exam is ours. It's our choice whether or not we choose to pursue a CPA license. We can gripe all we want about the wait for scores and the scoring method and the cost, but the only thing that does any good is to prepare like hell and do our best with the questions we get.

—from scoring FAQ—-

Q. Why will all scores be released only after the close of the window during the first three quarters of 2011?

A. The main reason that scores will be released after the close of the testing window during the first three quarters of 2011 is to gather candidate performance data for the new Task-Based Simulations (TBS). Candidate data from a full testing window is required to provide enough data for a proper analysis and validation.

The transition from standard simulations to TBSs in CBT-e makes faster score reporting possible since TBSs may be pre-tested. Pre-testing allows us to gather question performance data, as we have done with multiple-choice questions (MCQs): once we have enough statistical data, we will be able to score TBSs as quickly as we score MCQs.

—-from exam bulletin 3/11—-

As you may know, scores for the first three testing

windows of 2011 will be released to NASBA after

the close of the testing window. It is important to

note that scores are not being held until the end

of the window. Rather, the magnitude of changes

introduced by CBT-e require extra analysis to

verify that questions accurately capture candidate

performance, and ensure that only qualified

candidates enter the CPA profession. Additionally,

this analysis will enable scores to be released

faster and more predictably starting with the fourth

testing window of 2011.

The transition to Task-based Simulations (TBS) in

CBT-e, from the “classic simulations” used on the

CPA Exam before January 2011, is a key component

of the faster score release timeline in the fourth

testing window of 2011. The primary benefit a

TBS has over simulations used previously on the

CPA Exam is the ability to be pre-tested. Pre-

testing allows us to gather candidate and question

performance data, as we have done with multiple-

choice questions (MCQs): once we have enough

statistical data, we will be able to score TBSs as

quickly as we score MCQs.

Data from a full testing window are required in

order to properly analyze the performance of the

TBS questions. As a result, score releases for the

first three quarters in 2011 had to be delayed until

after the close of the testing window.

—from CPA exam passing rates page—-

In reviewing passing rates, it is important to remember that candidates are evaluated against an established standard of competence, and that the examination is scored and scaled so that scores are comparable across test forms and over time. The examination is not harder or easier to pass at different times. An increase in passing rates simply means that candidates are better prepared.

—from CPA annual report 2009—-

The computerized CPA Examination (Exam) is delivered through a

tri-party agreement (Agreement) between NASBA, Prometric and the

AICPA. The AICPA accounts for the Exam on a break-even basis over

the life of the Agreement. The AICPA had approximately $40.7 million in

net deferred costs associated with the Exam reflected in its Combined

Statement of Financial Position at July 31, 2008.

Fiscal 2009 marked the third straight year where the Exam’s revenue

exceeded expenses, as the AICPA recognized revenue of $21.5 million

and expenses of $14.1 million. Accordingly, costs equal to the revenue

recognized in the current year have been expensed, with the excess

of revenue over expenses of $7.4 million recognized as a reduction of

deferred costs. As such, at July 31, 2009, the balance of approximately

$33.3 million of net deferred costs is reflected in the Combined Statement

of Financial Position. Moreover, in July 2009, the one millionth Exam

section was administered. Since the Exam launched in 2004, the deferred

asset balance is more than $12 million lower than management’s

projections due to higher candidate volume and lower expenses.

Three main factors affect the recoverability of deferred costs associated

with the Exam and are tested annually for impairment: (1) future

candidate volume; (2) future price increases; and (3) cost management.

Given these factors and the financial performance in Fiscal 2009, the

projections continue to reflect the AICPA’s belief that the deferred costs

associated with the Exam will be fully recovered over the life of the

Agreement.

The candidate cost to take the examination is set by each licensing

jurisdiction and currently ranges between $978 and $1,293, assuming a

candidate takes all four parts. The AICPA currently earns $95 per section

or $380 for a candidate taking all four parts of the examination regardless

of the candidate charge set by the licensing jurisdiction. Although price

increases are permitted under the terms of the Agreement, no price

increase is planned for Fiscal 2010 or 2011.

FAR 4/1/11 - 89
AUD 4/15/11 - 85
REG 4/29/11 - 80
BEC 5/13/11 - 85

[Herbieherb]

Great post kb

NEW YORK- DONE

[HighMileageDude]

@kb24 Yes Thanks very much for posting this – great info! wasnt trying to be a pain beating this question again and again, but it's fun to learn everything about this whole process as we're awaiting scores Thanks

[Author]

Replies