From the AICPA Website. why is it so mysterious?
“The Examination is not curved. Every candidate’s score is entirely independent of other candidates’ Examination results.
The Examination is a criterion-referenced examination which means that it rests upon pre-determined standards. Every candidate’s performance is measured against established standards to determine whether the candidate has demonstrated the level of knowledge and skills that is represented by the passing score. Every candidate is judged against the same standards, and every score is an independent result.”
“What scoring method is used to score the Examination?
The AICPA uses Item Response Theory (IRT) for the objective portion of the Examination. IRT is a well-established psychometric approach to scoring used by licensing and certification examinations that administer many different test forms. IRT scoring ensures that scores and pass or fail decisions based on scores from different Examination forms are comparable. Based on the large amounts of data that are collected in pre-testing, the difficulty level as well as other statistical characteristics of Examination questions are known and taken into account in scoring.”
“What is the standard setting process that you will conduct in 2017?
After the close of the Q2 2017 testing window, the Board of Examiners will assemble panels that consist of CPAs with experience supervising the work of newly licensed CPAs. There is one panel of CPAs for each section of the Examination. These CPAs will review test questions and performance data for candidates in the Q2 2017 testing window and approve ratings to recommend a passing score for the Board of Examiners approval.
Once the passing score for each section is approved, it will remain in place until there are significant changes to the Examination in the future. Prior to the 2017 standard setting process, passing scores were most recently set in 2004 when the Examination was computerized, and again in 2011 following a revision of the Examination.
After the close of the Q3 and Q4 2017 testing windows, additional data analyses are require for quality assurance and validation purposes.”