Q: What is Computerized Adaptive Testing (CAT)?
A: CAT is the computerized delivery of exam items uniquely tailored to the ability of an individual candidate. Unlike fixed-form, linear exams, adaptive testing delivers items based on the demonstrated ability of a candidate during the exam. With CAT, the difficulty of each item a candidate receives is optimized to measure their ability with the greatest degree of efficiency possible. All English exams for CISSP are administered as a CAT exam. Based on the same exam content outline as the linear, fixed-form exam, CISSP CAT is a more precise and efficient evaluation of candidates’ competency. Learn more about CISSP CAT.
Q: Does everyone take the same test?
A: No. ISC2 maintains a large pool of questions which can be used to generate many different forms of the examinations. To ensure the integrity and security of the examination, the examination forms are updated on a regular basis. Furthermore, multiple forms of the examination are in use at any given time, and candidates are randomly assigned these forms.
Q: Is one test form easier than another?
A: No. ISC2 uses statistical data on how each test question performs to evaluate the difficulty of each examination form. The examinations are carefully constructed in order to minimize variations in difficulty levels from one test form to another. It is possible, however, to have slight variations in difficulty based on the particular combination of test questions that are selected for each examination form. This is taken into account when the passing scores are set or when test forms are equated, so it does not matter which exam form each candidate takes. Equating is a statistical method which allows psychometricians to create different examination forms of equal difficulty.
Q: How is the passing score determined for linear examinations?
A: The passing score (also called a “cut score”) is established by a panel of subject matter experts (SMEs). To have a diverse group of SMEs on the panel representing various geographic regions, ethnicity, practice settings and years of experience is important to ensure that the panel of experts adequately represents the field. The members on the panel are selected from a list of SME volunteers that ISC2 maintains. The list is updated yearly. Based on panel’s recommended passing score, the ISC2 Examination Committee, also called the Scheme Committee, which is made up of members of the ISC2 Board of Directors, finalize the passing score. During this process, the panel, working under the guidance of a psychometrician, discusses the minimum level of competence that is required for passing the examination and obtaining the credential. This discussion focuses on the specific knowledge, skills and abilities that qualified practitioners possess. The panel then systematically evaluates each question on the examination and rates their difficulty levels. The ratings are combined to arrive at the recommend passing score for that specific examination. ISC2 always uses psychometrically accepted standard setting methods to determine passing scores for all of its credentials.
Q: Why is the scale 0 - 1000 when I didn't see 1000 questions for linear examinations?
A: The scale used by ISC2 is similar to scales used by other large testing programs, such as the SAT, ACT, or GRE. The scale has more points on it than there are number of questions on the examination.
Q: What is a scaled score for linear examinations?
A: A scaled score is a different way of expressing a candidate’s achieved score in the examination without distorting the meaning of scores. In scaled score reporting, a candidate’s raw score (number of items answered correctly) is converted to a number within a predefined score range. In the case of ISC2, all raw scores are converted to scaled scores of 0-1000, with the passing scaled score of 700. It is similar to the conversion of temperature from Fahrenheit to Celsius or vice versa. For example, 85 degrees Fahrenheit is equivalent to 29.44 degrees Celsius. In this conversion, the temperature remains the same, but, depending on the scale used, the number is different (85° Fahrenheit versus 29.44° Celsius).
Q: Does scaling the scores affect who passes or fails the exam?
A: The use of scaled scores does not affect whether or not an individual candidate passes or fails the examination. The pass/fail decision is always made by comparing the number of questions answered correctly to the passing score that was established using the psychometrically accepted standard setting methods described above. All candidates who correctly answer more items than the number of items required for passing the examination will obtain scaled scores between 700 and 1000. Candidates who do not answer enough items to pass the test obtain scaled scores between 0 and 699. Scaling also does not affect the rank ordering of candidates. A candidate who answers more items correctly than another candidate taking the same examination form will obtain a higher scaled score; however; in the certification examination, candidates are not ranked in order; they are only informed whether they passed or failed the test.
Q: Why scale the scores?
A: The use of scaled scores allows us to directly compare scores from one examination form to another because the passing standard will always be the same - a scaled score of 700. In the long run, this process makes the scoring of ISC2 examinations easier to understand for all concerned. No matter which ISC2 credential examination form a candidate takes, the passing score will always remain constant of 700. This also provides stability in score reporting, all of which are critical to maintaining equality for all candidates, which provides the foundation for the ANSI/ISO/IEC accreditation of our credentials.
Q: What is the passing proficiency level for adaptive testing?
A: The proficiency is defined as meeting or exceeding the “passing standard.” Candidates must score above the proficiency level in order to pass the exam. A single pass/fail result is calculated on the total of all operational items administered for the examination. ISC2 exams are compensatory exams which allow for a higher number of items answered correctly in one domain to compensate for a lower performance in another domain. If a candidate performs very well in a more heavily weighted domain where a high number of items are included on the exam and only performs “near proficiency” or even “below proficiency,” in a lesser weighted domain where a fewer number of items are included, there is a possibility that a candidate may pass the exam, but there is no guarantee. Domain performance is provided to candidate who do not pass the exam for purposes of diagnostic feedback for future exam preparation, as follows:
Q: Can I find out how many questions I answered correctly?
A: ISC2 does not report to candidates the number of questions they answered correctly or the overall percentage of questions they answered correctly; however; failing candidates are provided with the rank ordering of domains based on their percentage of questions answered correctly in each domain of the examination. This information is provided to assist failing candidates who would know the domains that they need to focus in their further study.