This FAQ and best practice overview is divided into sections that address topics and questions organizations often ask about assessments. Use the links below to jump directly to a section.
Assessment Overview
Why should my organization use assessments?
Well-designed, valid assessments can have a dramatically positive effect on the quality of your hiring process, the quality of your new hires, and ultimately the effectiveness of your organization. Some key benefits of using a valid and consistent assessment process include:
- Hiring better suited employees who possess the right skills, abilities and fit for your organization and jobs.
- Applying consistent objective hiring criteria helps having a more defensible hiring process
- Improving critical business metrics, such as revenue and profitability
- Reducing employee withdrawal behaviors, such as absence and turnover
- Reducing overall hiring costs by allowing only those candidates who are successful on the cost-effective assessments to proceed to more costly steps later in the process
- Presenting a positive employment brand and image to candidates
Is First Advantage certified?
This is a common question, and different people often mean different things when they ask about “certification.” Below are responses to the most common certification questions.
- Is First Advantage or the individual tests provided by First Advantage certified by an official organization?
Within North America, there is no official body, government or otherwise, that certifies tests or test providers. The Equal Employment Opportunity Commission (EEOC), the Office of Federal Contract Compliance Programs (OFCCP), and the Society for Industrial and Organizational Psychology (SIOP) provide guidelines for the development and use of assessments in employment situations. All of these organizations emphasize the importance of using valid and job related assessments.
First Advantage has deep experience developing and implementing assessments that helps its clients’ comply with these guidelines, and meeting these guidelines is one of the primary aims of our consulting services when working with clients.
- Does my staff need to be certified to administer First Advantage assessments?
We do not require client test administrators to be certified to deliver our tests. Our assessments are delivered online which mitigates many of the concerns associated with providing direction and instructions when using paper and pencil assessments. We do, however, offer training for test administrators in how to use our online platform and create and use reports effectively.
It is important for organizations to ensure that employees understand and follow proper test administration protocols as defined by the EEOC and OFCCP. If this is an area of concern for you, First Advantage can offer guidance on test administration or provide more detailed training in test administration as needed.
- Do my report users need to be certified?
We do not require report user (recruiters, hiring managers, etc.) certification. Our standard assessment reports are designed to be easy to use and interpret. We also can create custom reports for clients to further streamline results interpretation.
Back to Top
Determining Assessment Return on Investment and Ensuring Continued Value
Are assessments worth the investment?
Absolutely. Valid, job-related assessments positively affect both hard and soft costs for organizations when implemented following best practices. First Advantage’s scientific research consistently demonstrates that assessments improve client business outcomes, including increased revenue, profit, productivity, improved customer service, reduced costs associated with absenteeism and turnover all of which contribute to an improved bottom line.
Can First Advantage help my organization measure assessment return on investment?
Our assessment scientists have considerable experience conducting business impact studies, which document assessment return on investment (ROI). This team works with clients to identify and measure key business metrics and outcomes, and then demonstrate how assessment results affect these metrics. Typically, results are provided back in financial terms, but our experts have deep experience working with many types of outcomes and reporting.
How do we ensure that the assessment process continues to add value in the future?
Assessments processes should not be static. Assessments should be continually evaluated and refined, as needed, to ensure that they are addressing business needs and hiring objectives. This process starts by appropriately validating and implementing assessments. Following implementation, we recommend conducting periodic business impact studies to identify opportunities for process refinement and improved validity for enhanced financial results.
How often should we evaluate our assessment process?
After launching a new process, First Advantage recommends careful monitoring for the first several weeks to ensure it is behaving as expected. After that initial period, the frequency of evaluation depends on the volume of candidates processed. Organizations processing thousands of candidates per month may want to review their system several times per year, but most organizations would be well positioned to evaluate their assessments once per year.
Note that certain types of changes might require more immediate assessment process reviews. For example, if legal requirements change, assessment processes should be evaluated and revised, if needed, to ensure compliance. Additionally, if the organization or a job changes enough that it alters worker requirements, the assessment process may need to be adjusted to address those new requirements.
Back to Top
Legal Compliance for Assessments
Will using assessments reduce our legal exposure?
Implementing job-related assessments in a consistent manner helps reduce legal risk by providing a more objective component to your hiring process. Further, when candidates perceive the hiring process is objective, research shows that they tend to have a more positive impression of the organization and feel they are being treated fairly. This can also decrease the likelihood that candidates will file complaints related to your hiring process.
What types of legal challenges can be made against my organization?
Any employment decision (e.g., hiring, promotion, down-sizing, selection into formal training or development programs) can be challenged by individuals or audited by the Equal Employment Opportunity Commission (EEOC) or the Office of Federal Contract Compliance Programs (OFCCP).
Many challenges to employment decision processes will claim that procedures are not job related or that they screen out a higher proportion of people in one or more protected groups (i.e., they have adverse impact). Other challenges can come from failing to abide by the Americans with Disabilities Act (ADA) or other laws and regulations. Using a well-developed and consistent selection process greatly reduces these risks, and First Advantage can help you address legal guidelines when implementing an assessment process.
What is adverse impact?
Adverse impact occurs when employment decisions disqualify a disproportionately greater number of candidates from protected groups (e.g., racial/ethnic minority, females or disabled candidates) than from non-protected groups (e.g., non-minority or male candidates). For example, if an assessment screens out substantially more female than male candidates, it may produce adverse impact.
While organizations should work to minimize it, adverse impact is not necessarily illegal if the tool(s) used to make decisions is valid (i.e., it accurately identifies candidates who can perform job-related work requirements).
Is it possible to avoid adverse impact?
Many valid assessments might produce some degree of adverse impact. This outcome often is unavoidable as cultural circumstances often give protected groups different access to educational opportunities or exposure to experiences that build job-related skills and abilities. Thus, valid assessments that reliably differentiate between those who do and do not possess job-related requirements can produce adverse impact.
These outcomes can be mitigated, however, through various activities like careful sourcing and recruiting candidates. In addition, properly designing and validating assessments will help ensure they measure job-related requirements and provide the basis for defending the process.
What do we do if a candidate asks for an accommodation on an assessment?
The Americans with Disabilities Act (ADA) provides specific direction on how to handle these situations. First Advantage strongly recommends that accommodation requests be carefully considered with your legal counsel to determine what reasonable accommodation could be made on the job. Once you have determined if and how to accommodate an individual’s need on the job, First Advantage will work with you around accommodation for assessment administration. For more information on accommodation testing, review First Advantage’s Assessment Accommodation Overview.
How do I make sure that our assessments meet legal requirements?
There are legal and professional guidelines that prescribe how to validate and implement assessment procedures. For example, the EEOC’s Uniform Guidelines on Employee Selection Procedures and SIOP’s Principles for the Validation and Use of Personnel Selection Procedures provide guidance on these matters. Essentially, the critical aspects of these guidelines specify the following:
- Carefully document job requirements (i.e., knowledge, skills, abilities and personal characteristics) using job analysis or competency modeling.
- Use assessments that directly measure critical job requirements identified through job analysis or competency modeling.
- Properly validate all components (prescreens, tests, interviews, etc.) of the employment decision-making process.
- Use multiple data collection methods and sources of information when making employment decisions. This will help increase process accuracy and potentially balance adverse impact from individual components.
- Track outcomes (e.g., passing rates of protected subgroups) for each decision-making component in the employment process.
Back to Top
Assessment Validity
What is validity?
Validity is the degree to which available evidence supports inferences made from assessment scores. That is, validity is all the information that shows how assessments measure essential job requirements and how well their scores predict individuals’ performance and other work outcomes in a target job(s).
Why is validity important?
Validity is essential for supporting the legal defensibility of assessments and for demonstrating their return on investment for your organization.
How important is it to document job requirements?
It’s critical. Professional and legal standards specify that assessments should measure requirements needed to perform the target job(s). The process used to document job requirements is called job analysis or competency modeling. Job analysis processes identify and document the primary work activities and knowledge, skills, abilities and personal characteristics (KSAPCs or competencies) required for successful job performance. So, job analysis serves as the basis for implementing job-relevant assessments.
How is job analysis completed?
Job analysis is a relatively simple process that can be completed using different methods or combinations of methods. Common approaches include using observations, interviews, focus groups, or surveys to ask job experts to document job requirements.
Isn’t there just one type of validity?
Actually, there are different types of validity that focus on demonstrating how assessments are related to job requirements and predict performance. The nature of the assessment and an organization’s ability to conduct validation research should dictate which type or combination of validity evidence should be documented. The following describe the most typical approaches to documenting validity.
- Content validity demonstrates the extent to which assessment content (i.e., its items) represents significant and important job requirements. This type of validity typically relies on job analysis and job expert judgments. For example, a data entry assessment would be content valid for a position when data entry skill is found be an important requirement of that job and the assessment was determined to effectively measure data entry skill.
- Criterion-related validity demonstrates a statistical relationship between assessment scores and measures of job performance or key work outcomes (e.g., sales, turnover). There are two strategies for conducting these studies, including:
a. Concurrent strategy – obtain test and performance data on job incumbents at the same time.
b. Predictive strategy – obtain applicant test scores and performance data after they have been on the job for a
given period of time.
- Validity transportability leverages job analysis data or job requirements information for the target job(s). These data are used to complete an overlap analysis that identifies similar jobs in other organizations in which the assessment has been criterion validated. If the jobs are substantially similar, the validity can be transported and documented for the target job(s).
How do we ensure our assessments are valid and job related?
By following the standards set out in legal and professional guidelines, such as the EEOC’s Uniform Guidelines on Employee Selection Procedures and SIOP’s Principles for the Validation and Use of Personnel Selection Procedures.
First Advantage strongly recommends local validation that follows legal and professional guidelines whenever it is feasible to do so. We have deep expertise in the development and validation of assessment processes that meet these guidelines, and we provide these services to our clients. Organizations also can elect to conduct their own validation studies.
Back to Top
Administration of Internet Assessments
What assessment administration options do we have?
The two primary options are proctored and unproctored administration. Each type has advantages and disadvantages, as well as degrees to which test security and applicant identity can be confirmed.
- Proctored Internet Testing involves administration of assessments by a human administrator that verifies test takers’ identities and supervises the testing session.
- Unproctored Internet Testing (UIT) involves remote administration of assessments, meaning there is no human administrator to verify test takers’ identities and supervise the test session.
What flexibility do we have within these administration options?
There are different methods that can help blend some of the advantages of proctored and unproctored assessment administration. These options include:
- Direct Invitation Unproctored Assessment – Involves administration with no direct human supervision, but assessments are only made available to known test takers.
· Advantage: Reduces potential of test-takers falsifying identities
· Disadvantage: This method still cannot fully ensure test-takers’ identities
- Unproctored Assessment with Proctored Follow-up – Involves combining both unproctored and proctored administration methods where the candidate initially completes the tests without verification. Those who pass the initial screen are scheduled for a proctored session and their scores from the two sessions are compared.
· Advantage: Reduces potential of test takers initially falsifying identities and provide the ability to confirm results
of promising candidates
· Disadvantage: Increases the number of times candidates are tested, which reduces some of the flexibility and
efficiencies of unproctored testing
- Third-Party Proctoring – Involves a third-party administering testing sessions.
· Advantage: This method can remove internal administrative burden or when assessing candidates in larger numbers
is difficult due to space or computer availability
· Disadvantage: This method can be cost prohibitive within hiring processes that are widely distributed or include very
small numbers of candidates
What administration method(s) should we use?
It really depends on the type of assessment and the stakes for the assessment process. Assessment type should drive assessment administration methods in that tests with right and wrong answers (e.g., tests of cognitive ability, like a math test, and knowledge tests) should be proctored. On the other hand, behavioral assessments (e.g., personality or biodata) and some skill assessments are suited for both proctored and unproctored environments.
Second, the stakes for the assessment process also should be a primary consideration. Higher stakes situations typically involve the need to verify test takers’ identities and true abilities or qualifications (e.g., candidates for jobs that involve public safety or when assessment is all or part of a certification process). As the stakes for the assessment process increase so does the need for proctored administration.
Are there any best practices and recommendations for UIT?
Before selecting a UIT administration model, consider the following points:
- Carefully weigh the advantages and disadvantages of UIT and make sure to take into account the type and stakes for the assessment process before selecting an administration method.
- Consider candidates’ reactions to how, when, and how often you will require assessment.
If you decide that UIT is best for your situation, consider the following:
- To help improve candidate identity verification and reduce the risk of assessment sharing or unauthorized use, force the use of a single point of entry. This could be done through your applicant tracking system with user authentication or by sending candidates a one-use URL.
- Use candidate contracts and warnings in the UIT process. These tools should inform candidates what they can and cannot do with assessments and include a warning that cheating or falsification of any type will lead to disqualification from the hiring process.
- If possible, conduct verification assessments in which you follow unproctored testing with a proctored testing session to verify candidate identities and scores.
- Use candidate reaction surveys to monitor how individuals perceive your organization and assessment process and use the information to refine your approach.
- Track assessment results against new hire performance and proactively track your applicant pool to ensure that the assessment process is consistently meeting your business and hiring objectives.
Back to Top
How to Effectively Use Assessments
How much weight should we place on an assessment when making employment decisions?
Assessment scores should be just one piece of information used to make employment decisions (e.g., hiring, promotion, etc.). When considering how much weight the assessment component should carry, the final answer will depend on some key considerations, including:
- How important is having the knowledge, skill, ability or personal characteristic (KSAPCs) to effectively performing the job? If it is very important (e.g., person cannot be successful without the KSAPCs), consider using a minimum score that helps ensure a basic level of those KSAPCs and screen out people who do not achieve that level.
- Another consideration is taking into account what other steps exist in the hiring process. Is there extensive prescreening, interviewing or other activities that are being done in addition to the assessment? If so, what KSAPCs are measured in those other steps and how objective are those other steps? Balance the weight of the assessment based on how much unique and objective information it provides to the overall process.
Note that weighting an assessment does not mean setting a high standard (i.e., passing score). Organizations must determine a reasonable standard based on actual job requirements. From there, First Advantage strongly recommends developing and following a standard and consistent approach for using assessment scores in the overall decision making process.
Should we set a time limit for assessments?
Some assessments are designed and validated using a specific time limit. These assessments should be used consistently with that same time limit to help ensure the reliability and validity of their results. For others, such as behavioral assessments, time limits may not be required, but may be added for efficiency if it allows the vast majority of test takers to complete the assessment within that time. First Advantage tracks assessment completion times and can provide recommendations for setting time limits on untimed assessments.
Is there a best practice in the sequencing of assessments within a test battery consisting of behavioral test and skills tests?
First Advantage recommends that timed tests come first, with any untimed tests administered at the end of the battery.
Should we set passing standards on our assessments?
Passing or qualification standards, also known as cut scores, can be very useful and increase the standardization of assessments and help users better interpret their results.
What do the assessment scores mean?
In general, the higher the score on an assessment, the more likely it is a test taker has a higher level of the knowledge, skill, ability, or personal characteristics (KSAPCs) the assessment measures. If the assessment is linked to job requirements, higher scores indicate individuals more likely to succeed on the job. To effectively interpret scores, users should understand what type of score is being reported (e.g., percentiles, percent correct, etc.) and how scores compare to the norm group for that test.
What is a percentile score and what does it mean?
Many First Advantage assessments use percentile scores. Percentile scores show an individual’s rank or the percentage of people he/she scored above in the normative population. Percentiles range from 1 to 99, where the 1st percentile represents the lowest value (minimum), and the 99th percentile represents the highest value (maximum). A test taker achieving a percentile score of 32 indicates that this person scored above 32% of the normative population. Normative groups include other individuals who have taken the same assessment. Norms are regularly updated to help ensure they reflect current applicant populations.
In addition, many assessment reports divide scores into “performance levels” (i.e., Excellent, Good, Average, Poor; or Substantial Strength, Adequate Strength, Not a Strength). These levels are designed to help users interpret scores based on how likely the test taker is to succeed in this area.
Do pre-set qualification standards exist for assessments?
First Advantage recommends that organizations examine their own unique qualification standards. Professional and legal guidelines highlight the need to determine reasonable standards, based on actual job requirements. Because each job and organization is different, assessment qualification standards should take these differences into account. Note that setting standards to low or too high will limit the usefulness of assessments. Organizations also should take into account potential adverse impact and ability to meet business objectives when setting qualification standards.
Should we allow people to retest?
It depends. The advantage of retesting is that it gives some individuals (e.g., those who are not used to taking tests or become nervous when taking tests) a chance to better show their qualifications. Retesting also may increase candidates’ perceptions of the organization in some cases. Disadvantages to retesting include increased access to the content of the assessment, which candidates could use unfairly prepare themselves or others; facilitating cheating behaviors; and decreased shelf-life of the assessment process. The decision to retest also should take into account the organization’s geography and access to candidates within that area.
How long should we wait to retest?
Again, the answer is that it depends on your specific situation. In many cases, a reasonable retesting interval is six months with a maximum number of retesting attempts set at two. If your organization is pressed for applicants, you might consider allowing a retest as soon as one to four weeks after the initial attempt, followed by a more standard 6-12 month retest interval for subsequent attempts.
Back to Top