No standard off-the-shelf survey can adequately describe the realities of every organization or the various positions within an organization. Organizations need to customize surveys so they align with actual employee performance and the organization’s unique business practices. To be considered valid, what’s being measured should accurately describe what an individual does at work. The survey should adequately sample the most important observable competencies of a given job. The following guidelines are an effective method for validating a local survey.
Assemble a team of Subject Matter Experts (SMEs). Identify five to seven people who understand the position. This group may include current and former successful job incumbents and managers who are familiar with the position. Include internal or external consultants, if involved. Frequently, the internal HR person facilitates the process and the external consultant contributes expertise.
Describe the responsibilities of the rated position. Collect as much information as possible about the job as it is presently being done. Have SMEs discuss how the job is done on a day-to-day basis. Determine which aspects of performance are crucial. Discuss examples of success and failure, either real or simulated. Record all information.
Describe the knowledge, skills and abilities (KSAs) needed to perform successfully. Determine which are the most important KSAs for the job. Determine whether others can observe these behaviors. The hard-to-measure behaviors become competencies to assess with a 360-degree feedback survey. Use existing items from your survey library or on-line resources to build a set of scaled responses. Refine the survey to include only the most important competencies. Revise the list of competencies.
Include other stakeholders in the final review of a competency list (e.g., senior management, direct reports, internal and external customers). Solicit input from qualified professionals outside the organization—other HR professionals or experts in technical aspects of the job. Revise the competencies as appropriate.
Pilot test the survey in the organization. Administer the survey to a sample group. Collect feedback about the survey from the participants. Revise if necessary. After the survey has been reviewed and confirmed by everyone who understands the position, it may be considered valid for the position.
Periodically review the survey and repeat the process. Over time, the requirements and expectations for a position can change. Review the survey every one to two years to determine if competencies and survey items are still valid for the position requirements.
