What does the term “HR Modernization” mean to you? In practicality, it means different things to different people. For example, as it relates to the public sector, HR Modernization often refers to the updating of an arcane process that is bulky, bureaucratic, costly, and difficult to manage. I think that it can be summed up as “This darn process just takes too long! By the time I can offer someone a job, they have already accepted a job with someone else.”
For the private sector, I believe HR Modernization takes on a subtly different flavor. It is typically much easier for private sector organizations to hire who they want when they want because, while they have to follow certain legal guidelines, they do not have intricate civil service rules. The closest they come to that is in a tight union shop such as the automotive industry. As a result, HR Modernization is primarily about “just in time hiring.” They have a need and want to fill it now with the best possible person. It is not that their process is too long and burdensome so much as it is that a faster process improves competitiveness.
Because HR Modernization in the public sector is more complicated—and there is more opportunity to actually make things worse—I will focus on this group of employers.
The most thoughtful approach to HR Modernization I have seen comes from Canada. They conducted extensive research into the kinds of complaints people had about the system, tried to identify ways to improve it, did some pilot testing, and generally approached the process in a step-by-step fashion.
In the US, much of the planning for HR Modernization has been of a “philosophical” nature. It is believed that the system takes to long. It is believed that we understand why, And a “solution” is proposed. No comprehensive research has been done (in my opinion) and the desired solutions are based around “perceived problems” rather than empirically derived evidence . Once the problem has been identified, the next step is to let an RFP to hire consultants or contractors who will provide services that will solve these perceived problems.
The main problem with this approach is, “what if the wrong problems were identified?” One could spend a tremendous amount of money in hardware, software and consultant hours just to find that the quality of the workforce has completely deteriorated.
Here is an example. People want to hire faster. They don’t like having to wait for a valid test to be developed (which could be avoided through effective planning) because they lose potential candidates.
Software is then purchased which may or may not allow for both automated applicant tracking and some kind of test administration. Even when the system allows for the delivery for valid tests they are not used because that would take time and defeat the purpose of purchasing the software in the first place.
So, what is a person to do? The natural thing, apparently, is to rely on what are called “education and experience surveys” or E&Es (some call them T&Es). These surveys typically ask people to self-report or provide ratings on how much experience they have in a given area, how good they are at doing a particular thing, how relevant their experience is, etc. After applicants complete the E&E surveys they are scored like a test and the person with the most relevant education and most experience gets the highest score.
You might say, “That sounds good to me. Where is the problem?”
All you have to do is step back and think about it critically. You will quickly see a series of serious problems with this approach.
First, candidate are being placed in a competitive situation. They have a built-in motivation to “inflate” their education and experience. In other words, they give themselves the benefit of the doubt. Who wouldn’t? The real problem is that more informed candidates may even lie to receive a higher score.
I have worked on several projects where we developed E&E surveys and included a “Fake Good” scale that was designed to determine the extent to which a person was artificially inflating their scores. We did this by asking questions that we knew they could not have experience with. This included made-up software packages, public laws, and statistical procedures. It was amazing to see how many people had extensive experience in their last job working with a software package that does not exist or who are well-versed in laws that have never been written. Even more interesting is the fact that if you were to subtract or adjust a person’s score based on the extent of their faking, you would see that your top candidates suddenly are in the middle or even at the bottom of your score range.
Another complicating factor is related to the issue of self-criticism. As if artificially inflating your scores is not bad enough, there is a substantial number of highly qualified people who are very critical of themselves. If you ask them to rate how much experience they have in a given area, a person who has worked in the area for 15 years may well say, “I know so and so who has worked for 20 years and they know so much more than me!” In this case, a highly qualified person will rate themselves low while far less qualified people are rating themselves high. How can you interpret this in a meaningful manner?
The natural solution is to verify the accuracy of the responses. You can’t just limit it to the people you are thinking about hiring or you will always miss out on the people who are overly critical about their abilities. You would need to do it for everyone to really get the best employees. Now you will be back to the same problem you had in the first place—a process that takes too long because you have to check the validity of the responses to your E&E survey.
And now we go from bad to worse. Incidentally, this is an area where I think the Canadians make a mistake as well.
The natural step is to “delegate the real decision to program managers because, after all, they know who they need.”
There are so many reasons that “delegation to program managers” is a bad idea that I will not get into it here—with the exception of one. Are you acquainted with different managers who seem to apply different standards to performance when they rate their employees? Some are “easy graders.” Some cannot be pleased by anyone. Some show favoritism. Do you really want to delegate your legal responsibility for legal defensibility to those with little or no training related to personnel selection and a history of inconsistent application of ratings..
The final problem I am concerned with is that if you automate the process and delegate selection to managers, the result will be HR professionals essentially becoming nothing more than “customer support/technical support staff.” The field will be further degraded, the workforce with deteriorate further, those who take pride in their HR craft will seek other jobs, and when the pendulum swings back, there will be few, if any, who maintain the knowledge base to develop, validate, develop, and retain a high qualified workforce.
Saturday, December 27, 2008
Sunday, November 9, 2008
Legally Defensible Competency Modeling For Use In Pre-Employment Testing
Competency modeling is a technique that is increasingly embraced as an alternative to the more traditional techniques of job analysis that have long been the foundation for developing and validating employee selection procedures. This strategy is fraught with potential peril and may expose employers to an increased risk of litigation.
It is the purpose of this article to discuss potential problems facing competency modelers. This is not to say competency modeling cannot be used effectively. Rather, it is a call to caution and serious reflection. Strategies will be presented to help protect employers.
What is a “Competency Model?” A model is a “hypothetical recreation of a complex process” or “a representation of a larger entity but on a smaller scale.” Statistical modeling is the process of creating a simpler version of a complex process by including the major or most important information necessary to make predictions. A competency model, therefore, is an artificially constructed small-scale representation of the important competencies that tend to be possessed by those people who are most successful on the job.
On a purely common-sense level, competency modeling has great appeal. It speaks a language we all seem to understand. The focus on a match between a person’s characteristics and broad aspects of the job generally just sounds “right.”
Competency modeling. as a technique for understanding the person/job interface is not new. Researchers have stated varying opinions on it for many years. It is actually similar to a “person centered” job analysis.
Guion and Highhouse (2oo6) in Essentials of Personnel Assessment and Selection, write “The notion behind the competency modeling movement is that traditional, or ‘old school,’ job analysis cannot meet the demands of the changing workplace.” Such a claim is unfounded. According to Schippmann, 2000), the problem facing competency modeling proponents is that the actual meaning of the term “competency” remains unclear, the notion behind it is that one should identify the characteristics or attributes related to exceptional performance on the job.
Competency modeling’s foundation is the mistaken assumption that traditional job analysis focuses solely on those tasks done on the job and not on the attributes required for success on the job (Sacjett and Laczom 2003).
More seriously, competency modeling lacks “methodological rigor” (Lievans, Shachez, & Corte, 2004) in spite of attempts to make it look more quantitative and objective in nature.
Another serious potential problem with competency modeling as a replacement for traditional job analysis is the fact that this author is not aware of a single case where it has had to face serious legal scrutiny and so, in the event that an employer’s selection decisions are ever challenged, it is difficult to know how solid their footing will be.
Finally, the most significant problem with competency modeling is related to how it may be used to develop selection instruments. The Federal Uniform Guidelines on Employee Selection Procedures recognize three traditional approaches to establishing the validity of pre-employment tests. These are Content Validation, Criterion Validation, and Construct Validation and each has withstood the tempests of the courtroom.
By far, the most commonly used approach to establish pre-employment test validity is content validation. Approximately 80% of all employment tests are based on this strategy (Biddle, 2007). The reason for this is that content validation is a simple and easy to follow process that links test content to critical knowledge, skills, and abilities that are required to perform critical job tasks and duties which are performed on the job. The traditional approach to conducting a job analysis is designed to facilitate this test-to-job linkage.
However, the Guidelines clearly state that a content validation strategy is NOT appropriate for creating tests that purport to measure traits such as intelligence, aptitude, personality, common sense, judgment, and leadership (UGESP, 4[b]). In many cases, the competencies that have been established as the benchmarks against which people are assessed are made based on aggregated duties or abilities that are essentially traits—the very things that are not appropriate for use in content validation processes.
The only safe way to proceed at this point would be to conduct a criterion validation study which generates statistical evidence that the test predicts job performance. It should be noted that very few employers use this approach to test validation, however. The reason for this is that it is a much more complicated and costly approach and has methodological issues of its own.
If a competency modeler is intent on using content validation, it is critical that they do the following: 1) Create competencies that are as objective and observable as possible, 2) Establish links between critical job duties and each competency, 3) Establish links between each critical KSAPC that is required at entry to each critical job duty, 4) Link actual assessment questions to KSAPCs—not to competencies as you will get the greatest level of objectivity. The result is a job analysis with the addition of an umbrella grouping of “competency” under which duties and KSAPCs are grouped.
--Jim
It is the purpose of this article to discuss potential problems facing competency modelers. This is not to say competency modeling cannot be used effectively. Rather, it is a call to caution and serious reflection. Strategies will be presented to help protect employers.
What is a “Competency Model?” A model is a “hypothetical recreation of a complex process” or “a representation of a larger entity but on a smaller scale.” Statistical modeling is the process of creating a simpler version of a complex process by including the major or most important information necessary to make predictions. A competency model, therefore, is an artificially constructed small-scale representation of the important competencies that tend to be possessed by those people who are most successful on the job.
On a purely common-sense level, competency modeling has great appeal. It speaks a language we all seem to understand. The focus on a match between a person’s characteristics and broad aspects of the job generally just sounds “right.”
Competency modeling. as a technique for understanding the person/job interface is not new. Researchers have stated varying opinions on it for many years. It is actually similar to a “person centered” job analysis.
Guion and Highhouse (2oo6) in Essentials of Personnel Assessment and Selection, write “The notion behind the competency modeling movement is that traditional, or ‘old school,’ job analysis cannot meet the demands of the changing workplace.” Such a claim is unfounded. According to Schippmann, 2000), the problem facing competency modeling proponents is that the actual meaning of the term “competency” remains unclear, the notion behind it is that one should identify the characteristics or attributes related to exceptional performance on the job.
Competency modeling’s foundation is the mistaken assumption that traditional job analysis focuses solely on those tasks done on the job and not on the attributes required for success on the job (Sacjett and Laczom 2003).
More seriously, competency modeling lacks “methodological rigor” (Lievans, Shachez, & Corte, 2004) in spite of attempts to make it look more quantitative and objective in nature.
Another serious potential problem with competency modeling as a replacement for traditional job analysis is the fact that this author is not aware of a single case where it has had to face serious legal scrutiny and so, in the event that an employer’s selection decisions are ever challenged, it is difficult to know how solid their footing will be.
Finally, the most significant problem with competency modeling is related to how it may be used to develop selection instruments. The Federal Uniform Guidelines on Employee Selection Procedures recognize three traditional approaches to establishing the validity of pre-employment tests. These are Content Validation, Criterion Validation, and Construct Validation and each has withstood the tempests of the courtroom.
By far, the most commonly used approach to establish pre-employment test validity is content validation. Approximately 80% of all employment tests are based on this strategy (Biddle, 2007). The reason for this is that content validation is a simple and easy to follow process that links test content to critical knowledge, skills, and abilities that are required to perform critical job tasks and duties which are performed on the job. The traditional approach to conducting a job analysis is designed to facilitate this test-to-job linkage.
However, the Guidelines clearly state that a content validation strategy is NOT appropriate for creating tests that purport to measure traits such as intelligence, aptitude, personality, common sense, judgment, and leadership (UGESP, 4[b]). In many cases, the competencies that have been established as the benchmarks against which people are assessed are made based on aggregated duties or abilities that are essentially traits—the very things that are not appropriate for use in content validation processes.
The only safe way to proceed at this point would be to conduct a criterion validation study which generates statistical evidence that the test predicts job performance. It should be noted that very few employers use this approach to test validation, however. The reason for this is that it is a much more complicated and costly approach and has methodological issues of its own.
If a competency modeler is intent on using content validation, it is critical that they do the following: 1) Create competencies that are as objective and observable as possible, 2) Establish links between critical job duties and each competency, 3) Establish links between each critical KSAPC that is required at entry to each critical job duty, 4) Link actual assessment questions to KSAPCs—not to competencies as you will get the greatest level of objectivity. The result is a job analysis with the addition of an umbrella grouping of “competency” under which duties and KSAPCs are grouped.
--Jim
Subscribe to:
Posts (Atom)
Posts
