Chinese Personality – Do we need indigenous psychometric tests?
It’s perhaps quite natural to believe that the Chinese personality is so different to others that it requires a special psychometric test to assess it. What better way to sell your new Chinese personality test than to state that it is “high time a test for the Chinese” were developed. However, this throws doubt upon the utility of rigorously developed international psychometric tests of personality.
Given the above, we embarked on a research program to assess whether Chinese people differ significantly comparied to others in terms of personality structure and whether personality tests that purport to assess Chinese Personality are able to predict any more work performance than internationally developed tests have already been proven to do!
You can read our research findings it: personality.cn, our Chinese Personality at Work Research Site.
No time to read the whole site? Here’s a quick summary:
Locally developed psychometric tests which purport to assess “indigenous” aspects of Chinese Personality were found to be less reliable than reputable internationally developed tests of personality. Furthermore, there is a big question as to whether so-called “indigenous” traits are Chinese-specific. Issues such as traditionalism or face also exist in other cultures! Moreover, the research has demonstrated that whatever we choose to believe about Chinese Personality, locally developed (Hong Kong) tests of “indigenous” personality add nothing to the prediction of performance at work that is not already accounted for by reputable internationally developed personality tests.
We present this research in a free HRM webinar which you can watch here. We held a vote at the beginning and end of our webinar whereby we asked attendees if they believed that Chinese Personality is so different that Chinese people need their own personality test. At the beginning of the webinar, the majority of the attendees said yes! By the end of the webinar only one attendee still believed this to be the case! We recommend choosing well designed psychometric tests with high reliability and validity. Personality is a universal construct, thus locally developed tests may have little benefit to the hiring manager!
HRM Training Course Singapore
PsyAsia International has released dates for their annual public course in Human Resource Management in Singapore. The course runs usually as an in-house course, but once per year the company offers it as a public course. This year all those who had previously registered their interest were surveyed for their preferred dates and locations so PsyAsia expects the course to run at or near capacity. There’s an early bird discount of 10% on offer for all those who register before 30 August entering discount code HRMEB on page 3 of the booking form. Full details and online registration at the HRM course webpage: http://www.psyasia.com/human_resource_management_training_course.php
Online Psychometric Test Mini-Course: Lesson 4
In this session we will explore the following:
1. The relationship between reliability and validity in psychometric assessment
2. How psychometric test administrators can impact the reliability of tests
Psychometric Test Reliability
When choosing a reputable test, whether it be aptitude or personality, one of the properties of the test you will need to look for is reliability. We’ll consider reliability in appropriate detail in a later section of the course. For now, think of reliability as consistency. In order to have absolute confidence in our test scores we need them to be consistent. However, we can’t test and retest our candidates in the real world. Despite this, reputable test publishers would already have done this for you. This would have been carried out under optimal conditions. So, now you know that you are using a reliable test (one that produces consistent scores), it’s your task as the test administrator to ensure that the test remains a reliable test.
Why is reliability so important?
Whenever you assess something, you expect the score you get to be reliable. For example, if you assess your weight using bathroom scales, you expect the reading you get to be consistent across at least the short term. If you weigh yourself over 2 consecutive days and get significantly different readings you know something is wrong with the scales! The same is true of psychometric tests. The publisher first ensures that the test scores will be consistent over time and then you, as the administrator, need to ensure that your actions do not make the test less reliable.
Not only do we want and expect test results to remain reliable over time, but we also know that reliability is a precursor to validity. It sets an upper limit on the test’s validity. In other words, if your test is not reliable then it is not valid. Confusing? Let’s use the weighing scales example again…
Let’s suppose a medical doctor does some research which shows that those who weight more than 120kg are significantly more likely to suffer a heart attack. His research shows that weight is a valid indicator for predicting the heart attack. The scales are fit for the purpose of predicting a heart attack. Validity is all about being fit for purpose. Now if those scales are not reliable, they will provide inconsistent data over the time of the research program. In this case would you have confidence in the doctor’s findings? Of course not!
So, to apply this to psychometric tests let’s take an aptitude test. We’ve carried out research which confirms that a new numerical reasoning test can predict the performance of accountants. Those who score better on the test are rated as better accountants. This is validity. The test is fit for the purpose of predicting accountant performance. You will hopefully have full confidence in this finding if you know the test is reliable. If however you expect the test is coming up with inconsistent scores for your candidates, it is unreliable, and, as in the scales example above, you will not have confidence in the test’s prediction of accountant performance. This is why reliability is a precursor to validity.
And why is all of this so important for this course? It’s because you as the test administrator can enhance or reduce the reliability of the test by how you administer it in the first place. Let’s now take a look at what factors you can and can’t influence in terms of reliability.
How psychometric test administrators can impact the reliability of tests
- Factors Affecting Psychometric Test Reliability (C)2010 PsyAsia International: No Copying
Take a look at the graphic on the left. It shows different factors which can impact the reliability of psychometric tests. This applies to both aptitude tests and personality assessments.
Factors within the test
Generally, a test administrator is not responsible for this. The test publisher must design tests that will be highly reliable. Factors within the test means that the questions chosen must be accessible to all groups for whom the test is intended. If a subsection finds some questions difficult based on their group membership (i.e. non-native-English speaking groups may not understand a colloquialism used in a test question), then the test will be less reliable for that group. Although the publisher needs to ensure a reliable test, not all test publishers are reputable or know what they are doing! This is why the person who purchases the test needs to know how to evaluate it. We’ll show you later how to evaluate the test in greater detail. Know for now that you do not evaluate a test or validate it by trialling it on yourself or your colleague as many untrained users think!
Factors within the respondent
Whilst the test administrator cannot control all the possible factors within a respondent, you can do your best to ensure you control for a much as possible. It’s a good idea to think here about how you would like to be treated if you were undergoing a psychometric assessment for the first time. You’d probably like a friendly invitation letter explaining what is going to happen and why. You’d like to know that your data and results will remain confidential and only shared with decision-makers and only for the purpose that you’re undertaking the test. You’d also like to know what you need to bring with you and if possible, a few example questions as approved by the test publisher might help to set your mind at rest. Finally it would be good to have a number to call should you have any special needs that you wish to convey to the administrators before the day. So, when you arrive at the test centre you already know what is going to happen and why, you won’t be overly concerned, you’ll have all the right things with you (e.g., reading glasses) and you’ll know how long the session is going to last. If it’s a personality test you’ll be more likely to be open and honest because you know your results won’t go further than the selection or development committee and won’t be used for reasons beyond the reason you’ve already been given.
Ultimately here you are attempting to control for mood and expectations. Ideally you don’t want these to vary between candidates in order to give everybody the same start line. On the actual day of the test you will go over all of these things again with the candidates in the room to ensure that they are all clear on what will happen and why. Again, this sets the scene and mood, demonstrates your organisation’s “humanness” in the assessment process and provides candidates with an opportunity to ask questions. Furthermore, on the day you will need to ensure that you administer the test instructions word for word and then administer the test exactly as intended by the test publisher. Doing all of this enhances consistency and thus increases reliability. This is essential as we saw before because reliability is the precursor to validity.
Factors within the environment
How well would you be able to complete an aptitude test in a noisy room? Or how about room that’s freezing from too much air conditioning or too hot due to broken air conditioning? Likewise, you need to ensure that the test environment is conducive to candidate performance each and every time. This applies to personality assessment too. Although there is no right or wrong, your candidate will certainly feel more able to make an effort and respond accurately if you provide them with the right environment! So, some time before the session you’ll need to check the room, make sure temperature controls work. On the day, switch them on in good time before the test so that by the time candidates arrive the room is just right. Place a sign on the door to ensure you are not disturbed during the testing session and be sure to silence all phones in the room. Candidates should of course have phones switched off too. Ensure that once the session is over, all candidates leave at the same time so that they do not disturb others. If a candidate really must make a restroom visit, they should be accompanied by an administrator and only one candidate at a time should go. Ensure that upon leaving and rejoining the room the candidate does not disturb others.
(Note: also a good idea to check there is no planned construction nearby and there are no fire drills scheduled on the day of testing. Do this before sending out your invitation to the candidate!)
Summary
By referring to these guidelines you’ll help to ensure that psychometric tests used by your organisation remain as reliable as the publisher intends them to be. By using short-cuts and not following the guidelines you’ll threaten the reliability and therefore the validity of the tests. If you threaten a test’s validity it becomes unfit for purpose which means your company is wasting its money buying psychometric tools!
Interested in learning more about psychometric testing for HRM? Keep reading – your next free session is not far away! To ensure you don’t miss a single instalment, we suggest you follow-us on twitter as each new post will be announced there. You may also like to join our face-to-face psychometric training courses in Singapore or Hong Kong – these range from simple introductory courses through to Certification Courses such as the BPS Level A and BPS Level B Certificates of Competence in Occupational Testing. Not in Singapore or Hong Kong? No problem – we also offer both recorded and live online training in psychometrics! For full details please see here or email us.
DO NOT COPY OR SAVE THIS ARTICLE TO YOUR COMPUTER.
THIS ARTICLE IS CLEARED FOR PUBLISHING ON PSYCHOLOGY1 GROUP SITES ONLY. IT REMAINS COPYRIGHT AND INTELLECTUAL PROPERTY OF PSYASIA INTERNATIONAL PTE. LTD. YOU ARE NOT AUTHORIZED TO PUBLISH IT ON ANY OTHER SITE. YOU ARE NOT PERMITTED TO COPY/PASTE THIS ARTICLE OR TO SAVE IT TO YOUR LOCAL DRIVE. YOU ARE ONLY PERMITTED TO READ IT ONLINE AT OUR WEBSITE. VIOLATION OF THESE TERMS WILL RESULT IN BANNING OF OFFENDING IPS AND LEGAL ACTION FOR THOSE WHO REPUBLISH THIS ARTICLE WHETHER IT BE WITH OR WITHOUT A REFERENCE TO THE ORIGINAL AUTHOR.
What to Look For in a Motivational Speaker
When you are running a business that involves some sort of marketing, and especially when you head a team, sometimes you will find the need to get a motivational speaker to egg them on. Motivational speaking has become a great business in the past few years. You’ll see a lot of them advertising their services online.
4 Reasons to Restock the Treat Cupboard
Our office treat cupboard was restocked today. So was employee morale.
A Contract Recruiter Provides Hiring Success
A company’s success is based on their ability to stay competitive by hiring the most qualified staff available. This can be a challenge especially when there is a need for multiple hires at once. That is why bringing on a contract recruiter might just be the best solution to help increase your company’s productivity and your bottom line.
Is Your Documentation System Ready For Legal Action?
Not having proper documentation or having erroneous documentation can come back to haunt you if your company is ever involved in an employee lawsuit. If you have good documentation then you are going want to easily find all of it when legal issues occur. Getting involved in a lawsuit of any type can mean spending a ton of time and energy searching through paper as the opposing legal team floods you with a wave of requests for relevant documentation.
Talent Acquisition is a Major Role in the Human Resource Segment
Today, where organizations are looking forward to gain a competitive edge in this impulsive market, their capacity to catch the attention and retain talented professionals has become their main focus. Keeping this in mind all small and large organizations are doing their best to gain over an effective talent acquisition approach.
Employee Contract Template – What to Include in an Employee Contract
What should be in an employee contract? Well firstly you could have a range of basic details such as the following.
NHS Reform and the Practice of Management
Many Doctors that we’ve worked with have expressed a deep loathing of ‘management’. They are not unique in this. Legal and accountancy practices once expressed a similar distrust of the professional manager in our work with them.
