Psychometrics Singapore | Psychometric Tests Singapore

Webinar: Saville Consulting Wave Psychometric Personality Assessment: What makes it more valid than competitors?

Free Psychometric Testing Webinar for Human Resources Professionals in Asia

In this complimentary webinar, PsyAsia International’s registered business psychologist will review aspects of the Saville Consulting Wave. The Wave is the 21st Century’s Revolution in Personality Assessment. It was developed by the best known name in modern Psychometrics, Professor Peter Saville. Independent, comparative research has shown that the validity of the Wave surpasses that usually expected of personality tests and that the tool beats its competitors in predicting performance and leadership at work. In fact, Wave has been so influential that its nearest rival felt the need to revisit their own personality test and to use the term revolution too in their update. However, it turned out to be a revamping rather than a revolution!! So, what makes the Wave such a fantastic tool. Why has Saville Consulting grown so fast over the past few years from nothing to representation in over 60 countries and a move to a larger office to house their growing team?

This webinar will take attendees through the revolutionary characteristics of the Wave in an accessible way. You’ll also get to see the validity study results. After attending the webinar, you’ll understand why the Saville Consulting Wave has been so well received and why it has changed the status quo in personality testing that has existed since the 1980s.

The complimentary webinar is open to all HR and related professionals in Asia who provide truthful data and a corporate email address when registering. It is not open to competitors.

Register here: http://www.psyasia.com/hr_seminars.php

Apollo Profile Online Psychometric Personality Test: New Online Training Course 50% Off

PsyAsia International is pleased to announce that the NEW Online Express Training Course for the Apollo Profile, named Apollo Module 1, is now ready! It’s been designed to be part of a wider series of online learning courses in psychometric tests. Module 1 covers the absolute basics of using the Apollo Online Personality Questionnaire and does so in around one hour. Upon completion, users are presented with a brief quiz and upon passing that will be able to use special pre-interpreted Apollo Profile reports. To sign up for the course, please visit our Online Learning Centre and choose the Apollo Profile Basic Personality Assessment Course.

Apollo Training

For a limited time we are offering this course at 50% off the regular price. This offer is open for the next 4 weeks only! The discount is already reflected in the fee found at the learning centre. Fees will increase in 4 week’s time!

The Apollo Profile is an Online Personality Assessment that compares candidates to those rated as excellent in various occupational groups from sales to executive management. It was developed by Psychometrician and Head of Psychology Department at Bond University, Professor Richard Hicks, and management consultant, Jim Bowden. Apollo is used by many leading organisations, including KPMG, Hudson, Chanel, Ford, Air New Zealand, DBS Bank, Philips Electronics and more. View a selection of different Apollo reports here

Apollo_Profile_Factsheet.pdf Download this file

Psychometric Testing Webinar: Saville Consulting Wave Personality Assessment

Saville-webinar-ad

In this complimentary webinar, PsyAsia International’s registered business psychologist will review aspects of the Saville Consulting Wave. The Wave is the 21st Century’s Revolution in Personality Assessment. It was developed by the best known name in modern Psychometrics, Professor Peter Saville. Independent, comparative research has shown that the validity of the Wave surpasses that usually expected of personality tests and that the tool beats its competitors in predicting performance and leadership at work. In fact, Wave has been so influential that its nearest rival felt the need to revisit their own personality test and to use the term revolution too in their update. However, it turned out to be a revamping rather than a revolution!! So, what makes the Wave such a fantastic tool. Why has Saville Consulting grown so fast over the past few years from nothing to representation in over 60 countries and a move to a larger office to house their growing team?

 This webinar will take attendees through the revolutionary characteristics of the Wave in an accessible way. You’ll also get to see the validity study results. After attending the webinar, you’ll understand why the Saville Consulting Wave has been so well received and why it has changed the status quo in psychometric personality testing that has existed since the 1980s.

The complimentary webinar is open to all HR and related professionals in Asia who provide truthful data and a corporate email address when registering. It is not open to competitors. The webinar runs on 26 April 2011 at 12pm Singapore/Hong Kong time.

To register, please visit http://webinars.psyasia.com

More on the Saville Consulting Wave Psychometric Personality Assessment

Slide4

Psychometric_Webinar_-_Wave_Personality_Assessment.pdf Download this file

Predicting talent with psychometric tests

Predicting talent with psychometric tests PDF Print E-mail
Written for & Published in HR Magazine: Recruitment
Friday, 25 February 2011

By Dr Graham Tyler, Registered Business Psychologist, 
Chartered Scientist and Executive Director at PsyAsia International: www.psyasia.com

Absolute objectivity
Psychometric tests are tools designed to assess psychological attributes such as aptitude and personality in an objective and quantitative manner. There has been over sixty years of research in this field and it is well documented that—psychometric tests, when used properly by trained individuals, add significantly to the ability of HR to select the best candidate for the job.

Organisations which have adopted more ‘scientific’ human resource processes usually incorporate psychometrics into their approach. Psychometrics is probably the only aspect of the selection process that allows for absolutely objective assessment of candidates. Moreover, they allow HR to assess large numbers of candidates in very short periods of time, which would be immaterial if the test did not do its job properly. However, research has shown that the ability of tests to predict successful hires often exceeds that of traditional interviews. Generally, aptitude tests have been shown to be more predictive of success than personality tests, although there are a couple of notable personality tools that are also on par with aptitude tests.

There are numerous tests currently on the market and many distributors in Hong Kong—so how can HR be certain they are choosing the right test and choosing to work with the right distributor? Two videos are available giving advice to those in HR on the right and wrong questions to ask psychometric test distributors and can be viewed via the link below.

VIDEO LINK: The right and wrong questions to ask psychometric test distributors

Professional understanding
Given that psychometric tests measure psychological characteristics, HR are advised to try and work with registered business psychologists, who will have the benefit of years of training in the psychology of people at work. Doctoral psychologists have the same level and years of training and experience as medical doctors, and can only re-register each year if they have undertaken significant professional development. In this respect, professionally qualified psychologists will have a much better understanding of psychometric tests. Some test distributors may have only undergone a few days of training in the tests they sell and will lack a proper understanding of the science and metrics beneath the tests, and are not accountable to anybody for their competence.

Which test?
As for choice of test, this poses a perennial problem for those without training in psychometrics. Many, new to the field, will ask which companies use a particular test—which would then require the distributor to break commercial confidence. Alternatively, HR may come pre-armed with the name of the test they want to use because a friend’s organisation uses that particular test. In fact, such recommendations should have little bearing on test choice, as tests have been designed for different situations and different people. A friend may have chosen their test when there was less market choice, and it may no longer be the best one currently available. So HR is advised to survey all tests currently available and select one that best suits their individual requirements bearing in mind the following key concerns.

How many questions?
One of the most frequently asked questions regarding personality tests is, “How many questions are there in the test.” When we reply with, “200” there is usually a gasp—it should be noted that the reliability, and therefore the validity, of tests generally increases with the number of questions—provided the test is not ridiculously long. Most good tests, which assess multiple relevant aspects of personality for work, do so with around 200 questions. Personality tests cannot do their job well with the 20-50 questions that many in HR may expect. This is one reason why some test distributors do not offer free trials, as a test cannot be evaluated on the basis of the questions alone. Trial users may also be discouraged by long questionnaires and repetition of questions—when in fact this results in higher reliability and a better measure of candidate consistency and attention.

Locally or internationally developed?
There has been a movement that supports Chinese Personality tests that are ‘designed by the Chinese for the Chinese’. The idea may sound sensible and quite seductive, however, research published in the British Psychological Society’s Selection & Development Review, entitled: The Chinese challenge to the Big-5, indicated that locally-developed tests lacked the rigour of internationally-developed ones and had unacceptable reliability levels. Additionally, they had no incremental validity over internationally developed tests. Traditionally, Chinese culture may place greater importance on certain aspects of personality, such as ‘face’, than other cultures, however, our research did not demonstrate that this had any utility in helping to predict candidate performance at work in Hong Kong companies.

Overcoming inertia
If psychometric tests are so useful, why does every organisation not use them? Firstly, cost—and many in procurement may not consider the return on investment. Every aspect of the selection process incurs costs. However, as capital invested in psychometric tests is usually paid to a vendor rather than to in-house staff, this often acts as a further deterrent to purchasing them in the first place. HR can help by explaining the tangible ROI benefits of psychometric tests to those in procurement so they are better aware that using good tests results in superior prediction of talent performance, which will lead to higher retention rates, lower absenteeism, a happier workforce and ultimately a more successful organisation.

Test administrator training essential
As with any new procedure, adopting psychometric tests requires planning, a company policy on test use, fairness, competence in testing, etc. and this all takes time and can be off-putting. HR is often surprised by the need for training test administrators. Proper training is, however, extremely important so that the HR staff administering tests and making decisions based the results know how to use and analyse them properly. Less reputable test suppliers may succumb to the reticence of organisations to train staff by offering tests that require little or no training, however, this is likely to fuel misunderstandings, misinterpretation and ill-informed decisions. For example, if a candidate scores low on an empathy scale—without proper understanding of the test—some may assume they have little empathy. In reality, the scale always means more than its label and the greatest validity can only be obtained from interpreting interactions of scores among multiple scales. Furthermore, interpretation in selection varies significantly depending on whether tests are used with ‘forced-choice questions’ or ‘rating scales’.

As in any assessment method, there is a degree of error in psychometric test scores and this must also be taken into account during interpretation. Quality psychometric training courses will help educate administrators how to do this. The best training courses also teach the right questions to ask distributors and how to evaluate the response—from asking for reliability and validity figures, to understanding the rationale behind the test itself and the test publisher’s background.

Training bloopers
Non-psychologists, who run training programmes, may lack a comprehensive understanding of the field. One trainer for a well-known personality type tool told his class that criterion-related validity was most important for his test—the reality being that construct validity is. The manager at a profiling company in Hong Kong told an enquirer that his questions were getting too complicated and most clients don’t get that technical. The client was only asking the questions they were taught on a six-day British Psychological Society course. One supplier defined test validity as being ‘how accurately the candidate believes the test represents them’ and claimed their tests average 95% validity. In fact, there are many types of technical validity and this is not recognised as one of them. So, HR should be cautious with validity claims that seem too good to be true—check exactly how the supplier defines the term ‘validity’. More credible suppliers would provide evidence that the test significantly predicts meaningful workplace performance variables. This evidence should come from a sizable sample of employees rather than students in universities.

Top 5 questions to ask suppliers

  1. What is the rationale or theory behind the test—is it a validated theory?
  2. May I see evidence of the test’s reliability? Look for internal consistency and test—retest reliabilities of 0.7+ for personality tests and 0.8+ for aptitude tests. Reliability is a precursor to validity.
  3. May I see evidence of the test’s validity? Look for either criterion-related validity or construct-related validity depending on how you will be using the test. You need to see that the test predicts something meaningful or accurately assesses your construct of interest.*
  4. What training is required to use the test?
  5. Do you employ experts in psychometrics, such as business psychologists, rather than test salespeople who are not experts in psychometrics or psychology?

*Notes
Criterion-related validity is evidence that the test actually predicts what it says it does, for example, leadership potential or performance on various competencies.
Construct-related validity is evidence that the test really does assess the construct it was designed to measure, for example, numerical reasoning or conscientiousness.

Conclusion
In summary, psychometric tests have an established record of being able to predict the skills and performance of potential hires and new recruits. However, for them to be effective, it is crucial that HR first equips itself with the right test. In order to choose the right test, HR needs to ask the right questions to suppliers so they can gain a proper understanding of the technical properties of the test, rather than simply relying on colleague recommendations. Psychometric tests are here to stay and both test vendors and HR have a responsibility to ensure competent and ethical test use and thus accurate and fair prediction of talent and performance in Hong Kong’s workplaces.

For more details visit: www.psychometricassessment.com

Saville Consulting Wave Personality Test: Complimentary Webinar from PsyAsia

PsyAsia International invites you to a complimentary webinar on the features of the Saville Consulting Wave Personality Test.

The webinar will run at 6pm Singapore time on 20 December 2010.

At the webinar we will discuss the following:

- Introduction to the Saville Consulting Wave

- What makes Wave a real revolution in personality assessment?

- Comparative validity of Wave with other personality assessments

- Applications of the Wave Framework (selection, team-building, 360 appraisal etc)

- Introduction to the new Wave Leadership and Interview reports

- Overview of other Wave reports

Register Now

New Leadership Assessment Report from Saville Consulting & PsyAsia features on BBC TV

Saville Consulting Wave Personality Assessment Asia

Professor Peter Saville recently appeared on BBC TV to discuss the Saville Consulting Wave Psychometric Personality Assessment. The BBC’s Technology Correspondent, Rory Cellan-Jones underwent the Wave questionnaire and then received feedback from Professor Saville.

The Leadership Report discussed by Professor Saville is powered by Wave Professional Styles and based on Saville Consulting’s brand new Leadership Model. It has been validated both against theoretical leadership constructs and empirically using international workplace performance and effectiveness criterion data.

It combines styles, situations and underlying leadership potential to assess the impact of leadership on people, tasks and the growth of the organization as a whole. As it is based on the Wave model, the Leadership Report can also distinguish between motives and talents and highlights areas of potential over or under-rating.

The Leadership Report has a wide range of applications including in selection, succession planning, coaching or development and in assessing a leader’s fit to their environment.

PsyAsia International is offering a 15% discount on Saville Consulting Wave training for anybody who registers after viewing this video and before 15 December 2010. Use promotion code BBC when booking your place. The next courses will run in March 2011 in Singapore and Hong Kong. Full details at http://www.psyasia.com/saville-consulting-wave-training-module.php.

Leadership Report from Saville Consulting Wave

Psychometric Test Training in Singapore: up to 25% Discount; includes BPS Level A and B & Online Psychometric Courses

Psychometric Assessment Training Courses - BPS Level A and B - Singapore - Hong Kong - Online - Discounts

View all Psychometric Training Courses

View all Course Dates

Course: All PSYCHOMETRIC courses – online and face-to-face
Location: Singapore, Hong Kong, Online
Date: February and March 2011
Details: Save up to 25% when registering 2 or more attendees at the same time. Discount amount reduces by 5% every 2 weeks until 19 December. The earlier you book, the higher the discount! Full details of this offer here.

A Review of Psychometric Tests in use in Singapore

Introduction

Psychologist Vincent Wong carried out an analysis of psychometric tests in use in Singapore and across Asia.  In this analysis, more than 40 tests were reviewed which involved no less than 20 test developers.  There were several focuses in the analysis which included practical information of the tests (information such as price and practical design issues), construct of the test, report design, technical details and training requirements.

Practical Information

Pricing

There exists a wide pricing range among tests developed by different test developers. In the lower end of the continuum one test provider provides tests for free in their entire product range and a section of the chargeable report will be produced. Obviously for user to obtain useful information they have to pay for the full report and this is certainly a marketing strategy. However in the perspective of psychometric this practice serious harm the integrity of the test as anybody can get access to the tests for unlimited number of times. Therefore it can only been seen as tests for people who are interested in trying out tests, rather than being usable in organizational settings. For more protected tests, prices range from USD$10 to more than USD$120 with some of the providers charge per usage while the others charge for subscription fee as well (usually paid annually).

Design

In this analysis, several design dimensions of the test were considered and they were the split between ipsative and normative measures, the type of scales that were employed, and other practical issue like medium of test administration.

The majority of the personality assessment tools (over 80%) employ normative measures (the type of psychometric tools that compare the respondent with a group of similar others, or the norm group) while the remaining ones employ an ipsative style (the type of psychometric tools that determine the preference among different personality traits within the respondent). Two exceptional case was identified which employs a mixed style, i.e. normative plus ipsative. The reason behind the popularity of normative style might down to the fact that for tests that were designed for selection purpose normative style was the better style to go with as it actually compare the respondents with the others. On the other hand ipsative measures can provide us with better knowledge about the preference or strength within the respondents. In line with this we found that most of the ipsative tests were preference or value tests which were designed for coaching or counselling purposes, although some ipsative measures that were designed for selection purposes were also identified. For the only tests that incorporated both normative and ipsative styles, the underlying connotation of the difference between normative and ipsative scales were utilized and it represented the discrepancy between the real and ideal self of the respondents.

The type of scale used by the tests is actually a function of whether they are ipsative or normative tests. For normative test the most popular scale type used was 5-point Likert Scale (Likert Scale is the type of scale that respondents choose among several options for the one that represent their thought most). 7-point scale was also quite common and there were a few occurrences of 3-point and 9-point scales. Other than using Likert scales, a few normative tests employed true or false scale. For ipsative tests force-choice scale was employed. One of the more popular version of ipsative scales asked the respondents to pick the option that describes them the best (usually termed as ‘most like me’) as well as option that describes them the worst (usually termed as ‘least like me’). Another appearing version of ipsative scale asked the respondents to put the available options into order, although this version was very uncommon.

Most of the surveyed tests, if not all, were designed for completing on computerized environment. While some of the tests can be administered online in an unsupervised manner, there were quite a few that required supervised administration. Whereas there were few test that provided different versions for supervised and unsupervised administration. Having more than one version allowed the result to be checked in a supervised manner after the candidates had passed the unsupervised session. Paper and pencil version of the tests were usually available with similar price of the computerized version although there were a few tests that did not provide paper and pencil version.

Although all the surveyed tests were not designed to be completed in a designated time, timer was identified in one test and it served the function of checking against random or thoughtful responses.

Measured Attributes

Personality

Among the different attributes, personality was the most popular one being measured. The majority of the personality measurements were built on the Big Five model of personality identified by Costa and McCrae (1985). While some of them retained the original five factors within the tests, about half of the surveyed tests restructured the factor compositions based on the result of the factor analysis or other theoretical support, for example one test split the factor of conscientiousness into ‘Industriousness’ and ‘Methodicalness’ while another developer incorporate the five factor model with behavioural tendencies and came up with a seven factor model. Another common phenomenon observed was that under each of the five factors the primary factors (ranges from 3-5 facets, also known as facets) were also measured, and they were actually more commonly used by test developers in report generation and interpretation. This was probably because the primary factors offer more detailed information thus higher flexibility in using them. Besides the Big Five model, another very popular personality model employed by test developers was Jung’s (1920) typology of personality. For instance two of the tests were developed from this theory as their entire theoretical foundation but one employed the original categorical model while the other one developed a continuum model.  Besides building upon one theory, many tests extract personality factors from multiple personality theories and some of them measured as many as 34 personality dimensions. Example of the measured personality dimension includes ambition, initiative, concern for others, flexibility, and energy. Nearly most of the surveyed personality tests served multiple functions which included selection, training/development need analysis, counselling and other related applications such as personal development, conflict management and team building. Test developers further added the applicability of personality tests in different situations by providing multiple versions of reports alongside with a general personality profile.

Value, Motive and Preference

Another popular attributes being measured were value, motive and preference. Although these are three distinct attributes, we found it was common that test publisher combine either two or all three attributes into one test. These tests were less commonly employed in the situation of selection but more widely used in counselling and developmental scenarios, although some of them were also designed to be used in selection as well. For tests that measures value and motive, normative measures were found to be more common and ipsative measures were more common among preference tests.  Another related attribute being measured was interest and they were mainly designed to be a career development tool.

Others

Other measured attributes included measure of leadership styles, team role, behavioural tendency, Emotional Intelligence, self-efficacy, work ethic, interpersonal communication, sales orientation, customer service orientation, learning style and even work effectiveness tendency.

Report Style

Nearly all of the surveyed tests have multiple reports and they are all in narrative form alongside with a graphic representation (usually bar charts) of the measured characteristic. However there was one test that did not employ narrative style in their report at all. Graphical representations with a sentence long description for each factor were employed instead of the narrative format. 2 dimensional typology graphs and score matrix were also employed for some type of reports. Some reports made use of different colours in representing different dimensions being measured yet some others used colour to indicate extreme scores (for example green representing high scores while red representing low scores). Colour was also frequently employed for matching test scores with a standard or an established profile, with green meaning a good match and red representing a poor match.

Report Content

Generic  Personality Profile

For all the surveyed tests, there was at least some form of generic personality profile provided in the report, whether in the form of narrative writing, matrix of scores, 2 dimensional typology graphs, bar charts or broken line graphs. Most commonly the personality profile was consisted of a graphical representation of the test scores on different dimensions with a brief descriptive narrative alongside it. In this generic personality profile the test scores, usually in form of sten scores or percentile were presented. Raw scores were also found in some reports. About half of the survey tests also presented the variation of the test score in the report and a few had an explanation on the meaning behind that. In all cases primary dimensions measured by the tests were reported in this section. Secondary or higher-level composite dimensions were also frequently reported in this section.

Strengths and Limitations

Strengths and limitations were another very popular qualities being reported, although we identified a few tests that do not report them. In reporting strengths and limitations some tests referred them to very specific behavioural terms while there existed some tests simply referred high or low scores in particular dimensions as strengths or limitations. Few tests incorporated contextual factors into the reporting of strengths and limitations were identified and they were more common in purpose-specific reports (for example reports designed for leadership development or team building). Overall tests tended to present information about strengths and limitations of the candidates.

Competencies

Leadership, team work, interpersonal skills or orientation and problem solving orientation were found to be the most popular competencies being tapped. Other competencies being tackled by the surveyed tests included achievement orientation, customer service orientation, management style, decision making, planning and organization, influence and negotiation, delivery, creativity, analytic orientation, coping style and thinking style. Rather than being measured directly in the tests, these competencies were often generated from several primary dimensions of personality. They were found to be written in context of work and behavioural terms were employed heavily in order to aid comprehensibility of the report. Furthermore competency based reports were identified and leadership related reports were the one which appeared most. Competency based reports for sales and managerial positions were also popular.

Interview prompts

Interview prompts were found in some reports. These included general instruction of how to use the report correctly to enhance the effectiveness of a follow-up interview as well as specific suggested interview questions to be asked for a particular candidate. The number of interview prompts varies from three to ten plus suggested questions and some reports even included the expected answer from the candidate. These interview prompts also served as a check or back up of the validity of the tests.

Training (Development) Needs

Several tests with a separated training need or developmental report were identified. For tests that did not have a designated report for training needs, it was surprising to found that the section outlining training was absent for majority of the surveyed tests, given most of them were designed to be used in training need analysis. When present, the training needs outlined (or some tests referred it to be ‘action plans’) were usually generated from the unfit aspects identified or areas that were not up to the normative standard. Simple description about the needs per se was common and a few reports were found to be providing concrete training suggestions.

Cultural Fit

Cultural fit information was identified in a few test reports. This information could include the fit of the candidate with the organizational culture, task nature as well as co-workers and it existed in several forms. The more popular way to compute it was comparing between the candidate’s score with the norm or an ideal profile. One test generated this information by comparing the candidate with the best performers. Yet another test presented the information in light of the candidate himself by stating what culture or environment will be the best fit for the candidate.

Technical Information

Technical information of the test included normative data, reliability and validity data as well as development procedure of the test. They are the most important information to be readily accessible to the public but unfortunately some of them were virtually absent for some of the surveyed tests. Normative data were found to be the most reported information and reliability data followed. However evidence for validity as well as development procedure of the test were absent for some of the tests despite the claim of ‘scientifically validated’ in their marketing materials. For tests that did not provide any of the above mentioned information the integrity of them were seriously in doubt.

Training Requirement

Training requirement of the tests varied from no need training for an extreme case (which was the free online test) to BPS Level B plus additional training (approximately 7 days of training in total). For most of the tests 2-3 days of training for the specific test was common but this type of training would not be recognized by a different test provider. The BPS (British Psychological Society) Competence in Occupational Testing was found to be the most widely accepted qualification by the test providers. Most of the tests could be administered by a BPS Level B qualified user but there existed some tests which required a conversion training (1-2 days long) in order to be a qualified user of them.

Chinese Personality – Do we need indigenous psychometric tests?

It’s perhaps quite natural to believe that the Chinese personality is so different to others that it requires a special psychometric test to assess it. What better way to sell your new Chinese personality test than to state that it is “high time a test for the Chinese” were developed. However, this throws doubt upon the utility of rigorously developed international psychometric tests of personality.

Given the above, we embarked on a research program to assess whether Chinese people differ significantly comparied to others in terms of personality structure and whether personality tests that purport to assess Chinese Personality are able to predict any more work performance than internationally developed tests have already been proven to do!

You can read our research findings it: personality.cn, our Chinese Personality at Work Research Site.

No time to read the whole site? Here’s a quick summary:

Locally developed psychometric tests which purport to assess “indigenous” aspects of Chinese Personality were found to be less reliable than reputable internationally developed tests of personality. Furthermore, there is a big question as to whether so-called “indigenous” traits are Chinese-specific. Issues such as traditionalism or face also exist in other cultures! Moreover, the research has demonstrated that whatever we choose to believe about Chinese Personality, locally developed (Hong Kong) tests of “indigenous” personality add nothing to the prediction of performance at work that is not already accounted for by reputable internationally developed personality tests.

We present this research in a free HRM webinar which you can watch here. We held a vote at the beginning and end of our webinar whereby we asked attendees if they believed that Chinese Personality is so different that Chinese people need their own personality test. At the beginning of the webinar, the majority of the attendees said yes! By the end of the webinar only one attendee still believed this to be the case!  We recommend choosing well designed psychometric tests with high reliability and validity. Personality is a universal construct, thus locally developed tests may have little benefit to the hiring manager!

Online Psychometric Test Mini-Course: Lesson 4

In this session we will explore the following:

1. The relationship between reliability and validity in psychometric assessment
2. How psychometric test administrators can impact the reliability of tests

Psychometric Test Reliability

When choosing a reputable test, whether it be aptitude or personality, one of the properties of the test you will need to look for is reliability. We’ll consider reliability in appropriate detail in a later section of the course.  For now, think of reliability as consistency.  In order to have absolute confidence in our test scores we need them to be consistent.  However, we can’t test and retest our candidates in the real world. Despite this, reputable test publishers would already have done this for you. This would have been carried out under optimal conditions.  So, now you know that you are using a reliable test (one that produces consistent scores), it’s your task as the test administrator to ensure that the test remains a reliable test.

Why is reliability so important?

Whenever you assess something, you expect the score you get to be reliable. For example, if you assess your weight using bathroom scales, you expect the reading you get to be consistent across at least the short term. If you weigh yourself over 2 consecutive days and get significantly different readings you know something is wrong with the scales!  The same is true of psychometric tests. The publisher first ensures that the test scores will be consistent over time and then you, as the administrator, need to ensure that your actions do not make the test less reliable.

Not only do we want and expect test results to remain reliable over time, but we also know that reliability is a precursor to validity. It sets an upper limit on the test’s validity. In other words, if your test is not reliable then it is not valid. Confusing?  Let’s use the weighing scales example again…

Let’s suppose a medical doctor does some research which shows that those who weight more than 120kg are significantly more likely to suffer a heart attack.  His research shows that weight is a valid indicator for predicting the heart attack.  The scales are fit for the purpose of predicting a heart attack.  Validity is all about being fit for purpose.  Now if those scales are not reliable, they will provide inconsistent data over the time of the research program.  In this case would you have confidence in the doctor’s findings? Of course not!

So, to apply this to psychometric tests let’s take an aptitude test. We’ve carried out research which confirms that a new numerical reasoning test can predict the performance of accountants. Those who score better on the test are rated as better accountants.  This is validity. The test is fit for the purpose of predicting accountant performance.  You will hopefully have full confidence in this finding if you know the test is reliable.  If however you expect the test is coming up with inconsistent scores for your candidates, it is unreliable, and, as in the scales example above, you will not have confidence in the test’s prediction of accountant performance. This is why reliability is a precursor to validity.

And why is all of this so important for this course?  It’s because you as the test administrator can enhance or reduce the reliability of the test by how you administer it in the first place.  Let’s now take a look at what factors you can and can’t influence in terms of reliability.

How psychometric test administrators can impact the reliability of tests

factors affecting psychometric test reliability
Factors Affecting Psychometric Test Reliability (C)2010 PsyAsia International: No Copying

Take a look at the graphic on the left. It shows different factors which can impact the reliability of psychometric tests. This applies to both aptitude tests and personality assessments.

Factors within the test

Generally, a test administrator is not responsible for this. The test publisher must design tests that will be highly reliable. Factors within the test means that the questions chosen must be accessible to all groups for whom the test is intended. If a subsection finds some questions difficult based on their group membership (i.e. non-native-English speaking groups may not understand a colloquialism used in a test question), then the test will be less reliable for that group. Although the publisher needs to ensure a reliable test, not all test publishers are reputable or know what they are doing! This is why the person who purchases the test needs to know how to evaluate it. We’ll show you later how to evaluate the test in greater detail.  Know for now that you do not evaluate a test or validate it by trialling it on yourself or your colleague as many untrained users think!

Factors within the respondent

Whilst the test administrator cannot control all the possible factors within a respondent, you can do your best to ensure you control for a much as possible.  It’s a good idea to think here about how you would like to be treated if you were undergoing a psychometric assessment for the first time. You’d probably like a friendly invitation letter explaining what is going to happen and why. You’d like to know that your data and results will remain confidential and only shared with decision-makers and only for the purpose that you’re undertaking the test. You’d also like to know what you need to bring with you and if possible, a few example questions as approved by the test publisher might help to set your mind at rest.  Finally it would be good to have a number to call should you have any special needs that you wish to convey to the administrators before the day.  So, when you arrive at the test centre you already know what is going to happen and why, you won’t be overly concerned, you’ll have all the right things with you (e.g., reading glasses) and you’ll know how long the session is going to last. If it’s a personality test you’ll be more likely to be open and honest because you know your results won’t go further than the selection or development committee and won’t be used for reasons beyond the reason you’ve already been given.

Ultimately here you are attempting to control for mood and expectations. Ideally you don’t want these to vary between candidates in order to give everybody the same start line.  On the actual day of the test you will go over all of these things again with the candidates in the room to ensure that they are all clear on what will happen and why.  Again, this sets the scene and mood, demonstrates your organisation’s “humanness” in the assessment process and provides candidates with an opportunity to ask questions.  Furthermore, on the day you will need to ensure that you administer the test instructions word for word and then administer the test exactly as intended by the test publisher. Doing all of this enhances consistency and thus increases reliability.  This is essential as we saw before because reliability is the precursor to validity.

Factors within the environment

How well would you be able to complete an aptitude test in a noisy room?  Or how about  room that’s freezing from too much air conditioning or too hot due to broken air conditioning?  Likewise, you need to ensure that the test environment is conducive to candidate performance each and every time.  This applies to personality assessment too. Although there is no right or wrong, your candidate will certainly feel more able to make an effort and respond accurately if you provide them with the right environment!  So, some time before the session you’ll need to check the room, make sure temperature controls work. On the day, switch them on in good time before the test so that by the time candidates arrive the room is just right.  Place a sign on the door to ensure you are not disturbed during the testing session and be sure to silence all phones in the room.  Candidates should of course have phones switched off too.  Ensure that once the session is over, all candidates leave at the same time so that they do not disturb others.  If a candidate really must make a restroom visit, they should be accompanied by an administrator and only one candidate at a time should go. Ensure that upon leaving and rejoining the room the candidate does not disturb others.
(Note: also a good idea to check there is no planned construction nearby and there are no fire drills scheduled on the day of testing. Do this before sending out your invitation to the candidate!)

Summary

By referring to these guidelines you’ll help to ensure that psychometric tests used by your organisation remain as reliable as the publisher intends them to be. By using short-cuts and not following the guidelines you’ll threaten the reliability and therefore the validity of the tests.  If you threaten a test’s validity it becomes unfit for purpose which means your company is wasting its money buying psychometric tools!

Interested in learning more about psychometric testing for HRM? Keep reading – your next free session is not far away! To ensure you don’t miss a single instalment, we suggest you follow-us on twitter as each new post will be announced there. You may also like to join our face-to-face psychometric training courses in Singapore or Hong Kong – these range from simple introductory courses through to Certification Courses such as the BPS Level A and BPS Level B Certificates of Competence in Occupational Testing. Not in Singapore or Hong Kong? No problem – we also offer both recorded and live online training in psychometrics! For full details please see here or email us.

DO NOT COPY OR SAVE THIS ARTICLE TO YOUR COMPUTER.
THIS ARTICLE IS CLEARED FOR PUBLISHING ON PSYCHOLOGY1 GROUP SITES ONLY. IT REMAINS COPYRIGHT AND INTELLECTUAL PROPERTY OF PSYASIA INTERNATIONAL PTE. LTD. YOU ARE NOT AUTHORIZED TO PUBLISH IT ON ANY OTHER SITE. YOU ARE NOT PERMITTED TO COPY/PASTE THIS ARTICLE OR TO SAVE IT TO YOUR LOCAL DRIVE. YOU ARE ONLY PERMITTED TO READ IT ONLINE AT OUR WEBSITE. VIOLATION OF THESE TERMS WILL RESULT IN BANNING OF OFFENDING IPS AND LEGAL ACTION FOR THOSE WHO REPUBLISH THIS ARTICLE WHETHER IT BE WITH OR WITHOUT A REFERENCE TO THE ORIGINAL AUTHOR.

Next Page »

Psychometrics Singapore | Psychometric Tests Singapore