Assessing Job Negotiation Competencies of College Students Using Evidence-Centered Design and Branching Simulations

The study explored the development of a valid assessment tool for job negotiation competencies using the Evidence Centered Design framework. It involved the creation of a competency model, evidence models, and task models that guided the development of a branching simulation tool to quickly diagnose college students' knowledge and skills in job negotiation. The online tool utilized three scenarios where students play the role of job seekers negotiating with their potential future employers. This study focused on two key behaviors in negotiation – making counteroffers and making reasonable concessions. A preliminary competency model was first developed based on a literature review of negotiation research. This model was then validated by a panel of experts. The experts also validated the evidence model (how to score performance on the simulation) and the task model (what tasks should be performed to elicit evidence of performance). These activities and the experts' feedback for improving the prototype simulation provided content validity for the tool. A total of 86 undergraduate and 51 graduate students participated in the study. The students completed an online tutorial, the scenarios in the simulation, a demographics survey, and two other survey instruments that provided alternative measures of negotiation abilities. Their performance on the assessment simulation was determined by their overall competency score and value of the negotiated outcome. Students were classified as experts or novices based on their negotiation experience and knowledge of negotiation strategies. Results from the study indicated that experts performed better than novices in terms of overall competency and negotiated outcome. The study also compared the outcomes of the assessment tool with outcomes from the alternative measures of negotiation ability (a survey on preference for competing, collaborating, compromising, and accommodating negotiation strategies and a survey to determine self-confidence in using distributive and integrative negotiation tactics). I hypothesized that students with a high preference for competing and collaborating strategies would also have higher scores from the assessment tool. On the other hand, students who indicated a high preference for accommodating and compromising strategies would have lower scores. The results from the Preferred Negotiation Strategies survey supported my hypothesis that students who highly prefer accommodating and compromising strategies would have lower scores on overall competency and negotiated outcome. But the mixed findings for competing and collaborating preferences only partially supported my hypotheses. I also hypothesized that students who were highly confident in the use of distributive and integrative negotiation tactics would have higher scores on the assessment compared to those who have low self-confidence. The results did not support my hypotheses because there were no significant relationships between confidence and the assessment outcomes. Finally, the study also found that gender, expertise, and negotiation training have an effect on overall competency score and the negotiated outcome. This dissertation provided a case study on how to develop an assessment tool that diagnoses negotiation competencies using the ECD framework. It also provided evidence of validity for the tool by demonstrating its ability to distinguish different levels of performance by expert and novice negotiators.