Competency-Based Training and Basic Principles
These report aim to give a global and practical view of the evaluation of working professionals in training focused on their performance in real situation performance or performances as illustrated in (4th level of the Miller’s pyramid). In the first part the evaluation of the performance is based, the different methods are exposed schematically and the advantages, disadvantages and limitations of its use are analyzed.
The second part deals with the types of forms used in the different methods, the keys to their design, the most common errors in their use, how to avoid them and how to minimize the subjectivity of the evaluation. The planning and implementation strategy is addressed in our context of specialized training, applying the basic principles of evaluation (Chauhan & Wellbeloved-Stone, 2015).
Training and evaluation based on competences. Basic principles
Competency-based training is more than a program written in terms of competencies. It is a strategy aimed at training professionals according to the needs of socio-working systems and needs to direct the training actions with a holistic, humanizing and global vision in the social, together with an institutional commitment and learning through situations or tasks.
Competency-based training requires:
1) On the part of the evaluated reflection, self-evaluation and an explicit commitment to its own training process is key.
2) Of the learning process, document learning outcomes, make the program more flexible (individualized), frequent and high-quality feedback, and leadership from the headquarters of the teaching units (welfare services), since it fosters commitment and participation active of all team professionals in the training process.
Evaluating is a process that generates information through the assessment of the qualities of the professional in training, through standardized criteria or parameters, and this information is used to form judgments and make decisions. It does not consist of applying a method determined by its psychometric value or an exercise of numerical calculations and weights. The aim is to document explicit evidence on performance to identify those areas of competence developed and those that need to be strengthened. It requires information from different sources, professional judgment and decision making by consensus of experts.
Therefore, it is not an action at a specific moment, it is a process with a well-planned strategy from before the training began(Jones, & Payne, 2016). The first day of the training begins with the departure diagnosis of the student / resident, which allows them to plan their training process and its development according to the learning curve . The concern of the managers and heads of the official institutions is, logically, in the summative or certifying evaluation. Trainers should put the emphasis on formative evaluation, comparing resident A with A, and not A with B. It is about moving from evaluating “learning” to evaluating “for” learning, with feedback being key. The evaluation becomes the axis of the training strategy, the learning engine, and the previous step is the definition of the profile of the professional to be trained, basing the entire evaluation process on that profile (Jones, & Payne, 2016).
- Professionalism: professional values, attitudes, behavior and ethics
- Communication
- Employees care and skills
- Working knowledge
- Practice based on the context of the Working System (public working, working systems)
- Practice based on learning and improvement (critical analysis and research)
- Information management
Formative Evaluation: Key to Competency Development
The formative evaluation is characterized by: early start and during the program, identify strengths and weaknesses, objectify progress and introduce corrective measures, develop competencies and guide and direct professional development. It has less rigor in psychometric terms. «No need to be afraid to rely more often on qualitative data and of professional judgment, since the overall quality of evaluations depends much more on a comprehensive evaluation program than on individual instruments.
The most important thing is to work well the training process with its different learning methodologies. The summative evaluation is the end of a formative evaluation process, it would be carried out by consensus of experts based on the periodic information collected and should express, in a brief report, the competences achieved. It is not about numerical calculations or weighing some competences over others, they all have the same value for the exercise of the profession (van Vendeloo, Brand, Kollen, & Verheyen, 2018).. Are knowledge and skills above ethics In practice, the important thing about methods is the information they provide, their educational impact and the modification of performance. Any method, even those that are less structured and standardized, can be useful depending on its use. There are no good or bad methods, this is relative, since the evaluation involves designing a plan, establishing the educational project and analyzing the resources for its implementation ».
Performance evaluation strategies Evaluate in real situation (performance) et al. They already showed the difference between competence and performance when comparing the results of the performance in real and simulated clinical practice of a group of general practitioners. The lowest score obtained in the real situation (performance), compared to the simulated (competence), was not a reflection of a loss of competition, but the real situation is subject to multiple external factors that influence the behavior of the professional (Verderame, & McCormack, 2018).
Therefore, competence and performance are different constructs. We propose to assimilate competence with “having competence”, possession of internal resources, and performance with “being competent”, showing those resources that are in real situation (4th level of the Miller’s pyramid), to solve problems or tasks integrating the knowledge (knowledge), know-how (skills and abilities), knowing how to be (attitudes and behaviors) and knowing how to be (ethics and values)
Difference between having competence and being Competent (evaluable in the action). Being competent is a way of knowing how to act in a relevant way in situations and contexts in which people face problems with a clear quality criterion, for which internal resources are articulated and mobilized ( knowledge, abilities, skills, values, experiences of context, people, data networks, etc.), being able to give reasons for their decisions and actions and take responsibility for the moral effects s and their social impact (Jones & Payne, 2016).
Performance Evaluation Strategies: Real Situation Performance
On the contrary, having competition is having the necessary resources to act with competence, but it can happen that a person has many competences and is not able to combine and mobilize them properly within a work situation. There is never a real situation like another, Therefore, the personnel in training must be evaluated in real and not simulated situations, except for elementary ethical principles. “The real competition will be that which can be evidenced by the result of the action (competition = action), given that internal resources are invisible.
Training and evaluation through the task (action) in which different competences are expressed through observable behavior. Performance evaluation strategies The performance ranges from the different working practices to management aspects, preventive, research and teaching activities, team cooperation, attention to the family and the community, etc., and for its evaluation it is necessary to combine different methods. The evaluation can be approached from two points of view:
1) temporal: during and after the action or work;
2) Through the person evaluated: self-assessment, assessment by others (observation and supervision) and by labor activity data.
Methods of performance evaluation During work Based on direct observation (immediate feedback) After work Based on Work results For a single evaluator in a specific working situation (Jonker & Ten Cate, 2015): Evaluation of performance or professional practice: fundamentals and methods.
Principles of working education: from the degree to professional development.
Evaluation during work (workplace-based assessment) Based on direct observation in the workplace and in the immediate feedback afterwards. Evaluations by a single observer Fundamentals, development and constraints. All the methods that we briefly describe have the same foundation : observation or discussion about a concrete clinical action and immediate feedback.
In order to alleviate a training deficit, since residents were rarely observed, supervised and oriented (feedback) during its clinical practice. The evaluator bases its judgment on the performance of the student / resident in a given working situation. To do this, select the employee and the working scenario according to the objectives of the program, request permission to the employee, observe and perform the feedback, documenting the weaknesses and strengths and the agreed improvement actions.
The instruments used are scale forms with precise descriptors of the skills to be evaluated to minimize subjectivity; the scales, from 3 to 6 levels, are verbal combined or not with numbers (Kupfer, 2016). The competencies that they describe vary according to the different organizations, specialties and contexts and include data on the working situation, the type of working problem and its complexity, the experience of the evaluators, an assessment of the satisfaction of the evaluated person, observations of the evaluator and, in some, a final scale with the level of performance or autonomy achieved. The application of these methods requires a good organization and teaching planning, availability of time, periodic meetings of the evaluators to reach consensus and correct errors, in addition to a good training of evaluators in the technique of feedback, since this is key.
Differentiating Competence and Performance
The evaluator chooses one of the 2 cases, according to the interest, and analyzes with the resident the most important aspects. The evaluator’s interest will focus on analyzing and understanding the resident’s reasoning process. It is a tool with high training value (reflection) and good psychometric characteristics, but demanding for those involved.
Description and utility: observation of the performance of a student / resident in a usual working encounter followed by feedback. The evaluator observes how the resident acts before a employee. It is a very well studied tool, with high training value and good psychometric characteristics. The studies carried out in- indicate that 8 evaluators with 2 evaluations carried out by each of them over a year (16 evaluations of different working situations) provide good validity and reliability Competencies evaluated: skills in the working interview and physical examination, communication, human and professional qualities, working reasoning, care and employee education, organization and efficiency and global working competence (Kupfer, J. M. 2016).
Competences evaluated: quality of the initial working assessment, working history, diagnostic tests requested, employee management according to its severity, time management, relationship with the work team, leadership, Income management or employee referral, employee safety, appropriate use of resources and a global working judgment. Example: Observation of technical skills (DOPS). Description and usefulness: the method is similar to the previous ones and allows to evaluate from simple procedures such as a lumbar puncture, a bladder catheterization, an arterial or venous catheterization, to more complex, such as a renal biopsy or an angiography (Rekman & Hamstra, 2016). It is an important evaluation, not only in the training, but to entrust techniques with full responsibility.
Competences that evaluates: the indication of the procedure, the identification of the anatomy and the technical procedure in itself, obtaining the informed consent, adequate preparation of the procedure, analgesia and sedation, aseptic techniques, skill in performing the technique, communication skills, employee safety, handling of complications, request for help when needed, interpretation of results or certain parameters provided by the procedure, professionalism, other important aspects of the technical procedure to be evaluated, a global assessment of the performance of the technique and the rational use of resources.
Good working practices, employee care, employee safety, management and organization, teaching, professionalism, communication, interpersonal relationships, application of knowledge, etc. It is important to adapt the skills to assess one’s own culture, especially in the case of behaviors.a) Evaluation by working and non-working professionals of the work environment. There are as many methods as teams of evaluators decide what to evaluate. (Kupfer, 2016).
Methods of Performance Evaluation
Among these factors is the different complexity of the employees, whether it is reference centers or not, or the different types of organization and management of the centers.
- The evaluation of different working situations is a clear advantage, specifying forms designed for effect.
2.Forced and opportunistic situation and errors in the development of the encounters.
- Disadvantages:
the resident and the evaluator must interrupt their normal working activity and ask the employee for permission, this forces the situation and causes that none of the “actors” »Have a normal behavior. The resident / student knows that he is being evaluated. B) Errors: – Sometimes, the employee addresses the evaluator and not the resident, so it is necessary to clearly explain to the employee what this encounter consists of. – The evaluator must distance himself enough, if the physical space allows it, not to interfere with the evaluated, avoiding being in sight or interrupting with gestures the interview. -If the time assigned to the meeting is not respected, the characteristic of being a real encounter is lost, transforming itself in a fictitious situation.
- Difficulty of the case. The different difficulty of the cases can be a source of error if you are evaluating and comparing residents of the same level of experience. It is necessary to select well the cases according to their difficulty(Kupfer, 2016).
- previous actions. The advantage of unique observations is that the evaluator focuses on judging a single action, avoiding this bias. This can limit viability.
- 360? evaluation. Its advantage lies in the fact that the trial is issued on the performance in different working situations, which reduces the specificity of the case, and in which filling out the form does not take more than 5 min. The disadvantage is that the evaluator can make judgments about aspects that he has not observed clearly or does not remember well Stevens, ( Hyde, Knight, Shires & Alexander, 2017). The judgments issued by different evaluators, of different professional profiles, as well as their training, will minimize this problem. But the fundamental problem is its viability, since the application clashes with the cultural resistance to which others evaluate our behaviors. Additional work is required with the equipment.
An adequate training reduces interobserver variability, gaining confidence in the use of these methods. It is necessary to take into account: A) The working experience of the evaluators. It is the biggest source of errors. Workingly experienced evaluators have different opinions about the performance in the same case, being able to assess the minor or greater difficulty of the cases based on that experience. It is therefore important to include several trained evaluators. B) Case specificity. The isolated observations do not predict the following nor do they represent the set of the resident’s performance. Several observations are needed with different employees, which means time and trained practitioners. In addition, these observations can detect other deficit competencies that require an evaluation with other methods. C) Aspects to be evaluated. To gain in reliability it is important to break down the task to be evaluated in different dimensions (competences); generally between 5 and 10 is enough, together with a good description of them. One mistake is to transform a working encounter into an interrogation about concepts of pathology(Kupfer, 2016). The methods by a single observer should be limited to evaluating working skills and not knowledge. For this there are other instruments. D) Equivalence between the evaluated ones. Sometimes it is not clear if the difference of results among the residents may be due to their different level of competence, to the difficulty of the cases, or to the different criteria of the evaluators, since not all the evaluators have the same rigor or training employee, nor all the cases the same complexity.
Workplace-Based Assessment: Fundamentals, Development and Constraints
However, it is not common to include proposals that allow evaluating the achievement of the results obtained in the acquisition of these competences. When analyzing this documentation, the existence of a significant gap is found as to how much should be observable of the competition or where this demonstrated competence should be reflected. In this way, elements that contribute to a complete description of competence such as the way to verify the degree of development of the competence acquired by the student and the basic source where to verify the evidence of its achievement, are usually undefined(Kupfer, 2016).
Therefore, we must bear in mind that two factors converge in the success of the current educational project aimed at improving higher education:
On the one hand, the knowledge and mastery that employees have about the specific and general competences of the degree or degrees where their teaching is developed.
On the other hand, the way in which the employee knows how to value in the student the results he has obtained from the learning process.
Conceptual framework: definition and elements of competition
Defines competence as the set of “personal attributes and (the) demonstrated aptitude to apply knowledge and skills”. It is expressly stated that it is synonymous with the “ability to solve problems in a specific context”. The analysis of this regulation allows us to indicate the following dimensions that integrate the concept of competence:
The personal attributes or any mood that belongs to the scope of the subject, that is, those features that a person has of birth or acquired by training and that define what the person is (in front of what he does) such as: talent, motivation, communication, cognitive abilities, values, emotional intelligence or others such as knowledge (knowledge) and skills (know-how) , that even though they are not typically traits, they are included by the majority of the authors under this category. The demonstrated aptitudes (to do) conform the observable behaviors as a response to the stimuli in a real environment. It is about prioritizing what a person does (in front of what he is). It is about applied skills and abilities(Patel & Wani, 2015).
With this objective, the employee develops his formative commitment assuming that the competence on which he must train must be planned and developed by coordinating the learning objectives that are intended to reach; the contents that will be taught; the methods and techniques that should be used; the resources that need to be used; the evidence that will be collected; and the methods and instruments of evaluation of the achievement that will be applied. In addition, you must place all these elements in a coordinate sufficient time so that the student can acquire the ability to solve problems and see enhanced their ability to apply knowledge and skills (Patel & Wani, 2015). Ultimately, when raising the teaching-learning process by competencies, it must answer the following questions: What competence?: What is it? What do I understand by the content of the statement of the competition? What is it about? How is the competition demonstrated? What knowledge does it include? What behaviors? What skills are put into practice? In what contexts? How much is made of the competition? How much is observable? What degree of development is the one carried out or required at a given moment in that competition? Where is this demonstrated competence reflected? That is, where do I consult the exercise of the competition? Where do I check your achievement? Where is it evident? The set of these questions form the framework in which the elements of a competition are inscribed and defined(Patel & Wani, 2015). Analyzing them in detail, we can see that the first two allude to the definition and scope of the competition, while the last two approximate the concept of the result of competition and, therefore, to that inherent in it, which is that of proven . Now we will deal with the first two questions, approaching the analysis of results in the next section dedicated to the evaluation of competences. If we analyze the reference of competences of a degree or of an organization, we can appreciate how each one of the competences is defined by a set of propositions that “constitute theoretical and practical units set in motion in a concrete way and that are the object of a description or a manifestation in the form of knowledge and expertise(Patel & Wani, 2015)
The propositions are expressed with an active or substantive verb, starting the sentence. Said propositions describe units of competence expressed in terms of knowledge (knowledge), of techniques (know-how), and of personal attitudes or attributes (being), whose evaluation is conceived in function of the necessary learning to pass from ignorance to knowledge and acquisition of knowledge. skills and abilities. Therefore, each level of competence corresponds to each level of competence. Institutions and society obtain qualitative and quantitative information on the degree of acquisition, construction and development of skills (Park & Tekian, 2016). This identification of the evaluation as the assessment of the achievement of the competence acquired by the student implies, in practice, a reorientation of the concept and evaluation process traditionally used in higher education, since the main objective cannot be limited to determining what a individual knows about a certain subject, but must assess to what extent the student possesses and dominates a certain competition.
The units of competence, as they have been defined in the previous section, appear as the criteria of evaluation par excellence when defining with clarity and precision the knowledge and skills that make up the competence in question(Park & Tekian, 2016).
That is, the type of evaluation to be applied (self-assessment, co-evaluation or hetero-evaluation) should be chosen, as well as the evaluation techniques and instruments that will be used to obtain objective information on the development and acquisition of competence by the student (direct observation, simulation, case studies, problem solving, tests, etc.). Once the strategy has been chosen and the evaluation techniques and instruments chosen, it will be applied to collect evidence, both knowledge – know- as of the performance – knowing how to do – and of the behavior – knowing how to be – demonstrated by the student in relation to the competence in question (results of written tests, exercises performed, oral presentations, degree of participation in the group, etc.). The next step would be to analyze the information provided by the evidence in order to be able to establish the level of achievement (Najolia & Zolnikov, 2014)..This is done by the student in the competition and then compared with the performance standards previously established by the evaluation criteria. Following the result of this comparison, the student’s achievement would be valued, that is, to elaborate a judgment about the degree of competence that has reached(Park & Tekian, 2016). Obviously, not all the individuals involved will achieve the same degree of mastery or achievement of the competition.
Conclusion
In this context, the role of the employee as an evaluator is also revolutionized. It is no longer enough to issue a final grade that reflects the level of knowledge acquired by the student. It is necessary that the evaluation be considered as a process that requires knowing to what degree the student has the competence before starting the training process (initial or diagnostic evaluation); how it advances in the acquisition or improvement of that competence (process or training evaluation); and, finally, to what degree does the competition have at the end of the process (final evaluation or promotion), that is, assessing what has been its achievement. The evaluation of the achievement of the competition by the student it appears as well as the cardinal axis that structures the entire evaluation process, which must be reconsidered from this new perspective. The essence of this change lies not so much in introducing new elements, applying different methodologies, or using innovative instruments, but in redirecting the tools, methods and evaluation criteria that we have traditionally been using, in such a way that allows us to achieve this objective: establish the degree to which the student has the competence or skills that enable him / her for his / her professional practice. In this context, the competency evaluation process would be considered as follows: First, and once the competences of the degree have been defined in detail, we should establish the evaluation criteria that will allow us to define the achievement of each competition.
References
Chauhan, P., Warren, J., Kois, L., & Wellbeloved-Stone, J. (2015). The significance of combining evaluations of competency to stand trial and sanity at the time of the offense. Psychology, public policy, and law, 21(1), 50.
Jones, J. H., Ziegler, M., Baughman, D. M., & Payne, C. (2016). Mock competencies: An intervention to improve student outcomes.
Jonker, G., Hoff, R. G., & Ten Cate, O. T. J. (2015). A case for competency-based anaesthesiology training with entrustable professional activities: an agenda for development and research. European Journal of Anaesthesiology (EJA), 32(2), 71-76.
Kupfer, J. M. (2016). The graying of US physicians: implications for quality and the future supply of physicians. Jama, 315(4), 341-342.
Najolia, G. M., Dunning, C., Smith, L., Knopp, P., & Zolnikov, B. (2014). Use of the Montreal Cognitive Assessment in Evaluations of Adjudicative Competency. Journal of Forensic Psychology Practice, 14(4), 342-354.
Park, Y. S., Zar, F. A., Norcini, J. J., & Tekian, A. (2016). Competency evaluations in the Next Accreditation System: Contributing to guidelines and implications. Teaching and learning in medicine, 28(2), 135-145.
Patel, S. G., Keswani, R., Elta, G., Saini, S., Menard-Katcher, P., Del Valle, J., … & Wani, S. (2015). Status of competency-based medical education in endoscopy training: a nationwide survey of US ACGME-accredited gastroenterology training programs. The American journal of gastroenterology, 110(7), 956.
Rekman, J., Gofton, W., Dudek, N., Gofton, T., & Hamstra, S. J. (2016). Entrustability scales: outlining their usefulness for competency-based clinical assessment. Academic Medicine, 91(2), 186-190.
Stevens, B., Hyde, J., Knight, R., Shires, A., & Alexander, R. (2017). Competency?based training and assessment in Australian postgraduate clinical psychology education. Clinical Psychologist, 21(3), 174-185.
van Vendeloo, S. N., Brand, P. L., Kollen, B. J., & Verheyen, C. C. (2018). Changes in Perceived Supervision Quality After Introduction of Competency-Based Orthopedic Residency Training: A National 6-Year Follow-Up Study. Journal of surgical education.
Verderame, M. F., Freedman, V. H., Kozlowski, L. M., & McCormack, W. T. (2018). Point of View: Competency-based assessment for the training of PhD students and early-career scientists. eLife, 7, e34801.
Order an Essay Now & Get These Features For Free:
Turnitin Report
Formatting
Title Page
Citation
Outline
