Meta Evaluation

On acquiring the skill of learning how to learn

This article discusses meta evaluation, or the skill of learning how to evaluate oneself effectively in the process of learning.

This process of learning how to learn must be grounded on the individual's goals and objectives. Without clear goals or objectives, it will be difficult for you to discern whether or not you are learning effectively.

This article ends with a 1 to 5 scoring mechanism that helps you evaluate your learning process.

The difference between Meta and Performance Evaluation

In simple terms:

• Performance evaluation means evaluating the skill being trained (e.g. you might be practicing how to type faster as a programmer).

• Meta evaluation means evaluating the method, style, or mindset of your learning (e.g. is this the most effective way for me to practice typing faster? Am I reaping good rewards from this process or am I better suited to explore a different method?)

To provide an example, let's look at basketball free-throw practice. Performance evaluation explores the following questions:

• Successful shots - How many hoops can you shoot successfully in a limited period of time (e.g. 20 out of 40 shots)?

Meta evaluation explores the following questions:

• Direction and alignment - Have I specifically set my goals for learning basketball free-throws?

• Introducing - Is my mind clouded and distracted because of fears, anxieties, worries, or insecurities?

• Familiarizing - Am I committed to my training?

• Training - Do I show up to the practice focused and attentive, and humbly requesting feedback from peers, educators, coaches, or mentors?

• Polishing - Am I actively practicing the recognition of my mistakes and the recommendation of improvements similar to my mentor's correction and guidance?

• Mastering - Do I explore other people's standards and define one of my own?

Answering those questions truthfully helps you recognize the shifts in the stages of training as your mindset and feelings shift, and helps direct you in both focus and pivoting your training or your short-term goals.

Motivation

Learning to evaluate your learning process helps individuals who might get stuck in accomplishing their goals because of bad habits in their learning process (distractions, insecurities, uncertainty, fear) and mastery of this skill helps individuals with effective double-loop learning.

Double-loop learning entails the modification of goals or decision-making rules in the light of experience. [...] Double-loop learning recognises that the way a problem is defined and solved can be a source of the problem.

Souce: Wikipedia

By understanding clearly the whole process of learning and diagnosing the stage of learning a student might find themselves in, then they can be more effective in directing themselves or reassessing whether their goals or directions ought to change.

Evaluation Scoring System

We follow a simple 5-score model for evaluation, whether for self-evaluation or peer evaluation by Community Members.

Score 1: Introducing

If you are still new to a topic, one might be intimidated by the variety of new words they are encountering. This is the Introducing stage of learning a new skill.

Feelings: Uncertain, anxious, intimidated, fearful, overwhelmed, adamant, hesitant

Score 2: Familiarizing

Once you are no longer overwhelmed by the newness of things, you would be at a Familiarizing stage to the concept, without committing to learning it yet. Whether you are convinced or not, you have not yet decided to allocate time and resources to practice the skill.

Feelings: Cautious, open, enlightened, non-committed, curious

Score 3: Training

When you have committed resources (time, energy, money) to learning a new skill, you are now at the Training stage of learning a new skill. You don't have the literacy yet to recognize mistakes, but you are slowly getting the mechanical aspects of the new skill you are learning. Here, external feedback is critical to ensure you gain insight on your growth progression.

Feelings: Focused, humbled, striving, disciplined, devoted, unaware, erroneous, goal-oriented

Score 4: Polishing

The Polishing stage of learning a new skill is when you are able to sufficiently perform the skill or activity, and have gained the literacy to self-correct. You and your mentor are similar in your assessments of yourself (e.g. the precise recognition of your mistakes and wins). You may not be confident in attaining perfect results, but you have acquired some level of literacy to sustain your positive feedback loop of learning independently.

Feelings: Self-aware, conscious, intellectually humble, sustainably critical, consistent, confident

Score 5: Mastering

You clearly understand your wins and mistakes. You actively seek out the complexity, nuance, and precise description of your mistakes. You no longer take on other people's standards and define your own. You are now at the Mastering stage of your skill or at the forefront of reinventing the skill.

Feelings: Independent, self-directed, nuanced, optimal, artful, elegant, expert

Benefits

• Map dispels uncertainty - Acts a simple 5-stage map of learning, which helps individuals diagnose what stage of learning they are in.

• Grounded and qualified goals - Having a simple but exhaustive model for learning, this helps individuals decide up to what level of proficiency they want for a certain skill (e.g. I only want to be familiar, not masterful).

• Create mental space - By defining the appropriate skill level to pursue, individuals can create space for other things that they want to pursue: rest, balance, other skills or goals.

Drawbacks

Why should we not do this?

• Meta evaluation is not based on correctness - Unlike other measures of evaluation that is focused on correctness (i.e. you are right or wrong), this measure focuses more on the person's learning and growth mindset rather than the correctness or incorrectness of the student's performance.

• Accuracy, subjectivity, truthfulness - For people performing self-recorded meta evaluations, there are no safeguards to ensure that the evaluations are indeed true or not. For mentor-based evaluations, it is also possible for subjective factors and biases from the mentor. As such, meta evaluation would be more reliable and useful as a signal for external evaluators only in the process of evaluating how reflective and adaptive the individual is, more than their concrete performance evaluations.

• Evaluation based on correctness is assumed to exist - In the context of software engineering, we assume that there exists an innate, positive feedback system for "correct" behavior. These could be from a wholistic scale such as the whole system which customers complain about, or at a granular perspective like unit-tests with green checkmarks, or merge requests or pull requests containing discussions, feedback, and approval.

• Costly - To consistently perform meta evaluations, whether self evaluations or not, in addition to existing performance evaluations takes time and effort.