Contents

How to create a rubric scale?

If you have been given permissions as a 'rubric creator' by the central admins, you will see the 'Templates' tab. Under the subtab 'Rubric Scales' you will see the rubric scales available within your institution. Each rubric scale can be translated into multiple languages.

Each level in a rubric scale has visible text, which is shown in the progress review, and additionally has an underlying percentage. The levels can be placed in any order and weighted by any percentage.

You create a rubric scale in the default language of your Portflow installation (e.g. Dutch). You can translate the rubric into your secondary language (e.g. English).

You can create and modify a rubric scale as follows

1. Create a new rubric scale by clicking'+ scale'. Open an existing rubric scale and 'Unpublish' it to modify it.

2. With the 3 dots next to the publish button you can change the Dutch and English names of the scale.

3. The 3 dots on the level card allow you to change the English and Dutch name of the level. You can also change the underlying percentage.

4. In the preview you can see what the rubric scale will look like in the progress review.

Open

How do you make good use of Rubric scales as an institution?

Rubric scales are used to rate quantitatively in progress evaluations. Different rubric scales may be used to value the same goal. This happens when a student has chosen a different rubric scale on different progress assessments on the same goal.

How do you ensure that the growth chart associated with a goal, and the spider diagram that contains multiple goals and progress assessments, still provide useful insights if, for example, you want to make the following rubric scales all available to students:

Open

Open

Open

Open

Open

The rules

Rule 1: Make sure the lowest level is always 0% and the highest level is 100%. Divide the other levels in between. This can be proportional, but this is not the main premise. Rather, look at rule 2 for the distribution between 0 and 100. You can also create a rubric scale where the highest level is "missing," so your rubric scale only goes up to 75%. Only in year 2 can they achieve expert level, for example.

Rule 2: Make sure that a level that generates a certain 'feeling' in different rubric scales also has the same percentage. For example, 'Satisfactory' in one rubric scale has the same percentage as ':)' in another rubric scale. For example, both 50%. It is not recommended to create multiple scales in which the word 'sufficient' appears, but each one actually means something different. For example: 'Unsatisfactory, sufficient, good, excellent' with a rubric scale 'Unsatisfactory, sufficient good' does not go well together. Then 'sufficient' and 'good' suddenly have a completely different meaning at the global level.

Rule 3: These scales need not correspond to what you see as "sufficient" during your formal assessment: the ratings in Portflow indicate growth, not absolute "grades”. Some courses only see an 8 as sufficient for formal assessment, others 5.5. For rubric scales to be used and combined across the institution, it is important to draw one line, separate from the grades in formal assessment. That 6 can be given in the LMS or the assessment program.

From these lines, for example, the following rubric scales follow:

Open

Open

Open

Open

Open

These rubric scales can be used interchangeably by a student because the percentages are used for the growth charts and they are interchangeable. A separate progress assessment where the "words" or emojis are involved still have value on their own.

How do these percentages show up in Portflow?

We'll take the example of these completed evaluations:

First evaluation 'goal 1', at time point 1
Teacher 1: 60%
Teacher 2: 80%

Second evaluation 'goal 1', at time point 2
Teacher 1: 70%
Teacher 2: 90%

Growth chart

In the growth chart (per goal), the average rating (in percentage) per role per evaluation is taken as a data point. A line is drawn between the different points at different times.

Example: In the growth graph of 'goal 1' we now see two data points. The data points are the average per role: 70% (60+80/2) at time point 1 and 80% (70+90/2) at time point 2.

Spider diagram

Results in a spider diagram (multiple goals, aggregated from multiple progress assessments) are based on a moving average: the last rating weighs 65% and the average of the previous results weighs 35%. The rating (in percentage) of the same role is averaged.

Example: In the spider chart after the first evaluation, we now see 70% at "goal 1. After the second evaluation we see 76.5% at 'goal 1' (calculation moving average: 35% * 70% + 65% * 80% = 76.5%)

Progress evaluation

Results in a spider diagram of 1 progress evaluation (multiple goals, 1 progress evaluation) are averaged per role.

Example: In the progress evaluation diagram we see the average score per role: 70% in evaluation 1 and 80% in evaluation 2.