The Reeve Guildmaster's Art: Making Good Reeve's Tests

by Matthias Fleewinter

Contents:

Introduction

The rules update, while certainly a positive development, places a burden on everyone. We are all having to learn and relearn the new rules. However, the burden of learning the new rules is arguably greater on reeves, and greater still on Guildmasters of Reeves since we have to give and sometimes write the tests that certify reeves for the new Rules of Play. If we Guildmasters of Reeves fail to give good tests, we may end up certifying unqualified reeves or hinder players from serving who would otherwise make good reeves.

It therefore falls to the Guildmasters to write and/or give the best tests they can.

But what makes a good test?

My theory on good reeve's tests is that what makes a good reeve, makes a good reeve's test. A good test will act as a set of scales from which a reeve can be weighed and found to be worthy or found wanting. This means that a test should be, as the Rules of Play describes a good reeve, impartial, fair, have a good eye, and be well-versed in the rules. Each of these points can be applied to the design of a good reeve's test.

Test Impartiality

Impartiality as far as tests go can be a tricky thing. Every Guildmaster of Reeves belonging to a group with rules clarifications is tempted to include questions on those clarifications in their tests. While this can be quite practical in the sense that you'd want your local reeves to be familiar with the "house rules" of your group, it is not good test-writing. A reeve's test is about one's knowledge of the official rules of play, not any one group's modifications.

A reeve candidate should be able to take a test written by a Guildmaster of Reeves from Dragonspine, Neverwinter, the Burning Lands, or Rising Winds and, if he passes, enjoy the same level of public trust regardless of whose test he took. One of the strengths of playing in a national LARP such as ours is that we all play by the same set of rules and that we can assume everyone else does too. If a reeve's test contains questions geared to any one kingdom's or group's "playing style", that solidarity is harmed. Therefore, an impartial reeve's test should be universally portable to any Amtgard group.

Test Fairness

The fairness of a reeve's test is derived from the fairness of its test questions. To be sure, no test question is perfect and some can exist in a gray area where the test question is fair for some candidates and not for others. Most tests can survive having a few unfair questions, but if those unfair questions are in a large enough number that it could make the difference between passing or failing the test, then the reeve's test is an unfair one.

There are at least six ways in which a test question can be unfair (and there may be more):

1) The unfair question is out-of-scope for the test. A test question must be Rules of Play-specific. For example, if a test question asks about the level abilities of a monster that appears only in the Dor Un Avathar and not the rulebook, or about the awards that a regent can bestow, it is an unfair question. The reeve's test is supposed to test a candidate's knowledge of the Rules of Play, not the monster handbook or the corpora.

2) The unfair question is too complex. While not specifically mandated, A reeve's test is best if it can be taken in a timely manner. No reeve candidate should be asked to take a test outside of normal Amtgard time unless it can't otherwise be helped, nor should a candidate have to spend hours taking it. Amtgard is not a correspondence course, and everyone wants to be able to show up and enjoy playing at least for a little while even if they have to take the test that day. If the test cannot be completed within 30 minutes to an hour, it is an unfair test. This means that a question requiring the candidate to write a theme paper or a thousand-word essay on some rules-related topic is an unfair question. By and large, a reasonably simple question should be one of the commonly-used types such as true/false, multiple-choice, matching, short answer, fill-in-the-blank, and listing. Any questions of these types should be acceptable so long as they are not unreasonably lengthy. (A question that asks for a list of fifty separate items is probably not a fair one.)

3) The unfair question is too obscure. There are literally an infinite number of obscure questions on the Rules of Play that may be devised. For example: "list the Nth word that appears in the main text on each page of the rulebook, not counting headers and section titles, and put them in reverse alphabetical order." While such questions are technically within the scope of the Rules of Play, if a candidate has to have memorized the rulebook word-for-word in order to answer the question, it is an unfair question.

Now that is an extreme example, but the fairness of such questions can be debatable. That is to say, some Guildmasters of Reeves may consider it to be a fair question to ask for the incantation of an uncommon magic such as Immolation, while others may not. A Guildmaster of Reeves needs to bear in mind that just because he knows the answer to the question, doesn't mean everyone else who would make a good reeve should be expected to know the answer as well. It doesn't make much sense to ask about very rare rules that the average reeve will mostly likely never encounter more than once or twice a year, if ever (although such questions are quite useful as bonus questions!) We are not testing reeves to see if they have photographic memories; we are testing reeves to see if they know the rules well enough to serve as a reeve.

While a reeve's test written to qualify people for reeving at their local group may not be as comprehensive as a reeve's test written to qualify people for kingdom-level reeve status, there is a point beyond which no player should reasonably be expected to know the answer to a given question, and such questions are unfair.

4) The unfair question does not contain enough information: a test question needs a certain amount of context which will let the test-taker determine what information is being asked for. For example: "If a player with an enchantment is hit with a magical ball, is he dead?" Without knowing the player's class, what enchantment he is carrying, and what type of magical ball struck him (and possibly where), the question is impossible to answer. If a reeve candidate can't tell what the question is asking for (even if he would otherwise be able to answer it), it's an unfair question.

5) The unfair question deals with a rule having multiple interpretations: While the current 7.0 Rules of Play are arguably more concise and rigorous than past editions, there have always been situations which the rulebook cannot adequately address either directly or by inference. Under earlier versions of the rules, most groups felt the need to adopt rules clarifications to fix these loopholes, gray areas, or unclear wording in the official rulebook. While the 7.0 Rules are young, it is a hobby of some to hunt for loopholes or gray areas or unclear wording for purposes of exploitation and gaining an unfair advantage on the field, so new instances of loopholes, gray areas, etc., are bound to be discovered sooner or later. Therefore it is important for test-writers to make sure their test questions are not susceptible to "creative interpretation" No test can be made "foolproof", but one has to do one's best. Under 6.0 Rules of Play, the question "Does a lightning bolt striking a player's weapon kill the player as well?" was an unfair question since the ambiguous wording of the effect of the Lightning Bolt spell in 6.0 led to some individuals believing the answer to be a "yes" and some believing it to be "no", all with complete intellectual honesty.

6) The unfair question is misleading. It is the habit of sadistic or playful teachers as well as Guildmasters of Reeves to incorporate so-called "trick questions" into their tests. Such practices are useful for testing to make sure you're paying attention, but their overuse can lead to a candidate's becoming frustrated or confused. It may also compel them to take a meticulous approach to completing the test as they scan for the next trick question. Some trick questions can be fair and some can be unfair; it mainly depends on the nature of the subtle error that makes the item a trick question. A fair trick question might contain a subtle error pertaining to a rule common enough that anyone who overlooks it really should have known better. An unfair trick question is one which would require the reeve candidate to read the test writer's mind or to otherwise be able to predict what he was really thinking, in order to sidestep the land mine.

The practice of incorporating trick questions into one's test is not a good thing to do, unless done in moderation. A reeve's test is not, after all, meant to test whether a candidate can find and avoid falling into all the traps laid by the trick questions; the test is intended to determine whether a candidate knows the rules. Therefore a question such as, "If a druid charges a flesh-to-stone magical ball and tosses it at a third-level monk, is the monk turned to stone?" is an unfair question if a "yes" answer is identified as wrong simply because Druids now call that particular magical ball Petrify and not Flesh to Stone.

On the other hand, a question such as "A wizard charges a fireball and hands it to a paladin with protection from flame, who tosses it and hits a first-level monk just as the monk tosses a throwing dagger at the wizard. Who was killed?" is, while more complicated, not a misleading question since the question has no subtle errors, but contains easily identifiable instances of the rules being broken which a knowledgeable reeve should catch: the paladin is trying to use a fireball he did not cast, and the first-level monk is trying to use throwing weapons but is not high enough level to use them. Yet in the first example, the term "flesh to stone" is still synonymous with "petrify" as a holdover from earlier editions of the rulebook.

Continue to "Having a Good Eye": Test Elegance