29 replies to this topic

[istaril]

For those of you who haven't heard, at GAMA this year FFG announced they'd be implementing a judge program this year. To be fair, they've announced this the last two years too, but they acknowledged that and promised it was 'for real this time', and I believe them because... umm... fool me once? Anyway, they've stated they'll have a judge certification program in July 2016.

 

In an effort to avoid me writing another "What are you Smoking?" when FFG comes around and missteps with it, I thought we should get a group discussion going about what we hope to see from a Judge program - and maybe I can compile it and send it FFG's way so they can at least take it into account when implementing this.

 

Some of the aspects I'd love to hear thoughts on are 

 

- How other games/sports/etc handle it, and what you like/dislike about the way they do (preferably games/sports at non-professional levels). 

 

- Judges playing in events (currently, the new tourney document allows a 'leader' to play in an event if there is a second leader, and only at the 'relaxed' teir - so not at Regionals and up)? How many judges/marshals for an event?

 

- What kind of 'certification' a judge should have. How often should it be updated. How should it be updated. Should it be tiered? Can it be revoked?

 

- How to incentivize Judge/Marshal participation (e.g. swag for them, or "the store can only run a regional if they have a marshal, so it's up to the store/community to incentivize, and so on, or "Only judges can order game night kits", etc) or traveling to events to help if they can't play.

 

- The extent of a Judge's authority (DQs, etc), how to back up that authority, and how to appeal a judge decision .

 

- Whether judges should be obliged, encouraged, or somehow granted special status with regards to participation on 'official' rules queries (like the rules board here, or the rules link fantasyflightgames deals with).

 

- Whether remote judging can be permitted. If someone can run a tournament, could I be available through skype to resolve rules disputes or answer rules queries? I mean, that sounds like a miserable gig, but I'd occasionally do it if it allowed somewhere to have an event they couldn't otherwise have. But it has significant disadvantages for handling issues pertaining to sportsmanship/code of conduct.

 

And, honestly, pretty much anything else you guys can think of. I haven't yet given this topic quite enough thought to even frame all the right questions!

[GalacticTaco]

"Can it be revoked?"

I'd say Yes. Now bad calls can happen, but no judge who devellops a bad rep, or who shows lacks objectivity or wavering ethics should stick around or be allowed to judge events without some sort of consequence.
In particular if those situations create tensions during finals.
Maybe suspensions for a certain amount of time, a year? Or for a # of tournaments or just suspended from a certain tier of event (formal, etc).

If there are tiers, would the size and level of the event determine a suggested amount of judges/marshals of each tier at a given tournament?

[Hinkbert]

An online quiz with pertinent rules questions to certify who can be a judge. 80% accuracy is required to succeed. That's was Pokemon does

[jaycr0]

Willingness to check online sources before ruling would be nice.

The whole Tears of Lys/immune to events thing is going to sound like you're totally trying to double talk and trick someone because it's so counterintuitive. It would be pretty awful to expect that to work only for a judge to use just a copy of the rules and "common sense" to rule. Not that I disagree with common sense but if the (not technically official but widely accepted) consensus isn't accepted, how do I know the rules going into each game?

[lmfeuer]

While I am very excited about an official judge program, in fact I plan to sign up. I am concerned about a requirement for there to be at least one judge who does not play at a regional. 

 

If I assume that future regionals will be in the same or similar locations as the last few  years I will only have a chance to participate in 1 a year. The regionals in my area are 1.5 hours, 6.5, and 8 hours away. So I will only be able to go to the nearest one. Giving up my one chance to even go to a regionals and not play isn't something I am willing to do. And I don't know anyone in the Florida meta who is. I am sure this situation would occur elsewhere too. 

 

Unless the game store owners go through the process to become judges I don't know how regionals will function in my area. I am lucky that we have one awesome game store (Dogs of War) who has hosted the Thrones regionals for several years now. The owner will likely just become a judge himself or have an employee do it. But the certification process needs to be easy enough that it can happen quickly. For instance testing can occur at anytime instead of just once or twice a year and results provided almost immediately. 

[keithsboredom]

I like the idea of judge only item like tokens, mats, cards. maybe give them free entry into higher level events like worlds or nationals

[Zigur]

- Judges playing in events (currently, the new tourney document allows a 'leader' to play in an event if there is a second leader, and only at the 'relaxed' teir - so not at Regionals and up)? How many judges/marshals for an event?

 

This seems fair to me - although, at that point, the playing judge doesn't need to be judging anything at all - the judge without a conflict can judge.  So the "active" judge can't play, basically, but I guess is allowed to assist the ruling judge?

 

- What kind of 'certification' a judge should have. How often should it be updated. How should it be updated. Should it be tiered? Can it be revoked?

 

Tiering is a good idea - SC/GNK, Regionals, Nationals and up seems like a natural divide.  I assume for nationals and worlds FFG will provide their own judges.

 

- How to incentivize Judge/Marshal participation (e.g. swag for them, or "the store can only run a regional if they have a marshal, so it's up to the store/community to incentivize, and so on, or "Only judges can order game night kits", etc) or traveling to events to help if they can't play.

 

Only judges can order kits seems a bit much to me.  I don't think GNK tourneys need judges, to be honest.  They're not part of the main OP circuit, so I don't think there's any real need to ensure judge participation.  Maybe an extra promo card or something, but nothing more than that.  

 

SCs and Regionals should require judges, and FFG should be involved in ensuring the judges show up, but I don't think there's anything wrong with stores offering extra incentives.  Extra copies of the mid-tiered SC/Regional goodies for judges, and perhaps a unique judge goody at regional tournaments would do the trick.  

 

At nationals and beyond, FFG itself should be compensating the best available judges from SC/Regionals - probably with $ and goodies.

 

- The extent of a Judge's authority (DQs, etc), how to back up that authority, and how to appeal a judge decision .

 

Judges should be able to DQ for cheating or extremely inappropriate behaviour.  There should be an avenue of appeal for DQs to get the DQ off your "record," as it were.  I don't think further compensation is warranted or appropriate.  As for enforcement, don't let organizers have another event if they ignore a judge's DQ.

 

Rulings should be binding for the event.  It's not practical to wait and see what Nate says every time there's an argument, and it makes no sense to offer play-overs every time a judge makes a bad call.  At nationals and up, FFG should have multiple judges, and an appeal should be available to the other judges.

[Ironswimsuit]

I don't think limiting regionals to stores that have marshals available is any more intuitive to FFG's current selection process. It'd obviously be nice to have someone with that authority at these events, but it may not be representative of the geographic locations of marshals, at least initially.  Allowing judges to order kits effectively gives them a TO hat, which they may already be wearing. Kits ordered by individuals not associated with a store may cause friction with the proprietors of the play space. However, I'd love to be able to Judge and TO something out of my house without having to go through a retailer. My area might see more events that way.  

 

 

I love the idea of special swag. Set up a place online for judges/marshals to log events and either earn points based hours worked(for Thrones swag...or other FFG product) or just send out the items per event worked. Other publishers already do something similar for their demo crews.

 

 

Judges should be tested regularly, probably every time there's a new faq. They should be on a mailing list to receive important rulings as they occur. The tiers already appear to exist with the rank of Marshal. These titles can be revoked based on performance or failure to keep up with testing, or other tom-foolery as listed in the unsportsmanlike conduct section of the tourney rules. 

 

For large events, like Nationals and Worlds, FFG should either supply their own arbitration or incentivize judges to partake. 

 

The number of judges could work at 1 for every 16 players, or 2 for 16 if judges are allowed to play in the event. Their authority for the duration of the event should be solid, allowing for appeals after the event as needed, with evidence presented as needed.

[Scottie]

I've been clamoring for a Judges program from FFG for years now. I've been involved in too many events for FFG products in which the people running the event and acting as judges really just don't know the rules for the game or how to run an event. And I'm talking not just at a GNK or SC level, but at Regionals. Even the FFG employees the Judge at Gencon at Worlds don't always display the needed level of rules knowledge or ability to enforce proper behavior from players.

My worry is that if FFG's Judges Program is handled as they tend to handle OP issues that it will just plain be ineffective.

As to the things brought up in the orginal posts.

1. Magic the Gathering has the most robust an formalized OP and Judges program that any game does. It would be easily to adapt to FFG'S games, especially the LCGs. FFGs new head of OP is/was a Lvl 3 MTG Judge as well from my understanding.

2. Judges should be free to play in any events they want, provided they aren't acting as a Judge for the event. No reason to prohibit thier partcipation provided they are acting as players. Regionals should require at least 1, but ideally 2 Judges as apart of the Leaders running the event. I think it is unrealistic to expect SCs to have certified Judges at this stage, but again ideally every "Official" event should require a Judge.

3. It should be a test on the rulebook, FAQ, General Event Guidlines, game specefic Event Rules, and Floor Rules for the specefic game. You should have to retest yearly.

4. Stores should not be given Regionals without having a Judge available to be a part of thier Leaders for the event. One of the largest issues with FFGS events right now is that you have no idea if the people running it know the game at all. I've been to multiple FFG Regionals in the past in which I had to insist that the TO go look at the rulebook or FAQ because I had overheard them giving a basically incorrect basic ruling. Part of the Regional kits should include Judge specefic swag as a form of compensation for sitting out and helping run the event.

5. This is where I fear FFG will have the biggest issues. Without Floor Rules in which they define various infractions along with requisite penalty there is no real ability for TOs/Judges to properly run an event. Yes right now TOs have unlimited authority to act on things as they see fit, but that actually causes TOs to not act. Things need to be clearly defined as to give TOs/Judges a clear indication of how and when to act on things such as player infractions. Only Netrunner as Floor Rules right now. You can test a person on the rules of a game all you want, but knowing the rules doesn't equip you with how to deal with what happens when players act outside of the rules. Which has led to issues in the past even at FFG hosted events.

[jaycr0]

If you don't allow judges to play you're just going to have TOs "judge" and ask the knowledgeable players anyway so you might as well just require a judge and a secondary judge who can both play.

That way you don't have tournaments in smaller regions just blowing off the rules entirel .

[Vorrt]

1. I agree with Scottie.  MtG has a super strong and robust Judging system that I think FFG should take a long hard look at and find away to adapt something like it to their needs

 

2. Starting at the Store Championship Level, Judges should not be allowed to play in any tournament they are judging.  GNK's are meant to be introductory to the concept of OP, so having judges play in those is a great idea.  A large number of the judges I've seen in other games use those type of events as a great place to teach new players, as well as help develop tournament experience for those looking to move up to more challenging tournaments.  SC's and up, the Judge of the event shouldn't be allowed to play, so they can be available for judge calls at any time and keep the tournament flowing.  Regionals and up, I think there should be at least 2 or more judges to handle the load of players (if you look at PPTQ's, PTQ's,  and GP's for MtG, they have multiple judges of staff to help manage the player base). Nationals and up, the judges should be between 4-6 Judges to help with proper coverage (maybe a couple extra to assist with scorekeeping and deck checking). Worlds, I'd say a minimum of 6 judges.

3. I think there should be a certification system (again, I'm going to reference MtG as my example).  Tests to establish that the judge is paying attention to all faq/errata/rules that are in effect for the game, along with any tournament rules.  I think it would also be very beneficial that the judges be organized into regional groups (i.e. the judges of Michigan, Ohio, Indiana, Illinois are the Great Lakes region in the US) so that they can assist with traveling to other stores to help out (which leads into my compensation answer).  I would imagine the Judging tiers would be: Store Judge, Regional Judge, National Judge, World Judge

- Store Judge: Shows competent understanding of the rules/faq, able to organize store level events to promote and increase player involvement

- Regional Judge: Shows advanced understanding of the rules/faq; coordinates with other Regional/Store Judges to schedule their SC's and Regional Events to eliminate overlapping tourneys

- National Judge: Shows expert understanding of the rules/faq: works with FFG to help run the National events smoothly and efficiently

- Worlds Judge: Shows mastery of the rules/faq; works with FFG to help run the Worlds events smoothly and efficiently

 

4. Compensation suggestions

- Store Judge: current practice (additional top prize (non-bye/plaque from SC) for the judge (i.e. playmat, coin, card, tokens, etc)

- Regional Judge: Travel per diem (mileage; gas money), a voucher worth $X that can be used in any FLGS; custom Regional Judge product (i.e. sleeves, playmat, promos)

- National Judge; Travel per diem, Food per diem, Lodging per diem; voucher higher valued than Regional; custom National Judge Product (i.e. shirt, tokens, and so on)

- Worlds Judge: Travel, Food, and Lodging per diem; voucher higher value than national; custom Worlds Judge Products

 

5. Same answer as Scottie

[istaril]

Before posting any of my own thoughts on the matter (probably this weekend - I'm still thinking a lot of it through), I'm adding a few more points of discussion I thought of. I've edited my original post to add the following questions:

- Whether judges should be obliged, encouraged, or somehow granted special status with regards to participation on 'official' rules queries (like the rules board here, or the rules link fantasyflightgames deals with).

 

- Whether remote judging can be permitted. If someone can run a tournament, could I be available through skype to resolve rules disputes or answer rules queries? I mean, that sounds like a miserable gig, but I'd occasionally do it if it allowed somewhere to have an event they couldn't otherwise have. But it has significant disadvantages for handling issues pertaining to sportsmanship/code of conduct.

[GalacticTaco]

I think only questions pertaining to ruling of cards/game rules could work through messaging, but never by demand of a player as this could be abused for slow play. Anything related to managing misconduct should only ever be handled on site (a pair of eyes and ears on site is necessary to handle it well)

All rulings by judges should be considered final say within that particular event only. If using tiers, higher level judges' decisions should be considered final say over lower tiered ones within that same tournament.

If a registered judge is not allowed to play at an event, they should be compensated with swag or ffg credit or something...

[Zigur]

Double. 

[Zigur]

- Whether judges should be obliged, encouraged, or somehow granted special status with regards to participation on 'official' rules queries (like the rules board here, or the rules link fantasyflightgames deals with).

 

- Whether remote judging can be permitted. If someone can run a tournament, could I be available through skype to resolve rules disputes or answer rules queries? I mean, that sounds like a miserable gig, but I'd occasionally do it if it allowed somewhere to have an event they couldn't otherwise have. But it has significant disadvantages for handling issues pertaining to sportsmanship/code of conduct.

 

Remote judging for rules questions is OK for SCs that have no other option, I think, but nothing further.  

 

I think encouraging participation on the forums is a good idea, but obliging might go too far.  As far as forum status, for my part I don't really care on the forum what tag someone has next to their name.  The best explanation is what's important, not who its coming from.

[Scottie]

Ideally FFG would set up a Judges only message board so that Judges could most easily keep up from rules, universally have access to FFG for quicker clarification and talk about issues without airing them out in public.

I'd say no to remote judging. The reason being the same for why I'd argue no player judging. Acting as a Judge just isn't answering rules question, or at least shouldn't be. It is also about enforcing the rules and ensuring proper play. To do that you can't be off site and you can't be playing. Now if FFG wants to allow Judges to consult other non-judging Judges on solely rules questions to ensure the most correct answer is given, whether those non-judging Judges be on site or not, that'd be fine. But you can't do the sheriff side of being a Judge through Skype or if engrossed in your own game.

[Masus04]

An online quiz with pertinent rules questions to certify who can be a judge. 80% accuracy is required to succeed. That's was Pokemon does

 

Online would be very useful in order to get more people into it. Also in areas where FFG is not as present (many places in europe)

[Khudzlin]

1. I agree with Scottie.  MtG has a super strong and robust Judging system that I think FFG should take a long hard look at and find away to adapt something like it to their needs

 

2. Starting at the Store Championship Level, Judges should not be allowed to play in any tournament they are judging.  GNK's are meant to be introductory to the concept of OP, so having judges play in those is a great idea.  A large number of the judges I've seen in other games use those type of events as a great place to teach new players, as well as help develop tournament experience for those looking to move up to more challenging tournaments.  SC's and up, the Judge of the event shouldn't be allowed to play, so they can be available for judge calls at any time and keep the tournament flowing.  Regionals and up, I think there should be at least 2 or more judges to handle the load of players (if you look at PPTQ's, PTQ's,  and GP's for MtG, they have multiple judges of staff to help manage the player base). Nationals and up, the judges should be between 4-6 Judges to help with proper coverage (maybe a couple extra to assist with scorekeeping and deck checking). Worlds, I'd say a minimum of 6 judges.

3. I think there should be a certification system (again, I'm going to reference MtG as my example).  Tests to establish that the judge is paying attention to all faq/errata/rules that are in effect for the game, along with any tournament rules.  I think it would also be very beneficial that the judges be organized into regional groups (i.e. the judges of Michigan, Ohio, Indiana, Illinois are the Great Lakes region in the US) so that they can assist with traveling to other stores to help out (which leads into my compensation answer).  I would imagine the Judging tiers would be: Store Judge, Regional Judge, National Judge, World Judge

- Store Judge: Shows competent understanding of the rules/faq, able to organize store level events to promote and increase player involvement

- Regional Judge: Shows advanced understanding of the rules/faq; coordinates with other Regional/Store Judges to schedule their SC's and Regional Events to eliminate overlapping tourneys

- National Judge: Shows expert understanding of the rules/faq: works with FFG to help run the National events smoothly and efficiently

- Worlds Judge: Shows mastery of the rules/faq; works with FFG to help run the Worlds events smoothly and efficiently

 

4. Compensation suggestions

- Store Judge: current practice (additional top prize (non-bye/plaque from SC) for the judge (i.e. playmat, coin, card, tokens, etc)

- Regional Judge: Travel per diem (mileage; gas money), a voucher worth $X that can be used in any FLGS; custom Regional Judge product (i.e. sleeves, playmat, promos)

- National Judge; Travel per diem, Food per diem, Lodging per diem; voucher higher valued than Regional; custom National Judge Product (i.e. shirt, tokens, and so on)

- Worlds Judge: Travel, Food, and Lodging per diem; voucher higher value than national; custom Worlds Judge Products

 

5. Same answer as Scottie

 

Looks great. But don't forget Continental championships (which have been announced at least for North America and Europe).

[istaril]

Some of my own thoughts, in parts because 20-month-olds don't like being ignored.

 

Certification 
My understanding is that games like Magic have a mentorship program, 'test' tournaments, and a judge needs to administer the test to certify to be a judge. This presumably ensures that judges are ready to handle in-person situations, enforce the many penalties and DQs that can come from play in Magic, and avoids the issues of an online test that, no matter how thorough, is more likely to be gamed or cheated. While I think that all sounds great, I just don't think it's a possibility for Thrones yet - I just don't think whatever your 'first wave' of judges would be sufficiently distributed worldwide to enable this to grow. 

 

I'd personally envision that to apply to be a judge, you have to show you've been 'leader' at a relaxed event (say, by the relevant entry in the Annals!), write an online test in the language of your choice (of languages to which the game has been officially localized), and that certification would renew every year provided you had been a Judge at at least one recorded *formal* event and been active on rules boards/judge boards, otherwise you'd have to re-test. In order to make the test more robust, I'd consider making a small entry fee - or a significant lockout period (entry fee actually best), and as part of the test you'd also have to submit a question to add to the test (approved by the test-setters). This would in theory eventually generate a pretty robust test, where you could pull 20 random questions from a database of thousands.

In order to be a marshal, which I'd probably require for Premier events, you'd probably need to show you'd been a Judge at a Formal event, been active on the boards, and you'd probably have to re-write the test with a higher grade cutoff, and have a phone interview with an existing Marshal to 'vet' you. The test (but not the interview) would probably have to be re-taken every year to be maintained.

 

Certification would naturally lapse after a year unless renewed as described above, and you'd have a judge account against which people could lodge complaints. 

 

Using Judges 
- A judge would be required for a Formal or premier event, probably one for every... 32 players?
- In addition to the judges, at least one marshal would be required for a premier event, regardles of size.

- When acting as a judge or marshal, the judge or marshal cannot participate in the event as a player.

- Judges & Marshals would be granted special status on Rules boards (title, etc), and be expected to be involved in answering rules questions online where/when able.

 

Judge Authority:
- If a marshal is not present, then within a tournament a judge's authority is absolute over matters of rules, DQ, etc - superceding even the RR book, FAQs, etc. He or she may consult with other judges and players, but once a judge has been called in - that judge gets the final say. While a complaint can be lodged against the judge for errors in judgement or mistakes and the juge may be censured for it, the ruling (within that tournament) will still stand. "A judge is right even when he or she is wrong", and is expected to have the full support of the players during the tournament itself.
- If a marshal is present, a judge's ruling may be appealed to a marshal, and the marshal will be the final authority (as above).

To come: Judge privileges/rewards/incentivizing.

[istaril]

Judge "Benefits"

Given that we're requiring judges to write an exam, possibly pay some small fee to do so, requesting forum involvement on rules questions, and (more important still) not allowing them to participate in events where they are required as a judge, we need to find some means of rewarding judges for their time & effort on behalf of the community. There also has to be acknowledgment that 'becoming a judge' isn't something everyone wants to (or should want to do), so it shouldn't offer a competitive advantage.

 

Let's start with the minor (and easy) ones I would envision:

• A Judge/Marshal would recieve that title on cardgamedb, a recognition of their contribution and acknowledgement of their authority in rules contributions. Other community sites would likely follow suite.
• A Judge/Marshal forum would be opened, and FFG OP would be present there as well to keep judges involved in changes to the organized play program, and discuss potentially problematic (non-rules) issues away from the community at large. They could proofread the damn Tournament Rules documents. 
• Having an attending Judge/Marshal is required for Regional and higher events, and therefore is required as part of the application process. While it sounds double-sided, that means that the privelege of being a judge is allowing your local community to host a high level event (and some of the burden of reward is on the community, as thanks). Higher 'rated' judges could factor heavily in the attribution of regionals.

 

More involved rewards (less obvious):

• 'Future event' rewards; Judges could receive, as part of the reward for juding a tournament, a free entry to a subsequent Regionals, Nationals, or Worlds (that year) as players. They could potentially recieve reserved spots at Worlds, just like regional winners. While neither of these is exactly easy to implement, as local stores set prices on regionals (and it's not obvious to have FFG pay them entry fees) - and it could potentially result in a lot of reserved slots at worlds, but it's not a bad start for a reward for 'sacrificing' your attendance at one event.
• 'Additional Swag': You could continue including additional swag for the judge. While this isn't really bad, I don't much like it as a reward structure because getting the same swag without paying for it feels cheap - and it doesn't scale with number of judges. Perhaps alternate judge-only swag ordered after the tournament (once data has been submitted to FFG/the Annals) for each Judge.
• 'Point system'. The old gold dragons reward system is something we'd really like to see brought back for players - where they accumulate points based on standing (making all standings matter, encouraging repeat particpating, choose-your own swag), but a Judge program might be the best point to pilot this. It could include such neat rewards as having your likeness appear on a card (for a sufficient number of judge points), something FFG already does for its employees (and isn't likely to be terribly expensive)
• Money/Product/Per Diem/Store Credit: There would seem to be something fitting in at least covering the cost of the hobby to someone who is sacrificing playtime to allow it to grow, and providing, say, 2 CPs per event judged (or some equivalent system) would not seem to be excessive, nor too expensive to FFG.
• Rather than use staff, FFG could offer accomodations/entry for Nationals/Continentals from a pool of eligeable judges, as a 'reward'. Not sure how they'd be chosen, but as it's high profile it would probably be a pretty solid PR move for not terribly high costs. I know they've done this in the past having ktom judge such events.