Official Review Guidelines

For reviewing RMS files, please use the RMS Review Guidelines; you can examine the old Review Guidelines here.

The five categories used in reviewing and described below should be used to define the various scores handed out by the reviewer. In the review all the scores handed out should be justified. However, this does not mean that a review must merely consist of ratings and a justification for each. This does not qualify as a proper review, as it is of little use to player or designer alike. Therefore, make sure your review will always be readable and understandable to those who have not played the scenario or campaign, as they want to know what to expect, and that the designer gets to know the weak and strong spots of his design.

Playability

Playability is probably the most subjective element of the scoring. It is simply a gauge of how much fun you had playing this particular scenario, in other words, an overall score of how enjoyable you found it.

There really are no specific criteria on how a score is given in Playability but there are quite a few things that can affect playability in a negative manner. Trigger bugs, victory condition bugs and any other playability-destroying bugs obviously can ruin a scenario's playability. Lag is another playability issue that a scenario can be marked down for. If a player is ever confused about the next goal to accomplish, that's a playability problem. If a player can complete an objective in a way that the author obviously did not intend to be possible (i.e. there's a hole in a wall that allows the player to skip half the scenario), that's a playability problem. Anything that adversely affects your enjoyment of a scenario can affect the Playability score, including factors considered in the other categories. It should be noted that the other categories of the review can themselves be marked down for issues of playability if the implementation of those areas negatively affects playability. The Playability category itself should be treated independently, and should be considered an overall judgement of the scenario's most important aspect: how entertaining it is.

Balance

Note: This category has been renamed "Pacing" for Cinematic Scenario ratings (effective 9 July, 2014).

Balance is also somewhat subjective since each player is a different skill level and what might be perfectly balanced for one player, might be way too easy or way too hard for another. As a reviewer, you must take your own skill level into account when giving a balance score to minimise this bias. A perfectly balanced scenario should provide a challenge for a veteran player while not being overly frustrating for a player fresh from the standard ES campaigns.

A player should not win by luck, the scenario should be constructed so that a player can learn from mistakes and use his skill to complete the objective.

It is extremely likely that some form of difficulty dynamics will have to be present for a high score to be achieved. However this alone is not sufficient. If a scenario is too easy even on the hardest available difficulty, or too hard on the easiest, this is a reason to deduct.

One important note about scoring the balance category for scenarios is that where no fighting takes place, such as some puzzle scenarios and some rpg style scenarios, is that just because the player cannot die in such scenarios, that doesn't mean the scenario isn't balanced. Difficulty can also be present via puzzles or other devices, and the balance of these should be taken into account.

For scenarios with no interactivity, such as cutscenes, this category should be used to examine the flow and technical merits of the cutscene: did it run smoothly? Was everything technically put together well?

Multi-player scenarios are reviewed a bit differently in terms of balance. Each human player should start out in an position of approximately the same strength, with an equal chance of winning provided the skill of the players is equal. Obviously, the players' positions don't have to match exactly, but they should be balanced. There are many creative ways that designers can make each player different, yet still balanced. In relevant cases, the map should also be examined to determine if all players have access to the same amounts of on-map resources.

Creativity

Creativity can be found in all aspects of a scenario; from trigger tricks, to map design, from the story, to the objectives, from sounds used, to what units a player is given. The most creative scenarios will generally have many aspects that are uniquely developed, and contribute to a greater impression of creativity and originality. However, considerations for scoring are not limited to original aspects and their sources of origin only. As such points should not be deducted from creativity if a design uses the same idea for a trick, or theme, or story, or anything else that may have been used in another scenario.

Creative aspects should be in an appropriate context to whatever they enhance. Whether it be the theme, objective, scene, game play, narrative, or mere elements, the aspects should give a unified impression. Therefore, anything which is new, or a fresh take, or highly developed while still fitting the intent of the scenario should be considered creative. Similarly, anything which seems haphazard, or is routine, or is undeveloped should by and large, detract from considerations of creativity.

A simple approach to scoring creativity is to reward aspects that are presented in new ways, that are striking especially in conception or style. The highest scores should generally be reserved for a scenario that displays creativity in many areas, or gives a great overall impression. Likewise, a scenario that lacks creative development in many areas should generally be given lower scores. Keep in mind however, that adaptation, and investing basic concepts with new forms and functions are keys to creativity. Hence, a scenario need not have any new tricks, nor a raft of new ideas to achieve the highest score.

Map Design

Map design is both an indication of the aesthetic quality of the map, and how well the map functions to aid playability. With this in mind, some general (but not hard-and-fast) rules can be established. A random map is usually a 2. Random maps can look fairly good, they can function well and there's nothing wrong with using a random map in a scenario if it is suitable, but it's below average (unless the random map is simply used as a base and is altered). If the scenario used a random map as a base, but proceeded to add details to it, then a score of 2 is obviously too low. There is nothing to stop a well-implemented random map scoring highly, but only if it suits the scenario's playability.

A rating of 1 is usually for a pathetic map, these typically consist of large blank areas with lots of square areas and straight lines. These maps look completely unrealistic and are quite unattractive. However, if a random map detracts from playability it can also be awarded a 1.

4s and 5s should be reserved for the best maps, ones that are easy on the eye, fit the purpose of the scenario, and are well implemented. It is up to the reviewer to decide whether a map is very good or exceptional, and rate accordingly. Remember to bear in mind how well the map design improves playability, as well as the map's appearance. This is particularly relevant in reviewing multiplayer scenarios.

One final note on scoring map design, only the portion of the map that can be seen during play should be scored. If there are large empty areas that a player never sees, that should not affect the map design rating.

Story/Objectives

This category is not as clear-cut as it first appears. Obviously a scenario without story or instructions would score a 1 (generally), but that does not mean that the automatic presence of either qualifies the scenario for a higher score. The presence of a decent story (not necessarily well-developed) and functional instructions should be the midpoint, a 3. Anything better or worse than these can be marked up or down accordingly.

If the instructions are wrong, misleading or confusing, the rating can go down.

An introductory bitmap is a nice touch and a good image can often raise the score, however, it is not required to score a 5. It certainly helps, but it is not an absolute requirement. Hints and History can also be judged here. These two areas are not required, but they can also help boost a scenario's score. While a bitmap, hints and history are not required, it would be difficult to give a rating of 5 if all three areas are missing. The rating should not be affected based on whether the story is fictional or historical. It doesn't make a difference as long as there's a story that draws the player into the scenario.

Another consideration that factors into the rating of the story and instructions is grammar and spelling. A designer should be diligent in this area of his scenario since it's very easy to copy the text into a word processor and spell check the instructions. There's no excuse for having spelling errors in a scenario, it simply shows a lack of effort on the part of the designer. The only exception we make is for designers whose primary language is not English, but many of our non-native English speaking visitors have excellent command of the language, so this is not a get-out clause. Instructions and story should be comprehensible in the language they are written in.

Finally, a story does not have to be concluded to attain a maximum score. Provided that there is no other reason to deduct, a story with an open ending or one that is part of a series warrants full marks. Only if the story ends abruptly and unnaturally can this be considered a flaw.

Multi-player scenarios are judged differently. With these, a story is purely optional, and this category is used as a rating for the quality of the instructions. If the instructions are overly complex, misspelled, inadequately delivered or uncomprehensive, points should be deducted. Points should not be deducted from a multiplayer scenario for absence of a storyline.

Official Reviewers

Exceptionally dedicated reviewers can be made Official Reviewers if they so wish. Any reviewer who has written 20 reviews can be promoted to this position if they request this. Once appointed, a reviewer will remain an official reviewer indefinitely and need not do anything to keep their title. Requests for Official Reviewer status should be sent to Leif Ericson.

Links

The Blacksmith -- Age of Kings Downloads and File Reviews

Official Review Guidelines Discussion Topic -- Discussion about evolving and changing the existing Review Guidelines

Review Requests, about Reviewing and Tutorials III -- Review requests and discussions

Archived Guidelines

Credits

"Story/Objectives" drafted by Julius999, intro by Kor, creativity by AnastasiaKafka.

"Official Reviewers" drafted by matty12345, adjustments made by Kor.

Based on the original guidelines created by Angel Spineman.