Archived Guidelines

These guidelines are currently outdated and continue to be retained for archive purposes only; the latest guidelines should be used while reviewing.

Introduction

This article will provide a description of how to write quality reviews for Age of Kings Heaven that are scored consistently between reviewers and are helpful to both the file's creator as well as the potential downloader.

I (Angel SpineMan) created the reviewing system almost three years ago and it was first used for the original Age of Empires. It has been tried and tested and I still believe that the 5 category system of Playability, Balance, Creativity, Map Design, and Story/Instructions, covers all the areas that a truly great scenario should excel in. Each category is scored on a range from 1 to 5 with 5 being the best score and 1 being the worst. The total score is achieved by averaging each category. The system is very effective if it is used correctly.

At the beginning, my sole responsibility at Heaven was writing reviews. As such, the reviews were very consistent since I wrote them all. However, over time, I took on more and more responsibilities at Heaven and I no longer had the time to write very many reviews myself. So, we got a small team of reviewers together and I explained the system to them in detail. That worked well for awhile but over time, some of the original team left and new reviewers have been constantly added to the team to try and keep up with the immense number of scenarios and campaigns that need to be reviewed. Unfortunately, I have not able to give each of the new reviewers as much guidance as they needed and the reviews have started to become very inconsistent.

This inconsistency is very troubling since scores can vary widely depending on who is doing the review. That's not fair to the designer or to the downloader. I know that there will always be some variation in scores depending on the reviewer, but lately, the amount of variation is unacceptable and this document will detail what I expect each score to represent. As the Angel in charge of reviewing, it's my job to ensure that reviews remain a valuable tool for both designers and downloaders.

General Guidelines

The first thing to always keep in mind is that reviews serve a double purpose. They are written for both the scenario designer and for the site visitor who is considering downloading the file. As such, reviews need to praise the designer for things that are done well and point out areas where the designer can improve. Reviews also need to provide enough information about the scenario so that potential downloaders will know if the scenario matches their interests. Obviously, don't give away the plot or reveal secrets that should be discovered while playing, but let the downloader know what the scenario is about, if it's mainly fighting, or mainly rpg, or mainly puzzles or a mix of everything. Whatever the case, a downloader should know what to expect from a scenario after reading a review.

Try as hard as possible to avoid vague statements in reviews. Make sure that your review answers more questions than it raises. Don't ever say something like "The first part of the 2nd scenario was good" or "the part with the wolf could be improved" without providing further explanation. Try to always include an example from the scenario to back up any points that you make. If you are pointing out something to the designer that you feel could be improved, try to provide some ideas that the author could build on. Do as much as you can to help the designer improve his work.

All scenarios have good aspects and bad aspects. Try to always say at least one good thing about any scenario you review and never, NEVER insult a designer. They might not be as good of a designer as you but even if you are handing out a score of 1.0, you still should never insult the designer. Be honest about the scenario but make every effort to encourage the designer to do better next time.

Always spell check your reviews. I've seen far too many reviews that take points off a rating because of poor spelling yet the review itself is full of errors. Don't embarrass yourself - spell check your work.

Lastly, the review should contain a short explanation of why you scored each category the way you did. This does not need to be lengthy, sometimes a sentence is enough but other times, a paragraph for each category is needed.

Category Scoring Guidelines

Before I detail what I expect for each category, there are some general scoring guidelines to make note of. All of the categories are subjective, some more than others, but try to be as consistent as you can with your own scoring. Also, take special note of a few things that should NOT affect the score of a scenario. These things should be noted in the review, but they should not affect the rating scores in any way.

First is the length of a scenario, or how many scenarios are included in a campaign. There is no rule that says a scenario must last more than 15 minutes or that a campaign must include at least 3 scenarios. The scores should only reflect how good the scenario was while it was being played. A great 5 minutes should score much higher than a mediocre 2 hours. No reduction in score should be made based on the length of a scenario.

Second is the number of triggers in a scenario. This is simply irrelevant to how good or bad a scenario scores. Lots of designers like to brag about how many triggers their scenario has but if the triggers are poorly constructed and don't contribute to gameplay, they might as well not have any triggers at all. A scenario does not need to have very many triggers to be a lot of fun to play. Scores should be based on playing the scenario, not opening the designer and counting the triggers.

Lastly, a scenario should not be penalized for not including special extras like music files or custom ai files. These extra items are great if used effectively and certainly can boost a score but a scenario should not be rated poorly just because of a lack of extras. A scenario should still be able to achieve a score of 5.0 even without using special extra files. The AoK design tool is so rich with extras already that a designer should not be required to use custom files if they can achieve their design goals using what is already built into the design tool.

Category Scoring

Playability

Playability is probably the most subjective element of the scoring. It is simply a gauge of how much fun you had playing this particular scenario. One thing to look out for when reviewing is to only play scenarios that use a style you enjoy. For example, if you hate playing RPG scenarios, don't try to review one since you are bound to not enjoy the scenario. Try to keep within styles that you enjoy.

There really is no specific criteria on how a score is given in Playability but there are quite a few things that can effect playability in a negative manner. Trigger bugs, victory condition bugs and any other playability-destroying bugs obviously can ruin a scenario's playability. Lag is another playability issue that a scenario can be marked down for. If a player is ever confused about the next goal to accomplish, that's a playability problem. If a player can complete an objective in a way that the author obviously did not intend to be possible (i.e. there's a hole in a wall that allows the player to skip half the scenario), that's a playability problem. Anything that adversely affects your enjoyment of a scenario can be deducted from the Playability score.

Balance

Balance is also somewhat subjective since each player is a different skill level and what might be perfectly balanced for one player, might be way too easy or way too hard for another. As a reviewer, you must take your own skill level into account when giving a balance score. A perfectly balanced scenario should provide a challenge for a veteran player. Most people who are downloading scenarios from the internet have at least played through the campaigns included with the game and have a good knowledge of the game.

Most perfectly balanced scenarios should not be able to be completed without the player losing a few times. If a player is able to complete the entire scenario the first time, the scenario is probably too easy. On the other hand, a player should not need to reload 15 times to get by a certain part of a scenario. That is frustrating and the scenario is probably way too difficult. The ideal scenario balance happens when a player gets stuck, but he knows that it's possible to complete the objective if only he did something a little differently. A player should not win by luck, the scenario should be constructed so that a player can learn from mistakes and use his skill to complete the objective.

One important item to note about scoring the balance category for scenarios where no fighting takes place, such as cut-scene scenarios, some puzzle scenarios and some rpg style scenarios, is that just because the player cannot die in such scenarios, that doesn't mean the scenario isn't balanced. You also need to take the author's original intent into account, giving the author some benefit of the doubt. If the author never intended the player to face a struggle to survive, then there's no reason to knock down the balance score if there isn't any fighting. So keep in mind that you do need to take the intent and goals of the scenario into account when scoring the balance category, especially for those scenarios where fighting is not included.

Multi-player scenarios are reviewed a bit differently in terms of balance. Each human player should start out in an equal position with equal starting resources and equal starting units. Obviously, the players don't have to match exactly, but they should be balanced. The map should also be examined to determine if all players have access to the same amounts of on-map resources. There are a lot of creative ways that map designers can use to make each player different, yet still balanced. If you choose to review multi-player scenarios, it's your job to ensure that each starting position is balanced with every other starting position.

Creativity

This area is probably second in subjectivity behind playability. Creativity is found in all aspects of a scenario, from trigger tricks, to map design, to the story, to what units a player is given, to the objectives, to sounds used, etc... Every aspect of a scenario factors into creativity. One thing to be careful for is not to knock points off of creativity if the designer uses a trick you've seen used in another scenario. There's nothing wrong with using the same trick that someone else used and no reason to deduct points because of that.

Probably the biggest creativity factors are the starting position and the victory conditions. For example, any scenario that starts with a TC and three villagers with a conquest victory condition is simply not very creative. The farther a player gets from a random-style scenario, the better the creativity score.

Map Design

Map design is one of the few categories that's very easy to define and give a rating to. I have pretty clear-cut rules on how map design is scored and this is how it should work. A random map is a 3. All a designer needs to do to score a 3 is to use a generated random map. Random maps look good, they function well and there's nothing wrong with using a random map in a scenario, but it's just average. From that basis, it's easy to figure out where scores of 1, 2, 4 and 5 come from.

A rating of 1 is for a pathetic map... these usually consist of large blank areas with lots of square areas and straight lines. These maps look completely unrealistic and are quite unattractive. A rating of 2 is somewhere between a pathetic map and a random map.

A rating of 5 is for an outstanding map with lots of special details and concentrated effort to make the map much better than a random map could possibly provide. Obviously, a rating of 4 is given for maps that are slightly better than a random.

One final note on score map design... only the portion of the map that can be seen during play should be scored. If there are large empty areas that a player never sees, that should not affect the map design rating.

Story/Instructions

This is another pretty clear-cut category. If there is no story or instructions, the score is easy... it's a 1. If there are instructions but no story, the max score is a 3. If there is any story at all, the rating goes up to a 4 and if the story is really good, the rating can be a 5. If the instructions are wrong, misleading or confusing, the rating goes down. Also, keep in mind that in Age of Kings, the instructions and the story goes far beyond the pre-scenario instruction screen. Often the story is continued throughout the scenario by using trigger events to move the story along. Also, since objectives can change in the middle of a scenario, the quality of the instructions must be judged throughout the playing of the scenario.

Some other guidelines on scoring this category: An introductory bitmap is a nice touch and a good image can often raise the score, however, an introductory bitmap is not required to score a 5. It certainly helps, but it's not an absolute requirement. Hints and History can also be judged here... these two areas are not required, but they can also help boost a scenario's score. While a bitmap, hints and history are not required, it would be difficult to give a rating of 5 if all three areas are missing. The rating should not be effected based on whether the story is fictional or historical. It doesn't make a difference as long as there's a story that draws the player into the scenario.

The last item that factors into the rating of the story and instructions is grammar and spelling. A designer should be diligent in this area of his scenario since it's very easy to copy the text into a word processor and spell check the instructions. There's no excuse for having spelling errors in a scenario... it simply shows a lack of effort on the part of the designer. The only exception I make is for designers whose primary language is not english... I am usually quite a bit more lenient with them.

Final Thoughts

The above instructions are specifically for writing scenario and campaign reviews. However, we are now allowing reviews of all file types available for download so you can rate and write reviews on Mods, Random Maps, AI Files, Utilities and Recorded Games! However, these reviews will not use a 5 category system, instead you will give a single rating to the file. When you write your review, simply include enough information to explain why you gave the rating that you did.

Now that you've made it all the way through this document, you are ready to write reviews! If you have any further questions about what is expected of you, please direct them to Angel SpineMan. Below, you will see the links that you can use to submit reviews. The review form will send an email to me so I can check your review to see if it follows the instructions given above and provides adequate information about the scenario. If the review passes my inspection, it will be posted on the site. Good luck and thanks for your interest in writing reviews!