1) Not abiding to the guidelines. Many reviewers do not seem to have read/remembered the reviewing tutorial. I see lots of reviews that penalize the reviewer for empty space on a part of the map that cannot be seen or that say a scenario is too long/short. I don't know how this should be fixed, but maybe have several "master reviewers" that read all submitted reviews and check if they abide to the guidelines. These people could look at many reviews per day because they only have to read one page for each instead of playing an entire campaign. Definitely a review that says, for example, "he used trees, cliffs and deserts" and gives 5 for map design just because of that should be considered dubious and should not get in. Also reviews that talk about the designer ("he's a scenario design god" or "some newbie I never heard of") should be penalized since this is not one of the categories required.
2) Story/Instructions score for Multiplayer. It says in the tutorial that the most a map can get is 3 if it has no story. However, it is much more difficult to have a story on a multiplayer map, and some maps, like castle blood or world map, do not need a story. The scoring should be different here depending on the type of map a designer made. For example, a multiplayer RPG had better have a story, but hitman doesn't need one. Also, keep in mind the limitations of the map editor for multiplayer: there are no AIs, no click-talk techniques, etc. So don't penalize for lack of this - just as you don't penalize people for not including AVIs because they have no AVI program. Basically, there are many multiplayer scenarios that should get a 5.0, but since the review system is mostly designed for single player, they never get that score.
3) Mods in Best of AoK. If you look at all the downloads rated 5.0, you'll find 5 campaigns, 4 single player scenarios, one random map We should try to fix these things and improve the system. What do you think?