Hi folks,
One of the things that I've been looking at recently for Rafe is our system for Reviews. With him, Dazler and Jizzo, we've come up with the following document for guidleines for reviewing, including style, content and how to score so we're consistent.
Please can you all read through and leave your comments. Almost everything is open for reasoned debate.
Come on guys lets have some comments. Agree / disagree?
Persoanlly I think there very good. I'll be posting my own comments in more detail later, but I think is very good and the first step towards generating a distinctive AAS style.
Rafe
Very entertaining and true, however there should be a clear point on editing someones review. For example, if I review something and a senior reviewer does edit something.....I as the initial reviewer must be informed of the changes...however what happens if I dont agree with the senior reviewer and the changes he made?
Two types of Edits would be from Senior Reviewers. The first would be (xxx - Rafe) comments, spelling changes, grammar, etc Those would stay as is.
The second, where the Reviewer and the 'Editor' disagree over an app is just tough. It's your own opinion. If you think it's crap, you say it's crap. If the Editor disagrees that much, then he can post his own review on the same app (nothing stopping us reviewing an app twice!)
Certainly I wouldn;t expect reviews to go out until reviewer and editor come agree (and agree to disagree if required).
The only point is over the Mega-App score. The reviewer must decided it's a 90%, PLUS two Senior Reviewers (we'll decide who they are shortly). If one Senior Reviewer says no, then it gets a maximum of 89%. This is to preserve the inegrity of the top award. THe 80-89% award should be for the run of the mill excellnet apps (of which there may not be that much).
Really good guidelines, and done in that special Ewan style 😃
Yeah, I agree on that.
How about some graphical "award" in addition to score.
I thought about that on the News section too, so that you can give some fast
indication of that app/game ??? something like thumbs up / down.
Nothing too specific, but just a first impression thingy 😃
Hope I made some sense :-?
[quote="JyriK"]Really good guidelines, and done in that special Ewan style
How about some graphical "award" in addition to score.
[/quote]
Yeah that makes sense.
Rafe is gonna do a "AAS Mega-Game / Mega-App" logo that 90%+ applications can carry on their website, and be starred as such on AAS.
There will also be a similar (but less shiny) award for AAS Recommended 80%-89%. Which will be awarded a bit more often, but not that much. Most apps should hit 65-75% if they are average apps. Not everythign is stunning.
Oh and we'll subtly downgrading existing reviews when the new scoring system goes public.
Actually if anyone wants to volunteer to do the logos that would be great. My graphics skills are crap.
axm......JyriK.....axm 😃
Ok, will be posting something really soon 😃
Just a quick message to say, I think it's a great idea....we need some sort of guidelines (norm) to follow otherwise as time goes on and we get more reviews up, we will get software companies compairing their apps to simlar ones, things could get a bit shirty 😉 :roll:
Where did this came from? Just left the office and I found new post..
Hm, this is good, and I agree with you all. It is just a matter of time until we will see between product name and NOKIA OK the AAS OK logo that JYRIK will design 😉
I was thinking to a way that can be more team-elaborated when it cames to % of pints given.
I think we should elaborate a AAS Scale for Sound, Graphics, Usability, Memory and so on.
How can I give 19/20 points to a game with good sound if a modest sound one got 20/20?
In rest, i am trying to restore my server backup data to a new laptop (my previous vas stolen 😞, and I will came up with 2 reviews that will knock the socks off on AAS :bday:
My feeling on the points is that spliting it to categories and adding it up gives shite scores. It's impossible to have an overall view. Dazler used to work like this, but gave up because it wasn't working.
If individual reviewers want to mark scores, great, but it's going to be hard enough to keep a level playing field with just the overall score,let alone the other fields too.
Here is a demo version that i was thinking, suggestions??
Does anybody like it? Or should I do something else?
Hey, that's great! 😃
That's pretty awesome - one question, can we have a look at a half size one, and maybe an icon sixed (48x48ish) one as well? I agree, this is top rate.
(We need Mega App as well for non-games)
Wait for a few minutes and those are ready 😃
Here these are:
And these:
comments?
If this is the trend may be like Platinum, gold and silver rated by aas..
The seal of excelence i think is too exagerated. We better come up with
"AAS reviewed"
"AAS Whatever Rated"
"AAS recommended"
"AAS Esential" :P
or on that cyrcle write www.allaboutsymbyan.com and in the center the game or APP with the ratings we decide to put in like:
Recommended
Essential
ehh, I am talking silly at 2.40 am. But this is write storming, isnt it?
Ewan,
When i tried to download your file, I got:
"General Error
The selected Attachment does not exist anymore
404 File Not Found: The File files/AAS_Review_Guide.doc does not exist."
Is this a legacy from the recent server problems?
Erm what was wrong with the reviews before? Seeing as I'm about to start reviewing again as I have manageable free time, I wanna know what was wrong with my reviews 😉. BTW MS Word, can we have it in something neutral like HTML or text.
It probably is a legacy problem in regards to downloading the Word file. I'm travelling just now can someone else repost the document please?
Switch., there was nothing essentially worng with the reviews before, but the guidelines means we can be consistent in terms of scoring, and have something down in print for new reviewers. We've already had problems from a certain company who compared the score one reviewer gave them because another reviewer marked a similar quality product from a competitor much higher.
They guidelines leave a huge scope for reviews, and only tie down a few things that can only help the impression of the site in the long run to both readers and companies.
Cheers Jyrik. As for the comment Ewan points out that different people scored the app differently, well it's not a surprise given that everyone has a different opinion.
Hey those are great - thanks JyriK.
I had been envisaging a rectangular button ot something, but those are excellent. Maybe have an in between size as well.
Rafe
[quote="SwitchBlade"]Cheers Jyrik. As for the comment Ewan points out that different people scored the app differently, well it's not a surprise given that everyone has a different opinion.[/quote]
Indeed, but at some point there needs to be a definitive AAS voice - you'll note that style and content of the text is left entirely up to you , the only thing we're asking is you follow some guidelines for the final score to make sure that we don't commit any majot boo boos.
Switch, can you think of a better way to do it (short of not having scores at all?)
Scores are an easy glance way to see how a product faired, no matter what system is used there is no way that you can have different people, or even the same person, reviewing apps to the exact same specs.
I think there's a need for it as a guideline to try and have some continuity between reviews.
Some sort of standard system on scores is a must to remain credible when reviewing stuff. There's no point reviewing stuff if somethign which is inferioir get a higher score than something else cause you have different reveiwer.
It may not be perfect, but thats something to be noted rather than reject the idea. Scores are rough glance guidelines, but they need to be off the same base.
rafe
If we have a post in admin section showing the review,
and then everyone that has used the app/game posts
their score/opinion, and then the total is based on that?
Of course the original reviewers score having a heavy influence.
Like if i give 67
and others think 76-83
then it would be something like 71/72
???? Thoughts ? Could this be feasible ?
Took the words out of my mouth J.
Of course the original reviewers score having a heavy influence.
Like if i give 67
and others think 76-83
then it would be something like 71/72
???? Thoughts ? Could this be feasible ?
I guess we can do this. Aren`t you all agree?
BTW. I have managed to rewrite the Review of one APP. Can I post it here in Word?
I fully support a set of guidelines for reviewers. Whatever the Admins agree on, once the system is created then all reviews should be to the defined standard. We cannot escape the fact that once Rafe or other Admin posts the review, it automatically gains an endorsement from AAS in the eyes of the reader. This is both a +ve reflection on the site and a responsibility that the team must (and does) take seriously.
The reviewer's freedom comes from how he interprets the guidelines, his assessment of the performance of the app against the guidelines and how he writes the piece. Anyone who feels that the guidelines are overly onerous or infringe his freedom of speech should not be posting reviews on behalf of AAS. They would, of course, be free to express their opinions in the forums or as comments posted against reviews by others. They would also have the right to propose amendments to the guidelines for review by Rafe et al.
Strong views perhaps and others may disagree - I do acknowledge their inherant right to be wrong!
Finally, IMHO, there is no reason why there should not be some basic categories to score against. Given that some categories will ot be appropriate for all apps, the reviewer could be left to selct those that apply and score accordingly.