Give us your input: The Future of Pre-Reviews
June 28th, 2007Back in February, Geoff and I were talking about games that were coming out and essentially how we could pretty much tell what kind of reviews a game would get before it even released. Thus sprang to life the idea for this website: Pre-Reviews, and part of the motivation for our website name.
We started with Sonic and the Secret Rings and came remarkably close to predicting its Gamerankings score, and moved onto just a few others, but just stopped working on them after awhile when it became apparent that most people who were finding the site didn’t seem to care about Pre-Reviews and instead were only commenting on our other posts. It was also a rather annoying process for us to write them, as even if we had a gut-feeling about what a game would garner, we would still try to research what it’s strengths and weaknesses would be before the game released and write them up. And of course, it became extremely apparent to us, particularly after the 1up SSX Blur “scandal” that previews would often have absolutely no bearing on a final review. As both Geoff and I have “real” jobs and have limited free time to post (let alone play the games that we like to post about), it just didn’t seem worth it to us to continue writing them.
Considering the original intent of the site, I’ve been trying to think of other ways that we may be able to re-introduce the feature that wouldn’t absorb so much of our time. So, after many months of no pre-reviews, I’m inviting you, our readers, to post your opinions on whether you’d like to see us return to these features in some way. Were they worthwhile reading? Did you enjoy our lengthy write-ups on them, or would something much shorter, say, a pro and con list, be better? Should we just abandon the concept entirely? Now is your chance to help shape the future of our site. What do you say?
Posted in Admin, Jeff |
June 28th, 2007 at 4:15 pm
Actually, one other reason I stopped being so enthusiastic about pre-reviews: simple math. Because we were using an average as our benchmark, we weren’t accounting for two major problems:
1.) Even great games can have a much lower average because only a few bad reviews can drop the score considerably. As a result, you reflect not the consensus but the extremes - if God of War gets 20 “10″ ratings and 1 “8″, its average is a 9.9. That one “8″ drops the average a tenth of a point… and variation is usually more extreme in real reviews. So we were often notably off on our averages, even though we were pretty close to the consensus view.
2.) Different sites use their scores very differently. 1up, which tends to grade on the lower end of many of its reviews, isn’t really comparable to IGN, which is often overly optimistic. Similarly, some sites use a 10-point scale (divided into tenths, 0.5, or whole increments), others use 5-points, etc. So the average again isn’t very reflective of an overall view of quality.
If we were to resume these pre-reviews, perhaps spotlighting only the major titles, we’d need to account for these caveats… maybe with a hand-picked group of reviewers.
June 28th, 2007 at 8:05 pm
I don’t know if it makes any sense or not but I’ll give it a shot.
I enjoyed your prereviews, but I didn’t really miss them when they were gone.
Does that make sense?
In all honesty, your opinions and writing are interesting enough and your angle on topics unique enough that you don’t need to do more than basic coverage to make this site worth visiting daily. I would say, in keeping with the theme of the site, maybe pick 1 game per month and do a pre-review, but don’t burder yourselves too much. Like you said, you have jobs, you are gamers, don’t sweat something that requires so much time to put out there. And in the end, pre-reviews are the prediction of reviews is a bit trivial for the labor it requires.
I think the site is great, just keep bringing your insight and I’ll keep bugging you with my dumb ass comments.