Problems in Game Journalism: 1up Edition

Here’s an easy quiz for you… All of the quotes below come from SSX Blur reviews (there’s more after the jump). Which of the following quotes came from 1up’s “Average” (according to them) 5.5 review, and which came from IGN’s “Impressive” (according to them) 8.4 review?

1) “Assigned to the Wii-mote, the theory is that the positioning of the remote translates directly to how your board moves. In practice, though, it’s spotty.”

2) “Prewinding a spin before a jump, then deftly moving from trick to trick, quickly linking spins and grabs…creating something as you’re airborne — which is usually second nature to a seasoned SSX player — is now much more convoluted.”

3) “Thankfully, you can set up in the control options to completely eliminate motion-based movement. If you’re not convinced, try this test: Run the first Slalom with the Nunchuk motion controls, then run it again with just the Nunchuk analog stick. If this is a concession from EA that not everybody’s going to “get” it, then we’ll gladly take it.”

4) “Even performing übertricks is a frustrating and unreliable endeavor — you’re required to “draw” overly complex lines in midair, but the Wii hardly ever recognizes what you’re doing. The “second-nature” feeling isn’t something that’s possible with the Wii Remote — the shoulder and face buttons of a traditional controller provide a quicker and more direct interface with a trick system that’s designed for multiple grab and spin changes per big air. (Why EA didn’t support the Wii’s “classic controller” is an interesting oversight.)”

5) “Notably, the developers didn’t ship the game with many control schemes, confident that the one they have is the way to go. At least for the basic maneuvers, that’s a hard point to argue, as turning and pulling off tricks works very well — at times feeling even tighter than on previous SSX games. Things take on a bit more of a learning curve when you move into the special tricks, and for us started to click after we had spent about 20 minutes with the game.”

6) “Regardless, it’s clear that the game does an impressive job making it easy and fun to jump in for a few minutes at a time. The new turning controls feel great, and performing tricks is easier than we expected. This is one of the best early 2007 games for the Wii.”

7) “How does it feel? For the most part, really good– the combo of pitching the Nunchuk and moving its analog stick made turning much more precise than in older SSX games.”

8 ) “This editor has always had a problem racking up big points in SSX, but not in this installment. Not only did gettin’ tricky with the Wii Remote feel surprisingly natural, but it was also way more rewarding than, say, pressing a bunch of shoulder buttons on the PS2 controller.”

9) “How do Ubertricks feel? Fantastic. This feature works so well in a game like this, we’d imagine that another racing/trick hybrid is pissed that it didn’t think of the mechanic first. Yes, Tony Hawk’s Downhill Jam, we’re talking about you….”

Ok, are you still with me? How do you think you did?

Well, I lied… and cheated a little. None of these quotes came from IGN’s review. Actually, every single one of these quotes are posted on 1up.com, by 1up or EGM editors (they are the same company and share some editors, to my understanding), but quotes 1-4 are from the review, and 5-9 are slightly modified quotes from their previews dated within the last 2 months, with 5-6 coming from 1up’s 1/19 preview, and 7-9 from EGM’s 1/3 preview. The “modifications” were only to hide the fact that these were previews rather than reviews, (if you’d like to see if I “cheated” too much, please feel free to check out the originals here and here).

What the hell happened? How did Ubertricks go from being “Fantastic” to “Frustrating and Unreliable”? How did the single control scheme go from a “hard point to argue” to complaining about the lack of classic controller support? The nunchuk movement from “really good” to “thankfully eliminateable?” The tricks from “surprisingly natural” and “way more rewarding than pressing a bunch of shoulder buttons on the PS2 controller” to “spotty” and not as “quick or direct” as the “shoulder and face buttons of a traditional controller?”

How did this go from “one of the best early 2007″ Wii games (with “depth” as their only reservation) to simply “average?” I suppose one could argue this last point, as there haven’t been too many Wii games that have come out this year yet, but I highly doubt 1up was thinking a “5.5″ would scream “Best Early 2007 Wii Game”, and you’ll notice that they never even mention the “depth” in the review. I suppose they were having too much trouble with the controls to really see if the game had any depth.

What bothers me isn’t that 1up didn’t like the controls and gave SSX Blur a bad review (and despite 1up’s claim that it’s an “average” review, their actual average, according to Gamerankings.com, is close to 7.0… so make no mistake, 5.5 is a sub-average, bad review). I haven’t played the game yet, and for all I know, it could be the biggest pile of crap ever made. In fact, maybe I’d even agree with every point that 1up makes, and it turns out that many of their complaints we actually predicted accurately in our pre-review. It is completely inconsequential.

What bothers me is the fact that their previews were so positive only a few weeks before it’s release, basically giving impressions about specific elements of the game that were completely opposite what their final review stated. I dunno about you, but after reading their previews, I was somehow under the impression that the game was great fun and felt pretty natural after a learning curve of about 20 minutes.

So what changed? I’m certainly not suggesting that reviews need to match up exactly with the expectations set by previews. Certainly, there are probably countless examples of games starting off strong and then sputtering out towards the end, or something potentially being fun for only a short time, but then gets boring rather quickly. But that doesn’t appear to be the case with this example. To fully accept the validity of both their previews and review, you have to believe that something was either fundamentally broken between their playtest and the review, or somehow hidden from them during their initial time with the game. Since neither of these is very likely (given that their biggest complain with the game is its control), it definitely seems like something is fishy with their preview.

But, it’s a preview, right? They’re supposed to be “puff-pieces” essentially, right? Well, the supposedly “new” trend in gaming previews is to be, believe it or not… honest. Absurd, I know… but IGN made the promise either on the Wii channel or in Matt Casamassina’s blog (I can’t find it at the moment, but I remember it… if someone else can find it, let me know) and 1up publishes it above their previews. It reads:

“The 1UP Network previews games with the philosophy that people want to hear our honest opinions on titles before they are released. If a game looks really promising, we’ll pass on our excitement. But if a game needs work, we’ll let you know.”

Apparently they feel they need to post that because, otherwise, people won’t believe that they are actually impartial when they preview games. This is certainly more of a widespread problem than just 1up, but perhaps 1up (and others) fear (perhaps even realistically) that big companies, like EA, the publisher of SSX Blur, would withhold information and pre-release play-testing of their games if they speak poorly of their games early on. After reading their previews and seeing their final review of the same game, I just can’t imagine how anyone would get that impression.

Game journalists have had a tough time gaining credibility as real, objective “journalists.” Having these kinds of discrepancies and contradictions certainly isn’t helping their case.