Ten days ago the oft-cantankerous YouTuber VideoGameDunkey, or just Dunkey, posted a video lambasting the profession of game critics (seemingly only of the written variety), and the game critics got together and reacted about as anyone would expect – with an equal amount of rancor and responded in kind discussing how he got it wrong.
One of the most-criticized segments of his video was his assertion that game critics rush reviews with the sole purpose of being the first on sites like Metacritic so their site can get an influx of traffic. Again, games critics assured everyone that Dunkey was mistaken.
Now, ten days after this whole kerfuffle, video game critics have gone and proved Dunkey right by once again rushing reviews of Nintendo’s Splatoon 2.
As someone with several years in this industry, and as someone who knows what the review process was like for Splatoon 2, I know that reviews coming out as soon as the embargo broke, as happened with almost every major outlet today, will not have adequate information to produce a fair review. In fact, I even read a review with a disclaimer that said their review should be considered provisional, which of course raises the obvious question: why release it as a review if not for the sake of getting it out early?
Games journalists use in-progress reviews all the time for these sorts of situations, a sort of placeholder for an eventual scored review that won’t immediately post a score to Metacritic but will shortly afterwards. If you already put a disclaimer on the review, it would follow that you shouldn’t treat it as a full review.
The truth is, too, that even the system of in-progress review followed shortly after a full review isn’t perfect, as the pressure to release a full review is constantly mounting, which could cause critics to release a review before its really ready. Before continuing, I want to be clear that I’m not coming at this from the perspective of someone upset about review scores – I haven’t even looked at Splatoon 2‘s Metacritic page.
In the case of Friday the 13th: The Game, it was especially clear to me that many critics didn’t play the game enough before releasing a review, including cases where critics got objective facts wrong. One person wrote that Jason doesn’t immediately know where the phone is, even though Jason starts with a fully equipped map with the phone house clearly labeled. Another wrote that there’s no way to meaningfully team up in Friday the 13th, even though that’s both generally and specifically untrue, as killing Jason requires at least two people to do two different things.
Ultimately that’s the rub when reviewing multiplayer games: you can easily miss things that you would find out if you just played it for a bit longer before reviewing. But, unlike most multiplayer titles that release, Splatoon 2 had pre-release multiplayer sessions in which journalists could participate. There were three days for Turf War and two days for the game’s new Salmon Run mode. There were also a few sessions for ranked battle, but that requires you to be level 10, which you won’t be for quite a few hours.
Of course, the issue there is obvious. What if you can’t make one of those times, or you’re not a high enough level when you do? Well you can try your luck in finding a different match with the other journalists who didn’t make those times, and that can take much longer, meaning the amount of matches you’ll get to play is much less. Especially if you’re releasing a scored review where you had only two set aside times (which you may have missed) to play Splatoon 2‘s brand new game mode, that’s going to be considerably inappropriate.
This may not be all that big of a deal to dedicated Switch players, especially because Splatoon 2 had a free one-day demo where any and all Nintendo Switch owners got to participate. This allowed players to make up their own minds. That said, the demo was only for one day, so if undecided players missed that opportunity all they will have to go off of is the reviews. If the reviews are artificially high because critics rushed through and missed the more contentious bits, gamers may buy it and be disappointed. If the reviews are artificially low because critics didn’t take enough time with it, gamers may not buy it when they would have otherwise enjoyed it. In either case, this is a disservice to the consumer when the only reward is a boost of traffic.
Now, I didn’t agree with everything Dunkey had to say in his now-infamous video – his statements about decentralized reviews can be solved simply by reading a byline and following that reviewer for consistency, for example – but his case about rushing reviews and improperly scoring them likely just got a huge boost.
