Do Polish journalists evaluate games differently than foreign ones?

When researching our clients’ games, we cooperate not only with gamers but also with Polish and foreign gaming journalists. Game developers and publishers often wonder to what extent Polish voices are representative of global opinions about a game and its eventual acceptance in the global game market. In light of this, we recently conducted an analysis to verify whether Polish gamers evaluate games differently than global players.  This time, we decided to take a closer look at the opinions of gaming journalists and determine whether the assessments of Polish experts differ from those of foreign experts.

One of the leading Try Evidence services are silent reviews (also known as mock reviews).   Gaming journalists with years of experience, selected according to their level of expertise in a particular genre or niche, are then given a build of the game (from any stage of production) and play it.  Then they write his review as if they did it for a gaming medium.  They also share their opinions on selected aspects of the game and often suggest interesting improvements.  Unsurprisingly, it’s very important to developers that the journalists selected for collaboration are, like the players selected for playtesting, “representative” of gaming experts in general.

As we did with gamers, we checked whether Polish journalists evaluate games in the same way as global journalists.  Or maybe because of the specific cultural conditions they live in, they judge games differently?

How did we study this?

We selected 24 culturally independent games and 21 culturally dependent games for analysis.  These are the titles we also considered in the first part of our study.  If you’re curious what this division is about, check out the article Do Poles evaluate games differently than the rest of the world?  where we present the exact criteria for the selection of games and their descriptions.

For each selected game, we looked at the ratings it was given in 8 most popular global gaming media and 8 most popular Polish gaming media.  We then extracted averages from them. In the next step, we compared them using a specialized statistical test. In this way, we checked whether the ratings of Polish journalists really differed from those of foreign journalists.

Unfortunately, in most cases we were unable to collect a set of 16 ratings – not all media outlets provided numerical ratings for the titles we selected.

Global media outlets were represented in our analysis by: gamespot.com, gamesradar.com, gamerant.com, ign.com, eurgamer.net, polygon.com, pcgamer.com, thegamer.com. Polish media outlet are: gry-online.pl, ppe.pl, CD-Action, gram.pl, eurogamer.pl, komputerswiat.pl, planetagracza.pl, ign.pl.

Results

As with the player studies, our analysis showed that

Polish journalists’ ratings generally do not differ significantly from those of foreign journalists, both in the case of culturally dependent and independent games.

The differences between Polish and foreign ratings (on a scale from 0 to 10 points) for culturally dependent games averaged 0.76 points, and for culturally independent games – 0.45 points on average.

It follows that

Polish gaming journalists experience and evaluate games very similarly to foreign journalists.

To be more precise: it is highly likely that Polish and foreign journalists do not differ at least in their general assessment of the games.  This is not to say that there won’t be a significant difference in any particular title. It can happen, but more as an exception rather than a consistent rule (as if Polish journalists are “different” from global ones).

Detailed results of our analysis are presented in the tables below**.

Culturally dependent games – rating tables

Polish journalists – game ratings

Global journalists – game ratings

 Culturally independent games – rating tables

Polish journalists – game ratings

Global journalists – game ratings

Games with the biggest difference in ratings – opinions of Polish and foreign journalists

Among the titles we researched, there were some with the difference between the average rating of Polish journalists and the average rating of foreign journalists of more than 1 point on a scale from 0 to 10.  Ergo on the face of it a lot. There were four culturally dependent games and three culturally independent games.  We decided to take a closer look at these titles and consider what these differences might result from.

1979: Revolution Black Friday

It’s a game that we categorized as culturally dependent.  It is a fairly niche, little known and ambitious production.  Its Polish rating is 2.5 points lower than the global rating.  It should be noted, however, that in the media outlet we selected for our analysis, we found only one Polish evaluation and two foreign ones. Therefore, such a large difference in ratings may be due to the small sample size – so it is difficult to infer from it actual differences in ratings.

The Polish review pointed out technical errors, uninteresting characters, poorly constructed script, unattractive models of less important characters and insignificant choices as the flaws of this production. As a plus, she singled out the filmmaking, the sound design and the subject matter, namely the 1979 Islamic Revolution that led to the overthrow of the dictatorship of Mohammad Reza Pahlavi in Iran.

Interestingly, two foreign reviewers also mentioned shortcomings in the visuals and illusory choices.  However, they paid much more attention to the story and relevant subject matter covered by the game.  One reviewer even mentions that for him, as an Iranian, the theme taken up by 1979: Revolution Black Friday made the game very refreshing, intelligent, and rich in educational value.

Thus, we suppose that the game theme itself was rated higher by people who were closer to it because of their cultural identity.

Kentucky Route Zero

For this title, we found two Polish and three global ratings, with the average Polish rating being 2.48 points lower than the global rating.  Again, this may be due to the small sample size, so it is difficult to talk about a general trend here.

Both Polish and foreign reviews noted the well-written dialogues and interesting form, praised the surrealist elements and cultural references, as well as the graphics and music.  The Polish reviewer, who rated Kentucky Route Zero lower than other journalists, criticized the game for its archaic mechanics, slow pacing, and plot being incompatible with the video game medium.

Paradise Lost

On average, the Polish reporters rated the game 1.25 points higher than the global ones, taking into account two Polish and two foreign ratings.  However, it is difficult to determine whether this is a general tendency in the case of this game and Polish journalists rate it better due to the fact that it deals with topics close to Poles, or whether the difference is due – again – to the small sample size.

Polish reviewers praised the attention to detail of the locations, atmosphere and story.  One foreign journalist, while appreciating the overall concept, pointed out the unused potential, the shallow story compared to other first-person adventures, and the lack of character development.

Raji: An Ancient Epic

In case of this quite niche production, we have only one Polish and one global evaluation (the Polish evaluation was 2 points lower).  The sample is too small to speak of any general trend.

Both reviews, Polish and foreign, praised the interesting and original subject matter of Hindu mythology.  The Polish reviewer also pointed out the game’s flaws like unsatisfying combat and too easy platformer elements.

Star Wars Jedi: Fallen Order

Star Wars Jedi: Fallen Order was a game rated one point higher on average by Poles.  Polish journalists paid attention mainly to the satisfying gameplay and combat, a well-designed open world and an interesting story.  They also liked the way the game expands the Star Wars universe.  Foreign journalists also praised these elements, but rated the game lower due to its technical shortcomings, problematic frame rate, overly distant checkpoints, and slow story development.

Control

On average, Polish journalists rated Control 1.12 points lower than foreign journalists.  In contrast to international ones, Polish experts more often pointed out the game’s ills such as uninteresting characters, unengaging plot, badly balanced difficulty level, and inadequately designed checkpoint system.

A Plague Tale: Innocence

Poles rated the game on average 1.28 points higher than foreign experts.   Interestingly, both Polish and foreign journalists pointed out practically the same flaws of the game – not very diverse gameplay, low difficulty level, one-dimensional antagonists, and shallow relationships between protagonists.  As for the positives, the indications were also similar.  They praised the atmosphere, the lore, and the well-written, exciting plot.

Summary

Polish gamers and journalists do not differ systematically, in an obvious way, from gamers and experts abroad in the way they evaluate games.  We suppose that cultural identity and country-specific conditions, do not significantly influence how we evaluate games in general.  There may, of course, be games uniquely immersed in a particular culture that will be judged dramatically differently by journalists and gamers from different countries, but there don’t seem to be any clear rules or regularities in this regard.  Differences are unlikely to occur.  Based on an analysis of reviews of individual titles, it can also be assumed that not only the numerical ratings of games given by Polish and foreign journalists are similar.  Polish and global experts in their reviews pointed out the same or very similar aspects of the reviewed games.

*Statistical significance is an assessment of whether a result occurred by chance.  If the result is statistically insignificant, it means that there is a high probability that it occurred by chance.  So – when we compare the results and it turns out that the differences obtained are statistically insignificant, we can say that the differences in the absolute values of the assessments between the studied groups do not have any practical significance.

**We could not find enough numerical ratings in the media outlets for the games Thea: The Awakening, Operencia: The Stolen Sun, and My Summer Car.  That’s why we couldn’t take those titles into consideration in our analysis, even though we had previously analyzed their reception among Polish and global players.

–

Want to cite this article? Do it in an elegant way:

Cieślak, K. (2021, 24 09). Do Polish journalists evaluate games differentlly than foreign ones?  https://tryevidence.com/blog/do-polish-journalists-evaluate-games-differently-than-foreign-ones/