Confession: I’m a bit of a self-help junky.

Not in the flaky, dime-store psychology, “I’m Okay You’re Okay” sense.

Rather, I tend to check Lifehacker at least once a day to see if they’re highlighting some spiffy new productivity engine that will miraculously make me into a better person. I inhale everything Malcolm Gladwell writes. I read books like The Power of Habit and Nerve, literature that purports to detail the latest discoveries in neurobiology when it in fact offers behavioural shortcuts—loosely based on the latest science—to help overcome our all-too human frailties.

Part of the magic of this kind of science-y self-help is that it convinces us we’re learning something new while essentially telling us—in sterile, academic language—what we already know. Like, for example, if you burn more calories than you consume, you will probably lose weight. If you better organize your time, you will be more productive.

This is essentially the same trick in a lot of recent research in football analytics—a good portion of it essentially provides data that confirms what we already know about conventional football tactics.

Not that this is necessarily a bad thing. When I first spoke with some major figures in the soccer analytics business, they were quick to temper hopes for a counterintuitive, sabermetrics-like revolution in how we understand soccer arising from the torrent of available data.

Rather in its present early stages, metrics would simply make statistically plain what many already knew about the game in its present state. Like how ball retention yields a significant competitive advantage over the long term, or how sustained possession via number of successful passes in the final third is more likely to lead to a goal than sustained possession in your half of the pitch.

This is a problem however in much of the analyses of Manchester City’s recent full-season Opta data dump.

The reason is some analytics writers are attempting to use statistical data from the 2011-12 Premier League season to reach broad conclusions about best practices in football tactics across all possible worlds.

This is not a trivial point. It’s much less clear for example in Major League Soccer that ball retention and possession is in of itself key to the success of a winning side. It may be the case that possession for its own sake is a better strategy in leagues with elite, technically-gifted players, but that’s not the same thing as saying superior ball-retention always wins football matches, no matter the league or the era or the in-vogue tactical thinking.

In essence, the MCFC data dump can only tell us what strategies were broadly successful for English clubs during the 2011-12 Premier League season. Many of the table leaders for example were managed by coaches who favoured a possession-based approach. Other teams presumably prepared and responded to this specific approach, or failed to, which in turn further affects the available data.

In other words, it’s not obvious where the conscious strategy of a good manager overseeing elite players ends and when universally effective, non-intuitive soccer metrics begin. Or even if the latter exists.

Moreover, analytics researchers who attempt to deduce how a specific tactical approach skewed the data risk a confirmation bias. For example, how can one deduce from the data the difference between ball retention as intentional tactic and ball retention as accident of employing superior players?

It certainly doesn’t present a crisis for soccer analytics if the data doesn’t yield universal truths about best practices in football. But it does limit its use. In the MCFC Opta case, the analytics only tell us in specific terms what winning teams in the Premier League did last season.

The allure of the data however has led a few writers to overreach. Recently, one analytics blogger finished an analysis of pass-completion and its relation to chance percentage with this claim:

By pooling such factors as passing sequence, final 3rd possession and the ability to take on and beat opponents we can highlight where and why scoring opportunities arise, prioritize attainable recruitment of new players and formulate tactics that play to a team’s strength and ability.

But would this present such a radical departure from current best practices in scouting and player recruitment? Generally football favours players able to retain the ball, beat their markers and pass intelligently.

It’s also the case that teams already look for ways to break up passing sequences, win back possession in the final third, and employ defenders who will win take-ons.

Soccer analytics is vital in our understanding in football, but writers should be wary in their attempt to use a single year of data from a single league to make broad claims about attacking strategy in soccer, full stop.