Python for Fantasy Football – APIs and JSON Data

Welcome to part 4 of the Python for Fantasy Football series! As always, if you missed any of the previous parts, go back and check those out first before reading any further. In part 3 I mentioned that your first port of call when looking to access data on a website should be the API (if there is one). I said last week that I wasn’t going to scrape the FPL site, but thanks to all the excellent feedback (keep it coming!) I have since found out that the site does in fact have an API. There is no documentation and the API isn’t publicly listed anywhere as far as I can tell, which makes me think that it’s not primarily intended for public use. However, there are already plenty of guides around showing people how to access it, so since we will be using JSON files later in the series I thought I would do a quick run-through of how to do so in Python. It goes without saying that this data should not be used commercially, as stated in the site terms and conditions.

Introduction to JSON

JSON (JavaScript Object Notation) is a text file format designed to facilitate the transmission of data from server to browser. Whilst initially intended to be used with JavaScript, there are libraries for creating and parsing JSON data in many of the most popular programming languages. The structure is similar to a nested Python dictionary, as you can see from the example below (via Wikipedia):

{
  "firstName": "John",
  "lastName": "Smith",
  "isAlive": true,
  "age": 27,
  "address": {
    "streetAddress": "21 2nd Street",
    "city": "New York",
    "state": "NY",
    "postalCode": "10021-3100"
  },
  "phoneNumbers": [
    {
      "type": "home",
      "number": "212 555-1234"
    },
    {
      "type": "office",
      "number": "646 555-4567"
    },
    {
      "type": "mobile",
      "number": "123 456-7890"
    }
  ],
  "children": [],
  "spouse": null
}

The FPL API has several JSON files that you can access, provided that you are logged in to the site (see here for an exhaustive list; note that you will need to be a league member or have authorisation to view some of these). I’ll go through a brief run-down of how to read JSON data into Python and access the pieces of information you want, showing you a few examples from the FPL API. To finish, I’ll look at player ownership in a mini-league to give an idea of what you can do with the data.

Loading JSON files into Python

To load JSON files into Python, we can use the ‘json’ library. We’ll also need to use ‘requests’ first to grab the data from the FPL API. The code below shows a standard function that you should be able to use to get any JSON file from the web, provided you have a link to it.

import pandas as pd
import json
import requests
from pandas.io.json import json_normalize

# Define a function to get info from the FPL API and save to the specified file_path
# It might be a good idea to navigate to the link in a browser to get an idea of what the data looks like
def get_json(file_path):
    r = requests.get('https://fantasy.premierleague.com/drf/bootstrap')
    jsonResponse = r.json()
    with open(file_path, 'w') as outfile:
        json.dump(jsonResponse, outfile)

# Run the function and choose where to save the json file
get_json('D:/Tom/Downloads/fpl.json')

# Open the json file and print a list of the keys
with open('D:/Tom/Downloads/fpl.json') as json_data:
    d = json.load(json_data)
    print(list(d.keys()))
['elements', 'total-players', 'player', 'element_types', 'watched', 'next-event', 'phases', 'stats', 'game-settings',
 'current-event', 'teams', 'stats_options', 'last-entry-event', 'entry', 'next_event_fixtures', 'events']

Parsing JSON

We can see that this file has several keys, each corresponding to a different subset of the data. Feel free to play around with each of these if you like, but I’ll give you a few examples of the most useful parts here. To get player information, we need to access the ‘elements’ key. We could write some code to parse this out manually, but luckily the ‘json_normalize’ function makes things a lot easier for us:

# See https://pandas.pydata.org/pandas-docs/stable/generated/pandas.io.json.json_normalize.html
df = json_normalize(d['elements'])
print('Columns:\n', list(df), '\n')
print('Dataframe Head:\n', df.head())

You can see that it’s not that difficult to get all the player info we would want from the FPL site! As for teams, that’s just as easy:

df = json_normalize(d['teams'])
print('Columns:\n', list(df), '\n')
print('Dataframe Head:\n', df.head())

If you’ve ever wondered how they get the fixture difficulty ratings, you might find the answer in there. In case you aren’t familiar with how bonus points are calculated, you can take a look at the game settings:

# The (not so) secret sauce (see https://www.premierleague.com/news/106533)
df = json_normalize(d['game-settings'])
print('Columns:\n', list(df), '\n')

As well as the above, there are other links we can access. Here is an example of how to get the standings for a particular league – in this case the overall standings. We could use the function from earlier, but to avoid having to download the JSON file and save it to disk we can use a much shorter piece of code:

d = json.loads(requests.get('https://fantasy.premierleague.com/drf/leagues-classic-standings/313').text)
print('Keys:\n', list(d.keys()), '\n')
df = json_normalize(d['standings'])
print('Columns:\n', list(df), '\n')
print('Dataframe Head:\n', df.head())

Hmm, not quite what we were after; the values we want are all inside the ‘results’ column. Instead, we can access the results key directly in json_normalize to parse out the league standings:

# See json_normalize documentation for further info
df = json_normalize(d['standings'], 'results')
print('Columns:\n', list(df), '\n')
print('Dataframe Head:\n', df.head())

As far as team names go this is a bit of a poor effort in my opinion! You can also see the data for your own team if you like by using the code below:

d = json.loads(requests.get('https://fantasy.premierleague.com/drf/entry/5746258').text)
print('Keys:\n', list(d.keys()), '\n')
df = json_normalize(d['entry'])
print('Columns:\n', list(df), '\n')
print('Dataframe Head:\n', df.head())

You will notice that my team is terrible! I actually didn’t play FPL this season, so this is just a random team I made to allow me to write this article. When FantasyBet were offering a real-money version with only 3808 entries it was hard to get excited for the free game, especially when the prize-pool looks like this:

If you missed it, don’t worry! FantasyBet also offer daily fantasy games where you can pick a team for just one gameweek and compete for cash prizes. This week there is a tournament with a guaranteed prize-pool of £3000, so make sure to check that out if you haven’t already!

FPL Mini-League Ownership

Accessing the data is just the tip of the iceberg, and people have already made some pretty cool Python scripts to do all sorts of things using the API. For example, see here for a script that allows you to find out captain choices and overall player ownership for a specific gameweek in any mini-league! I actually edited the script slightly to clean up the player names and automatically calculate ownership percentages; you can find it on my GitHub here. You can run this script from the command line as suggested in the readme, or alternatively you can run a script inside a Jupyter notebook using the method below:

# Note that this code might not work if you aren't in this league... I suggest replacing the league ID with one of your own mini-leagues
# If you pick a league with lots of players, this will take a while!
import os
filepath='C:/Users/Tom/Python/36/FantasyPremierLeague-Api.py-python3'
# Change directory to the filepath
os.chdir(filepath)
# Run Python script with specified paramaters
%run playersPickedInLeague.py --league 14226 --gameweek 9 --type classic

The script will create two csv files in the same directory, one with captain ownership and another with overall player ownership for the particular gameweek. As a side note, if you haven’t already checked out the FISO forums, they are a great resource for fantasy football, particularly FPL. If you load the files you will see a breakdown of ownership percentages for the FISO forum mini-league in gameweek 9:

# Note that you will need to update the league id and gw number here to match your input paramaters
captain = pd.read_csv(filepath + '/captain_14226_gw9.csv')
picked = pd.read_csv(filepath + '/picked_14226_gw9.csv')

print('The FISO forum mini-league captains, GW9:\n', captain.head(), '\n')
print('The FISO forum mini-league player ownership, GW9:\n', picked.head())

# Plot the top 20 owned players
sns.set(style='whitegrid')
sns.set_color_codes('muted')
f, ax = plt.subplots(figsize=(12, 10))
sns.barplot(x='percent', y='name', data=picked.head(20), color='b')
ax.set(ylabel='', xlabel='FISO mini-league ownership percentage (top 20)')
sns.despine(left=True, bottom=True)

Conclusion

That’s it for now! Hopefully the last part of the article has given you some ideas of the things you could create using the FPL API. Make sure to share any tools or visualisations in the comments here, or on Twitter @FantasyFutopia or @tom_whelan! The next part might take a little bit longer to write, but hopefully the wait will be worth it! I’ll be publishing the first of several articles on machine learning, looking at how you can create your own xG models in Python.

If you enjoyed the article, please don’t forget to share it on social media – it really helps!

Leave a Reply

Your email address will not be published. Required fields are marked *