We are Gearing up for Release!

Hello! We are currently gearing up for the release of a brand new API version. After more than 3 years since the release of v2 we felt an upgrade was in order and we have worked hard to try and improve the experience of using the API for all of you!

The goals we set out to achieve with v3 was primarily to make it faster, more flexible and more consistent than v2. In order to achieve this we have made a few fundamental changes to the API and the data-structures.

The full API reference can be found here.

Data-Structures

In order to allow for more consistent response-times and straightforward caching implementations all data-structures in v3 are "static". The functionality provided by the with[] parameter have been removed and related resources are always included by id only. This means that for example the matches key in the series data-structure is no longer optional, but is always included as a list of match ids (rather than the full resource). This is what we refer to as series having one (or more) "edge(s)" to the match resource. If a resource has an edge to another resource you can also expect the corresponding endpoint resource/{id}/edge to work. In this case series/id/matches.

In certain cases this will require a couple more requests to the API. However, doing this has allowed us to greatly simplify and optimize the more heavily used endpoints and we have thus increased the default page size to 50, and greatly increased the allowed request rate. Read more about that later on.

In addition to making data-structures static we have also modified some of them. We have added a new region resource and all occurrences of country has been moved inside of a region. This will allow us to at least provide e.g a player's region, even if we don't yet know what country he or she is from. Moving forward the tournament resource will also contain information both about the region and country where the tournament is hosted as well as regions and countries of the tournaments participants.

The series and match resources have been updated with a participants key. This key replaces seed, score, forfeit, winner and rosters and instead moves that information into an object next to an edge to the roster. In order words, instead of the match resource looking something like the following

{
   ...
   "winner": 18730,
   "scores": {
      "127": 9,
      "18730": 16
   },
   "forfeit": {
      "127": false,
      "18730": false
   },
   "seeding": {
      "1": 127,
      "2": 18730
   }
   ...
}

It will now look like

"participants": [
   {
      "seed":1,
      "score":9,
      "forfeit":false,
      "roster":{"id":127},
      "winner":null,
      "stats":null // battle-royale kills and placements when applicable
   },
   {
      "seed":2,
      "score":16,
      "forfeit":false,
      "roster":{"id":18730},
      "winner":true,
      "stats":null  battle-royale kills and placements when applicable
   }
]

The roster resource has also been updated. Instead of combining a list of teams and a list of players it instead groups together one team and one lineup, where a lineup is a unique set of players. The idea with a lineup is that you can more easily track a specific set of players throughout their career playing together, regardless of which team they are playing for.

Authentication

The authentication model has been reworked in an effort to simplify your implementations. Every API user can now generate as many API keys as they want for one API client. Each of these keys will last until deactivated manually and will share things such as authorization level and request rates. You will be able to view your requests patterns and logs per API key so for those of you who want more insight into how your applications end up querying the API you can generate a key for each. You will also be able to more easily rotate you API keys by keeping both the new and the old active during the update process.

Since the keys last until manually deactivated this means that you will no longer need to go through the process of re-generating a token every hour.

Rate Limiting

The rate limiting algorithm for v3 has been changed compared to v2. We now use a more standard GCRA which brings the behaviour much closer to, for example, nginx rate-limiting. This also means that the same request pattern should always work regardless of the exact time when it starts.

Moving through the beta up to the release v3 will only limit request on a per-second basis instead of both per-second and per-minute. We expect that this will also be the case after release. For those of you who cannot easily predict you request patterns we will now also provide per-second burst.

Every response from the API will now also include the header X-RateLimit-Limit, X-RateLimit-Burst, X-RateLimit-Remaining and X-RateLimit-Reset which describes you limit, burst, requests remaining until depleted and in how many milliseconds the rate-limit will reset, respectively. Using these headers you can dynamically check how many requests are appropriate to send for the near future.

If you do happened to meet the rate limit the header Retry-After will be provided and will specify how many milliseconds you should wait until the next attempt.

Pagination

Endpoints returning a list of resources (such as /series) no longer wrap the response in pagination information. Instead, the list is the top-level object and will be less than the page size if no more pages exist.

The page parameter has been replaced with the parameters skip and take. take refers to how many resources you want and will default to 50, which is also it's maximum value. skip refers to how many resources you want to "skip" and will default to 0, thus yielding the first page. The second page is then received by specifying skip=50, and the third by specifying skip=100 and so on.

Making the page size more than 3 times larger is hopefully something that you can all appreciate!

Filtering

We are very happy to announce that we have greatly improved the experience of filtering the response. All endpoints that returns a list of resources now accepts the filter query parameter, rather than parameter such as is_over and games[]. The idea behind the filter is that by looking at the JSON keys in the returned object you can constrain them in an arbitrary way.

Before delving into the specifics I'd like to start with a few examples and compare them to the equivalent v2 request. In v2 the request to get tier 1 series in Dota 2 is /series?games[]=1&tiers[]=1. This will fetch all upcoming series as is_over will default to false. We know that the series resource has a key called "tier", a key called "end" (that will be null when the series is live or upcoming) and a key called "game", which itself has a key called "id". The corresponding v3 request is thus /series?filter=tier=1,end=null,game.id=1.

Unlike the v2 query parameters, the v3 filter supports multiple operators. Getting all series that were played in June of this year in v2 can be done with the query /series?starts_after=2019-06-01T00:00:00Z&starts_before=2019-06-30T23:59:59Z. In v3 we only have to know about the start JSON key instead of the two different query parameters. The corresponding query is /series?filter=start>2019-06-01T00:00:00Z,start<2019-06-30T23:59:59Z. In order to include the specified timestamp you would simply use the non-strict version of the operator, e.g <=.

In the above examples we can identify two different kind of "terms". We have identifiers and concrete values. An identifier is the name of the JSON key (such as "start" or "game.id") whereas the concrete value is what you want to constrain it to (e.g "1" or "2019-06-01T00:00:00Z"). Both of these come in two different "types", scalar and set. A scalar value is a single concrete value such as "2" or "2019-06-30T23:59:59Z" whereas a scalar identifier is an identifier that represents a single value, such as "start" and "game.id". The rule is that if the JSON object referred to by the identifier is not a list, the identifier is a scalar identifier. If it is a list, the identifier is a set identifier. Scalar identifier can be constrained using both scalar values and set values. A set value is simply a list of scalar values enclosed in curly braces. For example, {1,2}. When constraining a scalar identifier with a set value the allowed operators are what you would expect when operating on a set with a couple of notable exceptions that we'll get to.

An example is probably in order for some clarification. Given the v2 query above for tier 1 Dota 2 games we can extend it to also include CS:GO games like so: /series?games[]=1&games[]=5&tier=1. We are specifying that the game should either be "1" or "5". In other words we are saying that we want the game of each returned series to be a subset of the set {1,5}. The subset operator in the filters is < (since the ⊂ character is not readily available on all keyboards). The equivalent v3 query is then /series?filter=game.id<{1,5},tier=1.

Set identifiers require the value to also be a set and the operator to be a valid set operator. As mentioned, set identifiers are the JSON keys whose referred to value is a list. In the case of series there is a list of participants under the "participants" key. Inside each object in this list is a key called "score" which tells us how that particular roster scored. When we refer to the identifier participants.score we refer to a set identifier which means that it can only be constrained with a set value. For example we might want to get all series that ended with the score "5-3". In v2 this is not possible without iterating through all series and doing the check ourselves. In v3 we can do the query /series?filter=participants.score={5,3}. In order words, we want the set that consists of each participants score to be equal to the set {5,3}. Set operators are defined without consideration of order so specifying {3,5} will yield the same result.

Perhaps a more interesting example is when querying for all previous series between two arbitrary rosters. In v2 you would need to find a series they played against each other and query /series/{id}?with[]=roster_perf in order to get a list of series between the two rosters in the past year. Using the filter functionality of v3 and constraining participants.roster.id to be the set of the sought after rosters we can iterate through all head to head encounters. For example, if we want to see the head to head encounters between roster 38226 and roster 53347 (which are the rosters OG and Liquid brought to the grand finals on The International 2019, respectively) we can give the filter /series?filter=participants.roster.id={38226,5337}.

Available set operators differs depending on if the identifier is a scalar or a set. For scalar identifiers only the subset (<) and strict subset(<=) exists. They do the same thing and essentially mean "one of the provided values in the set". For set identifiers the supported operators are equality (= and ==), inequality (!=), superset and strict superset (>= and >) and a custom operator called "meaningful intersection" (^) which is defined as intersection(identifier,value)!={}. The superset operators can be used when you only care about some entries in the set. For example if you want all series where roster 38226 was a participant you can use the filter /series?filter=participants.roster.id>={38226}. Making this a strict superset ensures that there is at least one more participants other than 38226.

The "meaningful intersection" operator (^) is used when you want the resource referred to by the set identifier to contain at least one of the specified values. For example /series?participants.score^{1,2} will ensure that at least one of the participants has scored either 1 or 2. If you want the complete history of both roster 38226 and roster 53347, including their head to head encounters you would supply the filter /series?filter=participants.roster.id^{38226,53347}.

All of these <identifier><operator><value> triplets can be combined in arbitrary ways. When separated by a comma (,) both must be true (equivalent to the logical AND) whereas if they are separated by two pipes (||) either of them must be true. Unfortunately due to URL query parameters being separated by an ampersand & the AND separator couldn't be &&. While filter containing || might with preference be two separate requests they are nonetheless available. You can also groups these triplets together using good old parentheses. For example, getting all tier 1 grand final series together with all series without an end date can be done with /series?filter=(tier=1,bracket_position.part=GF)||end=null. This is of course a ridiculous query but it is supported nonetheless.

All the scalar identifiers that are part of the base resource (i.e. does not refer to a JSON object) can be used to determine the order of the result. This is specified as <identifier>-<asc|desc> in the order query parameter. Getting all upcoming series with the most recent one at lower indices can be done with the query /series?filter=start!=null,end=null,deleted_at=null&order=start-asc.

External Ids

The long wait is (more or less) over! Starting today you can use external ids to fetch resources from v3. Using the id_space query parameter you can tell us how to interpret the {id} you specify in the /games/{id}, /tournaments/{id}, /series/{id}, /matches/{id}, /players/{id} and /teams/{id} endpoints. You can find the list of supported id spaces in the API reference documentation. Someone with the kambi identifier for the grand finals of The International 2019 (1005737562) can just as well do /series/1005737562?id_space=kambi as they can do /series/204247 and receive identical responses in both cases. In the future we hope to be able to also provide the corresponding external ids for the resources in the response.

Of course if no id_space is specified the Abios space is used.

Current State and Moving Forward

At the moment the equivalent of the Series and Match packages are available in v3 except for all data-structures containing aggregated data (such as series summary and team stats). We are still in the process of reworking what the Play by Play package entails and how the data will be accessed. The ambitions for all aggregated data (which includes all current play by play endpoints except match summary) is that instead of the months_back parameter you will be able to specify a from parameter and a to parameter and we will aggregate the statistics for you based on the data we have between those two arbitrary points in time. Another goal we have with aggregated statistics is that, where applicable, you can specify not only two arbitrary points in time but also which two competitors the data is about. In other words, we are aiming to provide endpoints both for a more general aggregation and an aggregation concerning the head to head performance between two competitors.

Moving forward we are also very excited about making more and more datapoints available to you! Apart from re-designing the current Play by Play endpoints and data-structures we have also been working on completely new endpoints that exposes data we haven't found a way to make available in a sensible manner before. Among these are the much requested player positions in order to make heatmaps of the movement in a map, timelines over e.g net worth for each player and team in a Dota 2 game and the starting position and velocity of all thrown grenades in a match of CS:GO and many, many more. Keeping true to our promise of data completeness these datapoints will not only be available for all upcoming matches but also for all historical matches that currently have Play by Play data.