Getting paged data from the Teamwork API using vue-resource and Promises
Teamwork Projects is an online project management app. Here at DesignHammer we are in the process of moving to Teamwork Projects. I am leading our transition and it’s a big job making sure everything goes smoothly. A key part of my job is identifying areas where standard Teamwork functionality doesn’t do exactly what we need and, if necessary, building tools to help meet that need.
Getting data from Teamwork
Our tools are built as a hosted Vue.js application. The app provides various reports and views which are not available in Teamwork itself but where the underlying data exists and is available via the Teamwork API.
Generating these reports typically requires fetching all of the available data for a given resource (e.g. all active projects, all time records in a given project, etc.). The Teamwork API provides these as REST-based URLs which are easy to fetch using vue-resource.
A basic request method in our Teamwork client looks like:
Vue.http.get() returns a Promise for the requested resource. We chain then() onto the request to pull out the project array from the response data. Code which calls the Teamwork client is therefore very simple:
However, some resources return a maximum number of records at one time (fetching all time records currently only returns 500 records per request). If there are more records in Teamwork for that query then the client needs to perform additional requests to get them all.
Client code can check the the response headers for information about how the data is broken up into pages. vue-resource provides these in headers.map in the response data. The main header we care about is X-Pages. If the value of this is 1 then we can just return the data from the first request; we got all of it.
If the value is greater than 1 then we need to perform additional requests. How we go about doing that in a clear and DRY way is an interesting question.
Supporting paged queries
Ideally, the code that is calling our Teamwork client shouldn’t have to care if the query is paged or not. Our UI is generally showing rolled up summaries of complete data sets, so we aren’t using paging for display. We can encapsulate all of the paging functionality within the Teamwork client as a reusable pagedGet() function and return full result sets from client methods.
Teamwork returns result data in a data type-specific property within the body of the response. The calling code needs to know the name of this property. We pass that to pagedGet() as recordKey from the resource specific method. We also check the X-Pages header and if there’s only one page we can return the results without firing off any additional requests.
If there are additional pages of data and we know exactly how many pages there are in total, we can fire off requests for all of the remaining pages simultaneously and then combine the results as the requests complete. Once all of the results are combined we return them.
We have added two parts to the pagedGet() method. First we loop through the remaining pages and add a new promise for the request to an array of promises. Vue.http.get accepts an options object as its second parameter. The params property can be an object with key-value pairs. Those key-value pairs are included in the request as GET parameters.
The second part executes all of the promises that were created simultaneously. Once all of them have been fulfilled we get results which is an array of result arrays. We get one result array for each page of data. All we need to do from there is combine them with our first page of data and return the full set.
The great part about this approach is that our calling code still looks exactly the same,
all of the resource-specific information is captured in resource-specific methods,
and the request and paging logic is in a single reusable method, pagedGet().
Supporting additional parameters
Many resources in the Teamwork API support additional parameters beyond just page. We can adjust pagedGet() to support these alongside the paging functionality:
We allow the caller to pass a params object and use Object.assign to override the page property. We can then provide custom params in our calling code: