tweeting-stackexchange-with-spring-social-part-1
StackExchange REST Client with Spring and RestTemplate
This article will cover a quick side-project – a bot to automatically
tweet Top Questions from the various Q&A StackExchange sites, such as
StackOverflow,
ServerFault,
SuperUser, etc. We will build a simple client
for the StackExchange API and then
we’ll setup the interaction with the Twitter API using
Spring Social – this first
part will focus on the StackExchange Client only.
The initial purpose of this implementation is not to be a full fledged
Client for the entire StackExchange API – that would be outside the
scope of this project. The only reason the Client exists is that I
couldn’t fine one that would work against the 2.x version of the
official API.
1. The Maven dependencies
To consume the StackExchange REST API, we will need very few
dependencies – essentially just an HTTP client – the Apache HttpClient
will do just fine for this purpose:
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.3.3</version>
</dependency>The
Spring
RestTemplate could also have been used to interact with the HTTP API,
but that would have introduced quite a lot of other Spring related
dependencies into the project, dependencies which are not strictly
necessary, so HttpClient will keep things light and simple.
2. The Questions Client
The goal of this Client is to consume the /questions REST Service that
StackExchange
publishes, not to
provide a general purpose client for the entire StackExchange APIs – so
for the purpose of this article we will only look at that.
The actual HTTP communication using HTTPClient is relatively
straightforward:
public String questions(int min, String questionsUri) {
HttpGet request = null;
try {
request = new HttpGet(questionsUri);
HttpResponse httpResponse = client.execute(request);
InputStream entityContentStream = httpResponse.getEntity().getContent();
return IOUtils.toString(entityContentStream, Charset.forName("utf-8"));
} catch (IOException ex) {
throw new IllegalStateException(ex);
} finally {
if (request != null) {
request.releaseConnection();
}
}
}This simple interaction is perfectly adequate for obtaining the
questions raw JSON that the API publishes – the next step will be
processing that JSON.
There is one relevant detail here – and that is the questionsUri
method argument – there are multiple StackExchange APIs that can
publish questions (as the
official
documentation suggests), and this method needs to be flexible enough to
consume all of them. It can consume for example the simplest API that
returns questions by setting questionUri set to
https://api.stackexchange.com/2.1/questions?site=stackoverflow or it
may consume the tag based
https://api.stackexchange.com/2.1/tags/{tags}/faq?site=stackoverflow
API instead, depending on what the client needs.
A request to the StackExchange API is fully configured with query
parameters, even for the more complex advanced search queries – there is
no body being send. To construct the questionsUri, we’ll build a basic
fluent RequestBuilder class that will make use of the URIBuilder
from the HttpClient library. This will take care of correctly encoding
the URI and generally making sure that the end result is valid:
public class RequestBuilder {
private Map<String, Object> parameters = new HashMap<>();
public RequestBuilder add(String paramName, Object paramValue) {
this.parameters.put(paramName, paramValue);
return this;
}
public String build() {
URIBuilder uriBuilder = new URIBuilder();
for (Entry<String, Object> param : this.parameters.entrySet()) {
uriBuilder.addParameter(param.getKey(), param.getValue().toString());
}
return uriBuilder.toString();
}
}So now, to construct a valid URI for the StackExchange API:
String params = new RequestBuilder().
add("order", "desc").add("sort", "votes").add("min", min).add("site", site).build();
return "https://api.stackexchange.com/2.1/questions" + params;3. Testing the Client
The Client will output raw JSON, but to test that, we will need a JSON
processing library, specifically
Jackson 2:
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.3.3</version>
<scope>test</scope>
</dependency>The tests we’ll look at will interact with the actual StackExchange API:
@Test
public void whenRequestIsPerformed_thenSuccess()
throws ClientProtocolException, IOException {
HttpResponse response = questionsApi.questionsAsResponse(50, Site.serverfault);
assertThat(response.getStatusLine().getStatusCode(), equalTo(200));
}
@Test
public void whenRequestIsPerformed_thenOutputIsJson()
throws ClientProtocolException, IOException {
HttpResponse response = questionsApi.questionsAsResponse(50, Site.serverfault);
String contentType = httpResponse.getHeaders(HttpHeaders.CONTENT_TYPE)[0].getValue();
assertThat(contentType, containsString("application/json"));
}
@Test
public void whenParsingOutputFromQuestionsApi_thenOutputContainsSomeQuestions()
throws ClientProtocolException, IOException {
String questionsAsJson = questionsApi.questions(50, Site.serverfault);
JsonNode rootNode = new ObjectMapper().readTree(questionsAsJson);
ArrayNode questionsArray = (ArrayNode) rootNode.get("items");
assertThat(questionsArray.size(), greaterThan(20));
}The first test has verified that the response provided by the API was
indeed a 200 OK, so the GET request to retrieve the questions was
actually successful. After that basic condition is ensured, we moved on
to the Representation – as specified by the Content-Type HTTP header –
that needs to be JSON. Next, we actually parse the JSON and verify that
there are actually Questions in that output – that parsing logic itself
is low level and simple, which is enough for the purpose of the test.
Note that these requests count towards your
rate limits specified by
the API – for that reason, the Live tests are excluded from the standard
Maven build:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.17</version>
<configuration>
<excludes>
<exclude>**/*LiveTest.java</exclude>
</excludes>
</configuration>
</plugin>4. The Next Step
The current Client is only focused on a single type of Resource from
the many available types published by the StackExchange APIs. This is
because it’s initial purpose is limited – it only needs to enable a user
to consume Questions from the various sites in the StackExchange
portfolio. Consequently, the Client can be improved beyond the scope of
this initial usecase to be able to consume the
other types of the
API.
The implementation is also very much raw – after consuming the
Questions REST Service, it simply returns JSON output as a String – not
any kind of Questions model out of that output. So, a potential next
step would be to unmarshall this JSON into a proper domain DTO and
return that back instead of raw JSON.
5. Conclusion
The purpose of this article was to show how to start building an
integration with the StackExchange API, or really an HTTP based API out
there. It covered how to write integration tests against the live API
and make sure the end to end interaction actually works.
The second part of this article will show how to interact with the
Twitter API by using the Spring Social library, and how to use the
StackExchange Client we built here to tweet questions on a new twitter
account.
I have already set up a few twitter accounts that are now tweeting the 2
Top Questions per day, for various disciplines:
-
SpringTip – Two of the best Spring
questions from StackOverflow each day -
JavaTopSO – Two of the best Java
questions from StackOverflow each day -
AskUbuntuBest – Two of the best
questions from AskUbuntu each day -
BestBash – Two of the best Bash
questions from all StackExchange sites each day -
ServerFaultBest – Two of the best
questions from ServerFault each day
The full implementation of this StackExchange Client is
on github.
