Sunday, October 4, 2015

Using Jira's REST API with Scala and Lift-Json

Before we started writing our little Jira Git Stats Collector we were thinking about if it was actually worth the trouble to write a separate tool just to print some git stats, but it turns out that writing it was as much fun as interpreting the information extracted by it. Since providing and consuming APIs is what the internet is all about (from a developer's point of view, that is) i thought today i would share a bit of information about how elegantly REST APIs can be consumed with scala.

There are multiple JSON libraries available (in java and also in scala) and each of them has their strengths and weaknesses. Usually in the scala world most people would probably go for Argonaut, Spray or use ScalaJson if they use Play. Our use case was simple enough to try something we hadn't used before: Lift-Json. For maximum type-safety one would usually create case classes conforming to the responses provided by the API, but since this was an ad-hoc project and we were only about to use exactly one field of the response we lowered ourselves into the dark waters of AST traversal and type casts ;)

Jira's REST API responses are easy to understand and allow for simple parsing and automation.

The Use Case

  • Provide one or multiple Epics as input
  • Extract all issue keys belonging to this epic from the REST API
  • Extract all subtasks belonging to each returned issue key
  • Return the combination of the three lists for further processing

Preparation

Epics

The issue type Epic in Jira is not implemented as a standard feature but as a custom field which is created when you install the Jira Agile Plugin. This means that depending on which custom fields you had before you installed the plugin (either created by yourself or by other plugins), the Epic field's ID will vary. In our case this id was cf[10147] which we found easily by using the advanced issue search in Jira, typing "epic" and looking at the autocomplete popup.

Authentication

Jira's REST API supports HTTP Basic Authentication, so it is easy enough to get authorized by providing some username and password in the correct format within the HTTP request. We wrote a little helper object to supply the required header:

package io.sourcy.jirastatscollector

import java.util.Base64

object HttpBasicAuth {
  private val BASIC = "Basic"
  val AUTHORIZATION = "Authorization"

  private def encodeCredentials(username: String, password: String): String =
    new String(Base64.getEncoder.encode((username + ":" + password).getBytes))

  def getHeader(username: String, password: String): String = BASIC + " " + encodeCredentials(username, password)
}

SSL

Since our use case was purely internal we decided to allow for disabling SSL verification:

private object NoSsl {
  def disableSslChecking(): Unit = {
    HttpsURLConnection.setDefaultSSLSocketFactory(NoSsl.socketFactory)
    HttpsURLConnection.setDefaultHostnameVerifier(NoSsl.hostVerifier)
  }

  private def trustAllCerts = Array[TrustManager] {
    new X509TrustManager() {
      override def getAcceptedIssuers: Array[X509Certificate] = null

      override def checkClientTrusted(x509Certificates: Array[X509Certificate], s: String): Unit = {}

      override def checkServerTrusted(x509Certificates: Array[X509Certificate], s: String): Unit = {}
    }
  }

  def socketFactory: SSLSocketFactory = {
    val sc = SSLContext.getInstance("SSL")
    sc.init(null, trustAllCerts, new SecureRandom())
    sc.getSocketFactory
  }

  def hostVerifier: HostnameVerifier = new HostnameVerifier() {
    override def verify(s: String, sslSession: SSLSession): Boolean = true
  }
}

Implementation

The implementation itself is surprisingly straight forward. First we need a possibility to run search queries against the API to be able to search for epics:

  private def runJql(jql: String): JValue = {
    NoSsl.disableSslChecking()
    val connection = new URL(Settings.jiraUrl + "jql=%s".format(jql)).openConnection
    connection.setRequestProperty(HttpBasicAuth.AUTHORIZATION, HttpBasicAuth.getHeader(Settings.jiraUser, Settings.jiraPassword))
    parse(Source.fromInputStream(connection.getInputStream).mkString)
  }

JSON data are represented as an AST in Lift-Json, so we need a method to exract an issue key from a node in the AST. In this case the actual type of a list of key-value pairs (as in {key=ISSUE-123, key=ISSUE-456}) is a List[Tuple2], so we extract the List using a type cast and only use each tuple's second value.

  private def extractIssuesFromJValue(values: JsonAST.JValue#Values): List[String] =
    values.asInstanceOf[List[(String, String)]].map(tuple => tuple._2)

Then we need a way to extract an Issue's subtasks:

  private def extractSubTasks(issue: String): List[String] = {
    val values = (runJql("parent=%s".format(issue)) \ "issues" \ "key").values
    var ret: List[String] = List()
    try {
      ret = extractIssuesFromJValue(values)
    } catch {
      case e: ClassCastException =>
    }
    issue :: ret
  }

...and a way to extract an Epic's child issues:

  private def extractChildIssues(epic: String): List[String] =
    epic :: extractIssuesFromJValue((runJql(Settings.epicCustomField + "=%s".format(epic)) \ "issues" \ "key").values)

In the end we just need to piece it all together:

  def extractAllIssues(epics: Seq[String]): Seq[String] =
    epics.flatMap(epic => extractChildIssues(epic).flatMap(issue => extractSubTasks(issue)))

This will return

  • The Originally provided Epic(s)
  • All of the Epic's child issues
  • All issues' subtasks
flattened into one Seq.

Take a look at the GitHub repository if you want to see more :)

Friday, October 2, 2015

Book Recommendation: Functional and Reactive Domain Modeling

I have recently finished reading what is already available of Debasish Ghosh's brilliant book Functional and Reactive Domain Modeling which is currently available via Manning's MEAP program. I have been doing functional programming with scala for about 2 years now after almost 15 years of working around side effects with OOP and other paradigms and have also inhaled other phantastic books like Functional Programming in Scala by RĂșnar Bjarnason and Paul Chiusano but what makes Functional and Reactive Domain Modeling so outstanding for me is the extremely practical and applicable approach to the more advanced - at least for me - topics of functional programming.

And no, i don't get paid by Manning for this, just in case you were wondering :D