Send new created Jira ticket info to MS-Teams Channels

Send new created Jira ticket info to MS-Teams Channels

Message Delivery “Architecture”

Summary

We are using (self hosted) Jira (the Atlassian Issue Tracker / Scrum / Kanban Board) at the office. And we wanted to receive a chat notification for each newly created ticket, to be received in Microsoft Teams Channels. Teams is a collaboration messenger tool like Slack or RocketChat.

Jira can send new ticket/story creations using a web-hook to external tools. The web-hook sends a Jira specific JSON payload, no configuration options possible. The project you are looking at, consists of a transformation service, which can transform the Jira JSON to a Microsoft Teams Channel formatted JSON. And while we are at it, it has the option to route different Jira project codes to different MS-Teams Channels. This way, colleagues can subscribe to the channels for the project codes they are interested in.

The transformer we use is the Comcast EEL transformer. See https://github.com/Comcast/eel.

To see my solution, look at my GitHub repository, which has the transformation configuration, an example, and some ways on how to deploy this in either docker or Kubernetes.

GitHub Project: https://github.com/atkaper/jira-to-ms-teams-notifier

Backstory

As mentioned before; at the office, we are using Jira as ticket system, for bugs/issues, and Scrum/Kanban stories/tasks. And we were using RocketChat as collaboration messenger tool. We used a setup, to send each newly created ticket from Jira to RocketChat. In Jira there is an outgoing web-hook defined, and in RocketChat there was a little JavaScript on the incoming hook to transform the Jira JSON to a RocketChat message. A simple and flexible setup. Super!

However… RocketChat was only used by our online-department (e.g. approximately 300 people), and not Corporate wide (thousands?). And as we already had to move away from using Gmail into using Outlook mail (MS Office 365), the logical next step was to start using Microsoft Teams as collaboration tool. This would give us corporate wide access for everyone.

Note: I personally do not like Office 365, as it has no proper support for Linux users (yet). Their attempt at a native “Teams” app for Linux is not up-to standard. It keeps logging me of daily (probably a corporate policy), and does not remember the login credentials (MS failure – or feature). So I’m stuck at the web version. I do like Gmail, and RocketChat much better. Oh well… corporate deals, we have to deal with it 😉 Will get used to it.

So we had to move the connection which went from Jira to RocketChat into a connection from Jira to MS-Teams. Unfortunately both Jira and MS-Teams lack the flexibility to connect them directly to each other. They both have their own fixed “message” formats. Therefore I had to look for some piece of connecting software, which is able to transform the data format between the two.

Another shortcoming of MS-Teams is that you can only send your message to a single Channel via a web-hook. RocketChat allowed us to setup a single incoming web-hook, which could be used to send to any of the defined channels. We did routing based on Jira project code inside the RocketChat translation JavaScript.

After googling around for a bit, and dismissing the payed Jira plugins which could do this, I found the opensource “Comcast EEL” project. It’s a JSON transformer, with built in listening web server and outgoing HTTP post option. And apart from simply transforming, you can add some simple logic and lookup tables / functions. Perfect for this use-case!

So I managed to find out what the data structure of Jira messages is, and what a possible message structure for MS-teams is. After this, I developed the transformation mapping configuration, and added the Channel routing to it.

Our Jira is running in our own Kubernetes cluster. So in our case, the Comcast EEL transformer is simply put in a separate namespace next to the Jira namespace, in the same cluster.

Running for a couple of days now, and seems to work fine.

Installation

For installation, please see the README.md file on GitHub. In summary:

  • Build the container.
  • Configure channels per project code, and a default channel.
  • Optional: run a transformation test.
  • Install container on a docker server, or in a Kubernetes cluster.
  • Configure Jira web-hook to point to the container.

That’s all!

Here is an example screenshot of the MS-Teams connector configuration page. From that page, take the URL path, without the server name to put in the project to channel mapping entry (the part starting with /webhook/).

MS-Teams connector configuration example

Technical Details

The following sections are here, in case you are wondering how this all works together, or in case you want to customize the transformation for your own use-case. For example you can add or remove Jira details to be shown. I actually tried displaying the profile image of the user who created the ticket, but gave up on it. It took quite some space (MS-Teams uses more visual space than RocketChat already), and it insisted on messing up my image URLs 😉 But in theory, it is possible.

Jira Outgoing Message Format

For a full example, see example-jira-data.json. It has been generated from our Jira, and I have only edited out some internal details (server names, id’s, mail-domain).

The interesting fields are the following ones:

  • /issue/key = ticket id.
  • /issue/self = a technical URL for the ticket. I cut it in two, to use the server name.
  • /issue/fields/summary = ticket title.
  • /issue/fields/description = ticket description.
  • /issue/fields/priority/name = ticket priority.
  • /issue/fields/labels = ticket labels.
  • /user/displayName = name of user who created the ticket.

But of course there are more fields, so feel free to tweak the transformation.

MS-Teams Expected Message Format

For a full example, see example-ms-teams-data.json. It has been generated from the example-jira-data.json via the transformation configuration.

You can copy the ms-teams JSON into a special “playground”; https://messagecardplayground.azurewebsites.net/ to see if they do understand the data, and they attempt to render a preview. Note that the preview uses different styling than the real end result in MS-Teams.

Interesting fields:

  • /summary and /title = message card title. I haven’t worked out yet where/when they show which of these, so I fill them with the same value.
  • /potentialAction/targets/uri = URL for a shown button to open on click.
  • /sections = can contain multiple display elements. Now using just one.
  • /sections[0]/facts/name+value = key/value pairs to show in a table.
  • /sections[0]/text = a bit of text to be shown in the section.

If you for example want to add a section with a profile photo, name, and some texts, you can add this above (or below) the current section element (you can leave fields off, if not needed – just play with it in the playground to see the effect):

        {
            "activityImage": "image-url",
            "activityTitle": "some title (user name?)",
            "activitySubtitle": "some more info",
            "activityText": "and even more..."
        },

Here are two images (using different message sources, not same the data), with the rendered result. One in the playground, and one in the real MS-Teams app.

Playground message preview (click to zoom)
MS-Teams Message Display

Note: In above MS-Teams message display, the truncation function for the detail text was not available yet, so it does show the non-truncated details, and the labels field was empty in the Jira source.

Transformation

The transformation configuration can be found here: transform-jira-to-ms-teams.json.

Interesting elements:

  • /Match and /IsMatchByExample = See the EEL handler-selection manual. Currently not used, but you can use this when creating more than one transformation file, and select the one to handle the action by configuring the proper Match expression.
  • /CustomProperties = A bunch of key/value pairs, which you can use later on in the transformation, using the propexists() and prop() functions. In here, we use this to map Jira project codes to the proper MS-Teams web-hook URL (path). But you can put any configuration items in there, for further use by the script.
  • /Transformations = This is not the main transformation, but a set of little helper transformations. I’ve used this to put the more complex expressions in, to keep the main transformation a bit more readable.
  • /Transformations/project = Helper expression, which uses a regex() function to get the Jira project code from the key. So it would return SBX for value SBX-123. The “{{/}}” in that rule indicates that it outputs the result as root element value (for the helpers just the result string value). See also the EEL jpath manual
  • /Transformations/webhook = Helper expression, which returns the MS-Teams web-hook path to use. It checks if a CustomProperties key of the project code exists, and if so, it returns it’s value. If it does not exist, it returns the default from the CustomProperties set. Here’s the use of the prop() function (read custom property), the propexists() function (check if a property exists), the transform() function (executes one of the Transformations), and the ifte() function (an if-then-else).
  • /Transformations/issueUrl = Helper expression, which returns the full URL to view the Jira ticket. I takes the base URL to your Jira instance from the /issue/self field, and appends the /browse/ path and ticket key to it.
  • /Transformations/partialDescription = Helper expression, which returns the first X characters of the description. It uses the js() function to pass the data to a JavaScript expression to do the truncating. There is a truncation function in EEL also, but that took much more code to use than the JS version. If a string is truncated, we add “…” to it at the end.
  • /Transformation = (note the missing S at the end) – This is the main transformation. It’s body consists of the JSON to be send to MS-Teams. In that JSON, the values are filled by using EEL expressions “{{…}}”.
  • /IsTransformationByExample = Having this set to TRUE, indicates that the Transformation body is the resulting JSON structure. If you set this to false, you are working with EEL expressions, like is done in the Transformations section (where all were set to false).
  • /Path = The path of the HTTP service to send the message to. In this case filled with the proper web-hook path.
  • /Verb = Indicating we want an HTTP POST.
  • /Endpoint = The base URL of the MS-Teams web-hook to send our data to.
  • /Protocol = Indicating the EEL output handler to use. In this case the HTTP send.
  • /HttpHeaders = Some HTTP request headers, which will be send with the output to MS-Teams. They don’t need them. Just added as example, and to use in the EEL debug log if enabled.

For better understanding of the possible EEL functions, I suggest you have a read of the EEL function manual, and look at the examples. And if you are planning to use multiple transformation files, make sure to look up the Match explanation in EEL. It also has a Filter option, that can be used to drop messages also. And you can configure if a transformation is the last one to execute, or that you want to execute multiple transformations in a row for the same source data. That would allow you for example to send the messages to more than one channel or more than one destination.

Security Considerations

The Comcast EEL container has no authentication configured on the incoming call’s. So it would be best not to expose that on a public network, to prevent silly messages from flooding your MS-Teams channels. In our situation, we are running the EEL container in the same non-public Kubernetes cluster as our Jira is running in, so we are not open to abuse. Also the EEL container does not listen outside of the cluster, so even internal networks (other than inside the cluster) can not reach the service.

You can setup a “Match” section in the transformation file. In there, you can check for certain request headers, or for certain pieces of data which must be available. That way you can make the EEL container not respond to messages without that data. Note that Jira does not allow any security settings, so you will be stuck at checking for example the existence and name of your Jira server (e.g. check that /issue/self starts with your Jira server address). I’m not sure if EEL can access query string parameters. If so, you could generate a key to put in the Jira outgoing web-hook configuration, and in EEL check the value matches.

Conclusion

This was a fun little project to build. EEL is quite flexible in what it can do, even using Java-Script fragments. It works quite well!

One addition I might look at later, is for creation of Jira sub-tasks, to see if we can show the parent task in the channel message also. If I do find out, I will amend this blog post with the new transformation configuration. In case I did not get to this, and you need it yourself, here are the steps to find out:

  • run the EEL container in debug level (see startup parameter).
  • create a sub-task in Jira.
  • look at the EEL log to get the Jira JSON data (it’s mixed in with other data, so you need to carefully look for the start and end of the Jira message – just compare it to the example).
  • find out where the parent ticket info is stored (to ease that, reformat the Jira message with a JSON-formatter, I use the “jq” command for that, but you can also use a suitable editor, or online formatter).
  • put the data in the transformation. This can be done simply as always displayed field (which will be blank if no parent). Or you can copy the transformation template twice, one with parent, and one without. And add a Match section to them which filters on the case to handle. In that case, make sure to map both files into the container.
  • don’t forget to disable debug logging on EEL again.

That’s it for now.

Thijs Kaper, March 21, 2020.

Leave a Reply

Your email address will not be published. Required fields are marked *