UTF-8 character problem with JIRA Application link

I have created a JIRA, and a Confluence add-on. They communicated with each other via REST API, using the Application link interface. I needed to call a REST function, implemented in my own JIRA add-on, from the Confluence add-on. 

It worked fine until any of the REST parameters contains a non ASCII character. In case of the parameter contains a German umlaut (öäüß), I got question marks at the JIRA side.

I wanted to prove, that the problem is not with my JIRA plugin, so I tried to call standard, built in REST functions via application link as well. The example code shows, how to call the REST API in order to retrieve project information.

public JsonJiraProject findJiraProject(String projectKey) {

    ApplicationLink appLink = appLinkService.getPrimaryApplicationLink(JiraApplicationType.class);
    ApplicationLinkRequestFactory factory = appLink.createAuthenticatedRequestFactory();
 
    ApplicationLinkRequest aplrq = factory.createRequest(Request.MethodType.GET, "http://localhost:2990/jira/rest/api/2/project/");

    aplrq.addRequestParameters("projectKey", projectKey);
    aplrq.setSoTimeout(APP_LINK_TIME_OUT);
    String jiraResponse = aplrq.execute();
    ...

I found, that the same problem occurs with basic, built-in JIRA REST functions as well. JIRA application link does not support UTF-8 parameters.

What I tried to get UTF-8 parameters working

• I tried to set header of ApplicationLinkRequest, but it does not help.
  alr.addHeader("Content-Type", "application/json;charset=UTF-8");
• I tried to define parameter directly at the end of the URL, without adding it as request parameter.
• I tried to URL encode the URL with the parameter
• I tried to call the function direct from a client (RESTClient add-on in Firefox), and I got the correct answer.

So I am sure, that the problem is with the ApplicationLinkRequest implementation, and it can not be solved by setting any option or header parameter.

As workaround I found, that I need to convert my input fields to ISO8859-1 character set. It does not cause any problem, and covers the characters (German umlauts), I needed to use. 

I created a utility class with following code to convert my input values:

public static String convertToIso8859(final String input) {
    if (StringUtils.isBlank(input)) {
        return "";
    }
    try {
        return new String(input.getBytes("UTF-8"), "ISO8859_1");
    } catch (UnsupportedEncodingException e) {
        return "";
    }
}

At the same time I restricted the allowed characters for my output fields, to avoid typing characters, outside of the ISO8859-1 set.

Multiton design pattern

The Multiton pattern is a design pattern similar to the singleton, which allows only one instance of a class to be created. The Multiton pattern expands on the singleton concept to manage a map of named instances as key-value pairs.

Rather than having a single instance per virtual machine, the Multiton pattern instead ensures a single instance per key. The key must not be a String, it is handy to define it as an enumeration. Using enumeration as Multiton key limits the number of possible elements in the hash, therefore eliminates the greatest risk of using the Multiton pattern, namely the memory leak. 

The pattern can be used when

• creating an object requires more effort, than storing it.
• some, but not too many instances of the given class should exists in the application
• instances should be created at the first demand
• instances should be accessible centrally

Advantages

• The pattern simplifies retrieval of shared objects in an application. Instances of the class should be accessed globally, via static getInstance method of the class.
• Makes it possible to store structures with high creation cost in the memory. If creation of the structure requires data reading, and high amount of calculation, it worth not to throw them away after once created, but store them in a Multiton structure.
  
  
• Possible examples can be like representation of domain specific configuration data

Disadvantages

• Memory leak. As object instances once created, are constantly referenced from the Multiton map, they can not be subject of garbage collection, until the end of the application. Therefore I would not recommend using Multiton for a set of objects with dynamic number of elements. 
    
• as with many creational pattern, Multiton introduces a global state, therefore makes it more complicated to test the application. 
  
  
• Makes dependency handling of related unit tests more complicated. If the constructor is private, and can be used only from inside the class, a more sophisticated mocking is necessary.

Examples of usage

• Storing database connection properties in a Multiton structure. If a predefined number of connections are used for the enterprise application, with well separated domains (Like HR database, Inventory database, Customer database, etc.) all connection information can be stored in a ConnectonProperties class. Instead of creating a ConnectonProperties instance every time, when a connection pool needs to be created, or a new element needs to be added to the pool, they can be stored in a Multiton structure. 
  A ConnectonProperties will be read only on demand, and stored from that point of time on, till end of the application.
• When implementing a GUI table, with requirement of showing detail view of the selected element, you can store detailed record information in a Multitone structure. The Multiton will be contain a dynamically growing set of data, therefore it is afforded to implement a feature to remove long time not used elements from the structure. 

Sources
http://www.blackwasp.co.uk/Multiton.aspx
https://en.wikipedia.org/wiki/Multiton_pattern
http://gen5.info/q/2008/07/25/the-multiton-design-pattern

Using JIRA REST API via Restclient.

While implementing JIRA plugins, I recently had to use JIRA REST API to create and modify issues.

As I did not want to install any desktop application, I decided to use the RESTclient plug-in for Firefox.

It provides a very convenient way to deal with REST requests, and makes it possible to try out your own REST functions fast.

I however faced some problems during setting up the client, so I list here, how to do it properly, in order to avoid wasting my time at the next ocassion.

• Install the RESTclient add-on for Firefox, from the Firefox application store
• Start the plugin
• Set up the request:
• Method: POST
• URL: http://localhost:2990/jira/rest/api/2/issue
• Define authentication by adding a new authentication element to the request. Select basic authentication, and set username and password
• Define content type by adding a new header element: Content-Type:application/json
• Define user agent: User-Agent:adminIt is necessary, for avoid JIRA error message: "403 Forbidden". If it is not defined, you get error from the REST API, and following log entry is shown in JIRA log:
  "XSRF checks failed for request"

• Define message body

{
    "fields": {
       "project":
       { 
          "key": "TEST"
       },
       "summary": "REST ye merry gentleme.",
       "description": "Creating of an issue using project keys and issue type names using the REST API",
       "issuetype": {
          "name": "Bug"
       }
   }
}

• Send the REST command.

For REST commands visit the JIRA documentation.

Lazy initialization design pattern

Creational design pattern

"... lazy initialization is the tactic of delaying the creation of an object, the calculation of a value, or some other expensive process until the first time it is needed."

Lazy init can be triggered by

• calling the assessor method of an object
• started at a low priority background thread at application startup
• started by a job

It is a good tactic to have a central class, responsible for lazy initialized objects in the application. It is usually a singleton class, that creates and stores the object target for the lazy init.

Advantages

• Makes it possible to initialize an object just when it is required by a client.
• Makes startup time of the application shorter.
• Can save resources, like database connections, opened files or memory.

Risks

• In multi-threading environment a special care needs to be taken in order to avoid multiple instantiation of the object. It can be inconvenient, or can cause a serious performance problems in the system. Therefore can be used for attacking your system.
• As for all optimization, it is true, that you need to implement it, if it solves a real performance problem. Try to avoid eager optimization, before even facing the problem.
• Like it is easily to experiment with JPA, you need to be careful when defining the amount of objects, created by lazy initialization or amount of effort to be spent on it. You can easily load to much at the same time.    
• During debugging, the lazy init can take place, triggered by the debugger. Therefore, the application can behave different than under normal circumstances.     

Real word examples

• JPA lazy initialization for collection of related objects.
• Database pools or any resource pool, where a resource is complicated and time consuming to be build.
• LoadingCache in Google Guava project. The cache loads a new element, after it was first requested, and not found among the cached objects. 

My implementations

1. I have created a small application, that provided data to our company Avatar server. The data shown in Avatar was synchronized with different user directories of the company, like Active Directory, JIRA Crowd, Corparate Directory and LDIF. The domain object called User contained two profile pictures, one with small and another one with big resolution. The small one was used in user lists, and in some other places. As the big one was used relatively rare, it was not loaded into the User object until it was really needed.  
   User objects were stored in a memory cache, with limited capacity, so not more needed Users were purged from the memory after defined time of inactivity. Also the user object and the image inside of it were both subjects of lazy initialization.  
2. I had to check if input values from a Confluence macro contain any html tags, and remove them if so. As it was not possible either to use any Confluence utility or install the OWASP packages under the plug-in, I implemented a small class to sanitize the input strings. It used a lots of Patterns to find tags to be removed. The pattern objects were initialized at the first usage, and stored in a static list.  

Sources
https://en.wikipedia.org/wiki/Lazy_initialization
http://www.martinfowler.com/bliki/LazyInitialization.html

Code templates for Eclipse

In my daily work, I found very productive to use my own code templates for Eclipse. By the time, I have collected a useful set of templates.

You can define your own templates by opening the dialog under Preferences\Java\Editor\Templates

Defining templates is quite straightforward. If you need a detailed introduction, please refer to the following source in Tutorialspoint.

Tips and tricks

Make it accessible

As it is possible to export and import templates, I keep my templates file under version control, so I can use them easily in all of my projects.

Naming

Naming my own templates is important, while I need to select them all for export, I want to handle them together, and I want to find them during the coding really fast.
Therefore I add the prefix "my" to the template name, and use namespaces to group my templates. So, my templates are named like this:

• my.junit.assertTrue
• my.log.createLogger
• my.stringUtils.notBlank

Variables

You can use variables in your template, which makes it possible to define complex expressions.
You can found a complete list of the variables found here. 

Imports

If you use any class in the template, that needs to be imported, define the required import statements as well. Sometime I generate a template for commonly used classes, just for spare the time of importing the needed package. Commons StringUtils is such an example for it. 

${:import(org.slf4j.Logger,org.slf4j.LoggerFactory)}
// Logger instance
private static final Logger log = LoggerFactory.getLogger(${enclosing_type}.class);${cursor}

Static imports

If you use methods, to be imported statically, you can use the staicImport keyword

${is:importStatic('org.junit.Assert.assertTrue')}

assertTrue(${cursor});

My Templates are stored under GitHub, and free to use by anyone. It contains following templates:

• Singleton
• Lazy initialization in getter method
• Basic JUnit assertions with imports
• AssertThat method calls with static exports
• Mockito commands (when, mock, times)
• JUnit exception testing with the @Rule annotation
• Logging (create logger, using different log methods, logging for exception)
• Input validation for methods, using commons validation
• generating enumeration with a field inside 
• Fixme and Todo templates

Using reflection in setup phase of unit testing

It is  a common practice to use mock framework in the setup phase of a unit test, in order to set the internal state of the dependent objects. Sometimes it is though easier to set a value of a given field directly, in order to test the class accordingly.

For modifying field value via reflection, I used the reflection utility classes from the org.apache.commons.lang3.reflect package.

In my example I had a class, responsible to holding the application configuration, which is loaded from an XML file. The default location of the configuration file is hard-coded into the class. I wanted however test the loadConfig method, with different test configuration files, in order to check all possible configuration cases.

public class ConfigurationHolder {

    private static final String DEFAULT_CONFIGURATION_FILE_LOCATION = "./src/main/conf/cleaner.xml";

    private final String configurationFileLocation;

    private CleanerConfiguration cleanerConfiguration;

    private final static ConfigurationHolder instance = new ConfigurationHolder();

    private ConfigurationHolder() {
        this.configurationFileLocation = DEFAULT_CONFIGURATION_FILE_LOCATION;
    }

    public CleanerConfiguration getConfiguration() {
        if (cleanerConfiguration == null) {
            loadConfig();
        }

        return cleanerConfiguration;
    }

    private void loadConfig() {
        try {
            JAXBContext jaxbContext = JAXBContext.newInstance(CleanerConfiguration.class);
            Unmarshaller jaxbUnmarshaller = jaxbContext.createUnmarshaller();
            File file = new File(configurationFileLocation);
            cleanerConfiguration = (CleanerConfiguration) jaxbUnmarshaller.unmarshal(file);
        } catch (JAXBException e) {
            throw new RuntimeException("Problem with cleaner configuration: " + e.getMessage());
        }
    }

    public static ConfigurationHolder getInstance() {
        return instance;
    }

}

With using mock framework I would have had design following possibilities:

• creating a class, like ConfigurationFileLocationProvider or ConfigurationFileProvider to deliver the location of the file, or the file itself, and use the class in ConfigurationHolder.
  In this case I believe, I would have introduced a dependency just to make testing possible.
   
• Passing the file location in the constructor. In this case I would have lost the possibility to use the class as singleton. 

So I decide to keep the simple implementation, and use reflection for the unit test. I decided to use the commons reflections package, while it seams to be a more sophisticated solution as the one from Spring framework.

As I wanted to set the value of a simple private field, I implemented the following method in the unit test class:

private ConfigurationHolder createConfigurationHolder(String configFileLocation) throws IllegalAccessException {
    ConfigurationHolder cr = ConfigurationHolder.getInstance();
    FieldUtils.writeField(cr, "configurationFileLocation", configFileLocation, true);
    FieldUtils.writeField(cr, "cleanerConfiguration", null, true);
    return cr;
}

In my test case I use this method to create a new instance, with the given file location

@Test
public void testGetConfiguration() throws Exception {
    String configFileLocation = "./src/test/conf/cleaner-config-test.xml";
    ConfigurationHolder cr = createConfigurationHolder(configFileLocation);

    CleanerConfiguration configuration = cr.getConfiguration();
    Assert.assertThat(configuration, notNullValue());
}

As always, you need to consider what brings you to modify the state for unit test, and which possible problems can you face with.

Advantages.

• you can change the internal state of your class without adding unwanted dependencies to it.
• you can test objects without setter methods like classes used for XML or JSON data modelling (with @XmlRootElement or @JsonAutoDetect annotation). These classes are generated usually without setter methods, and instances get filled up via reflection.  

Disadvantages

• as in case of the solution, where unit testing is done via mock framework, you need to design the class carefully, to avoid restrictions regarding reflection. In this case, I had to use the configurationFileLocation property, while, the DEFAULT_CONFIGURATION_FILE_LOCATION constant can not be modified by reflection.
• As name of the field to be modified via reflection is hard-coded into the unit test case, you need to be careful when refactoring the class. I do not see it as a big risk, however, while you need to run unit test during the refactoring anyway, so you will be forced the change the relevant test cases as well.  

Further useful methods from the FieldUtils class:

• removeFinalModifier() makes it possible to change even final fields
• writeStaticField() makes it possible to change even static fields