I’m happy to announce that we’ve just released the Atlassian Plugins SDK 4.1. Actually, we released it last week but didn’t tell anyone — we figured most of the Americans would be too busy chomping on turkey and being thankful. Those of you who are on SDK 4.0 and have used it since then should have gotten a notification that a new release is available. If you haven’t done it yet (and are using 4.0), go ahead and:

1
atlas-update

Some of you may be aware that at this year’s AtlasCamp we released the first major release of the SDK (4.0) in years, and while that was a big step forward, it could be argued that 4.1 is even bigger.

But before I dive into 4.1, I realized that we never blogged about what’s new in 4.0. So here’s a quick recap of the new features introduced in 4.0:

  • Live Reload speeds up the dev loop by allowing you to keep your focus in your IDE while you watch your changes applied in real-time in the browser.
  • Native Installers — no more untaring/unzipping binaries and setting environment variables. Our installers make it super easy to get started with plugin development. We’ve also added automatic update detection so that you’ll always be up-to-date.
  • Developer Toolbar When the dev is in an atlas-run/debug/run-standalone session, they’ll now have the ability to toggle a new Developer Toolbar that offers some nice developer productivity features, like the ability to search DAC, API docs, and Answers as well as toggle a variety of nice tools.

Great, now let’s dive into 4.1.

What’s New (at a glance)

  • Data Sources - That’s right, thanks to Adrien Ragot you can now use custom datasources when starting our products with the SDK.
    See : https://answers.atlassian.com/questions/11337/using-mysql-instead-of-hsql-db-with-atlassian-pdk
  • Resource Compression - We’ve updated the YUI compressor and also allow you to use Closure.
  • Fastdev 2.0 - Fastdev now uses the CLI / pi commands internally making it five times faster.
  • Multi-Fastdev - you can now map multiple plugins for fastdev from your pom. This makes developing multiple related plugins or plugins that use a shared “library plugin” a reasonable thing to do
  • “Wired” test framework - Introduced in 4.0, in 4.1 we’ve made in-product “wired” tests even awesomer… they run inside the products, have dependencies injected, and report back to the local JUnit result collector making testing fun (again?)
  • Plugins Test Console - A new in-product UI for running ALL of your plugin’s tests. This one rocks. Run tests without product restarts, run all, single, or batches of tests, re-run failed, fastdev test code, etc. see video below
  • Remote Testing - Run the above mentioned “Wired” tests against any already running product instance (OnDemand anyone?)
  • Various bug fixes.

Where do I get it?

If you’re on 4.0, you’ll get a notification to upgrade the next time you use the SDK. If not, you can download an installer here:

OSX, Windows, Debian or RPM

Features In-Depth

Data Sources

Datasources can be configured within a product inside of the amps plugin configuration section.
Here’s an example of configuring a Postgres datasource for JIRA:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
<plugin>
    <groupId>com.atlassian.maven.plugins</groupId>
    <artifactId>maven-jira-plugin</artifactId>
    <version>${amps.version}</version>
    <extensions>true</extensions>
    <configuration>
 
        ...
 
        <products>
            <product>
                <id>jira</id>
                <instanceId>jira50</instanceId>
                <version>${jira.version}</version>
                <dataVersion>${jira.version}</dataVersion>
                <dataSources>
                    <dataSource>
                        <jndi>jdbc/JiraDS</jndi>
                        <url>jdbc:postgresql://localhost:5432/jira</url>
                        <driver>org.postgresql.jdbcDriver</driver>
                        <username>jira</username>
                        <password>jira</password>
                        <libArtifacts>
                            <libArtifact>
                                <groupId>postgresql</groupId>
                                <artifactId>postgresql</artifactId>
                                <version>9.1-901-1.jdbc4</version>
                            </libArtifact>
                        </libArtifacts>
                    </dataSource>
                </dataSources>
            </product>
        </products>
    </configuration>
</plugin>

For more information about the datasource properties, see: http://cargo.codehaus.org/DataSource+and+Resource+Support

Resource Compression

In SDK 4.1, we’ve upgrade to YUI version 2.4.7 which resolves some bugs when compressing javascript and CSS. On top of that, we’ve added the ability to use the Google Closure Compiler for compiling javascript.

To enable Closure, simply add the following to your AMPS plugin configuration:

1
<closureJsCompiler>true</closureJsCompiler>

You can also enable it on the command line by passing:

1
-Dclosure.js.compiler=true

We’re hoping that enough people will test the Closure compiler throughout the initial 4.1 release that we can make it the default in the next SDK release.

Fastdev 2.0

As stated, Fastdev now uses CLI / pi internally to re-build plugins. However, now Fastdev is even better than using CLI / pi on the commandline because it is also aware of pom.xml changes and can restart the CLI automatically. This is something the commandline CLI can’t do.

The way it works is:

  • The first time a plugin needs to be reloaded, Fastdev boots up a CLI process for that particular plugin and stores a reference to it. Then it calls pi.
  • The next time that same plugin needs a reload and doesn’t have pom.xml changes, Fastdev simply calls pi on the already running CLI process for that plugin.
  • The next time that same plugin needs a reload AND has pom.xml changes, Fastdev stops the running CLI process for that plugin, starts a new CLI process (which reloads the pom) and stores the new process reference and then calls pi.

It’s been said that there are (overly) complex plugins that don’t play nice with pi. If you have one of these such plugins, you can revert back to having Fastdev run the complete maven process like it did before by adding the following to your AMPS plugin configuration:

1
<useFastdevCli>false</useFastdevCli>

So how fast is it?

On my own machine testing against the same plugin, the old Fastdev took an average of 30 seconds for every reload.

The new Fastdev takes an average of 18 seconds on the first CLI boot up and an average of 7 seconds to rebuild the plugin using pi.

Multi-Fastdev

With Fastdev 2.0 we’ve also added the ability to map additional plugins (other than the one your building) to be available for Fastdev/LiveReload.

To enable this, you can add a comma delimited list of resource paths to your AMPS plugin configuration:

1
2
3
4
<additionalResourceFolders>
/home/doklovic/data/IdeaProjects/atlassian-plugins-osgi-testrunner-bundle/src/main/resources
,/home/doklovic/data/IdeaProjects/developer-toolbox/src/main/resources
</additionalResourceFolders>

Why is this useful?

Well, when I was working on the new Test console, I had to make changes to the test console plugin itself, the plugin under test, and the Developer Toolbox plugin all at the same time. With multi-fastdev mapping, I could just make the changes I needed in any/all of the plugin and refresh my browser. Fastdev would figure out which plugins needed a rebuild and rebuild them all at the same time.

You may be thinking that this will only be used internally, however, combined with OBR support in UPM, this feature makes developing plugins that use a shared “library plugin” a reasonable thing to do and I think external plugin devs could also take advantage of it.

“Wired” Test Framework

What is a “wired” test? Simply put, it’s a JUnit 4 test that gets deployed as an Atlassian Plugin and as such is “wired” by Spring just like any other component. The test is actually run inside of the product container and reports back to the locally running JUnit.

Essentially this means you can do anything in your test that you can normally do in a JUnit test married with anything you can do in a plugin component and it all runs seamlessly during the integration-test phase of maven.

So how do you make a wired test? 3 simple steps:

  1. Add test runner dependencies to your POM
  2. Create a JUnit4 test class and annotate it with: @RunWith(AtlassianPluginsTestRunner.class)
  3. Add an atlassian-plugin.xml to your /src/test/resources/ folder and add anything your test needs injected.
    note: the plugin key should end with “-tests” but amps will add it if you forget

That’s it.

Once you have this, AMPS will do it’s magic to build a test bundle, deploy it, and make the proper test calls during the integration-test phase of the maven build. All test results will be added to the “normal” surefire report and you’ll get the same output on the commandline as you normally would with JUnit. To the system, the test is a plugin. To you, it’s just another test.

JUnit Enhancements

In SDK 4.0, you could write wired tests, however you were still constrained by the vanilla JUnit rules. With SDK 4.1 we’ve enhanced (and by “enhanced” I mean lifted) the JUnit constraints.

Here’s what’s different:

 

Normal JUnit Test Wired Atlassian Test
requires a single no-arg constructor promotes using constructor for dependency injection
@BeforeClass must be a static method @BeforeClass must NOT be static
@AfterClass must be a static method @AfterClass must NOT be static
Tests are stateless. Every method is run on it’s own instance of the test class. Tests are stateful. All methods are run on the same instance. Be careful to clean up any data at the end of your methods!

 

Here’s an example, note how much easier it is to get to writing a test… no mocks, no Selenium, etc.

Step 1: Maven Dependencies

Add these to your pom.xml (note: these are now added automatically when running atlas-create-<product>-plugin)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
<!-- the main testrunner api -->
<dependency>
    <groupId>com.atlassian.plugins</groupId>
    <artifactId>atlassian-plugins-osgi-testrunner</artifactId>
    <version>1.1-rc5</version>
    <scope>test</scope>
</dependency>
 
<!-- need to have junit 4.10 -->
<dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.10</version>
    <scope>test</scope>
</dependency>
 
<!-- these are here to override transitive versions -->
<dependency>
    <groupId>javax.ws.rs</groupId>
    <artifactId>jsr311-api</artifactId>
    <version>1.1.1</version>
    <scope>provided</scope>
</dependency>
 
<dependency>
    <groupId>com.google.code.gson</groupId>
    <artifactId>gson</artifactId>
    <version>2.2.2-atlassian-1</version>
</dependency>

Step 2: The Test Class

Step 3: The Plugin.xml

With everything in place, you can simply run atlas-integration-test to see it work.
It will also run as part of the maven default lifecycle as well.

Plugins Test Console

While developing the wired test framework, we found that although it makes writing tests a whole lot easier, actually running the tests slowed down the dev cycle due to….

  1. Write a test
  2. run atlas-integration-test
    1. recompiles the plugins
    2. starts the product
    3. runs the test
    4. shuts down the product
  3. inspect the results
  4. make changes and start over at step 1

The main problem is that rebuilding the project and cycling the container to run the tests for small changes takes at least 2-3 minutes. That alone makes you not want to write tests.To solve this, we built the Plugins Test Console that allows you to run all (or individual) tests from within the already running product container.

To make things even easier, we modified Fastdev to detect/rebuild/redeploy the test plugin as well as the main plugin when running tests. Keep in mind, when I say ALL tests, I’m not just talking about the new “wired” tests. The test console displays / runs “wired” tests, unit tests, and traditional integration tests.

The result? When testing on my own (super fast SSD + 16GB ram) laptop, the atlas-integration test cycle took 5 minutes.

During an 8 hour work day when focusing on writing tests, I was making lots of changes but let’s say I just made 50 changes that required a rerun of a test.

That’s 250 minutes or 4.1 hours that I just sat waiting for tests to run.

With the test console, I could rerun just the test I wanted and didn’t have to wait for the product to start. Combined with Fastdev 2.0 which I already stated rebuilds at an average of 7 seconds on my machine and given that an individual test of mine usually runs in under 5 seconds…

That’s 12 seconds per change which gives me a total of 10 minutes. That’s 4 hours I get back in a day!

Watch This:

Here’s a new video with a new song selection from http://www.youlicense.com

(please change quality to 720p and watch in fullscreen)

Here’s a screenshot of the console in action:

And now a warning…

For all of time, when amps generated any test code, either via atlas-create-<product>-plugin or atlas-create-<product>-plugin-module we have created the unit tests in the src/test/java folder using the same package as the base package in /src/main/java

This has always worked because the tests were never run inside of an OSGi container. You see, OSGi really hates “split packages” and since we deploy the base plugin and the tests plugin as separate bundles, we end up with split packages. So, with the addition of the test console which needs to be able to run unit tests inside of OSGi, we have changed the way amps generates test code and it now prefixes the unit test package with “ut.” similar to how we prefix integration tests with “it.”

What does this mean? Two things:

  1. You can no longer use package protected super classes, methods, etc. (from your main code)  in your tests.
  2. If you want to see unit tests from your already established plugin in the test console, you’ll have to move them into some sub package that doesn’t exist in the main codebase
  3. If you’re too lazy to move your unit tests, nothing bad will happen… they just won’t show up in the test console

Remote Testing

Finally in SDK 4.1 we’ve added an “experimental” feature that will most likely only be used by Atlassians, although there might be uses for external devs to, and that’s the ability to deploy and run wired tests in any already running product container.

Why? For one, it gives us the ability to test our plugins against OnDemand instances. This is already a requested feature for testing the UPM against varying environments.

For two, it should prove useful to test/approve/deny marketplace plugins. Imagine if we could automate testing marketplace entries against the products they mark as compatible…. obviously it’s only works if there are wired tests (for now) but I feel it’s a small step in the right direction for ensuring marketplace plugin quality.

So how does it work?

For now you need to have the source code of the plugin you’d like to test (this will not be a requirement in the future). From the plugin’s project directory run:

1
atlas-remote-test -Dserver= -Dhttp.port -Dcontext.path= -Dpdk.username= -Dpdk.password=

 

Property Description  Defaults
server the hostname to test against required
http.port the port of the server 80
context.path the app context path e.g. /jira /
pdk.username your username on the server admin
pdk.password your password on the server admin

Sometimes you may wish to deploy extra plugins to the remote container along with your plugin and the test framework. To do this, just add the following to your AMPS plugin configuration:

1
2
3
4
5
6
7
<deployArtifacts>
    <deployArtifact>
        <groupId>some.id</groupId>
        <artifactId>some.id</artifactId>
        <version>some-version</version>
    </deployArtifact>
</deployArtifacts>

Notes:

  1. You MUST be an admin (or sysadmin for OnDemand) on the server. If not, the upload/build will fail
  2. pdk.username and pdk.password can be set in your .m2/settings.xml so you can hide your password if needed (assuming your creds are the same on every server)

Thanks!

We welcome comments on this page and you can log any issues in these JIRA projects:

TestFramework / Console issues: https://ecosystem.atlassian.net/browse/PTRUNNER

General SDK issues: https://ecosystem.atlassian.net/browse/AMPS (please set fixfor to 4.1)

Comments (12)

  • Nice! Good write-up :). Can you add a section on server compatibility for plugins built with this SDK level? For example, can I build a Confluence plugin targeting 3.5.13 with this SDK release? Does atlas plugin install support obrs yet? Thanks.

    By Bob Swift on November 28, 2012

  • Hi, when installing the new SDK on Windows, I expirienced missing dependencies when following the tutorial. The atlas-mvn eclipse:eclipse command outputs

    Embedded error: Missing:
    ———-
    1) com.atlassian.plugins:atlassian-plugins-osgi-testrunner:jar:${plugin.testrunner.version}

    Try downloading the file manually from the project website.

    Then, install it using the command:
    mvn install:install-file -DgroupId=com.atlassian.plugins -DartifactId=atlassian-plugins-osgi-testrunner -Dversion=
    ${plugin.testrunner.version} -Dpackaging=jar -Dfile=/path/to/file

    Alternatively, if you host your own repository you can deploy the file there:
    mvn deploy:deploy-file -DgroupId=com.atlassian.plugins -DartifactId=atlassian-plugins-osgi-testrunner -Dversion=${
    plugin.testrunner.version} -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]

    Path to dependency:
    1) com.atlassian.tutorial:helloworld:atlassian-plugin:1.0-SNAPSHOT
    2) com.atlassian.plugins:atlassian-plugins-osgi-testrunner:jar:${plugin.testrunner.version}

    can somebody point me where I find the right version?

    many thanks

    By Stefan Ernst on November 29, 2012

  • @Bob – You should be able to use it to write plugins for old versions of the products, however, I can’t guarantee that the test console will look pretty if you target a really old version with a really old AUI.

    As for OBRs, the answer is: it depends. The installer tries to use UPM to install the plugin which supports OBRs (i think), however, if it can’t connect to UPM, it uses some old “trickery” to install the plugin which does not support OBR. Best bet is to give it a go.

    @Stefan Could you please log an issue with all of your details here: https://ecosystem.atlassian.net/browse/AMPS

    Thanks,

    - Jonathan

    By Jonathan Doklovic on November 29, 2012

  • I’m having the same problem as Stefan. When I try to compile I’m getting an error about ‘testrunner.jar’ missing.

    -Greg

    By Greg on November 29, 2012

  • Issue is opened here: https://ecosystem.atlassian.net/browse/AMPS-894 I don’t know if its the correct component.

    By Stefan Ernst on November 29, 2012

  • This is awesome, both the installer and custom Data Sources. Having gone through the previous 18-step tango, on those I gave it a try, but am unable to get the pom.xml correct to use a external database. Would you be willing to post or link to a fully-functional example with no ellipsis? I’m guessing I’m getting tripped up in something that’s elided in the original posting:

    Embedded error: Error configuring: com.atlassian.maven.plugins:maven-jira-plugin. Reason: Unable to parse the created DOM for plugin configuration
    com.atlassian.maven.plugins.jira.DataSource

    and a stacktrace with:
    Caused by: java.lang.ClassNotFoundException: com.atlassian.maven.plugins.jira.DataSource
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)

    Thanks!

    By Chris on November 30, 2012

  • This is really awesome! But i get the same error message as Chris, a small complete example would be awesome.

    Thanks a lot!

    By BJörn on December 1, 2012

  • Does the datasource feature work as well with Confluence ?
    I’ve tried the suggested pom from AMPS-896 and Confluence starts up fine but it’s still using the embedded HsqlDB instead of the configured postgres instance.

    By Jens Rutschmann on December 7, 2012

  • Is there a way to disable the update prompt?

    [INFO] [amps-dispatcher:debug]
    [INFO] [confluence:debug {execution: virtual-execution}]
    Version 4.1.2 of the Atlassian Plugin SDK is now available.
    Run the atlas-update command to install it.
    Press ENTER to continue:

    It’s friendly reminder, sure. But AMPS is not iTunes! Please let lhe empowered developer choose when to update.

    By Ulrich on January 2, 2013

    • Ulrich, you’ll want to add the following property to your pom.xml:
      true

      By Ben Woskow on January 4, 2013

    • It seems like my xml snippet got filtered out of my comment ;)
      “skipAllPrompts” should be set to “true”.

      By Ben Woskow on January 4, 2013