Maven in our development process. Part 4. Remaining issues
March 6, 2008 2:24 AM- The requirements for our Maven process.
- The infrastructure we have set up.
- The project configuration details.
As I went along, I also described some of the reasons for each part and some benefits. However, all of the work above does not guarantee that our process works smoothly every time. We do hit problems every now and again. Here, Samuel Le Berrigaud, James Dumay and I share some of the most frequent and some of the most sore issues we have to deal with from time to time.
Maven users need training.
We believe this is the most important and most overlooked issue of all.
One problem is if you are migrating projects from Ant or Maven 1 to Maven 2. Your developers used to know their build system very well and all of a sudden they lose control.
Another issue is when the problems occur - developers with little Maven exposure find it difficult to understand what is going on and how to fix it. Maven concepts need to be understood in order to deal with their issues.
Yet another issue is that Maven 2 comes with some very different concepts from either Ant or Maven 1. Some of those differences are lifecycle, working with Maven plugins and the dependency management for example. Those have to be explained carefully to the users. Otherwise your team of Maven "experts" will spend their time answering the same basic Maven 2 questions multiple times a day.
Here at Atlassian we started our Maven training sessions. The goal is to explain Maven philosophy and concepts, provide how-tos for most of the daily tasks and do away with some of the the myths. So far the training sessions are turning out to be successful.
Check your settings.xml
This is something we had to say a lot to our developers. As it was described in Part 2 of these series we have a few repositories - internal and external - that, combined with the proxies, we use to download from and deploy artifacts to. We also said that in order to benefit from this infrastructure developers must configure their Maven environment correctly and that we have a pre-configured settings.xml file available. The overwhelming majority of the cases when someone comes to us with: Maven can't find my artifact or I get authentication issue when deploying my artifact get resolved by making sure their settings.xml is correct.
IDE integration
The declarative nature of Maven allows for better integration with IDEs than other tools like ANT. This is true for Maven 1 and remains the same for Maven 2.
So for our developers running mvn idea:idea or mvn eclipse:eclipse at the root of their project should setup their development environment. It works well for most smaller projects and projects where Maven conventions are followed. However some teams have very complicated project structures where those plugins don't really do a good job. Having the possibility of setting up a development environment in the minimum amount of time is really a strong point in favour of Maven 2. However it seems that we will need to do more work on some projects to get them to the level of just using one command line for that.
IDEs are now coming with integrated Maven support as well. But they all have the same shortcomings as the Maven plugins and we won't be able to it use in all our projects. The best Maven support we have seen so far is definitely the one in Netbeans. IntelliJ IDEA has some good support as well but not quite to the level of Netbeans. As usual with Eclipse there are several plugins to choose from but none of them is quite as good as NetBeans.
Profiles
Build profiles are are a great idea. A lot of times profiles are used where they shouldn't be. This, unfortunately, makes us think that profiles are evil.
Profiles are useful when something needs to be activated seamlessly depending on user environment (JDK, OS, etc.). This seems reasonable. Another use of profiles is changing the database or an application server that you are going to be using to run your functional tests. Activating the profile on the command line makes sense here. But one needs to be able to run the same tests with a default environment out of the box without activating any profiles.
It is wrong to use a profile to activate the module which is responsible to functional testing. Such setup is tempting for it takes a long time to run functional tests and you might not want to be running them every time you run Maven. The problem, however, is that if you are going to do a release you have to remember to include this module by activating the profile. Very often this is forgotten.
Use of profiles is also not obvious unless you are intimately familiar with this projects setup. Therefore, when you try to run a Maven goal and it fails you are thinking that Maven is broken and get frustrated whereas you simply didn't invoke the right profile.
Group Id change
Each maven project (artifact) is uniquely defined by its group id and artifact id. Arguably the version is also part of the identifier.
Between Maven 1 and Maven 2 the conventions for group ids have changed. The convention (if it was a convention) in Maven 1 was to use the same value as the artifact id. However in Maven 2 group id looked more likes package name. For example if you take Atlassian Seraph its groupId:artifactId was atlassian-seraph:atlassian-seraph in Maven 1 and we changed for com.atlassian.seraph:atlassian-seraph in Maven 2.
This had a serious impact on projects that depend on this library. The greatest impact was when dealing with dependency management. As far as Maven 2 is concerned atlassian-seraph:atlassian-seraph and com.atlassian.seraph:atlassian-seraph represent two different libraries. This made it easy to end up with one of each on the classpath of the project.
This is not something your developers are going to enjoy. This is especially true because when you exclude the dependency from somewhere, it has the tendency of coming back from somewhere else. This will still be the case until someone fixes MNG-1977.
We migrated the group ids of almost all of our libraries pretty much at the same time. The more libraries you migrate - the worse the issue gets. Our first advice is to do it one library at the time. And don't start migrating it until you have all your project up to date with their dependencies - i.e. using the latest stable version.
Here is another issue with group ids. When the we took over the maven-clover-plugin code from the apache community it was decided to change the group id from org.apache.maven.plugins to com.atlassian.maven.plugins. While this seemed like a good idea it caused its own share of issues.
The biggest issue here is that if one is currently using the clover plugin on the command line, Maven will be using the old version of the plugin. This is true even if you define the Atlassian pluginGroup in your settings.xml or the exact plugin to use in the pluginManagement section of your POM. If you want to make it work you would have to use either the plugins section of your POM or the long command line as per mvn com.atlassian.maven.plugins:maven-clover-plugin::clover. We hope to fix this issue soon though.
Archiva issues
Out of all the proxy servers we have picked Archiva because it matched our requirements the most. However, when we started using it — it wasn't even version 1.0. We have discovered a lot of issues with it and Archiva guys so far have been very quick to address them. James Dumay also has adopted Archiva and has been contributing back to the project. It is slowly shaping up to be what we want in terms of stability, performance and even feature-wise. However, since it is a single most important part of our infrastructure, every time something goes wrong with it, everyone is affected.
Summary
These are some of the issues we have remaining with our Maven setup. On the grey side - they still cause some irritation for everyone. On the bright side, they are becoming less and less frequent.
We believe that after everyone had been properly introduced to Maven through our training sessions they will feel more confident dealing with Maven.
We also think that now we are dealing with issues that are related to our usage of Maven rather than Maven variables that we had no control over.
On the other hand — we would love to know what is your Maven experience like. What issues are you dealing with? What problems have you solved with Maven and in what way?
Since after the previous part we received some very valid comments about using Maven for our releases, we decided to write up another entry dedicated to the subject. It will be posted in the next week or two so stay tuned for more Maven lovin' and goodness!



Copyright © 2009 Atlassian Pty Ltd.

12 Comment(s)
Hey,
I just wanted to thank you for this great series of blog entries on Maven in corporate environment. That's something that would actually deserve to be included in one of the books on Maven in my opinion.
And I'm flattered by your opinion on the NetBeans integration. :)
Regards
Milos
By Milos Kleint at March 7, 2008 5:53 AM
Milos,
thank you. I am flattered back that you liked our material.
I also hope that it makes other people's Maven experience better.
cheers,
Sherali
By Sherali Karimov, X-Team Lead at March 7, 2008 10:39 AM
Hi,
Fantastic 4 piece article!
We've migrated to maven from a complex ant build process and the "command line version" works just great. A few tweaks here and there and some headaches as well to get everything sorted, but in the end it's better than before.
However, the IDE integration is ... well quite bad (in eclipse at least).
We've used m2eclipse and thinking on trying q4e as we have quite a lot of projects and people don't have all the projects checked out and are opening and closing them all the time.
How do you guys run eclipse:eclipse? Do you run it on every project you open/checkout?
Thanks,
Pedro
By Pedro Oliveira at March 12, 2008 2:55 AM
Pedro,
I am glad you liked the articles.
As for IDE integration, indeed it leaves much to be desired in a way of Maven integration at the moment. If you are IDEA or Eclipse user anyway.
I am not sure I understand your question. If you are asking whether we commit .project and .classpath files to SVN or not, the answer is no. We generate the IDE projects when we are setting up a new environment. This does not happen too often simply because it takes a fair bit of fiddling to tune your IDE environment even after generating it with maven. In fact I am currently researching making this simpler. I will post our findings.
As far as Eclipse plugins are concerned q4 and m2eclipse both seem to be improving but which one will get there sooner - I am not sure. I am just a bit sad that all of that great effort is being duplicated all the way.
By Sherali Karimov at March 12, 2008 3:32 PM
Sherali,
Example:
Parent Project
Project 1
...
Project N
In my team we usually check out only the projects that we required. The other ones are retrieved from the repository. The problem is that the all close/open causes a lot of problems in both eclipse plugins. Sometimes it does not add the new project to the classpath of the existing projects (as an eclipse projec), other times, the errors in dependant projects are not cleared, etc, etc, etc...
I tried to use the maven-eclipse-plugin, and it seems to work a lot better regarding this. The only issue is that you need to run eclipse:eclipse every time you checkout or open a new project.
Also, with maven-eclipse-plugin you can easily add stuff like checkstyle, pmd, etc... to you eclipse project without having to do it on every one of them manuall!
Regards,
Pedro
By Anonymous at March 13, 2008 2:02 AM
Pedro,
I understand a bit better now. In our case when we have a few projects that are parts of a bigger project (in other words if there is a parent to a few projects) we use 'module' feature of maven. Child projects are modules. mvn idea:idea or mvn eclipse:eclipse both support modules. Maven release process works well with the modules.
If the number 'N' is large, perhaps it is a good idea to split it into a few projects that have modules. Having a large number of child projects (i.e. modules) might not be a good idea.
As far as various maven plugins are concerned - I agree, there is a lot of work to be done and none of them do everything we want yet. We are also researching into this problem and might start contributing to maven IDEA plugin and one of the more promising maven Eclipse plugins to resolve this issue. I will blog about it as soon as we figure something out..
Having said that - any ideas or experience people have with this issue is always welcome since we hate duplicating effort if we can avoid it. :-)
cheers,
sherali
By Sherali Karimov, X-Team Lead at March 13, 2008 7:03 PM
Hi,
Great articles !
I have a question about the source code.
In your environment, how do you manage the source code ? Do you use Subversion or a similar product ? If it's the case how do you open the source code to the extern user ?
Thanks
By Lo at March 27, 2008 12:25 AM
Indeed we use Subversion.
We have two separate repositories. One of them is publicly available and hosts our open source libraries.
Within this repository we have two separate sections - contrib and atlassian. External people have write access to the public/contrib area and only a read-only access public/atlassian/
cheers,
sherali
By Sherali Karimov, X-Team Lead at March 27, 2008 1:26 AM
Thanks for sharing how Maven is used at Atlassian. Very informative.
I've a similar setup at my work place. Differences - we use CVS and Artifactory. I've been playing with Nexus (proxy server) for a couple of days now. I like it better over Artifactory and Archiva.
Keep up the good work with Confluence, JIRA, Bamboo, Fisheye, Crucible - we use them all and love it.
I would love to hear how Atlassian does releases using Maven.
By Rosh at April 23, 2008 6:13 AM
Tried buildr?
By Dave at May 27, 2008 4:53 PM
Hi,
I just wanted to thank you for this great series of blog entries about how Atlassian uses Maven.
And I also think this informations are "missing" in one of the official Maven documentations.
Finally, I agree that Netbeans contains by far one of the best Maven IDE integrations. ;)
By Frank-Michael Jaeschke at June 7, 2008 3:41 AM
This is an awesome set of articles! Thank you (and Atlassian) for making them available to the public.
I think the final part would be to share the contents of the training sessions if it is in addition to what is here.
Thanks again,
Fred.
By Fred at November 28, 2008 7:55 AM