Only Software matters

Experiences in software development

Archive for the ‘agile’ Category

Agile ALM Book Review

Posted by Patroklos Papapetrou on December 26, 2013

Some time ago I was looking for some resources around the Internet in order to combine the agile principles we were already following with a more flexible application lifecycle management. I wanted to find ideas and methods to get the best out of developers’ practices with source code infrastructure and well-known and mature tools that cover all development phases. I was so lucky that I found the Agile ALM book by Michael Hüttermann. I really enjoyed reading it and applying its concepts to real projects and I wanted a long time ago to write a review about it. Now, during Christmas time, it was the best period to do it, so here it is.

The initial paragraph of the “Preface” section summarizes the author’s intent:

Welcome to Agile ALM. This book has three main goals. The first is to describe Agile
Application Lifecycle Management (ALM) in practical terms and to provide a plan for
rolling out Agile strategies and best-of-breed tools. The second purpose is to explain
how to create ALM toolboxes based on standard tools used in advanced, real-world
cases. The third goal is to give you some guidance on how to choose the best tools, use
them correctly, and integrate them while applying Agile strategies. These three goals
are the focus of this book.

The author of Agile ALM bookMichael Hüttermannand is a Community Leader an Oracle Java Champion at, the Chairman at Java User Group Cologne a freelancer and delivery engineer that helps companies with SCM/ALM, Agile, Continuous Delivery and DevOps.

The title of the topic and the table of contents were the key factors that made me (I assume this is true for other readers) read it from the very early during MEAP program offered by Manning. The topics are suitable for both junior and experienced software developers. Delivery engineers will find it very useful as well even if they have no previous experience with the concepts introduced in the book. The Agile ALM is divided into four multi-chapter parts.

Part 1 (“Introduction to Agile ALM”) introduces  the basic concepts and building blocks of an agile ALM with practical use cases and best strategies. Chapter 1(“Getting started with Agile ALM”) sets the roots by discussing the evolution of ALM and how the use of lightweight tools can boost the effectiveness of an ALM through a real-life example. Chapter 2(“Agile and ALM strategies”) starts by explaining how project management can fit in an agile environment and propose a series of strategies to implement ALM processes. The remainder of the second chapter is about finding a balance between over-controlling the ALM and not controlling it at all.

Part 2 (“Functional Agile ALM”) touches the concepts of release management and task-based development in a higher, non-technical level. Chapter 3(“Using SCRUM for release management”), as its title implies is all about SCRUM and how practitioners can apply release management using SCRUM principles. Readers, after finishing this chapter, whey will be able to implement, following a detailed step-by-step guide, a release management process. The Subversion supporting strategies that finish third chapter are the bonus included topic. Chapter 4(“Task-based development”) talks about a series of mature tools such as Jira, Trac, Bamboo and others that facilitate task-based development, bug tracking and project management.

Part 3 (“Integration and Release management”) is considered more technical as it describes techniques using a variety of tools to manage and release artifacts and apply continuous integration. Chapter 5 introduces the Maven  ecosystem and explains the process of releasing using some of the most well-known Maven features. After reading this chapter you will be able to do release management with Maven and get the foundation ideas for the rest of the chapters. Chapter 6(“Creating a productive development environment”) discusses Mockito and its important role in mocking, stubbing and isolating systems during test execution. This chapter, also, teaches us the art of creating and maintaining workspace environments for developing, integrating and testing software systems.  The rest of the chapter deals with various advanced continuous integration techniques. Although there are plenty of great books for each topic discussed in this chapter, what makes it special is that everything is placed in the context of agile ALM. Readers, especially those that have already a small experience with such processes will find it really useful to see all these well-established tools fitting in their development lifecycle.

The last Part 4 (“Outside-in and barrier-free development”) of the book moves agile ALM even further by introducing some approaches for requirements and test management. What is really cool, is that, again, in chapter 8(“requirements and test management”) these concepts are presented with practical advice and examples using testing frameworks such as Selenium, TestNG, FitNesse and others. The topic of collaborative tests was one of my favorites throughout the book. Not only it’s perfectly explained but it made me also realize how testing processes and should be considered and implemented. The last, but not least, chapter of the book (“Collaborative and barrier-free development with Groovy and Scala”) explains how it’s more productive to use different languages for development and testing activities (polyglot programming). Spock, easyb and spec2 are some of the tools presented in this chapter to describe using practical examples the concepts of Business Driven Development (BDD) and Domain Specific Languages (DSL)

Conclusion : The Agile ALM book is not just another “agile” book. The high expectations set when reading the abstract and the back page of the book were met. One might think that the large number of techniques and tools included in the book might annoy or confuse the readers but this is not happening because as already said, everything is discussed withing the context of ALM. You will learn a lot of new ideas and strategies and you will be able to adopt them immediately. Even if you’re not familiar with some of them, the book explains very nice the basics and moves directly to the approach that will solve a particular problem in the development lifecycle.

Posted in agile, book review, java, jenkins, software | Leave a Comment »

20 +2 Subjects Every Software Engineer Should Know … and the books you need

Posted by Patroklos Papapetrou on July 19, 2012

I recently read an extremely interesting and useful article about the 20 subjects that every software engineer should know or learn….
What is really cool is that it’s not restricted to products, languages but it describes generally accpepted technologies, methodologies and practices.
It applies both to  junior and exeperienced software engineers. The former have a guideline about the fields that need to focus whereas the latter have the chance to re-evaluate their knowledge.
What’s missing, IMHO, is to give the reader a clue about which are the best book(s) related to these subjects so in this post I give my advices on that. Of course the list of books is not complete and it’s just my opinion based on my experience.

Hope you find it useful as well!

1. Object oriented analysis & design

2. Software quality factors

3. Data structures & algorithms: Basic data structures like array, list, stack, tree, map, set etc. and useful algorithms are vital for software development. Their logical structure should be known.
6. Software processes and metrics
8. Operating systems basics
10. Network basics
13. Dependency management
15. ORM (Object relational mapping)
18. Internationalization (i18n)

Posted in agile, cdi, ci, continuous integration, java, quality, software, sonar, testing | 10 Comments »

Best Agile Practices Blog Articles during September 2011

Posted by Patroklos Papapetrou on October 3, 2011

For September I have chosen to present a personal collection of articles that address / discuss one or more agile practices. Estimation, CI, Continuous Delivery and much more. Enjoy!.

4 Values of the Innovative Team
What does it take to have team be successful in an innovation?  All teams operate with a set of values, usually implicitly.  In this post,Paul Boos discusses the 4 primary values that a innovative team will exhibit and according to him,  these values are what allow a team to be creative in constructing and implementing an innovation.

Estimating a full backlog based on a sample of it
My favorite article for last month. See how easy it is to have a full backlog estimation just by estimating only a portion of the total user stories. Mike Cohn offered to all agile practicioners another excellent post.

The value of stable teams
Forget about multi-tasking. Forget about multi-project assignments. Bring your team members together and make them last ( forever!!! )

Agile is NOT a game of perfect
Agile isn’t about perfection; it’s about bringing value to your organization and to your customers in the most cost-effective, team-centric, transparent way.

How does QA fit in?
Do you have a separated department of testers. How can they work smoothly in an agile team? Or even better… how can they be a part of that team?

How to introduce a Test-Driven Mindset
Test Driven Development is considered to be one of the most difficult agile practices to adopt and that is because is primarly a mindset problem and not a technology or tools issue. This post, by Mike Caspar, presents an excellent approach on how you can introduce a TDD mindset in an agile team

Continuous Deployment is Continuous Business Improvement
Why it is extremely important to achieve continuous deployment. It’s not about technology… it’s about business improvement

Continuous Delivery – Why you don’t really “get it”?
Are you sure you understand what CI is?

Feel free to add any article you found interesting in the list

Posted in agile, continuous integration, quality, software, testing | 1 Comment »

Best Java Testing Articles during Summer 2011

Posted by Patroklos Papapetrou on August 31, 2011

Starting from today I will try to post every month or so , a list of articles  that cover a specific topic and were initially posted during the last month.

For the beginning I the selected topic is Testing and covers not only August but also July. Enjoy!!

5 reasons to write tests for you code
If you are not yet convinced about the necessity of testing your code then this article is your last chance 🙂

Unit Testing for Java EE
Think testing Java EE applications is difficult, inconvenient, or too complex? Learn why this isn’t the case and how do unit testing efficiently in this article.Unit testing and mocking of Java EE 6 (CDI, JPA and EJB) applications.

Why testing code for thread safety is flawed
A blog for testing thread-safety code and the problems behind that.

Test Driven: It is the mindset not the tool!
Interersting point of view on how you have to approach TDD.

DBUnit Express Tips – Setup Simplification and Custom Data File Convention
Simple Guide for DBUnit Express. If you plan to perform Database Unit testing then drop an eye on this post

Good Unit Test Check List
Are you writing Unit Tests? Then check this list if you are on the correct path.

Unit Tests are better than excellent specification
Excellent specifications = Very refined exhaustive view of future potential features
Good unit tests
 = Very refined exhaustive view of actual features

What is the cost of Unit Tests
Simple blog showing how much you earn by writing unit tests instead of running manual tests or debugging your application

UDIAS – The five levels of an ideal agile testing strategy
The UDIAS (unit, database, integration, acceptance, stress) testing strategy is probably not a silver bullet, but it covers all the different layers and views of an application. The time needed to set running all the above build jobs is significantly large but the ROI of investment worth the effort. I don’t think that there is something more important in a system under development / maintenance than an automated testing plan.


Posted in agile, quality, software, testing | 1 Comment »

UDIAS – The 5 levels of an ideal agile and automated testing strategy

Posted by Patroklos Papapetrou on July 24, 2011

Recently my team started working on a new project which is actually a re-developing of a legacy system in a new platform. We decided to move from a windows based application to a new web-based J2EE application. It is a project that has all the odds against failure. We know the domain, we know the new technology, the team is working together many years and there is only one restriction about it. We ought to use the same database schema since it may be used in conjunction with the old windows – client. Good news since we don’t have to redesign the database. Bad news since we have to stick on some bad database smells of the past. Anyway we have the chance to complete a project on time with no serious risks. It is also a very good opportunity to apply in practice the testing strategy we dreamed for all these years. A strategy that is fully automated, repeatable with a single key press and covers all aspects of the system. We decided to use Jenkins as our build system and in the following lines I briefly describe the testing strategy

Unit  Testing

Since it is a Java system we had to choose between the two famous unit test frameworks. JUnit and TestNG. For no particular reason (maybe because we are already experienced with it) we have chosen JUnit. All unit tests run after each commit with our first Jenkins job (let’s call it app-trunk). If coverage falls below a predefined threshold (i.e <75%) the the build automatically fails,team members are notified and last commiter(s) strive to fix the build.

Database Unit Testing / Database Integration Testing

You can name the next level of testing as you wish. I prefer to call it Database Integration Testing since we test how well is our system integrated with the database. Remember that we had to keep the database unchanged so we have a double challenge here. Test our code (EJB3 entities) against an existing / productive database with real data and test our code against a database that is automatically created with sample data. If am not mistaken the only available mature framework for database testing is DBUnit. It provides some flexibility about database testing but we had some more requirements so we created a layer over it to automate things like data generation, testing of entity objects manipulation etc. Maybe in another post we will describe this layer in details. For each entity we test functions like create, edit, find, delete and some schema-related issues, such as indexes, foreign keys and primary keys. Due to the fact that there are some tables with many columns we would like to sure that all mappings (columns and relations are correct). All kinds of these tests run through an automated build-job,that is being triggered whenever app-trunk succeeds. There are two matrix jobs in Jenkins that test our code against all supported database platforms with existing and sample data. Obviously, if something goes wrong, again all team members are notified and try to fix the problem.

Integration Testing

Our system runs on an application server so we need somehow to test its behavior and how well is integrated with some services and frameworks like JSF and CDI. Here comes Arquillian to make our life much easier (with JSFUnit extensions). The idea of writing tests just like unit tests is brilliant and since we don’t have to learn a new framework we can adopt it quickly enough, from the day – one of our project. Each class that uses services within the container should be tested at this level. Same rules with unit testing about coverage apply here as well and integration tests run after the success of all database integration test jobs. We have created once again a matrix build job in Jenkins that runs all integration tests against supported platforms and application servers.

Acceptance Testing

Probably the hardest job because this is the point where we have to test our system if it meets the end-user’s requirements. It is also the step that requires the strongest hardware since the software has to be deployed in a real environment and automatically be tested for the most critical scenarios and the most used application flows. There are plenty of tools, however we have chosen to use Selenium, due to the fact that has a quite stable integration with Jenkins CI and has a very powerful add on for Firefox browser. We try to cover in depth not all possible user screens but those that are the most . All acceptance tests run during our nightly build once a day and only if there is an existing commit since last run. Although it is very difficult to test our system in all different environments we try to run the tests in the most commonly used. Obviously acceptance testing does not end with the automated build job. It is QA team’s responsibility to perform a complete acceptance test of all scenarios, but we strongly believe that core functionality should be always tested automatically to catch serious defects before QA team.

Stress Testing

Our application stores a large amount of data and is accessed by many users so we have to ensure that its performance is acceptable by them and does not fall below some predefined thresholds. Without any automation it would be very hard to achieve this, so to conquer the last frontier of testing we have used Apache JMeter. Although JMeter has no tight integration with Jenkins we have created a different project (JAR) that includes all performance and stress tests, triggered also during our nightly build job. Stress testing is not our number one priority, however at the end of each iteration we evaluate the usefulness of our existing tests and we modify them accordingly if some serious requirements change has occurred during the last iteration.

The UDIAS (unit, database, integration, acceptance, stress) testing strategy is probably not a silver bullet, but it covers all the different layers and views of an application. The time needed to set running all the above build jobs is significantly large but the ROI of investment worth the effort. I don’t think that there is something more important in a system under development / maintenance than an automated testing plan.

Thanks for reading this post and feel free to rate it, post your comments or share it with others.


Posted in agile, ci, continuous integration, jenkins, software, testing | 4 Comments »

Get Your Customers On Board – Continuous Feedback with Demos

Posted by Patroklos Papapetrou on May 14, 2011

Most of us,during the development of a software system, have probably received some major changes requests from customers near the end of the project. Even worse, during a demo session, the end users as soon as see (for the first time) the final(?) product jump out of the window screaming and cursing you and your team:”This is not what I wanted. I don’t like it. Get it back and bring me something else”. What exactly is this “something else”? No one is able to explain it and at this point of the project being under budget/time pressures it is something extremely difficult to clarify. With a simple (greek) word: A CATASTROPHE. If you find your self smiling at this moment then at some time in the past you have been in such a position and of course, for that we usually blame this unsatisfied and indecisive creature that we call “the customer”. No matter how many times you have tried to “freeze” the requirements in the early stages of the project either because you had a fixed-price contract, or because “we need to know in advance what we are going to develop”, the only thing you achieve is to “lose” the customer from the beginning of the project and obviously, this is not what you would expect. So, are we really unprotected against to our customers? Of course not. Our purpose is, or at least should be, the delivery of a system that covers as many as possible, and always within budget, customer business needs. One of the agile practices I like and I believe that help towards this aim is the one i simply present (for agile dummies) in this blog.

Some simple rules and guidelines

Frequent demos for immediate feedback are extremely imporant and come to fill the gap between the development team (what is implemented) and end users (what they really want from the system).The fact that from the early stages of the project, the customer  participate at regular intervals in such meetings, ensure that critical requirement changes and misunderstandings are identified as soon as possible. Thus the implementation cost is far lower than the scenario where the same change is “discovered” much later when the project would be in advance.As for each agile practice, however, frequent demos should follow some rules and guidelines in order to get the maximum of the. The following brief instructions are  mainly focusing to projects where there is a single customer.

  • Since the beginning of the project, therefore, describe in details the practice you intend to follow and try to explain its return of investment. Make clear that these demos have a single purpose. To understand early in the project what are the actual needs of end users and get as soon as possible feedback on the track you follow, so if something is extremely wrong you have adequate time to fix it.
  • If there is some kind of dissatisfaction about the large number of releases, it is good to clarify that these are not real releases, but internal demos, which will be not available to all users. There are nevertheless valuable for the early diagnosis of requirements misunderstandings.
  • Respect customers’ free time. Try to find an approach that will make everyone happy. If the customer can not attend a demo at the end of each iteration (for example every two or three weeks) try to make it at the end of each month. This, IMHO, is the maximum allowed period between two demos.
  • Try to ensure that you really have to show something. New features, modified or redesigned screens are essential in a demo and should work with no problems. If the customer is frustrated by a non-working demo, then it is most likely that he loses confidence in the development team and distrusts any attempt for future meetings.
  • The demo should be done only by development-team members so that he/she will show the customer features that need his/her attention and feedback. Do not let the customer “play” with the release. Politely remind him the purpose of this meeting and that the demo version of the system is not suitable yet for beta testing.
  • Remember to make clear before each demo that the product/system is still under development and that this is NOT what the end users will get. Try to focus on core and critical business requirements so you can get important feedback, but this does not mean that you ignore proposals for the usability, the user interface, etc.

Product vs Project

What happens though, when we do not have a single customer, but our product is targeting thousands of customers? Obviously, the frequent demo aproach seems to be unrealistic or it may be applied to a limited extent. For that reason, I propose below a couple of alternative actions that will replace this practice aiming at the same results (get early feedback).
  • Try to create videos that demonstrate the new feature of the upcoming release- several days before its official announcement. Something like a pre-release announcement.The main advantage of this approach is that with minimum cost, we can briefly show (the duration of the video should not exceed 2-3 minutes), the core features for that we would like to get feedback and comments from our customers, before the new version is actually released
  • Try to provide a try-it-live demo site where customers (existing and potential) are allowed to test the new upcoming version (Beta versions, Release Candidate versions) and send with a predefined way comments and feedback.
  • Try to gather some of the customers (if they are located in some or nearby places) or a webinar for long distance-customers and perform a demo session instead. Although this will not have the same results as if you would showing the product in a single customer, but you may get some important feedback.
Some of the practice’s benefits are summarized right below:
  • Early detection of misunderstood requirements.
  • Correction of the project “path” at a very low cost.
  • Continuous communication with the customer.
  • Increase confidence between the development team and end users.
  • Creative collaboration to achieve the same purpose (meaningful and working product)
  • Most critical and core features are delivered first and complete without having to reach at the latest phases of the project.
Probably the above list is not completed, so if you have some more ideas (remember this post is for agile beginners and not experienced users) feel free to post your comments.

Posted in agile, demo, feedback, frequent | Leave a Comment »

%d bloggers like this: