Summary of the Schneide Dev Brunch at 2012-10-14

Two weeks ago, we held another Schneide Dev Brunch. The Dev Brunch is a regular brunch on a sunday, only that all attendees want to talk about software development and various other topics. If you bring a software-related topic along with your food, everyone has something to share. The brunch was so well attended that we had trouble to all find a chair and fit at the table. We had to stay inside as the weather was rainy and too cold for prolonged outdoor sessions. Let’s have a look at the main topics we discussed:

Work hard, play hard

The first topic was a summary of the contents of the documentary movie “work hard play hard” about our modern work places. The documentary is a recommended watch for everyone thinking about joining this side of the industry. It’s beautiful sometimes and very painful to watch most times. You might cherish some of the rougher edges on your workplace afterwards. The DVD is out now.

Dual Monitoring

A short discussion about the efficiency increase that happens just by adding another monitor to your desk. There was no dispute: If you don’t at least try it, you waste money. That’s what I meant when I blogged about the second monitor being an profitable investment. Just one downfall, it shouldn’t end like this.

Management by Directive

Another discussion about the management of large departments. The “directive issuer” manager style is a common sight in this environment. I won’t repeat the discussion itself, but rather add an amusing story about an ex-military commander running a software development company. Enjoy!

Review of the Sneak Preview “Quality Assurance Best Practices in Karlsruhe”

There was a “sneak preview” organised by the VKSI, a local association of software engineers a few weeks ago. The topic of the whole event was “Quality Assurance Best Practices in Karlsruhe“. The event was divided into three independent presentations with different topics:

• “Non-Functional Software Tests” by Gebhard Ebeling: The talk was about realistic load- and performance testing of complex applications (and websites). While the presentation omitted tools and code completely, there were some take-aways even for developers that had never performed these types of tests before. This was arguably the best presentation of the event.
• “Contracts im Software Engineering” by Ben Romberg and Stefan Schürle: This talk was about the benefits of software contracts (think about checked method or class invariants) and the presentation of a particular implementation in Java, namely C4J. The perceived problem with this solution was the rather clumsy source code necessary to define the contracts.
• “MoDisco Software Modernization & Analysis” by Benjamin Klatt: MoDisco builds a model out of source code that is detailled enough to apply meaningful transformations to it and have the exact same source code (plus transformed code) as output. The idea looked very promising, but the presentation lacked actual source code examples. Nonetheless, MoDisco proves that there is a future for modell-driven analysis.

We had a lengthy discussion about software contracts and Design By Contract (DBC) in general. One tool that got mentioned several times was “CoFoJa” from (at least initially) Google.

Book review: Java Application Architecture: Modularity Patterns with Examples Using OSGI

In the rather new book of the Robert C. Martin signature series, Kirk Knoernschild tackles the hard task to teach software architecture through a book. One participant read the book and is very happy about the experience and insight he got from it. The book itself is repetitive at times, but that adds to the accessibility of the topic at hand when you jump right into a chapter. Additionally to the modularity and architecture aspects, you’ll learn OSGI through the code examples. This books gets a recommendation.

Book review: ATDD by Example

Another new book is from Markus Gärtner, of the Kent Beck signature series this time. It takes the reader by the hand and shows a way to use Cucumber, FitNesse and of course Behavior-Driven Development as a tool-and-process framework to implement (Acceptance-) Test Driven Development. None of our participants read the book fully yet, but it’s already a promising start. If you are looking for a new book about testing (after having read the great GOOS book), don’t hesitate. Another recommendation to read.

Visitor design pattern breaks modularization

One participant brought up the problem that he wanted absolute modularization in his application layout, but used a visitor design pattern at some central place. This breaks modularization, as the type information is exposed too much. We discussed the problem with some diagrams and sketches and came up with several alternatives, each with their own advantages and drawbacks. That was a great code design session among seasoned professionals.

Why are services included into Grails?

Another discussion was about the Grails web framework and the necessity for a service layer or service classes explicitly. We sketched out the fundamental architecture of a Grails application and discussed different possible alternatives to a dedicated service layer. There are some nice features about Grails services (like injection by convention, transaction and scope), but nothing really too sophisticated to distinguish them from POGOs. The discussion was open-ended, as usual with complex topics.

Review of a workshop on agile software-engineering

Lately, a participant visited a workshop on agile software-engineering, focussing a lot on SCRUM and XP. The workshop ran for several days and included lots of hands-on exercises. The workshop itself provided not much new content for seasoned agile developers, but served as an accurate and thorough introduction for younger developers. A major part of the workshop were social aspects of agile environments. Concepts like team empowerement are usually not taught in technical workshops. Important additional topics comprised of agile planning and estimation and proper retrospectives. The workshop itself was more of a entry-level introduction to agile development, but very effective in that regard.

Epilogue

As usual, the Dev Brunch contained a lot more chatter and talk than listed here. The high number of attendees makes for an unique experience every time. We are looking forward to the next Dev Brunch at the Softwareschneiderei. And as always, we are open for guests and future regulars. Just drop us a notice and we’ll invite you over next time.

Grails and the query cache

Look at the following code:

class Node {
  Node parent
  String name
  Tree tree
}

Tree tree = new Tree()
Node root = new Node(name: 'Root', tree: tree)
root.save()
new Node(name: 'Child', parent: root, tree: tree).save()

What happens when I query all nodes by tree?

List allNodesOfTree = Node.findAllByTree(tree, [cache: true])

Of course you get 2 nodes, but what is the result of:

allNodesOfTree.contains(Node.get(rootId))

It should be true but it isn’t all the time. If you didn’t implement equals and hashCode you get an instance equals that is the same as ==.
Hibernate guarantees that you get the same instance out of a session for the same domain object. (Node.get(rootId) == Node.get(rootId))

But the query cache plays a crucial role here, it saves the ids of the result and calls Node.load(id). There is an important difference between Node.get and Node.load. Node.get always returns an instance of Node which is a real node not a proxy. For this it queries the session context and hits the database when necessary. Node.load on the other hand never hits the database. It returns a proxy and only when the session contains the domain object it returns a real domain object.

So allNodesOfTree returns

• two proxies when no element is in the session
• a proxy and a real object when you call Node.get(childId) beforehand
• two real objects when you call get on both elements first

Deactivating the query cache globally or for this query only, returns two real objects.

Grails 2.0.0 Update: Test Problems

Grails 2 has some nice improvements over the previous 1.3.x versions and we thought we give it a try. Upgrading our application and its 18 plugins went smooth (we already used the database migration plugin). The application started and ran without problems. The better console output and stacktraces are a welcomed improvement. So all in all a pleasant surprise!
So just running the tests for verification and we can commit to our upgrade branch. Boom!

junit.framework.AssertionFailedError:
No more calls to 'method' expected at this point. End of demands.

Looking at the failing unit test showed that we did not use any mock object for this method call. Running the test alone let it pass. Hhhmm seems like we hit GRAILS-8530. The problem even exists between unit and integration tests. So when you mock something in your unit test it is also mocked in the integration tests which are run after the unit tests.
Even mocking via Expando metaclass and the map notation did not work reliably. So upgrading for us is not viable at the moment.

HTTP Get: The problem with Percent Encoded Parameters

Encoding problems are common place in software development but sometimes you get them in unexpected places.
About the setup: we have a web application written in Grails (though the choice of framework here doesn’t really matter) running on Tomcat. A flash application sends a HTTP Get request to this web application.
As you might know parameters in Get request are encoded in the URL with the so called percent encoding for example: %20 for space. But how are they encoded? UTF8?
Looking at our tomcat configuration all Get parameters are decoded with UTF8. Great. But looking at the output of what the flash app sends us we see scrambled Umlauts. Hmmm clearly the flash app does not use UTF8. But wait! There’s another option in Tomcat for decoding Get parameters: look into the header and use the encoding specified there. A restart later nothing changed. So flash does not send its encoding in the HTTP header. Well, let’s take a look at the HTTP standard:

If a reserved character is found in a URI component and no delimiting role
is known for that character, then it must be interpreted as representing the
data octet corresponding to that character's encoding in US-ASCII.

Ah.. US-ASCII and what about non ASCII ones? Wikipedia states:

For a non-ASCII character, it is typically converted to its byte sequence
in UTF-8, and then each byte value is represented as above.

Typically? Not in our case, so we tried ISO-8859-1 and finally the umlauts are correct! But currency signs like the euro are again garbage. So which encoding is similar to Latin-1 but not quite the same?
Yes, guess what: cp1252, the Windows native encoding.
And we tested all this on a Mac?!

Grails Gems: Command Objects

Besides domain objects command objects are another way to get validation and data binding of parameters. But why (or when) should you use them?
First when you do not want to persist the data. Like validating parameters for a search query.
Second when you just want a subset of the parameters which has no corresponding domain object. For example for keeping malicious data away from your domain objects.
Third when you get a delta of the new data. When you just want to add to a list and do not want to check if you get a single or a multiple value for your a parameter.

Usage

Usually you put the class of the command in the same file as the controller you use them in. The command object is declared as a parameter of the action closure. You can even use multiple one:

class MyController {
  def action = { MyCommand myCommand, YourCommand yourCommand ->
    ...
  }
}

Grails automatically binds the request parameters to the commands you supply and validates them. Then you can just call command.hasErrors() to see if the validation succeeded.

Grails: Beware of the second level cache

Recently we were hunting a strange bug. Take the following domain model:

class Computer {
  Coder coder
}

class Coder {
  static hasMany = [projects:Project]
}

Querying the computer and iterating over the respective coder and projects sometimes resulted in strange number of projects: 1. Looking into the underlying database we quickly found out that the number of 1 was not correct. It got even more strange: getting the coder in question via Coder.get in the loop yielded the correct results. What was the problem?
After some code reading and debugging another query which was called after the first one but before accessing the coder in the loop gave some insight:

  Coder.withCriteria {
    projects {
      idEq(projectId)
    }
  }

This second query also queried the Coder but constrained the projects to a specific one. These coders were populated into the second level cache and when we called computer.coder the second level cache returned the before queried coder. But this coder had only one project!
Since we only needed the number of coders with this project we changed the second
query to using count, so no instances of Coder are returned and thus saved in the second level cache. Bug fixed.

A big benefit of Convention over Configuration

Recently I joined a long running Grails project and had to complete some issues in a short time. After a quick introduction I was ready to dive in and instantly could chunk out code. Convention over Configuration helped a lot getting a fast start:

• Which classes are persisted? Look into the domain folder.
• What controller handles xyz? It is named after xyz.
• What is the page that is served for URL u? Look into the corresponding folder and find the view named accordingly.

IMHO this is a benefit that many conventions are determined by the framework. If you know the framework, you know the layout of the project. The opposite holds also true: if the project departs from these “standards” you have to look closely into the configuration.
So think twice before you do something your way. Sometimes you have no choice like when you are using Oracle and have to cut all table and column names to 30 characters. But normally you should keep the defaults.

Grails: The good, the bad, the ugly

After 3 years of Grails development it is time to take a step back and look how well we went.
(Info: we made several Grails apps ranging from small (<15 domain classes) to medium sized (50-70 domain classes) using front ends like Flex/Flash and AJAX)

The good parts

Always start with praise. So I tell you what in my opinion was and is good about developing in Groovy and Grails.

Groovy is Java with sugar

The Groovy syntax and the type system are so close to Java, so that when you come from a Java background you feel right at home.

Standard web stack

If you are accustomed to standard technologies like Spring and Hibernate you see Grails as a vacation.

Sensible defaults aka Convention over configuration

Many of the configuration options are filled with sensible defaults.

Fast start

You get from 0 to 100 in almost no time.

The bad things

Things which are not easily avoided.

Bugs, bugs, bugs

Grails has many, many bugs, unfortunately even in such fundamental things such as data binding and validation. A comment from a previous blog post: “To me, developing with Grails always felt like walking on eggs.”

Regression

Some bugs sneak back in again or are even reopened. Note that this is not the same as bugs, bugs, bugs because fixed bugs should be secured by a test.

Leaky abstractions

You have to know the underlying technologies especially Hibernate and Spring to get a foot on the ground. The GORM layer inherits all the complexity from Hibernate.

Slow integration tests

The ramp up time is 45 s on a decent build/development machine and then the first test hasn’t even started.

Uses the Java way of solving problems

Got a problem? There’s a framework for that!

Abandoned or prototype like plugins

Take a look at the list of plugins like Autobase, Flex.

Problems with incremental compiling

Don’t know where the real cause is buried: but using IntelliJ for developing Grails projects results in comments like:
Not working? Have you cleaned, invalidated your caches, rebuilt your project, deleted the .grails directory?

The ugly things

Things which are easily avoided or just a minor issue.

Groovys use of == and equals

Inherited from Java and made even worse: compare two numbers or a String and a GString

Groovys definition for the boolean truth

0, [], “”, null, false are all false

Groovys use of the NullObject and the plus operator

Puzzler: what is null + null ?

Uses unsupported/discontinued technologies

Hibernates SchemaExport comes to mind.

Mix of technology and intention

hasMany, hasOne, belongsTo have not only an intention revealing function but also determine how cascading works and the schema is generated.

Summary, opinionated

Grails has deficits and is bug ridden. But this will be better in the future (hopefully).
When you compare Grails with standard web stacks in the Java world you can gain a lot from it.
So if you want to know if you should use Grails in your next project ask yourself:

• do you have or want to use Spring and Hibernate?
• can you live without static typing? (remember: with freedom comes responsibility)
• are you ready to work around or even fix an issue or bug?
• is Java your home?

If you can answer all those questions with Yes, then Grails is for you. But beware: no silver bullets!

The Grails performance switch: flush.mode=commit

– Disclaimer –
This optimization requires more manual work and is error prone but isn’t this with most (big) performance improvements?
For it to really work you have to structure your code accordingly and flush explicitly.
–
Recently in our performance measurements of a medium sized Grails project we noticed a strange behavior: every time we executed the same query the time it took increased. It started with 40ms and every time it took 1 ms more. The query was simple like Child.findAllByParent(parent)
The first thought: indexes! We looked at the database (a postgresql db) and we had indexes on the parent column.
Next: maybe the session cache got too large. But session.flush() and session.clear() did not solve that problem.
Another post suggested using a HQL query. Changing to

Child.executeQuery("select new Child(c.name, c.parent) from Child c where parent=:parent", [parent: parent])

had no effect.
Finally after countless more attempts we tried:

session.setFlushMode(FlushMode.COMMIT)

And not even the query executed in constant time it was also 10x faster?!
Hmmm…why?
The default flush mode in Grails is set to AUTO
Which means that before every query made the session is flushed. Every query regardless of the classes effected. The problem is known for hibernate but after 4! years it is still unresolved.
So my question here is: why did Grails chose AUTO as default?

Open Source Love Day October 2010

On Friday two weeks ago, we held our Open Source Love Day for October 2010. This day was special in several ways. We strayed very far from the usual schedule for this day, there were several internal tasks that couldn’t be delayed and we introduced a “fun practice” event. But we eventually produced something valuable this day.

The Open Source Love Day

We introduced a monthly Open Source Love Day (OSLD) to show our appreciation to the Open Source software ecosystem and to donate back. We heavily rely on Open Source software for our projects. We would be honored if you find our contributions useful. Check out our first OSLD blog posting for details on the event itself.

The distractions

• A regular project needed an urgent cost estimation by the whole team. This was the last opportunity because of an upcoming parental leave to have the team together for a long time.
• Another regular project needed an urgent problem solved. This turned out to be so obscure that one of our developers had to be on-site. You can read about it in this blog entry now.
• We received several shipments of office furniture and computer parts. They had to be checked and placed in.
• We had a fun practice event. We discovered the online “game” typeracer and practiced our raw typing skill against each other for some time. Pro tip for beginners: don’t look at the highscores!

On this OSLD, we accomplished the following tasks:

• A new version 1.8 of the cmakebuilder hudson plugin implements several feature requests. You can now choose to NOT clean your workspace before building and set different paths for the cmake installation for every job or node (hudson slaves). The latter option can be applied using an environment variable.

On this OSLD, we also tried the following tasks:

• We keep an eye on Scala and its associated web framework Lift as a promising technology. One issue with Lift that bugs us is the use of “sun bastard format” properties for internationalization. We tried to teach Lift to accept UTF-8 encoded property files. After a lot of “downloading the internet” (you can always tell which project uses maven by their initial setup delay), we quickly implemented our own ResourceBundle.Control. But the Lift framework itself could not be built: “Error occurred during initialization of VM: Could not reserve enough space for object heap”. We ran out of time and will investigate in this issue on the next OSLD.
• Grails is another web framework we use in projects. There are some bugs that really annoy us, and the OSLD is the perfect time to fix them. One of these bugs is GRAILS-6475, which we tried to reproduce with the latest code base. After writing a test case that would go green unexpectedly, we tried to provoke the error by setting up a sample project. The bug didn’t show up there, too. We left a comment in the issuetracker and ceased development.

What were our lessons learnt today?

• You can’t tear off massive amounts of time from the OSLD and expect it to still be working. An OSLD doesn’t scale down apparently.
• Most issues that can’t be done fail with the project’s build. The build process of a foreign project is the most crucial phase in your decision on commitment. If it fails, your participation in the project is at risk. We’ve seen many brittle, undocumented and incredible complex build processes now. And we can state one thing: It doesn’t stop with throwing maven at a project, you still have to “think the build”.

Retrospective of the OSLD

This OSLD was special in the amount of non-OSLD work done. The remaining efforts weren’t as successful as we wished. This has been an ongoing issue with our OSLD for the last months now and we are looking forward to adapt our workstyle to yield better results in the future. The distraction by typeracer was fun, though.