The state of functional testing in Flex

Functional testing of UIs is an important and often neglected way of ensuring quality and prevent regression. The Flex world of functional tests seems at the very beginning. We evaluated some of the tools available and used the following criterias:

• OS independence
  the tool and the created test scripts should run on at least every platform the Flex SDK and the Flash platform are available
• Tool changes
  how much we need to change or adapt the tool to suit our needs
• Code pollution
  how much the actual code needs to be polluted to support this testing tool
• Capture/Replay
  the tool needs at least an option to capture and replay test scripts
• Additional License Costs
  if we need to pay additional (besides the tool) license costs for things like the FlexBuilder Pro

	OS ind.	Tool changes	Code pollution	Capture / Replay	Add. costs
Automation based tools	+	-	0	+	-
SeleniumFlex	+	0	+	0	+
FunFx	-	0	+	-	0
Fluorida	+	-	+	-	+

Automation based tools (like FlexMonkey, QTP and RIATest) use the Flex automation API and have additional costs for FlexBuilderPro (700$ per license). For custom components you have to add automation code to them (pollution) and introduce them and their events in FlexMonkeys event map (tool changes).
SeleniumFlex uses the JavascriptBridge (ExternalInterface) of the FlashPlayer and needs you to add the custom components and events to this external interface which resides in the tool/test code (therefore a 0 at tool changes). You can use the Selenium plugin for spy (the ids)/replay but the capture option isn’t working yet (0 for capture/replay).
FunFx also uses the ExternalInterface and is written in Ruby but runs only on Windows (- for OS independence) because it connects to the Flex application via Win32OLE. I found no capture/replay (-) and the website says you need FlexBuilder (I don’t know why therefore a 0 for license costs, we use IntelliJ IDEA for Flex development)
Fluorida seems to be at the beginning and there is very little documentation so it looks like to need an investment (- for tool changes). It has no capture/replay (-).

Conclusion

So our tool of choice is SeleniumFlex and we hope to get capture/replay working in the near future.
What experience have you made with functional testing in Flex? Which one do you use?

---

Possibly related posts: (automatically generated)

• Review of Effective Software Test Automation: Developing an Automated Softw…
• Choose Test Tools that Integrate With Existing Testing Setup
• A short rant about test automation tools