Functional testing of UIs is an important and often neglected way of ensuring quality and prevent regression. The Flex world of functional tests seems at the very beginning. We evaluated some of the tools available and used the following criterias:
- OS independence
the tool and the created test scripts should run on at least every platform the Flex SDK and the Flash platform are available
- Tool changes
how much we need to change or adapt the tool to suit our needs
- Code pollution
how much the actual code needs to be polluted to support this testing tool
the tool needs at least an option to capture and replay test scripts
- Additional License Costs
if we need to pay additional (besides the tool) license costs for things like the FlexBuilder Pro
|OS ind.||Tool changes||Code pollution||Capture / Replay||Add. costs|
|Automation based tools||+||–||0||+||–|
Automation based tools (like FlexMonkey, QTP and RIATest) use the Flex automation API and have additional costs for FlexBuilderPro (700$ per license). For custom components you have to add automation code to them (pollution) and introduce them and their events in FlexMonkeys event map (tool changes).
FunFx also uses the ExternalInterface and is written in Ruby but runs only on Windows (- for OS independence) because it connects to the Flex application via Win32OLE. I found no capture/replay (-) and the website says you need FlexBuilder (I don’t know why therefore a 0 for license costs, we use IntelliJ IDEA for Flex development)
Fluorida seems to be at the beginning and there is very little documentation so it looks like to need an investment (- for tool changes). It has no capture/replay (-).
So our tool of choice is SeleniumFlex and we hope to get capture/replay working in the near future.
What experience have you made with functional testing in Flex? Which one do you use?