BDD analysis

BDD, when applied to test automation, allows Agile teams to write tests in pseudo natural language interpretable by automation tools such as Cucumber, Easyb, Concordion, RSpec and Behat. Fundamentally it hard-wires the link between ACs and Tests. BDD provides a solution, with varying degrees of success, to the issues faced with Agile Acceptance Criteria.

What are these issues:

1. Insufficient time to produce sufficiently comprehensive ACs
2. Unstructured ACs written in narrative leading to ambiguity.
3. No traceability between ACs and the automated tests ... no acceptance test coverage
4. Poor cross-team visibility of original ACs and subsequent changes
5. Tests for ACs being implemented in different unrelated technologies
6. Automated tests requiring significant refactoring when new stories are developed 
7. Loose relationship between ACs and tests 
8. Previous Stories become redundant and no single view of what the App does
9. Little focus on AC context such as; scope and assumptions
10. Little focus on wider ACs such as; app wide, business domain, technical domain.

How does BDD help:

1. The enforced structuring involved in writing BDD Scenarios imposes an increased focus on writing ACs and hence a greater investment of time hence more deliberation. This benefit is a marginal byproduct of the process and so can be mute.
2. BDD imposes a stricter (e.g. Gherkin) style which needs to be implementable in test logic, thus reducing the scope for ambiguities
3. There is a programmatic link between Stories (Features), ACs (Scenarios) and Tests
4. ACs become a tangible test artifact. Tests = ACs to Test better reflect the requirements, rather than going thru a level of interpretation.
5. ACs are linked to the corresponding tests which can be implemented in different technologies e.g. GUI test or Unit style test
6. 'Fixtures' directly model ACs and therefore are likely to be less throw-away. Organising automation into; state setting, follow-up actions, and test assertions, modularise the tests thus increasing re-use. Additionally the separation of data-driven inputs and their associated test improves the design.
7. Assertions and flow are taken out of the test and implemented in the BDD code so the AC is the test. 
8. Stories need to be kept up-to-date and so can be used to document App functionality

By using a BDD tool, the QA and BA can focus on writing ACs, leaving the implementation of the AC open to QAs or Devs, whilst retaining the traceability. BDD suports ATDD in that automated tests are written at a high level before delivery. The building of fixtures is generally a mechanical exercise.

BDD tests should focus on the functionality and not on the Integration and so can be heavily geared around the Unit tests.

Concerns:

• The relationship between ACs and test is not always 1:1. Sometimes it's better to amend existing tests rather than creating new ones.  
• The most important requirement for AC writing is sufficient time and flexibility, and only a process change will provide this
• Structure can be achieved through process and practices
• POs are not focussed on ACs, they are focussed on results and building confidence in QAs
• BDD tests are still at quite a high level and there is still room for ambuguity thus misinterpretation when righting fixture code.
• Overhead incurred for maintaining previous Stories where new Stories have changed behaviour