Why we’re backing Qameta – Allure Framework and why you should too.

What is Allure anyway ?

If you are looking for a free lightweight universal (i.e. multi-language ) test report tool you might not need to look further; Allure Framework could be all you need.

It is the right testing framework for reporting self-documented test case specification and test report.

“Allure” is “horse gait” in French because tests have steps, and “charm” in English because it looks great !

@aandryashin (Alexander Andryashin) is, in fact, the person who coined the name “Allure Report”.

Since June 2018, PDFAnalytics supports the Allure Framework initiation by contributing to the project via the Open Collective (web-based platform for open source crowdfunding).

The development team behind the project Dmitry @baev, Artem @eroshenkoam, Stanislav @sseliverstov and Vitaliy @vbragin have achieved so far :

What does it look like and what can it do ?

  • Generate nice test execution report in HTML
  • Attach execution artefacts after a step dynamically (e.g. a screenshot from a failed test step, or the log)
  • Link your test cases with the defect ID / requirement ID / user story ID from your management tool
  • Sort by severity i.e. blocker, critical, normal , minor and trivial
  • Check between broken and failed test
  • Test execution trend
  • Analyse the execution time
  • Analyse the test execution timeline
  • Analyse the frequency of the defects

It supports testing frameworks in the following language:  Java,  Python, JavaScript,  Ruby,  GroovyPHP,  .Net, and Scala including python-behave, karma, cucumberjs and SpecFlow.

Finding the balance between ‘Just Enough’ Documentation in your Agile Project

Documentation is a necessary evil; it may look like a cumbersome and time-consuming task but the project cannot survive without it. So, we need to find ways to do just enough documentation — no more, no less.

Allure report can support you to your journey to produce the test documentation for your agile team. Image if there was a single place to collect and display all test execution results from the CI pipeline to your agile team (e.g. unit tests, integration and system level tests) .

The history and the future

In 2019 the Allure family will be expanded with:

  • Allure 3 — realtime support, test environments, result analytics, customization and a brand new UI rewritten using ReactJS.
  • Allure Pro — the same report but more features and integrations (dark theme inc!).
  • Allure Hub — report manager that keeps all the reports in one place.
  • Allure Enterprise — test management system, that would keep both manual and automated tests in one place

Resources and Support

On blog posts and youtube

Everyone needs some help sometimes

Aerokube founded by @aandryashin and @vania-pooh specialises on tools for Selenium infrastructure.

Integration with HipTest and Travis CI

HipTest is web-based test management platform that supports continuous delivery and bridges the gap from manual to automated testing. It provides a real-time environment for creating, executing and maintaining tests.

Travis CI a hosted, distributed continuous integration service used to build and test software projects hosted at GitHub.

We used both Hiptest and Travis CI to automatically executed our python behave test cases located at GitHub and here below we will demonstrate what we learned.

Installation and Configuration

  1. Installation of the overall setup is fairly easy since both Travis CI and HipTest provide quite useful documentation. We had to create:
  2. We set Travis CI to execute all the test cases every time the repository is changed. The status page of this Travis job is publicly available to everyone python-pdf-analytics-client .
  3. After the execution, all the test case results are set to be uploaded to HipTest test run execution called ‘Automated execution via Travis’.

What we learned

    • Both Travis CI and HipTest documentation was very useful and up-to-date based on the current configuration of their platforms
    • We liked Travis CI , it is so much easier to demonstrate the features and the capabilities of the python-pdf-analytics-client library in real time using tow different python versions Python 2.7 and Python 3.6  :
    • We liked HipTest too, all our test execution results are now stored stored in a single platform monitoring the performance over time is now becomes more easierhiptest.PNG



Checking a payslip

In this article we will demonstrate how the PDFAnalytics web service can be used to verify automatically the content of a payslip PDF file.

We will use the PyPI library python-pdf-analytics-client which allows to automate the most common functions of the PDFAnalytics service.  The test cases will be written by using the python behave library.

The demo payslip PDF file is located inside the GitHub repository https://github.com/pdf-analytics/python-pdf-analytics-client .

        1. The payslip document is located inside the GitHub repository 
        2. Get an account on PDFAnalytics and log in
        3. From the PDFAnalytics, upload the payslip to start the inspection as show belowupload_pdf
        4. Check the coordinates on where the expected text boxes are located inspector
        5. Create a directory for your behave_payslip
          $ mkdir ~/behave_payslip
        6. Setup a virtualenv python environment from the
          $ cd ~/behave_payslip
          $ virtualenv venv
          $ source venv/bin/activate
          $ pip install python-pdf-analytics-client
        7. Create the python behave file structure
          • environment.py : to run your environmental code
          • payslip.py : the PDF payslip document
          • features/ : to store your feature test steps
          • steps/ : to store your python test steps
          • steps/basic_steps.py, to store the actual test steps. For the test steps you can reuse (i.e. copy ) the definitions from the file on Github  basic_steps.py.
        8. Create your feature file at features/payslip_pdf.feature . Check the example below to get an idea :
           Scenario: Verify payments and logo
           Given the pdf file "payslip.pdf" is sent to be analysed
           Then I "can" see the image "payslip_logo.png", at [left, top] ["100", "100"] on page "1" in pdf
           And I read "Total gross <br>payments:", at [left, top] ["74", "599"] on page "1" in pdf
           And I read "£1021.43", at [left, top] ["160", "584"] on page "1" in pdf
           And I read "Total <br>deductions:", at [left, top] ["250", "603"] on page "1" in pdf
           And I read "£304.92", at [left, top] ["334", "587"] on page "1" in pdf
           And I read "Net pay:", at [left, top] ["430", "570"] on page "1" in pdf
           And I read "£716.51", at [left, top] ["516", "586"] on page "1" in pdf
        9. Run you behave test cases from the command line:
          $ behave -D token=|your-personal-token-ID-from-the-PDFAnalytics-site|
        10. Done !


You have find a copy of this code at the Github/examples .

If you have any questions do not hesitate to contact us.