Tool send email with tripleo tempest results

https://blueprints.launchpad.net/tripleo/+spec/send-mail-tool

To speed up the troubleshooting, debugging and reproducing TripleO tempest results, we should have a list of people responsible to receive email status about tempest failures, containing a list of all the failures and failures that are known issues and are being covered by some opened bug in launchpad.

Problem Description

Currently there is periodic TripleO jobs running tempest, and these results are not being verified whether is failing or passing. Even if there is someone responsible to verify these runs, still is a manual job go to logs web site, check what’s the latest job, go to the logs, verify if tempest ran, list the number of failures, check against a list if these failures are known failures or new ones, and only after all these steps, start to work to identify the root cause of the problem.

Proposed Change

Overview

TripleO should provide a unified method for send email for a list of users who would be responsible to take action when something goes wrong with tempest results. The method should run at the end of every run, in the validate-tempest role, and read the log file, either by the output generated by tempest, or by the logs uploaded to the logs website, identifying failures on tempest and report it by mail, or save the mail content in a file to be verified later. The mail should contain information such list of failures, list of known failures, date, link to the logs of the run, and any other information that might be relevant.

Alternatives

One of the alternatives would be openstack-health, where the user can subscribe into the rss feed of one of the jobs using a third party application. Right now, openstack-health doesn’t support user subscription or send emails.

Security Impact

None, since it will use a API running in some cloud service to send the email, so the username and password remain secure.

Other End User Impact

None.

Performance Impact

None.

Other Deployer Impact

None.

Developer Impact

Developers in different teams will be more involved in TripleO CI debugging.

Implementation

Assignee(s)

Primary assignee:

arxcruz

Work Items

  • The script should be writen in Python

  • Should be part of validate-tempest role in tripleo-quickstart-extras

  • Should be able to read the logs in any run in http://logs.openstack.org

  • Once it reads the log, collect information about the failures, passing and known failures or taking tempest output and parsing it directly.

  • Be able to work with Jinja2 template to send email, so it’s possible to have different templates for different types of job

  • Read the list of address that the report should be sent * The list is a dictionary mapping the email address to a list of tests and/or jobs where the users are interested.

  • Render the template with the proper data

  • Send the report

Dependencies

None.

Testing

As part of CI testing, the new tool should be used to send a report to a list of interested people

Documentation Impact

Documentation should be updated to reflect the standard ways to send the report and call the script at the end of every periodic run.

References

Sagi mail tempest: https://github.com/sshnaidm/various/blob/master/check_tests.py