There is a natural workflow when using the Achilles Satellite; you set
up the device that you want to test, configure the network settings, and
choose the monitors to use and the tests to run. Once the tests have
executed, you can examine the test results to find out what happened
when unusual traffic was sent to the device. The final, optional step
in the workflow is the generation of a PDF report detailing all this
information. The report is a useful 'takeaway' that can be distributed
amongst management or other testers, or filed for testing and auditing
records. The development team has been working on improving the
reporting capabilities of the Satellite over the past year. This week
we will tell you a bit about what our reports contain and suggest ways
in which you can use them.
What Type of Reports Can You Create?
The Satellite can generate several types of reports: a history report containing varying amounts of history information, a graph report containing particular graph history for selected monitors, and a certification analysis report that analyzes Level 1 test suite results to predict whether the tested device would pass Achilles certification.
History is an Achilles term for all information that is displayed in the Satellite UI after testing. It includes information such as the device name, the total number of test cases that were executed, the number of test cases that were executed but could not complete, a summary of unusual device behavior (anomalies) and monitor results, test parameters, link mode information, event log information for test exceptions and so on. There are three types of history reports: verbose, standard, and compact, each containing varying amounts of history information. In addition, you can choose to report on only those test cases that are of interest to you. Graph history displays graphical information from each enabled monitor for a particular test case.
A certification analysis report is a useful tool when preparing embedded devices or host-based devices for Achilles Certification, but be aware that the analysis is an informal interpretation of test results and does not guarantee certification!
How Is A Report Useful?
A history report is a useful record of having executed tests. Perhaps you include Achilles tests as part of your QA process, or you run external tests using the Satellite. In both cases, a report is proof that procedure was followed. If a test finds unacceptable device behavior, you can use a history report to record and then recreate the conditions under which the behavior was exhibited so that, when you have corrected this behavior, you can verify that it no longer exists. You can also use history reports for auditing compliance purposes.
The configurable nature of the history reports enable you to produce an overview for management which could be helpful when making the case for further testing or to demonstrate the communications robustness of a device. Conversely, you can produce a highly detailed report for those interested in more technical details.
A certification analysis report is a cost-effective means to determine whether your device is as prepared as possible for Achilles Certification. It allows you to fix any issues before submitting the device to be certified.
These are just some ways in which our reports might be used. The reporting capabilities of the Satellite are driven by customer requests and feedback: if you have any reporting suggestions, please let us know!