Agilent Technologies is one of the leading manufacturers of test and measurement equipment. At the moment the High Speed Digital Test division is developing a new Pulse- and Pattern generator. In order to improve the quality of the firmware development process, the firmware should be automatically tested. This will help find errors faster and identify side effects of any changes in the source code.
For this, a test framework that can carry out these automatic tests was developed. The test cases are defined in XML to make the processing easy. The results are also saved to XML. This allows both human and machine readability and to transform the result to a HTML formatted report.
Contents
1 Introduction
1.1 Agilent Technologies - Company Structure
1.2 Project Environment
1.3 Task Description
1.4 Strategic Approach
1.5 Structure of this Thesis
1.6 Time Schedule
2 Basic Concepts
2.1 Test Devices
2.2 GPIB
2.3 Programming Language
2.3.1 SCPI
2.3.2 .NET Framework
2.3.3 The Programming Language C#
2.4 Driver Library
2.5 XML
2.5.1 Basic Components of an XML Document
2.5.2 XML Schema
2.5.3 XPath
2.5.4 XML Transformation Agilent Technologies
2.6 Gnuplot
3 Implementation
3.1 Input Formats
3.1.1 The Test Suite XML File
3.1.2 The Test XML File
3.2 Perl Component
3.2.1 GetResourceString
3.2.2 GetOutputDir
3.2.3 GetTestSuiteName
3.2.4 GetTestName
3.2.5 SetParameter
3.2.6 GetParameter
3.2.7 Error
3.2.8 WriteLink
3.2.9 WriteText
3.2.10 Log
3.2.11 ShouldAbort
3.2.12 Wait
3.3 The Code
3.3.1 Windows Application
3.3.2 Class Library
3.3.3 Setup Project
3.4 Creation of the HTML test report
4 Validation
5 Usage of the GUI Tool
5.1 Setup
5.2 The GUI
5.3 Demo Test Suite Agilent Technologies
6 Future Prospects
A XML Element Overview
B Code Listings
C Glossary
D Listings
E List of Figures
F Bibliography
1 Introduction
1.1 Agilent Technologies - Company Structure
Agilent Technologies is a leading manufacturer of Test and Measurement equipment, Communication and Electronic Solutions as well as Chemical and Biological Analysis tools. The company is a spin off of the Hewlett-Packard Corporation. It was first listed on the New York stock market in November 1999. At the moment there are more than 19.000 employees working for Agilent in over 40 countries with customers in more than 110 countries. In the financial year 2005 Agilent produced a turnover of 5.1 billion US- Dollar [ATD05].
illustration not visible in this excerpt
Figure 1.1: Agilent - Company Structure
The Agilent Technologies Deutschland GmbH has about 1400 employees. In Germany all Agilent divisions are represented. The Life Sciences and Chemical Analysis (LSCA) sector, which is located at the Waldbronn site, provides complete solutions for the pharmaceutical industry, especially in the sectors of health research, development of new drugs and research for new active ingredients. They also work on the quality assurance during the manufacturing process of drugs.
At the Böblingen, site digital test equipment and modules are developed. The Research and Development (R&D), as well as the production of several Agilent products, are also located in Boblingen.
The Network and Digital Solutions Business Unit (NDS) is part of the Electronic Measurement Group (EMG). The EMG is one of the main groups the company comprises. One of the departments of NDS is the Digital Verification Solution Division (DVS), which has its headquarters in Böblingen. DVS develops measurement equipment for high speed digital applications. Its product range includes Digital Communication Analyzers (DCA), Bit Error Rate Testers (BERT) as well as Pulse and Pattern generators (PPG).
The main part of DVS is called High Speed Digital Test (HSDT) and develops the High Frequency Patten Generators and Analyzers. In this part of Agilent the project described in this thesis took place. [ADVS]
1.2 Project Environment
HSDT is developing a new Pulse and Pattern generator at the moment. The internal project name is Pulsar. The firmware running on the Pulsar is developed in parallel with the hardware. Since the firmware is a very complex piece of software developed by a team of several engineers, it is an important task to verify that the software is working as expected. Experience has shown that the cost of fixing an error is smaller the earlier the error is found. A change in one part of the firmware could also have a side-effect on other parts.
Therefore the demand for a tool that can carry out some automated tests has occur. Agilent Technologies
1.3 Task Description
The task of this diploma thesis is to design and implement a test framework that allows the software engineers to automatically test the firmware of the Pulse Generator. The tests will not be defined in the actual code but in a number of extra XML files. This ensures the reusability of the test framework. This way, for a new product new test cases can be defined and there is no need to adapt the software.
The tests are carried out through the programming interfaces of the instruments. The available interfaces are LAN (Local Area Network), USB (Universal Serial Bus) and GPIB (General Purpose Interface Bus). Testing the instruments through several interfaces ensures that the I/O (Input/Output) paths are all working correctly.
There are several different test types required, simple query/response tests as well as execution time measurements. In addition, the possibility to sweep over one or more parameters is needed to determine the allowed value range of certain parameters. The result will be saved in XML files so that it can be easily processed and transformed into another format like HTML.
For the implementation, C# will be used since it supports XML very well and allows to simply create a comprehensive graphical user interface (GUI).
In addition to the test framework, some example tests will be created to demonstrate the usage of the test tool.
1.4 Strategic Approach
In the first phase, the objective of this project was discussed with the project sponsor Thomas Burger, the main point being what the test framework should be able to do. After the theoretical fundamentals and knowledge were defined, a first prototype of the test framework was developed. From the experience gained by the development and usage of this prototype, the project scope and deliverables where defined in more detail. During the whole development phase of the final test framework there was a close cooperation with Thomas Burger to ensure that the software met the requirements of the firmware developers.
The software was tested with different test cases and also by different users to ensure the usability.
1.5 Structure of this Thesis
The first part of this thesis explains the basic concepts of the technologies used in this project. This includes the Agilent instruments as well as the software and programming languages used.
Later on, the actual implementation is described. The format of the test definition files is laid out in detail and the code of the application is explained.
Chapter 4 covers the validation process of the project.
The instructions in chapter 5 explain the usage of the test framework.
At the end of this thesis some future prospects are given to illustrate the parts where there is still room for further development.
1.6 Time Schedule
The following figure shows the initial schedule for this project. Not everything was completed on the exact day planned. But the major milestones were completed as planned so the project stayed as scheduled most of the time.
Figure 1.2: Time schedule of the project
2 Basic Concepts
This chapter will give an overview of the different technologies used in this diploma thesis.
The programming language C# is used for the GUI and to control the program flow. The VISA (Virtual Instrument Software Architecture) library is used to communicate with the instruments by so called SCPI (Standard Commands for Programmable Instrumentation) commands. The physical connection to the instrument is established by either GPIB, USB or LAN. For the data storage and processing a number of XML technologies are used.
2.1 Test Devices
The test framework was primarily developed for the new Pulsar but it can also be used with any instrument that uses the SCPI command set.
The new Pulsar will also have the functionality of an Arbitrary Waveform Generator. For remote programming USA, LAN and GPIB interfaces are available.
The firmware running on the Pulsar is developed with MS Visual Studio for the .NET Compact Framework. This allows the firmware to be compiled for a standard Windows PC as well. This simulates the Pulsar on a PC.
illustration not visible in this excerpt
Figure 2.2: The Pulsar Simulation running on a PC
2.2 GPIB
The General Purpose Interface Bus (GPIB) was developed by Hewlett-Packard (HP) in the 1960s and was standardized by the IEEE in the late 1970s under the name IEEE-488. It was developed as a general purpose bus and was used for connecting plotters, external hard disk and tape drives. Nowadays it is still widely used in the lab environment to communicate with measurement devices.
It allows for 15 devices to be connected at the same time. The slowest device connected will determine the maximum speed on the whole bus. The maximum theoretical speed is 1 MByte/s. GPIB uses 24-pin Amphenol Micro ribbon connectors. Sixteen wires are used for signals, eight for the data, three for the handshake and five for the bus management. The other eight wires are used for grounding. [WGPIB]
2.3 Programming Language
2.3.1 SCPI
The Standard Commands for Programmable Instrumentation (SCPI) is a standard set of commands that is mainly used to control programmable test and measurement devices. SCPI does not define the physical layer. Instead, it can use a variety of bus systems like GPIB, USB, RS232 and LAN to send the commands to the instrument. For some instrument classes like power supplies, voltmeters and oscilloscopes standard command sets are already predefined. SCPI commands are pure ASCII text strings, which are handled well by most programming languages. [WSCPI]
2.3.2 .NET Framework
The .NET Framework is a new environment for developing and running software applications featuring ease of development of web based services, rich standard run time services available to components written in various programming languages, as well as inter language and inter machine operability.
A number of programming languages are integrated in the Visual Studio development environment. Such languages are “C#”, “C++”, “VisualBasic.NET”, “J#” and “JScript .NET”. The architecture of the .NET framework also allows the integration of programming languages by other vendors. There are more than thirty additional languages with the corresponding compiler available. These include “Ada”, “COBOL”, “Eiffe”, “Fortran”, “Perl”, “Ruby” and “Smalltalk”, just to name a few.
.NET provides an object oriented API (Application Programming Interface) as a set of classes that are accessible from any programming language.
illustration not visible in this excerpt
Figure 2.3: Overview of the Common Language Infrastructure [WCLI]
illustration not visible in this excerpt
Figure 2.3 shows how the main components of the .NET framework work together. First, high level languages are translated into the Common Intermediate Language (CIL), which then gives access to the .NET system services. This intermediate language is not interpreted, but rather compiled in a manner known as just-in-time compilation (JIT)
into native code. The combination of these concepts is called the Common Language Infrastructure (CLI). Microsoft’s implementation of the CLI is known as the Common Language Runtime (CLR).
The .NET Framework introduces a Common Type System, or CTS. The CTS specification defines all possible data types and programming constructs supported by the CLR and how they may or may not interact with each other. [WNET]
2.3.3 The Programming Language C#
The language C# is Microsoft’s approach to create a pure object oriented language that combines the best feature of Delphi, Visual Basic, Java and C++ with a particular emphasis on simplification.
The design goals for C# were:
- C# is intended to be a simple, modern, general purpose, object oriented programming language.
- The language, and implementations thereof, should provide support for software engineering principles such as strong type checking, array bounds checking, detection of attempts to use uninitialized variables, and automatic garbage collection. Software robustness, durability, and programmer productivity are important.
- The language is intended for use in developing software components suitable for deployment in distributed environments.
- Source code portability is very important, as is programmer portability, especially for those programmers already familiar with C and C++.
- Support for internationalization is very important.
- C# is intended to be suitable for writing applications for both hosted and embedded systems, ranging from the very large that use sophisticated operating systems, down to the very small having dedicated functions.
- Although C# applications are intended to be economical with regards to memory and processing power requirements, the language was not intended to compete directly on performance and size with C or assembly language. [WCSP]
2.4 Driver Library
VISA stands for Virtual Instrument Software Architecture and is a widely used I/O API in the Test & Measurement industry for communicating with instruments from a PC. The VISA library allows controlling different instruments in the same way by defining virtual instruments that work in a standard way. An instrument which is set up in the VISA library is called a VISA resource and is identified by its resource address; this is a unique identifier for the instrument. The resource address includes the physical path to the instrument as well as some additional identifiers. So the resource address shows which physical bus the instrument uses. One physical instrument can have several interfaces; therefore several VISA resources can map the same physical instrument but use different means of communication with this instrument. [WVISA]
Abbildung in dieser Leseprobe nicht enthalten
Listing 2.1: Examples of a VISA Resource Address for a TCP/IP and a USB device
2.5 XML
The Extensible Markup Language is a standard by the W3C (World Wide Web Consortium) for creating machine and human readable documents in the form of a tree structure. The main purpose of XML is to share data between different systems. XML is a meta language that can be used to define a special language. There is a wide variety of XML languages defined by standard organizations like the W3C for very different
purposes like text make up, mathematical notations, music description and also vector graphics. With the help of an XML Schema one can define its own XML language. There are also a number of technologies related to XML which allow different types of access and the manipulation of the data inside the XML document. [OXML], [WXML]
2.5.1 Basic Components of an XML Document
The most basic components of an XML Document are:
Elements to mark up the sections of an XML document. An element has the following form: ElementNameElement Content/ElementName or /EmptyElement. An Element consists of the start tag ElementName, the end tag /ElementName and the content in between the start and end tags. The content can either be a text, a number of XML elements or mixed content. One special case is the empty element /EmptyElement. It has no content, so there is no need for an end tag.
An Attribute is a mechanism for adding more information to the element start tag. Attributes can be used to add a unique label or a Boolean flag to an element. It can also be used to distinguish between elements of the same name. One element can have many attributes. The syntax of an attribute looks like this:
Abbildung in dieser Leseprobe nicht enthalten
Listing 2.2: Declaration of an XML entity in the document prolog
The entity can be referred many times in a document. An entity can be a placeholder for a single character, a text string or even the content of a whole file. The entity reference consists of an ampersand (&), the entity name and a semicolon (;).
Abbildung in dieser Leseprobe nicht enthalten
Listing 2.3: Usage of an XML entity inside the document
Comments are notes in the XML document that are not interpreted by the XML processor. They can be used to identify the purpose of a file or a section or to help other people working on the same file to understand it. A comment starts with the delimiter
“! ” and ends with “ ”. Between the delimiters goes the actual comment, which can be any kind of text, including spaces, newlines and XML elements.
2.5.2 XML Schema
The purpose of a schema is to define the structure of an XML document. It has become a W3C recommendation in May 2001. The schema expresses a number of rules to which an XML document must conform to be considered valid according to the schema. It defines which elements and attributes are allowed to appear in a document, which elements are valid child elements of what other element, the order of them, whether an element is empty or can include text, data types for elements and attributes and also the default and fixed value for elements and attributes. XML Schema allows a more detailed description of the structure than DTD (Document Type Declaration) which is older than XML Schema but still widely used. [WXSCH]
2.5.3 XPath
XPath allows random access to the content of an XML Document. It can be used in conjunction with other XML Technologies including XSL Transformation. An XPath expression can refer to all parts of an XML document, text, values and data in XML elements, attributes, processing instructions, comments etc. It is also possible to access the name of elements and attributes. [WXPA]
2.5.4 XML Transformation
Extensible Stylesheet Language Transformations (XSLT) is a language used to transform an XML document to another document. The new document can be an XML document as well, but also any other format like HTML or plain text. The so called XSLT stylesheet defines how the document is transformed. The XSLT stylesheet itself follows the XML standard. An XSLT stylesheet consists of a number of template rules. Each template rule matches some part of the source document and defines what to add to the result document. [WXSLT]
2.6 Gnuplot
Gnuplot is a freely available program that can be used to generate two- and threedimensional plots of functions and data. It can output the graph directly on the screen or in a number of different graphic files, including PNG, EPS, SVG, JPEG and many others. Gnuplot does not have a graphical user interface but is controlled by the command line and the usage of scripts. [WGPT]
3 Implementation
3.1 Input Formats
The definition of a test suite is made up of two kinds of XML files. The first defines which tests are loaded for a test suite. The second kind is the actual definition of the test. A test suite is usually made up of several test files.
3.1.1 The Test Suite XML File
The test suite defines which test files are loaded for a test. This ensures the reusability of test files in several test suites.
The XML structure of a test suite file is defined by an XML Schema file. This file is called “testsuite.xsd” and is located in the installation directory. This schema file makes sure that the syntax of a test suite always complies with certain rules.
The XML standard defines that an XML file always has a single root element that encloses all other elements. In case of a test suite, this root element is called “testsuite”. The “testsuite” element can contain an arbitrary number of “testfile” elements. Each “testfile” element defines exactly one test file to be loaded. The “testfile” element must contain a “path” and a “selected” element. The “path” element defines where the test file is located, and it can either be relative to the location of the test suite or absolute. The “selected” element is a Boolean that indicates whether this test is selected when the test is loaded into the GUI or not. The user can change the selection before starting the test. The only allowed values for the “selected” element are true, 1, false and 0.
3.1.2 The Test XML File
The test file contains the actual definition of a test; mainly these are the commands that will be sent to the instrument. The structure of a test file is defined by an XML schema called “input.xsd” which is also located in the installation directory. The root element of a test file is the “test” element. It must contain a “title” and a “description” element. Both elements contain a string and are used to identify the test. The title and description will also be displayed in the GUI when a test suite is loaded. After the “title” and “description” the “test” element can contain an arbitrary number of any of the following elements, which are described in detail below:
- scpi
- sweep
- stopwatch
- resultlog
- perlstatement
- perlfile
- comment
In the input file variables in the form of “$variable$” can be used. Some elements like the “send” and “stopwatch” can define these variables and other elements can use them. These variables can also be accessed within a Perl script.
3.1.2.1 The “scpi” Element
The “scpi” element is used to send commands and/or queries to an instrument. The “scpi” element has a required attribute of type string called “instrument”. This is a symbolic name which will be mapped to the resource address of a physical device in the GUI. The “scpi” element contains a number of the following elements:
- command
- query
- resultlog
-comment
The number and order of these elements are interchangeable.
3.1.2.2 The “command” Element
The “command” element is a simple string with an optional Boolean attribute “errorExpected”. The content of the “command” element will be sent to the instrument. Within the command, variables of the form $name$ can be used. This string will be substituted by the current values of the variable before the command is sent to the instrument. If the variable was not previously defined, this will lead to an error and the command is not sent to the instrument. The “errorExpected” attribute indicates if the command is expected to produce an error. If the “errorExpected” attribute is set to true, which means that an error is expected, and an error occurred this error is logged but the command element is treated as success. On the other hand if no error occurred, but the “errorExpected” attribute is set to true, this “command” element is treated as a failure.
3.1.2.3 The “query” Element
The “query” element is used to send queries to the instrument. It can have the following child elements:
- send
- expectedResponse
- resultlog
- comment
The “send” element is required, while all other child elements are optional. “send” and “expectedResponse” can only occur once inside a query element and must be placed in this order. In-between the “send” and “expectedResponse” element there must not be any other element.
3.1.2.4 The “send” Element
The “send” element contains the actual query that will be sent to the instrument. Just like the “command” element the “send” can contain variables that will be substituted with the current values before the query is sent to the instrument. It also has two optional attributes. The “errorExpected” attribute works the same way as it does with the command element (see section 3.1.2.2). The second attribute is the “log” attribute, which allows to save the response of the instrument in a variable, that is valid during the processing of this test file. The value of the “log” attribute is the name of the variable where to save the response. The variable name has to be in the form of $name$.
3.1.2.5 The “expectedResonse” Element
The “expectedResonse” element allows comparing the actual response from the instrument with some expected value. It is of type string and has the two optional attributes “resolution” and “accuracy”. The resolution indicates how many significant figures and the accuracy how many decimal places are compared. If both resolution and accuracy are specified, the number is first truncated accordingly to the resolution and then accordingly to the accuracy.
Some examples:
12.3456 with resolution of 3 = 12.3
0.012375 with resolution of 4 = 0.01237
12.3456 with accuracy of 0.01 = 12.34
[...]
-
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X.