This paper deals with the substitution of defective sensor pixels regarding an Infrared Energy (IR) camera module. The work includes information about the development of algorithms and the optimization regarding hardware usage of a Field Programmable Gate Array (FPGA). Furthermore, the work outlines the test of the algorithms with the aid of Matrix Laboratory (Matlab) and the implementation in Very high speed integrated circuit Hardware Description Language (VHDL). Simulations of the VHDL module conclude the work.
Contents
Preface
1 Task
2 Realization
2.1 Definition of defective Pixels
2.1.1 Types of defects concerning monolithic pyroelectric arrays
2.1.2 Types of defects concerning microbolometer technology
2.2 Substitution algorithms
2.2.1 Single pixel substitution
2.2.2 Cluster error correction
2.2.3 Row and column errors
2.3 Tests with Matlab
2.3.1 Import of test pictures
2.3.2 Correction and display of test pictures
2.4 VHDL programming
2.4.1 Entity declaration and data transfer
2.4.2 Memory management
2.4.3 Implementation of algorithms
2.4.4 Simulation and tests
3 Conclusion
3.1 Findings
3.2 Outlook
Figures
Listings
List of abbreviations
Bibliography
Preface
Active Photonics AG works on thermal imaging systems. In order to enhance the market position, the development of a complete IR camera system, from the sensor to the display, was initiated. The whole system should be a single chip solution, running on an FPGA. The following figure 0.1 shall demonstrate the modules necessary to realize the task.
illustration not visible in this excerpt
Figure 0.1: Overview of the whole system
The system will run on an FPGA, which will be of the order of a XILINX Spartan-3A DSP XC3SD3400A. The chip provides a sufficient amount of logic cells and distributed Random Access Memory (RAM) - as well as block RAM bits. On this account the idea of a system without the need of a external RAM chip emerged. The matter of fact, that the algorithms which are fulfilling the tasks of the individual modules (see figure 0.1), will run on an FPGA, should be regarded in the design process. The more complex the algorithms are, the more hardware (Configurable Logic Blocks (CLB's), Look Up Tables (LUT's) etc.) is required. The dead pixel substitution module (see figure 0.1) is topic of this paper. It serves as interface between the offset- and gain-correction module and the image scaling block. The purpose of the defective pixel module is to substitute degraded sensor elements. Due to the fact that the sensor (ULIS Long-Wave InfraRed (LWIR) uncooled microbolometer) which should be employed, is not available in the first phase of the development process, the data format, timing, resolution and the serial configuration interface of the sensor was emulated by the use of a KITE pyroelectric sensor (see section
2.4 and figure 2.28 for the block diagram of the sensor emulator). Figure 0.2 illustrates the hardware which was worked with.
illustration not visible in this excerpt
Figure 0.2: Hardware view of the camera module
The sensor construction, including the optics, can be seen in the image above to the left. The data is analyzed and processed by using a XILINX Spartan3 XA3S1000 evaluation board, adapted for the application (see figure 0.2 to the right).
1 Task
The electromagnetic spectrum is divided into three segments by wavelength, which is measured in microns (1/1,000,000 of a meter) [12].
1. 0.76 to 1.5 microns = near infrared
2. 1.5 to 5.6 microns = middle infrared
3. 5.6 to 1000 microns = far infrared
illustration not visible in this excerpt
Figure 1.1: IR wavelength diagram
This wavelength (highlighted in figure 1.1) of light warms objects without warming the air between the source and the object. Radiant heat can also be called IR. Infrared waves are not visible to human eyes but can be seen by special instruments that translate infrared into colors that are visible to our eyes. Section 2.1.1 and 2.1.2 give a review of infrared sensors.
Dead elements, meaning defective sensor pixels, may interfere images remarkable. In order to avoid these negative effects on the image quality, substitution algorithms have to be designed. Infrared camera systems, valid for microbolometer sensors as for pyroelectric sensors, suffer from dead elements. The avoidance of these impairments is an important issue for all imaging systems. The development of algorithms, in order to substitute degraded pixels, is an important affair. An approach to a solution is the replacement of the unwanted values with values derived from "good" pixels in the vicinity of the "bad" ones (see section 2.2).
illustration not visible in this excerpt
Figure 1.2: Infrared Image containing defective pixels
Figure 1.2 shows an IR image containing defective pixels. As can be seen, the image quality is interfered by the dead elements.
2 Realization
2.1 Definition of defective Pixels
Infrared detectors typically suffer from defective pixels. They are either non responsive, permanently dark or saturating, or are otherwise incapable of sampling the scene. In the Short-Wave InfraRed (SWIR) band bad pixels typically appear as saturating while in LWIR microbolometer detector arrays bad pixels are essentially dead and lie at the zero signal level.
2.1.1 Types of defects concerning monolithic pyroelectric arrays
In a pyroelectric device, a change in temperature creates a change in polarization. Electrical polarization change is related to a surface-charge change with respect to time. Thus, a pyroelectric device produces current only as it experiences a temperature rise or fall. For that reason, a chopper wheel is used to release the view to the environment periodically. Thus, a dynamic measurement is possible (see reference [15] for further information about pyroelectric detectors).
A pixel of the KITE pyroelectric detector array is defined as defect, if it applies to one of the following categories.
- Category 1 - elements with a response in the range 0 to 40μV/K inclusive.
- Category 2 - elements with a response below 50% of the mean
- Category 3 - elements with a response above 200% of the mean
- Category 4 - elements with an out-of-range noise value
- Category 5 - elements with an Noise Equivalent Temperature Difference (NETD) below 20% of the mean
- Category 6 - elements with an NETD above 300% of the mean
- Category 7 - elements with an out-of-range offset value
- Category 8 - elements with a negative response
- Category 9 - elements with an out of range average noise value
The information is gathered from [2].
2.1.2 Types of defects concerning microbolometer technology
Bolometers consist of a thin carrier foil on which a resistance film is brought up. Thermal radiation causes a temperature change of the film and for this reason a variation of the resistance. This serves as a rule for the amount of absorbed radiancy (see reference [15] for further information about bolometers).
This specification is guilty for a ULIS LWIR Focal Plane Array (FPA). The sensor is sensitive to radiation in the 8 - 14^m range. It includes a microbolometer FPA comprised of a 384 x 288 elements two dimensional detector array made from amorphous Silicon resistive bolometer connected to a silicon ReadOut Integrated Circuit (ROIC). A pixel of the infrared image sensor is defined as non-operating if
- its responsivity is < 0.8 x average responsivity or > 1.2 x average respon-
- its NETD is > 1.5 x average NETD defined in related document
See reference [131 for detailed information about the sensor.
2.2 Development of substitution algorithms
In order to substitute a degraded pixel, its value is replaced with one derived four in the horizontal and vertical directions (see figure 2.1 left), called the close neighbors, and four diagonal neighbors(see figure 2.1 right), called distant neighbors[4\.
illustration not visible in this excerpt
Figure 2.1: Close and distant neighbors of a single pixel
Each of them comes into consideration in order to substitute the central defective one. This process suppresses the display of dead pixels and avoids the disturbance of the image quality. By the correction of pixels marked for substitution, a repaired image is produced. Defective pixels may appear as single pixel errors (see subsection 2.2.1), as cluster errors (see subsection 2.2.2) or as row and column errors (see subsection 2.2.3).
2.2.1 Single pixel substitution
Single defective pixels are defined as any one bad pixel, if it is not nearby any other bad pixel. A pixel pair, defined as any two adjacent bad pixels, is an exception. In the following, it won't be distinguished between single pixels and a pixel pair. Figure 2.2 illustrates an original image and an intentional destroyed image to demonstrate the appearance of single pixel errors.
illustration not visible in this excerpt
Figure 2.2: Single pixel errors
As can be seen, already a few single defective pixels may interfere images remarkable. For this reason substitution algorithms have to be developed.
2.2.1.1 Single neighboring pixel substitution
The usage of the value of a neighboring pixel suggests itself to replace the bad one. In doing so, one should pay attention to select an applicable one. As shown in figure 2.3 it is not always possible to take the same good pixel to kill the defective one.
The blue, yellow, red and green pixels stand for dead pixels and the surrounding grey ones for pixels which could replace the bad ones. It is obvious that it is not possible to replace the value of the dead pixel with that one derived from the previous or the following good pixel at all times. The very same holds for the pixels above, below and diagonal to the pixel to adjust. The information of potential good pixels to substitute the bad one has to be known prior the correction algorithm starts.
illustration not visible in this excerpt
[...]
-
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X. -
Upload your own papers! Earn money and win an iPhone X.