IMPLEMENTATION OF MULTIMEDIA LIBRARIES ON ANALOG DEVICES BLACKFIN – 533 PROCESSOR Project Report Submitted by
Ajay.A.V 0221073
Deepti.S.M
Rajendra.C.Y
Venkatesan.N.E
0221115
0221134
0221089
Under the guidance of Mr. Gurudath.N.V
Dr.Sumam David
DSP Applications Engineer, Analog Devices India Pvt. Limited, Bangalore.
Professor, Dept of ECE, NITK, Surathkal.
Department of Electronics & Communication Engineering, National Institute of Technology Karnataka, Surathkal, Srinivasanagar – 575025, Karnataka May 2006
Department of Electronics & Communication Engineering National Institute Of Technology Karnataka, Surathkal Srinivasanagar – 575025, Karnataka
CERTIFICATE This is to certify that the project entitled “Implementation of Multimedia Libraries on Analog Devices BlackFin – 533 Processor” is a bonafide work done by Ajay.A.V (Reg. No 0221073), Deepti.S.M (Reg. No .0221089), Rajendra.C.Y (Reg. No. 0221115) and Venkatesan.N.E (Reg. No .0221134) in partial fulfillment of the requirements for the award of the Degree of Bachelor of Engineering in Electronics and Communication Engineering at the National Institute of Technology Karnataka, Surathkal during the academic year 2005 – 2006.
Mr.Gurudath.N.V
Dr.Sumam David
DSP Applications Engineer, Analog Devices India Pvt. Limited, Bangalore.
Professor, Dept of ECE, NITK, Surathkal.
Dr. B. Shankarananda Head of Department, Dept of ECE, NITK, Surathkal
Place: NITK, Surathkal Date:
i
Dedicated To
All Our Teachers “The Best Teachers will not give us
something to drink. But they make us thirsty” And We are proud to have them
ii
ACKNOWLEDGEMENTS It gives us immense pleasure to thank all those who helped us during the tenure of our project. We would like to thank our guide and mentor Dr. Sumam David, Professor, NITK, Surathkal.
“One who wishes to teach us a truth would not tell it to us, but simply suggest it with a brief gesture, a gesture which starts an ideal trajectory in the air along which we glide until we find ourselves at the feet of the new truth” José Ortega y Gasset (1883 - 1955), Spanish writer and philosopher. She will be a constant source of inspiration for us. Her treasured support and guidance through the course of the project was instrumental in the successful completion of the project. We would like to thank Mr. Srinivas.K, ADI, Bangalore, for providing us the opportunity to work on an Analog Devices sponsored project. We would like to thank Mr. Kunal Singh who provided us with all the technical support we needed during the course of the project, without which we would have been stuck up at various junctures of the project. We are also thankful to Mr. Gurudath.N.V, and Mr. Rupjyothi Sarmah for their support during the later stages of the project. We would also like to thank Mr. Ganesh Prabhu, Mr.Niranjan of DSP Lab, ECE dept, for their persevered, relentless help they gave by keeping the lab open 24x7 and troubleshooting all the technical snags. We would also like to convey our gratitude to all our friends for their moral support, without which we wouldn’t have tread so far.
iii
ABSTRACT The basic aim of the project is to capture the TV signal, which is in NTSC format, perform format conversion, perform operations like contrast stretching, edge detection, image enhancement etc on the image frame, convert it back into NTSC and display it on the TV. In our project we capture the NTSC signal from a camera and then process it. All these functions have to be implemented in the BF-533 processor for real time operation. This finds a variety of applications in the market as most of the video systems are digital in nature. There are various issues that need to be taken care of while implementing the algorithms for real time operation on BF-533. The major issues are memory management and real time transfer between two kits. We are making use of two kits as a BF-533 EZKIT Lite supports either video input or video output at a time. BF-533 has a limited internal memory and for operations that need to be done on a video frame (720 x 525), we need to make use of external memory interfaced to the processor. For efficient utilization of this memory, memory management techniques such as Direct Memory Access (DMA), Cacheing of memory etc need to be used. These issues are discussed in detail in the report. For real time transfer between the kits, Serial Ports (SPORTs) need to be used. For this purpose, clock speed reduction, acknowledgement and other issues need to be considered. All these issues are discussed in detail. The image enhancement algorithms that we have implemented include Histogram equalization, Gamma correction, Log Transformation, Auto White Balancing and Edge Detection. We have also implemented format conversion among different image formats. For this purpose we developed our own algorithm. All these operations were then optimized for real time operation using memory management techniques. The developed algorithm for resizing has been submitted as a paper titled “Resizing of Images by Arbitrary Factors in the Spatial Domain and Implementation on BlackFin BF533 Processor” for International Conference on Visual Information Engineering 2006, Bangalore conducted by the Institution of Engineering and Technology (IET).
iv
CONTENTS Acknowledgement…………………………………………………………………ii Abstract……………………………………………………………………………..iii List of Figures……………………………………………………………………..vi List of tables………………………………………………………………..……...vi 1. Introduction……………………………………………………………………..1 2. BlackFin – 533 DSPr 2.1 Overview………………………………………………………………......2 2.2 Peripherals…………………………………………………………….......2 2.3 Core architecture………………………………………………...….…… 3 2.4 Memory architecture…………………………………………………...... 5 2.4.1 Internal Memory………………………………………………....6 2.4.2 External Memory……………………………………………......6 2.4.3 I/O Memory space……………………………………………....6 2.5 DMA………………………………………………………………………...7 2.6 Serial ports (SPORTs)…………………………………………………....8 2.7 Instruction set description………………………………………………..9 2.8 Development tools……………………………………………………....10 3. ADSP BF – 533 EZ Kit Lite 3.1 Video output mode…………………………………………………..…..13 3.2 Video input mode………………………………………………………..13 4. Image Enhancement Algorithms 4.1 Histogram Equalization…………………………………………………14 4.2 Gamma Correction……………………………………………………...15 4.3 Auto White Balancing…………………………………………………..16 4.4 Edge Detection and Enhancement…………………………………...18 4.5 Log transformation……………………………………………………...20 5. Image Resizing 5.1 Introduction……………………………………………………………....21 5.2 Images seen as a vector in Euclidean Space………………………..22 5.3 Discrete Cosine Transform of Images……………………………......22 5.4 Determination of Transformation matrices………………………......23 5.5 Decomposition and recomposition of images into spatial blocks…..25 5.6 Comparison with standard algorithms………………………………...27 6. Implementation on BF – 533
v 6.1 Processor implementation…………………………………………….28 6.2 Results……………………………………………………………….....31 7.
Conclusion and future work 7.1 Work Done……………………………………………………………….35 7.2 Scope for future work……………………………………………………35
8.
Publications………………………………………………………………...36
9.
Bibliography and References……………………………………………37
vi
LIST OF FIGURES Figure – 1: Figure – 2: Figure – 3: Figure – 4: Figure – 5: Figure – 6: Figure – 7: Figure – 8: Figure – 9: Figure – 10: Figure – 11: Figure – 12: Figure – 13: Figure – 14: Figure – 15: Figure – 16: Figure – 17: Figure – 18a: Figure – 18b: Figure – 19: Figure – 20: Figure – 21a: Figure – 21b: Figure – 21c: Figure – 22a: Figure – 22b:
Processor block diagram…………………………………………………….....……3 Processor core architecture………………………………………………………....4 BlackFin – 533 system Architecture………………………….............................12 Histogram Equalization……………………………………………………………..15 Gamma correction……………………………………………………………….….16 Color temperatures for various light sources…………………………………….17 AWB Correction……………………………………………………………………..18 Edge detection using Laplacian Operator………………………………………..20 Log Transformation…………………………………………………………………20 Block Description of Resizing Algorithm………………………………………….26 Image Resizing………………………………………………………………………26 Block Diagram of Hardware Setup………………………………………………...29 Input Image and its Histogram……………………………………………………..31 Output Image and its Histogram…………………………………………………...31 Input images and the Gamma corrected image………………………………….31 Image with red tinge and the White Balanced image……………………………32 Input NTSC (525x720) image and the resized VGA (320x240) image………..32 Image to be negated and transmitted……………………………………………..32 Negated and received image in the second kit…………………………………..32 Original showing SPORT transfer between kits scaled to 25%........................33 QCIF image obtained from NTSC image scaled to 50 %.................................33 Image used for Edge Detection……………………………………………………33 Image after Edge Detection………………………………………………………..33 Image Enhancement using Edge Detection………………………………………33 Original image for Log Transformation……………………………………………34 Log Transformed Image…………………………………………………………….34
LIST OF TABLES Table – 1: Table – 2: Table – 3:
Memory Comparison………………………………………………………………….5 PSNR values for conversion of images from NTSC to QVGA………………….27 Cycle count for various algorithms for NTSC image input………………………34
Implementation of Multimedia Libraries on BF - 533
1. INTRODUCTION Video signals are one of the ways of transferring visual information from one point to another. Earlier, video signals were mostly analog in nature. Digital video was mainly used for professional applications such as video editing. However, due to the developments in VLSI technology and falling costs of digital equipments, the average customer has access to digital video systems. We can see digital video systems in almost all the places. The digital video information may be from a VCR, DVD player, a channel on the local broadcast, cable television, or satellite system, the internet, game console, or one of many other sources. These then need to be displayed on digital displays such as the computer monitor, TV, LCD displays etc. Digital video processing has become so important that we have dedicated video processors for processing heavy video content signals such as video gaming and so on. During the transmission of video information or during video capture due to improper lighting conditions or due to problems in the capturing devices, the video signal may get distorted. In such cases, video frame enhancement forms an important role. Also while displaying the video; different display devices may have different resolutions. In such a case, image resizing algorithms may need to be used. When we are implementing such video processing algorithms, we need to do this on a DSPr. We have made use of Analog Devices BF-533 processor for this purpose. We shall initially give a brief overview of the BF-533 processor and the BF-533 EZ-KIT Lite that we have used. We then discuss about the image enhancement algorithms that we have implemented. For format conversion, we developed our own algorithm which suits our needs. We shall discuss this in detail. We shall then discuss about the implementation aspects on BF-533 and interfacing between the kits for real time operation.
Dept of E & C
-1-
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
2. BLACKFIN – 533 PROCESSOR 2.1 OVERVIEW The ADSP – BF 533, ADSP BF – 532 and BF - 531 processors are enhanced members of the BlackFin processor family that offer significantly higher performance and lower power than previous BlackFin processors while retaining their ease-of-use and code compatibility benefits. The three new processors are completely pin compatible, differing only in their performance and on-chip memory, mitigating many risks associated with new product development. The BlackFin processor core architecture combines a dual MAC signal processing engine, an orthogonal RISC-like microprocessor instruction set, flexible Single Instruction, Multiple Data (SIMD) capabilities, and multimedia features into a single instruction set architecture. BlackFin products feature dynamic power management. The ability to vary both the voltage and frequency of operation optimizes the power consumption profile to the specific task.
2.2 PERIPHERALS The processor system peripherals include: • Parallel Peripheral Interface (PPI) • Serial Ports (SPORTs) • Serial Peripheral Interface (SPI) • General-purpose timers • Universal Asynchronous Receiver Transmitter (UART) • Real – Time Clock (RTC) • Watchdog timer • General – purpose I/O (programmable flags) These peripherals are connected to the core via several high bandwidth buses, as shown in figure - 1. All of the peripherals, except for general-purpose I/O, Real-Time Clock, and Timers, are supported by a flexible DMA structure. There are also two separate memory DMA channels dedicated to data transfers between the processor’s memory spaces, which include external SDRAM and asynchronous memory. Multiple on-chip buses provide enough bandwidth to keep the processor core running even when there is also activity on all of the on-chip and external peripherals.
Dept of E & C
-2-
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
Figure – 1: Processor block diagram
2.3 CORE ARCHITECTURE The processor core contains two 16-bit multipliers, two 40-bit accumulators, two 40-bit arithmetic logic units (ALUs), four 8-bit video ALUs, and a 40-bit shifter, shown in figure - 2. The computational units process 8-, 16-, or 32-bit data from the register file. The compute register file contains eight 32-bit registers. When performing compute operations on 16-bit operand data, the register file operates as 16 independent 16-bit registers. All operands for compute operations come from the multi-ported register file and instruction constant fields. Each MAC can perform a 16- by 16-bit multiply per cycle, with accumulation to a 40-bit result. Signed and unsigned formats, rounding, and saturation are supported. The ALUs perform a traditional set of arithmetic and logical operations on 16-bit or 32-bit data. Many special instructions are included to accelerate various signal processing tasks. These include bit operations such as field extract and population count, modulo 232 multiply, divide primitives, saturation and rounding, and sign/exponent detection. The set of video instructions include byte alignment and packing operations, 16-bit and 8-bit adds with clipping, 8-bit average
Dept of E & C
-3-
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
operations, and 8-bit subtract/absolute value/accumulate (SAA) operations. Also provided are the compare/select and vector search instructions. For some instructions, two 16-bit ALU operations can be performed simultaneously on register pairs (a 16-bit high half and 16-bit low half of a compute register). By also using the second ALU, quad 16-bit operations are possible.
Figure – 2: Processor core architecture
The 40-bit shifter can deposit data and perform shifting, rotating, normalization, and extraction operations. A program sequencer controls the instruction execution flow, including instruction alignment and decoding. For program flow control, the sequencer supports PC – relative and indirect conditional jumps (with static branch prediction) and subroutine calls. Hardware is provided to support zero-overhead looping. The architecture is fully interlocked, meaning there are no visible pipeline effects when executing instructions with data dependencies. The address arithmetic unit provides two addresses for simultaneous dual fetches from memory. It contains a multi-ported register file consisting of four sets of 32-bit Index, Modify, Length, and Base registers (for circular buffering) and eight additional 32-bit pointer registers (for C-style indexed stack manipulation).
Dept of E & C
-4-
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
BlackFin processors support a modified Harvard architecture in combination with a hierarchical memory structure. Level 1 (L1) memories typically operate at the full processor speed with little or no latency. At the L1 level, the instruction memory holds instructions only. The two data memories hold data, and a dedicated scratchpad data memory stores stack and local variable information. In addition, multiple L1 memory blocks are provided, which may be configured as a mix of SRAM and cache. The Memory Management Unit (MMU) provides memory protection for individual tasks that may be operating on the core and may protect system registers from unintended access. The architecture provides three modes of operation: User, Supervisor, and Emulation. User mode has restricted access to a subset of system resources, thus providing a protected software environment. Supervisor and Emulation modes have unrestricted access to the system and core resources. The ADSP-BF53x BlackFin processor instruction set is optimized so that 16-bit opcodes represent the most frequently used instructions. Complex DSP instructions are encoded into 32-bit opcodes as multifunction instructions. BlackFin products support a limited multi-issue capability, where a 32-bit instruction can be issued in parallel with two 16-bit instructions. This allows the programmer to use many of the core resources in a single instruction cycle. The ADSP-BF53x BlackFin processor assembly language uses an algebraic syntax. The architecture is optimized for use with the C compiler.
2.4 MEMORY ARCHITECTURE The BlackFin processor architecture structures memory as a single, unified 4G byte address space using 32-bit addresses. All resources, including internal memory, external memory, and I/O control registers occupy separate sections of this common address space. The memory portions of this address space are arranged in a hierarchical structure to provide a good cost/performance balance of some very fast, low latency on-chip memory as cache or SRAM, and larger, lower cost and lower performance off-chip memory systems. Table -1 shows the memory comparison for the ADSP-BF531, ADSPBF532, and ADSP-BF533 processors. Type of memory Instruction SRAM/Cache Instruction SRAM Data SRAM/Cache Data SRAM Scratchpad Total
ADSP – BF531 16 K byte 16 K byte 16 K byte 4 K byte 84 K byte
ADSP – BF532 16 K byte 32 K byte 32 K byte 4 K byte 116 K byte
ADSP – BF533 16 K byte 32 K byte 32 K byte 32 K byte 4 K byte 148 K byte
Table – 1: Memory Comparison
Dept of E & C
-5-
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
The L1 memory system is the primary highest performance memory available to the core. The off-chip memory system, accessed through the External Bus Interface Unit (EBIU), provides expansion with SDRAM, flash memory and SRAM, optionally accessing up to 132M bytes of physical memory. The memory DMA controller provides high bandwidth data movement capability. It can perform block transfers of code or data between the internal memory and the external memory spaces. 2.4.1 INTERNAL MEMORY The processor has three blocks of on-chip memory that provide high bandwidth access to the core: • L1 instruction memory, consisting of SRAM and a 4-way set-associative cache. This memory is accessed at full processor speed. • L1 data memory, consisting of SRAM and/or a 2-way set-associative cache. This memory block is accessed at full processor speed. • L1 scratchpad RAM, which runs at the same speed as the L1 memories but is only accessible as data SRAM and cannot be configured as cache memory. 2.4.2 EXTERNAL MEMORY External (off-chip) memory is accessed via the External Bus Interface Unit (EBIU). This 16-bit interface provides a glue less connection to a bank of synchronous DRAM (SDRAM) and as many as four banks of asynchronous memory devices including flash memory, EPROM, ROM, SRAM, and memorymapped I/O devices. The PC133-compliant SDRAM controller can be programmed to interface to up to 128M bytes of SDRAM. The asynchronous memory controller can be programmed to control up to four banks of devices. Each bank occupies a 1M byte segment regardless of the size of the devices used, so that these banks are only contiguous if each is fully populated with 1M byte of memory. 2.4.3 I/O MEMORY SPACE BlackFin processors do not define a separate I/O space. All resources are mapped through the flat 32-bit address space. Control registers for on-chip I/O devices are mapped into memory-mapped registers (MMRs) at addresses near the top of the 4G byte address space. These are separated into two smaller blocks: one contains the control MMRs for all core functions and the other
Dept of E & C
-6-
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
contains the registers needed for setup and control of the on-chip peripherals outside of the core. The MMRs are accessible only in Supervisor mode. They appear as reserved space to on-chip peripherals.
2.5 DMA The processor has multiple, independent DMA controllers that support automated data transfers with minimal overhead for the core. DMA transfers can occur between the internal memories and any of its DMA-capable peripherals. Additionally, DMA transfers can be accomplished between any of the DMAcapable peripherals and external devices connected to the external memory interfaces, including the SDRAM controller and the asynchronous memory controller. DMA-capable peripherals include the SPORTs, SPI port, UART, and PPI. Each individual DMA-capable peripheral has at least one dedicated DMA channel. The DMA controller supports both one-dimensional (1D) and twodimensional (2D) DMA transfers. DMA transfer initialization can be implemented from registers or from sets of parameters called descriptor blocks. The 2D DMA capability supports arbitrary row and column sizes up to 64K elements by 64K elements, and arbitrary row and column step sizes up to +/32K elements. Furthermore, the column step size can be less than the row step size, allowing implementation of interleaved data streams. This feature is especially useful in video applications where data can be de-interleaved on the fly. Examples of DMA types supported include: • A single, linear buffer that stops upon completion • A circular, auto-refreshing buffer that interrupts on each full or fractionally full buffer • 1D or 2D DMA using a linked list of descriptors • 2D DMA using an array of descriptors specifying only the base DMA address within a common page In addition to the dedicated peripheral DMA channels, there is a separate memory DMA channel provided for transfers between the various memories of the system. This enables transfers of blocks of data between any of the memories—including external SDRAM, ROM, SRAM, and flash memory—with minimal processor intervention. Memory DMA transfers can be controlled by a very flexible descriptor-based methodology or by a standard register-based auto buffer mechanism.
Dept of E & C
-7-
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
2.6 SERIAL PORTS (SPORTs) The processor incorporates two dual-channel synchronous serial ports (SPORT0 and SPORT1) for serial and multiprocessor communications. The SPORTs support these features: • Bidirectional, I2S capable operation Each SPORT has two sets of independent transmit and receive pins, enabling eight channels of I2S stereo audio. • Buffered (eight-deep) transmit and receive ports Each port has a data register for transferring data words to and from other processor components and shift registers for shifting data in and out of the data registers. • Clocking Each transmit and receive port can either use an external serial clock or can generate its own in a wide range of frequencies. • Word length Each SPORT supports serial data words from 3 to 32 bits in length, transferred in most significant bit first or least significant bit first format. • Framing Each transmit and receive port can run with or without frame sync signals for each data word. Frame sync signals can be generated internally or externally, active high or low, and with either of two pulse widths and early or late frame sync. • Companding in hardware Each SPORT can perform A-law or μ-law Companding according to ITU recommendation G.711. Companding can be selected on the transmit and/or receive channel of the SPORT without additional latencies. • DMA operations with single cycle overhead Each SPORT can automatically receive and transmit multiple buffers of memory data. The processor can link or chain sequences of DMA transfers between a SPORT and memory.
Dept of E & C
-8-
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
• Interrupts Each transmit and receive port generates an interrupt upon completing the transfer of a data word or after transferring an entire data buffer or buffers through DMA. • Multichannel capability Each SPORT supports 128 channels out of a 1024-channel window and is compatible with the H.100, H.110, MVIP-90, and HMVIP standards.
2.7 INSTRUCTION SET DESCRIPTION The ADSP-BF53x processor family assembly language instruction set employs an algebraic syntax designed for ease of coding and readability. The instructions have been specifically tuned to provide a flexible, densely encoded instruction set that compiles to a very small final memory size. The instruction set also provides fully featured multifunction instructions that allow the programmer to use many of the processor core resources in a single instruction. Coupled with many features more often seen on microcontrollers, this instruction set is very efficient when compiling C and C++ source code. In addition, the architecture supports both user (algorithm/application code) and supervisor (O/S kernel, device drivers, debuggers, ISRs) modes of operation, allowing multiple levels of access to core resources. The assembly language, which takes advantage of the processor’s unique architecture, offers these advantages: • Seamlessly integrated DSP/CPU features optimized for both 8-bit and 16-bit operations • A multi-issue load/store modified Harvard architecture, which supports two 16-bit MAC or four 8-bit ALU + two load/store + two pointer updates per cycle • All registers, I/O, and memory mapped into a unified 4G byte memory space, providing a simplified programming model • Microcontroller features, such as arbitrary bit and bit field manipulation, insertion, and extraction; integer operations on 8-, 16-, and 32-bit data types; and separate user and supervisor stack pointers.
Dept of E & C
-9-
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
Code density enhancements include intermixing of 16 and 32-bit instructions with no mode switching or code segregation. Frequently used instructions are encoded in 16 bits.
2.8 DEVELOPMENT TOOLS The processor is supported with a complete set of Cross Core® software and hardware development tools, including Analog Devices emulators and the VisualDSP++ development environment. The same emulator hardware that supports other Analog Devices products also fully emulates the ADSP-BF53x processor family. The VisualDSP++ project management environment lets programmers develop and debug an application. This environment includes an easy-to-use assembler that is based on an algebraic syntax, an archiver (librarian/library builder), a linker, a loader, a cycle-accurate instruction-level simulator, a C/C++ compiler, and a C/C++ runtime library that includes DSP and mathematical functions. A key point for these tools is C/C++ code efficiency. The compiler has been developed for efficient translation of C/C++ code to BlackFin processor assembly. The BlackFin processor has architectural features that improve the efficiency of compiled C/C++ code. Debugging both C/C++ and assembly programs with the VisualDSP++ debugger, programmers can: • View mixed C/C++ and assembly code (interleaved source and object information) • Insert breakpoints • Set conditional breakpoints on registers, memory, and stacks • Trace instruction execution • Perform linear or statistical profiling of program execution • Fill, dump, and graphically plot the contents of memory • Perform source level debugging • Create custom debugger windows The VisualDSP++ Integrated Development Environment (IDE) lets programmers define and manage software development. Its dialog boxes and
Dept of E & C
- 10 -
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
property pages let programmers configure and manage all development tools, including Color Syntax Highlighting in the VisualDSP++ editor. These capabilities permit programmers to: • Control how the development tools process inputs and generate outputs. • Maintain a one-to-one correspondence with the tool’s command - line switches. Analog Devices emulators use the IEEE 1149.1 JTAG test access port of the processor to monitor and control the target board processor during emulation. The emulator provides full speed emulation, allowing inspection and modification of memory, registers, and processor stacks. Nonintrusive in-circuit emulation is assured by the use of the processor’s JTAG interface—the emulator does not affect target system loading or timing. In addition to the software and hardware development tools available from Analog Devices, third parties provide a wide range of tools supporting the BlackFin processor family. Hardware tools include the ADSP-BF533 EZ-KIT Lite™ standalone evaluation/development cards. Third party software tools include DSP libraries, real-time operating systems, and block diagram design tools.
Dept of E & C
- 11 -
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
3. ADSP BF – 533 EZ Kit Lite The EZ-KIT Lite has been designed to demonstrate the capabilities of the ADSP-BF533 BlackFin processor. The processor has IO voltage of 3.3V. The core voltage and the core clock rate can be set on the fly by the processor. The input clock is 27 MHz. A 32.768 kHz crystal supplies the Real Time Clock (RTC) inputs of the processor. The External Bus Interface Unit (EBIU) connects an external memory to the ADSP-BF533 device. It includes 16-bit wide data bus, address bus, and a control bus. Both 16-bit and 8-bit access is supported. On the EZ-KIT, the EBI unit is connected to SDRAM and Flash memory.
Figure – 3: BlackFin – 533 System architecture
Dept of E & C
- 12 -
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
3.1 VIDEO OUTPUT MODE In the video output mode, the PPI interface is configured as output and connects to the on-board video encoder device, ADV7171. The ADV7171 encoder device generates three analog video channels on DAC B, DAC C, and DAC D outputs. The PPI data connects to P7-0 of the encoder’s pixel inputs. The encoder’s PPI input clock runs at 27 MHz, and it is in phase with CLK IN of the ADSP-BF533 processor. The encoder’s synchronization signals, HSYNC and VSYNC, can be configured as inputs or outputs. Video Blanking control signal is at level “1”. The HSYNC and VSYNC signals can be connected to the ADSP-BF533 processor’s multiplexed sync pins and to the on-board video decoder, ADV7183, via the SW3 switch.
3.2 VIDEO INPUT MODE In the video input mode, the PPI interface is configured as input and connects to the on-board video decoder device, ADV7183. The ADV7183 decoder receives three analog video channels on AIN1, AIN4, and AIN5 input. The decoder’s pixel data outputs P15-8 drive the PPI data (PPI3-0 and PF15-12). The decoder’s 27 MHz pixel clock output can be selected to drive PPI clock
Dept of E & C
- 13 -
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
4. IMAGE ENHANCEMENT ALGORITHMS Due to improper lighting conditions, camera calibration etc the captured video frames may be of poor quality and hence they need to be enhanced. The common enhancement algorithms that we have implemented on the processor include histogram equalization, gamma correction, log transformation, edge detection and enhancement and auto white balancing.
4.1 HISTOGRAM EQUALIZATION The Histogram of a digital image with gray levels in the range [0, L-1], is the discrete function h(rk ) = nk where rk is the
k th
gray level and nk is the
number of pixels in the image having gray level rk . A low contrast image will have a narrow histogram. An image, whose pixels tend to occupy the entire range of possible gray levels with uniform distribution, will have an appearance of high contrast. Histogram equalization not only provides useful image statistics but also can be effectively used for image enhancement, image segmentation, image compression etc. Histogram matching is also known to be used for decrypting encoded images i.e. in steganography. Histogram equalization is however not reversible in nature and the histogram equalized image is unique in nature. Histogram equalization can be viewed as a transformation operation shown as follows
s = T (r )
where s is the transformed pixel value. The transformation is given by k
nj
j =0
n
sk = ∑
k = 0,1, 2,3........., L − 1 This equation could be directly implemented without any need for external parametric specifications. The original image and the histogram equalized images are shown below. These images are obtained from MATLAB.
Dept of E & C
- 14 -
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
Figure – 4: Histogram Equalization
4.2 GAMMA CORRECTION Many of the image capturing devices responds according to a power law. The transformation that may need to be applied to correct this power law is known as Gamma correction. In case the image is not gamma corrected, it may look bleached out. The power law transform applied is as follows
s = cr γ It can be seen that for γ < 1 the lower gray levels are compressed while for γ > 1 the higher gray levels are compressed. Gamma correction is important if displaying an image accurately on a computer screen is of concern. Images not corrected properly can look either bleached out or too dark. While reproducing colors accurately, value of gamma correction should be appropriately chosen as when the value of Gamma is varied the relative ratios of red, green and blue are also varied. Gamma correction is also used in the medical field when a certain portion of the magnetic resonance (MR) image needs to be viewed in particular and so on.
Dept of E & C
- 15 -
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
Figure – 5a
Figure – 5b
Figure – 5c
Figure – 5d: Gamma correction
4.3 AUTO WHITE BALANCING An object may appear different in color when it is illuminated with different light sources. This is due to the color temperature difference of the light sources, which induces the shift of the reflection spectrum of the object from the "true" color. When a white object is illuminated with low color temperature light source, the reflection becomes reddish. On the other hand, the high temperature light source causes bluish in color to the same white object. The human vision may not be able to distinguish the color difference due to the "color constancy" of human eyes. However, for an image recording media such as photographic films or video tapes, the reflected light of an object from a given light source will be recorded, and results in a color difference, if the light source is not standard Temperature, measured in degrees Kelvin, describes the relation between the degrees of heat applied on a light source to the specified color of light generated.
Dept of E & C
- 16 -
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
Figure – 6: Color temperatures for various light sources.
Color temperature can affect the scanner because the scanner is calibrated for fluorescent light. Different lighting conditions will produce adverse lighting effects in the output. For example a camera calibrated for 5600 degrees Kelvin (near fluorescent) reproduces light from a 2900 degree light source (tungsten) with a reddish tint. To compensate this color difference caused by various light sources, a white balance mechanism is employed in most electronic cameras. This could be through hardware. This could be done by using color sensors that are sensitive to Red, Green and Blue colors present in the environment. These are amplified using a logarithmic amplifier and then the obtained values could be stored in a ROM which could be used for providing appropriate gains to the three color components. Another method to achieve this is using software. Here we assume that the input is an RGB image. The first step is to isolate a white image in the desired lighting environment. This could be done either by locating areas of white in our datasets or another option would be to scan a white board or piece of paper by itself to be used as a reference image. The next step is to calculate the average value for each color channel, which is then used to determine a gain factor for the green and blue channels. The main obstacle regarding this method is the need for a white reference scan in every lighting condition the scanner is exposed to. Also, another problem with this method is that white balancing generally will correct only for color discrepancies, it will not significantly aid in adjusting the brightness of the image.
Dept of E & C
- 17 -
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
Hence to avoid user input we try to shift the histogram channels of two of the color planes so that they align with the third channel. The mean of all the three color planes
( μ r , μ g , μb ) are
calculated and then the scaling factor that
needs to be applied to each channel is calculated. Suppose we fix the red channel as the reference, then the scaling factors for the other two color channels are given as follows.
μg μr μ γb = b μr γg =
Where
γg
and
γb
are the scaling factors for the green and blue channels
respectively. The corrupted image and the AWB corrected image are shown below.
Figure – 7a: Corrupted image
Figure – 7c: AWB corrected image
4.4 EDGE DETECTION AND ENHANCEMENT Edge detection is a problem of fundamental importance in image analysis. In typical images, edges characterize object boundaries and are therefore useful for segmentation, registration, and identification of objects in a scene. An edge is a jump in intensity. The cross section of an edge has the shape of a ramp. An ideal edge is a discontinuity (i.e., a ramp with an infinite slope). The first derivative , where x assumes a local maximum at an edge. For a continuous image and y are the row and column coordinates respectively, we typically consider the and . Of particular interest in edge two directional derivatives detection are two functions that can be expressed in terms of these directional
Dept of E & C
- 18 -
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
derivatives: the gradient magnitude and the gradient orientation. The gradient magnitude is defined as , And the gradient orientation is given by . Local maxima of the gradient magnitude identify edges in . When the first derivative achieves a maximum, the second derivative is zero. For this reason, an alternative edge-detection strategy is to locate zeros of the second . The differential operator used in these so-called zeroderivatives of crossing edge detectors is the Laplacian . The Laplace operator is a very popular operator approximating the second derivative which gives the gradient magnitude only. The Laplacian is approximated in digital images by a convolution sum. A 3 x 3 mask for 4neighborhoods and 8-neighborhood is shown below
We have made use of the first Laplacian operator shown above. A Laplacian operator with stressed significance of the central pixel or its neighborhood is sometimes used. In this approximation it loses invariance to rotation
The Laplacian operator has a disadvantage that it responds doubly to some edges in the image. Once the edges are detected, these need to be added to the original image for enhancement purposes. The original image and the edge detection done using Laplacian operator is shown below.
Dept of E & C
- 19 -
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
Figure – 8: Edge detection using Laplacian Operator
4.5 LOG TRANSFORMATION This transformation tries to map the input Gray level pixels to a wider range in the output. This is usually used to expand the dark pixels in an image while compressing the lower range of pixels. This is similar to power law transformations such as gamma correction. However, the important characteristic of log transformation is that it compresses the dynamic range of images with large variations in the pixel values. The general form of log transformation is given as
s = c log(1 + r ) Where c is a constant.
Figure – 9: Log Transformation
Dept of E & C
- 20 -
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
5. IMAGE RESIZING 5.1 INTRODUCTION Image resizing finds application mainly in digital displays. With the advent in digital technology, there have been a variety of display devices starting from digital TV, digital cameras, handy cams, mobile phones and so on. Each of these devices has variable screen sizes. Even among the same family of devices, the size of the display unit differs. When the same data needs to be displayed in different devices, it needs to be resized. The constraint is more so in the case of video applications as the processing time available is very less. Most of the present algorithms deal with resizing by factors that are powers of two or integral factors. However, certain applications demand resizing by rational factors. Also, many of the present approaches are to resize in the compressed domain. This is useful during processing of stored images and videos as they are usually in the compressed form. However, in real time applications such as while using a handy cam, the video frames generated will be in the uncompressed format and these algorithms cannot be used efficiently. Most of the proposed algorithms are based on resizing both the rows and columns by the same factor. However, in certain video format conversions it may be required to resize the columns and rows by different factors. Various approaches are presently available for resizing of images for both in the spatial as well as the DCT domain. In particular, Dugad and Ahuja presented a simple fast computation technique for image halving and doubling in the frequency domain. Later, HyunWook Park, Senior Member, IEEE, YoungSeo Park, and Seung-Kyun Oh presented an algorithm for L/M-Fold Image Resizing in Block-DCT Domain Using Symmetric Convolution. Then, S K Mitra and J Mukherjee, have presented an algorithm for arbitrary resizing using DCT subband approximation. We developed an approach for resizing of images by rational factors in the spatial domain which is also computationally efficient. We make use of the results obtained by Dugad and Ahuja. Our approach eliminates the requirement of a low pass filter which is needed during up sampling and down sampling, thereby reducing the number of computations. The resizing is accomplished in a single step without passing through the steps of up sampling and down sampling. We look at the images from an Euclidean space perspective and derive transformations for image resizing. The proposed algorithm supports resizing by different scaling factors along both rows and columns.
Dept of E & C
- 21 -
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
5.2 IMAGES SEEN AS A VECTOR IN EUCLIDEAN SPACE A vector by definition is an ordered set of numbers. The numbers could be either complex or real. Depending upon this we define whether the vector is defined over a real field R or a complex field C. A collection of such vectors closed under addition and scalar multiplication constitutes a vector space. A vector space closed under the operation of dot product is a Euclidean space. A gray scale image of size m × n can be viewed as a mn-D vector defined over R. We shall deal with Gray scale images as the results derived hold good for color images also when extended to each of the color planes. A vector space of dimension mn can be described as the set of all m × n images in this context. Let us denote this vector space as I MN . The pixel values are constrained to lie between 0 and 255. Image resizing can be thought of a linear transformation from one vector space ( I MN ) to another vector space ( I PQ ) where M, N, P, Q are arbitrary integers. mn is the dimension of the original vector space and pq is the dimension of the vector space containing the resized images. Linear transformation can be viewed as a matrix multiplication. Thus if X MN is a vector i.e. an image in this case in I MN and YPQ is a vector in I PQ then we have
where
and
are the transformation matrices.
Thus if we know the transformation matrices we can convert any vector in I MN to a corresponding vector in I PQ . Since the transformation matrices are common for all the vectors, they are needed to be computed only once and then stored in the memory from where they can be accessed as required.
5.3 DISCRETE COSINE TRANSFORM OF IMAGES The DCT of a 2-D image X (m,n); 0 ≤ m ≤ M-1, 0 ≤ n ≤ N-1, is given by
Thus for an image block of size M × N, we get M × N DCT coefficients. We exploit the energy compaction property of DCT transformation for resizing. Consider that we have an M × N DCT coefficient block. Suppose that
Dept of E & C
- 22 -
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
we neglect the last i rows and the last j columns. We then get a DCT matrix of size P × Q where P = M −i and Q = N −j. When we apply the IDCT to this we will get an image of size P × Q. This image is an approximation of the original image. Similarly if we had added i rows and j columns, the extra elements being zeroes, we would have got an image of size P × Q, where P = M+i and Q = N +j. In this case we can view the original image to have been an approximation of the larger image. However since we are not adding more information to the image, both the images essentially contain the same information. We shall use these concepts as a basis for our resizing algorithm.
5.4 DETERMINATION OF TRANSFORMATION MATRICES The 2-D DCT of an image can be got by multiplying the image with a pre pre post and post matrices which are denoted by DMM and DNN respectively. The matrices can be calculated as follows.
Thus the DCT of an image XMN is given by
The resizing in the DCT domain is done by multiplying CMN by a pre and pre post post matrix respectively denoted by RPM and RNQ . These matrices for various cases can be computed as follows.
where I is the identity matrix and Z is the matrix of zeroes. Once resizing is done, we can obtain the resized image by taking the IDCT which is also done by matrix multiplications.
Dept of E & C
- 23 -
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
The matrices for obtaining the IDCT are given by
Thus the resized image YPQ is obtained by multiplying the resized DCT coefficient matrix with the IDCT matrices.
Combining all the above equation’s we get
Comparing this equation with
We get
Thus once the required transformation is given we can calculate post the A and ANQ matrices and apply them to the image. The equations required pre PM
pre post for computation of APM and ANQ can be calculated from the above set of
equations. They are given as follows
where R = min(M, P) and C = min(N,Q) The equation for the intermediate matrix TPN with the pre matrix is given by
Dept of E & C
- 24 -
obtained after multiplying
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
The final resized image matrix is given by
It should be noted that the above transformation is not always invertible. Invertibility exists only when P > M and Q > N. This is true because in such a case the resized image is an interpolated version of the original image. Hence there is no loss of information and thus we can exactly reconstruct the original image by decimation using suitable factors. However, when either P < M or Q < N there is a loss of information and the original image cannot be exactly reconstructed. From the point of view of linear transformations, it can be said that the transformation are not orthogonal in nature as the size of the vector is changing during the transformation.
5.5 DECOMPOSITION AND RECOMPOSITION IMAGES INTO SPATIAL BLOCKS
OF
The spatial correlation reduces when an image of large block size is taken. Hence, we split the image into smaller blocks and then apply the algorithm to each block. Pseudo code Input: M × N image Output: P × Q image ¾ Compute R = HCF (M,P) and C = HCF (N,Q) ¾ Compute input block size (Ri,Ci) as follows : Ri = M/R and Ci = N/C ¾ Compute output block size (Ro,Co) as follows : Ro =P/R and Co = Q/C pre
¾ Compute the pre matrix ARoRi and post matrix conversion of input block to the output block
post ACiCo
for
¾ Multiply each input block with the obtained matrices and arrange all together to obtain the resized image. Dept of E & C
- 25 -
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
Figure –10: Block Description of Resizing Algorithm
The resizing results obtained for different output sizes are as shown.
Figure –11: Image resizing
a) Original NTSC (525 × 720) image b) QVGA (240 × 320) image from NTSC c) CIF (352 × 288) image from QCIF d) QCIF (176 × 144) image from QVGA Dept of E & C
- 26 -
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
5.6 COMPARISON WITH STANDARD ALGORITHMS We have compared our algorithm with some of the standard algorithms that are currently available. We have used MATLAB as our simulation tool for these calculations. The images for which the calculations are done are standard images that are available in MATLAB. 5.6.1 PSNR CALCULATION The PSNR is calculated as follows
Where MSE is the mean square error
Table 2: PSNR values for conversion of images from NTSC to QVGA
5.6.2 COMPUTATIONAL REQUIREMENTS The resizing operation involves matrix multiplications. Let the cost of a multiplication be ρ and that of an addition be μ. The computational cost Γ for resizing a MN image to a PQ image is given by
The computational cost per pixel Γ n is given by
where Ri x Ci is the input image block size and Ro x Co is the output image block size obtained as shown in section 5.5.
Dept of E & C
- 27 -
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
6. IMPLEMENTATION ON BF – 533 6.1 PROCESSOR IMPLEMENTATION A real time implementation of the algorithms on the processor requires usage of DMA and Cache. The internal memory of BF-533 is not sufficient to hold the entire video frame. Hence each captured video frame is stored in the external SDRAM. The video frames are captured using the ADV7183 video decoder present on the BF-533 EZ-kit. The ADV7183 is an integrated video decoder that automatically detects and converts a standard analog base band television signal compatible with worldwide standards NTSC or PAL into 4:2:2 or 4:1:1 component video data compatible with 16/8-bit CCIR601/CCIR656. The advanced and highly flexible digital output interface enables video decoding and conversion in both frame-buffer-based and line-locked clock-based systems. For our purpose, the PPI is initialized and one of the DMA ports is setup in stop mode. The ADV7183 clock is linked to the PPI. The DMA channel then stores the image in the SDRAM starting from 0x0 locations. The video frame will have two fields - odd and even. The processing is then done after removing the blanking lines. While giving the output, the blanking lines need to be reinserted. There is also a video encoder on the EZ-kit. The ADV7171 is an integrated digital video encoder that converts Digital CCIR-601 4:2:2 16-bit component video data into a standard analog base band television signal compatible with worldwide standards. The on-board SSAF (Super Sub-Alias Filter) with extended luminance frequency response and sharp stop band attenuation enables studio quality video playback on modern TVs, giving optimal horizontal line resolution. The video encoder can also generate HSYNC, VSYNC, and FIELD timing signals. However, at a time either the encoder or the decoder could be active. Hence we need to use two kits for real time operation. In one kit the video frames are processed and then transferred serially to the second kit where some more image enhancement could be done and then is sent to the video encoder. The block diagram of the implementation is given below.
Dept of E & C
- 28 -
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
Figure – 12: Block Diagram of Hardware Setup
It is highly inefficient and time consuming to retrieve the image data from the external memory and process. For this purpose MDMA is made use of. Here, Ping-Pong DMA approach is used. Since most of the algorithms are point processing algorithms, the operations could be done on image blocks. For this purpose, the internal memory is divided into two blocks. The DMA is initialized for transfer of a suitable image block size into one of the internal memory blocks. When this DMA transfer is taking place, processing is done on the other block. Thus, it effectively seems that the operation is being done on an image that is present in the internal memory itself. While writing back into the external memory DMA is again made use of. Pseudo Code: ¾ Load image frame into external memory ¾ Initialize DMA transfer for Block A ¾ While (all the blocks are processed) .. • • • •
Wait until DMA transfer is complete If (odd cycle) Initialize DMA for Block B, Process on Block A Else Initialize DMA for Block A, Process on Block B Initialize DMA for transfer through SPORT
¾ End loop Since for real time operation, the image needs to be sent out serially, it is not written back into the external memory. Instead the SPORT DMA is initialized which starts transferring the frame blocks to the second kit. This is done in parallel along with the processing. For this purpose the control registers are
Dept of E & C
- 29 -
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
suitably initialized and also the clock frequency at which the data is to be transferred between the kits is suitably fixed. Pseudo code: Input: M × N raw NTSC image Output: M × N enhanced NTSC image ¾ Initialize the video decoder ¾ Initialize MDMA for transfer of image blocks from external memory to internal memory ¾ Initialize DMA transfer for the next block and process on the block transferred in the previous cycle ¾ Initialize DMA for SPORT transfer after suitably fixing the transmitter Clock ¾ Check for DMA completion interrupt before initializing next DMA transfer ¾ At the receiver kit a Video out code is run to display the received frames. Another method by which optimization could be done is by Cacheing the external memory. Here the external SDRAM is configured as a cache. In case the code is also large then we could have both instruction as well as data cache. The cache block size is suitably fixed. The replacement policy could also be selected as LRU (Least recently used) or FIFO. Once this is set, cache operation is transparent to the programmer. The processor takes care of the block transfers. However, while serial transfer is used, SPORT DMA needs to be used. We have implemented the resizing algorithm for a single image frame. The results are shown in the next section. The PSNR obtained after resizing in the processor will be lesser than that given in the table 2 as the processor is a fixed point processor and hence errors will be more.
Dept of E & C
- 30 -
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
6.2 RESULTS The results obtained after implementing the enhancement and resizing algorithms are shown in this section.
Figure – 13: Input Image and its Histogram
Figure – 14: Output Image and its Histogram
Figure – 15: Input image and the Gamma corrected image
Dept of E & C
- 31 -
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
Figure – 16: Image with red tinge and the White Balanced image
Figure – 17: Input NTSC (525x720) image and the resized QVGA (320x240) image
Figure – 18a Figure – 18b Figure – 18a: Image to be negated and transmitted Figure – 18b: Negated and received image in the second kit
Dept of E & C
- 32 -
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
Figure – 19: Original showing SPORT transfer between kits scaled to 25 percent
Figure – 20: QCIF image obtained from NTSC image scaled to 50 percent
Figure – 21a
Figure – 21b
Figure – 21c
Figure – 21a: Image used for edge detection Figure – 21b: Image after edge detection Figure – 21c: Image enhancement using edge detection
Dept of E & C
- 33 -
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
Figure – 22a Figure – 22b Figure – 22a: Original image for Log Transformation Figure – 22b: Log Transformed image
Algorithms Auto White Balancing Edge Detection Enhancement using Edge Detection Gamma Correction Histogram Log Transformation Image Negative Resizing( NTSC to QCIF)
Cycle count after implementing DMA Techniques 242,693,662 256,307,340 256,307,340 2,820,608,429 242,250,632 1,053,719,914 10,182,045 1,311,502,180
Table – 3: Cycle counts for various algorithms for NTSC image input
Dept of E & C
- 34 -
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
7. CONCLUSION AND FUTURE WORK 7.1 WORK DONE The work done by us can be summarized as follows: •
Development of a resizing algorithm.
•
Implementation of the following image enhancement functions on the processor: Histogram Equalization Gamma Correction Edge Detection Image enhancement using Edge Detection Log Transformation Image Negative Auto White Balance correction
•
Implementation of image resizing algorithm on the processor
•
Achieving Serial Port transfer between the kits
•
Optimization of all the above functions and algorithms for real time operation using DMA
•
Optimization of the functions for real time using Cache.
•
Video frame capturing from NTSC source.
7.2 SCOPE FOR FUTURE WORK Future work that could be done include: •
A single SPORT transfer is time consuming and is not suitable for real time operation. For this purpose, both primary and second channels of SPORT could be used to possibly double the speed of operation.
•
The programs could be implemented on a higher version of Blackfin processor which supports both Video-In and Video-Out on the same kit thereby avoiding serial transfer.
•
For digital zooming, the resizing algorithm itself could be used recursively.
Dept of E & C
- 35 -
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
8. PUBLICATIONS Papers based on this project have been submitted/presented in the following conferences/symposiums: 1. A paper tilted “Resizing of Images by Arbitrary Factors in the Spatial Domain and Implementation on BlackFin BF533 Processor” has been submitted for International Conference on Visual Information Engineering 2006, Bangalore conducted by the Institution of Engineering and Technology (IET). 2. A paper titled “Resizing of Images by Arbitrary Factors in the Spatial Domain and Implementation in real time on Blackfin-533 DSP Processor” was adjudged the best paper in the electronics stream at “Blueprint – Paper presentation contest” conducted by IEEE, NITK, conducted as a part of Engineer – 2006 , a national level technical symposium. 3. A manuscript on the Implementation of the project is currently under preparation.
Dept of E & C
- 36 -
NITK – Surathkal
Implementation of Multimedia Libraries on BF - 533
9. BIBLIOGRAPHY AND REFERENCES [1] J.Mukherjee, S.K.Mitra: Arbitrary Resizing of Images in DCT space— IEE Proc.-Vis. Image Signal Process., Vol. 152, No. 2,Apr 2005 [2] Rafael.C.Gonzalez, Richard.E.Woods: Digital Image Processing—Second Edition— Pearson Education, 2003 [3] Yung-Cheng Liu, Wen-Hsin Chan and Ye-Quang Chen: Automatic White Balance for Digital Camera— IEEE Transactions on Consumer Electronics, Vol. 41. No. 3. AUGUST 1995 [4] Jason C. Yang: A Light Field Camera For Image Based Rendering— Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, May 2000 [5] Keith Jack: Video Demystified—Third Edition—LLH Technology Publishing, 2001 [6] ADSP-BF533 Blackfin Processor Hardware Reference Manual [7] HyunWook Park, YoungSeo Park, and Seung-Kyun Oh: L/M-Fold Image Resizing in Block- DCT Domain Using Symmetric Convolution — IEEE Transactions on image processing , Vol. 12, No. 9, Sept 2003 [8] J.Mukherjee, S.K.Mitra: Resizing Of Images in the DCT space by Arbitrary factors— International Conference on Image Processing (ICIP), 2004. [9] Rakesh Dugad, Narendra Ahuja: A Fast Scheme for Image Size Change in the Compressed Domain—IEEE Transactions on circuits and systems for video technology, Vol. 11, No. 4, April 2001 [10] Ci Wang, Hong-Bin Yu, and Meng Zheng: A Fast Scheme for Arbitrarily Resizing of Digital Image in the Compressed Domain —IEEE Transactions on Consumer Electronics, Vol 49, No 2, MAY 2003 [11] Gilbert Strang: Linear Algebra and Its Applications—Third Edition— Thompson Books/Cole, 2005 [12] Fundamentals of Image Processing http://www.cs.dartmouth.edu/~farid
Dept of E & C
- 37 -
-
[email protected]
NITK – Surathkal
