A 1.1W Single-Chip MPEG-2 HDTV CODEC LSI for Embedding in Consumer-oriented Mobile CODEC Systems Hiroe Iwasaki, Jiro Naganuma, Yasuyuki Nakajima, Yutaka Tashiro, Ken Nakamura, Takeshi Yoshitome, Takayuki Onishi, Mitsuo Ikeda, Takaaki Izuoka and Makoto E n d o NTT Cyber Space Laboratories, NTT Corporation 1-1 Hikarinooka, Yokosuka-Shi, Kanagawa Pref. 239-0847, Japan [email protected]

.

AbsbacI-This paper proposes a 1.1 W single-chip MPEG-2 HDTV codec LSI for embedding in consumer-oriented mobile codec systems, and demonstrates its flexibility and usefulness. This amhitecture consists of a halfduplex 720/30P encoding core. a halfduolex pore. m d i n.DSP. il .__ RWC. ~ ~ ~lOSDl ~ . ...decodins .~ ...., an .~~ ___. - -,a d a multiplexerlde-multiplexer core with a dual-memory scheme for supplying data at high speeds. The LSI, which integrates 3.8

-

~~

~~~~

-.... ~~~~

~~

...-

~

Flexible resolution conversion: The LSIs for consumes are connected to various types of resolution input/output equipment, resolution conversion(FRC) is very imp0rtant for implementing flexible mapping between inputloutput resolution and codec resolution. However, FRC requires another base band pixel data transfer while

~. . .. ..... --. ..... ...-~~. .. ~

for cOmUmerSto

HDTV quality equipment on a

more widespread scale.

I. INTRODUCTiON Recent progress in video and audio compression technology has made it possible to provide a much greater volume and range of digital multimedia. The MPEG-2 standard [I] has emerged as a method for effectively compressing video and audio With high quality. In Particular, this standard is currently seeing extensive world wide use in many transmission and storage applications, such as digital satellite broadcasting, digita1 cable television, video conferencing, DVD and CD-ROM storage media, video on demand, and time-shifted viewing. The digitization of HDTV format T V broadcasting is descending upon the world in the form of a global wave. This can be seen, for example, in the advent of terrestrial digital broadcasting, which will be offered in Japan by the end of 2W3. This'spread of HDTV contents will require HDTV codec systems for not only professionals but also consumers. In recent years several single-chip MPEG-2 codec LSIS [21, 131, [41, 151, [61, 171, 181 have been developed to implement MPEG-'2 .codecs, but generally they are for MP@ML or 422P@ML. Some of them [6], [71, [SI, 191 provide HDTV codecs. but their power consumption is too large to use for consumer systems such as portable HD camcoders, portable HDD recorders, and small boards for PCs. We have also developed a professional-quality single-chip MPEG-2 422P@HL CODEC LSI 191 with 5.0 W for terrestrial broadcasting. There are three major problems involved in implementing single-chip MPEG-2 HDTV codec LSIs for embedding in consumer-oriented mobile codec systems, as follows:

tive communication. Full-duplex codecs require individual memories for full encoding and decoding processing. Low power consumption: Low power consumption of less than 1.5 W is required for embedding in consumeroriented mobile codec systems such as portable HD camcoders and portable HDD recorders. T~ solve these we a 1.1 w single. chip M P E G . ~HDTV codec LSI for embedding in consumeroriented mobile codec systems. This architecture consists of a half.duplex 720130~ core, a half.duplex 10801 decoding core, an audio DSP, a ~ s c , a multiplexer/demultiplexer core with a dual-memory scheme for supplying data at high speeds. The LSI, which integrates 3.8 million mnsistors on a 9.7 mm x 9.7 mm die using the 0 . 1 3 + ~ sevenmetal CMOS process, implements 720/30P encoding with 1.1 W, 10801decoding with 0.8 W, and full-duplex 480P encoding and decoding simultaneously with 1.4 w . Section 2 describes the chip Structure of the paper. architecture, and Section 3 explains how it was implemented in a fabricated LSI and evaluates the resultant LSI's performance. Section 4 presents the applications for this LSI. 11. ARCHITECTURE A.

The data transfer rate requirements in video encoding or decoding are shown in Figure 1. To implement half-duplex encoding or decoding with flexible resolution conversion, or full-duplex encoding and decoding, 72013OP encoding and 10801 decoding, which need one SDRAM as an external memory, are chosen to achieve maximum resolution. Utilizing dual memories. they implement full-duplex encoding and

9-2-1 0-7803-7842-3/03/$17.00 0 2003 IEEE

IEEE 2003 CUSTOM INTEGRATED CIRCUITS CONFERENCE

177

.-.-E-"

(a) Data Row

in codec mode.

I -

Rg..l. Data mnsfer quiremenu.

(b) Data Raw in encoding mode

(c) Data flow

I

1

I

Fig. 3. Dual memory scheme.

various mixing functions is executed on a real-time OS with multi-tasking, which makes it easy for various functions to run on the same RISC.

Fig. 2. Block diagram.

decoding including audio encoding, decoding and systems. Half-duplex encoding and decoding with flexible resolution conversion is also implemented. This dual-memory scheme achieves performance optimization in the encoding, decoding and codec modes, a target range less than that of 720/30P encoding, full decoding of less than 10801, and a codec of less than 480P. The hardwarelsoftware partitioning must be optimized in terms of power consumption and shortened development time. The main functions with enormous computational complexity, such as video encoding and decoding, are mapping application-specific hardware because of their power. Audio encoding processing is mapped on DSP to enable the supporting of a multi audio encoding format. This partitioning is possible not only for half-duplex encoding or half-duplex decoding but also for full-duplex encoding and decoding. To support full-duplex encoding and decoding with low power consumption, the RISC software integrates a number of processing functions. Audio decoding processing with the least computational complexity is merged in RISC middleware, and the muddemux controller function is merged in RISC software. The video decoding controller and total controller functions are also executed in RISC software. Software with

178

in decoding mods

B. Hardware Architecture

I ) Block Diagram: The block diagram of this LSI is shown in Figure 2. It consists of a half-duplex 72Ol3OP video encoding core(E-CORE) with 32-bit RISC(VRISC), a halfduplex 10801 video decoding core(D-CORE), a total control RISC(TRISC), an audio DSP, a multiplexerlde-multiplexer core(MDX), and flexible resolution conversion(FRC) with a dual-memory scheme for supplying data at high speeds. The data path to SDRAM-A from SDlF is active in the encoding or codec modes, and the data path to SDRAM-A from MIF-A is active in the decoding mode. The data path to SDRAM-B is connected to FRC, D-CORE, TRISC, and DSP through the MIF-B. The E-CORE consists of several types of applicationspecific hardware and the RISC. The core executes MPEG-2 video encoding processing, and outputs elementary streams to the MDX. The encoding core makes it possible to achieve 720/30P encoding in real-time in the encoding mode. The D-CORE consists of several types of application-specific hardware. Its addressing for frame memory for decoding is controlled by the TRISC software. The decoding core and many processing functions share one memory?and the memory

9-2-2

Hardware Layler

Fig. 4.

Chip

Software Layer Configuration

controller allows the passing of various kinds of data for several blocks in the codec mode. The decoding core enables 10801 decoding in real-time in the decoding mode. The MDX has two buffers for full-duplex, and its control is executed by TRISC software.

2) Dual memory scheme: The dual memory scheme, which consists of two individual memories, is shown in Figure 3. The memories are 128 Mbit and 32 bit bandwidths. The full-duplex codec processing is shown in Figure 3(a). One memory is for E-CORE, and the other is for D-CORE. the DSP for audio encoding and the TRISC for audio decoding. The half-duplex encoding processing is shown in Figure 3(b). One memory is for E-CORE, and the other is for DSP in audio encoding and FRC. The FRC converts from higher resolution to lower resolution. The half-duplex decoding processing is shown in Figure 3(c). One memory is for D-CORE, and the other is for DSP in audio decoding and FRC. The FRC converts from lower resolution to higher resolution. For example, the FRC makes it possible to convert from 72Ol3OP resolution to 10801 resolution. This scheme makes it possible to convert video resolution in decoding or encoding processing, and the LSI can provide flexible connection to various kinds of inpulloutput equipment.

C. Sojiwore Archiiecture The software architecture is shown in Figure 4. It consists of three layers: the bottom is the hardware layer; the middle is the OS layer, which consists of pITRON and drivers; the top is the user layer, which includes the main task, the video task, the audio task, and so on. The tasks in the top layer are shown in Figure 5 in codec mode. Tasks divided into functional features are in a treelike structure, and these tasks communicate only with next tasks, avoiding complex control and communication. The main task has the highest priority because of its urgency, and tasks for audio decoding are assigned the lowest priority for continuous processing. The video task, the DISP task, and the VDEC control task are assigned higher priority than the audio decoding tasks. and each of them has a deadline for display in every frame.

Fig. 5. Task Configuration.

111. IMPLEMENTATION

The physical features of the fabricated LSI are shown in Table I. The LSI integrates 30.7 million transistors in a 9.7 mm x 9.7 mm die size using 0.13pm 7-level metal CMOS technology. The clock frequency in half-duplex 720/30P encoding mode is 144-MHz with 1.5-V to the internal circuits and 3.3-V to the U 0 circuit supply source. The half-duplex 10801 decoding mode is at 1 4 4 M H z with 0.8 W, and the full-duplex 480P codec mode is at 108-MHz with 1.4 W.The chip is mounted on a 457-pin BGA. A micro-photograph of the LSI is shown in Figure 6. All logic circuits on the chip were constructed using only standard cells in order to shorten design time. Hard macros were not used except for the memories and TRISC. Encoding and decoding cores on the chip used modified versions of our previous design cores [71, (91. The function features of the LSI are shown in Table 11. The MPEG-2 standards for the video encoding algorithms are supported by various profiles and levels with a single chip. And some audio encoding formats are supported by firmware on DSP and RISC. The maximum encoding size and rate support 1280 x 720 encoding at up to 30 frames per second, and the decoding size and rate suppolt 1920 x 1080 decoding at up to 30 frames per second. IV. APPLICATION Since the LSI provides an HDTV codec with low consump lion, it is suitable for portable HD camcoders. portable HDD recorders, and so on. Examples are shown in Figure 7. This LSI with low power consumption and HDTV quality can be embedded in portable HD camcoders, portable HDD recorders, mobile PCs, and mobile communication terminals. V. CONCLUSION This paper proposed a 1.1 W single-chip MPEG-2 HDTV codec LSI for embedding in consumer-oriented mobile codec

9-2-3

179

TABLE I PHYSICAL FEATURES Technology Number of transistors Die si= Clock frequency Supply vollage Pawcr consumption

Package Extsml memories

0.13-pm 7-level metal CMOS 30.7 million 9.7 mm I 9.7 mm 81 / 108 / 144 MHz 1.5 W3.3 V O.l-W(at MPEG-2 10801 decoding) 1.1-W(at MPEG-2 720/30P encoding) 1.4-W(al MPEG-2 480P Full-Duplex) 457-pin B G A ( 1 7 m I 17mm) 128-Mbit(32-bit) SDRAM I 2 TABLE II Fig. 6. Chip Photo.

FUNCTIONAL FEATURES Vldeo Profile and level

Mobile mmmpication

MPEG-2 SP.MP@ML,H-I4 (for encoding) MPEG-2 [email protected] . (far . decodine) N m w : .I13.5/+99.5(H).-57.5/+57.5(V) Wide: -225.5/+211.5(H), -1 13.5/+1 l3.5(V) 1280 I 720 at up to 30 fps (for encoding) 1920 x 1080 a1 up to 30 fps (far decoding) . I _

Scarch range Resolution and rate

Audio Sampling Encoding format System

U 0 format

32 KHz. 44.1 KHz. 48 KHz MPEC-I Layerll. MPEG-I Layerlll MPEG-2 T S

MoDlle PC

P a l e nDD remder

systems, and demonstrated its flexibility and usefulness. The architecture consists of a half-duplex 720/30P encoding core, a half-duplex 10801 decoding core, an audio DSP, a RISC. and a multiplexerlde-multiplexer core with a dual-memory scheme for supplying data at high speeds. The LSI, which integrates 3.8 million transistors on a 9.7 mm x 9.7 mm die using the 0.13-pm seven-metal CMOS process, implements 720/30Pencoding with 1.1 W, 10801decoding with 0.8W, and full-duplex 480P encoding and decoding simultaneously with 1.4 W. This LSI will help make it possible for HDTV quality equipment to gain more widespread use among consumers.

Fig. 7. Examples.

151

161

REFERENCES

Ill lSO/lEC 13818-1/2/3 International Standard. - Generic Coding of Moving Piclures omi

Infomation Technology Associated Audio: Sys-

remsVduoUAudio -, November 1994 121 M. Miruno. Y. Ooi, N. Hayashi. 1. Goto. M. Hozumi, K. Furuta. A. Shibayama. Y. N a b w a . 0. Ohnirhi. S. Y. Zhu. Y. Yakoyama. Y. Katayama. H. Takano. N. Miki. Y. Ssndn. and M. Yamashina. "A 1.5-W singlechip MPEG-2 MP@ML vidm encoder wilh low power motion estimation and clocking." IEEE Solid.Slorc Circuits, vol. 32. pp. 1807-1816, November 1997. 131 E. Miyagoshi. T. Araki, T. Sayama, A. Ohbani. T. M i n a " . K. Obmofo. H. M a m a . T. Marishige. A. Watabe. K. Aoki. T, Mit. rumori, H. Imanishi. T. Jinbo. Y.Tan&. M. Taniyama. T. Shingou, singleT. Fukumoto. H. Morimoto. and K. Aono, "A 100mmZ0 . 9 5 ~ chip MPEGZ MPBML video encoder wilh a 128GOPS motionestimator and B multi-tasking RISC-type conuollcr:' in InlemtioMI Solid-Stma

171

[E] 191

Circuits Conference. 1998, pp. 3&3l. 141 E. Ogura, M. Takashim. D. Hiranaka, T. Ishikawa. Y. Yanagita. S . Suruki. T. Fukuda, and T. Ishii. "A 1 . 2 ~single-chip MPEG2

180

9-2-4

M P 4 M L video cncoder LSI including wide search range motion estimation and 8lMOPS controller." in Inremationol Solid-State Circuits Conference. 1998. pp. 32-33. S. Ishiwala, T. Yamakagc. Y. Tsuboi. T. Shimazawa. T. Kitalawa, S. Michinaka. K. Yahagi. H. Takcda, A. 0"s. T. Kodama. N. Mmumoto. T. Kamei, T. Miyamoti. G . Ootomo. and M. Mauui. "A SinglcChip in MPEG-2 C d c c B a d on Customizablc Media Mimopmessor:' IEEE 2W2 Custom Inregrated Circuits Confcrcncr, 2032. pp. 163-166. S. Kumaki, H. Takata, Y. Ajioka. T. Ooishi. K. Ishihara. A. H"i. T,Tsuji. T, Watanabe. C. Morishima. T. Yorhizawa, H. Sato. S. Hatloti. A. Koshio. K. Tsukamoto, and T. MaUumw. "A W-mmz 0.7-W Single-Chip MPEC-2 422POML Video. Audio, and System E n d e r With a M-Mb Embsddd DRAM For Portable 422PBHL E n d e r System:' in IEEE Joumal of Solid-Slate Circuits. 2oM. pp. 4 5 M 5 4 . M.Ikeda, T. Kondo, K. Nitta. K. Suguri. T. Yoshitome. T. Minami. J. Naganuma, and T. Og-. "An MPEG-2 Video hcodsr LSI wilh Scalability for HDTV bawd on Thnc-layer Cooperative Archiactwe:' in Design. Aulomation and Test in Eumpc Confcrmce 1999. March 1999. pp. 44-50, LSt Logic Corporation., "DoMiNo: (2002) LSI LOGIC h o m page.." [Online]. Available: http:/lwww.lsil~gie.em. H. Iwasaki. 1. Nagamma. K. Nitta. K. N a k a " , T. Yoshitoms. M. O g w Y, Nakajima, Y. Tashim, T. Onishi, M. Ikcda, and M.Endo, "Single-chip MPEG-2 422PbHL CODEC LSl with Multi-chip Configuration far Large Scale Processing beyond H M V Lcvel." in &sign. Automation and Test in Eumpr Conference 2W3 Designers' Forum, March 2033, pp. 2-1.