USOORE42848E

(19) United States (12) Reissued Patent

(10) Patent Number:

Sato et a]. (54)

(45) Date of Reissued Patent:

INFORMATION TRANSMISSION SYSTEM

(56)

U~S~ PATENT DOCUMENTS

METHOD, IMAGE PICKUP DEVICE, AND

5,033,214 A *

3/1991 Miida ~~~~~~~~~~~~~~~~~~~~~~ ~~ 348/2071

5,956,082 A *

9/1999

Yamazaki ........ ..

348/2221

COMPUTER PROGRAMMED PRODUCT

6,208,378 B1*

3/2001 Barraclough et a1.

348/2111

6,522,889 B1*

2/2003

Aarnio ............. ..

Inventors:

6,701,058 B1 *

3/2004

Tsubaki . . . . . .

. . . . . .. 386/226

9/2004 Brumitt et a1.

.. 348/20711

Kunio Sato, Ome (JP); Nobuo Iizuka,

HamuraUP)

6,798,445 B1* 2003/0025803 A1*

10/2004

* Cited by examiner

(21)

Primary Examiner * Duc Q Dinh

_

(22) Filed:

(74) Attorney, Agent, or Firm * HoltZ, HoltZ, Goodman &

Aug. 23, 2007

Chick, p_C_

Reissue of Related US. Patent Documents

(57)

'

(30)

348/2181

Matsushima ............ .. 348/2221

(73) Assignee: Casio Computer Co., Ltd., Tokyo (JP) Appl' NO': 11/895,260

455/456.5

2/2003 Nakamura et a1. .

2004/0201717 A1*

(64)

Oct. 18, 2011

References Cited

USING LIGHT AS COMMUNICATION MEDIUM, INFORMATION TRANSMISSION

(75)

US RE42,848 E

ABSTRACT

A CPU 36 of a light emitting unit 30 fetches ith bit of trans

Patent No.:

6,933,956

m1ss1on information TX stored in a transm1ss1on data

Issued'

Aug“ 23’ 2005

memory 33 in synchronization With a clock signal CK from a

APPI- NOJ

10/246,042

timing generator 35. The CUP 36 determines the bit value and

PCT Flled3

seP- 18, 2002

fetches a ?rst pattern sequence SA from a pattern data memory 44 When the bit is logic signal 1, and fetches a second pattern sequence SB from a pattern data memory 34 When the

_

.

Foreign Application Priority Data

.

.

.

.

.

.

bit is logic signal 0, and outputs the ?rst pattern sequence SA Sep. 21, 2001 Aug. 12, 2002

(JP) ............................... .. 2001/288428 (JP) ............................... .. 2002/235225

and second pattern sequence SB to a light emitting section 37. While, a CPU 39 of a light receiving unit 40 extracts a pixel area With a time sequence luminance variation pattern from a

(51) Int, Cl, G09G 5/10 H04N 5/232

frame image captured into a capture image buffer 46, and binarizes the luminance variation pattern, and performs logic determination. A liquid crystal display 44 displays informa 31012;” llght emlmng area to be overlapped Wlth a “bled

(200601) (200601) (52) US. Cl. .................... .. 345/690; 345/204; 348/2112

(58)

Field of Classi?cation Search ................. .. 345/690

'

See application ?le for complete search history.

20 Claims, 35 Drawing Sheets

42

1'4

4w

J45

IMAGE SECTION PICKUP

49

47

44

4B

Q

PATTERN DATA

P

46

/

CPU

memonv

CAPTURE IMAGE

REFERENCE

BUFFER

IMAGE BUFFER

5“ N

51

DISPLAY

BUFFER

FRAME TIME

5&2???

LIQUID CRYSTAL DISPLAY

CORRELATION EVALUATION IMAGE BUFFER

TIMING GENERATOR

/52

I

~53

, ‘ WORKING BUFFER '~54 FOH BINARIZATION

PCK

[55 UPDATE REQUEST 7

552

US. Patent

Oct. 18, 2011

Sheet 1 0f 35

US RE42,848 E

>52 \\/

.50:GE -am ZOF um M

Z\ OF m >mE

025 F

mw/

02mt
.ZE5<“LmOZD

\60A

20 4 ZOE.me

i

US. Patent

Oct. 18, 2011

US RE42,848 E

Sheet 2 0f 35

gi?

<

mM685Q0a:358.°6:;

>me

0 _.

HT5l 2

mm

5Foo mm

o

Tlo

mud!"

le?5%:

A rum?n

m

US. Patent

Oct. 18, 2011

cv~\azoc.bmw

alul

.6. .h..%G

-\8\

US RE42,848 E

Sheet 3 0f 35

.a? qu\m w

JO 2 0 5 % zam0ot6k.w

z5m0.ut8kn.

moz w zoFum<0wKof25th5“.025.%a-_<26_m mozaw \

a/\\/

US. Patent

Oct. 18, 2011

Sheet 4 0f 35

US RE42,848 E

53

FIG.4A

31~

QD

43

/

FIG.4B 42

FIG.4C 44

US. Patent

0a. 18, 2011

Sheet 5 0f 35

US RE42,848 E

FIG.5A

29

[33

/34

TRANSMISSION DATA MEMORY

/

PATTERN DATA MEMORY \

?

[35

P

I /36

TIMING

_

GENERATOR

T

I

[37

/Z\/

LIGHT EMITTING CPU

T

sEcTION

4”

K

\ CK 42

FIG.5B /45

1-

IMAGE

/Z_-—v

PICKUP

)7,

SECTION

/

49

59

CPU

\

p

PATTERN DATA N

MEMORY

I

45\ cAPTURE IMAGE

—

BUFFER

47

T

DISPLAY

L

_

BUFFER

T

T

_

REFERENCE

T

IMAGEBUFFER

~

51

/52 _

44

5°

FRAME TIME

8%‘325255 \ LIQUID CRYSTAL _ DIsPLAY

T

coRRELATION

48

\

—

TIMING

.Txea‘ITF'asn

T53

GENERATOR

WORKING BUFFER ~54 T

T

FOR BINARIZATION

PCK

43\

[55 UPDATE REQUEST N

snag-{ER

=|

LIST LIsT ENTRY

551 #552

BIT BUFFER

553

US. Patent

Oct. 18, 2011

Sheet 6 0f 35

US RE42,848 E

FIG.6

Ema .EIIIHII 1:11:35 53:1

65A

6?A

“Z 59A

61A

Q“ V

60A

‘1

I

25D

.... “Tower

55', cu

/ H\l

/ 63A

A

"

II]

55:1

66A 69

68A

1 64A

_/STA

US. Patent

0a. 18, 2011

Sheet 7 0f 35

US RE42,848 E

FIG.7

(LIGHT EMITTING PROCESSING ) II

FETCHES ONE BYTE RETURN TO MOST SIGNIFICANT YTE IN THE CASE OF DATA END) I FETCH 1 an

/ S10

S11

S1 2 LOGIC

DETERMINATION

'-

L06": 1

~

“LOGIC 0"

fsm

[$13

FLASH LIGHT EMITTING SECTION 37 WITH

FLASH LIGHT EMITTING SECTION 37 WITH

SECOND PATTERN

FIRST PATTERN

SEQUENCE

SEQUENCE

END EQUIVALENT FOR ONE BYTE 7

S15

US. Patent

0a. 18, 2011

Sheet 9 0f 35

US RE42,848 E

FIG.9

CB SA

H

SA

m

s9

_;

LIGHT

.... ..p

E ITTING 1

SIDE)

‘ n 339% SIDE)

:

5

g

5

CG

US. Patent

Oct. 18, 2011

US RE42,848 E

Sheet 10 0f 35

FIG.10 (ngHT RECEIVING ROCESSING

I [$20 INITIATE TIMING GENERATOR f321

FRAME BUFFER REGISTRATION

PROCESSING

No

a

$22

YES

SIGNAL DETECTION AND BIT FETCHING

/$23

PROCESSING

DISPLAY PROCESSING

f824

US. Patent

Oct. 18, 2011

US RE42,848 E

Sheet 11 0f 35

FIG.11

G.

FRAME BUFFER EGISTRATION PROCESSIN

G) $21 A

CAPTURE FRAME IMAGE

" FILTER PROCESSING

/ 821 B

sucu As SMOOTHING 1

CALCULATE MOTION VEcTOR / 8210 WITH REFERENCE FRAME

AMOUNT

521”

OF MOTION Is WITHIN CORRECTION

THRESHOLD

STOPQ§§E%“}EP%“SA%%§"°

PERFORM MOTION

OORREOTION AND

POINTER N 0F FRAME

STIS’L'Z'ilg?'FUL-ggmg?m

"ME SE?BEQQEYBUFFE"

POINTER N OF FRAME TIME

f521G

SEQUENCE BUFFER

U

UPDATE REFERENcE FRAME IMAGE

[521 F

7 UPDATE FRAME BUFFER

y 321“

PosITION POINTER n ~— (n+1)mod N

I

STORE To

DISPLAY BUFFER

II

( RETURN )

/521'

US. Patent

Oct. 18, 2011

US RE42,848 E

Sheet 12 0f 35

SIGNAL DETEcTION AND

BM FETCHING PROCESSING

[$23A INITIATE UPDATE REQUEST UST

[8238

I

SET gSESTItIgI?D \11ALUE

OBTAIN AVERAGE IMAGE

=_ I

WHERE DOT VALUES 0F

LESS THAN ' o

FRAME IMAGE GROUP OORRESPONDING T0 ON

TIMING OF LIGHT P

/S23°

fszm

GENERATE BINARIZED IMAGE FROM cORRELATION EVALUATION IMAGE WITH THRESHOLD VALUE

l

$23H

I

/

PROVIDE LABELING TO RESPECTIVE AREAS AND

OBTAIN AVERAGE IMAGE

A°3§E§§§EET°I3€RRQ€ES USING AS FIRST PATTERN

$R§?I%?I?Ié’€b%%%3"

SE°%%’$%%§?J§?“E

TIMING OF LIGHT P

‘

CORRESPONDING To OFF

Gm

OBTAIN VARIED-WIDTH IMAGE HAVING MAXIMUM

VALUE FOR EACH DOT POSITION OF THE ENTIRE FRAME IMAGE GROUP

[52$

GENEIsIESIa'e'ERI'IAGE EVALUATION IMAGE WITH VALUE OBTAINED BY

PERFORMING SIGN INVERSION 0N THRESHOLD VALUE HRESHOLD VALUE LESS HAN = 1, MORE THAN = 0)

‘

[$235

°%B‘A§é€§8'a?“ EVALUATION IMAGE

/S23F

mmmmm

EVALUATION IMAGE OF

ABSOLUTE cORRELATION EVALUATION IMAGE!

‘

Pm"

PROVIDE LABELING TO RESPECTIVE AREAS AND

ADD cENTRAL cOORDINATES

0F RESPEcTIVE AREAS

"SILSISNSSECSSDUIQTI?" REQUEST LIST

VARIED-WIDTH IMAGE * 255

LIST UPDATE PROCESQNG V

( RETURN )

US. Patent

0a. 18, 2011

Sheet 13 0f 35

US RE42,848 E

FIG.13 ( LIST UPDATE PROCESSING) II

EXTRACT one mom

[525A

UPDATE REQUEST usn

THERE AREA MATCHING COORDINATES OF REGISTERED ENTRY WITH LESS THAN

ERROR

GENERATE LIST ENTRY

YES

NEWLY, PERFORM COORDINATE

REGISTRATION PROCESSING, AND INITIALIZE BIT BUFFER

$250 REQUEST

"SECOND PATTERN SEQUENCE"

PATTERN "FIRST PATTERN

SEQUENCE"

ADD TO CORRESPONDING ENTRY

YES RETURN

Jsst

ADD TO CORRESPONDING ENTRY

=

END ALL REQUEST LIST ENTRIES '2

¢

I

S256 "0

US. Patent

0a. 18, 2011

Sheet 14 0f 35

US RE42,848 E

FIG.14 ( DISPLAY PROCESSING ) THERE ONE THAT HAS NO UPDATE

°F Baasszssma?am THIS TIME

0

'-’

DELETE CORRESPONDING ENTRY FROM DATA LIST MEMORY

No

[$248

S24C BIT DATA CORRESPONDING To ONE BYTE BUFFERED

m arr DDEFER No

[$249 ADD BIT DATA To CORRESPONDING BYTE DATA FIFO AND CLEAR an BUFFER

II

I

DISPLAY IMAGE OF DISPLAY BUFFER

/824E

1 SELECT CENTERMOST SPOT

AND DISPLAY AN INFORMATION

/324F

CHARACTER STRING, WHICH

IS OVERLAPPED WITH DISPLAY IMAGE AND WHICH IS TRANSMITTED BY ARROW

I ARROW IS OVERLAPPED AND DISPLAYED WITH

RESPECT TO OTHER SPOTS EXCEPT THE ABOVE SPOT I

I RETURN I

[$24G

US. Patent

Oct. 18, 2011

Sheet 15 0f 35

E

2.6-“

[ID I

UDJDU DUDIJDU

N":

US RE42,848 E

U.S. Patent

0a. 18, 2011

Sheet 16 0135

US RE42,848 E

FIG.16

(0, 0)

75A

(320, 240)

EXAMPLE OF UPDATE REQUEST LIST

(THREE REQUESTS ARE GENERATED)

-

“856R351’11TT'EQL

52%TJEI'1‘6‘E

(300, 98)

SA

(159, 121)

SA

(20, 24)

$5

.___--

US. Patent

Oct. 18, 2011

US RE42,848 E

Sheet 17 0f 35

** o P .8de a

h. x .QaF a $2.3: I

P x .mF ao518: o =I

.<20m5:9Bcqun<.mx2a

Ma$m2n05c.k
oEQdQQI

$520

US. Patent

0a. 18, 2011

Sheet 18 0f 35

US RE42,848 E

FIG.18A

FIG.1BB

550

FIG.1 8C X Tower Tel :

50°

9 0120- .... .. )

l

r’

510

540

530

\29

US. Patent

Oct. 18, 2011

Sheet 19 0135

US RE42,848 E

42

[45

’7.”

“T

M5?

4,

SECTION

WK

P—1 46\ CAPTURE IMAGE

1

E l

_

BUFFER

47

49

Q

PA‘ITERN DATA

MEMORY

N

REFERENOE

~

' IMAGE BUFFER

5° 5’1

DISPLAY

BUFFER

'

/52 l

v

I

FRAME TIME sEOUENcE

44

BUFFER LIQUID cRYsTAI. E

DISPLAY

48

\

‘

=

CORRELATION IIEXéEUB‘Yéggn

1

_

WORKING BUFFER

~53

TIMING GENERATOR

\\ PCK ,J

FOR BINARIZATION

~54

[55

UPDATE REQUEsT ~

‘

'

100\ i

READING POSITION

_

i

=

cONTROI. MEMORY

'

5

7

usT

551

LIST ENTRY

552

BIT BUFFER

553

SHUTTER KEY

~43