교 정: 파 일:

Indoor navigation system via sound with microphone and loudspeakers ETRI Journal Editorial Office

This paper proposes a novel technique for estimating the position of a microphone using loudspeakers and a hidden audio-embedded signal. Unlike traditional positioning systems using an electromagnetic wave, the proposed positioning system employs an acoustic signal for position estimation. Thus, the proposed positioning system only requires a conventional loudspeaker pre-placed in the living space and a microphone in the smart device to estimate the user’s position. To achieve this goal, we embed a time synchronization signal that is inaudible to human ear into the audio signal. After receiving this synchronization signal from the microphone, our proposed system analyzes the hidden synchronization signal embedded in the audio signal. The synchronization information provides the exact location of the microphone. The proposed positioning system can be applied to an indoor positioning system for shopping malls and museums. Keywords: Positioning System, Indoor Navigation, Audio Embedded Signal

I. Introduction 1. Indoor positioning system with sound Because the Global Positioning System (GPS) is generally unavailable indoors, various attempts have been made to realize indoor navigation using a Wireless Local Area Network

Manuscript received submission_date; revised last_revision_date; accepted accepted_date. This work was supported by the Funding_Program_name of Funding_Org, Country (grant_ number, project_name). First_author (phone: +82-42-860-0825, email: [email protected]), Second_author (email: [email protected])

ETRI Journal, Volume x, Number y, Date

Audio Signal

Estimated Microphone Position

Time Synchronization Signal Embedding

TDOA Position Calculation

Playback Device

Acoustic Channel Mic.

Receiving Device

Generalized Cross Correlation

Estimated Delay

Fig. 1. Data flow of proposed positioning system. (WLAN) [1]. Although these techniques work with a reasonable level of performance, they require a signal beacon transmitting an electro-magnetic (EM) wave signal to a moving receiver such as a smartphone. In addition, most techniques based on a wireless EM signal require processing hardware with a unified protocol and antenna. On the other hand, the proposed positioning system only needs a microphone in the mobile device for the speech signal. In addition, conventional loudspeakers already exist in locations such as shopping malls and museums, and can work as a signal beacon for transmitting a synchronization signal. Therefore, if there are a sufficient number of loudspeakers in the room, the positioning system can be built without installing a wireless transmitter for sending the synchronization signal.

2. Technical issues and previous research Our proposed system is based on a service scenario assumes that the loudspeaker is always transmitting a constant acoustic signal, such as background music. Such constant acoustic signals contain a time synchronization signal that achieves the time delay measurement. When transmitting a signal through a loudspeaker with a microphone in a mobile unit, the human listener should not be aware of the embedded synchronization signal because such signals are unpleasant to the human ear. Therefore, the time synchronization signal should be embedded while preserving its inaudibility. At the same time, First Author et al.

1

the detection probability of the correct positioning should be guaranteed. To achieve this, an adequate optimizing technique should be applied to design a time-synchronization signal. Thus, we employed a genetic algorithm to obtain an optimized signal. In addition, a robust detection technique is applied to the system to guarantee high detection probability in a noisy environment. The overall data flow of our proposed system is shown in figure 1. Most positioning systems consist of beacons with a fixed location and a moving device for receiving signals. To estimate the position of a device, several different techniques can be applied to measure the position of a moving signal transmitter [2]. In this paper, we employ a Time difference Of Arrival (TDOA) for measuring the time difference of the arrival time. TDOA only measures the difference between estimated time delays. Therefore, if there are more than three loudspeakers, the position can be measured without any synchronization with the main system, unlike in our previous work [3], which assumed a system with two loudspeakers with reference timing. A similar work on locating a microphone-equipped camcorder used to illegally record a movie in a theater was conducted by Nakashima et al [4]. Unlike Nakashima’s work, we employed a different type of hidden synchronization signal that locates the microphone more quickly. Whereas Nakashima’s work is based on the use of an SS (Spread Spectrum) technique, our proposed system employs an OFDM (Orthogonal Frequency Division Multiplexing) time synchronization signal that has been employed in a communication system [5]. Moreover, unlike Nakashima’s work, we are targeting indoor navigation systems.

II. Time Synchronization Signal Design 1. Time synchronization signal for the proposed system As previously mentioned, our proposed system employs a TDOA technique to estimate the position of a microphone. To measure the exact time delay of a signal, an effective time synchronization signal should be designed. Because the possibility of interference in an acoustic channel always exists, the time synchronization signal should be robust to noise interference from real-world conditions. Moreover, the positioning system should work properly at any location. Therefore, the magnitude and time synchronization in each channel should be balance for each channel and should not interfere with other channels. Furthermore, because microphones have a different frequency response, a time synchronization signal should not be concentrated within a certain frequency band. Therefore, we employed a technique that is able to equally distribute the time

2

First Author et al.

synchronization within the frequency domain while preserving the synchronization performance. Through this approach, microphones with a limited frequency response can also analyze the signal by receiving partial data below the limited cutoff frequency. In this regard, to design the desired time synchronization signal properly, we considered the following properties and techniques for our time synchronization signal design. 2. Peak-to-Side Peak Ratio (PSPR) The PSPR describes the ratio between the main and side peaks of the autocorrelation function. If we define a time synchronization signal as s (n) , the PSPR of the time synchronization signal can be described as below. é N -1 ù r (t ) = Re ê å s( n ) s* ( n + t ) ú ë n =0 û r (0) PSPR = max r (t )

(1)

t ¹0

wheret is the time delay. The time synchronization signal with a high PSPR has less opportunity to estimate the time delay incorrectly because it has a huge gap in magnitude between the biggest autocorrelation value and the second-biggest autocorrelation value. 3. Signal optimization Because our positioning system assumes multiple channel inputs and a single microphone, the three time synchronization signals in each channel should be designed with equal magnitude and PSPR. Therefore, we employed a time synchronization signal optimization technique that was previously employed for an OFDM communication system [5]. This OFDM time synchronization technique replaces the particular carrier frequency with the time synchronization signal. When selecting this carrier frequency, the combination of carrier frequencies determines the PSPR of the time synchronization signal. Unfortunately, this combination of carrier frequencies cannot be determined through an analytic solution. Thus, we solve this problem by employing a genetic algorithm to determine the carrier frequency set for each channel. The process of the genetic algorithm is described below and illustrated in figure 2. (1) The chromosome from the former generation has same number of carrier frequencies. (2) Each chromosome splits its sequence and mates with another chromosome in the gene pool. (3) After mating, the numbers of carrier frequencies in the

ETRI Journal, Volume x, Number y, Date

U ch1 (n) =

å sin(w n + Q(k ) k

ch1

)

kÎCh1

U ch 2 (n) =

å sin(w n + Q(k )

(2)

k

ch 2

)

å sin(w n + Q(k )

ch 3

)

kÎCh 2

U ch 3 (n) =

k

kÎCh 3

where Ch1, Ch2, and Ch3 are a gene pool set for each channel, and Qch1 , Qch 2 , and Qch 3 indicate the phase for each channel and frequency. As described in figure 3, these three time synchronization signals have no overlapping carrier frequency. Henceforth, we call this property the “orthogonality” of the time synchronization signal, which can be described as below: N -1 CN

(3)

åÕU [k ] = 0 r

k = 0 r =1

where U r [ k ] is the DFT of U Ch1 (n) , U Ch 2 (n) , and U Ch 3 (n) . This orthogonality assures the reliability of the positioning system when a microphone approaches a particular loudspeaker. If the synchronization signal has no orthogonality in the frequency domain, the signal from the nearest loudspeaker will dominate the microphone input. Therefore, without orthogonality, the positioning system will not work in the area near the loudspeaker. Unlike with GPS, it is highly possible for a microphone to be placed near a loudspeaker because loudspeakers are usually placed near the human listener, in contrast to a GPS satellite. Fig. 2. Example of three-channel genetic algorithm process.

chromosome are equalized. (4) After equalizing the carrier number, the chromosomes are sorted based on the PSPR value. The chromosomes with a higher PSPR value survive, and the chromosomes with a lower PSPR value are discarded. Each square in figure 2 describes a single carrier frequency. If the number in the box is 1, the carrier frequency contains a signal. Otherwise, if the number in the square box is zero, the carrier frequency contains no signal. After obtaining the optimized carrier frequency set for each channel, the time synchronization signal is generated as below:

ETRI Journal, Volume x, Number y, Date

D. Embedding of time synchronization signal Because our proposed system needs to embed the time synchronization signal into an audio signal (such as music), we embed the time synchronization signal at the frequency above band-edge frequency fb , as described in figure 3. All frequency bins of each channel over band-edge frequency fb are erased except for the set of frequency bins selected in the genetic algorithm, as shown in figure 3. In addition, to ensure the inaudibility of the time synchronization signal, we preserve the signal magnitude, which can be amplified at a certain ratio to increase the detection probability, as shown in figure 4. However, a highly amplified synchronization signal degrades the listening quality. The result is verified through a listening test with a human subject.

First Author et al.

3

Fig. 4. Example spectrogram of (a) original signal (b) signal with a 0dB of gain (c) signal with a 15dB of gain (d) signal with a 30dB of gain. 1. Time delay estimation from a received signal

Fig. 3. Example of time synchronization signal embedding process and received time synchronization signal. Figure 3 also describes the embedding process. The magnitude of the original signal is preserved and the phase of the time synchronization is inserted to the original audio signal for the selected carrier frequencies. As mentioned above, all three audio signals are generated with no overlapping frequency bin. After the audio signal goes through a loudspeaker, acoustic channel, and microphone, the receiving device analyzes the received signal. The received signal picked up from the microphone has a time synchronization signal from all three channels, and is analyzed during the analysis process.

III. Signal Analysis

4

First Author et al.

Using a microphone, the receiving device obtains multiple time synchronization signals from the acoustic channel. Since both the playback and receiving devices have the same time synchronization signal, the receiving device can estimate the exact time delay by calculating the cross-correlation with the received signal and the original time synchronization signal. However, a typical time-domain based cross correlation does not give the best result because we modify the magnitude of the time synchronization signal in the frequency domain to ensure inaudibility. Therefore, there is need to compensate this magnitude altering process in the frequency domain. To calculate the exact time delay, we therefore employ Generalized Cross Correlation (GCC) with a Phase Transform (PHAT) technique [6]. This GCC-PHAT technique can be described simply as follows:

GPHAT (k ) =

U i (k ) × U TS (k )* U i (k ) × U TS (k )*

(4)

where U i (k ) is the DFT of the received time synchronization signal, U TS ( k ) is DFT of the original time synchronization signal, and k is the frequency index. However, since our proposed system is based on a single mono microphone, the time synchronization signal should be extracted for each channel from the mono channel. To achieve this, we employed frequency selection matrix M to select a certain frequency from a mono signal. Thus, single-column matrix M contains only ETRI Journal, Volume x, Number y, Date

Cost function F(x,y) 5

1 4

y (m)

2

Fig. 5. Time delay between each speaker and a microphone. ones and zeros which were determined during the application of the genetic algorithm, as shown in figure 2. Multiplying frequency selection matrix M with the DFT of the input signal enables single channel information to be extracted from the DFT of a mono signal. Consequently, the GCC-PHAT function for the t sample-delayed i-th channel time synchronization signal can be described as below:

GPHAT , i (k ,t ) =

M i (k ) × U i (k )e - jwkt × U TS (k )* M i (k ) × U i (k ) × U TS (k )*

(5)

where M i (k ) is the frequency selection matrix of the i-th channel. After setting up the GCC-PHAT function, we calculate the argument maximum of the GCC-PHAT function to obtain the estimated delay as shown below: é

ù

ë kÎFS

û

tˆi = arg max ê å GPHAT , i (k ,tˆi ) ú tˆi

(6)

where tˆi is the estimated delay for the i-th channel time synchronization signal, and FS is the total carrier frequency set for the i-th time synchronization signal. 2. Position estimation using TDOA Once the time delay of each channel is obtained, the location of the microphone can be obtained based on the difference in the delays. The merit of using the difference in the delays is that there is no need for the reference timing. Therefore, unlike our previous work in [3], we can measure the location without having common reference timing between the playback and receiving devices. Using an analytical approach, a three-point based position calculation can be easily calculated by finding the intersection of two hyperboloids. However, in a computer system, an analytical approach is somewhat demanding because the computer system calculates the analytical solution through a ETRI Journal, Volume x, Number y, Date

(2.52,2.52) 3

3 2

4

1

5

0

6

-1

1

2

3 4 5 6 x (m) Fig. 6. Example plot of cost function F ( x, y ) .

numerical approximation. Thus, we employed a numerical approach that fits a computer system. Example arbitrary positions of the loudspeakers and microphone are shown in figure 5. The sample delay between a loudspeaker ( xi , yi ) and an arbitrary position ( x, y ) in figure 5 can be defined as below:

t%i ( x, y ) = Fs ×

( xi - x) 2 + ( yi - y ) 2 c

(7)

where c is the speed of sound, and Fs is the sampling frequency. With this delay, difference of delay can be defined as below

D ij = tˆi - tˆ j D% ij ( x, y ) = t%i ( x, y ) - t% j ( x, y )

(8)

where D ij is the difference in the delay estimated from equation (6), D% ij is the difference in the delay dependent on arbitrary position ( x, y ) , and i, j is the channel index. With these measures, to estimate the location, we find the position that minimizes the following equation.

F ( x, y ) =

(D + (D

12

- D% 12 ( x, y )

% 23 - D 23

2

) ( x, y ) )

2

(9)

Finally, the coordinate of the microphone can be obtained by the following equation, which finds arguments minimizing cost function F ( x, y ) .

First Author et al.

5

The performance of the proposed system should be tested in two separate ways. First, the positioning system should be tested to determine how exactly the proposed positioning system can measure the microphone position. Second, the time synchronization embedded in an audio signal should not be detectable by the human auditory system. Thus, we carried out the following positioning and listening tests. 1. Positioning test

Fig. 7. Loudspeaker position setup for positioning performance test. Each picture describes (a) setup 1 (b) setup 2 (c) setup 3 and (d) setup 4. The red filled circles represent the microphone positions for the test, and the hollow blue circles represent the positions of the loudspeakers. Table 1. Loudspeaker positions (m). Loudspeaker position Setup 1 Loudspeaker position Setup 2 Loudspeaker position Setup 3 Loudspeaker position Setup 4

Loudspeaker 1

Loudspeaker 2

Loudspeaker 3

(1.5, 0.4, 1.05)

(3, 0, 1.05)

(4.5, 0.4, 1.05)

(0.4, 1.6 , 1.05)

(3, 6, 1.05)

(5.6 , 1.6 , 1.05)

(1.3, 1.5, 3)

(3, 0.75, 3)

(4.7, 1.5, 3)

(1.4, 1.3, 3)

(3, 5.4, 3)

(4.6, 1.3, 3)

Table 2. Microphone positions for test.

Test positions

x interval (m)

y interval (m)

z (m)

1

1

1.2

( x, y ) = arg min [ F ( x, y ) ]

(10)

x, y

Figure 6 shows an example of F ( x, y ) in equation (9) and the estimated loudspeaker position ( x, y ) . The black cross in figure 6 represents the position of the loudspeaker, and the black empty circle represents the estimated position of the microphone.

IV. Experimental Results

6

First Author et al.

The positioning test was conducted using a studio monitor loudspeaker and microphone equipped in a smartphone that is able to pick up sound frequencies below 22 kHz. We used an audio signal with a 44.1 kHz sampling rate and window length of 2048 samples. In addition, we tested four different loudspeaker positional setups for the positioning test. The positioning configuration is shown in figure 7. The loudspeaker positions in setups 1 and 2 were employed to test the positioning performance, where each loudspeaker was placed at a location that shares the same surface as the microphone location. On the other hand, loudspeaker positional setups 3 and 4 were employed to test the positioning performance with the loudspeakers positioned on a surface near the ceiling. The loudspeaker positional setups 3 and 4 are more similar to realworld loudspeaker positions in a public location. Detailed positional information of the speaker setup for the experiment is given in table 1, and the microphone position for the experiment is provided in table 2. To test the performance of our proposed technique in a different environment, we tested the proposed technique under a few different conditions. First, we used three different synchronization signal gains, as shown in figure 4. A higher synchronization signal gain has a higher risk of audibility of hidden synchronization signals by the human auditory system, but ensures a higher detection probability. In addition, we tested three different loudspeaker loudness levels to evaluate the degradation in performance as the loudness level of the loudspeaker was lowered. To set the reference loudness level of the loudspeakers, we measured the sound pressure level at position (3, 3, 1.05), shown in figure 7, with a loudspeaker placed at position (3, 0, 1.05), and applied a certain loudspeaker loudness level. While measuring the sound pressure level at this position, we played amplitude-normalized white Gaussian noise with the loudspeaker position placed at position (3, 0, 1.05). The measured reference sound pressure level was 66.5 dB. The SPL of the quiet test room was 33.5 dB. We varied the loudness level of the loudspeaker at 10 dB intervals for the test, as described in the legend of figure 8. We measured the performance of our proposed positioning system based on the detection probability. If the Euclidean

ETRI Journal, Volume x, Number y, Date

classic1 classic2 pop1 pop2 rock1 rock2 jazz1 jazz2

100

80

60

40

20

0

HR

0dB

15dB

30dB 48MP3 32MP3 Anchor

Fig. 9. MUSHRA listening test results of a mono audio signal generated using the proposed synchronization signal embedding technique.

Fig. 8. The detection probability of the proposed positioning system with four different loudspeaker positional setups: (a) setup 1, (b) setup 2, (c) setup 3, and (d) setup 4. distance between the estimated position and the reference test position is shorter than 0.25 m, the positioning result is considered a success. The detection probability is calculated by dividing the number of successful position estimations with total trial number. Each trial used a 0.8 s sound signal recorded by a microphone equipped in a smartphone. As the results in figure 8 show, the test with a higher time synchronization signal gain achieves a higher detection probability. In addition, the loudness level of the loudspeaker affects the detection probability. Although a lower loudness level deteriorates the detection probability, a low detection probability does not mean the positioning system is not working. In practice, the positioning system will take a longer time to locate the current position exactly when the loudness level of the loudspeaker is too low. Therefore, the positioning speed can be measured based on the detection probability. The main reason for the low detection probability in setups 1 and 3 is because of the loudspeaker interval. If the space between loudspeakers is narrow, a small measurement error will cause a significant estimation error because the relative position of the loudspeaker is a reference for the proposed positioning system. Therefore, a loudspeaker setup with large gap between loudspeakers such as setups 2 and 4, obtains more accurate positioning results.

synchronization signal should have a similar listening fidelity as the original audio signal. Therefore, to test the listening quality of a time synchronization signal embedded audio signal, we employed a Multiple Stimuli with Hidden Reference and Anchor (MUSHRA) test [7]. We conducted a listening test using the following samples.   Audio signal with 0 dB time synchronization signal gain   Audio signal with 15 dB time synchronization signal gain   Audio signal with 30 dB time synchronization signal gain   Audio signal encoded with mono 48 kbps MP3   Audio signal encoded with mono 32 kbps MP3   Audio signal filtered with 3.5 kHz low-pass filter   Hidden reference audio signal Eight different mono music signals in four different genres were selected for testing the audio signals to determine the dependency on the music genre. Ten male and female listeners unrelated in the proposed work participated in the test. The listening test results in figure 9 show the dependency on the time synchronization signal gain. As the time synchronization signal gain increases, the listening test score drops slightly. However, the three time synchronization signal embedded audio signals clearly show a better listening score when compared to the two reference MP3 encoded signals. From the above results, we can conclude that without advance information for the time synchronization signal, the listener will have difficulty noticing that the manipulated signal is different from the original audio signal.

2. Listening test of synchronization signal embedded in audio As mentioned above, an audio signal containing a time ETRI Journal, Volume x, Number y, Date

V. Conclusions First Author et al.

7

In this paper, a novel indoor navigation system based on the acoustic signal propagated from loudspeakers to a microphone was proposed. The proposed system is targeted for locations with loudspeakers playing background music. Thus, by hiding a time synchronization signal in the audio signal and playing the signal through a loudspeaker, a microphone in a mobile device can pick up the signal using a hidden time synchronization signal that a human listener cannot recognize. After analyzing the received time-synchronization signal with the given protocol and information, the position of the microphone can be successfully estimated. In addition, the listening quality of the manipulated audio signal is fairly similar to the original audio signal. The proposed positioning system can be applied to places where an indoor navigation system is needed such as a shopping mall or museum. Moreover, the proposed technique can be applied to a home theater audio system to locate the listener’s position to set up the appropriate sweet spot.

References [1] M. Vossiek, L. Wiebking, P. Gulden, J. Wieghardt, C. Hoffmann, P. Heide, “Wireless local positioning,” IEEE Microwave Magazine, vol. 4, no. 4, Dec. 2003, pp. 77-86. [2] L. Hui, H. Darabi, P. Banerjee, L. Jing, “Survey of Wireless Indoor Positioning Techniques and Systems,” Systems, Man, and Cybernetics, Part C: Applications and Reviews IEEE Transactions on, vol. 37, no. 6, Nov. 2007, pp. 1067-1080, [3] T.J. Park and K.O. Kang, “Position estimation using a microphone and stereo loudspeaker with an audio-embedded hidden time synchronization signal,” IEEE Int. Conf. Acoustics, Speech and Signal Processing on, Vancouver, Canada, May 2631, 2013, pp. 473-477 [4] Y. Nakashima, R. Tachibana, and N. Babaguchi, “Watermarked Movie Soundtrack Finds the Position of the Camcorder in a Theater,” Multimedia, IEEE Transactions on, vol. 11, no. 3, Apr 2009, pp. 443-454. [5] O. Ureten and S. Tascıoglu, “Autocorrelation Properties of OFDM Timing Synchronization Waveforms Employing Pilot Subcarriers,” EURASIP Journal on Wireless Communications and Networking, no. 10, Jan 2009, pp. 13-18. [6] C.H. Knapp and G.C. Carter, “The generalized correlation method for estimation of time delay,” Acoustics, Speech and Signal Processing, IEEE Transactions on, vol. 24, no. 4, Aug 1976, pp. 320-327. [7] ITU-R Rec. BS.1534-1, Method for the subjective assessment of intermediate quality level of coding systems, Jan. 2003.

8

First Author et al.

ETRI Journal, Volume x, Number y, Date