Understanding the Capacity of Information Retrieval from Long Term Memory Misha Tsodyks Sandro Romani (WIS,CU,Janelia), Itai Pinkoviezky (WIS) Alon Rubin (WIS), Misha Katkov (WIS) Bennet Murdock (Toronto) and Mike Kahana (Upenn)

Memory retrieval

Memory retrieval – with cues

Memory retrieval – without cues

Free recall VS Recognition

Recognition

Free recall

Fig: Standing (1973), Q J Exp Psy. Free Recall: Binet & Henri (1894), Murdock (1960) J Exp Psy

Retrieval from long-term memory – power law

C V

1 2

Research Questions

• What prevents information stored in longterm memory to be efficiently retrieved? • Is there a parsimonious explanation for the power-law scaling of recall capacity?

Neural network models of long-term memory (Hopfield, 1982) Memories are represented as attractors (stable states) of network dynamics.  Attractor = internal representation (memory) of a stimulus  Each attractor: a subset of neurons that has elevated persistent activity.  Synaptic changes => Changes in attractor landscape = changes in memory  Convergence to an attractor = recall of item from memory

Hopfield model with sparse random coding i  1,..., N

Neurons ( N ):

J ij

Connections ( N 2): Memory patterns ( L):

Storage:





i  0,1 Pr ob( i  1)  f   1,..., L J ij (  )  (i   f )(i   f ) (Tsodyks, Feigelman 1988)

Hopfield model with sparse random coding: Storage capacity

Pmax 

1 1 2 f log( ) f

N

N: number of neurons in the network f: average fraction of neurons in the network encoding a memory

Tsodyks & Feigelman 1988

Mathematical model Similarities (intersections)

N

S ,  i i i 1





    S 2,1    S  L ,1

S1, 2  S L,2

 S1, L    S 2, L        

Mathematical model Similarities (intersections)

N

S ,  i i 



i 1

One parameter (f)

    S 2,1    S  L ,1

S1, 2  S L,2

 S1, L    S 2, L        

Associative retrieval: graph representation 1

4

7

?

2

3

5

6

1 2 26 64

8

9

 Romani et al 2013

Retrieval capacity: analytical solution

1

2

3

k

L

Retrieval capacity: analytical solution

1

2

3



 Nwr  L var( Nwr )

k

L

1 1 f  2 1 f

2

L

Romani et al 2013

Retrieval capacity: analytical solution

2

1

3

Паста+шоколадка



 Nwr  L var( Nwr ) ‘Naïve model’:

k

L

1 1 f  2 1 f

2

L

var( Nwr )



L

Romani et al 2013

1. Random asymmetric matrix of similarities: exact solution of the model p0 

1

1 L 1

2

p(k )  ? 3

k

1 2 k  2 k 1 p( k )  (1  )(1  )...(1  ) L 1 L 1 L 1 L 1

L

1. Random asymmetric matrix of similarities: exact solution of the model p0 

1

1 L 1

2

p(k )  ? 3

k

L

Power law scaling p0 

1

1 L 1

2

p(k )  ? 3

1 2 k 1 k p( k )  (1  )(1  )...(1  ) L 1 L 1 L 1 L 1

k

k  L

 k2  k  1 k  k  exp    i   exp    L L L 2 L i 1     2

k x x  p ( x )  x exp(  ) 2 L

Normalized probability distribution

k 

 2

L

1/2



Var (k )  (2  ) L 2

Bennet Murdock (Toronto)

Retrieval capacity: longer lists

Courtesy of B. Murdock

Retrieval capacity: longer lists (data courtesy B. Murdock)

Nwr

0.41

L

Var ( Nwr )

1.08

L

Research Questions • What prevents information stored in long-term memory to be efficiently retrieved? Answer: randomness of long-term memory representations that results in repeated recall of same items. • Is there a parsimonious explanation for the power-law scaling of recall capacity? Answer: power-law scaling emerges from random distribution of transitions between different items.

Free recall data set (Mike Kahana, Upenn) 170 subjects 112 trials/6 sessions per subject L=16 words per list

‘Easy’ vs ‘difficult’ words

L  16

‘Easy’ vs ‘difficult’ words

words of fixed size

Katkov et al, 2014

More subtle recall statistics

More subtle recall statistics

Katkov et al, 2014

Model predictions

• Easy vs difficult words • Nontrivial interactions between recall of easy vs difficult words (‘shielding’)

Distribution of recall probabilities over a pool of 1638 words (141 subjects, 112 trials/subject, L=16)

Easy vs difficult words

Recall statistics: data vs model

Katkov et al, 2014

Summary • Randomness of long-term memory representations results in repeated recall of same items and hence limits the recall capacity. • Power-law scaling of retrieval capacity emerges from random distribution of transitions between different items. • Recall capacity can be improved by applying recall strategies based on temporal presentation order.

Understanding the Capacity of Information Retrieval from Long Term ...

What prevents information stored in long- term memory to be efficiently retrieved? • Is there a parsimonious explanation for the power-law scaling of recall ...

2MB Sizes 0 Downloads 205 Views

Recommend Documents

Long-term behavior understanding based on the expert ...
digging as we do not employ an advanced detector (recall that we measure it ... interesting events in the form of a Twitter message that contains textual and ... The textual information allows a user to quickly search in historic data and the visual.

Long-term Optimization of Update Frequencies for Decaying Information
taining a high-quality database of specialist doctors and selling the contact information to suppliers of medical technology and to phar- maceutical companies. As medical equipment can be expensive, suppliers are willing to pay considerable amounts f

pdf-148\the-organizational-contract-from-exchange-to-long-term ...
... apps below to open or edit this item. pdf-148\the-organizational-contract-from-exchange-to-l ... an-contract-law-markets-and-the-law-by-stefan-grun.pdf.

pdf-1425\advances-in-information-retrieval-recent-research-from-the ...
... the apps below to open or edit this item. pdf-1425\advances-in-information-retrieval-recent-resear ... formation-retrieval-the-information-retrieval-series.pdf.

Method of wireless retrieval of information
Dec 24, 1996 - mitted to the database service provider which then provides a response back to .... a CNI, an e-mail address, a fax number, etc. and any of this.

Method of wireless retrieval of information
Dec 24, 1996 - provider can establish a database which can be accessed by subscribers to ... message center then forwards the messages to the designated mobile station via ... tion to place a call via the wireless network and a PSTN to an.

Long-Term Load Forecasting of Jordanian - ijeecs.org
May 1, 2013 - natural resources, his imports from energy represent around ... [3] . (ii) Nonlinear models such as Artificial. Neural Networks (ANN) [4], Support Vector ..... Load Forecasting, Renewable Energy, Nuclear Energy, Economic and.

Long-Term Load Forecasting of Jordanian - ijeecs.org
May 1, 2013 - 95% of his consumption [2] , around every ten years has a waves of refugees come in, and with the big fluctuation in energy prices, long-term load forecasting represent really a challenging ... 1; one input layer (source nodes with ....

Parallel Pursuit of Near-Term and Long-Term Mitigation.pdf ...
Page 1 of 2. 526 23 OCTOBER 2009 VOL 326 SCIENCE www.sciencemag.org. POLICYFORUM. It is well accepted that. reduction of carbon diox- ide (CO2. ) emissions is. the lynchpin of any long-term. climate stabilization strat- egy, because of the long life-

Exploiting the Short-Term and Long-Term Channel Properties in ...
Sep 18, 2002 - approach is the best single-user linear detector1 in terms of bit-error-ratio (BER). ..... structure of the mobile radio channel, short-term process-.

Lessons from Modelling Long-Term Depression in Sheffield.pdf ...
which just under half a billion pounds are spent by the NHS on the treatment of depression. Although depression has been historically viewed as an acute, ...

DOES IMMIGRATION AFFECT THE LONG-TERM EDUCATIONAL ...
Feb 20, 2007 - the Bank of Israel, Ben-Gurion University, LSE, the NBER Education .... on natives' educational outcomes, although the effect is sometimes ...

3G Long Term Evolution - 3g4g.co.uk
Mar 27, 2007 - FDD preferred if paired spectrum available ... layer transmission, and to enable frequency-domain channel ... time-domain already for HSPA.

3G Long Term Evolution - 3g4g.co.uk
Mar 27, 2007 - EPC: Evolved Packet Core. MME: Mobility Management ..... and unicast on the same carrier as well as dedicated multicast/broadcast carrier ...

pdf-1938\sustainable-investing-the-art-of-long-term-performance ...
... Accountability Journal. Page 3 of 9. pdf-1938\sustainable-investing-the-art-of-long-term-pe ... ental-markets-insights-series-from-brand-routledge.pdf.

The Dynamics of Long-Term Growth - Jacques Vallee
and a number of other publications [3] call attention to the consequences of unlimited growth ... effects and the consequences of demographic and technological growth. .... animal energy in' antiquity to modern interplanetary rockets, ..... Investiga

The Long-Term Impact of University Budget Cuts: A ... - CiteSeerX
Mar 12, 2006 - 1997, the California state budget appropriated more money for corrections ... not sufficient simply to have a college or university in a region; an institution starved of resources ... [10, p. 68]. Education is recognized as a crucial

What is the Long-Term Impact of Changes? - CiteSeerX
ever, if the user enters an end-of-file (Control-D) char- acter ... ical change, i.e. the more files are influenced by this change, the lower .... QS(A) = Σn i=1 QS(Ai) n.

Stunted Firms: The Long-Term Impacts of Colonial ...
Aug 3, 2017 - This paper uses the geocoded 2008 census of Peruvian firms to study how ... forced labor system (1573–1812) caused long-run impacts on firm ..... agriculture and financial services regardless of their tax-filing status, focusing.

The dynamics of long forward rate term structures
*Correspondence author, School of Economics and Finance, The University of Hong Kong, Pokfulam Road,. Hong Kong, P. R. China. ..... imum, standard deviation, skewness, kurtosis, and the sample autocorrelations at displacements of 1, 21, and 126 days.

The Long-Term Impact of University Budget Cuts: A ... - CiteSeerX
Mar 12, 2006 - as path analysis to account for interaction effects [35] and identical twin data. [36,37]. .... in a small town or open economy. Burdens are ...... 612, The World Bank, Washington, D.C., 1990. [15] L. J. ... Australian Econ. Rev., vol.

What is the Long-Term Impact of Changes?
personal or classroom use is granted without fee provided that copies are not made or ... Linux OS that turned the BKL non-preemptible. Re- moving the BKL ... ever, if the user enters an end-of-file (Control-D) char- acter ... maintain a system.