2010年3月31日,星期三

On 祖母细胞s and 平行 分散式 models

杰夫·鲍尔斯(Jeff Bowers)发表了一两篇论文,论证了 祖母细胞s -- cells that represent whole "objects" such as a specific face (or your grandmother's face). At issue, of course, is whether the brain represents information in a localist or 分散式 fashion and 杰夫 has used his case for 祖母细胞s as evidence against a basic assumption of 平行 分散式 处理中 (PDP) models. But the 等离子显示器 folks don't seem to think "分散式" is a necessary property of 等离子显示器 models. So in the guest post below, 杰夫 asks, What does the D in 等离子显示器 actually mean? This is an interesting question, and 杰夫 would like to know your thoughts (see the new survey to respond). I'd also be interested in your thoughts on 祖母细胞s!

杰夫 Bowers的来宾帖子:
I’最近,我参与了有关本地化表示形式和在并行分布式处理(PDP)网络中学习的分布式表示形式的相对优点的辩论。本地人,我的意思是交互式激活(IA)模型中的单词单元–代表特定单词的单位(例如“grandmother cell”)。分布式的意思是,将熟悉的单词(或物体,面部等)编码为跨一组单元的激活模式,没有单个单元足以表示一个项目(您需要考虑完整的模式) )。在Bowers(2009,2010)中,我认为与PDP网络中的分布式表示相比,神经科学与本地编码的一致性更高,这与认知科学界的普遍假设相反。也就是说,皮质和海马中神经元的单细胞记录通常揭示出在其响应中具有显着选择性的神经元(例如,对许多人中的一个人有响应的神经元)。与分布式PDP理论相比,我认为这与本地主义者更加一致。

但是,该文章与本地化或PDP模型在生物学上是否更合理无关。而是我’我对人们认为PDP模型背后的理论感到好奇;具体来说,您对分布式表示和PDP模型之间的关系有何了解?在Bowers(2009,2010)中,我声称PDP模型致力于声称信息是以分布式格式而不是本地格式编码的。根据这种观点,包括单个单元以对特定单词进行编码的单词识别的IA模型(例如DOG单元)不是PDP模型。像格罗斯贝格(Grossberg)的ART模型一样,学习本地主义者表示的神经网络也不是。根据我的理解,Seidenberg和McClelland单词命名模型的一个关键(必要)功能使其成为PDP家族的一部分,即它学习了单词的分布式表示形式–它摆脱了本地化的单词表示形式。
However, Plaut and McClelland (2010) challenge this characterization of 等离子显示器 models. That is, they write:

In accounting for human behavior, one aspect of 等离子显示器 models
特别重要的是他们对互动的依赖以及
分级约束满足度以得出输入的解释
或选择一个与所有
系统’的知识(以介于
units). In this regard, models with local and 分散式 representations
可能非常相似,并且保留了许多本地化模型
很有用且有影响力(例如,Dell,1986; McClelland&
埃尔曼(Elman),1986年;麦克莱兰德& Rumlehart, 1981; McRae, Spivey-
Knowlton和Tenenhaus,1998年)。实际上,鉴于其明确和
extensive reliance on 平行 分散式 处理中, we think it
makes perfect sense to speak of localist 等离子显示器 models alongside
分散式 ones. (p 289).

That is, they argue that the 等离子显示器 approach is not in fact committed to 分散式 representations. Elsewhere they write:

实际上,该方法对要
应该积极代表给定实体或学位
给定单元响应的实体的相似性。
相反,该方法的主要原则之一是发现
而不是规定陈述(第286页)

So on this view, the 等离子显示器 approach does not rule out the possibility that a neural network might actually learn localist 祖母细胞s in the appropriate training conditions.

以此为背景,我会对人们感兴趣’对此的看法。这是我的问题:

Are 等离子显示器 theories of cognition committed to the claim that knowledge is coded in a 分散式 rather than a localist format? [see new survey]

谢谢你的想法

杰夫

参考文献

Bowers JS (2009). On the biological plausibility of 祖母细胞s: implications for neural network theories in psychology and neuroscience. 心理审查,116 (1),220-51 PMID: 19159155

Bowers JS (2010). More on 祖母细胞s and the biological implausibility of 等离子显示器 models of cognition: a reply to Plaut and McClelland (2010) and Quian Quiroga and Kreiman (2010). 心理审查,117 (1)PMID: 20063980

普拉特(D.)& McClelland, J. (2010). Locating object knowledge in the brain: Comment on Bowers’s (2009) 在 tempt to revive the 祖母细胞 hypothesis. 心理评论,117 (1),284-288 DOI: 10.1037 / a0017101

7条评论:

格里格·德·祖比卡雷说过...

杰夫,你好

感谢您的有趣帖子。我在2006年的B版文章中谈到了与神经影像有关的一些问题&C,同时引用自己的作品和Mike Page's BBS position paper. So, I think 等离子显示器 theories are not committed to that claim. Similarly, most localist models incorporate some form of 分散式 处理中, so 我不'也不认为本地主义者的方法很严格。

最良好的祝愿,

格雷格

Page, M. (2000). Connectionist modelling in psychology: A localist manifesto. Behavioral and Brain Sciences, 23, 443–467. http://journals.cambridge.org/action/displayAbstract?aid=65457

de Zubicaray, G. (2006). Cognitive neuroimaging: cognitive science out of the armchair. Brain and Cognition, 60, 272-281. http://dx.doi.org/10.1016/j.bandc.2005.11.008

马克·乔尼斯(Marc Joanisse)说过...

I'll go out on a limb here and say: I do think actual neural coding happens in a 分散式 sense, to the extent that 在 the very least a) groupings of cells will be used to code individual perceptuomotor or cognitive categories; and b) that 在 least a subset of these cells are likely re-used for coding other categories; and c) the degree of similarity among categories can be captured as the degree of similarity in the activity of neurons used to code them.

I'我确信杰夫在所有这些方面都有很多话要说,但就我所知'm saying is right, this is what I believe to be true. The issue is the extent to which a 等离子显示器 model must use 分散式 coding to capture the phenomena 在 hand. In that sense 我不't think 分散式 coding is the sine qua non of the 等离子显示器 enterprise. Rather what is 分散式 is the connections among these different neurons, and indeed this is where much of the work (i.e, 处理中) is being done.

与此相关的更广泛的评论:我'我一直觉得'平等地评估所有连接主义者/ 等离子显示器企业是很危险的。研究人员在承诺/遵守并行分布式处理的任何基本原则方面差异很大。即使使用原始PDP体积(McClelland& Rumelhart & the 等离子显示器 research group, 1986, MIT Press) there is considerable variability in how 分散式 coding is used or not. For instance the TRACE-II model (now known as TRACE) codes word identities using unique units that could be thought of as lexical entries or 祖母细胞s; in contrast the McClelland &Rumelhart过去时模型将单词作为Wickelfeatures的组合进行编码,这是一个可用于语音系统的分布式位置敏感编码系统(在上述Seidenberg中使用了相同的系统&McClelland 1989阅读模型,尽管在原始文章中引用的Plaut等人的研究中对此进行了更新。

More tangentially, there can be some confusion about whether 等离子显示器 is a framework or a theory. I see it as 实施理论的框架和consequently individual models 需要 to be assessed on their own merits (including what representational coding scheme they use).

回到最初的问题,局部性与分布式特征的适当性取决于它是否代表理论上的主张,还是为了简化计算和解释性而做出的简化假设。在上面的TRACE模型示例中,他们对声音信息使用分布式编码,而对词汇单元使用局部编码。动机可能是,关于所模拟现象的任何假设都不取决于模型是否能够在词汇/语义域中编码相似度,而至关重要的是,它必须对音素之间的声学​​相似性进行编码。这样一来,它就从语义相关性中抽象出来,这一事实并不会使它对它要解决的亚词法现象的主张无效。

我们要不要 需要 至少在某些情况下进行分布式编码?我认为我们确实是在某种意义上说,对认知现象的许多解释都基于相似度的概念(有关语义记忆的探讨,请参见Cree。& McRae, 2003). That said, is it necessary in all cases? 我不'不这么认为。考虑到我自己的工作,我们以前曾使用连接主义模型来检查失语症的语义与语音缺陷的影响,以解释这些患者的分离'过去式的问题(Joanisse&Seidenberg,1998年)。我们为语音学使用了分布式代码,但对语义使用了本地编码。广泛的说法是,语义相关性可能不像语音相似性那样重要。像这样的简化假设是所有模型的一部分。否则他们不会't be models.

未知说过...

Hi Marc, you may be right that the brain relies on 分散式 coding –我文章的主要目的是要指出,不应如此迅速地取消本地编码方案。在对话中,我’ve been struck the extent to which advocates of the 等离子显示器 approach often rely on the biological plausibility of 分散式 compared to localist coding in support of their view. That underlying assumptions makes it difficult to even take localist models seriously I think. What I show in the papers (I think) is this widespread assumption is unsafe.

你写:
Even with the original 等离子显示器 volumes (McClelland & Rumelhart & the 等离子显示器 research group, 1986, MIT Press) there is considerable variability in how 分散式 coding is used or not. For instance the TRACE-II model (now known as TRACE) codes word identities using unique units that could be thought of as lexical entries or 祖母细胞s;

的确,几乎所有PDP模型都包含本地化表示形式–通常,它们包括本地输入输出单元并学习分布式内部表示(这被认为是使它们成为PDP模型的关键)。有时,PDP方法的倡导者开发了包含本地词表示形式的模型。但是在两种情况下,这都被视为实现上的便利。仍然有这样的主张,即应该分配输入层和隐藏层的知识-在简化模型中使用本地代码会更容易。但是,本地化建模者将本地化代码作为其模型的核心特征–不是在更现实的模型中可以摆脱的东西。

未知说过...

嗨,格雷格,感谢您对本文的引用。在其中写:

“At its outset, connectionism made the central claim that knowledge is coded in a 分散式 manner (Rumelhart et al., 1986 Rumelhart, D. E., McClelland, J. L., & the 等离子显示器 Research Group (1986). Parallel 分散式 处理中: Explorations in the microstructure of cognition (Vol. 1 and 2). Cambridge, MA: MIT Press.Rumelhart, McClelland, &(PDP研究小组,1986年)。但是,在连接主义模型中确实出现了本地主义表示,并且连接主义建模的最新方法已明确地纳入了本地主义表示(参见Bowers,2002和Page,2000)。”

I agree with you that 等离子显示器 modelers originally claimed that knowledge is coded in a 分散式 manner. In fact, this was the key claim that distinguished this type of neural network model view from previous models that included (and sometimes learned) localist representations. So while I agree with you that neural network models can learn localist representations, this was not supposed to be the case with 等离子显示器 models. Here is a nice quote that captures this I think:

“人们偶尔会以某种未知的方式提出改变我们思维方式的想法。我相信,连接主义体现了一些真正的原创思想。特别是,存在一种新颖的表示知识的方式– in terms of pattern of activation over units encoding 分散式 representations... The people who developed this framework—鲁梅尔哈特,麦克莱兰德,欣顿和其他人—设法提出了一些见解,从而扩展了思考思维方式的范围。这导致了关于思想和大脑的新发现(Seidenberg,1993,第234页)”

然后是术语“并行分布式处理”–D代表什么?一世’有人告诉我"distributed"在PDP中是"processing” (so that the D does not imply a commitment to 分散式 representations). But if that is the case, what is the difference between “distributed” and “parallel” in 等离子显示器? Why not call the 分散式 分散式 处理中? If 等离子显示器 models are now said to be consistent with learning localist or 分散式 representations, what is the core claim of the approach? Why introduce the term 等离子显示器 在 all? Why not just stick with the term neural network?

So my question to you everyone who votes NO to the poll question: Are you using the term 等离子显示器 and neural network synonymously? In that case, are the symbolic neural network models by Hummel, ART models by Grossberg, “实施神经网络” advocated by Pinker, etc. all 等离子显示器 models? (despite the fact that the these authors all take their models to be inconsistent with the 等离子显示器 approach).

我不’认为这只是一个术语问题。 等离子显示器建模者与Coltheart,Pinker,Besner,Hummel,Grainger,Davis等研究人员之间存在着长期的辩论,重要的是每个人都必须清楚辩论的内容。无关紧要的是大脑是神经网络–每个人都同意这一点。每个人还都同意神经网络可以学习(并且可以学习本地表示和分布式表示)。 Coltheart,Pinker,Besner,Hummel,Grainger和Davis等都不同意PDP方法,因为他们认为这种方法拒绝了本地化(和象征性)表示。他们对此有错吗–是否也将他们描述为PDP建模者(仅仅是相信本地主义者而不是分布式表示形式的PDP建模者)?如果每个人都同意这些条款,那么进行辩论会更有效率。

马克·乔尼斯(Marc Joanisse) said...

杰夫,您好,感谢您的回应和看法。

For me it all boils down to the idea that 等离子显示器 is a framework for implementing and testing theories, and not a theory in itself. The up-side is that people who purport to disagree connectionism might find something to like in symbolic-based approaches - in a similar way that I'在当前的生成语法理论中,我们发现许多事物都喜欢随机的而不是严格的确定性规则。

格里格·德·祖比卡雷说过...

杰夫,你好

I see what you are getting 在 . However, I agree with Marc. I think that 等离子显示器 is a framework for modelling, and we 需要 to examine the explicit assumptions instantiated in the models, otherwise we could just as easily ask why many localist/symbolic models incorporate forms of 分散式 处理中.

帕万说过...

嗨,您好,

感谢您的启发性讨论。我尚未查找此处引用的各种论文,并且对本领域不熟悉,可能对专业术语不熟悉,但是我已经对该问题进行了一些思考。没有特定顺序的想法:

在我看来,"parallel" in 等离子显示器 refers to distribution in time, whereas "distributed"指空间分配。

我同意马克的观点"PDP" (or 分散式 spatiotemporal representation) is a framework and not a theory. If it were a complete theory, then we would just be explaining away 祖母细胞s with grandmother networks, i.e. we would instantiate a new network which takes on a 分散式 spatiotemporal representation for each and every new sensory/cognitive phenomenon or memory representation. This would not be a theory with predictive power, but just an arbitrary look-up table of networks and functions. I feel that neuroimagers should be careful of this eventuality.

Another question I have is: have connectionist modelers tried to explain with their models how unified percepts arise from 分散式 representations? Presumably, a localist/GMcell approach does not suffer from this issue. Or do scientists believe this is a completely independent problem, with models having no bearing on such emergence.

欢迎任何反馈或指点。

谢谢和最诚挚的问候,
帕万