Connectionist Modelling of Language Acquisition

In this talk I will start by giving an introduction on a connectionist model, a.k.a. parallel distributed processing model (Rumelhart & McClelland, 1986), that I have been working on. I will focus the discussion on a model known as Simple Recurrent Network (SRN) that was introduced as a biologically plausible model for dealing with sequential information (Elman, 1990). Basic working mechanisms and the behaviours of SRNs will be illustrated with case studies of speech segmentation and acquisition of syntactic categories.

I will then talk about the relevance of modelling and simulation to empirical work in language acquisition and linguistic theories. Experiments on infants learning artificial languages (Saffran, Aslin, & Newport, 1996) suggested that infants can segment continuous speech into words on the basis of transitional probabilities. I will illustrate that such a behaviour could be well captured by SRNs. Marcus et al. on the other hand are in favour of a rule-based account (Marcus, Vijayan, Bandi Rao, & Vishton, 1999), they argued that SRNs fail to exhibit learning capabilities observed in humans. This brings out the very fundamental issue of generalisation.

Several types of generalisations that are relevant to language acquisition will be discussed including generalisation towards combinatorial productivity (van der Velde et al., 2004), an aspect of productivity of language arises from novel combinations of lexical items. I will report my work (Wong, Minett, & Wang, 2006) in this area as the final case study of the talk.

References

Elman, J. L. (1990). Finding structure in time. Cognitive Science, 14, 179-211.
Marcus, G. F., Vijayan, S., Bandi Rao, S., & Vishton, P. M. (1999). Rule Learning by Seven-Month-Old Infants. Science, 283(5398), 77-80.
Rumelhart, D. E., & McClelland, J. L. (1986). Parallel distributed processing: explorations in the microstructure of cognition (Vol. 1). Cambridge, Mass.: MIT Press.
Saffran, J. R., Aslin, R. N., & Newport, E. L. (1996). Statistical learning by 8-month-old infants. Science, 274(5294), 1926-1928.
van der Velde, F., van der Voort van der Kleij, G. T., & de Kamps, M. (2004). Lack of combinatorial productivity in language processing with simple recurrent networks. Connection Science, 16(1), 21-46.
Wong, F. C. K., Minett, J. W., & Wang, W. S.-Y. (2006). Reassessing combinatorial productivity exhibited by simple recurrent networks in language acquisition. In Proceedings of the 2006 International Joint Conference on Neural Networks (pp. 2905-2912). Vancouver, Canada.