Category Archives: language

Major conscious and unconscious processes in the brain: part 4: the easy problem of A-consciousness

This is the part 4 of the multi–part series on conscious and unconscious processes in the brain.

I’ll like to start with a quote from the Mundaka Upanishads:

Two birds, inseparable friends, cling to the same tree. One of them eats the sweet fruit, the other looks on without eating.


On the same tree man sits grieving, immersed, bewildered, by his own impotence. But when he sees the other lord contented and knows his glory, then his grief passes away.

Today I plan to delineate the major conscious processes in the brain, without bothering with their neural correlates or how they are related to unconscious processes that I have delineated earlier. Also I’ll be restricting the discussion mostly to the easy problem of Access or A- consciousness.  leaving the hard problem of phenomenal or P-consciousness for later.

I’ll first like to quote a definition of consciousness form Baars:

The contents of consciousness include the immediate perceptual world; inner speech and visual imagery; the fleeting present and its fading traces in immediate memory; bodily feelings like pleasure, pain, and excitement; surges of feeling; autobiographical events when they are remembered; clear and immediate intentions, expectations and actions; explicit beliefs about oneself and the world; and concepts that are abstract but focal. In spite of decades of behaviouristic avoidance, few would quarrel with this list today.

Next I would like to list the subsystems identified by Charles T tart that are involved in consciousness:

  • EXTEROCEPTION (sensing the external world)
  • INTEROCEPTION (sensing the body)
  • INPUT-PROCESSING (seeing meaningful stimuli)
  • EMOTIONS
  • MEMORY
  • SPACE/TIME SENSE
  • SENSE OF IDENTITY
  • EVALUATION AND DECISION -MAKING
  • MOTOR OUTPUT
  • SUBCONSCIOUS

With this background, let me delineate the major conscious processes/ systems that make up the A-consciousness as per me:-

  1. Perceptual system: Once the spotlight of attention is available, it can be used to bring into focus the unconscious input representations that the brain is creating.  Thus a system may evolve that has access to information regarding the sensations that are being processed or in other words that perceives and is conscious of what is being sensed. To perceive is to have access to ones sensations.  In Tarts model , it is the input-processing module  that ‘sees’ meaningful stimuli and ignores the rest / hides them from second-order representation. This is Baars immediate perceptual world.
  2. Agency system: The spotlight of attention can also bring into foreground the unconscious urges that propel movement. This access to information regarding how and why we move gives rise to the emergence of A-consciousness of will/ volition/agency. To will is to have access to ones action-causes. In tarts model , it is the motor output module that enables sense of voluntary movement. In Baars definition it is clear and immediate intentions, expectations and actions.
  3. Memory system:  The spotlight of attention may also bring into focus past learning. This access to information regarding past unconscious learning gives rise to A-consciousness of remembering/ recognizing. To remember is to have access to past learning. The Tart subsystem for the same is Memory and Baars definition is autobiographical events when they are remembered. 
  4. Feeling (emotional/ mood) system: The spotlight of attention may also highlight the emotional state of the organism. An information about one’s own emotional state gives rise to the A-consciousness of feelings that have an emotional tone/ mood associated. To feel is to have access to ones emotional state. The emotions system of Tart and Baars bodily feelings like pleasure, pain, and excitement; surges of feeling relate to this.
  5. Deliberation/ reasoning/thought system: The spotlight of attention may also highlight the decisional and evaluative unconscious processes that the organism indulges in. An information about which values guided decision can lead to a reasoning module that justifies the decisions and an A-consciousness of introspection. To think is to have access to ones own deliberation and evaluative process. Tarts evaluative and decision making module is for the same. Baars definition may be enhanced to include intorspection i.e access to thoughts and thinking (remember Descartes dictum of I think therefore I am. ) as part of consciousness.
  6. Modeling system that can differentiate and perceive dualism: The spotlight of attention may highlight the dual properties of the world (deterministic and chaotic ). An information regarding the fact that two contradictory models of the world can both be true at the same time, leads to modeling of oneslf that is different from the world giving rise to the difference between ‘this’ and ‘that’ and giving rise to the sense of self. One models both the self and the world based on principles/ subsystems of extereocpetion and interoception and this give rise to A-consciousness of beliefs about the self and the world. To believe is to have access to one’s model of something. One has access to a self/ subjectivity different from world and defined by interoceptive senses ; and a world/ reality different from self defined by exterioceptive senses. The interocpetive and exteroceptive subsystems of  Tart and Baars  explicit beliefs about oneself and the world are relevant here. This system give rise to the concept of a subjective person or self.
  7. Language system that can report on subjective contents and propositions. The spotlight of awareness may  verbalize the unconscious communicative intents and propositions giving rise to access to inner speech and enabling overt language and reporting capabilities. To verbally report is to have access to the underlying narrative that one wants to communicate and that one is creating/confabulating. This narrative and story-telling capability should also in my view lead to the A-consciousness of the stream of consciousness. This would be implemented most probably by Tart’s unconscious and space/time sense modules and relates to Baars the fleeting present and its fading traces in immediate memory- a sense of an ongoing stream of consciousness. To have a stream of consciousness is to have access to one’s inner narrative.
  8. Awareness system that can bring into focal awareness the different conscious process that are seen as  coherent. : the spotlight of attention can also be turned upon itself- an information about what all processes make a coherent whole and are thus being attended and amplified gives rise to a sense of self-identity that is stable across time and  unified in space. To be aware is to have access to what one is attending or focusing on or is ‘conscious’ of. Tarts Sense of identity subsystem and Baars concepts that are abstract but focal relate to this. Once available the spotlight of awareness opens the floodgates of phenomenal or P-consciousness or experience in the here-and-now of qualia that are invariant and experiential in  nature. That ‘feeling of what it means to be’ of course is the subject matter for another day and another post!

Major conscious and unconcoscious processes in the brain: part 2

This is the second in the series about major conscious and unconscious processes in the brain.  The first part can be found here. This post  tries to document a few more processes / functions in the brain and their neural substrates.
To recap, the major processes  in brain (along with sample broad brain regions (grossly simplified) associated) are :

  1. Sensory (occipital)
  2. Motor (parietal)
  3. Learning (hippocampal formation in medial temporal)
  4. Affective (amygdalar and limbic system)
  5. Evaluative/decisional (frontal)
These are supplanted by the following processes and mechanisms.
6. Modeling system/ Hemispheric laterlaization: Another system/ mechanism that the brain may find useful and develop is the ability to model the world and model the self and others . This presents the following problem. The world consist of objects that follow deterministic casual laws thus lending order to it as well as seeming agents that act by their own volition and thus leading to chaos. The modeling required to model causal, deterministic world may suffer from different design constraints than that required to model a chaotic, agentic world.  The brain, I propose, solves this, by having two hemispheres, one specialized for interacting with the world based on the model of the world as orderly, deterministic , statistically regular world; while the other hemisphere specialized for interacting with the world assuming it as a chaotic , agentic, probabilistically undetermined world. The two hemispheres co-operate with each other and respond using the advantages offered by the different strategies of both hemispheres. To recap, left hemisphere is specialized for causal patterns, sequences, analysis and interpretation, classifying objects (categorical spatial represnetation) , verbal abilities depending on analysis of sequences, uses prototypes (statistical mean) and uses Match strategy of responding in a statistical pattern, Music lyrics, and works on local stimuli (components) and parses high spatial frequency and values relativity. The right brain on the other hand is specialized for random/unperdicatble associations, scenes, synthesis and documentation, acting on objects (co-ordinate spatial representation), Spatial abilities depending on synthesis of objects making the scene, uses exemplars (actual events) and uses Maximizing strategy of responding as per probability at the moment, Music melody, and works on  global stimuli (wholes) and parses low spatial frequency and values absoluteness. To summarize, left hemisphere is best suited to model temporal dimensions in an analytical and causal manner, while right hemisphere is best suited to model the spatial dimensions in an holistic and agentic manner. This modeling, it needs to be emphasized, need not be  conscious, but could be entirely unconscious and have unconscious effects. 

7. Communciation system/ perisylvian area/ mirror neurons?: Once an organism has discovered/ realized unconsciously that there are other agents/ con specifics in the world , a brain system that communicates (on an unconscious level) with the others can evolve. A system can evolve that signals the emotional/internal state to others and can also sense the emotional/ internal state of others. This can be used as an aid to predict how the agent will act – as the agent is similar to oneself, one can predict how the other will act based on its internal state by simulating how one would act himself , given the same internal state. Sensing the internal state of others is one side of the coin, the other part is signalling your own internal state honestly to others to aid communication and enhance fitness by group selection. Remember that none of these consdireations need to be conscious. Even unicellular bacteria that live in colonies/ cultures evolve communication systems based on sensing and emitting chemicals etc.  In humans the mirror neuron system activated by others actions, the emotional expression and contagious unconscious empathy may all be the unconscious communciation system driven by non-verbal communication based on mirroring and mirror neurons.

8. Attention system : The last (for now!) system to evolve might be related to directing attention or selectivity processing relevant inputs, actions, affects, evaluations, associations, models and communciations while suppressing irrelevant ones. At any time , one is bombarded by many (all unconscious ) different stimuli, urges, activated associations, body states, values, models and communications from con specifics- these may or may not be relevant to current situation/ goal.  Not everything can be processed equally as the brain has limited computational resources. This leads to a mechanism/system to gauze relevance and thus bias the other systems by selectively processing some aspects in detail while ignoring others. This attentional/orientational mechanism may be covert, may be unconscious and might be triggered by external events/ voluntarily directed; important thing to realize is that  attention seems to integrate the output and inputs of other brain systems/ mechanisms  by selectivity highlighting a few features that are relevant and coherent. This also ultimately leads to  opening the doors to the next higher level of processing by brain – the conscious processing, which is computationally more demanding and thus requires attention to restrict the inputs that it can process. The attentional system opens the floodgates of heaven (consciousness) for the humans/ animals that are able to use it appropriately.

The spotlight of attention once created leads to conscious experiences of perception, agency, memory, feelings, thoughts, self-awareness, inner speech and identity. That of course is material for another post!

Linguistic effects on unconscious color perception

The post headline may seem an oxymoron , but it is indeed possible to perceive colors unconsciously. How do we know that someone has perceived a color, when he doesn’t report the qualia. We do so by measuring the effects on subsequent behavior. Consider subliminal priming. Consider a subliminal stroop test, in which color patches are presented subliminally and then color lexical terms are presented consciously in neutral (say black) ink. I’m sure with this subliminal modified stroop test one could still get a color and lexical term interaction effect; the point is that color , when not perceived, may still influence subsequent behavior.

The experimental paradigm in this PNAS article did not go so far, but restricted itself to color stimuli that was not attended to; that is, the color was indeed perceived, but it was not attended to (the task involved attention to form rather than color) and so as the color was not attended to, they presumed that the effects that the color information would have on behavior would be completely unconscious. I’m not convinced, but that doesn’t invalidate their otherwise very beautiful study that once again provides strong evidence for the milder version of the Sapir-Whorf hypothesis, at least as it relates to categorical color perception. 

Now, I have written previously about Sapir- Whorf hypothesis in general,  and in particular about the ability of Russians( who have two separate terms for light and dark blue) to visually discriminate between light and dark blue significantly better than their English counterparts, thanks to their rich color lexicon; so this new study that found that Greek-natives (who also have different lexical terms for light and dark blue) were superior to English-natives in terms of discriminating categorical color perception for light and dark blue color, did not come as a surprise or seemed ground-breaking; but there are important differences both in terms of the procedures used and the processes involved.

This study, works at pre-attentive level, uses physiological measures like ERP (they studied the vMMN – visual Mismatch Negativity) to determine whether the color stimuli had differential effect even at pre-attentive perception and thus provides independent evidence for the effect of Language on color perception. I’ll now quote from the abstract and discussion section:

It is now established that native language affects one’s perception of the world. However, it is unknown whether this effect is merely driven by conscious, language-based evaluation of the environment or whether it reflects fundamental differences in perceptual processing between individuals speaking different languages. Using brain potentials, we demonstrate that the existence in Greek of 2 color terms—ghalazio and ble—distinguishing light and dark blue leads to greater and faster perceptual discrimination of these colors in native speakers of Greek than in native speakers of English. The visual mismatch negativity, an index of automatic and preattentive change detection, was similar for blue and green deviant stimuli during a color oddball detection task in English participants, but it was significantly larger for blue than green deviant stimuli in native speakers of Greek. These findings establish an implicit effect of language-specific terminology on human color perception.

This study tested potential effects of color terminology in different languages on early stages of visual perception using the vMMN, an electrophysiological index of perceptual deviancy detection. The vMMN findings show a greater distinction between different shades of blue than different shades of green in Greek participants, whereas English speakers show no such distinction. To our knowledge, this is the first demonstration of a relationship between native language and unconscious, preattentive color discrimination rather than simply conscious, overt color categorization.

To conclude, our electrophysiological findings reveal not only an effect of the native language on implicit color discrimination as indexed by preattentive change detection but even electrophysiological differences occurring as early as 100 ms after stimulus presentation, a time range associated with activity in the primary and secondary visual cortices (22). We therefore demonstrate that language-specific distinctions between 2 colors affect early visual processing, even when color is task irrelevant. At debriefing, none of the participants highlighted the critical stimulus dimension tested (luminance) or reported verbalizing the colors presented to them. The findings of the present study establish that early stages of color perception are unconsciously affected by the terminology specific to the native language. They lend strong support to the Whorfian hypothesis by demonstrating, for the first time, differences between speakers of different languages in early stages of color perception beyond the observation of high-level categorization and discrimination effects strategically and overtly contingent on language specific
distinctions.

I think this fits in with predictive models of perception, wherein, earlier stages of visual processing, that are unrelated to color discrimination, may still be primed by color information that one has obtained earlier and has processed pre-attentively. I, as always , am excited by this proof of whorfian hypotheses.

Language and intentionality

Michael Tomasello has a new book out titled ” The origins of human communication” and the book seems to be promising, though has been a bit harshly reviewed at the Babel’s Dawn. In it Tomasello proposes that a pre-requisite for language is ‘a psychological infrastructure of shared intentionality’. It is based on Jean Nicod lectures and you can read a review here too.
What I am most interested is in this intentionality business. I have commented on orders of intentionality previously and this shared intentionality seems to fit the third order of intentionality that I proposed was necessary for communication.

But first for the premise of the book:

Tomasello opens his book with a consideration of the “infrastructure” that enables people to tell one another things. Apes do not have this infrastructure and the absence leads to scenes like this one:

A “whimpering chimpanzee child” is searching for its mother; the other chimps in the area are smart enough and social enough to recognize why the chimpanzee is whimpering; sometimes one of the chimps present will know where the mother is, and of course chimps have the physical ability to raise an arm point out the mother; even so, chimpanzees never help forlorn infants by pointing to the mother.

Why not?

There is a straightforward, Darwinian explanation for the ape’s mum’s-the-word behavior. Individuals don’t help non-kin. There is nothing in it for the informed adults to help the whimpering child of another. But Tomasello comes at the question from another perspective. Humans typically do help out whimpering children, even if the child is a stranger. An adult, happening upon a solitary, unknown, whimpering child is very likely to stop and ask what is wrong, take charge, and stick around until the problem is resolved. This activity strikes us as perfectly natural, normal behavior, even though it is contrary to so many of the rules in Darwin’s book. What, Tomasello wonders, is there about humans that makes such behavior easy and routine? His answer: “a psychological infrastructure of shared intentionality” [p. 12].

Thus, the premise is that pro-social behaviour and the shared intentionality underlying it are the pre-requisites for any meaningful language to evolve. And for this some tools are required.

The psychological tools Tomasello refers to are cognitive and emotional. The cognitive tools give us the understanding to engage in joint purposes and joint attention. The emotional tools provide us with the motivation for helping and sharing with others. These tools enable people to act together on a “common ground.”

Ebolles goes on further to speculate that this could be tied to Autistics’ difficulty with language and I concur that the cognitive deficits related to intentionality as opposed to affective deficits empathy or mindblindness may be the roots of Autistics’ language and communicative difficulties. We already know that they lack ToM to an extent and they also have communicative and social difficulties; might lack of shared intentionality, or intentionality at all or the lack of feeling of one has an intentional agent,  lie at the heart of the autism issue?

Immediately one can imagine all sorts of peculiarities that would arise in people who lack some part of these needs. Some people might have the prosocial motivation but not the cognitive ability to form a bird’s eye view. Perhaps autistic-spectrum disorder includes this difficulty. Others might have the cognitive ability, but not the prosocial motivation. There’s your sociopath, in a nutshell.

I think this common ground and ‘infrastructure of shared intentionality’ concept is awesome and I intend to read the book and review it soon on this blog. 

Intentionality: order, order!

I have been reading, of late, some articles that have invoked the concept of intentionality and its orders. More specifically, this has been with respect to Social Brian hypothesis of Robin Dunbar, whereby he claims that humans evolved intelligence to be able to cope with your in-laws (and other social members of one’s groups). Leaving asides the main premise of the social brain hypothesis, which I find convincing to an extent, he also claims that monkeys have only first order intentionality, while apes have second order and humans are able to function at about fifth order of intentionality, with some like Shakespeare being able to work on the sixth order. To quote at length form the ‘beginner’s guide to intentionality’:

Computers can be said to know things because their memories contain information; however, it seems unlikely that they know that they know these things, in that we have no evidence that they can reflect on their states of ‘‘mind.’’ In the jargon of the philosophy of mind, computers are zero-order intentional machines. Intentionality is the term that philosophers of mind use to refer to the state of having a state of mind (knowing, believing, thinking, wanting, understanding, intending, etc).

Most vertebrates are probably capable of reflecting on their states of mind, at least in some crude sense: they know that they know. Organisms of this kind are first-order intentional. By extension, second-order intentional organisms know that someone else knows something, and third-order intentional organisms know that someone else knows that someone else knows something. In principle, the sequence can be extended reflexively indefinitely, although, in practice, humans rarely engage in more than fourth-order intentionality in everyday life and probably face an upper limit at sixth-order (‘‘Peter knows that Jane believes that Mark thinks that Paula wants Jake to suppose that Amelia intends to do something’’).

A minimum of fourth-order intentionality is required for literature that goes beyond the merely narrative (‘‘the writer wants the reader to believe that character A thinks that character B intends to do something’’). Similar abilities may be required for science, since doing science requires us to ask whether the world can be other than it is (a second-order problem at the very least) and then ask someone else to do the same (an additional order of intentionality).

I find the above definitions (and other I have found on the web), slightly problematic, so I’ll attempt my own synthesis on the matter:

  1. Zeroth order or No intentionality: Having knowledge but no ‘awareness ‘ of knowledge. Mere representation of information, but no meta awareness of that representation. Computers and machines , and even simple life forms like bacteria etc, may have this (no) intentionality, wherein they have ‘facts’ about the world, but no beliefs, desires etc.
  2. First Order Intentionality: Awareness of knowledge that is distinct from mere knowledge. A belief system. Knowing that something you know may be incorrect from the actual world scenario. You know what you know and you know what you don’t know. Meta cognition. Beliefs, desires etc. Important thing to note is that only ‘I know’ is covered in this definition. A limited ‘You know as I know’ may be covered at this order as one may be aware of other people as being intentional agents , but whose beliefs are congruent with one’s own! ‘You know something that may be different from what I know’ is not possible yet. Most mammals including rats and monkeys are at this level. Awareness ta this level may be that others too have facts of world at their disposal.
  3. Second order intentionality: Awareness of a belief-system that is distinct from the belief system itself. A Theory of Mind. You know that someone else may know things differently from both as they are and as you think they are. Awareness that others have a mind or a belief-system. Ability to keep two different belief systems in the mind- one of your own and the other of another third person. Apes and children age 4 demonstrate this level and order of intentionality. They have a theory of mind as to the fact that others have beliefs and that these are after all beliefs and can be false too. Awareness that others have beliefs, but still no awareness that they have a ToM too!
  4. Third order intentionality: Awareness of a ToM that is distinct from the ToM itself. A communicative intent. Joint attention. Language. symbol grounding. Knowing that someone else may have different views regarding what you yourself believe and thus it is important to communicate your internal intentions, beliefs , desires etc to others so that there is common ground on which communication and speech acts can proceed. this also enables grounds for lies and deceptions in the sense that one can deliberately lead someone to believe what one oneself does not believe. As per this source , communication requires third order of intentionality. To quote:

    5. Suppose my little brother intends for me to jump. He might (and sometimes does) achieve this by sneaking up behind me and yelling “Boo!”. But that’s not communication, in the fullest sense of the word. It would be quite a different sort of action were he to instead request of me, “please jump.” (I don’t think he’d find that nearly so fun, for one thing.) Such a speech-act would show not only that he intends me to jump, but also that he intends for me to recognize that he wants me to jump.

    Purposive communication requires an intentional state of at least third-order complexity. The speaker wants his audience to recognize what the speaker intends by his utterance. Put another way, you don’t just communicate ‘X’, you rather communicate, “I am trying to convey ‘X'”. (This is the difference between discreetly insulting someone, or making it clear to him that you want him to know you’re insulting him.) Anything less would fail to qualify as ‘communication’, in the fullest sense of the word.

  5. Fourth-order Intentionality: Awareness of a communicative act that is distinct from the communicative act itself: A narrative or story telling/ story understanding capability. An ability to weave experiences into a running narrative such that it incorporates different communicative acts or ‘scenes’. An understanding of ‘roles’ that one is playing that give shape to all the communicative acts one participate sin and the narrative one weaves for oneself. A limited awareness that others are also communicative agents , but not a full awareness , that like oneself, they are also acting a script/ playing a role/ having a running narrative using which they interpret events. It is important to emphasize that story telling requires one to visit a new world in which the protagonist is separate, but also one is in a state of willful suspension of disbelief and thus one feels along-with the protagonist, but still retains one’s own narrative: separate, and quite distinct, form the story-teller’s narrative. Story-telling, and story understanding and the interpreter module of humans that gives rise to stream of consciousness to me are the hallmarks of fourth order of intentionality and most of us juts stop there. One may mistakenly believe that there is only one role / narrative and that everybody shares the same narrative.
  6. Fifth order intentionality: Awareness of roles and narratives that are distinct from the role or narrative. An organizing system of religion/ myths using which one interprets stories. Awareness that others too have their own narratives and are playing a script/ performing their roles. Awareness that one’s role/ stance / understanding of world can be radically different from someone having the same experiences but using a different interpretation. A culture . A worldview. It is instructive to note that Dunbar considers that religion and story telling are higher level intentional activities.

I’ll leave things as they are for now as this fits nicely with my obsession with 5 + 3 stage developmental process. Higher orders of intentionality may exist, but probably we humans are not yet evolved to appreciate their subtleties/ find practical examples.

‘A’ is for a RED apple and ‘V’ is for a PURPLE van!

New research has unearthed that the grapheme-color synesthesia is not idiosyncratic , but follows some typical patterns. Grapheme – color synesthesia is one of the common types of synesthesia wherein one sees color associated with visualizing an alphabet / letter. Thus, whenever one see the alphabet ‘A’ one may also have a perception of color ‘red’. Till now, it was believed that this association of colors with alphabets was random and idiosyncratic; but new research has now revealed that it follows a pattern with most synesthetes more likely to associate typical colors with alphabets and for example report ‘A’ as red and V as ‘purple’.

Jamie Ward’s team that found this phenomenon speculates that the hue could be associated with the frequency of the word. Thus, as ‘A’ is a frequently used word it is associated with a common color ‘red’. ‘V’ which is infrequently used in the lexicon is associated with a similar infrequently encountered color purple. I am not sure how their new study is different from their earlier study that also found thus association and I believe that there would be some truth to their theory. however, the science daily article also talks about saturation. So I though I would jump in.

Colors can be conceptualized as per the HSV/ HSL or HSB system and understood in terms of hue , saturation and value/ brightness. I would personally be inclined to interpret the ‘A’ is red and ‘V’ is purple mapping as the outcome of a mapping of the alphabet order (a, b, c, ….x, y, z) to the color order in the rainbow / hue dimension (VIBGYOR). ‘A’ is one end ofthe spec trim and thus red in color, while ‘V’ is on another end of spectrum and thus more likely to be ‘violet’ in color. The frequency of usage of the alphabet should ideally map to brightness/ value of the synesthete color as in color space value is mapped to the amount of light reflected. saturation or ‘purity’ of color is a bit difficult to map onto the alphabet; but one could venture forth and suggest it has to do with how ‘pure’ the alphabet is ….is it always pronounced in one way….or are their multiple pronunciations associated with the same alphabet.

Mapping a linear progression of hues along VIBGYOR axis to alphabet order or numeral oredr is not that hard to envisage or visualize. If neurons of adjacent colorotopic and lexicotopic maps (assuming there are such maps for color and lexicon in the brain) in the brain overlap/ cross-over we would have the phenomenon of grapheme-color synestehesia that accounted for the commonalities in hues and alphabet association. However, we just know of retinotopic sort of maps in brains and these fit in with our existing knowledge. How the brain stores information about saturation/ value and correspondingly frequency and purity of alphabets and maps between the too, can lead to novel insights as to how information is stored in the brain.

I am excited and believe that we are on verge of breaking new ground ( I haven’t read the new Jamie ward paper though yet) and I have my own theories on why color is so important and may provide us many more clues (color and music are two most interesting phenomenon I believe). Are you excited? Do you have any theories?

PS: I just found that Jamie Ward is writing a book called “The Frog who Croaked Blue: Synaesthesia and the Mixing of the Senses” in which he recounts the experience of a synesthete who heard frog croaks as blue and chirping of cricket as red. To me this immediately conjures up the colortopic map with red at one end (high, feminine, shrill noises) and blue at the other (more manly, bass noise). This mapping of sound with colors may again follow the hue, saturation and value (three dimensions) with loudness of sound being proportional to the value of color being perceived and the hue and pitch mapped. Also , this may be an idiosyncratic experience, or this may be true of the species as a whole that we map more shrill noises to red and soothing and duller sounds to blue/ violet.

Categorical color perception: the language effect

I touched on the sapir-whorf hypothesis and how Russians are better able to do better Categorical Perception (CP) of color, thanks to the fact that they have a richer color terms lexicon than English, last month.

I have also covered the research of P. Kay earlier regarding color terms and their evolution. Now a new PNAS paper by Kay et al shows that while the left hemisphere(LH) , which is involved in language, shows superior CP effect in adults, the reverse trend is shown in infants i.e.e the infants show a stronger CP of colors when the stimuli is presented to Left Visual field (LVF) and hence processed by RH.

Their hypothesis was that while the CP of colors in adults is mediated by language, the CP in infants is non-verbal and the cP in adults may or may not build on this childhood CP ability. The results go on to show that not only doers language affect the left hemisphere dominance on categorical perception of colors ; it does so by overriding an inborn RH dominance for the same task. thus, there is no doubt that the color term lexicon heavily influences how we categorize colors in the adulthood.

Here is their conclusion:

Evidence suggesting that color CP varies cross-linguistically, and that color CP is eliminated by verbal interference, has supported the hypothesis that color CP depends on access to lexical codes for color . However, the finding of color category effects in prelinguistic infants and toddlers has led others to argue that language cannot be the only origin of the effect . The current study finds evidence to support both positions. Color CP is found in 4- to 6-month-old infants, replicating previous infant studies. However, the absence of a category effect in the LH for infants, but the presence of a greater LH than RH category effect for adults, suggests that language-driven CP in adults may not build on prelinguistic CP, but that language instead imposes its categories on a LH that is not categorically prepartitioned. The current findings may therefore suggest a compromise between the two positions: there is a form of CP that is nonlinguistic and RH based (found in infancy) and a form of CP that is lexically influenced and biased to the LH (found in adulthood). Color CP is found for both infants and adults, but the contribution of the LH and RH to color CP appears to change across the life span.

Autism / Psychosis: Agency and Joint Attention

A recent study by Tomosello’s group indicates that children with autism, can help a stranger pick a pen (and thus can apparently infer goal and intentional states of others), but cannot indulge in co-operative behavior that may involve shared goals and shared attention.

As per Translating Autism blog:

This fresh-off-the-press article comes to us from Dr. Michael Tomasello’s group at the Max Planck institute in Germany. The authors present the results of two studies looking at helping and cooperation in children with autism. The first study compared 15 children with ASD (14 with Autism and 1 with PDD-NOS) with 15 children with other non-ASD developmental delays (40 months of age average). During this study the children were place in situations that either called for helping behaviors (such as picking up a pen that the researcher dropped and could not reach) or a similar situation that did not necessarily call for helping behaviors (such as when the researcher threw the pen on purpose and did not attempt to pick it up). Both groups (children with Autism and children with other developmental delays) showed more helping behaviors when placed in the situation that called for such behaviors. That is, when the experimenter was “trying” to reach an out-of-reach object, both groups were more likely to help than when the experimenter was not trying to reach for the object. The authors concluded that these behaviors showed that both groups understood the adult’s goals and were motivated to help her. In the second study, the same children were placed in situations that called for “cooperative” behaviors, such as a task requiring them to work with the researcher by simultaneously pulling at two cylinders to reach a toy. The results showed that children with autism were less likely than kids with other developmental delays to successfully complete the cooperation tasks. Furthermore, the children with autism were less likely to initiate additional attempts to complete the task when the task was interrupted. The authors concluded that, at least at this developmental period, children with autism seem to understand the social components of situations that call for “helping” behaviors and engage in helping behaviors, but only when such help does not require interpersonal cooperation. However, when cooperation is required to complete the task, these children are less likely to correctly engage with another partner, possibly because the unique “shared” component of cooperation. That is, cooperation requires shared goals, shared attention, and a shared plan of action, processes that seem to be affected in children with autism.

Here is the abstract of the Tomosello paper:

Helping and cooperation are central to human social life. Here, we report two studies investigating these social behaviors in children with autism and children with developmental delay. In the first study, both groups of children helped the experimenter attain her goals. In the second study, both groups of children cooperated with an adult, but fewer children with autism performed the tasks successfully. When the adult stopped interacting at a certain moment, children with autism produced fewer attempts to re-engage her, possibly indicating that they had not formed a shared goal/shared intentions with her. These results are discussed in terms of the prerequisite cognitive and motivational skills and propensities underlying social behavior

From the above it is clear that children with Autism lack shared attention: a pre-requisite for language and their language impediments may also be due to this fact. If we contrast this with Schizophrenia/ Psychosis ( and assuming they are at opposite ends) it is not hard to see that with too much shared goals/ intentions/ attention, one may likely confuse between one’s own goals and those of others and in a joint scenario be more susceptible to delusions of control/ though insertion, wherein the shared space has become so vast that one seems to be controlled by the other or intruded by the other. thus , I propose that children susceptible to psychosis should show enhanced cooperating behavior indicating an overactive shared goals/ attention module.

Another interesting study I would like to discuss is the recent reporting of a dysfunctional ‘self’ module/model in a Trust game as compared to the ‘other’ module/ model. Here is how the Science Daily describes the Trust game that was used in the game.

In the trust game, one player receives an amount of money and then sends whatever amount he or she wants to the other player via computer message. The amount sent is tripled and the player at the other end then decides how much of the tripled amount to send back. The game has several rounds.

The ‘self’ module was identified as the brain areas (cingulate cortex) involved when making the decision to share the initial amount of money with another person. The ‘other’ module was defined as network region activated when the decision of the other player was revealed to them.

It was found that autistics showed lowered activity in the ‘self’ module. The authors construe this as evidence that they have a defective self concept.

“To have a good self concept, you have to be able to decide if the shared outcome is due to the other person or due to you,” said Montague. “If people can’t see themselves as a distinct entities at deeper levels, there is a disconnect.”

I beg to differ. In my view the findings can be explained using the joint attention / goal/ outcome defect outlined above. Although I believe that their explanation that people with autism may have a diminished sense of self or Agency also makes intuitive sense and I have argued the same previously. I contrast that with the Psychotic case where one attributes too much agency- even to inanimate objects or animals for example. However, in this case a more parsimonious explanation can be that the autistics were not able to model the others goal as their own (the familiar simulation argument) and could not indulged in joint goal intention and thus failed to optimally use the ‘self’ module i/e failed to take whatever actions were needed for a co-operative and trustful behavior .

The Friths adequately sum that up:

In a preview in the journal Neuron, Chris and Uta Frith wrote, “This is an exciting result because it suggests that some mechanisms of social interaction are intact in these high-functioning cases. What is the critical difference between the self phase and the other phase? We believe that the simple distinction of self versus other is not adequate. “It involves higher-order mentalizing: you care what another person thinks of you, and even further, you care that the other person trusts you. You would not do this when playing against a computer. In autism there is no difference,” wrote the Friths, who are at University College London.

Russsinas have a richer discriminative experience of light and dark blue qualia

I have blogged extensively earlier regarding language, color and the sapir -whorf hypothesis. My position in the above is clear, I lean towards the sapir-whorf hypothesis and a mild form of linguistic determinism. Now a new study (which I had missed earlier) by Lera Boroditsky presents further corroborating evidence that language influences even such basic functions as color perception. As per their 2007 PNAS paper, Russians are better (more speedily) able to distinguish between the light blue and dark blue color in an objective color perception task, thanks to the fact that Russian has a different color term for dark blue and a different one for the light blue. It is an excellent paper and I present some excerpts from the introduction:

Different languages divide color space differently. For example,the English term ‘‘blue’’ can be used to describe all of the colors in Fig. 1. Unlike English, Russian makes an obligatory distinction between lighter blues (‘‘goluboy’’) and darker blues (‘‘siniy’’). Like other basic color words, ‘‘siniy’’ and ‘‘goluboy’’ tend to be learned early by Russian children (1) and share many of the usage and behavioral properties of other basic color words (2). There is no single generic word for ‘‘blue’’ in Russian that can be used to describe all of the colors in Fig. 1 (nor to adequately translate the title of this work from English to Russian). Does this difference between languages lead to differences in how people discriminate colors?

The question of cross-linguistic differences in color perception has a long and venerable history (e.g., refs. 3–14) and has been a cornerstone issue in the debate on whether and how much language shapes thinking (15). Previous studies have found cross-linguistic differences in subjective color similarity judgments and color confusability in memory (4, 5, 10, 12, 16). For example, if two colors are called by the same name in a language, speakers of that language will judge the two colors to be more similar and will be more likely to confuse them in memory compared with people whose language assigns different names to the two colors. These cross-linguistic differences develop early in children, and their emergence has been shown to coincide with the acquisition of color terms (17). Further, cross-linguistic differences in similarity judgments and recognition memory can be disrupted by direct verbal interference (13, 18) or by indirectly preventing subjects from using their normal naming strategies (10), suggesting that linguistic representations are involved online in these kinds of color judgments.

Because previous cross-linguistic comparisons have relied on memory procedures or subjective judgments, the question of whether language affects objective color discrimination performance has remained. Studies testing only color memory leave open the possibility that, when subjects make perceptual discriminations among stimuli that can all be viewed at the same time, language may have no influence. In studies measuring subjective similarity, it is possible that any language-congruent bias results from a conscious, strategic decision on the part of the subject (19). Thus, such methods leave open the question of whether subjects’ normal ability to discriminate colors in an objective procedure is altered by language.

Here we measure color discrimination performance in two language groups in a simple, objective, perceptual task. Subjects were simultaneously shown three color squares arranged in a triad (see Fig. 1) and were asked to say which of the bottom two color squares was perceptually identical to the square on top.

This design combined the advantages of previous tasks in a way that allowed us to test for the effects of language on color perception in an objective task, with an implicit measure and minimal memory demands.

First, the task was objective in that subjects were asked to provide the correct answer to an unambiguous question, which they did with high accuracy. This feature of the design addressed the possibility that subjects rely only on linguistic representations when faced with an ambiguous task that requires a subjective judgment. If linguistic representations are only used to make subjective judgments in ambiguous tasks, then effects of language should not show up in an objective unambiguous task with a clear correct answer.

Second, all stimuli involved in a perceptual decision (in this case, the three color squares) were present on the screen simultaneously and remained in full view until the subjects responded. This allowed subjects to make their decisions in the presence of the perceptual stimulus and with minimal memory demands.

Finally, we used the implicit measure of reaction time, a subtle aspect of behavior that subjects do not generally modulate explicitly. Although subjects may decide to bias their decisions in choosing between two options in an ambiguous task, it is unlikely that they explicitly decide to take a little longer in responding in some trials than in others.

In summary, this design allowed us to test subjects’ discrimination performance of a simple, objective perceptual task. Further, by asking subjects to perform these perceptual discriminations with and without verbal interference, we are able to ask whether any cross-linguistic differences in color discrimination depend on the online involvement of language in the course of the task.

The questions asked here are as follows. Are there crosslinguistic differences in color discrimination even for simple, objective, perceptual discrimination tasks? If so, do these differences depend on the online involvement of language? Previous studies with English speakers have demonstrated that verbal interference changes English speakers’ performance in speeded color discrimination (21) and in visual searching (22, 23) across the English blue/green boundary. If a color boundary is present in one language but not another, will the two language groups differ in their perceptual discrimination performance across that boundary? Further, will verbal interference affect only the performance of the language group that makes this linguistic distinction?

They then go on to discuss their experimental setup (which I recommend you go and read). Finally they present their findings:

We found that Russian speakers were faster to discriminate two colors if they fell into different linguistic categories in Russian (one siniy and the other goluboy) than if the two colors were from the same category (both siniy or both goluboy). This category advantage was eliminated by a verbal, but not a spatial, dual task. Further, effects of language were most pronounced on more difficult, finer discriminations. English speakers tested on the identical stimuli did not show a category advantage under any condition. These results demonstrate that categories in language can affect performance of basic perceptual color discrimination tasks. Further, they show that the effect of language is online, because it is disrupted by verbal interference. Finally, they show that color discrimination performance differs across language groups as a function of what perceptual distinctions are habitually made in a particular language.

They end on a philosophical note:

The Whorfian question is often interpreted as a question of whether language affects nonlinguistic processes. Putting the question in this way presupposes that linguistic and nonlinguistic processes are highly dissociated in normal human cognition, such that many tasks are accomplished without the involvement of language. A different approach to the Whorfian question would be to ask the extent to which linguistic processes are normally involved when people engage in all kinds of seemingly nonlinguistic tasks (e.g., simple perceptual discriminations that can be accomplished in the absence of language). Our results suggest that linguistic representations normally meddle in even surprisingly simple objective perceptual decisions.

To me this is another important paper that puts sapir-whorf hypothesis on the forefront. I would love to hear from those who do not endorse the spair-whorf hypothesis as to what they make of these results?

hat tip: Neuroanthropology blog.

Music and Language: dissociation between rule-crunching and memory-retrieval systems

I have previously written about how concepts are stored in the brain: they involve rule-based systems (A is bachelor if A is Single AND A is male) and memory based systems (prototypes and exemplars). I have also looked at how language involves both rules (the syntax of the language) as well as memory (semantics or word meanings) systems and our normal language comprehension as well as productions engages both types of systems.

It is a popular paradigm in cognitive linguistic research to present unexpected words in sentences (such as, “I’ll have my coffee with milk and concrete”), while monitoring brain activity using ERP, and find that the presentation of an unexpected word leads to a N400 peak in the temporal lobe areas. This violation of semantics is differentiated from when the syntax of the sentence is wrong, in which case we get changed activity in frontal lobes.

“Up until now, researchers had found that the processing of rules relies on an overlapping set of frontal lobe structures in music and language. However, in addition to rules, both language and music crucially require the memorization of arbitrary information such as words and melodies,” says the study’s principal investigator, Michael Ulmann, Ph.D., professor of neuroscience, psychology, neurology and linguistics.

For the first time , similar results have been obtained for music. If one assumes that changing an in-key note in a familiar melody is akin to an unexpected word in a sentence, then the same N400 peak is observed. Also , if a violation of harmonical rules , like an off-key note in an unfamiliar harmony, is akin to violations of linguistic syntax, then here too similar changes in frontal lobe activity were observed.

The subjects listened to 180 snippets of melodies. Half of the melodies were segments from tunes that most participants would know, such as “Three Blind Mice” and “Twinkle, Twinkle Little Star.” The other half included novel tunes composed by Miranda. Three versions of each well-known and novel melody were created: melodies containing an in-key deviant note (which could only be detected if the melody was familiar, and therefore memorized); melodies that contained an out-of-key deviant note (which violated rules of harmony); and the original (control) melodies.

For listeners familiar with a melody, an in-key deviant note violated the listener’s memory of the melody ? the song sounded musically “correct” and didn’t violate any rules of music, but it was different than what the listener had previously memorized. In contrast, in-key “deviant” notes in novel melodies did not violate memory (or rules) because the listeners did not know the tune.

Out-of-key deviant notes constituted violations of musical rules in both well-known and novel melodies. Additionally, out-of-key deviant notes violated memory in well-known melodies.

Miranda and Ullman examined the brain waves of the participants who listened to melodies in the different conditions, and found that violations of rules andmemory in music corresponded to the two patterns of brain waves seen in previous studies of rule and memory violations in language. That is, in-key violations of familiar (but not novel) melodies led to a brain-wave pattern similar to one called an “N400” that has previously been found with violations of words (such as, “I’ll have my coffee with milk and concrete”). Out-of-key violations of both familiar and novel melodies led to a brain-wave pattern over frontal lobe electrodes similar to patterns previously found for violations of rules in both language and music. Finally, out-of-key violations of familiar melodies also led to an N400-like pattern of brain activity, as expected because these are violations of memory as well as rules.

“This tells us that these two aspects of music, that is rules and memorized melodies, depend on two different brain systems – brain systems that also underlie rules and memorized information in language,” Ullman says. “The findings open up exciting new ways of thinking about and investigating the relationship between language and music, two fundamental human capacities.”

To me this seems exciting. My thesis has been that Men are better at rule-based things (syntax and harmony); while women are better at memory-based things (semantics and melody), so I’ll like to know whether the authors observed any gender effects. If so, this would be further proof for abstract vs concrete gender difference theory.