Access to information is critical for enabling awareness and stimulating social connectedness in mediated environments. In this article, which is wholly based on my doctoral thesis, I will discuss the cognitive processes necessary for promoting social connectedness in mediated environments. The information presented could be relevant for designers leveraging peripheral technologies to support communication in mediated settings. 

Face-to-Face Versus Computer-Mediated Communication

In ‘real’/face-to-face communication scenarios, one must perceive and pay attention to available cues, in order to discern the ‘states’ and ‘traits’ of another (Bodenhausen & Hugenberg, 2009). However, this becomes more difficult in mediated communication (i.e., communication with information communication technology), as multiple stimuli are competing for the perceiver’s attention, which might affect the quality of the interaction. Consequently, a number of cognitive processes including perception, attention, and memory (Matlin, 2005) are discussed. Moreover, C. A. Wisneski (1999) brings to the fore, the need to understand human cognitive abilities as they are believed to be highly relevant to tailor how the display content is presented, with much attention given to the ‘channel capacity’ limitations of the human perceptual system.

What is Perception?

According to Styles (2005), perception is a sensory process whereby sense organs (e.g., eye, nose, ear, etc.) transmit ‘physical energy’ detected from the world around us, which is encoded and dispatched to the brain through sensory neurons for analysis by the perceptual system. Styles (2005) describes a typical example of visual perception as follows. When light patterns enter the eye, it is encoded by the photoreceptors (i.e., via the rods and cones) in the retina, this information is then relayed via the pathways that handle visual input and is later transferred to the cortical regions of the brain primarily responsible for colour, shapes, movement, etc. perception. Basically, Styles (2005) suggests that the preliminary stages of perception are mostly governed by automatic and unconscious processes (i.e., executed without conscious awareness). On the other hand, Bodenhausen and Hugenberg (2009) more clearly explain the perceptual experience as the process where stimuli sensed from the outer world, are received by the observer who later converts these signals into more meaningful representations that define their inner feelings, sensations, or thoughts of the outside world.

Psychologists Engel, Fries & Singer (2001) distinguish between two approaches to perception namely: 

1. top-down processing
2. bottom-up processing

The top-down approach requires conscious awareness and refers to the process whereby stimuli are interpreted based on prior knowledge or experience (Gregory, 1970). Consequently, the top-down process is conceptually driven, such that one’s expectations of the incoming sensed data may influence how the information is perceived (Styles, 2005). A number of perception theorists (Gregory, 1970; Henle, 1989; Rock, 1983) assume the significance of top-down mechanisms, for instance, to trigger the recollection of stored memories (Engel et al., 2001). However, the bottom-up viewpoint implies unconscious processing and suggests that perceptual processes are ‘stimulus’ or ‘data’ driven by incoming data received from sense organs (Rollinson, 2008; Styles, 2005).

What is Attention?

Attention is considered a necessary process to facilitate perception (Bodenhausen & Hugenberg, 2009; Styles, 2005). James (1890) a 19th-century psychologist and philosopher, defined attention as follows:

“Every one knows what attention is. It is the taking possession by the mind, in clear and vivid form, of one out of what seem several simultaneously possible objects or trains of thought. Focalization, concentration, of consciousness are of its essence. It implies withdrawal from some things in order to deal effectively with others” (p. 403).

In essence, James (1890) suggests that attention is a conscious process, which is inherently selective, whereby irrelevant details are filtered, to interpret and analyze only one item at a time. Theories of selective attention, e.g., Broadbent’s filter theory (Broadbent, 1958) and Treisman’s attenuation theory (A. Treisman, 1964) are often classified as filter theories, whereby certain stimuli are attended to while irrelevant stimuli are eliminated or attenuated. This assumption implies that the human perceptual system is unable to simultaneously process all incoming stimuli due to its limited capacity channels (Pashler & Sutherland, 1998).

However, Neisser (1967) proposed a counter-theory to describe the selectivity of attention through his analysis-by-synthesis theory. He believed that attention is functionally divided between two processes: (i) pre-attentive and (ii) focal-attentive processing (Neisser, 1967).

What is Pre-attentive Processing? 

Pre-attentive processing occurs in parallel across the visual field (A. M. Treisman & Gelade, 1980) i.e., the brain’s ability to process several stimuli at once. According to Wyer (2014), it is characterized by salient features such as:

• rapid,
• effortless,
• unconscious, and
• automatic.

In essence, pre-attentive processes perform preliminary sorting and organization of sensory data for further analysis by focal-attentive processes. Also, it facilitates the interpretation of physical features such as colour, orientation, brightness, the direction of movement, and controls the perceptual grouping of stimuli (A. M. Treisman & Gelade, 1980). Furthermore, this construct underpins and supports gestalt principles (Banerjee, 1994) such as figure and ground, similarity, closure, symmetry, etc., which enable visual perception in pattern recognition scenarios. From a design standpoint, Ware (2004) pointed out that pre-attentive attributes such as colour, form, spatial position, speed, and motion are relevant to trigger perception at a glance.

Focal Attentive Processing 

In focal-attentive processing - the movement of information occurs serially (A. M. Treisman & Gelade, 1980) i.e., attention can solely be attuned to one task at a time. It is considered to be 

• slow,
• controlled,
• and comes after pre-attentive processing.

Divided Attention

Divided attention theories examine the notion of multi-tasking in which multiple attentional tasks are executed in parallel. However, Kahneman (1973) in his theory of divided attention, implies that only a limited number of mental resources are available for distribution across different tasks, so the number of activities that can be executed concurrently is constrained. Thus, in multi-tasking activities, mental resources are allocated based on the degree of complexity of the task.

To gain a deeper understanding of divided attention, both controlled and automatic information processing (Shiffrin & Schneider, 1977) must be taken into account. As previously examined, controlled processes require conscious awareness and can only be executed one item at a time while automatic processes are fast, unconscious, and can be executed in parallel. Spaulding (1994) argues that automatic processes develop with time and practice; thus preventing cognitive overload by reducing the amount of mental effort required. This is reflected in the following example. 

In controlled activities such as when we first learn to ride a bicycle, the activity of riding demands conscious awareness and a substantial amount of mental effort. Basically, one has to concentrate on learning how to balance, pedal, brake, stop, switch gears, etc. Thus, these activities cannot be performed in parallel with others. However, with time, practice, and experience riding becomes automated or habituated (requiring less mental effort). Consequently, it can be executed in parallel with other activities such as steering, enjoying the environment, and reacting to anomalies in vehicular traffic.

Revisiting the theory of divided attention, Kahneman (1973) believed that the allocation of mental effort is based on the following four factors.

• Involuntary action (e.g., the allocation of mental resources to unusual signals or abrupt changes in movement).
• Momentary intentions (e.g., like deciding the switch the channel from a cooking show in order to watch the news).
• Evaluation of demands (e.g., whenever the request from two activities exceeds more resources than available, then only one is executed).
• Effects of arousal (heightened arousal will more likely increase attention to controlled processes, (e.g., being highly engaged in watching an exciting motion picture).

Basically, by exploring the theory of divided attention, it is apparent that we can multitask to a certain extent. In the case of mediated environments, one means by which you could improve social connectedness is to support perception at a glance. Fundamentally, this is achieved through peripheral interaction, which according to Hausen (2014), is aimed at “divided attention and thus multitasking but with the specific goal of moving tasks to the periphery of attention (visual and cognitive)” (p. 22). Furthermore, Buxton (1995) underscores the importance of the periphery in supporting awareness and communication in the field of computer-mediated environments. More specifically, Buxton (1995) differentiates between foreground and background interactions such that:

• Foreground activities – are ‘intentional’ or ‘pre-mediated’, requiring consciousness, much focus, and concentration, e.g., having a conversation on a mobile phone.
• Background activities – are undertaken in the periphery of consciousness, e.g., being aware of someone cooking in the kitchen while one reads this dissertation in another room.

Essentially, Buxton’s notion of the background is invaluable for enabling peripheral interaction in mediated environments. Primarily, through the filter mechanisms embodied in selective attention, it is feasible to maintain awareness of information on the periphery of one’s attention (Vogel, 2005). This is achieved through habituation (i.e., repeated exposure to a stimulus generates reduced reactions, which eventually fades into the background) and dishabituation (speedy recovery of a response, which was previously debilitated by habituation) (J. A. Gray, 1975; Vogel, 2005). However, Vogel (2005) argues that certain circumstances can cause sudden or unexpected changes to background stimuli; thus, triggering dishabituation. Following the preliminary operations on background stimuli, if a stimulus is detected as highly relevant, then it is selected for complete attentive processing.

Thus, supporting the seamless transition between the foreground and background attentional processes, which is a primary goal of peripheral interaction. Moreover, since, peripheral interaction aspires to reduce the user’s cognitive mental effort, then habituation could be useful in peripheral interactions (Hausen, 2014). 

Memory

Cognitive psychologist, Nathan Spreng implies that memory is the cornerstone of social behavioural processes (Spreng, 2013). Specifically, Spreng (2013) outlines how memory is utilized within a social context in the statement below. “In navigating the social world, we must often retrieve, maintain, manipulate, and update the information we have about other people” (p. 1). Generally speaking, memory involves three main processes where information is: (i) encoded, (ii) stored, and (iii) retrieved (Atkinson & Shiffrin, 1968). It is further categorized into three sub-components, which are summarized by (Styles, 2005) below.

1. Sensory memory – This is the shortest form of memory and is determined by the ability to retain traces of a stimulus even when it has ended. When visual traces are retained, this is known as iconic memory, e.g., when you still see the camera flashing even though it is turned off.
2. Short-term/working memory – acts as the temporary storage of sensory information, which has already been processed for recall from long-term processing, is limited in processing capacity and requires conscious awareness.
3. Long-term memory – This is an information storehouse for prolonged periods of time. It further decomposed into the following types of memory:

• explicit memory – requires conscious effort to enable recollection,
• implicit memory – is automatic and unintentional, i.e., does not require conscious awareness for task execution,
• declarative memory – facilitates the recollection of facts or events – requires conscious awareness,
• procedural memory – enables recollection for demonstration of proficiency and task execution activities – is an unconscious process,
• episodic memory – ability to recall personal experiences, self-identity, and life history requires conscious awareness, and
• semantic memory – ability to recall facts – requires conscious awareness.

Summary

Overall, this article considers the relevance of visual perception, attention, and memory to decode information received from the outer world through peripheral technology. This is achieved by filter mechanisms, which enable divided attention and support interaction in the attentional periphery. Furthermore, Kahneman (1973) implies that attention and arousal are tightly coupled. Therefore, to support awareness and enhance the sense of belongingness in mediated environments, there is a need for a transition medium (e.g., ambient lighting and visual storytelling)  that can propagate visual signals through abstraction, while evoking a sense of connectivity, causing minimal distraction and support peripheral interaction. 

Further reading: 

Davis, K. A. (2017). Social hue: a bidirectional human activity-based system for improving social connectedness between the elderly and their caregivers. Technische Universiteit Eindhoven.

References 

1. Bodenhausen, G. V. & Hugenberg, K. (2009). Attention, perception, and social cognition. Social cognition: The basis of human interaction, 1–22.
2. Engel, A. K., Fries, P. & Singer, W. (2001). Dynamic predictions: oscillations and synchrony in top–down processing. Nature Reviews Neuroscience, 2(10), 704–716.
3. Matlin, M. W. (2005). Cognition, 6th edition. John Wiley & Sons.
4. Gray, J. A. (1975). Elements of a two-process theory of learning. Academic Press.
5. Gregory, R. L. (1970). The intelligent eye. ERIC.
6. Hausen, D. (2014). Peripheral interaction: exploring the design space (Unpublished doctoral dissertation). München, Ludwig-Maximilians-Universität, Diss., 2014.
7. Henle, M. (1989). Some new gestalt psychologies. Psychological Research, 51(2), 81–85.
8. Ishii, H., Wisneski, C., Brave, S., Dahley, A., Gorbet, M., Ullmer, B. & Yarin, P. (1998). ambientroom: Integrating ambient media with architectural space. In Chi 98 conference summary on human factors in computing systems (pp. 173–174). New York, NY, USA: ACM.
9. James, W. (1890). The principles of psychology new york. Holt and company.
10. Kahneman, D. (1973). Attention and effort. Prentice-Hall.
11. Matthews, T., Dey, A. K., Mankoff, J., Carter, S. & Rattenbury, T. (2004). A toolkit for managing user attention in peripheral displays. In Proceedings of the 17th annual acm symposium on user interface software and technology (pp. 247–256). New York, NY, USA: ACM.
12. Neisser, U. (1967). Cognitive psychology. Appleton-Century-Crofts.
13. Pashler, H. E. & Sutherland, S. (1998). The psychology of attention (Vol. 15). MIT press Cambridge, MA.
14. Rock, I. (1983). The logic of perception. MIT Press
15. Rollinson, D. (2008). Organisational behaviour and analysis: an integrated approach. Pearson Education.
16. Shiffrin, R. M. & Schneider, W. (1977). Controlled and automatic human information processing: Ii. perceptual learning, automatic attending and a general theory. Psychological review, 84(2), 127.
17. Spaulding, W. D. (1994). Cognitive technology in psychiatric rehabilitation. University of Nebraska Press. Spreng, R. N. (2013). Examining the role of memory in social cognition. Frontiers in psychology, 4, 437.
18. Styles, E. A. (2005). Attention, perception and memory: an integrated introduction. Routledge.
19. Treisman, A. (1964). Monitoring and storage of irrelevant messages in selective attention. Journal of Verbal Learning and Verbal Behavior, 3(6), 449–459.
20. Treisman, A. M. & Gelade, G. (1980). A feature-integration theory of attention. Cognitive psychology, 12(1), 97–136.
21. Vogel, D. J. (2005). Interactive public ambient displays (Unpublished doctoral dissertation). University of Toronto.
22. Wyer, R. S. (2014). The automaticity of everyday life: Advances in social cognition (Vol. 10). Psychology Press.