Lecture 8

The Representation and Communication of Auditory and Visual Information

April 7, 2004



Psychophysics -- Relating physical stimuli to perception References

Sound and hearing

The University of Colorado site Audio Processing says all there is know about the subject in a two minute capsule.

Loudness ("prothetic" attribute reference):

SPL (Sound Power Levels) dB Sound Power Levels

The Fletcher-Munson equal loudness curve:

To quote from Music and the Human Ear:

"SPL is an objective measurement of sound pressure, or power in watts, and is independent of frequency.  In 1933 Fletcher and Munson of Bell Labs did a study that showed that subjective sound levels varied significantly from the SPL level.  That is, when two tones were played at the
exactly the same SPL level, one sounded louder than the other.  And the results were very dependent on how loud the tones were to begin with. This is illustrated by the set of Fletcher-Munson curves. The vertical axis is the objective SPL sound level. Each of the curves in the graph represents a constant subjective sound level, which are in units called "phones." The lowest curve is the minimum audible level of sound. As noted above, the ear is most sensitive around 2-5 kHz.  To be audible at this minimum level, a sound at 20Hz must be 80 dB (100 million times!) more powerful than a sound at 3 kHz.

"Near the top, the curve at 100 phones is a fairly loud level. To sound equally loud at this level the sound at 20 Hz must be about 40 dB more powerful.  This change in subjective level for different loudness levels means that music played softly will seem to be lacking in bass. For years pre-amps have come equipped with "loudness" controls to compensate for this. …"

Pitch ("metathetic" attribute reference):
See theories of Pitch Perception from the authoritative notes from the course on Sensation and Perception: Hearing at the University of Sussex.

See also our dicussion of Critical Bands and  Psychacoustics: Critical Bands etc.

Light and seeing

The distinction between light and seeing has taken a long time to gel.  See Light and Seeing: Evolution of the Concept
Brightness ("prothetic" attribute):
Brightness, Luminance, and Confusion: a March 1993 article by Charles P. Halsted in Information Displaywhich introduces and defines the "Six Light Words."
  • Light
  • Luminous Flux
  • Luminous Intensity
  • Luminance
  • Brightness
  • Illuminance
The following two quotes may extend the confusion:
"The concept that is now known as 'luminance' was for many years designated by the term 'brightness.'  This led to much confusion between the objective concept of 'brightness' as intensity per unit of projected area, and the subjective concept of 'brightness' which referred to a sensation in the consciousness of a human observer. The newer term). 'luminance' was adopted to avoid this confusion. - from Optics by Francis Weston Sears (Addison-Wesley, 1949)

 
"Brightness (a term of photometry): The subjective description of luminance.  A perceived characteristics of objects which does not vary directly in a simple mathematical relationship with their physical or measured brightness, which is corretly termed their luminance. Brightness is often misused to mean luminance. If so, it should be qualified as 'measured' brightness, to distinguish between the absolute attributes ('measured' brightness) and the perceived attributes ('apparent' or 'subjective' brightness) of the object in question."
 

Luminance, Illuminance, Lumens, Lux, . . .

In case you even need to use it, here is Luminance: Unit Conversion

Visual Acuity:
See Visual Acuity from Webvision: The Organization of the Retina and Visual System: from the University of Utah
Color ("metathetic" attribute):
Color Science: a collection of pictures and links relating to color perception -- from the Department of Physics and Astronomy, Austin State University.  See in particular the material on incandescence or "backbody" radiation -- for example.

Additive mixing

The photopic luminosity function:

The "brightness" characteristics of visual perception.

Early Attempts at Communicating Visual Information:

Ancient telegraphic ( distant writing ) systems

Analog Representations and Communication of Visual Information:

Picture or pixel scanning: a fundamental communication concept which is often attributed to  Alexander Bain (1811-1877)
The picture scanning concept

In 1883 Paul Nipkow (1860-1940) envisaged and demonstrated a complete scanning system.

A schematic of the Nipkow's Disk system (source)
An early AT&T facsimile system based on Bain's scanning concept.
Broadcast Analog TV: References
Worldwide  TV Standards

TV Systems: A Comparison

Details of NTSC

Black and white signals:
Horizontal/Vertical scan
Frame Form
Luminance signal
Spectrum
Black and White Transmitter
Black and White Receiver
Composite color signals:
Discussion of NTSC color system
Spatial harmonics:  Picture representation in terms of spatial harmonics is a fundamentally different communication concept which is the basis of many image compression schemes.
Examples of the nonlocalized spatial harmonic view point
Examples of a localized spatial harmonic view point

Gabor Function References:

Gabor Functions
Using Gabor functions to make atmosphere in computer graphics
Gabor Model
2-D Gabor Function - Interactive Visualization
l

 

Psychophysics References:

A wonderful set of web resources on "light and seeing; sound and hearing" collected and created by George Mather and linked to the Perception Home Page.  See in particular:
The Joy of Visual Perception: a web book on perception by Peter K. Kaiser of the Department of Psychology, York University.
 
Visual Physiology: an interactive tutorial by George Mather of the School of Biological Sciences, University of Sussex.
Lance Hahn's Retina Reference

Psychology 280 Class Notes from the University of Toronto

A Computer Model of the Auditory Periphery

Television References:

The Revolution in Television Technology: a exposition of new television technologies -- from the Department of Physics and Astronomy, University of Delaware.

A Guide to Digital Terrrestrial Television Broadcasting (DTTB): an extensive report of a International Telecommunications Union (ITU) study group.

The Evolution of Broadcast TV Technology

Basic Video: a useful basic reference by Roger Inman & Greg Smith from the 1996 Television Production Handbook
 
 

This page was prepared and is maintained by R. Victor Jones
Comments to: jones@deas.harvard.edu.

Last updated April 14, 2004