Download PDF file

The Impact of Media Literacy
Instruction on Students’ Academic Performance

Renee Hobbs and Richard Frost

Babson College

Wellesley, MA USA

The Importance of Measuring Impact of ML

  • Develop instruments to measure media analysis skills
  • Determine relationship between ML instruction and academic performance
  • Examine relationship between ML instruction and attitudes and behaviors
  • Provide feedback to teachers about students’ skill development over time

Using Quantitative Measures of Student Performance

  • Permits generalization about a large sample
  • Demonstrates competence in terms that are familiar to education leaders
  • Useful to test specific research hypotheses
  • Helps identify patterns and relationships between skills, attitudes and behaviors

Context for the Research

  • 1997 English department at Concord H.S. reorganizes curriculum to accommodate Grade 9
  • 1998 School Committee approves a year-long English course for all Grade 11 students called “Communications/Media”
  • 1998 Three teachers attend “Teaching the Humanities in a Media Age” conference at Clark University

Grade 11 Curriculum

  • Analysis focus: five critical questions used as a framework
  • Non-fiction reading
    • Brill’s Content
    • Team Rodent
    • News is a Verb
    • Glued to the Set
  • Fiction reading
    • One Flew over the Cuckoo’s Nest
    • Frankenstein
    • Beloved
    • As I Lay Dying
  • Media reading
    • Film adaptations of books
    • TV newsmagazines
    • Newspapers
    • Magazines
    • Print and TV advertising
  • Media production activities
    • News articles
    • Press releases
    • Posters
    • Ad campaigns
    • Film adaptations and scriptwriting
    • Video production of ads and film scenes

The Concord English 11 Faculty

  • Senior faculty leadership and support within the department
  • Collaborative approach included sharing of resource materials and lesson plans
  • First year: a time of adjustment
    • unpredictability of classroom activity
    • deciding what to emphasize and omit
    • discovering what works best

Research Hypotheses

1. ML instruction improves media analysis skills. 

Operational definition: pre-post measures of the ability to identify:

  • Purpose
  • target audience
  • construction techniques
  • point of view/subtext
  • omissions

2. ML instruction improves literacy skills. 

Operational definition: pre-post measures of the ability to communicate effectively in writing:

  • Quality of writing (holistic scale)
  • Quantity of writing (word count)
  • Errors (spelling and usage error count)
  • Coherence-incoherence
  • Count of main ideas identified
  • Count of specific information used in writing

3. ML instruction leads to to a shift in viewing motivations, greater knowledge about the mass media, increased awareness of economic motives in media, increased sense of personal efficacy and decreased parasocial attachment. 

Operationalized using multiple choice test, likert-scale survey questionnaire and focus groups.

4. ML instruction leads to greater use of non-fiction media, greater use of print media, and decreased use of teen-focused programming. 

Operationalized using survey questionnaire and focus groups.

5. ML instruction leads to greater trust between student and teacher, increased perceptions of closeness, and decreased perceptions of authority and competence.

Operationalized using survey questionnaire and focus groups.

Research Methodology

  • Pre-post design (repeated measures)
  • Survey questionnaire used to measure attitudes, knowledge of the mass media, and media use
  • Video stimuli with open-ended questions used to measure knowledge of media production
  • Teacher-student relationship (Purdue Teacher Evaluation Scale)
  • Measures of media analysis used multimedia stimuli:
  • Time magazine article (critical reading)
  • National Public Radio (listening skills)
  • Beer print ad (advertising analysis)
  • TV news (critical viewing)

Characteristics of the Sample

  • N = 287
  • All Grade 11 students including special needs students mainstreamed into regular ed program
  • Diverse social class mix

Research Hypotheses

ML instruction improves media analysis skills.

1. Who is the target audience for this message? (check all that apply)

____ 2-11 year olds 

____ 12-17 year olds  

____ 18-25 year olds

____ 25-40 year olds

____ 40-60 year olds

____ 60 + year olds

____ whites   

____ blacks 

____ hispanics  

____ asians   

____ other  

 

____ men

____ women

 

____ poor people

____ working class

____ middle class

____ upper middle class

____ wealthy people

 

What visual information in the ad supports your answer?

2. What techniques are used to attract your attention?

3. What is the purpose of the message?

4. What is the subtext of the message?  What values are embodied in this message?

5. Comment on the meaning of the small logo,  “Think When You Drink.”

    Ad Analysis Results

    Interpreting Ad Subtext

    Analyzing TV News

    • Main ideas and supporting facts
    • Purpose
    • Target audience
    • Construction techniques
    • Point of view/subtext
    • Omissions
    • Compare/contrast to other TV news formats

    Analyzing Purpose of TV News

    News Analysis Results

    • Graph plus
    • Identifies omitted information
    • Identifies construction techniques
    • Identifies differences between stimulus and TV news
    • Identifies similarities

    Research Hypotheses

    ML instruction improves literacy skills.

    Literacy Results

    • Graph plus
    • Ability to identify omitted information
    • Ability to identify construction techniques
    • Usage errors

    Literacy Results

    Word Count (viewing)               Pre = 12.76  Post = 14.62          p. < .001

    Holistic writing (listening)         Pre = 2.61 Post = 2.77              p. < .0001

    Research Hypotheses

    3. ML instruction leads to a shift in viewing motivations, greater knowledge about the mass media, increased awareness of economic motives in media, increased sense of personal efficacy and decreased parasocial attachment.

    Viewing Motivations

    • Social facilitation
    • Habit/pass time
    • Mood altering / entertainment
    • Companionship
    • Escape / avoidance
    • Learning / information acquisition

    Habit & Social Facilitation

    • “I enjoy watching TV to get a break from some of the pressures in my life.”
    • “I often watch TV to pass the time”.
    • “When there’s nothing better to do, I watch TV.”
    • “I like watching TV  shows that I know my friends will want to talk about.”

     

    Viewing Motivation Results

    Parasocial Attachment

    “I think my favorite TV personality is like an old friend.”

    “My favorite TV personality makes me feel comfortable, as if I am with friends.”

    “If my favorite TV personality appeared on another program, I would watch that program.”

    “I like to compare my ideas with what my favorite TV personality says.”

    Parasocial Attachment Results

    Media Dependence                                                                                       “It would be practically impossible for me to go a whole week without watching TV.”

    Knowledge of Media Economics

    Economic control over media usually comes from:

    • Government
    • Public
    • Advertisers
    • Networks

    PRE = 34% POST = 66%

    P = .0001

      How much of a newspaper is comprised of advertising?

      • 25%
      • 40%
      • 60%
      • 85%

      PRE = 17% POST = 51%

      P = .0001

      Magazines usually make most of their money through subscription income. 

      • true
      • false

      PRE = 55% POST = 71%

      P = .0001

        Typically, articles in a magazine are written by a staff that works exclusively for that magazine.

        • true
        • false

        PRE = 39% POST = 55%

        P = .0001

          The government agency responsible for granting broadcasters licenses is:

          • National Association of Broadcasters
          • Federal Communications Commission
          • Communications Industry Commission
          • Federal Trade Commission

          PRE = 49% POST = 56%

          P = .05

            Knowledge of Media Production Processes

            Measured by showing ad stimulus and asking students to list all the steps involved in constructing the message.

            Pre-Production Results

            • Target audience
            • Planning
            • Firm selects a concept
            • Storyboard

            Post-Production Results

            • Company approval
            • Media placement
            • Measure effect on sales

            Research Hypotheses

            4.   ML instruction leads to greater use of non-fiction media, greater use of print media, and decreased use of teen-focused programming. 

             

            Media Products in the Home

            TVs                        2.9            3.2

            VHS                        1.9             2.1

            Computer               .91             .90

            Internet                   .71            .71

            Cable                      .88             .92

            DBS                        .06             .06

            Media Products in the Home

            • Newspaper  .69   .68
            • Sunday .75  .71
            • Magazine .89  .88
            • # of Mags  2.43  2.69
            • Increased product purchase?
            • Increased awareness?

            TV Viewing Behaviors                                                       “How often do you use…” 6 pt. scale

            Media Use Behaviors
            “How often do you use…” 6 pt. scale

            Favorite Types of Programs
            “how often do you watch…” 10 pt scale

            • Crime dramas
            • Dramas  
            • Science fiction 
            • Game shows
            • Cartoons
            • Local news
            • History/science 
            • National news
            • Newsmagazines
            • Comedies
            • Situation comedies
            • Morning shows
            • Sports
            • Music video
            • Shopping / infomercials
            • Late nite shows

            Favorite Types of Programs

            Explanation of Results

            • Educational intervention
            • Maturation

            Renee Hobbs and Richard Frost History and Society Division Babson College                           One College Drive              Wellesley, MA 02457                  (781) 239-4975

            Email: [email protected]