HighMissing

Review – Green Living

Introduction.

I started working with Tableau Desktop just after Stephen and Eileen McDaniel gave a presentation to the Seattle Tableau User’s group (March, 2013).  On Tableau’s community site they named Eileen’s Green Living Dashboard as a good example of visual data presentation.  This sounded like a good initial visualization for me to work with.   I started by doing a quick review.  It turned up several problems, which triggered a more detailed review.  Eventually, the quantity and severity of problems suggested that the visualization should never have been created. The dashboard did turn out to be a great example  ̶  one which offers reviewers many opportunities to discover problems.   The dashboard is available at the McDaniel’s site.

Summary of Review

Story

The dashboard presents a model for improving programs which promote green activities. It suggests working with experts to:

  • list important green activities which are appropriate for the locality,
  • develop a survey which measures the frequency of those activities,
  • classify the activities into meaningful categories,
  • administer the survey,
  • and suggest best practices for local sustainability programs based on the results.

The model presents 21 green activities which are pooled into these three categories: save money, easy to do, and beneficial to the environment.  The results found that “easy to do” activities were done most often, those that “save money” next and those “most beneficial to the environment” least often.  These stated outcomes are no surprise and of little interest.  However, the methodology could be quite interesting.  How were the 21 items ranked and placed into the three categories.  Did each item have a save money score in dollars and cents, an easy to do score  in minutes per day, and an environmental score in yearly pounds of carbon saved?  Unfortunately none of this information is available on the site or from the author.  We are left with So What?.

Survey

The survey method (phone, personal interview, mail, or on-line) and the response rate are not reported or available.

The respondents’ education levels are significantly higher than state norms.

The respondents’ distribution by state is significantly different from census data.

Measurement – Likert scale

The five point scale for the Likert items is poorly constructed.

There is no information on what Likert items were assigned to each Likert scale.

Measurement – Clarity of Questions

The wording of the questions is often ambiguous.

Several of the questions do not apply to all respondents.

Missing data

Analysis of missing data suggests that some of the questions were not applicable for some respondents.   There were numerous questions which were left blank (i.e. missing data) but the three pooled categories: save money, easy to do, and beneficial to the environment were calculated for each respondent.  The process used to handle missing values for pooled categories is not specified.

Transformation

The study is based on aggregated data. Unfortunately the aggregation loses all individual data and renders the dataset useless. The original data are not available.

Transparency

The original data and key information about the survey (method, response rate), the questions (what items went to which scales) and how missing values contributed to the three summary categories are not presented on the site.  Attempts to get additional information resulted in an email stating that they were too busy to respond to questions.

Communicate

There are several minor issues on the dashboard.

  • The average response is displayed with different resolution in each segment (the map shows integer values, the chart one decimal place, and table two decimal places).
  • The map and table show “Count of Persons” while the chart shows “Count of Responses”.
  • The dashboard header uses “Green Lives” while the blog and link use “Green Living”.

These issues foreshadow the lack of rigor apparent in all categories reviewed.  The review found enough serious problems to invalidate the data and the methods.  Therefore and extended review of Communication is not required.

Categories used in this review (see General topics and Special topics in Tools>Checklists) : 
Key: Red=Summarized here and detailed at link, Black=This page only, Strike through=Not reviewed
General: Argument, Communicate, Comparisons, Measurement, Research, Review, Statistics, Story, Transparency 
Special: Causality, Missing, Survey, Time-series, Transformations 

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>