KN '09 Breakfast Forum

Attaining Accuracy, Maximum Coverage and Representation in Web-based Research

KN's current Web-based panel methodological research was recently discussed with about 100 clients in the Boston and Washington DC area. In the spirit of transparency, we've created this page of resources, based on these fora.

Key Take-aways

Summary of Knowledge Networks methodological research: Presentation and topics

Yeager/Krosnick Study

A new peer-reviewed study by Stanford University researchers David Yeager, Prof. Jon Krosnick, and their co-authors provides an extensive comparison of seven non-probability Internet survey panels with two other survey platforms based instead on probability sampling. One is based on KnowledgePanel® from Knowledge Networks; the other probability-based solution uses random-digit-dialing telephone methodology. The authors concluded that the non-probability Internet surveys were less accurate, and customary weighting adjustments did not uniformly improve them.

Jon Krosnick Forum Presentation – Full Video (39:11)
Windows Streaming Video

Presentation Clips - Windows Streaming Video

1. Moving from Telephone to Internet (00:41)
2. Indicator of validity #1: LinChiat Chang's experiment: Mode comparison with sample held constant/latent variable analysis (02:29)
3. Indicator of validity #2: Systematic measurement error: Response order effects (01:53)
4. RDD recruitment with Internet survey administration: Description of new analysis (03:19)
5. Weigh constructs and why they matter/unweighted comparisons among panel types (03:17)
6. Internet survey accuracy/response rates (01:51)
7. Assessing validity (Telephone vs. Internet results) (00:43)
8. Estimating reliability via latent variable structural equation modeling (01:39)
9. Internet data collection: Probability versus opt-in sample (results) (07:15)
10. Has anything improved since 2004? (01:02)

• Report on the Study - .pdf
  
• Online Supplement - .pdf
  
• Krosnick/Yeager/Javitz respond to questions about their study – ABCNews.com

Other Resources

• The Decision Maker's Guide to Online Research™, 2009-10
  
• Accuracy's Impact on Research – Fall 2009

Photo: (c) Jannyjus-Dreamstime.com