Quantifying the User Experience: Practical Statistics for User Research, Second Edition, provides practitioners and researchers with the information they need to confidently quantify, qualify, and justify their data. The book presents a practical guide on how to use statistics to solve common quantitative problems that arise in user research. It addresses questions users face every day, including, Is the current product more usable than our competition? Can we be sure at least 70% of users can complete the task on their first attempt? How long will it take users to purchase products on the website?
This book provides a foundation for statistical theories and the best practices needed to apply them. The authors draw on decades of statistical literature from human factors, industrial engineering, and psychology, as well as their own published research, providing both concrete solutions (Excel formulas and links to their own web-calculators), along with an engaging discussion on the statistical reasons why tests work and how to effectively communicate results. Throughout this new edition, users will find updates on standardized usability questionnaires, a new chapter on general linear modeling (correlation, regression, and analysis of variance), with updated examples and case studies throughout.
1. Chapter 1: Introduction and how to use this book - p. xxi 2. Chapter 2: Quantifying user research - p. 8 3. Chapter 3: How precise are our estimates? Confidence intervals - p. 18 4. Chapter 4: Did we meet or exceed our goal? - p. 38 5. Chapter 5: Is there a statistical difference between designs? - p. 60 6. Abstract - p. 60 7. Introduction - p. 61 8. Comparing two means (rating scales and task times) - p. 61 9. Comparing completion rates, conversion rates, and A/B testing - p. 73 10. Chapter review questions - p. 95 11. Answers to chapter review questions - p. 97 12. Chapter 6: What sample sizes do we need? Part 1: summative studies - p. 102 13. Abstract - p. 102 14. Introduction - p. 103 15. Estimating values - p. 106 16. Comparing values - p. 112 17. What can I do to control variability? - p. 118 18. Sample size estimation for binomial confidence intervals - p. 119 19. Sample size estimation for chi-squared tests (independent proportions) - p. 125 20. Sample size estimation for McNemar Exact Tests (matched proportions) - p. 128 21. Chapter review questions - p. 134 22. Answers to chapter review questions - p. 135 23. Chapter 7: What sample sizes do we need? Part 2: formative studies - p. 141 24. Abstract - p. 141 25. Introduction - p. 143 26. Using a probabilistic model of problem discovery to estimate sample sizes for formative user research - p. 143 27. Assumptions of the binomial probability model - p. 148 28. Additional applications of the model - p. 149 29. What affects the value of p? - p. 157 30. What is a reasonable problem discovery goal? - p. 157 31. Reconciling the “Magic Number Five” with “Eight is Not Enough” - p. 160 32. More about the binomial probability formula and its small-sample adjustment - p. 167 33. Other statistical models for problem discovery - p. 171 34. Chapter review questions - p. 179 35. Answers to chapter review questions - p. 179 36. Chapter 8: Standardized usability questionnaires - p. 183 37. Abstract - p. 183 38. Introduction - p. 185 39. Post-study questionnaires - p. 189 40. Post-task questionnaires - p. 216 41. Questionnaires for assessing perceived usability of websites - p. 225 42. Other questionnaires of interest - p. 231 43. Chapter review questions - p. 239 44. Answers to chapter review questions - p. 240 45. Chapter 9: Six enduring controversies in measurement and statistics - p. 248 46. Chapter 10: An introduction to correlation, regression, and ANOVA - p. 276 47. Abstract - p. 276 48. Introduction - p. 277 49. Correlation - p. 278 50. Coefficient of determination (R2) - p. 284 51. Regression - p. 288 52. Analysis of variance - p. 295 53. Chapter review questions - p. 308 54. Answers to chapter review questions - p. 308 55. Appendix: derivation of sample size formulas for regression - p. 319 56. Chapter 11: Wrapping up - p. 320 57. Appendix: A crash course in fundamental statistical concepts - p. 325 58. Subject Index - p. 343
Quantifying the User Experience: Practical Statistics for User Research, Second Edition, provides practitioners and researchers with the information they need to confidently quantify, qualify, and justify their data. The book presents a practical guide on how to use statistics to solve common quantitative problems that arise in user research. It addresses questions users face every day, including, Is the current product more usable than our competition? Can we be sure at least 70% of users can complete the task on their first attempt? How long will it take users to purchase products on the website?
This book provides a foundation for statistical theories and the best practices needed to apply them. The authors draw on decades of statistical literature from human factors, industrial engineering, and psychology, as well as their own published research, providing both concrete solutions (Excel formulas and links to their own web-calculators), along with an engaging discussion on the statistical reasons why tests work and how to effectively communicate results. Throughout this new edition, users will find updates on standardized usability questionnaires, a new chapter on general linear modeling (correlation, regression, and analysis of variance), with updated examples and case studies throughout.
1. Chapter 1: Introduction and how to use this book - p. xxi 2. Chapter 2: Quantifying user research - p. 8 3. Chapter 3: How precise are our estimates? Confidence intervals - p. 18 4. Chapter 4: Did we meet or exceed our goal? - p. 38 5. Chapter 5: Is there a statistical difference between designs? - p. 60 6. Abstract - p. 60 7. Introduction - p. 61 8. Comparing two means (rating scales and task times) - p. 61 9. Comparing completion rates, conversion rates, and A/B testing - p. 73 10. Chapter review questions - p. 95 11. Answers to chapter review questions - p. 97 12. Chapter 6: What sample sizes do we need? Part 1: summative studies - p. 102 13. Abstract - p. 102 14. Introduction - p. 103 15. Estimating values - p. 106 16. Comparing values - p. 112 17. What can I do to control variability? - p. 118 18. Sample size estimation for binomial confidence intervals - p. 119 19. Sample size estimation for chi-squared tests (independent proportions) - p. 125 20. Sample size estimation for McNemar Exact Tests (matched proportions) - p. 128 21. Chapter review questions - p. 134 22. Answers to chapter review questions - p. 135 23. Chapter 7: What sample sizes do we need? Part 2: formative studies - p. 141 24. Abstract - p. 141 25. Introduction - p. 143 26. Using a probabilistic model of problem discovery to estimate sample sizes for formative user research - p. 143 27. Assumptions of the binomial probability model - p. 148 28. Additional applications of the model - p. 149 29. What affects the value of p? - p. 157 30. What is a reasonable problem discovery goal? - p. 157 31. Reconciling the “Magic Number Five” with “Eight is Not Enough” - p. 160 32. More about the binomial probability formula and its small-sample adjustment - p. 167 33. Other statistical models for problem discovery - p. 171 34. Chapter review questions - p. 179 35. Answers to chapter review questions - p. 179 36. Chapter 8: Standardized usability questionnaires - p. 183 37. Abstract - p. 183 38. Introduction - p. 185 39. Post-study questionnaires - p. 189 40. Post-task questionnaires - p. 216 41. Questionnaires for assessing perceived usability of websites - p. 225 42. Other questionnaires of interest - p. 231 43. Chapter review questions - p. 239 44. Answers to chapter review questions - p. 240 45. Chapter 9: Six enduring controversies in measurement and statistics - p. 248 46. Chapter 10: An introduction to correlation, regression, and ANOVA - p. 276 47. Abstract - p. 276 48. Introduction - p. 277 49. Correlation - p. 278 50. Coefficient of determination (R2) - p. 284 51. Regression - p. 288 52. Analysis of variance - p. 295 53. Chapter review questions - p. 308 54. Answers to chapter review questions - p. 308 55. Appendix: derivation of sample size formulas for regression - p. 319 56. Chapter 11: Wrapping up - p. 320 57. Appendix: A crash course in fundamental statistical concepts - p. 325 58. Subject Index - p. 343
