This book considers classical and current theory and practice, of supervised, unsupervised and semi-supervised pattern recognition, to build a complete background for professionals and students of engineering. The authors, leading experts in the field of pattern recognition, have provided an up-to-date, self-contained volume encapsulating this wide spectrum of information. The very latest methods are incorporated in this edition: semi-supervised learning, combining clustering algorithms, and relevance feedback.
· Thoroughly developed to include many more worked examples to give greater understanding of the various methods and techniques
· Many more diagrams included--now in two color--to provide greater insight through visual presentation
· Matlab code of the most common methods are given at the end of each chapter.
· More Matlab code is available, together with an accompanying manual, via this site
· Latest hot topics included to further the reference value of the text including non-linear dimensionality reduction techniques, relevance feedback, semi-supervised learning, spectral clustering, combining clustering algorithms.
· An accompanying book with Matlab code of the most common methods and algorithms in the book, together with a descriptive summary, and solved examples including real-life data sets in imaging, and audio recognition. The companion book will be available separately or at a special packaged price (ISBN: 9780123744869).
1. Chapter 1: Introduction - p. xvii 2. Chapter 2: Classifiers Based on Bayes Decision Theory - p. 13 3. 2.1 Introduction - p. 13 4. 2.2 Bayes decision theory - p. 13 5. 2.3 Discriminant functions and decision surfaces - p. 19 6. 2.4 Bayesian classification for normal distributions - p. 20 7. 2.5 Estimation of unknown probability density functions - p. 34 8. 2.6 The nearest neighbor rule - p. 61 9. 2.7 Bayesian networks - p. 64 10. 2.8 Problems - p. 71 11. Matlab programs and exercises - p. 79 12. Chapter 3: Linear Classifiers - p. 91 13. 3.1 Introduction - p. 91 14. 3.2 Linear discriminant functions and decision hyperplanes - p. 91 15. 3.3 The perceptron algorithm - p. 92 16. 3.4 Least squares methods - p. 103 17. 3.5 Mean square estimation revisited - p. 110 18. 3.6 Logistic discrimination - p. 117 19. 3.7 Support vector machines - p. 119 20. 3.8 Problems - p. 142 21. Matlab programs and exercises - p. 144 22. Chapter 4: Nonlinear Classifiers - p. 151 23. 4.1 Introduction - p. 151 24. 4.2 The xor problem - p. 151 25. 4.3 The two-layer perceptron - p. 152 26. 4.4 Three-layer perceptrons - p. 158 27. 4.5 Algorithms based on exact classification of the training set - p. 160 28. 4.6 The backpropagation algorithm - p. 162 29. 4.7 Variations on the backpropagation theme - p. 169 30. 4.8 The cost function choice - p. 172 31. 4.9 Choice of the network size - p. 176 32. 4.10 A simulation example - p. 181 33. 4.11 Networks with weight sharing - p. 183 34. 4.12 Generalized linear classifiers - p. 184 35. 4.13 Capacity of the l -dimensional space in linear dichotomies - p. 187 36. 4.14 Polynomial classifiers - p. 189 37. 4.15 Radial basis function networks - p. 190 38. 4.16 Universal approximators - p. 194 39. 4.17 Probabilistic neural networks - p. 196 40. 4.18 Support vector machines: the nonlinear case - p. 198 41. 4.19 Beyond the svm paradigm - p. 203 42. 4.20 Decision trees - p. 215 43. 4.21 Combining classifiers - p. 222 44. 4.22 The boosting approach to combine classifiers - p. 230 45. 4.23 The class imbalance problem - p. 237 46. 4.24 Discussion - p. 239 47. 4.25 Problems - p. 240 48. Matlab programs and exercises - p. 244 49. Chapter 5: Feature Selection - p. 261 50. 5.1 Introduction - p. 261 51. 5.2 Preprocessing - p. 262 52. 5.3 The peaking phenomenon - p. 265 53. 5.4 Feature selection based on statistical hypothesis testing - p. 268 54. 5.5 The receiver operating characteristics (ROC) curve - p. 275 55. 5.6 Class separability measures - p. 276 56. 5.7 Feature subset selection - p. 282 57. 5.8 Optimal feature generation - p. 288 58. 5.9 Neural networks and feature generation/selection - p. 297 59. 5.10 A hint on generalization theory - p. 299 60. 5.11 The bayesian information criterion - p. 309 61. 5.12 Problems - p. 310 62. Matlab programs and exercises - p. 314 63. Chapter 6: Feature Generation I: Data Transformation and Dimensionality Reduction - p. 323 64. 6.1 Introduction - p. 323 65. 6.2 Basis vectors and images - p. 324 66. 6.3 The karhunen–loève transform - p. 326 67. 6.4 The singular value decomposition - p. 334 68. 6.5 Independent component analysis - p. 341 69. 6.6 Nonnegative matrix factorization - p. 349 70. 6.7 Nonlinear dimensionality reduction - p. 350 71. 6.8 The discrete fourier transform (DFT) - p. 362 72. 6.9 The discrete cosine and sine transforms - p. 366 73. 6.10 The hadamard transform - p. 367 74. 6.11 The haar transform - p. 369 75. 6.12 The haar expansion revisited - p. 371 76. 6.13 Discrete time wavelet transform (DTWT) - p. 375 77. 6.14 The multiresolution interpretation - p. 384 78. 6.15 Wavelet packets - p. 386 79. 6.16 A look at two-dimensional generalizations - p. 388 80. 6.17 Applications - p. 389 81. 6.18 Problems - p. 396 82. Matlab programs and exercises - p. 399 83. Chapter 7: Feature Generation II - p. 411 84. 7.1 Introduction - p. 411 85. 7.2 Regional features - p. 412 86. 7.3 Features for shape and size characterization - p. 435 87. 7.4 A glimpse at fractals - p. 443 88. 7.5 Typical features for speech and audio classification - p. 451 89. 7.6 Problems - p. 464 90. Matlab programs and exercises - p. 469 91. Chapter 8: Template Matching - p. 481 92. Chapter 9: Context-Dependent Classification - p. 521 93. 9.1 Introduction - p. 521 94. 9.2 The bayes classifier - p. 521 95. 9.3 Markov chain models - p. 522 96. 9.4 The viterbi algorithm - p. 523 97. 9.5 Channel equalization - p. 527 98. 9.6 Hidden markov models - p. 531 99. 9.7 HMM with state duration modeling - p. 545 100. 9.8 Training markov models via neural networks - p. 551 101. 9.9 A Discussion of markov random fields - p. 553 102. 9.10 Problems - p. 556 103. Matlab programs and exercises - p. 557 104. Chapter 10: Supervised Learning: The Epilogue - p. 567 105. Chapter 11: Clustering: Basic Concepts - p. 595 106. Chapter 12: Clustering Algorithms I: Sequential Algorithms - p. 627 107. Chapter 13: Clustering Algorithms II: Hierarchical Algorithms - p. 653 108. 13.1 Introduction - p. 653 109. 13.2 Agglomerative algorithms - p. 654 110. 13.3 The cophenetic matrix - p. 679 111. 13.4 Divisive algorithms - p. 680 112. 13.5 Hierarchical algorithms for large data sets - p. 682 113. 13.6 Choice of the best number of clusters - p. 690 114. 13.7 Problems - p. 692 115. Matlab programs and exercises - p. 695 116. Chapter 14: Clustering Algorithms III: Schemes Based on Function Optimization - p. 701 117. 14.1 Introduction - p. 701 118. 14.2 Mixture decomposition schemes - p. 702 119. 14.3 Fuzzy clustering algorithms - p. 712 120. 14.4 Possibilistic clustering - p. 733 121. 14.5 Hard clustering algorithms - p. 739 122. 14.6 Vector quantization - p. 749 123. Appendix - p. 751 124. 14.7 Problems - p. 752 125. Matlab programs and excercises - p. 754 126. Chapter 15: Clustering Algorithms IV - p. 765 127. 15.1 Introduction - p. 765 128. 15.2 Clustering algorithms based on graph theory - p. 765 129. 15.3 Competitive learning algorithms - p. 779 130. 15.4 Binary morphology clustering algorithms (BMCAs) - p. 789 131. 15.5 Boundary detection algorithms - p. 797 132. 15.6 Valley-seeking clustering algorithms - p. 801 133. 15.7 Clustering via cost optimization (Revisited) - p. 803 134. 15.8 Kernel clustering methods - p. 811 135. 15.9 Density-based algorithms for large data sets - p. 815 136. 15.10 Clustering algorithms for high-dimensional data sets - p. 821 137. 15.11 Other clustering algorithms - p. 837 138. 15.12 Combination of clusterings - p. 839 139. 15.13 Problems - p. 846 140. Matlab programs and exercises - p. 849 141. Chapter 16: Cluster Validity - p. 863 142. 16.1 Introduction - p. 863 143. 16.2 Hypothesis testing revisited - p. 864 144. 16.3 Hypothesis testing in cluster validity - p. 866 145. 16.4 Relative criteria - p. 876 146. 16.5 Validity of individual clusters - p. 893 147. 16.6 Clustering tendency - p. 895 148. 16.7 Problems - p. 905 149. Appendix A: Hints from Probability and Statistics - p. 915 150. Appendix B: Linear Algebra Basics - p. 927 151. Appendix C: Cost Function Optimization - p. 930 152. Appendix D: Basic Definitions from Linear Systems Theory - p. 946 153. Index - p. 949
This book considers classical and current theory and practice, of supervised, unsupervised and semi-supervised pattern recognition, to build a complete background for professionals and students of engineering. The authors, leading experts in the field of pattern recognition, have provided an up-to-date, self-contained volume encapsulating this wide spectrum of information. The very latest methods are incorporated in this edition: semi-supervised learning, combining clustering algorithms, and relevance feedback.
· Thoroughly developed to include many more worked examples to give greater understanding of the various methods and techniques
· Many more diagrams included--now in two color--to provide greater insight through visual presentation
· Matlab code of the most common methods are given at the end of each chapter.
· More Matlab code is available, together with an accompanying manual, via this site
· Latest hot topics included to further the reference value of the text including non-linear dimensionality reduction techniques, relevance feedback, semi-supervised learning, spectral clustering, combining clustering algorithms.
· An accompanying book with Matlab code of the most common methods and algorithms in the book, together with a descriptive summary, and solved examples including real-life data sets in imaging, and audio recognition. The companion book will be available separately or at a special packaged price (ISBN: 9780123744869).
1. Chapter 1: Introduction - p. xvii 2. Chapter 2: Classifiers Based on Bayes Decision Theory - p. 13 3. 2.1 Introduction - p. 13 4. 2.2 Bayes decision theory - p. 13 5. 2.3 Discriminant functions and decision surfaces - p. 19 6. 2.4 Bayesian classification for normal distributions - p. 20 7. 2.5 Estimation of unknown probability density functions - p. 34 8. 2.6 The nearest neighbor rule - p. 61 9. 2.7 Bayesian networks - p. 64 10. 2.8 Problems - p. 71 11. Matlab programs and exercises - p. 79 12. Chapter 3: Linear Classifiers - p. 91 13. 3.1 Introduction - p. 91 14. 3.2 Linear discriminant functions and decision hyperplanes - p. 91 15. 3.3 The perceptron algorithm - p. 92 16. 3.4 Least squares methods - p. 103 17. 3.5 Mean square estimation revisited - p. 110 18. 3.6 Logistic discrimination - p. 117 19. 3.7 Support vector machines - p. 119 20. 3.8 Problems - p. 142 21. Matlab programs and exercises - p. 144 22. Chapter 4: Nonlinear Classifiers - p. 151 23. 4.1 Introduction - p. 151 24. 4.2 The xor problem - p. 151 25. 4.3 The two-layer perceptron - p. 152 26. 4.4 Three-layer perceptrons - p. 158 27. 4.5 Algorithms based on exact classification of the training set - p. 160 28. 4.6 The backpropagation algorithm - p. 162 29. 4.7 Variations on the backpropagation theme - p. 169 30. 4.8 The cost function choice - p. 172 31. 4.9 Choice of the network size - p. 176 32. 4.10 A simulation example - p. 181 33. 4.11 Networks with weight sharing - p. 183 34. 4.12 Generalized linear classifiers - p. 184 35. 4.13 Capacity of the l -dimensional space in linear dichotomies - p. 187 36. 4.14 Polynomial classifiers - p. 189 37. 4.15 Radial basis function networks - p. 190 38. 4.16 Universal approximators - p. 194 39. 4.17 Probabilistic neural networks - p. 196 40. 4.18 Support vector machines: the nonlinear case - p. 198 41. 4.19 Beyond the svm paradigm - p. 203 42. 4.20 Decision trees - p. 215 43. 4.21 Combining classifiers - p. 222 44. 4.22 The boosting approach to combine classifiers - p. 230 45. 4.23 The class imbalance problem - p. 237 46. 4.24 Discussion - p. 239 47. 4.25 Problems - p. 240 48. Matlab programs and exercises - p. 244 49. Chapter 5: Feature Selection - p. 261 50. 5.1 Introduction - p. 261 51. 5.2 Preprocessing - p. 262 52. 5.3 The peaking phenomenon - p. 265 53. 5.4 Feature selection based on statistical hypothesis testing - p. 268 54. 5.5 The receiver operating characteristics (ROC) curve - p. 275 55. 5.6 Class separability measures - p. 276 56. 5.7 Feature subset selection - p. 282 57. 5.8 Optimal feature generation - p. 288 58. 5.9 Neural networks and feature generation/selection - p. 297 59. 5.10 A hint on generalization theory - p. 299 60. 5.11 The bayesian information criterion - p. 309 61. 5.12 Problems - p. 310 62. Matlab programs and exercises - p. 314 63. Chapter 6: Feature Generation I: Data Transformation and Dimensionality Reduction - p. 323 64. 6.1 Introduction - p. 323 65. 6.2 Basis vectors and images - p. 324 66. 6.3 The karhunen–loève transform - p. 326 67. 6.4 The singular value decomposition - p. 334 68. 6.5 Independent component analysis - p. 341 69. 6.6 Nonnegative matrix factorization - p. 349 70. 6.7 Nonlinear dimensionality reduction - p. 350 71. 6.8 The discrete fourier transform (DFT) - p. 362 72. 6.9 The discrete cosine and sine transforms - p. 366 73. 6.10 The hadamard transform - p. 367 74. 6.11 The haar transform - p. 369 75. 6.12 The haar expansion revisited - p. 371 76. 6.13 Discrete time wavelet transform (DTWT) - p. 375 77. 6.14 The multiresolution interpretation - p. 384 78. 6.15 Wavelet packets - p. 386 79. 6.16 A look at two-dimensional generalizations - p. 388 80. 6.17 Applications - p. 389 81. 6.18 Problems - p. 396 82. Matlab programs and exercises - p. 399 83. Chapter 7: Feature Generation II - p. 411 84. 7.1 Introduction - p. 411 85. 7.2 Regional features - p. 412 86. 7.3 Features for shape and size characterization - p. 435 87. 7.4 A glimpse at fractals - p. 443 88. 7.5 Typical features for speech and audio classification - p. 451 89. 7.6 Problems - p. 464 90. Matlab programs and exercises - p. 469 91. Chapter 8: Template Matching - p. 481 92. Chapter 9: Context-Dependent Classification - p. 521 93. 9.1 Introduction - p. 521 94. 9.2 The bayes classifier - p. 521 95. 9.3 Markov chain models - p. 522 96. 9.4 The viterbi algorithm - p. 523 97. 9.5 Channel equalization - p. 527 98. 9.6 Hidden markov models - p. 531 99. 9.7 HMM with state duration modeling - p. 545 100. 9.8 Training markov models via neural networks - p. 551 101. 9.9 A Discussion of markov random fields - p. 553 102. 9.10 Problems - p. 556 103. Matlab programs and exercises - p. 557 104. Chapter 10: Supervised Learning: The Epilogue - p. 567 105. Chapter 11: Clustering: Basic Concepts - p. 595 106. Chapter 12: Clustering Algorithms I: Sequential Algorithms - p. 627 107. Chapter 13: Clustering Algorithms II: Hierarchical Algorithms - p. 653 108. 13.1 Introduction - p. 653 109. 13.2 Agglomerative algorithms - p. 654 110. 13.3 The cophenetic matrix - p. 679 111. 13.4 Divisive algorithms - p. 680 112. 13.5 Hierarchical algorithms for large data sets - p. 682 113. 13.6 Choice of the best number of clusters - p. 690 114. 13.7 Problems - p. 692 115. Matlab programs and exercises - p. 695 116. Chapter 14: Clustering Algorithms III: Schemes Based on Function Optimization - p. 701 117. 14.1 Introduction - p. 701 118. 14.2 Mixture decomposition schemes - p. 702 119. 14.3 Fuzzy clustering algorithms - p. 712 120. 14.4 Possibilistic clustering - p. 733 121. 14.5 Hard clustering algorithms - p. 739 122. 14.6 Vector quantization - p. 749 123. Appendix - p. 751 124. 14.7 Problems - p. 752 125. Matlab programs and excercises - p. 754 126. Chapter 15: Clustering Algorithms IV - p. 765 127. 15.1 Introduction - p. 765 128. 15.2 Clustering algorithms based on graph theory - p. 765 129. 15.3 Competitive learning algorithms - p. 779 130. 15.4 Binary morphology clustering algorithms (BMCAs) - p. 789 131. 15.5 Boundary detection algorithms - p. 797 132. 15.6 Valley-seeking clustering algorithms - p. 801 133. 15.7 Clustering via cost optimization (Revisited) - p. 803 134. 15.8 Kernel clustering methods - p. 811 135. 15.9 Density-based algorithms for large data sets - p. 815 136. 15.10 Clustering algorithms for high-dimensional data sets - p. 821 137. 15.11 Other clustering algorithms - p. 837 138. 15.12 Combination of clusterings - p. 839 139. 15.13 Problems - p. 846 140. Matlab programs and exercises - p. 849 141. Chapter 16: Cluster Validity - p. 863 142. 16.1 Introduction - p. 863 143. 16.2 Hypothesis testing revisited - p. 864 144. 16.3 Hypothesis testing in cluster validity - p. 866 145. 16.4 Relative criteria - p. 876 146. 16.5 Validity of individual clusters - p. 893 147. 16.6 Clustering tendency - p. 895 148. 16.7 Problems - p. 905 149. Appendix A: Hints from Probability and Statistics - p. 915 150. Appendix B: Linear Algebra Basics - p. 927 151. Appendix C: Cost Function Optimization - p. 930 152. Appendix D: Basic Definitions from Linear Systems Theory - p. 946 153. Index - p. 949
