Machine Learning: A Constraint-Based Approach, Second Edition provides readers with a refreshing look at the basic models and algorithms of machine learning, with an emphasis on current topics of interest that include neural networks and kernel machines. The book presents the information in a truly unified manner that is based on the notion of learning from environmental constraints. It draws a path towards deep integration with machine learning that relies on the idea of adopting multivalued logic formalisms, such as in fuzzy systems. Special attention is given to deep learning, which nicely fits the constrained-based approach followed in this book.
The book presents a simpler unified notion of regularization, which is strictly connected with the parsimony principle, including many solved exercises that are classified according to the Donald Knuth ranking of difficulty, which essentially consists of a mix of warm-up exercises that lead to deeper research problems. A software simulator is also included.
1. Machine Learning - p. i 2. Cover - p. i 3. Title page - p. ii 4. Copyright - p. iv 5. Table of Contents - p. v 6. Preface - p. xi 7. Notes on the exercises - p. xvii 8. Chapter 1 The big picture - p. 1 9. 1.1: Why do machines need to learn? - p. 2 10. 1.2: Principles and practice - p. 25 11. 1.3: Machine learning experiments - p. 34 12. 1.4: Challenges in machine learning - p. 43 13. 1.5: Scholia - p. 47 14. Chapter 2 Learning principles - p. 53 15. 2.1: Environmental constraints - p. 54 16. 2.1.1: Loss and risk functions - p. 54 17. 2.1.2: Ill-position of constraint-induced risk functions - p. 62 18. 2.1.3: Risk minimization - p. 64 19. 2.1.4: The bias–variance dilemma - p. 69 20. 2.2: Statistical learning - p. 75 21. 2.3: Information-based learning - p. 86 22. 2.4: Learning under the parsimony principle - p. 94 23. 2.5: Scholia - p. 105 24. Chapter 3 Linear threshold machines - p. 113 25. 3.1: Linear machines - p. 114 26. 3.2: Linear machines with threshold units - p. 128 27. 3.3: Statistical view - p. 141 28. 3.4: Algorithmic issues - p. 147 29. 3.5: Scholia - p. 159 30. Chapter 4 Kernel machines - p. 169 31. 4.1: Feature space - p. 170 32. 4.2: Maximum margin problem - p. 177 33. 4.3: Kernel functions - p. 189 34. 4.4: Regularization - p. 201 35. 4.5: Scholia - p. 210 36. Chapter 5 Deep architectures - p. 217 37. 5.1: Architectural issues - p. 218 38. 5.2: Realization of Boolean functions - p. 228 39. 5.3: Realization of real-valued functions - p. 244 40. 5.4: Convolutional networks - p. 258 41. 5.5: Learning in feedforward networks - p. 274 42. 5.6: Advances in deep learning - p. 288 43. 5.7: Complexity issues - p. 307 44. 5.8: Scholia - p. 320 45. Chapter 6 Learning with constraints - p. 339 46. 6.1: Constraint machines - p. 342 47. 6.1.1: Walking through learning and inference - p. 342 48. 6.1.2: A unified view of constrained environments - p. 351 49. 6.1.3: Functional representation of learning tasks - p. 359 50. 6.1.4: Reasoning with constraints - p. 364 51. 6.2: Logic constraints in the environment - p. 371 52. 6.3: Diffusion machines - p. 389 53. 6.4: Algorithmic issues - p. 400 54. 6.5: Life-long learning agents - p. 420 55. 6.6: Scholia - p. 432 56. Chapter 7 Epilogue - p. 443 57. Chapter 8 Answers to exercises - p. 449 58. Section 1.1 - p. 450 59. Section 1.2 - p. 451 60. Section 1.3 - p. 452 61. Section 1.4 - p. 452 62. Section 2.1 - p. 452 63. Section 2.2 - p. 457 64. Section 3.1 - p. 462 65. Section 3.2 - p. 465 66. Section 3.3 - p. 468 67. Section 3.4 - p. 468 68. Section 4.1 - p. 470 69. Section 4.2 - p. 472 70. Section 4.3 - p. 474 71. Section 4.4 - p. 479 72. Section 5.1 - p. 480 73. Section 5.2 - p. 482 74. Section 5.3 - p. 483 75. Section 5.4 - p. 485 76. Section 5.5 - p. 486 77. Section 5.7 - p. 487 78. Section 6.1 - p. 490 79. Section 6.2 - p. 492 80. Section 6.3 - p. 494 81. Section 6.4 - p. 496 82. Appendix A Constrained optimization - p. 501 83. Appendix B Regularization operators - p. 505 84. Appendix C Calculus of variations - p. 511 85. Appendix D Index to notation - p. 521 86. Index - p. 525
Machine Learning: A Constraint-Based Approach, Second Edition provides readers with a refreshing look at the basic models and algorithms of machine learning, with an emphasis on current topics of interest that include neural networks and kernel machines. The book presents the information in a truly unified manner that is based on the notion of learning from environmental constraints. It draws a path towards deep integration with machine learning that relies on the idea of adopting multivalued logic formalisms, such as in fuzzy systems. Special attention is given to deep learning, which nicely fits the constrained-based approach followed in this book.
The book presents a simpler unified notion of regularization, which is strictly connected with the parsimony principle, including many solved exercises that are classified according to the Donald Knuth ranking of difficulty, which essentially consists of a mix of warm-up exercises that lead to deeper research problems. A software simulator is also included.
1. Machine Learning - p. i 2. Cover - p. i 3. Title page - p. ii 4. Copyright - p. iv 5. Table of Contents - p. v 6. Preface - p. xi 7. Notes on the exercises - p. xvii 8. Chapter 1 The big picture - p. 1 9. 1.1: Why do machines need to learn? - p. 2 10. 1.2: Principles and practice - p. 25 11. 1.3: Machine learning experiments - p. 34 12. 1.4: Challenges in machine learning - p. 43 13. 1.5: Scholia - p. 47 14. Chapter 2 Learning principles - p. 53 15. 2.1: Environmental constraints - p. 54 16. 2.1.1: Loss and risk functions - p. 54 17. 2.1.2: Ill-position of constraint-induced risk functions - p. 62 18. 2.1.3: Risk minimization - p. 64 19. 2.1.4: The bias–variance dilemma - p. 69 20. 2.2: Statistical learning - p. 75 21. 2.3: Information-based learning - p. 86 22. 2.4: Learning under the parsimony principle - p. 94 23. 2.5: Scholia - p. 105 24. Chapter 3 Linear threshold machines - p. 113 25. 3.1: Linear machines - p. 114 26. 3.2: Linear machines with threshold units - p. 128 27. 3.3: Statistical view - p. 141 28. 3.4: Algorithmic issues - p. 147 29. 3.5: Scholia - p. 159 30. Chapter 4 Kernel machines - p. 169 31. 4.1: Feature space - p. 170 32. 4.2: Maximum margin problem - p. 177 33. 4.3: Kernel functions - p. 189 34. 4.4: Regularization - p. 201 35. 4.5: Scholia - p. 210 36. Chapter 5 Deep architectures - p. 217 37. 5.1: Architectural issues - p. 218 38. 5.2: Realization of Boolean functions - p. 228 39. 5.3: Realization of real-valued functions - p. 244 40. 5.4: Convolutional networks - p. 258 41. 5.5: Learning in feedforward networks - p. 274 42. 5.6: Advances in deep learning - p. 288 43. 5.7: Complexity issues - p. 307 44. 5.8: Scholia - p. 320 45. Chapter 6 Learning with constraints - p. 339 46. 6.1: Constraint machines - p. 342 47. 6.1.1: Walking through learning and inference - p. 342 48. 6.1.2: A unified view of constrained environments - p. 351 49. 6.1.3: Functional representation of learning tasks - p. 359 50. 6.1.4: Reasoning with constraints - p. 364 51. 6.2: Logic constraints in the environment - p. 371 52. 6.3: Diffusion machines - p. 389 53. 6.4: Algorithmic issues - p. 400 54. 6.5: Life-long learning agents - p. 420 55. 6.6: Scholia - p. 432 56. Chapter 7 Epilogue - p. 443 57. Chapter 8 Answers to exercises - p. 449 58. Section 1.1 - p. 450 59. Section 1.2 - p. 451 60. Section 1.3 - p. 452 61. Section 1.4 - p. 452 62. Section 2.1 - p. 452 63. Section 2.2 - p. 457 64. Section 3.1 - p. 462 65. Section 3.2 - p. 465 66. Section 3.3 - p. 468 67. Section 3.4 - p. 468 68. Section 4.1 - p. 470 69. Section 4.2 - p. 472 70. Section 4.3 - p. 474 71. Section 4.4 - p. 479 72. Section 5.1 - p. 480 73. Section 5.2 - p. 482 74. Section 5.3 - p. 483 75. Section 5.4 - p. 485 76. Section 5.5 - p. 486 77. Section 5.7 - p. 487 78. Section 6.1 - p. 490 79. Section 6.2 - p. 492 80. Section 6.3 - p. 494 81. Section 6.4 - p. 496 82. Appendix A Constrained optimization - p. 501 83. Appendix B Regularization operators - p. 505 84. Appendix C Calculus of variations - p. 511 85. Appendix D Index to notation - p. 521 86. Index - p. 525
