Pattern recognition | Mauricio Orozco-Alzate |

: Home :: Teaching :: Research :: Publications :: Students :: CV :: CvLAC : | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

We, human beings, perceive information of the surrounding environment by the senses.
Using a series of general concepts or The aforementioned examples and, in general, all the processes of recognition,
involve a The complex and repetitive classification tasks, the necessity of increasing the reliability and objectivity of decisions, judgments or diagnoses and the research works
about the human brain promoted the development of algorithms —also called The core of my research is related to the field of pattern recognition and particularly to statistical PR; that is, the PR based on probability theory i.e. decision-theoretical approach. There are several definitions of PR in the classical textbooks (Fukunaga, 1990; Duda, 2001; Webb, 2002; van der Heijden, 2004; Theodoridis, 2006). Some of them are very general while others are quite specific. Theodoridis and Koutroumbas (2006) define PR as "the scientific discipline whose goal is the classification of objects into a number of categories"; Duda et al. (2001) define it as "the act of taking raw data and taking an action based on the category of the pattern". Taking into account those definitions, I give mine here with the intention to be a comprehensive one: "Pattern recognition is the discipline that attempts to find ways of imitating the human capacity of using sensorial information and knowledge intelligently, providing mathematical foundations, models and methods for learning from a limited number of examples, in order to automate the process of classification or categorization". Usually, the knowledge cited in my definition is called prior knowledge. It is that we —or the machine— know about the object beforehand. Similarly, sensorial information, perceived through our senses or acquired by sensors according to the case, is called empirical knowledge. The combined use of both sources of information produces the posterior knowledge, in which decisions and actions for minimizing errors or losses are based.
## Recommended booksR. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification., 2nd ed. New York: Wiley-Interscience, 2001.
Table of Contents
- Preface
- Chapter 1: Introduction
- Chapter 2: Bayesian decision theory
- 2.1 Introduction
- 2.2 Bayesian decision theory–Continuous features
- 2.3 Minimum-error-rate classification
- 2.4 Classifiers, discriminant functions, and decision surfaces
- 2.5 The normal density
- 2.6 Discriminant function for the normal density
- *2.7 Error probabilities and integrals
- *2.8 Error bounds for normal densities
- 2.9 Bayes decision theory—discrete features
- *2.10 Missing and noisy features
- *2.11 Bayesian belief networks
- *2.12 Compound Bayesian decision theory and context
- Summary
- Bibliographical and historical remarks
- Problems
- Computer exercises
- Bibliography
- Chapter 3: Maximum-likelihood and Bayesian parameter estimation
- 3.1 Introduction
- 3.2 Maximum-likelihood estimation
- 3.3 Bayesian estimation
- 3.4 Bayesian parameter estimation: Gaussian case
- 3.5 Bayesian parameter estimation: General theory
- *3.6 Sufficient statistics
- 3.7 Problems of dimensionality
- *3.8 Component analysis and discriminants
- *3.9 Expectation-maximization (EM)
- 3.10 Hidden Markov models
- Summary
- Biographical and historical remarks
- Problems
- Computer exercises
- Bibliography
- Chapter 4: Nonparametric techniques
- 4.1 Introduction
- 4.2 Density estimation
- 4.3 Parzen windows
- 4.4 kn-nearest-neighbor estimation
- 4.5 The nearest-neighbor rule
- 4.6 Metrics and nearest-neighbor classification
- *4.7 Fuzzy classification
- *4.8 Reduced Coulomb energy networks
- 4.9 Approximations by series expansions
- Summary
- Bibliographical and historical remarks
- Problems
- Computer exercises
- Bibliography
- Chapter 5: Linear discriminant functions
- 5.1 Introduction
- 5.2 Linear discriminant functions and decision surfaces
- 5.3 Generalized linear discriminant functions
- 5.4 The two-category linearly separable case
- 5.5 Minimizing the perceptron criterion function
- 5.6 Relaxation procedures
- 5.7 Nonseparable behavior
- 5.8 Minimum squared-error procedures
- 5.9 The Ho-Kashyap procedures
- *5.10 Linear programming algorithms
- *5.11 Support vector machines
- 5.12 Multicategory generalizations
- Summary
- Bibliographical and historical remarks
- Problems
- Computer exercises
- Bibliography
- Chapter 6: Multilayer neural networks
- 6.1 Introduction
- 6.2 Feedforward operation and classification
- 6.3 Backpropagation algorithm
- 6.4 Error surfaces
- 6.5 Backpropagation as feature mapping
- 6.6 Backpropagation, Bayes theory and probability
- *6.7 Related statistical techniques
- 6.8 Practical techniques for improving backpropagation
- 6.8.1 Activation function
- 6.8.2 Parameters for the sigmoid
- 6.8.3 Scaling input
- 6.8.4 Target values
- 6.8.5 Training with noise
- 6.8.6 Manufacturing data
- 6.8.9 Learning rates
- 6.8.10 Momentum
- 6.8.11 Weight decay
- 6.8.12 Hints
- 6.8.13 On-line, stochastic or batch training?
- 6.8.14 Stopped training
- 6.8.15 Number of hidden layers
- 6.8.16 Criterion function
- *6.9 Second-order methods
- *6.10 Additional networks and training methods
- 6.11 Regularization, complexity adjustment and pruning
- Summary
- Bibliographical and historical remarks
- Problems
- Computer exercises
- Bibliography
- Chapter 7: Stochastic methods
- Chapter 9: Algorithm-independent machine learning
- 9.1 Introduction
- 9.2 Lack of inherent superiority of any classifier
- 9.3 Bias and variance
- 9.4 Resampling for estimating statistics
- 9.5 Resampling for classifier design
- 9.6 Estimating and comparing clasifiers
- 9.6.1 Parametric models
- 9.6.2 Cross-validation
- 9.6.3 Jackknife and bootstrap estimation of classification accuracy
- 9.6.4 Maximum-likelihood model comparison
- 9.6.5 Bayesian model comparison
- 9.6.6 The problem-average error rate
- 9.6.7 Predicting final performance from learning curves
- 9.6.8 The capacity of a separating plane
- 9.7 Combining classifiers
- Summary
- Bibliographical and historical remarks
- Problems
- Computer exercises
- Bibliography
- Chapter 10: Unsupervised learning and clustering
- 10.1 Introduction
- 10.2 Mixture densities and identifiability
- 10.3 Maximum-likelihood estimates
- 10.4 Application to normal densities
- 10.5: Unsupervised Bayesian learning
- 10.6 Data description and clustering
- 10.7: Criterion functions for clustering
- *10.8: Iterative optimization
- 10.9: Hierarchical clustering
- *10.10: The problem of validity
- *10.11: On-line clustering
- *10.12: Graph-theoretic methods
- 10.13: Component analysis
- 10.14: Low-dimensional representtions and multidimensional scaling (MDS)
- Summary
- Bibliographical and historical remarks
- Problems
- Computer exercises
- Bibliography
- Appendix: Mathematical foundations
- A.1: Notation
- A.2: Linear algebra
- A.3: Lagrange optimization
- A.4: Probability theory
- A.4.1: Discrete random variables
- A.4.2: Expected values
- A.4.3: Pairs of discrete random variables
- A.4.4: Statistical independence
- A.4.5: Expected values of functions of two variables
- A.4.6: Conditional probability
- A.4.7: The law of total probability and Bayes rule
- A.4.8: Vector random variables
- A.4.9: Expectations, mean vectors and covariance matrices
- A.4.10: Continuous random variables
- A.4.11: Distributions of sums of independent random variables
- A.4.12: Normal distributions
- A.5: Gaussian derivatives and integrals
- A.6: Hypothesis testing
- A.7: Information theory
- Computational complexity
- Index
K. Fukunaga, Introduction to Statistical Pattern Recognition. New York: Aca
demic Press, 1990.
C. M. Bishop, Pattern Recognition and Machine Learning. Singapore: Springer,
2006. A. R. Webb, Statistical Pattern Recognition, 2nd ed. London, UK: Wiley, 2002. F. van der Heijden, R. P. W. Duin, D. de Ridder, and D. M. J. Tax, Classification,
Parameter Estimation and State Estimation: An Engineering Approach Using
MATLAB. Chichester , UK: Wiley, 2004. S. Theodoridis and K. Koutroumbas, Pattern Recognition, 3rd, Ed. Amsterdam:
Elsevier Academic Press, 2006. P. A. Devijver and J. Kittler, Pattern Recognition: a Statistical Approach. Lon
don: Prentice Hall International, 1982.
## Software
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

: Home :: Teaching :: Research :: Publications :: Students :: CV :: CvLAC : | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

Mauricio Orozco-Alzate |