Computational Statistics

  1. Numerical Optimization in Statistics
    1. The Role of Optimization in Statistical Modeling
      1. Maximum Likelihood Estimation (MLE)
        1. Likelihood Functions
          1. Log-Likelihood
            1. Score Functions
              1. Fisher Information
              2. Least Squares Estimation
                1. Linear Regression
                  1. Nonlinear Regression
                    1. Weighted Least Squares
                    2. Method of Moments
                      1. Moment Equations
                        1. Generalized Method of Moments
                        2. Bayesian Estimation
                          1. Maximum A Posteriori (MAP)
                            1. Posterior Mode Finding
                          2. Optimization Algorithms
                            1. Gradient-Based Methods
                              1. Gradient Descent
                                1. Step Size Selection
                                  1. Convergence Criteria
                                    1. Line Search Methods
                                    2. Stochastic Gradient Descent
                                      1. Mini-batch Methods
                                        1. Online Learning
                                          1. Adaptive Learning Rates
                                          2. Conjugate Gradient Methods
                                            1. Application to Large Systems
                                              1. Preconditioning
                                            2. Newton's Method and Variants
                                              1. Newton-Raphson Method
                                                1. Hessian Matrix Computation
                                                  1. Convergence Properties
                                                  2. Quasi-Newton Methods
                                                    1. BFGS Algorithm
                                                      1. L-BFGS Algorithm
                                                        1. DFP Algorithm
                                                        2. Fisher Scoring
                                                          1. Application in Generalized Linear Models
                                                            1. Information Matrix
                                                          2. Derivative-Free Methods
                                                            1. Nelder-Mead Simplex
                                                              1. Simulated Annealing
                                                                1. Genetic Algorithms
                                                                2. Constrained Optimization
                                                                  1. Lagrange Multipliers
                                                                    1. Penalty Methods
                                                                      1. Barrier Methods
                                                                    2. The Expectation-Maximization (EM) Algorithm
                                                                      1. The E-Step (Expectation)
                                                                        1. Calculating Expected Values
                                                                          1. Complete Data Log-Likelihood
                                                                          2. The M-Step (Maximization)
                                                                            1. Maximizing the Expected Log-Likelihood
                                                                              1. Parameter Updates
                                                                              2. Theory and Convergence
                                                                                1. Monotonicity of Likelihood
                                                                                  1. Convergence Criteria
                                                                                    1. Rate of Convergence
                                                                                    2. Variants of EM
                                                                                      1. Generalized EM (GEM)
                                                                                        1. Expectation Conditional Maximization (ECM)
                                                                                          1. Monte Carlo EM (MCEM)
                                                                                          2. Applications with Missing Data and Latent Variables
                                                                                            1. Mixture Models
                                                                                              1. Hidden Markov Models
                                                                                                1. Factor Analysis
                                                                                                  1. Incomplete Data Problems

                                                                                              Previous

                                                                                              3. Resampling Methods

                                                                                              Go to top

                                                                                              Next

                                                                                              5. Bayesian Computational Methods