Generalized Linear Models
Generalized Linear Models (GLMs) provide a powerful and flexible framework for modeling a response variable ($Y$) that is non-normally distributed or related...
Posts by tag
- statistics 11
- normal-distribution 10
- derivative 9
- loss-function 9
- vector 9
- variance 9
- linear-regression 8
- gradient-descent 6
- probability-density-function 6
- hypothesis-testing 6
- regression 6
- differentiation 5
- matrix 5
- nonlinear-regression 5
- probability 5
- confidence-interval 5
- MLE 5
- calculus 4
- second-derivative 4
- Taylor-series 4
- mean 4
- residuals 4
- probability-distribution 4
- sample 4
- maximum-likelihood-estimation 4
- slope 3
- gradient 3
- machine-learning 3
- penalization 3
- learning-rate 3
- span 3
- svd 3
- dot-product 3
- kernel 3
- symmetric-matrix 3
- positive-definite-matrix 3
- hessian 3
- steepest-descent 3
- classification 3
- significance-test 3
- z-score 3
- binomial-distribution 3
- cosine 3
- conditional-probability 3
- covariance 3
- PDF 3
- error-term 3
- survival-function 3
- hazard-function 3
- survival-analysis 3
- derivation 2
- functions 2
- integration 2
- polynomial 2
- basis 2
- inverse 2
- determinant 2
- matrix-diagonalization 2
- column-space 2
- principal-component-analysis 2
- rank 2
- ill-conditioned-matrix 2
- taylor-series 2
- taylor-expansion 2
- Hessian 2
- simplex 2
- secant-line 2
- minimization 2
- smoothing 2
- lowess 2
- outliers 2
- dimensionality-reduction 2
- pseudoinverse 2
- covariance-matrix 2
- null-hypothesis 2
- p-value 2
- significance-level 2
- false-discovery-rate 2
- margin-of-error 2
- degrees-of-freedom 2
- t-test 2
- t-distribution 2
- heteroscedasticity 2
- homoscedasticity 2
- natural-language-processing 2
- Levenshtein-distance 2
- median 2
- variation 2
- cost-function 2
- Kullback–Leibler-divergence 2
- KL-divergence 2
- gaussian-process 2
- prior-distribution 2
- posterior-distribution 2
- Bayes-theorem 2
- ELBO 2
- sigmoid-function 2
- stationarity 2
- autocorrelation 2
- AR 2
- unit-root 2
- autoregression 2
- moving-average 2
- lag 2
- complex-number 2
- exponential-function 2
- mean-shift 2
- z-value 2
- central-limit-theorem 2
- Gaussian-distribution 2
- cumulative-distribution-function 2
- probability-mass-function 2
- standard-error 2
- sampling-distribution 2
- statistical-inference 2
- likelihood-function 2
- log-likelihood 2
- time-series 2
- kalman-filter 2
- expectation 2
- logarithm 2
- limits 1
- continuity 1
- discontinuity 1
- rate-of-change 1
- extremities 1
- extreme-points 1
- analysis 1
- saddle-point 1
- minimum 1
- maximum 1
- concavity 1
- extreme-value-theorem 1
- equations 1
- exponential-model 1
- logistic-model 1
- Maclaurin-series 1
- local-minimum 1
- global-minimum 1
- SGD 1
- momentum 1
- adagrad 1
- adadelta 1
- adam 1
- RMS 1
- decaying-average 1
- vector-space 1
- dimensions 1
- unit-vector 1
- linear-equations 1
- linear-transformation 1
- vector-reflection 1
- householder-reflection 1
- matrix-decomposition 1
- gram-schmidt 1
- vector-projection 1
- identity-matrix 1
- PCA 1
- SVD 1
- singular-value-decomposition 1
- full-rank-matrix 1
- condition-number 1
- null-space 1
- linear-combination 1
- Schwarz’s-theorem 1
- nonlinear-least-squares 1
- Gauss-Newton-method 1
- Jacobian 1
- Levenberg–Marquardt-algorithm 1
- linear-programming 1
- constraints 1
- objective-function 1
- maximization 1
- tangent-line 1
- Wolfe-conditions 1
- Armijo-rule 1
- search-direction 1
- cauchy-point 1
- dogleg-method 1
- line-search 1
- supervised-machine-learning 1
- svm 1
- function-minimization 1
- rank-one-update 1
- jacobian 1
- Sherman-Morrison-formula 1
- cross-product 1
- secant-method 1
- bisection-method 1
- Brent's-method 1
- Newton's-method 1
- zero-finding 1
- false-position-method 1
- Broyden's-method 1
- inverse-quadratic-interpolation 1
- scatterplot 1
- least-squares 1
- weighting 1
- non-parametric-regression 1
- local-regression 1
- integral 1
- area 1
- net-change 1
- Riemann-sum 1
- fundamental-theorem-of-calculus 1
- improper-integrals 1
- pdf 1
- moment 1
- mgf 1
- probability-density-distribution 1
- skewness 1
- kurtosis 1
- expected-value 1
- Hoeffding's-inequality 1
- Markov's-inequality 1
- Chebyshev's-inequality 1
- Chernoff-bound 1
- moment-generating-function 1
- Weak-law-of-large-numbers 1
- Bernoulli-distribution 1
- principal-components 1
- pca 1
- matrix-transformation 1
- feature-extraction 1
- eigendecomposition 1
- singular-vectors 1
- singular-values 1
- scatter-matrix 1
- truncated-svd 1
- linear-model 1
- type-1-error 1
- type-2-error 1
- p-value-hacking 1
- alternative-hypothesis 1
- power-analysis 1
- sample-size 1
- family-wise-error-rate 1
- multiple-testing 1
- Benjamini–Hochberg-procedure 1
- FWER 1
- Pearson's-chi-square-test 1
- z-test 1
- F-distribution 1
- Student's-distribution 1
- continuity-correction 1
- Welch's-t-test 1
- exact-Fisher's-test 1
- Barnard's-test 1
- contingency-table 1
- G-test 1
- paired-t-test 1
- McNemar's-test 1
- ANOVA 1
- analysis-of-variance 1
- F-test 1
- F-statistic 1
- Games-Howell-test 1
- ols 1
- normal-equations 1
- qr-decomposition 1
- Breusch–Pagan-test 1
- White-test 1
- nlp 1
- documents-similarity 1
- vector-magnitude 1
- sign-test 1
- Wilcoxon-signed-rank-test 1
- Mann–Whitney-U-test 1
- U-statistic 1
- change-point 1
- anomaly-detections 1
- search-method 1
- Hausdorf-metric 1
- breakpoint 1
- break-point 1
- F1-score 1
- vector-length 1
- vector-multiplication 1
- Euclidean-distance 1
- Manhattan-distance 1
- Minkowski-distance 1
- vector-norm 1
- normed-vector-space 1
- Mahalanobis-distance 1
- cosine-distance 1
- cosine-similarity 1
- Hamming-distance 1
- relative-entropy 1
- bayesian-statistics 1
- evidence-lower-bound 1
- mean-field-variational-family 1
- coordinate-ascent-variational-inference 1
- CAVI 1
- gaussian-distribution 1
- log-marginal-likelihood 1
- sparse-variational-gaussian-process 1
- variational-inference 1
- likelihood 1
- marginal-likelihood 1
- whitened-parameterization 1
- parameterization-trick 1
- binary-classification 1
- logistic-function 1
- odds 1
- log-odds 1
- regularization 1
- overfittings 1
- L2-regularization 1
- L1-regularization 1
- Ridge-regularization 1
- Lasso-regularization 1
- Ridge-regression 1
- Lasso-regression 1
- Elastic-Net 1
- logistic-regression 1
- activation-function 1
- neuron 1
- perceptron 1
- deep-neural-networks 1
- hidden-layer 1
- node 1
- backpropagation 1
- ReLU 1
- bias 1
- vanishing-gradient 1
- saturated-neuron 1
- dying-ReLU 1
- ELU 1
- softmax 1
- feed-forward-networks 1
- mse 1
- l2 1
- mean-squared-error 1
- quadratic-loss 1
- l1 1
- mean-absolute-error 1
- mae 1
- m-estimator 1
- huber-loss 1
- bisquare-loss-function 1
- entropy 1
- cross-entropy 1
- Durbin–Watson-test 1
- Durbin–Watson-statistic 1
- white-noise 1
- Breusch–Godfrey-test 1
- ADF 1
- KPSS 1
- Kwiatkowski-Phillips-Schmidt-Shin-test 1
- Dickey-Fuller-test 1
- MA 1
- ARMA 1
- random-walk 1
- differencing 1
- ACF 1
- autocorrelation-function 1
- PACF 1
- partial-autocorrelation-function 1
- SARIMA 1
- SARIMAX 1
- decision-tree 1
- bagging 1
- random-forest 1
- bootstrap-aggregation 1
- ensemble-techniques 1
- extremely-randomized-trees 1
- extra-trees 1
- mean-decrease-in-impurity 1
- MDI 1
- AdaBoost 1
- boosting 1
- gradient-boosting 1
- histogram-based-gradient-boosting 1
- xgboost 1
- extreme-gradient-boosting 1
- gradient-boosting-machine 1
- GBM 1
- LightGBM 1
- gradient-based-one-side-sampling 1
- GOSS 1
- exclusive-feature-bundling 1
- EFB 1
- Euler-number 1
- real-number 1
- trigonometric-function 1
- Euler's-formula 1
- vector-rotation 1
- Euler-identity 1
- missing-values 1
- spline 1
- nearest-neighbour 1
- Vandermonde-matrix 1
- Lagrange-multipliers 1
- Newton's-divided-differences 1
- cubic-spline 1
- Bezier-curve 1
- periodic-function 1
- sine 1
- fundamental-period 1
- harmonic-frequency 1
- RBF 1
- radial-basis-function-kernel 1
- squared-exponential-kernel 1
- matern-kernel 1
- rational-quadratic-kernel 1
- periodic-kernel 1
- linear-kernel 1
- dot-product-kernel 1
- kernel-density-estimation 1
- KDE 1
- kernel-bandwidth 1
- segmentation 1
- unsupervised-learning 1
- hierarchical-clustering 1
- agglomerative-clustering 1
- dendrogram 1
- k-means 1
- ward-linkage 1
- affinity-propagation 1
- spectral-clustering 1
- DBSCAN 1
- HDBSCAN 1
- OPTICS 1
- BIRCH 1
- unsupervised-machine-learning 1
- semi-supervised-machine-learning 1
- anomaly-detection 1
- novelty-detection 1
- local-outlier-factor 1
- LOF 1
- GLOSH 1
- isolation-forest 1
- LLM 1
- z-table 1
- six-sigma 1
- CDF 1
- distribution 1
- inference 1
- student-distribution 1
- PMF 1
- chi-square-distribution 1
- Bernoulli-trial 1
- geometric-distribution 1
- exponential-distribution 1
- poisson-distribution 1
- uniform-distribution 1
- negative-binomial-distribution 1
- beta-distribution 1
- gamma-function 1
- Central-Limit-Theorem 1
- CLT 1
- standard-deviation 1
- sample-variance 1
- sample-mean 1
- unbiased-estimate 1
- confidence-level 1
- t-value 1
- t-statistic 1
- z-statistic 1
- goodness-of-fit 1
- joint-probability 1
- likelihood-ratio 1
- likelihood-ratio-test 1
- Wald-test 1
- Wald-statistic 1
- chi-squre-distribution 1
- Lagrange-multiplier-test 1
- score-test 1
- trend 1
- seasonal-component 1
- level-component 1
- Holt-linear-trend 1
- damped-trend 1
- Holt-Winters-seasonal-method 1
- Holt-linear-trend-method 1
- state-space-model 1
- additive-error 1
- multiplicative-error 1
- non-linear-regression 1
- ARIMA 1
- exponential-smooting 1
- Kalman-filter 1
- exponential-smoothing 1
- Rauch-Tung-Striebel 1
- prior-probability 1
- Bayes-rule 1
- posterior-probability 1
- MAP 1
- maximum-a-posteriori 1
- conjugate-prior-distribution 1
- expected-a-posteriori 1
- EAP 1
- marginal-probability 1
- hidden-variables 1
- maximum-likelihood 1
- Evidence-Lower-Bound 1
- Jensen-inequality 1
- convexity 1
- correlation 1
- correlation-coefficients 1
- covariation 1
- Pearson's-correlation-coefficient 1
- coefficient-of-determination 1
- r-squared 1
- adjusted-r-squared 1
- regression-model 1
- variance-of-coefficients 1
- multiple-regression 1
- multivariable-regression 1
- gaussian-noise 1
- multicollinearity 1
- correlation-coefficient 1
- Cochrane–Orcutt-estimation 1
- Prais–Winsten-estimation 1
- weighted-least-squares 1
- WLS 1
- generalized-least-squares 1
- GLS 1
- feasible-generalized-least-squares 1
- FGLS 1
- clustering 1
- computer-vision 1
- CV 1
- NLP 1
- function 1
- non-linearity 1
- law-of-large-numbers 1
- convergence 1
- censoring 1
- Kaplan-Meier-Estimator 1
- density-function 1
- life-table 1
- delta-method 1
- partial-likelihood 1
- Weibull-distribution 1
- accelerated-failure-time 1
- GLM 1
- OLS 1
- iteratively-reweighted-least-squares 1
- IRLS 1
- generalized-estimating-equations 1
- GEE 1
- information-matrix 1
- sandwitch-estimator 1
- Fisher-information 1
- Cramér-Rao-lower-bound 1
statistics
Survival Analysis
Survival Analysis, also known as Time-to-Event Analysis, is a specialized branch of statistics focused on analyzing the time until an event of interest occur...
Machine Learning
Machine Learning (ML) is the field of study dedicated to building algorithms that allow computers to learn patterns from data without being explicitly progra...
Regression Analysis
Statistical inference allows making educated guesses about a population using a sample. Regression analysis extends this idea by providing means to understan...
Bootstrapping
Introduction
Statistical Inference
Introduction
Probability Distributions
A probability distribution is a function that describes the likelihood of a random variable taking on certain values. Think of it as a model or blueprint for...
Main Probability Concepts
Introduction
Descriptive Statistics
Introduction
Probability & Statistics
Introduction
Normal Distribution
Introduction
normal-distribution
Generalized Linear Models
Generalized Linear Models (GLMs) provide a powerful and flexible framework for modeling a response variable ($Y$) that is non-normally distributed or related...
Delta Method
The Big Idea: Linearizing the Non-Linear
Linear regression
Linear regression is the simplest and most widely used form of regression analysis. It assumes a straight-line relationship between a dependent variable ($Y$...
Sampling Distribution
Introduction
Probability Distributions
A probability distribution is a function that describes the likelihood of a random variable taking on certain values. Think of it as a model or blueprint for...
Normal Distribution
Introduction
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
Hypothesis test non-parametric statistics
This is an overview of the most commonly used non-parametric statistics in hypothesis testing explaining when to use each. Unlike the parametric counterparts...
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
derivative
Fisher Information
Fisher Information, denoted $I(\theta)$, is a foundational concept in mathematical statistics. It quantifies the amount of information that an observable ran...
Kaplan-Meier Estimator
The Kaplan-Meier (KM) Estimator is the standard way to estimate the survival function $S(t)$ from observed data.
Understanding integrals
Building intuition
Mean value theorem
The mean value theorem states that if a function is continuous and differentiable between two points then there exists a point between them where the tangent...
Newton’s method
Similarly to gradient descent, Newton’s method is used for finding a minimum of a function through iterations. Unlike gradient descent however, it uses a sec...
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
Taylor series
One usually neglected topic in calculus is understanding Taylor series. At first glance they seem to be rather impractical, however, they form building block...
Analyzing functions
Given an equation, understanding of derivatives allows analyzing functions beyond simple plotting and eyeballing them. Below I will describe some of the tech...
Understanding derivatives
Building intuition
loss-function
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
Regularization
Regularization is a process of simplifying complex machine learning models in order to prevent overfitting. It may be applied explicitly by adding a penalty ...
Quasi-Newton methods
The classical Newton’s method for optimization is known for being computationally expensive when it comes to calculating the gradient of the Hessian matrix a...
Conjugate gradient
Conjugate gradient methods are well suited for solving both linear and non-linear systems of equations.
Trust region
Trust region is a general strategy of optimization of an objective function by constructing an approximation of the function around a certain point and measu...
Line search
Generally speaking, line search is one of the two (the other is trust region) major strategies in non-linear optimization. Line search methods first determin...
Nedler-Mead algorithm
The Nedlear-Mead optimization algorithm employs the concept of simplex - a special type of polytope in $n$-dimensional space which has $n+1$ vertices, that i...
Newton’s method
Similarly to gradient descent, Newton’s method is used for finding a minimum of a function through iterations. Unlike gradient descent however, it uses a sec...
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
vector
Fourier series
A Fourier series is a way to approximate a periodic function as the sum of simple sine and cosine functions which makes it useful for analyzing and synthesiz...
Imaginary number and rotation of vectors
Introduction
Distance metrics
In machine learning the distance metrics are used as a measure of similarity between two datapoints - smaller distance means more similarity. They are utiliz...
Text similarity
Doing text analysis you may encounter a case when you need to select the most similar text or a group of text to another chosen text. One way to do it is to...
Principal component analysis and singular value decomposition
Principal component analysis
Matrix properties
Below I’ve provided intuition behind some of the properties of matrices, which are useful to understand in order to apply them in more complex algorithms. R...
Eigenvectors and eigenvalues
In machine learning eigenvectors and eigenvalues is one of the heavily used concepts, which among other things is fundamental for dimension reduction and fin...
Change of basis
Let’s recall that the basis is a set of linearly independent vectors which form the full span of vector space. With regard to matrices the basis can be viewe...
Linear transformations
Linear algebra is all about making transformations of linear combinations of numbers, called vectors, and arrays of vectors, called matrices. The elements of...
variance
Fisher Information
Fisher Information, denoted $I(\theta)$, is a foundational concept in mathematical statistics. It quantifies the amount of information that an observable ran...
Delta Method
The Big Idea: Linearizing the Non-Linear
Correlation and variance in linear regression
What is covariance?
Jensen’s Inequality
The Statement of the Inequality
Sampling Distribution
Introduction
Heteroscedasticity
What is heteroscedasticity
Degrees of freedom in statistics
In statistics degrees of freedom is the number of independent elements forming a final statistic which are free to vary without violation of an imposed const...
Principal component analysis and singular value decomposition
Principal component analysis
Probability bounds
When applying a model to a certain process it is mostly impossible to be fully certain about the values of the parameters, the output of the process, and the...
linear-regression
Generalized Linear Models
Generalized Linear Models (GLMs) provide a powerful and flexible framework for modeling a response variable ($Y$) that is non-normally distributed or related...
Linear regression
Linear regression is the simplest and most widely used form of regression analysis. It assumes a straight-line relationship between a dependent variable ($Y$...
Regression Analysis
Statistical inference allows making educated guesses about a population using a sample. Regression analysis extends this idea by providing means to understan...
Linear mixed model
This type of model is suited when observations are not completely independent. Perhaps we measured the same individuals multiple times over a study, or we co...
Neural Networks
In this article
Logistic regression
Logistic regression is one of the simplests algorithms for binary classification, and it is based on the linear regression. In its core, it uses a linear com...
Heteroscedasticity
What is heteroscedasticity
Linear least squares
Ordinary least squares method (OLS) is a widely used method which estimates the parameters of a linear regression by minimizing the sum of the squares of the...
gradient-descent
Neural Networks
In this article
Linear least squares
Ordinary least squares method (OLS) is a widely used method which estimates the parameters of a linear regression by minimizing the sum of the squares of the...
Conjugate gradient
Conjugate gradient methods are well suited for solving both linear and non-linear systems of equations.
Line search
Generally speaking, line search is one of the two (the other is trust region) major strategies in non-linear optimization. Line search methods first determin...
Non-linear regression
Although application of linear regression is straightforward and characterized by many useful statistical properties, in the real life many processes have no...
Newton’s method
Similarly to gradient descent, Newton’s method is used for finding a minimum of a function through iterations. Unlike gradient descent however, it uses a sec...
probability-density-function
Generalized Linear Models
Generalized Linear Models (GLMs) provide a powerful and flexible framework for modeling a response variable ($Y$) that is non-normally distributed or related...
Bayesian inference
In statistics, Bayesian inference is a powerful method for updating a probability estimate when new data becomes available. This process can be seen as an el...
Probability Distributions
A probability distribution is a function that describes the likelihood of a random variable taking on certain values. Think of it as a model or blueprint for...
Normal Distribution
Introduction
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
Understanding integrals
Building intuition
hypothesis-testing
Linear regression
Linear regression is the simplest and most widely used form of regression analysis. It assumes a straight-line relationship between a dependent variable ($Y$...
Probability & Statistics
Introduction
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
Hypothesis test non-parametric statistics
This is an overview of the most commonly used non-parametric statistics in hypothesis testing explaining when to use each. Unlike the parametric counterparts...
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
regression
Machine Learning
Machine Learning (ML) is the field of study dedicated to building algorithms that allow computers to learn patterns from data without being explicitly progra...
Correlation and variance in linear regression
What is covariance?
Time Series Analysis
Time series analysis is a specialized field of statistics that focuses on data points collected over time. From daily stock prices and hourly temperature rea...
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
Neural Networks
In this article
Heteroscedasticity
What is heteroscedasticity
differentiation
Mean value theorem
The mean value theorem states that if a function is continuous and differentiable between two points then there exists a point between them where the tangent...
Taylor series
One usually neglected topic in calculus is understanding Taylor series. At first glance they seem to be rather impractical, however, they form building block...
Differential equations
Differential equation is an equation with a function and at least one of its derivatives. Here is an example:
Analyzing functions
Given an equation, understanding of derivatives allows analyzing functions beyond simple plotting and eyeballing them. Below I will describe some of the tech...
Understanding derivatives
Building intuition
matrix
Principal component analysis and singular value decomposition
Principal component analysis
Matrix properties
Below I’ve provided intuition behind some of the properties of matrices, which are useful to understand in order to apply them in more complex algorithms. R...
Eigenvectors and eigenvalues
In machine learning eigenvectors and eigenvalues is one of the heavily used concepts, which among other things is fundamental for dimension reduction and fin...
Change of basis
Let’s recall that the basis is a set of linearly independent vectors which form the full span of vector space. With regard to matrices the basis can be viewe...
Linear transformations
Linear algebra is all about making transformations of linear combinations of numbers, called vectors, and arrays of vectors, called matrices. The elements of...
nonlinear-regression
Conjugate gradient
Conjugate gradient methods are well suited for solving both linear and non-linear systems of equations.
Trust region
Trust region is a general strategy of optimization of an objective function by constructing an approximation of the function around a certain point and measu...
Line search
Generally speaking, line search is one of the two (the other is trust region) major strategies in non-linear optimization. Line search methods first determin...
Nedler-Mead algorithm
The Nedlear-Mead optimization algorithm employs the concept of simplex - a special type of polytope in $n$-dimensional space which has $n+1$ vertices, that i...
Non-linear regression
Although application of linear regression is straightforward and characterized by many useful statistical properties, in the real life many processes have no...
probability
Survival Analysis
Survival Analysis, also known as Time-to-Event Analysis, is a specialized branch of statistics focused on analyzing the time until an event of interest occur...
Probability Distributions
A probability distribution is a function that describes the likelihood of a random variable taking on certain values. Think of it as a model or blueprint for...
Probability & Statistics
Introduction
Normal Distribution
Introduction
Understanding integrals
Building intuition
confidence-interval
Cox Proportional Hazards Model
While the Kaplan–Meier estimator tells us how survival probability changes over time, it doesn’t explain why — it cannot handle predictors or covariates.
Bootstrapping
Introduction
Sampling Distribution
Introduction
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
Probability bounds
When applying a model to a certain process it is mostly impossible to be fully certain about the values of the parameters, the output of the process, and the...
MLE
Fisher Information
Fisher Information, denoted $I(\theta)$, is a foundational concept in mathematical statistics. It quantifies the amount of information that an observable ran...
Generalized Linear Models
Generalized Linear Models (GLMs) provide a powerful and flexible framework for modeling a response variable ($Y$) that is non-normally distributed or related...
Expectation Maximization
The Expectation-Maximization (EM) algorithm is a powerful iterative method used in statistics to find the maximum likelihood or maximum a posteriori (MAP) es...
Bayesian inference
In statistics, Bayesian inference is a powerful method for updating a probability estimate when new data becomes available. This process can be seen as an el...
Maximum Likelihood Estimation
In statistical inference, the likelihood function is used to estimate the goodness of fit of the parameters to the observed data. The maximum likelihood esti...
calculus
Differential equations
Differential equation is an equation with a function and at least one of its derivatives. Here is an example:
Analyzing functions
Given an equation, understanding of derivatives allows analyzing functions beyond simple plotting and eyeballing them. Below I will describe some of the tech...
Understanding derivatives
Building intuition
Limits and continuity
Let’s build an idea about formal definition of limit.
second-derivative
Quasi-Newton methods
The classical Newton’s method for optimization is known for being computationally expensive when it comes to calculating the gradient of the Hessian matrix a...
Newton’s method
Similarly to gradient descent, Newton’s method is used for finding a minimum of a function through iterations. Unlike gradient descent however, it uses a sec...
Taylor series
One usually neglected topic in calculus is understanding Taylor series. At first glance they seem to be rather impractical, however, they form building block...
Analyzing functions
Given an equation, understanding of derivatives allows analyzing functions beyond simple plotting and eyeballing them. Below I will describe some of the tech...
Taylor-series
Imaginary number and rotation of vectors
Introduction
Moment generating functions
In general moment generating functions (MGF) are used as an alternative for using probability density distribution. The name implies that with MGF it is poss...
Mean value theorem
The mean value theorem states that if a function is continuous and differentiable between two points then there exists a point between them where the tangent...
Non-linear regression
Although application of linear regression is straightforward and characterized by many useful statistical properties, in the real life many processes have no...
mean
Sampling Distribution
Introduction
Change point analysis
Change point analysis is basically used for determining whether and where an ordered series of values (usually time series) changed their behaviour. The chan...
Degrees of freedom in statistics
In statistics degrees of freedom is the number of independent elements forming a final statistic which are free to vary without violation of an imposed const...
Probability bounds
When applying a model to a certain process it is mostly impossible to be fully certain about the values of the parameters, the output of the process, and the...
residuals
Linear regression
Linear regression is the simplest and most widely used form of regression analysis. It assumes a straight-line relationship between a dependent variable ($Y$...
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
Heteroscedasticity
What is heteroscedasticity
Degrees of freedom in statistics
In statistics degrees of freedom is the number of independent elements forming a final statistic which are free to vary without violation of an imposed const...
probability-distribution
Bayesian inference
In statistics, Bayesian inference is a powerful method for updating a probability estimate when new data becomes available. This process can be seen as an el...
Maximum Likelihood Estimation
In statistical inference, the likelihood function is used to estimate the goodness of fit of the parameters to the observed data. The maximum likelihood esti...
Probability Distributions
A probability distribution is a function that describes the likelihood of a random variable taking on certain values. Think of it as a model or blueprint for...
Normal Distribution
Introduction
sample
Bootstrapping
Introduction
Statistical Inference
Introduction
Sampling Distribution
Introduction
Probability & Statistics
Introduction
maximum-likelihood-estimation
Fisher Information
Fisher Information, denoted $I(\theta)$, is a foundational concept in mathematical statistics. It quantifies the amount of information that an observable ran...
Generalized Linear Models
Generalized Linear Models (GLMs) provide a powerful and flexible framework for modeling a response variable ($Y$) that is non-normally distributed or related...
Parametric Survival Models
Parametric survival models assume that survival times follow a known probability distribution. Unlike non-parametric methods (Kaplan–Meier) or semi-parametri...
Bayesian inference
In statistics, Bayesian inference is a powerful method for updating a probability estimate when new data becomes available. This process can be seen as an el...
slope
Mean value theorem
The mean value theorem states that if a function is continuous and differentiable between two points then there exists a point between them where the tangent...
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
Understanding derivatives
Building intuition
gradient
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
Understanding derivatives
Building intuition
machine-learning
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
penalization
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
Change point analysis
Change point analysis is basically used for determining whether and where an ordered series of values (usually time series) changed their behaviour. The chan...
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
learning-rate
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
Line search
Generally speaking, line search is one of the two (the other is trust region) major strategies in non-linear optimization. Line search methods first determin...
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
span
Matrix properties
Below I’ve provided intuition behind some of the properties of matrices, which are useful to understand in order to apply them in more complex algorithms. R...
Change of basis
Let’s recall that the basis is a set of linearly independent vectors which form the full span of vector space. With regard to matrices the basis can be viewe...
Linear transformations
Linear algebra is all about making transformations of linear combinations of numbers, called vectors, and arrays of vectors, called matrices. The elements of...
svd
Linear least squares
Ordinary least squares method (OLS) is a widely used method which estimates the parameters of a linear regression by minimizing the sum of the squares of the...
Principal component analysis and singular value decomposition
Principal component analysis
QR decomposition
QR decomposition may be a numerically stable algorithm for solving systems of linear equations. Similar to singular value decomposition it provides mechanism...
dot-product
Vector projection
In order to understand more advanced concepts from vector operations it is good to have an intuition on such terms as vector projection. Below is a two-dimen...
Text similarity
Doing text analysis you may encounter a case when you need to select the most similar text or a group of text to another chosen text. One way to do it is to...
QR decomposition
QR decomposition may be a numerically stable algorithm for solving systems of linear equations. Similar to singular value decomposition it provides mechanism...
kernel
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
Matrix properties
Below I’ve provided intuition behind some of the properties of matrices, which are useful to understand in order to apply them in more complex algorithms. R...
symmetric-matrix
Principal component analysis and singular value decomposition
Principal component analysis
Newton’s method
Similarly to gradient descent, Newton’s method is used for finding a minimum of a function through iterations. Unlike gradient descent however, it uses a sec...
Matrix properties
Below I’ve provided intuition behind some of the properties of matrices, which are useful to understand in order to apply them in more complex algorithms. R...
positive-definite-matrix
Principal component analysis and singular value decomposition
Principal component analysis
Newton’s method
Similarly to gradient descent, Newton’s method is used for finding a minimum of a function through iterations. Unlike gradient descent however, it uses a sec...
Matrix properties
Below I’ve provided intuition behind some of the properties of matrices, which are useful to understand in order to apply them in more complex algorithms. R...
hessian
Fisher Information
Fisher Information, denoted $I(\theta)$, is a foundational concept in mathematical statistics. It quantifies the amount of information that an observable ran...
Quasi-Newton methods
The classical Newton’s method for optimization is known for being computationally expensive when it comes to calculating the gradient of the Hessian matrix a...
Newton’s method
Similarly to gradient descent, Newton’s method is used for finding a minimum of a function through iterations. Unlike gradient descent however, it uses a sec...
steepest-descent
Conjugate gradient
Conjugate gradient methods are well suited for solving both linear and non-linear systems of equations.
Trust region
Trust region is a general strategy of optimization of an objective function by constructing an approximation of the function around a certain point and measu...
Line search
Generally speaking, line search is one of the two (the other is trust region) major strategies in non-linear optimization. Line search methods first determin...
classification
Machine Learning
Machine Learning (ML) is the field of study dedicated to building algorithms that allow computers to learn patterns from data without being explicitly progra...
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
Support vector machine
Support vector machine (SVM) is mainly used for building classification models in supervised machine learning, however the ideas employed by SVM can be also ...
significance-test
Hypothesis test non-parametric statistics
This is an overview of the most commonly used non-parametric statistics in hypothesis testing explaining when to use each. Unlike the parametric counterparts...
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
z-score
Sampling Distribution
Introduction
Normal Distribution
Introduction
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
binomial-distribution
Probability Distributions
A probability distribution is a function that describes the likelihood of a random variable taking on certain values. Think of it as a model or blueprint for...
Hypothesis test non-parametric statistics
This is an overview of the most commonly used non-parametric statistics in hypothesis testing explaining when to use each. Unlike the parametric counterparts...
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
cosine
Fourier series
A Fourier series is a way to approximate a periodic function as the sum of simple sine and cosine functions which makes it useful for analyzing and synthesiz...
Vector projection
In order to understand more advanced concepts from vector operations it is good to have an intuition on such terms as vector projection. Below is a two-dimen...
Text similarity
Doing text analysis you may encounter a case when you need to select the most similar text or a group of text to another chosen text. One way to do it is to...
conditional-probability
Bayesian inference
In statistics, Bayesian inference is a powerful method for updating a probability estimate when new data becomes available. This process can be seen as an el...
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
Variational inference
Variational inference is a technique which is usually employed for approximating complex models, where computation of the exact conditional distribution is i...
covariance
Time Series Smoothing
In time series analysis, data often contains a high degree of noise—random, short-term fluctuations that can obscure the underlying patterns. Smoothing is th...
Kernels overview
In this article What are kernels Kernel density estimation Kernels in Gaussian process Squared exponential kernel Matern kernel ...
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
Probability Distributions
A probability distribution is a function that describes the likelihood of a random variable taking on certain values. Think of it as a model or blueprint for...
Normal Distribution
Introduction
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
error-term
Linear regression
Linear regression is the simplest and most widely used form of regression analysis. It assumes a straight-line relationship between a dependent variable ($Y$...
Correlation and variance in linear regression
What is covariance?
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
survival-function
Parametric Survival Models
Parametric survival models assume that survival times follow a known probability distribution. Unlike non-parametric methods (Kaplan–Meier) or semi-parametri...
Kaplan-Meier Estimator
The Kaplan-Meier (KM) Estimator is the standard way to estimate the survival function $S(t)$ from observed data.
Survival Analysis
Survival Analysis, also known as Time-to-Event Analysis, is a specialized branch of statistics focused on analyzing the time until an event of interest occur...
hazard-function
Parametric Survival Models
Parametric survival models assume that survival times follow a known probability distribution. Unlike non-parametric methods (Kaplan–Meier) or semi-parametri...
Cox Proportional Hazards Model
While the Kaplan–Meier estimator tells us how survival probability changes over time, it doesn’t explain why — it cannot handle predictors or covariates.
Survival Analysis
Survival Analysis, also known as Time-to-Event Analysis, is a specialized branch of statistics focused on analyzing the time until an event of interest occur...
survival-analysis
Parametric Survival Models
Parametric survival models assume that survival times follow a known probability distribution. Unlike non-parametric methods (Kaplan–Meier) or semi-parametri...
Cox Proportional Hazards Model
While the Kaplan–Meier estimator tells us how survival probability changes over time, it doesn’t explain why — it cannot handle predictors or covariates.
Kaplan-Meier Estimator
The Kaplan-Meier (KM) Estimator is the standard way to estimate the survival function $S(t)$ from observed data.
derivation
Analyzing functions
Given an equation, understanding of derivatives allows analyzing functions beyond simple plotting and eyeballing them. Below I will describe some of the tech...
Understanding derivatives
Building intuition
functions
Taylor series
One usually neglected topic in calculus is understanding Taylor series. At first glance they seem to be rather impractical, however, they form building block...
Analyzing functions
Given an equation, understanding of derivatives allows analyzing functions beyond simple plotting and eyeballing them. Below I will describe some of the tech...
integration
Understanding integrals
Building intuition
Differential equations
Differential equation is an equation with a function and at least one of its derivatives. Here is an example:
polynomial
Interpolation
Generally speaking, interpolation is a way of generating new data points which exactly fit into some given set of points. It is used for approximation of com...
Taylor series
One usually neglected topic in calculus is understanding Taylor series. At first glance they seem to be rather impractical, however, they form building block...
basis
Eigenvectors and eigenvalues
In machine learning eigenvectors and eigenvalues is one of the heavily used concepts, which among other things is fundamental for dimension reduction and fin...
Linear transformations
Linear algebra is all about making transformations of linear combinations of numbers, called vectors, and arrays of vectors, called matrices. The elements of...
inverse
Principal component analysis and singular value decomposition
Principal component analysis
Linear transformations
Linear algebra is all about making transformations of linear combinations of numbers, called vectors, and arrays of vectors, called matrices. The elements of...
determinant
Eigenvectors and eigenvalues
In machine learning eigenvectors and eigenvalues is one of the heavily used concepts, which among other things is fundamental for dimension reduction and fin...
Linear transformations
Linear algebra is all about making transformations of linear combinations of numbers, called vectors, and arrays of vectors, called matrices. The elements of...
matrix-diagonalization
Eigenvectors and eigenvalues
In machine learning eigenvectors and eigenvalues is one of the heavily used concepts, which among other things is fundamental for dimension reduction and fin...
Change of basis
Let’s recall that the basis is a set of linearly independent vectors which form the full span of vector space. With regard to matrices the basis can be viewe...
column-space
Matrix properties
Below I’ve provided intuition behind some of the properties of matrices, which are useful to understand in order to apply them in more complex algorithms. R...
Eigenvectors and eigenvalues
In machine learning eigenvectors and eigenvalues is one of the heavily used concepts, which among other things is fundamental for dimension reduction and fin...
principal-component-analysis
Principal component analysis and singular value decomposition
Principal component analysis
Eigenvectors and eigenvalues
In machine learning eigenvectors and eigenvalues is one of the heavily used concepts, which among other things is fundamental for dimension reduction and fin...
rank
Hypothesis test non-parametric statistics
This is an overview of the most commonly used non-parametric statistics in hypothesis testing explaining when to use each. Unlike the parametric counterparts...
Matrix properties
Below I’ve provided intuition behind some of the properties of matrices, which are useful to understand in order to apply them in more complex algorithms. R...
ill-conditioned-matrix
Linear least squares
Ordinary least squares method (OLS) is a widely used method which estimates the parameters of a linear regression by minimizing the sum of the squares of the...
Matrix properties
Below I’ve provided intuition behind some of the properties of matrices, which are useful to understand in order to apply them in more complex algorithms. R...
taylor-series
Delta Method
The Big Idea: Linearizing the Non-Linear
Newton’s method
Similarly to gradient descent, Newton’s method is used for finding a minimum of a function through iterations. Unlike gradient descent however, it uses a sec...
taylor-expansion
Trust region
Trust region is a general strategy of optimization of an objective function by constructing an approximation of the function around a certain point and measu...
Newton’s method
Similarly to gradient descent, Newton’s method is used for finding a minimum of a function through iterations. Unlike gradient descent however, it uses a sec...
Hessian
Non-linear regression
Although application of linear regression is straightforward and characterized by many useful statistical properties, in the real life many processes have no...
Newton’s method
Similarly to gradient descent, Newton’s method is used for finding a minimum of a function through iterations. Unlike gradient descent however, it uses a sec...
simplex
Nedler-Mead algorithm
The Nedlear-Mead optimization algorithm employs the concept of simplex - a special type of polytope in $n$-dimensional space which has $n+1$ vertices, that i...
Simplex method
Linear programming aims at finding the best outcome of mathematical problems provided constraints. The requirements for the linear programming problems are d...
secant-line
Root finding algorithms
In numeric analysis root finding is equivalent to finding zeros of a continuous function. In case of complex and composite function finding values of a varia...
Mean value theorem
The mean value theorem states that if a function is continuous and differentiable between two points then there exists a point between them where the tangent...
minimization
Trust region
Trust region is a general strategy of optimization of an objective function by constructing an approximation of the function around a certain point and measu...
Line search
Generally speaking, line search is one of the two (the other is trust region) major strategies in non-linear optimization. Line search methods first determin...
smoothing
Time Series Analysis
Time series analysis is a specialized field of statistics that focuses on data points collected over time. From daily stock prices and hourly temperature rea...
LOESS
It is quite common to have a relation between two values represented as a scatterplot. However, visual representation of a scatterplot alone can often be non...
lowess
LOESS
It is quite common to have a relation between two values represented as a scatterplot. However, visual representation of a scatterplot alone can often be non...
LOESS
It is quite common to have a relation between two values represented as a scatterplot. However, visual representation of a scatterplot alone can often be non...
outliers
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
LOESS
It is quite common to have a relation between two values represented as a scatterplot. However, visual representation of a scatterplot alone can often be non...
dimensionality-reduction
Machine Learning
Machine Learning (ML) is the field of study dedicated to building algorithms that allow computers to learn patterns from data without being explicitly progra...
Principal component analysis and singular value decomposition
Principal component analysis
pseudoinverse
Linear least squares
Ordinary least squares method (OLS) is a widely used method which estimates the parameters of a linear regression by minimizing the sum of the squares of the...
Principal component analysis and singular value decomposition
Principal component analysis
covariance-matrix
Correlation and variance in linear regression
What is covariance?
Principal component analysis and singular value decomposition
Principal component analysis
null-hypothesis
Hypothesis test non-parametric statistics
This is an overview of the most commonly used non-parametric statistics in hypothesis testing explaining when to use each. Unlike the parametric counterparts...
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
p-value
Hypothesis test non-parametric statistics
This is an overview of the most commonly used non-parametric statistics in hypothesis testing explaining when to use each. Unlike the parametric counterparts...
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
significance-level
Sampling Distribution
Introduction
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
false-discovery-rate
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
margin-of-error
Sampling Distribution
Introduction
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
degrees-of-freedom
Hypothesis test non-parametric statistics
This is an overview of the most commonly used non-parametric statistics in hypothesis testing explaining when to use each. Unlike the parametric counterparts...
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
t-test
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
t-distribution
Probability Distributions
A probability distribution is a function that describes the likelihood of a random variable taking on certain values. Think of it as a model or blueprint for...
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
heteroscedasticity
Linear regression
Linear regression is the simplest and most widely used form of regression analysis. It assumes a straight-line relationship between a dependent variable ($Y$...
Heteroscedasticity
What is heteroscedasticity
homoscedasticity
Linear regression
Linear regression is the simplest and most widely used form of regression analysis. It assumes a straight-line relationship between a dependent variable ($Y$...
Heteroscedasticity
What is heteroscedasticity
natural-language-processing
Machine Learning
Machine Learning (ML) is the field of study dedicated to building algorithms that allow computers to learn patterns from data without being explicitly progra...
Text similarity
Doing text analysis you may encounter a case when you need to select the most similar text or a group of text to another chosen text. One way to do it is to...
Levenshtein-distance
Distance metrics
In machine learning the distance metrics are used as a measure of similarity between two datapoints - smaller distance means more similarity. They are utiliz...
Text similarity
Doing text analysis you may encounter a case when you need to select the most similar text or a group of text to another chosen text. One way to do it is to...
median
Bootstrapping
Introduction
Hypothesis test non-parametric statistics
This is an overview of the most commonly used non-parametric statistics in hypothesis testing explaining when to use each. Unlike the parametric counterparts...
variation
Correlation and variance in linear regression
What is covariance?
Change point analysis
Change point analysis is basically used for determining whether and where an ordered series of values (usually time series) changed their behaviour. The chan...
cost-function
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
Change point analysis
Change point analysis is basically used for determining whether and where an ordered series of values (usually time series) changed their behaviour. The chan...
Kullback–Leibler-divergence
Variational inference
Variational inference is a technique which is usually employed for approximating complex models, where computation of the exact conditional distribution is i...
Distance metrics
In machine learning the distance metrics are used as a measure of similarity between two datapoints - smaller distance means more similarity. They are utiliz...
KL-divergence
Variational inference
Variational inference is a technique which is usually employed for approximating complex models, where computation of the exact conditional distribution is i...
Distance metrics
In machine learning the distance metrics are used as a measure of similarity between two datapoints - smaller distance means more similarity. They are utiliz...
gaussian-process
Kernels overview
In this article What are kernels Kernel density estimation Kernels in Gaussian process Squared exponential kernel Matern kernel ...
Variational inference
Variational inference is a technique which is usually employed for approximating complex models, where computation of the exact conditional distribution is i...
prior-distribution
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
Variational inference
Variational inference is a technique which is usually employed for approximating complex models, where computation of the exact conditional distribution is i...
posterior-distribution
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
Variational inference
Variational inference is a technique which is usually employed for approximating complex models, where computation of the exact conditional distribution is i...
Bayes-theorem
Bayesian inference
In statistics, Bayesian inference is a powerful method for updating a probability estimate when new data becomes available. This process can be seen as an el...
Variational inference
Variational inference is a technique which is usually employed for approximating complex models, where computation of the exact conditional distribution is i...
ELBO
Expectation Maximization
The Expectation-Maximization (EM) algorithm is a powerful iterative method used in statistics to find the maximum likelihood or maximum a posteriori (MAP) es...
Variational inference
Variational inference is a technique which is usually employed for approximating complex models, where computation of the exact conditional distribution is i...
sigmoid-function
Neural Networks
In this article
Logistic regression
Logistic regression is one of the simplests algorithms for binary classification, and it is based on the linear regression. In its core, it uses a linear com...
stationarity
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
autocorrelation
Linear regression
Linear regression is the simplest and most widely used form of regression analysis. It assumes a straight-line relationship between a dependent variable ($Y$...
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
AR
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
unit-root
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
autoregression
Time Series Analysis
Time series analysis is a specialized field of statistics that focuses on data points collected over time. From daily stock prices and hourly temperature rea...
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
moving-average
Time Series Smoothing
In time series analysis, data often contains a high degree of noise—random, short-term fluctuations that can obscure the underlying patterns. Smoothing is th...
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
lag
Time Series Smoothing
Similarly to ARIMA models the idea upon which the exponential smoothing models are built is that the most recent observations contribute to the currently obs...
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
complex-number
Fourier series
A Fourier series is a way to approximate a periodic function as the sum of simple sine and cosine functions which makes it useful for analyzing and synthesiz...
Imaginary number and rotation of vectors
Introduction
exponential-function
Fourier series
A Fourier series is a way to approximate a periodic function as the sum of simple sine and cosine functions which makes it useful for analyzing and synthesiz...
Imaginary number and rotation of vectors
Introduction
mean-shift
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
z-value
Sampling Distribution
Introduction
Normal Distribution
Introduction
central-limit-theorem
Delta Method
The Big Idea: Linearizing the Non-Linear
Normal Distribution
Introduction
Gaussian-distribution
Probability Distributions
A probability distribution is a function that describes the likelihood of a random variable taking on certain values. Think of it as a model or blueprint for...
Normal Distribution
Introduction
cumulative-distribution-function
Survival Analysis
Survival Analysis, also known as Time-to-Event Analysis, is a specialized branch of statistics focused on analyzing the time until an event of interest occur...
Normal Distribution
Introduction
probability-mass-function
Generalized Linear Models
Generalized Linear Models (GLMs) provide a powerful and flexible framework for modeling a response variable ($Y$) that is non-normally distributed or related...
Probability Distributions
A probability distribution is a function that describes the likelihood of a random variable taking on certain values. Think of it as a model or blueprint for...
standard-error
Generalized Linear Models
Generalized Linear Models (GLMs) provide a powerful and flexible framework for modeling a response variable ($Y$) that is non-normally distributed or related...
Sampling Distribution
Introduction
sampling-distribution
Bootstrapping
Introduction
Statistical Inference
Introduction
statistical-inference
Regression Analysis
Statistical inference allows making educated guesses about a population using a sample. Regression analysis extends this idea by providing means to understan...
Maximum Likelihood Estimation
In statistical inference, the likelihood function is used to estimate the goodness of fit of the parameters to the observed data. The maximum likelihood esti...
likelihood-function
Bayesian inference
In statistics, Bayesian inference is a powerful method for updating a probability estimate when new data becomes available. This process can be seen as an el...
Maximum Likelihood Estimation
In statistical inference, the likelihood function is used to estimate the goodness of fit of the parameters to the observed data. The maximum likelihood esti...
log-likelihood
Cox Proportional Hazards Model
While the Kaplan–Meier estimator tells us how survival probability changes over time, it doesn’t explain why — it cannot handle predictors or covariates.
Maximum Likelihood Estimation
In statistical inference, the likelihood function is used to estimate the goodness of fit of the parameters to the observed data. The maximum likelihood esti...
time-series
Time Series Smoothing
In time series analysis, data often contains a high degree of noise—random, short-term fluctuations that can obscure the underlying patterns. Smoothing is th...
Regression Analysis
Statistical inference allows making educated guesses about a population using a sample. Regression analysis extends this idea by providing means to understan...
kalman-filter
Expectation Maximization
The Expectation-Maximization (EM) algorithm is a powerful iterative method used in statistics to find the maximum likelihood or maximum a posteriori (MAP) es...
Time Series Smoothing
In time series analysis, data often contains a high degree of noise—random, short-term fluctuations that can obscure the underlying patterns. Smoothing is th...
expectation
Jensen’s Inequality
The Statement of the Inequality
Expectation Maximization
The Expectation-Maximization (EM) algorithm is a powerful iterative method used in statistics to find the maximum likelihood or maximum a posteriori (MAP) es...
logarithm
Parametric Survival Models
Parametric survival models assume that survival times follow a known probability distribution. Unlike non-parametric methods (Kaplan–Meier) or semi-parametri...
Kaplan-Meier Estimator
The Kaplan-Meier (KM) Estimator is the standard way to estimate the survival function $S(t)$ from observed data.
limits
Limits and continuity
Let’s build an idea about formal definition of limit.
continuity
Limits and continuity
Let’s build an idea about formal definition of limit.
discontinuity
Limits and continuity
Let’s build an idea about formal definition of limit.
rate-of-change
Understanding derivatives
Building intuition
extremities
Analyzing functions
Given an equation, understanding of derivatives allows analyzing functions beyond simple plotting and eyeballing them. Below I will describe some of the tech...
extreme-points
Analyzing functions
Given an equation, understanding of derivatives allows analyzing functions beyond simple plotting and eyeballing them. Below I will describe some of the tech...
analysis
Analyzing functions
Given an equation, understanding of derivatives allows analyzing functions beyond simple plotting and eyeballing them. Below I will describe some of the tech...
saddle-point
Analyzing functions
Given an equation, understanding of derivatives allows analyzing functions beyond simple plotting and eyeballing them. Below I will describe some of the tech...
minimum
Analyzing functions
Given an equation, understanding of derivatives allows analyzing functions beyond simple plotting and eyeballing them. Below I will describe some of the tech...
maximum
Analyzing functions
Given an equation, understanding of derivatives allows analyzing functions beyond simple plotting and eyeballing them. Below I will describe some of the tech...
concavity
Analyzing functions
Given an equation, understanding of derivatives allows analyzing functions beyond simple plotting and eyeballing them. Below I will describe some of the tech...
extreme-value-theorem
Analyzing functions
Given an equation, understanding of derivatives allows analyzing functions beyond simple plotting and eyeballing them. Below I will describe some of the tech...
equations
Differential equations
Differential equation is an equation with a function and at least one of its derivatives. Here is an example:
exponential-model
Differential equations
Differential equation is an equation with a function and at least one of its derivatives. Here is an example:
logistic-model
Differential equations
Differential equation is an equation with a function and at least one of its derivatives. Here is an example:
Maclaurin-series
Taylor series
One usually neglected topic in calculus is understanding Taylor series. At first glance they seem to be rather impractical, however, they form building block...
local-minimum
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
global-minimum
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
SGD
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
momentum
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
adagrad
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
adadelta
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
adam
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
RMS
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
decaying-average
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
vector-space
Linear transformations
Linear algebra is all about making transformations of linear combinations of numbers, called vectors, and arrays of vectors, called matrices. The elements of...
dimensions
Linear transformations
Linear algebra is all about making transformations of linear combinations of numbers, called vectors, and arrays of vectors, called matrices. The elements of...
unit-vector
Linear transformations
Linear algebra is all about making transformations of linear combinations of numbers, called vectors, and arrays of vectors, called matrices. The elements of...
linear-equations
Linear transformations
Linear algebra is all about making transformations of linear combinations of numbers, called vectors, and arrays of vectors, called matrices. The elements of...
linear-transformation
Change of basis
Let’s recall that the basis is a set of linearly independent vectors which form the full span of vector space. With regard to matrices the basis can be viewe...
vector-reflection
QR decomposition
QR decomposition may be a numerically stable algorithm for solving systems of linear equations. Similar to singular value decomposition it provides mechanism...
householder-reflection
QR decomposition
QR decomposition may be a numerically stable algorithm for solving systems of linear equations. Similar to singular value decomposition it provides mechanism...
matrix-decomposition
QR decomposition
QR decomposition may be a numerically stable algorithm for solving systems of linear equations. Similar to singular value decomposition it provides mechanism...
gram-schmidt
QR decomposition
QR decomposition may be a numerically stable algorithm for solving systems of linear equations. Similar to singular value decomposition it provides mechanism...
vector-projection
QR decomposition
QR decomposition may be a numerically stable algorithm for solving systems of linear equations. Similar to singular value decomposition it provides mechanism...
identity-matrix
Eigenvectors and eigenvalues
In machine learning eigenvectors and eigenvalues is one of the heavily used concepts, which among other things is fundamental for dimension reduction and fin...
PCA
Eigenvectors and eigenvalues
In machine learning eigenvectors and eigenvalues is one of the heavily used concepts, which among other things is fundamental for dimension reduction and fin...
SVD
Eigenvectors and eigenvalues
In machine learning eigenvectors and eigenvalues is one of the heavily used concepts, which among other things is fundamental for dimension reduction and fin...
singular-value-decomposition
Eigenvectors and eigenvalues
In machine learning eigenvectors and eigenvalues is one of the heavily used concepts, which among other things is fundamental for dimension reduction and fin...
full-rank-matrix
Matrix properties
Below I’ve provided intuition behind some of the properties of matrices, which are useful to understand in order to apply them in more complex algorithms. R...
condition-number
Matrix properties
Below I’ve provided intuition behind some of the properties of matrices, which are useful to understand in order to apply them in more complex algorithms. R...
null-space
Matrix properties
Below I’ve provided intuition behind some of the properties of matrices, which are useful to understand in order to apply them in more complex algorithms. R...
linear-combination
Matrix properties
Below I’ve provided intuition behind some of the properties of matrices, which are useful to understand in order to apply them in more complex algorithms. R...
Schwarz’s-theorem
Newton’s method
Similarly to gradient descent, Newton’s method is used for finding a minimum of a function through iterations. Unlike gradient descent however, it uses a sec...
nonlinear-least-squares
Non-linear regression
Although application of linear regression is straightforward and characterized by many useful statistical properties, in the real life many processes have no...
Gauss-Newton-method
Non-linear regression
Although application of linear regression is straightforward and characterized by many useful statistical properties, in the real life many processes have no...
Jacobian
Non-linear regression
Although application of linear regression is straightforward and characterized by many useful statistical properties, in the real life many processes have no...
Levenberg–Marquardt-algorithm
Non-linear regression
Although application of linear regression is straightforward and characterized by many useful statistical properties, in the real life many processes have no...
linear-programming
Simplex method
Linear programming aims at finding the best outcome of mathematical problems provided constraints. The requirements for the linear programming problems are d...
constraints
Simplex method
Linear programming aims at finding the best outcome of mathematical problems provided constraints. The requirements for the linear programming problems are d...
objective-function
Simplex method
Linear programming aims at finding the best outcome of mathematical problems provided constraints. The requirements for the linear programming problems are d...
maximization
Simplex method
Linear programming aims at finding the best outcome of mathematical problems provided constraints. The requirements for the linear programming problems are d...
tangent-line
Mean value theorem
The mean value theorem states that if a function is continuous and differentiable between two points then there exists a point between them where the tangent...
Wolfe-conditions
Line search
Generally speaking, line search is one of the two (the other is trust region) major strategies in non-linear optimization. Line search methods first determin...
Armijo-rule
Line search
Generally speaking, line search is one of the two (the other is trust region) major strategies in non-linear optimization. Line search methods first determin...
search-direction
Line search
Generally speaking, line search is one of the two (the other is trust region) major strategies in non-linear optimization. Line search methods first determin...
cauchy-point
Trust region
Trust region is a general strategy of optimization of an objective function by constructing an approximation of the function around a certain point and measu...
dogleg-method
Trust region
Trust region is a general strategy of optimization of an objective function by constructing an approximation of the function around a certain point and measu...
line-search
Trust region
Trust region is a general strategy of optimization of an objective function by constructing an approximation of the function around a certain point and measu...
supervised-machine-learning
Support vector machine
Support vector machine (SVM) is mainly used for building classification models in supervised machine learning, however the ideas employed by SVM can be also ...
svm
Support vector machine
Support vector machine (SVM) is mainly used for building classification models in supervised machine learning, however the ideas employed by SVM can be also ...
function-minimization
Conjugate gradient
Conjugate gradient methods are well suited for solving both linear and non-linear systems of equations.
rank-one-update
Quasi-Newton methods
The classical Newton’s method for optimization is known for being computationally expensive when it comes to calculating the gradient of the Hessian matrix a...
jacobian
Quasi-Newton methods
The classical Newton’s method for optimization is known for being computationally expensive when it comes to calculating the gradient of the Hessian matrix a...
Sherman-Morrison-formula
Quasi-Newton methods
The classical Newton’s method for optimization is known for being computationally expensive when it comes to calculating the gradient of the Hessian matrix a...
cross-product
Quasi-Newton methods
The classical Newton’s method for optimization is known for being computationally expensive when it comes to calculating the gradient of the Hessian matrix a...
secant-method
Root finding algorithms
In numeric analysis root finding is equivalent to finding zeros of a continuous function. In case of complex and composite function finding values of a varia...
bisection-method
Root finding algorithms
In numeric analysis root finding is equivalent to finding zeros of a continuous function. In case of complex and composite function finding values of a varia...
Brent's-method
Root finding algorithms
In numeric analysis root finding is equivalent to finding zeros of a continuous function. In case of complex and composite function finding values of a varia...
Newton's-method
Root finding algorithms
In numeric analysis root finding is equivalent to finding zeros of a continuous function. In case of complex and composite function finding values of a varia...
zero-finding
Root finding algorithms
In numeric analysis root finding is equivalent to finding zeros of a continuous function. In case of complex and composite function finding values of a varia...
false-position-method
Root finding algorithms
In numeric analysis root finding is equivalent to finding zeros of a continuous function. In case of complex and composite function finding values of a varia...
Broyden's-method
Root finding algorithms
In numeric analysis root finding is equivalent to finding zeros of a continuous function. In case of complex and composite function finding values of a varia...
inverse-quadratic-interpolation
Root finding algorithms
In numeric analysis root finding is equivalent to finding zeros of a continuous function. In case of complex and composite function finding values of a varia...
scatterplot
LOESS
It is quite common to have a relation between two values represented as a scatterplot. However, visual representation of a scatterplot alone can often be non...
least-squares
LOESS
It is quite common to have a relation between two values represented as a scatterplot. However, visual representation of a scatterplot alone can often be non...
weighting
LOESS
It is quite common to have a relation between two values represented as a scatterplot. However, visual representation of a scatterplot alone can often be non...
non-parametric-regression
LOESS
It is quite common to have a relation between two values represented as a scatterplot. However, visual representation of a scatterplot alone can often be non...
local-regression
LOESS
It is quite common to have a relation between two values represented as a scatterplot. However, visual representation of a scatterplot alone can often be non...
integral
Understanding integrals
Building intuition
area
Understanding integrals
Building intuition
net-change
Understanding integrals
Building intuition
Riemann-sum
Understanding integrals
Building intuition
fundamental-theorem-of-calculus
Understanding integrals
Building intuition
improper-integrals
Understanding integrals
Building intuition
Understanding integrals
Building intuition
moment
Moment generating functions
In general moment generating functions (MGF) are used as an alternative for using probability density distribution. The name implies that with MGF it is poss...
mgf
Moment generating functions
In general moment generating functions (MGF) are used as an alternative for using probability density distribution. The name implies that with MGF it is poss...
probability-density-distribution
Moment generating functions
In general moment generating functions (MGF) are used as an alternative for using probability density distribution. The name implies that with MGF it is poss...
skewness
Moment generating functions
In general moment generating functions (MGF) are used as an alternative for using probability density distribution. The name implies that with MGF it is poss...
kurtosis
Moment generating functions
In general moment generating functions (MGF) are used as an alternative for using probability density distribution. The name implies that with MGF it is poss...
expected-value
Moment generating functions
In general moment generating functions (MGF) are used as an alternative for using probability density distribution. The name implies that with MGF it is poss...
Hoeffding's-inequality
Probability bounds
When applying a model to a certain process it is mostly impossible to be fully certain about the values of the parameters, the output of the process, and the...
Markov's-inequality
Probability bounds
When applying a model to a certain process it is mostly impossible to be fully certain about the values of the parameters, the output of the process, and the...
Chebyshev's-inequality
Probability bounds
When applying a model to a certain process it is mostly impossible to be fully certain about the values of the parameters, the output of the process, and the...
Chernoff-bound
Probability bounds
When applying a model to a certain process it is mostly impossible to be fully certain about the values of the parameters, the output of the process, and the...
moment-generating-function
Probability bounds
When applying a model to a certain process it is mostly impossible to be fully certain about the values of the parameters, the output of the process, and the...
Weak-law-of-large-numbers
Probability bounds
When applying a model to a certain process it is mostly impossible to be fully certain about the values of the parameters, the output of the process, and the...
Bernoulli-distribution
Probability bounds
When applying a model to a certain process it is mostly impossible to be fully certain about the values of the parameters, the output of the process, and the...
principal-components
Principal component analysis and singular value decomposition
Principal component analysis
pca
Principal component analysis and singular value decomposition
Principal component analysis
matrix-transformation
Principal component analysis and singular value decomposition
Principal component analysis
feature-extraction
Principal component analysis and singular value decomposition
Principal component analysis
eigendecomposition
Principal component analysis and singular value decomposition
Principal component analysis
singular-vectors
Principal component analysis and singular value decomposition
Principal component analysis
singular-values
Principal component analysis and singular value decomposition
Principal component analysis
scatter-matrix
Principal component analysis and singular value decomposition
Principal component analysis
truncated-svd
Principal component analysis and singular value decomposition
Principal component analysis
linear-model
Degrees of freedom in statistics
In statistics degrees of freedom is the number of independent elements forming a final statistic which are free to vary without violation of an imposed const...
type-1-error
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
type-2-error
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
p-value-hacking
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
alternative-hypothesis
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
power-analysis
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
sample-size
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
family-wise-error-rate
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
multiple-testing
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
Benjamini–Hochberg-procedure
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
FWER
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
Pearson's-chi-square-test
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
z-test
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
F-distribution
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
Student's-distribution
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
continuity-correction
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
Welch's-t-test
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
exact-Fisher's-test
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
Barnard's-test
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
contingency-table
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
G-test
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
paired-t-test
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
McNemar's-test
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
ANOVA
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
analysis-of-variance
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
F-test
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
F-statistic
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
Games-Howell-test
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
ols
Linear least squares
Ordinary least squares method (OLS) is a widely used method which estimates the parameters of a linear regression by minimizing the sum of the squares of the...
normal-equations
Linear least squares
Ordinary least squares method (OLS) is a widely used method which estimates the parameters of a linear regression by minimizing the sum of the squares of the...
qr-decomposition
Linear least squares
Ordinary least squares method (OLS) is a widely used method which estimates the parameters of a linear regression by minimizing the sum of the squares of the...
Breusch–Pagan-test
Heteroscedasticity
What is heteroscedasticity
White-test
Heteroscedasticity
What is heteroscedasticity
nlp
Text similarity
Doing text analysis you may encounter a case when you need to select the most similar text or a group of text to another chosen text. One way to do it is to...
documents-similarity
Text similarity
Doing text analysis you may encounter a case when you need to select the most similar text or a group of text to another chosen text. One way to do it is to...
vector-magnitude
Text similarity
Doing text analysis you may encounter a case when you need to select the most similar text or a group of text to another chosen text. One way to do it is to...
sign-test
Hypothesis test non-parametric statistics
This is an overview of the most commonly used non-parametric statistics in hypothesis testing explaining when to use each. Unlike the parametric counterparts...
Wilcoxon-signed-rank-test
Hypothesis test non-parametric statistics
This is an overview of the most commonly used non-parametric statistics in hypothesis testing explaining when to use each. Unlike the parametric counterparts...
Mann–Whitney-U-test
Hypothesis test non-parametric statistics
This is an overview of the most commonly used non-parametric statistics in hypothesis testing explaining when to use each. Unlike the parametric counterparts...
U-statistic
Hypothesis test non-parametric statistics
This is an overview of the most commonly used non-parametric statistics in hypothesis testing explaining when to use each. Unlike the parametric counterparts...
change-point
Change point analysis
Change point analysis is basically used for determining whether and where an ordered series of values (usually time series) changed their behaviour. The chan...
anomaly-detections
Change point analysis
Change point analysis is basically used for determining whether and where an ordered series of values (usually time series) changed their behaviour. The chan...
search-method
Change point analysis
Change point analysis is basically used for determining whether and where an ordered series of values (usually time series) changed their behaviour. The chan...
Hausdorf-metric
Change point analysis
Change point analysis is basically used for determining whether and where an ordered series of values (usually time series) changed their behaviour. The chan...
breakpoint
Change point analysis
Change point analysis is basically used for determining whether and where an ordered series of values (usually time series) changed their behaviour. The chan...
break-point
Change point analysis
Change point analysis is basically used for determining whether and where an ordered series of values (usually time series) changed their behaviour. The chan...
F1-score
Change point analysis
Change point analysis is basically used for determining whether and where an ordered series of values (usually time series) changed their behaviour. The chan...
vector-length
Vector projection
In order to understand more advanced concepts from vector operations it is good to have an intuition on such terms as vector projection. Below is a two-dimen...
vector-multiplication
Vector projection
In order to understand more advanced concepts from vector operations it is good to have an intuition on such terms as vector projection. Below is a two-dimen...
Euclidean-distance
Distance metrics
In machine learning the distance metrics are used as a measure of similarity between two datapoints - smaller distance means more similarity. They are utiliz...
Manhattan-distance
Distance metrics
In machine learning the distance metrics are used as a measure of similarity between two datapoints - smaller distance means more similarity. They are utiliz...
Minkowski-distance
Distance metrics
In machine learning the distance metrics are used as a measure of similarity between two datapoints - smaller distance means more similarity. They are utiliz...
vector-norm
Distance metrics
In machine learning the distance metrics are used as a measure of similarity between two datapoints - smaller distance means more similarity. They are utiliz...
normed-vector-space
Distance metrics
In machine learning the distance metrics are used as a measure of similarity between two datapoints - smaller distance means more similarity. They are utiliz...
Mahalanobis-distance
Distance metrics
In machine learning the distance metrics are used as a measure of similarity between two datapoints - smaller distance means more similarity. They are utiliz...
cosine-distance
Distance metrics
In machine learning the distance metrics are used as a measure of similarity between two datapoints - smaller distance means more similarity. They are utiliz...
cosine-similarity
Distance metrics
In machine learning the distance metrics are used as a measure of similarity between two datapoints - smaller distance means more similarity. They are utiliz...
Hamming-distance
Distance metrics
In machine learning the distance metrics are used as a measure of similarity between two datapoints - smaller distance means more similarity. They are utiliz...
relative-entropy
Distance metrics
In machine learning the distance metrics are used as a measure of similarity between two datapoints - smaller distance means more similarity. They are utiliz...
bayesian-statistics
Variational inference
Variational inference is a technique which is usually employed for approximating complex models, where computation of the exact conditional distribution is i...
evidence-lower-bound
Variational inference
Variational inference is a technique which is usually employed for approximating complex models, where computation of the exact conditional distribution is i...
mean-field-variational-family
Variational inference
Variational inference is a technique which is usually employed for approximating complex models, where computation of the exact conditional distribution is i...
coordinate-ascent-variational-inference
Variational inference
Variational inference is a technique which is usually employed for approximating complex models, where computation of the exact conditional distribution is i...
CAVI
Variational inference
Variational inference is a technique which is usually employed for approximating complex models, where computation of the exact conditional distribution is i...
gaussian-distribution
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
log-marginal-likelihood
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
sparse-variational-gaussian-process
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
variational-inference
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
likelihood
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
marginal-likelihood
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
whitened-parameterization
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
parameterization-trick
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
binary-classification
Logistic regression
Logistic regression is one of the simplests algorithms for binary classification, and it is based on the linear regression. In its core, it uses a linear com...
logistic-function
Logistic regression
Logistic regression is one of the simplests algorithms for binary classification, and it is based on the linear regression. In its core, it uses a linear com...
odds
Logistic regression
Logistic regression is one of the simplests algorithms for binary classification, and it is based on the linear regression. In its core, it uses a linear com...
log-odds
Logistic regression
Logistic regression is one of the simplests algorithms for binary classification, and it is based on the linear regression. In its core, it uses a linear com...
regularization
Logistic regression
Logistic regression is one of the simplests algorithms for binary classification, and it is based on the linear regression. In its core, it uses a linear com...
overfittings
Regularization
Regularization is a process of simplifying complex machine learning models in order to prevent overfitting. It may be applied explicitly by adding a penalty ...
L2-regularization
Regularization
Regularization is a process of simplifying complex machine learning models in order to prevent overfitting. It may be applied explicitly by adding a penalty ...
L1-regularization
Regularization
Regularization is a process of simplifying complex machine learning models in order to prevent overfitting. It may be applied explicitly by adding a penalty ...
Ridge-regularization
Regularization
Regularization is a process of simplifying complex machine learning models in order to prevent overfitting. It may be applied explicitly by adding a penalty ...
Lasso-regularization
Regularization
Regularization is a process of simplifying complex machine learning models in order to prevent overfitting. It may be applied explicitly by adding a penalty ...
Ridge-regression
Regularization
Regularization is a process of simplifying complex machine learning models in order to prevent overfitting. It may be applied explicitly by adding a penalty ...
Lasso-regression
Regularization
Regularization is a process of simplifying complex machine learning models in order to prevent overfitting. It may be applied explicitly by adding a penalty ...
Elastic-Net
Regularization
Regularization is a process of simplifying complex machine learning models in order to prevent overfitting. It may be applied explicitly by adding a penalty ...
logistic-regression
Neural Networks
In this article
activation-function
Neural Networks
In this article
neuron
Neural Networks
In this article
perceptron
Neural Networks
In this article
deep-neural-networks
Neural Networks
In this article
node
Neural Networks
In this article
backpropagation
Neural Networks
In this article
ReLU
Neural Networks
In this article
bias
Neural Networks
In this article
vanishing-gradient
Neural Networks
In this article
saturated-neuron
Neural Networks
In this article
dying-ReLU
Neural Networks
In this article
ELU
Neural Networks
In this article
softmax
Neural Networks
In this article
feed-forward-networks
Neural Networks
In this article
mse
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
l2
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
mean-squared-error
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
quadratic-loss
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
l1
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
mean-absolute-error
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
mae
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
m-estimator
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
huber-loss
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
bisquare-loss-function
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
entropy
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
cross-entropy
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
Durbin–Watson-test
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
Durbin–Watson-statistic
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
white-noise
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
Breusch–Godfrey-test
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
ADF
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
KPSS
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
Kwiatkowski-Phillips-Schmidt-Shin-test
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
Dickey-Fuller-test
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
MA
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
ARMA
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
random-walk
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
differencing
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
ACF
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
autocorrelation-function
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
PACF
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
partial-autocorrelation-function
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
SARIMA
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
SARIMAX
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
decision-tree
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
bagging
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
random-forest
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
bootstrap-aggregation
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
ensemble-techniques
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
extremely-randomized-trees
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
extra-trees
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
mean-decrease-in-impurity
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
MDI
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
AdaBoost
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
boosting
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
gradient-boosting
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
histogram-based-gradient-boosting
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
xgboost
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
extreme-gradient-boosting
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
gradient-boosting-machine
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
GBM
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
LightGBM
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
gradient-based-one-side-sampling
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
GOSS
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
exclusive-feature-bundling
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
EFB
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
Euler-number
Imaginary number and rotation of vectors
Introduction
real-number
Imaginary number and rotation of vectors
Introduction
trigonometric-function
Imaginary number and rotation of vectors
Introduction
Euler's-formula
Imaginary number and rotation of vectors
Introduction
vector-rotation
Imaginary number and rotation of vectors
Introduction
Euler-identity
Imaginary number and rotation of vectors
Introduction
missing-values
Interpolation
Generally speaking, interpolation is a way of generating new data points which exactly fit into some given set of points. It is used for approximation of com...
spline
Interpolation
Generally speaking, interpolation is a way of generating new data points which exactly fit into some given set of points. It is used for approximation of com...
nearest-neighbour
Interpolation
Generally speaking, interpolation is a way of generating new data points which exactly fit into some given set of points. It is used for approximation of com...
Vandermonde-matrix
Interpolation
Generally speaking, interpolation is a way of generating new data points which exactly fit into some given set of points. It is used for approximation of com...
Lagrange-multipliers
Interpolation
Generally speaking, interpolation is a way of generating new data points which exactly fit into some given set of points. It is used for approximation of com...
Newton's-divided-differences
Interpolation
Generally speaking, interpolation is a way of generating new data points which exactly fit into some given set of points. It is used for approximation of com...
cubic-spline
Interpolation
Generally speaking, interpolation is a way of generating new data points which exactly fit into some given set of points. It is used for approximation of com...
Bezier-curve
Interpolation
Generally speaking, interpolation is a way of generating new data points which exactly fit into some given set of points. It is used for approximation of com...
periodic-function
Fourier series
A Fourier series is a way to approximate a periodic function as the sum of simple sine and cosine functions which makes it useful for analyzing and synthesiz...
sine
Fourier series
A Fourier series is a way to approximate a periodic function as the sum of simple sine and cosine functions which makes it useful for analyzing and synthesiz...
fundamental-period
Fourier series
A Fourier series is a way to approximate a periodic function as the sum of simple sine and cosine functions which makes it useful for analyzing and synthesiz...
harmonic-frequency
Fourier series
A Fourier series is a way to approximate a periodic function as the sum of simple sine and cosine functions which makes it useful for analyzing and synthesiz...
RBF
Kernels overview
In this article What are kernels Kernel density estimation Kernels in Gaussian process Squared exponential kernel Matern kernel ...
radial-basis-function-kernel
Kernels overview
In this article What are kernels Kernel density estimation Kernels in Gaussian process Squared exponential kernel Matern kernel ...
squared-exponential-kernel
Kernels overview
In this article What are kernels Kernel density estimation Kernels in Gaussian process Squared exponential kernel Matern kernel ...
matern-kernel
Kernels overview
In this article What are kernels Kernel density estimation Kernels in Gaussian process Squared exponential kernel Matern kernel ...
rational-quadratic-kernel
Kernels overview
In this article What are kernels Kernel density estimation Kernels in Gaussian process Squared exponential kernel Matern kernel ...
periodic-kernel
Kernels overview
In this article What are kernels Kernel density estimation Kernels in Gaussian process Squared exponential kernel Matern kernel ...
linear-kernel
Kernels overview
In this article What are kernels Kernel density estimation Kernels in Gaussian process Squared exponential kernel Matern kernel ...
dot-product-kernel
Kernels overview
In this article What are kernels Kernel density estimation Kernels in Gaussian process Squared exponential kernel Matern kernel ...
kernel-density-estimation
Kernels overview
In this article What are kernels Kernel density estimation Kernels in Gaussian process Squared exponential kernel Matern kernel ...
KDE
Kernels overview
In this article What are kernels Kernel density estimation Kernels in Gaussian process Squared exponential kernel Matern kernel ...
kernel-bandwidth
Kernels overview
In this article What are kernels Kernel density estimation Kernels in Gaussian process Squared exponential kernel Matern kernel ...
segmentation
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
unsupervised-learning
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
hierarchical-clustering
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
agglomerative-clustering
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
dendrogram
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
k-means
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
ward-linkage
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
affinity-propagation
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
spectral-clustering
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
DBSCAN
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
HDBSCAN
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
OPTICS
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
BIRCH
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
unsupervised-machine-learning
Outliers detection
This article explores machine learning models used to identify unusual or atypical observations within data sets.
semi-supervised-machine-learning
Outliers detection
This article explores machine learning models used to identify unusual or atypical observations within data sets.
anomaly-detection
Outliers detection
This article explores machine learning models used to identify unusual or atypical observations within data sets.
novelty-detection
Outliers detection
This article explores machine learning models used to identify unusual or atypical observations within data sets.
local-outlier-factor
Outliers detection
This article explores machine learning models used to identify unusual or atypical observations within data sets.
LOF
Outliers detection
This article explores machine learning models used to identify unusual or atypical observations within data sets.
GLOSH
Outliers detection
This article explores machine learning models used to identify unusual or atypical observations within data sets.
isolation-forest
Outliers detection
This article explores machine learning models used to identify unusual or atypical observations within data sets.
LLM
Linear mixed model
This type of model is suited when observations are not completely independent. Perhaps we measured the same individuals multiple times over a study, or we co...
z-table
Normal Distribution
Introduction
six-sigma
Normal Distribution
Introduction
CDF
Normal Distribution
Introduction
distribution
Probability & Statistics
Introduction
inference
Probability & Statistics
Introduction
student-distribution
Probability Distributions
A probability distribution is a function that describes the likelihood of a random variable taking on certain values. Think of it as a model or blueprint for...
PMF
Probability Distributions
A probability distribution is a function that describes the likelihood of a random variable taking on certain values. Think of it as a model or blueprint for...
chi-square-distribution
Probability Distributions
A probability distribution is a function that describes the likelihood of a random variable taking on certain values. Think of it as a model or blueprint for...
Bernoulli-trial
Probability Distributions
A probability distribution is a function that describes the likelihood of a random variable taking on certain values. Think of it as a model or blueprint for...
geometric-distribution
Probability Distributions
A probability distribution is a function that describes the likelihood of a random variable taking on certain values. Think of it as a model or blueprint for...
exponential-distribution
Probability Distributions
A probability distribution is a function that describes the likelihood of a random variable taking on certain values. Think of it as a model or blueprint for...
poisson-distribution
Probability Distributions
A probability distribution is a function that describes the likelihood of a random variable taking on certain values. Think of it as a model or blueprint for...
uniform-distribution
Probability Distributions
A probability distribution is a function that describes the likelihood of a random variable taking on certain values. Think of it as a model or blueprint for...
negative-binomial-distribution
Probability Distributions
A probability distribution is a function that describes the likelihood of a random variable taking on certain values. Think of it as a model or blueprint for...
beta-distribution
Probability Distributions
A probability distribution is a function that describes the likelihood of a random variable taking on certain values. Think of it as a model or blueprint for...
gamma-function
Probability Distributions
A probability distribution is a function that describes the likelihood of a random variable taking on certain values. Think of it as a model or blueprint for...
Central-Limit-Theorem
Sampling Distribution
Introduction
CLT
Sampling Distribution
Introduction
standard-deviation
Sampling Distribution
Introduction
sample-variance
Sampling Distribution
Introduction
sample-mean
Sampling Distribution
Introduction
unbiased-estimate
Sampling Distribution
Introduction
confidence-level
Sampling Distribution
Introduction
t-value
Sampling Distribution
Introduction
t-statistic
Sampling Distribution
Introduction
z-statistic
Sampling Distribution
Introduction
goodness-of-fit
Maximum Likelihood Estimation
In statistical inference, the likelihood function is used to estimate the goodness of fit of the parameters to the observed data. The maximum likelihood esti...
joint-probability
Maximum Likelihood Estimation
In statistical inference, the likelihood function is used to estimate the goodness of fit of the parameters to the observed data. The maximum likelihood esti...
likelihood-ratio
Maximum Likelihood Estimation
In statistical inference, the likelihood function is used to estimate the goodness of fit of the parameters to the observed data. The maximum likelihood esti...
likelihood-ratio-test
Maximum Likelihood Estimation
In statistical inference, the likelihood function is used to estimate the goodness of fit of the parameters to the observed data. The maximum likelihood esti...
Wald-test
Maximum Likelihood Estimation
In statistical inference, the likelihood function is used to estimate the goodness of fit of the parameters to the observed data. The maximum likelihood esti...
Wald-statistic
Maximum Likelihood Estimation
In statistical inference, the likelihood function is used to estimate the goodness of fit of the parameters to the observed data. The maximum likelihood esti...
chi-squre-distribution
Maximum Likelihood Estimation
In statistical inference, the likelihood function is used to estimate the goodness of fit of the parameters to the observed data. The maximum likelihood esti...
Lagrange-multiplier-test
Maximum Likelihood Estimation
In statistical inference, the likelihood function is used to estimate the goodness of fit of the parameters to the observed data. The maximum likelihood esti...
score-test
Maximum Likelihood Estimation
In statistical inference, the likelihood function is used to estimate the goodness of fit of the parameters to the observed data. The maximum likelihood esti...
trend
Time Series Smoothing
Similarly to ARIMA models the idea upon which the exponential smoothing models are built is that the most recent observations contribute to the currently obs...
seasonal-component
Time Series Smoothing
Similarly to ARIMA models the idea upon which the exponential smoothing models are built is that the most recent observations contribute to the currently obs...
level-component
Time Series Smoothing
Similarly to ARIMA models the idea upon which the exponential smoothing models are built is that the most recent observations contribute to the currently obs...
Holt-linear-trend
Time Series Smoothing
Similarly to ARIMA models the idea upon which the exponential smoothing models are built is that the most recent observations contribute to the currently obs...
damped-trend
Time Series Smoothing
Similarly to ARIMA models the idea upon which the exponential smoothing models are built is that the most recent observations contribute to the currently obs...
Holt-Winters-seasonal-method
Time Series Smoothing
Similarly to ARIMA models the idea upon which the exponential smoothing models are built is that the most recent observations contribute to the currently obs...
Holt-linear-trend-method
Time Series Smoothing
Similarly to ARIMA models the idea upon which the exponential smoothing models are built is that the most recent observations contribute to the currently obs...
state-space-model
Time Series Smoothing
Similarly to ARIMA models the idea upon which the exponential smoothing models are built is that the most recent observations contribute to the currently obs...
additive-error
Time Series Smoothing
Similarly to ARIMA models the idea upon which the exponential smoothing models are built is that the most recent observations contribute to the currently obs...
multiplicative-error
Time Series Smoothing
Similarly to ARIMA models the idea upon which the exponential smoothing models are built is that the most recent observations contribute to the currently obs...
non-linear-regression
Regression Analysis
Statistical inference allows making educated guesses about a population using a sample. Regression analysis extends this idea by providing means to understan...
ARIMA
Time Series Analysis
Time series analysis is a specialized field of statistics that focuses on data points collected over time. From daily stock prices and hourly temperature rea...
exponential-smooting
Time Series Analysis
Time series analysis is a specialized field of statistics that focuses on data points collected over time. From daily stock prices and hourly temperature rea...
Kalman-filter
Time Series Analysis
Time series analysis is a specialized field of statistics that focuses on data points collected over time. From daily stock prices and hourly temperature rea...
exponential-smoothing
Time Series Smoothing
In time series analysis, data often contains a high degree of noise—random, short-term fluctuations that can obscure the underlying patterns. Smoothing is th...
Rauch-Tung-Striebel
Time Series Smoothing
In time series analysis, data often contains a high degree of noise—random, short-term fluctuations that can obscure the underlying patterns. Smoothing is th...
prior-probability
Bayesian inference
In statistics, Bayesian inference is a powerful method for updating a probability estimate when new data becomes available. This process can be seen as an el...
Bayes-rule
Bayesian inference
In statistics, Bayesian inference is a powerful method for updating a probability estimate when new data becomes available. This process can be seen as an el...
posterior-probability
Bayesian inference
In statistics, Bayesian inference is a powerful method for updating a probability estimate when new data becomes available. This process can be seen as an el...
MAP
Bayesian inference
In statistics, Bayesian inference is a powerful method for updating a probability estimate when new data becomes available. This process can be seen as an el...
maximum-a-posteriori
Bayesian inference
In statistics, Bayesian inference is a powerful method for updating a probability estimate when new data becomes available. This process can be seen as an el...
conjugate-prior-distribution
Bayesian inference
In statistics, Bayesian inference is a powerful method for updating a probability estimate when new data becomes available. This process can be seen as an el...
expected-a-posteriori
Bayesian inference
In statistics, Bayesian inference is a powerful method for updating a probability estimate when new data becomes available. This process can be seen as an el...
EAP
Bayesian inference
In statistics, Bayesian inference is a powerful method for updating a probability estimate when new data becomes available. This process can be seen as an el...
marginal-probability
Bayesian inference
In statistics, Bayesian inference is a powerful method for updating a probability estimate when new data becomes available. This process can be seen as an el...
maximum-likelihood
Expectation Maximization
The Expectation-Maximization (EM) algorithm is a powerful iterative method used in statistics to find the maximum likelihood or maximum a posteriori (MAP) es...
Evidence-Lower-Bound
Expectation Maximization
The Expectation-Maximization (EM) algorithm is a powerful iterative method used in statistics to find the maximum likelihood or maximum a posteriori (MAP) es...
Jensen-inequality
Expectation Maximization
The Expectation-Maximization (EM) algorithm is a powerful iterative method used in statistics to find the maximum likelihood or maximum a posteriori (MAP) es...
convexity
Jensen’s Inequality
The Statement of the Inequality
correlation
Correlation and variance in linear regression
What is covariance?
correlation-coefficients
Correlation and variance in linear regression
What is covariance?
covariation
Correlation and variance in linear regression
What is covariance?
Pearson's-correlation-coefficient
Correlation and variance in linear regression
What is covariance?
coefficient-of-determination
Correlation and variance in linear regression
What is covariance?
r-squared
Correlation and variance in linear regression
What is covariance?
adjusted-r-squared
Correlation and variance in linear regression
What is covariance?
regression-model
Correlation and variance in linear regression
What is covariance?
variance-of-coefficients
Correlation and variance in linear regression
What is covariance?
multiple-regression
Linear regression
Linear regression is the simplest and most widely used form of regression analysis. It assumes a straight-line relationship between a dependent variable ($Y$...
multivariable-regression
Linear regression
Linear regression is the simplest and most widely used form of regression analysis. It assumes a straight-line relationship between a dependent variable ($Y$...
gaussian-noise
Linear regression
Linear regression is the simplest and most widely used form of regression analysis. It assumes a straight-line relationship between a dependent variable ($Y$...
multicollinearity
Linear regression
Linear regression is the simplest and most widely used form of regression analysis. It assumes a straight-line relationship between a dependent variable ($Y$...
correlation-coefficient
Linear regression
Linear regression is the simplest and most widely used form of regression analysis. It assumes a straight-line relationship between a dependent variable ($Y$...
Cochrane–Orcutt-estimation
Linear regression
Linear regression is the simplest and most widely used form of regression analysis. It assumes a straight-line relationship between a dependent variable ($Y$...
Prais–Winsten-estimation
Linear regression
Linear regression is the simplest and most widely used form of regression analysis. It assumes a straight-line relationship between a dependent variable ($Y$...
weighted-least-squares
Linear regression
Linear regression is the simplest and most widely used form of regression analysis. It assumes a straight-line relationship between a dependent variable ($Y$...
WLS
Linear regression
Linear regression is the simplest and most widely used form of regression analysis. It assumes a straight-line relationship between a dependent variable ($Y$...
generalized-least-squares
Linear regression
Linear regression is the simplest and most widely used form of regression analysis. It assumes a straight-line relationship between a dependent variable ($Y$...
GLS
Linear regression
Linear regression is the simplest and most widely used form of regression analysis. It assumes a straight-line relationship between a dependent variable ($Y$...
feasible-generalized-least-squares
Linear regression
Linear regression is the simplest and most widely used form of regression analysis. It assumes a straight-line relationship between a dependent variable ($Y$...
FGLS
Linear regression
Linear regression is the simplest and most widely used form of regression analysis. It assumes a straight-line relationship between a dependent variable ($Y$...
clustering
Machine Learning
Machine Learning (ML) is the field of study dedicated to building algorithms that allow computers to learn patterns from data without being explicitly progra...
computer-vision
Machine Learning
Machine Learning (ML) is the field of study dedicated to building algorithms that allow computers to learn patterns from data without being explicitly progra...
CV
Machine Learning
Machine Learning (ML) is the field of study dedicated to building algorithms that allow computers to learn patterns from data without being explicitly progra...
NLP
Machine Learning
Machine Learning (ML) is the field of study dedicated to building algorithms that allow computers to learn patterns from data without being explicitly progra...
function
Delta Method
The Big Idea: Linearizing the Non-Linear
non-linearity
Delta Method
The Big Idea: Linearizing the Non-Linear
law-of-large-numbers
Delta Method
The Big Idea: Linearizing the Non-Linear
convergence
Delta Method
The Big Idea: Linearizing the Non-Linear
censoring
Survival Analysis
Survival Analysis, also known as Time-to-Event Analysis, is a specialized branch of statistics focused on analyzing the time until an event of interest occur...
Kaplan-Meier-Estimator
Survival Analysis
Survival Analysis, also known as Time-to-Event Analysis, is a specialized branch of statistics focused on analyzing the time until an event of interest occur...
density-function
Survival Analysis
Survival Analysis, also known as Time-to-Event Analysis, is a specialized branch of statistics focused on analyzing the time until an event of interest occur...
life-table
Survival Analysis
Survival Analysis, also known as Time-to-Event Analysis, is a specialized branch of statistics focused on analyzing the time until an event of interest occur...
delta-method
Kaplan-Meier Estimator
The Kaplan-Meier (KM) Estimator is the standard way to estimate the survival function $S(t)$ from observed data.
partial-likelihood
Cox Proportional Hazards Model
While the Kaplan–Meier estimator tells us how survival probability changes over time, it doesn’t explain why — it cannot handle predictors or covariates.
Weibull-distribution
Parametric Survival Models
Parametric survival models assume that survival times follow a known probability distribution. Unlike non-parametric methods (Kaplan–Meier) or semi-parametri...
accelerated-failure-time
Parametric Survival Models
Parametric survival models assume that survival times follow a known probability distribution. Unlike non-parametric methods (Kaplan–Meier) or semi-parametri...
GLM
Generalized Linear Models
Generalized Linear Models (GLMs) provide a powerful and flexible framework for modeling a response variable ($Y$) that is non-normally distributed or related...
OLS
Generalized Linear Models
Generalized Linear Models (GLMs) provide a powerful and flexible framework for modeling a response variable ($Y$) that is non-normally distributed or related...
iteratively-reweighted-least-squares
Generalized Linear Models
Generalized Linear Models (GLMs) provide a powerful and flexible framework for modeling a response variable ($Y$) that is non-normally distributed or related...
IRLS
Generalized Linear Models
Generalized Linear Models (GLMs) provide a powerful and flexible framework for modeling a response variable ($Y$) that is non-normally distributed or related...
generalized-estimating-equations
Generalized Linear Models
Generalized Linear Models (GLMs) provide a powerful and flexible framework for modeling a response variable ($Y$) that is non-normally distributed or related...
GEE
Generalized Linear Models
Generalized Linear Models (GLMs) provide a powerful and flexible framework for modeling a response variable ($Y$) that is non-normally distributed or related...
information-matrix
Generalized Linear Models
Generalized Linear Models (GLMs) provide a powerful and flexible framework for modeling a response variable ($Y$) that is non-normally distributed or related...
sandwitch-estimator
Generalized Linear Models
Generalized Linear Models (GLMs) provide a powerful and flexible framework for modeling a response variable ($Y$) that is non-normally distributed or related...
Fisher-information
Generalized Linear Models
Generalized Linear Models (GLMs) provide a powerful and flexible framework for modeling a response variable ($Y$) that is non-normally distributed or related...
Cramér-Rao-lower-bound
Fisher Information
Fisher Information, denoted $I(\theta)$, is a foundational concept in mathematical statistics. It quantifies the amount of information that an observable ran...