Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
Posts by tag
- loss-function 9
- vector 9
- derivative 7
- normal-distribution 7
- gradient-descent 6
- variance 6
- differentiation 5
- matrix 5
- nonlinear-regression 5
- hypothesis-testing 5
- linear-regression 5
- calculus 4
- second-derivative 4
- Taylor-series 4
- probability-density-function 4
- mean 4
- regression 4
- residuals 4
- slope 3
- gradient 3
- machine-learning 3
- penalization 3
- learning-rate 3
- span 3
- svd 3
- dot-product 3
- kernel 3
- symmetric-matrix 3
- positive-definite-matrix 3
- steepest-descent 3
- confidence-interval 3
- error-term 3
- significance-test 3
- degrees-of-freedom 3
- z-score 3
- binomial-distribution 3
- probability-distribution 3
- cosine 3
- conditional-probability 3
- derivation 2
- functions 2
- integration 2
- polynomial 2
- basis 2
- inverse 2
- determinant 2
- matrix-diagonalization 2
- column-space 2
- principal-component-analysis 2
- rank 2
- ill-conditioned-matrix 2
- hessian 2
- taylor-expansion 2
- Hessian 2
- simplex 2
- secant-line 2
- minimization 2
- classification 2
- lowess 2
- outliers 2
- variation 2
- covariance-matrix 2
- pseudoinverse 2
- null-hypothesis 2
- p-value 2
- significance-level 2
- false-discovery-rate 2
- margin-of-error 2
- t-test 2
- t-distribution 2
- Student's-distribution 2
- z-value 2
- likelihood-function 2
- MLE 2
- heteroscedasticity 2
- homoscedasticity 2
- Levenshtein-distance 2
- autocorrelation 2
- cost-function 2
- PDF 2
- Bayes-theorem 2
- Kullback–Leibler-divergence 2
- KL-divergence 2
- gaussian-process 2
- prior-distribution 2
- posterior-distribution 2
- covariance 2
- sigmoid-function 2
- stationarity 2
- AR 2
- unit-root 2
- lag 2
- complex-number 2
- exponential-function 2
- mean-shift 2
- limits 1
- continuity 1
- discontinuity 1
- rate-of-change 1
- extremities 1
- extreme-points 1
- analysis 1
- saddle-point 1
- minimum 1
- maximum 1
- concavity 1
- extreme-value-theorem 1
- equations 1
- exponential-model 1
- logistic-model 1
- Maclaurin-series 1
- local-minimum 1
- global-minimum 1
- SGD 1
- momentum 1
- adagrad 1
- adadelta 1
- adam 1
- RMS 1
- decaying-average 1
- vector-space 1
- dimensions 1
- unit-vector 1
- linear-equations 1
- linear-transformation 1
- vector-reflection 1
- householder-reflection 1
- matrix-decomposition 1
- gram-schmidt 1
- vector-projection 1
- identity-matrix 1
- PCA 1
- SVD 1
- singular-value-decomposition 1
- full-rank-matrix 1
- condition-number 1
- null-space 1
- linear-combination 1
- taylor-series 1
- Schwarz’s-theorem 1
- nonlinear-least-squares 1
- Gauss-Newton-method 1
- Jacobian 1
- Levenberg–Marquardt-algorithm 1
- linear-programming 1
- constraints 1
- objective-function 1
- maximization 1
- tangent-line 1
- Wolfe-conditions 1
- Armijo-rule 1
- search-direction 1
- cauchy-point 1
- dogleg-method 1
- line-search 1
- supervised-machine-learning 1
- svm 1
- function-minimization 1
- rank-one-update 1
- jacobian 1
- Sherman-Morrison-formula 1
- cross-product 1
- secant-method 1
- bisection-method 1
- Brent's-method 1
- Newton's-method 1
- zero-finding 1
- false-position-method 1
- Broyden's-method 1
- inverse-quadratic-interpolation 1
- smoothing 1
- scatterplot 1
- least-squares 1
- weighting 1
- non-parametric-regression 1
- local-regression 1
- integral 1
- area 1
- net-change 1
- Riemann-sum 1
- fundamental-theorem-of-calculus 1
- improper-integrals 1
- probability 1
- pdf 1
- moment 1
- mgf 1
- probability-density-distribution 1
- skewness 1
- kurtosis 1
- expected-value 1
- Hoeffding's-inequality 1
- Markov's-inequality 1
- Chebyshev's-inequality 1
- Chernoff-bound 1
- moment-generating-function 1
- Weak-law-of-large-numbers 1
- Bernoulli-distribution 1
- correlation 1
- correlation-coefficients 1
- covariation 1
- Pearson's-correlation-coefficient 1
- coefficient-of-determination 1
- r-squared 1
- adjusted-r-squared 1
- regression-model 1
- variance-of-coefficients 1
- principal-components 1
- pca 1
- dimensionality-reduction 1
- matrix-transformation 1
- feature-extraction 1
- eigendecomposition 1
- singular-vectors 1
- singular-values 1
- scatter-matrix 1
- truncated-svd 1
- linear-model 1
- type-1-error 1
- type-2-error 1
- p-value-hacking 1
- alternative-hypothesis 1
- power-analysis 1
- sample-size 1
- family-wise-error-rate 1
- multiple-testing 1
- Benjamini–Hochberg-procedure 1
- FWER 1
- Pearson's-chi-square-test 1
- z-test 1
- F-distribution 1
- continuity-correction 1
- Welch's-t-test 1
- exact-Fisher's-test 1
- Barnard's-test 1
- contingency-table 1
- G-test 1
- paired-t-test 1
- McNemar's-test 1
- ANOVA 1
- analysis-of-variance 1
- F-test 1
- F-statistic 1
- Games-Howell-test 1
- ols 1
- normal-equations 1
- qr-decomposition 1
- sample 1
- Central-limit-theorem 1
- standard-error 1
- standard-deviation 1
- sample-variance 1
- sample-mean 1
- unbiased-estimate 1
- confidence-level 1
- t-value 1
- t-statistic 1
- z-statistic 1
- statistical-inference 1
- goodness-of-fit 1
- joint-probability 1
- log-likelihood 1
- likelihood-ratio 1
- likelihood-ratio-test 1
- Wald-test 1
- Wald-statistic 1
- chi-squre-distribution 1
- Lagrange-multiplier-test 1
- score-test 1
- Breusch–Pagan-test 1
- White-test 1
- nlp 1
- natural-language-processing 1
- documents-similarity 1
- vector-magnitude 1
- multiple-regression 1
- multivariable-regression 1
- gaussian-noise 1
- multicollinearity 1
- correlation-coefficient 1
- Cochrane–Orcutt-estimation 1
- Prais–Winsten-estimation 1
- weighted-least-squares 1
- WLS 1
- generalized-least-squares 1
- GLS 1
- feasible-generalized-least-squares 1
- FGLS 1
- median 1
- sign-test 1
- Wilcoxon-signed-rank-test 1
- Mann–Whitney-U-test 1
- U-statistic 1
- change-point 1
- anomaly-detections 1
- search-method 1
- Hausdorf-metric 1
- breakpoint 1
- break-point 1
- F1-score 1
- statistics 1
- z-table 1
- six-sigma 1
- central-limit-theorem 1
- probability-mass-function 1
- chi-square-distribution 1
- Bernoulli-trial 1
- geometric-distribution 1
- exponential-distribution 1
- poisson-distribution 1
- uniform-distribution 1
- negative-binomial-distribution 1
- studentized-range-distribution 1
- beta-distribution 1
- gamma-function 1
- Gaussian-distribution 1
- prior-probability 1
- Bayes-rule 1
- maximum-likelihood-estimation 1
- posterior-probability 1
- MAP 1
- maximum-a-posteriori 1
- conjugate-prior-distribution 1
- expected-a-posteriori 1
- EAP 1
- marginal-probability 1
- vector-length 1
- vector-multiplication 1
- Euclidean-distance 1
- Manhattan-distance 1
- Minkowski-distance 1
- vector-norm 1
- normed-vector-space 1
- Mahalanobis-distance 1
- cosine-distance 1
- cosine-similarity 1
- Hamming-distance 1
- relative-entropy 1
- bayesian-statistics 1
- evidence-lower-bound 1
- ELBO 1
- mean-field-variational-family 1
- coordinate-ascent-variational-inference 1
- CAVI 1
- gaussian-distribution 1
- log-marginal-likelihood 1
- sparse-variational-gaussian-process 1
- variational-inference 1
- likelihood 1
- marginal-likelihood 1
- whitened-parameterization 1
- parameterization-trick 1
- binary-classification 1
- logistic-function 1
- odds 1
- log-odds 1
- regularization 1
- overfittings 1
- L2-regularization 1
- L1-regularization 1
- Ridge-regularization 1
- Lasso-regularization 1
- Ridge-regression 1
- Lasso-regression 1
- Elastic-Net 1
- logistic-regression 1
- activation-function 1
- neuron 1
- perceptron 1
- deep-neural-networks 1
- hidden-layer 1
- node 1
- backpropagation 1
- ReLU 1
- bias 1
- vanishing-gradient 1
- saturated-neuron 1
- dying-ReLU 1
- ELU 1
- softmax 1
- feed-forward-networks 1
- mse 1
- l2 1
- mean-squared-error 1
- quadratic-loss 1
- l1 1
- mean-absolute-error 1
- mae 1
- m-estimator 1
- huber-loss 1
- bisquare-loss-function 1
- entropy 1
- cross-entropy 1
- Durbin–Watson-test 1
- Durbin–Watson-statistic 1
- white-noise 1
- Breusch–Godfrey-test 1
- ADF 1
- KPSS 1
- Kwiatkowski-Phillips-Schmidt-Shin-test 1
- Dickey-Fuller-test 1
- MA 1
- ARMA 1
- autoregression 1
- moving-average 1
- random-walk 1
- differencing 1
- ACF 1
- autocorrelation-function 1
- PACF 1
- partial-autocorrelation-function 1
- SARIMA 1
- SARIMAX 1
- trend 1
- seasonal-component 1
- level-component 1
- Holt-linear-trend 1
- damped-trend 1
- Holt-Winters-seasonal-method 1
- Holt-linear-trend-method 1
- state-space-model 1
- additive-error 1
- multiplicative-error 1
- decision-tree 1
- bagging 1
- random-forest 1
- bootstrap-aggregation 1
- ensemble-techniques 1
- extremely-randomized-trees 1
- extra-trees 1
- mean-decrease-in-impurity 1
- MDI 1
- AdaBoost 1
- boosting 1
- gradient-boosting 1
- histogram-based-gradient-boosting 1
- xgboost 1
- extreme-gradient-boosting 1
- gradient-boosting-machine 1
- GBM 1
- LightGBM 1
- gradient-based-one-side-sampling 1
- GOSS 1
- exclusive-feature-bundling 1
- EFB 1
- Euler-number 1
- real-number 1
- trigonometric-function 1
- Euler's-formula 1
- vector-rotation 1
- Euler-identity 1
- missing-values 1
- spline 1
- nearest-neighbour 1
- Vandermonde-matrix 1
- Lagrange-multipliers 1
- Newton's-divided-differences 1
- cubic-spline 1
- Bezier-curve 1
- periodic-function 1
- sine 1
- fundamental-period 1
- harmonic-frequency 1
- RBF 1
- radial-basis-function-kernel 1
- squared-exponential-kernel 1
- matern-kernel 1
- rational-quadratic-kernel 1
- periodic-kernel 1
- linear-kernel 1
- dot-product-kernel 1
- kernel-density-estimation 1
- KDE 1
- kernel-bandwidth 1
- segmentation 1
- unsupervised-learning 1
- hierarchical-clustering 1
- agglomerative-clustering 1
- dendrogram 1
- k-means 1
- ward-linkage 1
- affinity-propagation 1
- spectral-clustering 1
- DBSCAN 1
- HDBSCAN 1
- OPTICS 1
- BIRCH 1
- unsupervised-machine-learning 1
- semi-supervised-machine-learning 1
- anomaly-detection 1
- novelty-detection 1
- local-outlier-factor 1
- LOF 1
- GLOSH 1
- isolation-forest 1
loss-function
Regularization
Regularization is a process of simplifying complex machine learning models in order to prevent overfitting. It may be applied explicitly by adding a penalty ...
Quasi-Newton methods
The classical Newton’s method for optimization is known for being computationally expensive when it comes to calculating the gradient of the Hessian matrix a...
Conjugate gradient
Conjugate gradient methods are well suited for solving both linear and non-linear systems of equations.
Trust region
Trust region is a general strategy of optimization of an objective function by constructing an approximation of the function around a certain point and measu...
Line search
Generally speaking, line search is one of the two (the other is trust region) major strategies in non-linear optimization. Line search methods first determin...
Nedler-Mead algorithm
The Nedlear-Mead optimization algorithm employs the concept of simplex - a special type of polytope in $n$-dimensional space which has $n+1$ vertices, that i...
Newton’s method
Similarly to gradient descent, Newton’s method is used for finding a minimum of a function through iterations. Unlike gradient descent however, it uses a sec...
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
vector
Fourier series
A Fourier series is a way to approximate a periodic function as the sum of simple sine and cosine functions which makes it useful for analyzing and synthesiz...
Imaginary number and rotation of vectors
Introduction
Distance metrics
In machine learning the distance metrics are used as a measure of similarity between two datapoints - smaller distance means more similarity. They are utiliz...
Text similarity
Doing text analysis you may encounter a case when you need to select the most similar text or a group of text to another chosen text. One way to do it is to...
Principal component analysis and singular value decomposition
Principal component analysis
Matrix properties
Below I’ve provided intuition behind some of the properties of matrices, which are useful to understand in order to apply them in more complex algorithms. R...
Eigenvectors and eigenvalues
In machine learning eigenvectors and eigenvalues is one of the heavily used concepts, which among other things is fundamental for dimension reduction and fin...
Change of basis
Let’s recall that the basis is a set of linearly independent vectors which form the full span of vector space. With regard to matrices the basis can be viewe...
Linear transformations
Linear algebra is all about making transformations of linear combinations of numbers, called vectors, and arrays of vectors, called matrices. The elements of...
derivative
Understanding integrals
Building intuition
Mean value theorem
The mean value theorem states that if a function is continuous and differentiable between two points then there exists a point between them where the tangent...
Newton’s method
Similarly to gradient descent, Newton’s method is used for finding a minimum of a function through iterations. Unlike gradient descent however, it uses a sec...
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
Taylor series
One usually neglected topic in calculus is understanding Taylor series. At first glance they seem to be rather impractical, however, they form building block...
Analyzing functions
Given an equation, understanding of derivatives allows analyzing functions beyond simple plotting and eyeballing them. Below I will describe some of the tech...
Understanding derivatives
Building intuition
normal-distribution
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
Statistical distributions
A set of discrete values or a range of continuous values of a random variable is characterized by a certain probability distribution. In statistics various d...
Hypothesis test non-parametric statistics
This is an overview of the most commonly used non-parametric statistics in hypothesis testing explaining when to use each. Unlike the parametric counterparts...
Linear regression
Regression analysis is used for estimating the relationship between variables, usually one dependent and one or several independent variables. Having a regre...
Sampling distribution
In real life situations it is often impossible to calculate statistics such as mean and variance for the whole population. Instead we may only be able to dra...
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
gradient-descent
Neural Networks
In this article
Linear least squares
Ordinary least squares method (OLS) is a widely used method which estimates the parameters of a linear regression by minimizing the sum of the squares of the...
Conjugate gradient
Conjugate gradient methods are well suited for solving both linear and non-linear systems of equations.
Line search
Generally speaking, line search is one of the two (the other is trust region) major strategies in non-linear optimization. Line search methods first determin...
Non-linear regression
Although application of linear regression is straightforward and characterized by many useful statistical properties, in the real life many processes have no...
Newton’s method
Similarly to gradient descent, Newton’s method is used for finding a minimum of a function through iterations. Unlike gradient descent however, it uses a sec...
variance
Heteroscedasticity
What is heteroscedasticity
Sampling distribution
In real life situations it is often impossible to calculate statistics such as mean and variance for the whole population. Instead we may only be able to dra...
Degrees of freedom in statistics
In statistics degrees of freedom is the number of independent elements forming a final statistic which are free to vary without violation of an imposed const...
Principal component analysis and singular value decomposition
Principal component analysis
Correlation and variance in linear regression
What is covariance?
Probability bounds
When applying a model to a certain process it is mostly impossible to be fully certain about the values of the parameters, the output of the process, and the...
differentiation
Mean value theorem
The mean value theorem states that if a function is continuous and differentiable between two points then there exists a point between them where the tangent...
Taylor series
One usually neglected topic in calculus is understanding Taylor series. At first glance they seem to be rather impractical, however, they form building block...
Differential equations
Differential equation is an equation with a function and at least one of its derivatives. Here is an example:
Analyzing functions
Given an equation, understanding of derivatives allows analyzing functions beyond simple plotting and eyeballing them. Below I will describe some of the tech...
Understanding derivatives
Building intuition
matrix
Principal component analysis and singular value decomposition
Principal component analysis
Matrix properties
Below I’ve provided intuition behind some of the properties of matrices, which are useful to understand in order to apply them in more complex algorithms. R...
Eigenvectors and eigenvalues
In machine learning eigenvectors and eigenvalues is one of the heavily used concepts, which among other things is fundamental for dimension reduction and fin...
Change of basis
Let’s recall that the basis is a set of linearly independent vectors which form the full span of vector space. With regard to matrices the basis can be viewe...
Linear transformations
Linear algebra is all about making transformations of linear combinations of numbers, called vectors, and arrays of vectors, called matrices. The elements of...
nonlinear-regression
Conjugate gradient
Conjugate gradient methods are well suited for solving both linear and non-linear systems of equations.
Trust region
Trust region is a general strategy of optimization of an objective function by constructing an approximation of the function around a certain point and measu...
Line search
Generally speaking, line search is one of the two (the other is trust region) major strategies in non-linear optimization. Line search methods first determin...
Nedler-Mead algorithm
The Nedlear-Mead optimization algorithm employs the concept of simplex - a special type of polytope in $n$-dimensional space which has $n+1$ vertices, that i...
Non-linear regression
Although application of linear regression is straightforward and characterized by many useful statistical properties, in the real life many processes have no...
hypothesis-testing
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
Hypothesis test non-parametric statistics
This is an overview of the most commonly used non-parametric statistics in hypothesis testing explaining when to use each. Unlike the parametric counterparts...
Linear regression
Regression analysis is used for estimating the relationship between variables, usually one dependent and one or several independent variables. Having a regre...
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
linear-regression
Neural Networks
In this article
Logistic regression
Logistic regression is one of the simplests algorithms for binary classification, and it is based on the linear regression. In its core, it uses a linear com...
Linear regression
Regression analysis is used for estimating the relationship between variables, usually one dependent and one or several independent variables. Having a regre...
Heteroscedasticity
What is heteroscedasticity
Linear least squares
Ordinary least squares method (OLS) is a widely used method which estimates the parameters of a linear regression by minimizing the sum of the squares of the...
calculus
Differential equations
Differential equation is an equation with a function and at least one of its derivatives. Here is an example:
Analyzing functions
Given an equation, understanding of derivatives allows analyzing functions beyond simple plotting and eyeballing them. Below I will describe some of the tech...
Understanding derivatives
Building intuition
Limits and continuity
Let’s build an idea about formal definition of limit.
second-derivative
Quasi-Newton methods
The classical Newton’s method for optimization is known for being computationally expensive when it comes to calculating the gradient of the Hessian matrix a...
Newton’s method
Similarly to gradient descent, Newton’s method is used for finding a minimum of a function through iterations. Unlike gradient descent however, it uses a sec...
Taylor series
One usually neglected topic in calculus is understanding Taylor series. At first glance they seem to be rather impractical, however, they form building block...
Analyzing functions
Given an equation, understanding of derivatives allows analyzing functions beyond simple plotting and eyeballing them. Below I will describe some of the tech...
Taylor-series
Imaginary number and rotation of vectors
Introduction
Moment generating functions
In general moment generating functions (MGF) are used as an alternative for using probability density distribution. The name implies that with MGF it is poss...
Mean value theorem
The mean value theorem states that if a function is continuous and differentiable between two points then there exists a point between them where the tangent...
Non-linear regression
Although application of linear regression is straightforward and characterized by many useful statistical properties, in the real life many processes have no...
probability-density-function
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
Bayesian inference
Bayesian inference is the method of statistical inference where an estimated probability is updated when new data arrives. In a way, it may be viewed as an u...
Statistical distributions
A set of discrete values or a range of continuous values of a random variable is characterized by a certain probability distribution. In statistics various d...
Understanding integrals
Building intuition
mean
Change point analysis
Change point analysis is basically used for determining whether and where an ordered series of values (usually time series) changed their behaviour. The chan...
Sampling distribution
In real life situations it is often impossible to calculate statistics such as mean and variance for the whole population. Instead we may only be able to dra...
Degrees of freedom in statistics
In statistics degrees of freedom is the number of independent elements forming a final statistic which are free to vary without violation of an imposed const...
Probability bounds
When applying a model to a certain process it is mostly impossible to be fully certain about the values of the parameters, the output of the process, and the...
regression
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
Neural Networks
In this article
Heteroscedasticity
What is heteroscedasticity
Correlation and variance in linear regression
What is covariance?
residuals
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
Linear regression
Regression analysis is used for estimating the relationship between variables, usually one dependent and one or several independent variables. Having a regre...
Heteroscedasticity
What is heteroscedasticity
Degrees of freedom in statistics
In statistics degrees of freedom is the number of independent elements forming a final statistic which are free to vary without violation of an imposed const...
slope
Mean value theorem
The mean value theorem states that if a function is continuous and differentiable between two points then there exists a point between them where the tangent...
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
Understanding derivatives
Building intuition
gradient
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
Understanding derivatives
Building intuition
machine-learning
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
penalization
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
Change point analysis
Change point analysis is basically used for determining whether and where an ordered series of values (usually time series) changed their behaviour. The chan...
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
learning-rate
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
Line search
Generally speaking, line search is one of the two (the other is trust region) major strategies in non-linear optimization. Line search methods first determin...
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
span
Matrix properties
Below I’ve provided intuition behind some of the properties of matrices, which are useful to understand in order to apply them in more complex algorithms. R...
Change of basis
Let’s recall that the basis is a set of linearly independent vectors which form the full span of vector space. With regard to matrices the basis can be viewe...
Linear transformations
Linear algebra is all about making transformations of linear combinations of numbers, called vectors, and arrays of vectors, called matrices. The elements of...
svd
Linear least squares
Ordinary least squares method (OLS) is a widely used method which estimates the parameters of a linear regression by minimizing the sum of the squares of the...
Principal component analysis and singular value decomposition
Principal component analysis
QR decomposition
QR decomposition may be a numerically stable algorithm for solving systems of linear equations. Similar to singular value decomposition it provides mechanism...
dot-product
Vector projection
In order to understand more advanced concepts from vector operations it is good to have an intuition on such terms as vector projection. Below is a two-dimen...
Text similarity
Doing text analysis you may encounter a case when you need to select the most similar text or a group of text to another chosen text. One way to do it is to...
QR decomposition
QR decomposition may be a numerically stable algorithm for solving systems of linear equations. Similar to singular value decomposition it provides mechanism...
kernel
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
Matrix properties
Below I’ve provided intuition behind some of the properties of matrices, which are useful to understand in order to apply them in more complex algorithms. R...
symmetric-matrix
Principal component analysis and singular value decomposition
Principal component analysis
Newton’s method
Similarly to gradient descent, Newton’s method is used for finding a minimum of a function through iterations. Unlike gradient descent however, it uses a sec...
Matrix properties
Below I’ve provided intuition behind some of the properties of matrices, which are useful to understand in order to apply them in more complex algorithms. R...
positive-definite-matrix
Principal component analysis and singular value decomposition
Principal component analysis
Newton’s method
Similarly to gradient descent, Newton’s method is used for finding a minimum of a function through iterations. Unlike gradient descent however, it uses a sec...
Matrix properties
Below I’ve provided intuition behind some of the properties of matrices, which are useful to understand in order to apply them in more complex algorithms. R...
steepest-descent
Conjugate gradient
Conjugate gradient methods are well suited for solving both linear and non-linear systems of equations.
Trust region
Trust region is a general strategy of optimization of an objective function by constructing an approximation of the function around a certain point and measu...
Line search
Generally speaking, line search is one of the two (the other is trust region) major strategies in non-linear optimization. Line search methods first determin...
confidence-interval
Sampling distribution
In real life situations it is often impossible to calculate statistics such as mean and variance for the whole population. Instead we may only be able to dra...
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
Probability bounds
When applying a model to a certain process it is mostly impossible to be fully certain about the values of the parameters, the output of the process, and the...
error-term
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
Linear regression
Regression analysis is used for estimating the relationship between variables, usually one dependent and one or several independent variables. Having a regre...
Correlation and variance in linear regression
What is covariance?
significance-test
Hypothesis test non-parametric statistics
This is an overview of the most commonly used non-parametric statistics in hypothesis testing explaining when to use each. Unlike the parametric counterparts...
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
degrees-of-freedom
Statistical distributions
A set of discrete values or a range of continuous values of a random variable is characterized by a certain probability distribution. In statistics various d...
Hypothesis test non-parametric statistics
This is an overview of the most commonly used non-parametric statistics in hypothesis testing explaining when to use each. Unlike the parametric counterparts...
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
z-score
Statistical distributions
A set of discrete values or a range of continuous values of a random variable is characterized by a certain probability distribution. In statistics various d...
Sampling distribution
In real life situations it is often impossible to calculate statistics such as mean and variance for the whole population. Instead we may only be able to dra...
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
binomial-distribution
Statistical distributions
A set of discrete values or a range of continuous values of a random variable is characterized by a certain probability distribution. In statistics various d...
Hypothesis test non-parametric statistics
This is an overview of the most commonly used non-parametric statistics in hypothesis testing explaining when to use each. Unlike the parametric counterparts...
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
probability-distribution
Bayesian inference
Bayesian inference is the method of statistical inference where an estimated probability is updated when new data arrives. In a way, it may be viewed as an u...
Statistical distributions
A set of discrete values or a range of continuous values of a random variable is characterized by a certain probability distribution. In statistics various d...
Maximum likelihood estimation
In statistics the likelihood function is used to estimate the goodness of fit of the parameters to the observed data. The maximum likelihood estimation (MLE)...
cosine
Fourier series
A Fourier series is a way to approximate a periodic function as the sum of simple sine and cosine functions which makes it useful for analyzing and synthesiz...
Vector projection
In order to understand more advanced concepts from vector operations it is good to have an intuition on such terms as vector projection. Below is a two-dimen...
Text similarity
Doing text analysis you may encounter a case when you need to select the most similar text or a group of text to another chosen text. One way to do it is to...
conditional-probability
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
Variational inference
Variational inference is a technique which is usually employed for approximating complex models, where computation of the exact conditional distribution is i...
Bayesian inference
Bayesian inference is the method of statistical inference where an estimated probability is updated when new data arrives. In a way, it may be viewed as an u...
derivation
Analyzing functions
Given an equation, understanding of derivatives allows analyzing functions beyond simple plotting and eyeballing them. Below I will describe some of the tech...
Understanding derivatives
Building intuition
functions
Taylor series
One usually neglected topic in calculus is understanding Taylor series. At first glance they seem to be rather impractical, however, they form building block...
Analyzing functions
Given an equation, understanding of derivatives allows analyzing functions beyond simple plotting and eyeballing them. Below I will describe some of the tech...
integration
Understanding integrals
Building intuition
Differential equations
Differential equation is an equation with a function and at least one of its derivatives. Here is an example:
polynomial
Interpolation
Generally speaking, interpolation is a way of generating new data points which exactly fit into some given set of points. It is used for approximation of com...
Taylor series
One usually neglected topic in calculus is understanding Taylor series. At first glance they seem to be rather impractical, however, they form building block...
basis
Eigenvectors and eigenvalues
In machine learning eigenvectors and eigenvalues is one of the heavily used concepts, which among other things is fundamental for dimension reduction and fin...
Linear transformations
Linear algebra is all about making transformations of linear combinations of numbers, called vectors, and arrays of vectors, called matrices. The elements of...
inverse
Principal component analysis and singular value decomposition
Principal component analysis
Linear transformations
Linear algebra is all about making transformations of linear combinations of numbers, called vectors, and arrays of vectors, called matrices. The elements of...
determinant
Eigenvectors and eigenvalues
In machine learning eigenvectors and eigenvalues is one of the heavily used concepts, which among other things is fundamental for dimension reduction and fin...
Linear transformations
Linear algebra is all about making transformations of linear combinations of numbers, called vectors, and arrays of vectors, called matrices. The elements of...
matrix-diagonalization
Eigenvectors and eigenvalues
In machine learning eigenvectors and eigenvalues is one of the heavily used concepts, which among other things is fundamental for dimension reduction and fin...
Change of basis
Let’s recall that the basis is a set of linearly independent vectors which form the full span of vector space. With regard to matrices the basis can be viewe...
column-space
Matrix properties
Below I’ve provided intuition behind some of the properties of matrices, which are useful to understand in order to apply them in more complex algorithms. R...
Eigenvectors and eigenvalues
In machine learning eigenvectors and eigenvalues is one of the heavily used concepts, which among other things is fundamental for dimension reduction and fin...
principal-component-analysis
Principal component analysis and singular value decomposition
Principal component analysis
Eigenvectors and eigenvalues
In machine learning eigenvectors and eigenvalues is one of the heavily used concepts, which among other things is fundamental for dimension reduction and fin...
rank
Hypothesis test non-parametric statistics
This is an overview of the most commonly used non-parametric statistics in hypothesis testing explaining when to use each. Unlike the parametric counterparts...
Matrix properties
Below I’ve provided intuition behind some of the properties of matrices, which are useful to understand in order to apply them in more complex algorithms. R...
ill-conditioned-matrix
Linear least squares
Ordinary least squares method (OLS) is a widely used method which estimates the parameters of a linear regression by minimizing the sum of the squares of the...
Matrix properties
Below I’ve provided intuition behind some of the properties of matrices, which are useful to understand in order to apply them in more complex algorithms. R...
hessian
Quasi-Newton methods
The classical Newton’s method for optimization is known for being computationally expensive when it comes to calculating the gradient of the Hessian matrix a...
Newton’s method
Similarly to gradient descent, Newton’s method is used for finding a minimum of a function through iterations. Unlike gradient descent however, it uses a sec...
taylor-expansion
Trust region
Trust region is a general strategy of optimization of an objective function by constructing an approximation of the function around a certain point and measu...
Newton’s method
Similarly to gradient descent, Newton’s method is used for finding a minimum of a function through iterations. Unlike gradient descent however, it uses a sec...
Hessian
Non-linear regression
Although application of linear regression is straightforward and characterized by many useful statistical properties, in the real life many processes have no...
Newton’s method
Similarly to gradient descent, Newton’s method is used for finding a minimum of a function through iterations. Unlike gradient descent however, it uses a sec...
simplex
Nedler-Mead algorithm
The Nedlear-Mead optimization algorithm employs the concept of simplex - a special type of polytope in $n$-dimensional space which has $n+1$ vertices, that i...
Simplex method
Linear programming aims at finding the best outcome of mathematical problems provided constraints. The requirements for the linear programming problems are d...
secant-line
Root finding algorithms
In numeric analysis root finding is equivalent to finding zeros of a continuous function. In case of complex and composite function finding values of a varia...
Mean value theorem
The mean value theorem states that if a function is continuous and differentiable between two points then there exists a point between them where the tangent...
minimization
Trust region
Trust region is a general strategy of optimization of an objective function by constructing an approximation of the function around a certain point and measu...
Line search
Generally speaking, line search is one of the two (the other is trust region) major strategies in non-linear optimization. Line search methods first determin...
classification
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
Support vector machine
Support vector machine (SVM) is mainly used for building classification models in supervised machine learning, however the ideas employed by SVM can be also ...
lowess
LOESS
It is quite common to have a relation between two values represented as a scatterplot. However, visual representation of a scatterplot alone can often be non...
LOESS
It is quite common to have a relation between two values represented as a scatterplot. However, visual representation of a scatterplot alone can often be non...
outliers
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
LOESS
It is quite common to have a relation between two values represented as a scatterplot. However, visual representation of a scatterplot alone can often be non...
variation
Change point analysis
Change point analysis is basically used for determining whether and where an ordered series of values (usually time series) changed their behaviour. The chan...
Correlation and variance in linear regression
What is covariance?
covariance-matrix
Principal component analysis and singular value decomposition
Principal component analysis
Correlation and variance in linear regression
What is covariance?
pseudoinverse
Linear least squares
Ordinary least squares method (OLS) is a widely used method which estimates the parameters of a linear regression by minimizing the sum of the squares of the...
Principal component analysis and singular value decomposition
Principal component analysis
null-hypothesis
Hypothesis test non-parametric statistics
This is an overview of the most commonly used non-parametric statistics in hypothesis testing explaining when to use each. Unlike the parametric counterparts...
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
p-value
Hypothesis test non-parametric statistics
This is an overview of the most commonly used non-parametric statistics in hypothesis testing explaining when to use each. Unlike the parametric counterparts...
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
significance-level
Sampling distribution
In real life situations it is often impossible to calculate statistics such as mean and variance for the whole population. Instead we may only be able to dra...
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
false-discovery-rate
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
margin-of-error
Sampling distribution
In real life situations it is often impossible to calculate statistics such as mean and variance for the whole population. Instead we may only be able to dra...
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
t-test
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
t-distribution
Statistical distributions
A set of discrete values or a range of continuous values of a random variable is characterized by a certain probability distribution. In statistics various d...
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
Student's-distribution
Statistical distributions
A set of discrete values or a range of continuous values of a random variable is characterized by a certain probability distribution. In statistics various d...
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
z-value
Statistical distributions
A set of discrete values or a range of continuous values of a random variable is characterized by a certain probability distribution. In statistics various d...
Sampling distribution
In real life situations it is often impossible to calculate statistics such as mean and variance for the whole population. Instead we may only be able to dra...
likelihood-function
Bayesian inference
Bayesian inference is the method of statistical inference where an estimated probability is updated when new data arrives. In a way, it may be viewed as an u...
Maximum likelihood estimation
In statistics the likelihood function is used to estimate the goodness of fit of the parameters to the observed data. The maximum likelihood estimation (MLE)...
MLE
Bayesian inference
Bayesian inference is the method of statistical inference where an estimated probability is updated when new data arrives. In a way, it may be viewed as an u...
Maximum likelihood estimation
In statistics the likelihood function is used to estimate the goodness of fit of the parameters to the observed data. The maximum likelihood estimation (MLE)...
heteroscedasticity
Linear regression
Regression analysis is used for estimating the relationship between variables, usually one dependent and one or several independent variables. Having a regre...
Heteroscedasticity
What is heteroscedasticity
homoscedasticity
Linear regression
Regression analysis is used for estimating the relationship between variables, usually one dependent and one or several independent variables. Having a regre...
Heteroscedasticity
What is heteroscedasticity
Levenshtein-distance
Distance metrics
In machine learning the distance metrics are used as a measure of similarity between two datapoints - smaller distance means more similarity. They are utiliz...
Text similarity
Doing text analysis you may encounter a case when you need to select the most similar text or a group of text to another chosen text. One way to do it is to...
autocorrelation
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
Linear regression
Regression analysis is used for estimating the relationship between variables, usually one dependent and one or several independent variables. Having a regre...
cost-function
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
Change point analysis
Change point analysis is basically used for determining whether and where an ordered series of values (usually time series) changed their behaviour. The chan...
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
Statistical distributions
A set of discrete values or a range of continuous values of a random variable is characterized by a certain probability distribution. In statistics various d...
Bayes-theorem
Variational inference
Variational inference is a technique which is usually employed for approximating complex models, where computation of the exact conditional distribution is i...
Bayesian inference
Bayesian inference is the method of statistical inference where an estimated probability is updated when new data arrives. In a way, it may be viewed as an u...
Kullback–Leibler-divergence
Variational inference
Variational inference is a technique which is usually employed for approximating complex models, where computation of the exact conditional distribution is i...
Distance metrics
In machine learning the distance metrics are used as a measure of similarity between two datapoints - smaller distance means more similarity. They are utiliz...
KL-divergence
Variational inference
Variational inference is a technique which is usually employed for approximating complex models, where computation of the exact conditional distribution is i...
Distance metrics
In machine learning the distance metrics are used as a measure of similarity between two datapoints - smaller distance means more similarity. They are utiliz...
gaussian-process
Kernels overview
In this article What are kernels Kernel density estimation Kernels in Gaussian process Squared exponential kernel Matern kernel ...
Variational inference
Variational inference is a technique which is usually employed for approximating complex models, where computation of the exact conditional distribution is i...
prior-distribution
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
Variational inference
Variational inference is a technique which is usually employed for approximating complex models, where computation of the exact conditional distribution is i...
posterior-distribution
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
Variational inference
Variational inference is a technique which is usually employed for approximating complex models, where computation of the exact conditional distribution is i...
covariance
Kernels overview
In this article What are kernels Kernel density estimation Kernels in Gaussian process Squared exponential kernel Matern kernel ...
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
sigmoid-function
Neural Networks
In this article
Logistic regression
Logistic regression is one of the simplests algorithms for binary classification, and it is based on the linear regression. In its core, it uses a linear com...
stationarity
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
AR
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
unit-root
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
lag
Exponential smoothing models
In this article we shall make a breakdown of the exponential smoothing models which are used in time series forecasting.
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
complex-number
Fourier series
A Fourier series is a way to approximate a periodic function as the sum of simple sine and cosine functions which makes it useful for analyzing and synthesiz...
Imaginary number and rotation of vectors
Introduction
exponential-function
Fourier series
A Fourier series is a way to approximate a periodic function as the sum of simple sine and cosine functions which makes it useful for analyzing and synthesiz...
Imaginary number and rotation of vectors
Introduction
mean-shift
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
limits
Limits and continuity
Let’s build an idea about formal definition of limit.
continuity
Limits and continuity
Let’s build an idea about formal definition of limit.
discontinuity
Limits and continuity
Let’s build an idea about formal definition of limit.
rate-of-change
Understanding derivatives
Building intuition
extremities
Analyzing functions
Given an equation, understanding of derivatives allows analyzing functions beyond simple plotting and eyeballing them. Below I will describe some of the tech...
extreme-points
Analyzing functions
Given an equation, understanding of derivatives allows analyzing functions beyond simple plotting and eyeballing them. Below I will describe some of the tech...
analysis
Analyzing functions
Given an equation, understanding of derivatives allows analyzing functions beyond simple plotting and eyeballing them. Below I will describe some of the tech...
saddle-point
Analyzing functions
Given an equation, understanding of derivatives allows analyzing functions beyond simple plotting and eyeballing them. Below I will describe some of the tech...
minimum
Analyzing functions
Given an equation, understanding of derivatives allows analyzing functions beyond simple plotting and eyeballing them. Below I will describe some of the tech...
maximum
Analyzing functions
Given an equation, understanding of derivatives allows analyzing functions beyond simple plotting and eyeballing them. Below I will describe some of the tech...
concavity
Analyzing functions
Given an equation, understanding of derivatives allows analyzing functions beyond simple plotting and eyeballing them. Below I will describe some of the tech...
extreme-value-theorem
Analyzing functions
Given an equation, understanding of derivatives allows analyzing functions beyond simple plotting and eyeballing them. Below I will describe some of the tech...
equations
Differential equations
Differential equation is an equation with a function and at least one of its derivatives. Here is an example:
exponential-model
Differential equations
Differential equation is an equation with a function and at least one of its derivatives. Here is an example:
logistic-model
Differential equations
Differential equation is an equation with a function and at least one of its derivatives. Here is an example:
Maclaurin-series
Taylor series
One usually neglected topic in calculus is understanding Taylor series. At first glance they seem to be rather impractical, however, they form building block...
local-minimum
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
global-minimum
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
SGD
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
momentum
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
adagrad
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
adadelta
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
adam
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
RMS
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
decaying-average
Gradient descent
Gradient descent is bread and butter of machine learning. It is a fundamental method of finding the minimum of the loss function in neural networks, hence it...
vector-space
Linear transformations
Linear algebra is all about making transformations of linear combinations of numbers, called vectors, and arrays of vectors, called matrices. The elements of...
dimensions
Linear transformations
Linear algebra is all about making transformations of linear combinations of numbers, called vectors, and arrays of vectors, called matrices. The elements of...
unit-vector
Linear transformations
Linear algebra is all about making transformations of linear combinations of numbers, called vectors, and arrays of vectors, called matrices. The elements of...
linear-equations
Linear transformations
Linear algebra is all about making transformations of linear combinations of numbers, called vectors, and arrays of vectors, called matrices. The elements of...
linear-transformation
Change of basis
Let’s recall that the basis is a set of linearly independent vectors which form the full span of vector space. With regard to matrices the basis can be viewe...
vector-reflection
QR decomposition
QR decomposition may be a numerically stable algorithm for solving systems of linear equations. Similar to singular value decomposition it provides mechanism...
householder-reflection
QR decomposition
QR decomposition may be a numerically stable algorithm for solving systems of linear equations. Similar to singular value decomposition it provides mechanism...
matrix-decomposition
QR decomposition
QR decomposition may be a numerically stable algorithm for solving systems of linear equations. Similar to singular value decomposition it provides mechanism...
gram-schmidt
QR decomposition
QR decomposition may be a numerically stable algorithm for solving systems of linear equations. Similar to singular value decomposition it provides mechanism...
vector-projection
QR decomposition
QR decomposition may be a numerically stable algorithm for solving systems of linear equations. Similar to singular value decomposition it provides mechanism...
identity-matrix
Eigenvectors and eigenvalues
In machine learning eigenvectors and eigenvalues is one of the heavily used concepts, which among other things is fundamental for dimension reduction and fin...
PCA
Eigenvectors and eigenvalues
In machine learning eigenvectors and eigenvalues is one of the heavily used concepts, which among other things is fundamental for dimension reduction and fin...
SVD
Eigenvectors and eigenvalues
In machine learning eigenvectors and eigenvalues is one of the heavily used concepts, which among other things is fundamental for dimension reduction and fin...
singular-value-decomposition
Eigenvectors and eigenvalues
In machine learning eigenvectors and eigenvalues is one of the heavily used concepts, which among other things is fundamental for dimension reduction and fin...
full-rank-matrix
Matrix properties
Below I’ve provided intuition behind some of the properties of matrices, which are useful to understand in order to apply them in more complex algorithms. R...
condition-number
Matrix properties
Below I’ve provided intuition behind some of the properties of matrices, which are useful to understand in order to apply them in more complex algorithms. R...
null-space
Matrix properties
Below I’ve provided intuition behind some of the properties of matrices, which are useful to understand in order to apply them in more complex algorithms. R...
linear-combination
Matrix properties
Below I’ve provided intuition behind some of the properties of matrices, which are useful to understand in order to apply them in more complex algorithms. R...
taylor-series
Newton’s method
Similarly to gradient descent, Newton’s method is used for finding a minimum of a function through iterations. Unlike gradient descent however, it uses a sec...
Schwarz’s-theorem
Newton’s method
Similarly to gradient descent, Newton’s method is used for finding a minimum of a function through iterations. Unlike gradient descent however, it uses a sec...
nonlinear-least-squares
Non-linear regression
Although application of linear regression is straightforward and characterized by many useful statistical properties, in the real life many processes have no...
Gauss-Newton-method
Non-linear regression
Although application of linear regression is straightforward and characterized by many useful statistical properties, in the real life many processes have no...
Jacobian
Non-linear regression
Although application of linear regression is straightforward and characterized by many useful statistical properties, in the real life many processes have no...
Levenberg–Marquardt-algorithm
Non-linear regression
Although application of linear regression is straightforward and characterized by many useful statistical properties, in the real life many processes have no...
linear-programming
Simplex method
Linear programming aims at finding the best outcome of mathematical problems provided constraints. The requirements for the linear programming problems are d...
constraints
Simplex method
Linear programming aims at finding the best outcome of mathematical problems provided constraints. The requirements for the linear programming problems are d...
objective-function
Simplex method
Linear programming aims at finding the best outcome of mathematical problems provided constraints. The requirements for the linear programming problems are d...
maximization
Simplex method
Linear programming aims at finding the best outcome of mathematical problems provided constraints. The requirements for the linear programming problems are d...
tangent-line
Mean value theorem
The mean value theorem states that if a function is continuous and differentiable between two points then there exists a point between them where the tangent...
Wolfe-conditions
Line search
Generally speaking, line search is one of the two (the other is trust region) major strategies in non-linear optimization. Line search methods first determin...
Armijo-rule
Line search
Generally speaking, line search is one of the two (the other is trust region) major strategies in non-linear optimization. Line search methods first determin...
search-direction
Line search
Generally speaking, line search is one of the two (the other is trust region) major strategies in non-linear optimization. Line search methods first determin...
cauchy-point
Trust region
Trust region is a general strategy of optimization of an objective function by constructing an approximation of the function around a certain point and measu...
dogleg-method
Trust region
Trust region is a general strategy of optimization of an objective function by constructing an approximation of the function around a certain point and measu...
line-search
Trust region
Trust region is a general strategy of optimization of an objective function by constructing an approximation of the function around a certain point and measu...
supervised-machine-learning
Support vector machine
Support vector machine (SVM) is mainly used for building classification models in supervised machine learning, however the ideas employed by SVM can be also ...
svm
Support vector machine
Support vector machine (SVM) is mainly used for building classification models in supervised machine learning, however the ideas employed by SVM can be also ...
function-minimization
Conjugate gradient
Conjugate gradient methods are well suited for solving both linear and non-linear systems of equations.
rank-one-update
Quasi-Newton methods
The classical Newton’s method for optimization is known for being computationally expensive when it comes to calculating the gradient of the Hessian matrix a...
jacobian
Quasi-Newton methods
The classical Newton’s method for optimization is known for being computationally expensive when it comes to calculating the gradient of the Hessian matrix a...
Sherman-Morrison-formula
Quasi-Newton methods
The classical Newton’s method for optimization is known for being computationally expensive when it comes to calculating the gradient of the Hessian matrix a...
cross-product
Quasi-Newton methods
The classical Newton’s method for optimization is known for being computationally expensive when it comes to calculating the gradient of the Hessian matrix a...
secant-method
Root finding algorithms
In numeric analysis root finding is equivalent to finding zeros of a continuous function. In case of complex and composite function finding values of a varia...
bisection-method
Root finding algorithms
In numeric analysis root finding is equivalent to finding zeros of a continuous function. In case of complex and composite function finding values of a varia...
Brent's-method
Root finding algorithms
In numeric analysis root finding is equivalent to finding zeros of a continuous function. In case of complex and composite function finding values of a varia...
Newton's-method
Root finding algorithms
In numeric analysis root finding is equivalent to finding zeros of a continuous function. In case of complex and composite function finding values of a varia...
zero-finding
Root finding algorithms
In numeric analysis root finding is equivalent to finding zeros of a continuous function. In case of complex and composite function finding values of a varia...
false-position-method
Root finding algorithms
In numeric analysis root finding is equivalent to finding zeros of a continuous function. In case of complex and composite function finding values of a varia...
Broyden's-method
Root finding algorithms
In numeric analysis root finding is equivalent to finding zeros of a continuous function. In case of complex and composite function finding values of a varia...
inverse-quadratic-interpolation
Root finding algorithms
In numeric analysis root finding is equivalent to finding zeros of a continuous function. In case of complex and composite function finding values of a varia...
smoothing
LOESS
It is quite common to have a relation between two values represented as a scatterplot. However, visual representation of a scatterplot alone can often be non...
scatterplot
LOESS
It is quite common to have a relation between two values represented as a scatterplot. However, visual representation of a scatterplot alone can often be non...
least-squares
LOESS
It is quite common to have a relation between two values represented as a scatterplot. However, visual representation of a scatterplot alone can often be non...
weighting
LOESS
It is quite common to have a relation between two values represented as a scatterplot. However, visual representation of a scatterplot alone can often be non...
non-parametric-regression
LOESS
It is quite common to have a relation between two values represented as a scatterplot. However, visual representation of a scatterplot alone can often be non...
local-regression
LOESS
It is quite common to have a relation between two values represented as a scatterplot. However, visual representation of a scatterplot alone can often be non...
integral
Understanding integrals
Building intuition
area
Understanding integrals
Building intuition
net-change
Understanding integrals
Building intuition
Riemann-sum
Understanding integrals
Building intuition
fundamental-theorem-of-calculus
Understanding integrals
Building intuition
improper-integrals
Understanding integrals
Building intuition
probability
Understanding integrals
Building intuition
Understanding integrals
Building intuition
moment
Moment generating functions
In general moment generating functions (MGF) are used as an alternative for using probability density distribution. The name implies that with MGF it is poss...
mgf
Moment generating functions
In general moment generating functions (MGF) are used as an alternative for using probability density distribution. The name implies that with MGF it is poss...
probability-density-distribution
Moment generating functions
In general moment generating functions (MGF) are used as an alternative for using probability density distribution. The name implies that with MGF it is poss...
skewness
Moment generating functions
In general moment generating functions (MGF) are used as an alternative for using probability density distribution. The name implies that with MGF it is poss...
kurtosis
Moment generating functions
In general moment generating functions (MGF) are used as an alternative for using probability density distribution. The name implies that with MGF it is poss...
expected-value
Moment generating functions
In general moment generating functions (MGF) are used as an alternative for using probability density distribution. The name implies that with MGF it is poss...
Hoeffding's-inequality
Probability bounds
When applying a model to a certain process it is mostly impossible to be fully certain about the values of the parameters, the output of the process, and the...
Markov's-inequality
Probability bounds
When applying a model to a certain process it is mostly impossible to be fully certain about the values of the parameters, the output of the process, and the...
Chebyshev's-inequality
Probability bounds
When applying a model to a certain process it is mostly impossible to be fully certain about the values of the parameters, the output of the process, and the...
Chernoff-bound
Probability bounds
When applying a model to a certain process it is mostly impossible to be fully certain about the values of the parameters, the output of the process, and the...
moment-generating-function
Probability bounds
When applying a model to a certain process it is mostly impossible to be fully certain about the values of the parameters, the output of the process, and the...
Weak-law-of-large-numbers
Probability bounds
When applying a model to a certain process it is mostly impossible to be fully certain about the values of the parameters, the output of the process, and the...
Bernoulli-distribution
Probability bounds
When applying a model to a certain process it is mostly impossible to be fully certain about the values of the parameters, the output of the process, and the...
correlation
Correlation and variance in linear regression
What is covariance?
correlation-coefficients
Correlation and variance in linear regression
What is covariance?
covariation
Correlation and variance in linear regression
What is covariance?
Pearson's-correlation-coefficient
Correlation and variance in linear regression
What is covariance?
coefficient-of-determination
Correlation and variance in linear regression
What is covariance?
r-squared
Correlation and variance in linear regression
What is covariance?
adjusted-r-squared
Correlation and variance in linear regression
What is covariance?
regression-model
Correlation and variance in linear regression
What is covariance?
variance-of-coefficients
Correlation and variance in linear regression
What is covariance?
principal-components
Principal component analysis and singular value decomposition
Principal component analysis
pca
Principal component analysis and singular value decomposition
Principal component analysis
dimensionality-reduction
Principal component analysis and singular value decomposition
Principal component analysis
matrix-transformation
Principal component analysis and singular value decomposition
Principal component analysis
feature-extraction
Principal component analysis and singular value decomposition
Principal component analysis
eigendecomposition
Principal component analysis and singular value decomposition
Principal component analysis
singular-vectors
Principal component analysis and singular value decomposition
Principal component analysis
singular-values
Principal component analysis and singular value decomposition
Principal component analysis
scatter-matrix
Principal component analysis and singular value decomposition
Principal component analysis
truncated-svd
Principal component analysis and singular value decomposition
Principal component analysis
linear-model
Degrees of freedom in statistics
In statistics degrees of freedom is the number of independent elements forming a final statistic which are free to vary without violation of an imposed const...
type-1-error
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
type-2-error
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
p-value-hacking
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
alternative-hypothesis
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
power-analysis
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
sample-size
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
family-wise-error-rate
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
multiple-testing
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
Benjamini–Hochberg-procedure
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
FWER
Hypothesis testing
In this article What is hypothesis testing P-value hacking Type I and type II errors Statistical power Power analysis Multiple hypothesis testin...
Pearson's-chi-square-test
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
z-test
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
F-distribution
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
continuity-correction
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
Welch's-t-test
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
exact-Fisher's-test
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
Barnard's-test
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
contingency-table
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
G-test
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
paired-t-test
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
McNemar's-test
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
ANOVA
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
analysis-of-variance
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
F-test
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
F-statistic
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
Games-Howell-test
Hypothesis test parametric statistics
This is an overview of the most commonly used parametric statistics in hypothesis testing explaining when to use each. The parametric statistics here assume ...
ols
Linear least squares
Ordinary least squares method (OLS) is a widely used method which estimates the parameters of a linear regression by minimizing the sum of the squares of the...
normal-equations
Linear least squares
Ordinary least squares method (OLS) is a widely used method which estimates the parameters of a linear regression by minimizing the sum of the squares of the...
qr-decomposition
Linear least squares
Ordinary least squares method (OLS) is a widely used method which estimates the parameters of a linear regression by minimizing the sum of the squares of the...
sample
Sampling distribution
In real life situations it is often impossible to calculate statistics such as mean and variance for the whole population. Instead we may only be able to dra...
Central-limit-theorem
Sampling distribution
In real life situations it is often impossible to calculate statistics such as mean and variance for the whole population. Instead we may only be able to dra...
standard-error
Sampling distribution
In real life situations it is often impossible to calculate statistics such as mean and variance for the whole population. Instead we may only be able to dra...
standard-deviation
Sampling distribution
In real life situations it is often impossible to calculate statistics such as mean and variance for the whole population. Instead we may only be able to dra...
sample-variance
Sampling distribution
In real life situations it is often impossible to calculate statistics such as mean and variance for the whole population. Instead we may only be able to dra...
sample-mean
Sampling distribution
In real life situations it is often impossible to calculate statistics such as mean and variance for the whole population. Instead we may only be able to dra...
unbiased-estimate
Sampling distribution
In real life situations it is often impossible to calculate statistics such as mean and variance for the whole population. Instead we may only be able to dra...
confidence-level
Sampling distribution
In real life situations it is often impossible to calculate statistics such as mean and variance for the whole population. Instead we may only be able to dra...
t-value
Sampling distribution
In real life situations it is often impossible to calculate statistics such as mean and variance for the whole population. Instead we may only be able to dra...
t-statistic
Sampling distribution
In real life situations it is often impossible to calculate statistics such as mean and variance for the whole population. Instead we may only be able to dra...
z-statistic
Sampling distribution
In real life situations it is often impossible to calculate statistics such as mean and variance for the whole population. Instead we may only be able to dra...
statistical-inference
Maximum likelihood estimation
In statistics the likelihood function is used to estimate the goodness of fit of the parameters to the observed data. The maximum likelihood estimation (MLE)...
goodness-of-fit
Maximum likelihood estimation
In statistics the likelihood function is used to estimate the goodness of fit of the parameters to the observed data. The maximum likelihood estimation (MLE)...
joint-probability
Maximum likelihood estimation
In statistics the likelihood function is used to estimate the goodness of fit of the parameters to the observed data. The maximum likelihood estimation (MLE)...
log-likelihood
Maximum likelihood estimation
In statistics the likelihood function is used to estimate the goodness of fit of the parameters to the observed data. The maximum likelihood estimation (MLE)...
likelihood-ratio
Maximum likelihood estimation
In statistics the likelihood function is used to estimate the goodness of fit of the parameters to the observed data. The maximum likelihood estimation (MLE)...
likelihood-ratio-test
Maximum likelihood estimation
In statistics the likelihood function is used to estimate the goodness of fit of the parameters to the observed data. The maximum likelihood estimation (MLE)...
Wald-test
Maximum likelihood estimation
In statistics the likelihood function is used to estimate the goodness of fit of the parameters to the observed data. The maximum likelihood estimation (MLE)...
Wald-statistic
Maximum likelihood estimation
In statistics the likelihood function is used to estimate the goodness of fit of the parameters to the observed data. The maximum likelihood estimation (MLE)...
chi-squre-distribution
Maximum likelihood estimation
In statistics the likelihood function is used to estimate the goodness of fit of the parameters to the observed data. The maximum likelihood estimation (MLE)...
Lagrange-multiplier-test
Maximum likelihood estimation
In statistics the likelihood function is used to estimate the goodness of fit of the parameters to the observed data. The maximum likelihood estimation (MLE)...
score-test
Maximum likelihood estimation
In statistics the likelihood function is used to estimate the goodness of fit of the parameters to the observed data. The maximum likelihood estimation (MLE)...
Breusch–Pagan-test
Heteroscedasticity
What is heteroscedasticity
White-test
Heteroscedasticity
What is heteroscedasticity
nlp
Text similarity
Doing text analysis you may encounter a case when you need to select the most similar text or a group of text to another chosen text. One way to do it is to...
natural-language-processing
Text similarity
Doing text analysis you may encounter a case when you need to select the most similar text or a group of text to another chosen text. One way to do it is to...
documents-similarity
Text similarity
Doing text analysis you may encounter a case when you need to select the most similar text or a group of text to another chosen text. One way to do it is to...
vector-magnitude
Text similarity
Doing text analysis you may encounter a case when you need to select the most similar text or a group of text to another chosen text. One way to do it is to...
multiple-regression
Linear regression
Regression analysis is used for estimating the relationship between variables, usually one dependent and one or several independent variables. Having a regre...
multivariable-regression
Linear regression
Regression analysis is used for estimating the relationship between variables, usually one dependent and one or several independent variables. Having a regre...
gaussian-noise
Linear regression
Regression analysis is used for estimating the relationship between variables, usually one dependent and one or several independent variables. Having a regre...
multicollinearity
Linear regression
Regression analysis is used for estimating the relationship between variables, usually one dependent and one or several independent variables. Having a regre...
correlation-coefficient
Linear regression
Regression analysis is used for estimating the relationship between variables, usually one dependent and one or several independent variables. Having a regre...
Cochrane–Orcutt-estimation
Linear regression
Regression analysis is used for estimating the relationship between variables, usually one dependent and one or several independent variables. Having a regre...
Prais–Winsten-estimation
Linear regression
Regression analysis is used for estimating the relationship between variables, usually one dependent and one or several independent variables. Having a regre...
weighted-least-squares
Linear regression
Regression analysis is used for estimating the relationship between variables, usually one dependent and one or several independent variables. Having a regre...
WLS
Linear regression
Regression analysis is used for estimating the relationship between variables, usually one dependent and one or several independent variables. Having a regre...
generalized-least-squares
Linear regression
Regression analysis is used for estimating the relationship between variables, usually one dependent and one or several independent variables. Having a regre...
GLS
Linear regression
Regression analysis is used for estimating the relationship between variables, usually one dependent and one or several independent variables. Having a regre...
feasible-generalized-least-squares
Linear regression
Regression analysis is used for estimating the relationship between variables, usually one dependent and one or several independent variables. Having a regre...
FGLS
Linear regression
Regression analysis is used for estimating the relationship between variables, usually one dependent and one or several independent variables. Having a regre...
median
Hypothesis test non-parametric statistics
This is an overview of the most commonly used non-parametric statistics in hypothesis testing explaining when to use each. Unlike the parametric counterparts...
sign-test
Hypothesis test non-parametric statistics
This is an overview of the most commonly used non-parametric statistics in hypothesis testing explaining when to use each. Unlike the parametric counterparts...
Wilcoxon-signed-rank-test
Hypothesis test non-parametric statistics
This is an overview of the most commonly used non-parametric statistics in hypothesis testing explaining when to use each. Unlike the parametric counterparts...
Mann–Whitney-U-test
Hypothesis test non-parametric statistics
This is an overview of the most commonly used non-parametric statistics in hypothesis testing explaining when to use each. Unlike the parametric counterparts...
U-statistic
Hypothesis test non-parametric statistics
This is an overview of the most commonly used non-parametric statistics in hypothesis testing explaining when to use each. Unlike the parametric counterparts...
change-point
Change point analysis
Change point analysis is basically used for determining whether and where an ordered series of values (usually time series) changed their behaviour. The chan...
anomaly-detections
Change point analysis
Change point analysis is basically used for determining whether and where an ordered series of values (usually time series) changed their behaviour. The chan...
search-method
Change point analysis
Change point analysis is basically used for determining whether and where an ordered series of values (usually time series) changed their behaviour. The chan...
Hausdorf-metric
Change point analysis
Change point analysis is basically used for determining whether and where an ordered series of values (usually time series) changed their behaviour. The chan...
breakpoint
Change point analysis
Change point analysis is basically used for determining whether and where an ordered series of values (usually time series) changed their behaviour. The chan...
break-point
Change point analysis
Change point analysis is basically used for determining whether and where an ordered series of values (usually time series) changed their behaviour. The chan...
F1-score
Change point analysis
Change point analysis is basically used for determining whether and where an ordered series of values (usually time series) changed their behaviour. The chan...
statistics
Statistical distributions
A set of discrete values or a range of continuous values of a random variable is characterized by a certain probability distribution. In statistics various d...
z-table
Statistical distributions
A set of discrete values or a range of continuous values of a random variable is characterized by a certain probability distribution. In statistics various d...
six-sigma
Statistical distributions
A set of discrete values or a range of continuous values of a random variable is characterized by a certain probability distribution. In statistics various d...
central-limit-theorem
Statistical distributions
A set of discrete values or a range of continuous values of a random variable is characterized by a certain probability distribution. In statistics various d...
probability-mass-function
Statistical distributions
A set of discrete values or a range of continuous values of a random variable is characterized by a certain probability distribution. In statistics various d...
chi-square-distribution
Statistical distributions
A set of discrete values or a range of continuous values of a random variable is characterized by a certain probability distribution. In statistics various d...
Bernoulli-trial
Statistical distributions
A set of discrete values or a range of continuous values of a random variable is characterized by a certain probability distribution. In statistics various d...
geometric-distribution
Statistical distributions
A set of discrete values or a range of continuous values of a random variable is characterized by a certain probability distribution. In statistics various d...
exponential-distribution
Statistical distributions
A set of discrete values or a range of continuous values of a random variable is characterized by a certain probability distribution. In statistics various d...
poisson-distribution
Statistical distributions
A set of discrete values or a range of continuous values of a random variable is characterized by a certain probability distribution. In statistics various d...
uniform-distribution
Statistical distributions
A set of discrete values or a range of continuous values of a random variable is characterized by a certain probability distribution. In statistics various d...
negative-binomial-distribution
Statistical distributions
A set of discrete values or a range of continuous values of a random variable is characterized by a certain probability distribution. In statistics various d...
studentized-range-distribution
Statistical distributions
A set of discrete values or a range of continuous values of a random variable is characterized by a certain probability distribution. In statistics various d...
beta-distribution
Statistical distributions
A set of discrete values or a range of continuous values of a random variable is characterized by a certain probability distribution. In statistics various d...
gamma-function
Statistical distributions
A set of discrete values or a range of continuous values of a random variable is characterized by a certain probability distribution. In statistics various d...
Gaussian-distribution
Statistical distributions
A set of discrete values or a range of continuous values of a random variable is characterized by a certain probability distribution. In statistics various d...
prior-probability
Bayesian inference
Bayesian inference is the method of statistical inference where an estimated probability is updated when new data arrives. In a way, it may be viewed as an u...
Bayes-rule
Bayesian inference
Bayesian inference is the method of statistical inference where an estimated probability is updated when new data arrives. In a way, it may be viewed as an u...
maximum-likelihood-estimation
Bayesian inference
Bayesian inference is the method of statistical inference where an estimated probability is updated when new data arrives. In a way, it may be viewed as an u...
posterior-probability
Bayesian inference
Bayesian inference is the method of statistical inference where an estimated probability is updated when new data arrives. In a way, it may be viewed as an u...
MAP
Bayesian inference
Bayesian inference is the method of statistical inference where an estimated probability is updated when new data arrives. In a way, it may be viewed as an u...
maximum-a-posteriori
Bayesian inference
Bayesian inference is the method of statistical inference where an estimated probability is updated when new data arrives. In a way, it may be viewed as an u...
conjugate-prior-distribution
Bayesian inference
Bayesian inference is the method of statistical inference where an estimated probability is updated when new data arrives. In a way, it may be viewed as an u...
expected-a-posteriori
Bayesian inference
Bayesian inference is the method of statistical inference where an estimated probability is updated when new data arrives. In a way, it may be viewed as an u...
EAP
Bayesian inference
Bayesian inference is the method of statistical inference where an estimated probability is updated when new data arrives. In a way, it may be viewed as an u...
marginal-probability
Bayesian inference
Bayesian inference is the method of statistical inference where an estimated probability is updated when new data arrives. In a way, it may be viewed as an u...
vector-length
Vector projection
In order to understand more advanced concepts from vector operations it is good to have an intuition on such terms as vector projection. Below is a two-dimen...
vector-multiplication
Vector projection
In order to understand more advanced concepts from vector operations it is good to have an intuition on such terms as vector projection. Below is a two-dimen...
Euclidean-distance
Distance metrics
In machine learning the distance metrics are used as a measure of similarity between two datapoints - smaller distance means more similarity. They are utiliz...
Manhattan-distance
Distance metrics
In machine learning the distance metrics are used as a measure of similarity between two datapoints - smaller distance means more similarity. They are utiliz...
Minkowski-distance
Distance metrics
In machine learning the distance metrics are used as a measure of similarity between two datapoints - smaller distance means more similarity. They are utiliz...
vector-norm
Distance metrics
In machine learning the distance metrics are used as a measure of similarity between two datapoints - smaller distance means more similarity. They are utiliz...
normed-vector-space
Distance metrics
In machine learning the distance metrics are used as a measure of similarity between two datapoints - smaller distance means more similarity. They are utiliz...
Mahalanobis-distance
Distance metrics
In machine learning the distance metrics are used as a measure of similarity between two datapoints - smaller distance means more similarity. They are utiliz...
cosine-distance
Distance metrics
In machine learning the distance metrics are used as a measure of similarity between two datapoints - smaller distance means more similarity. They are utiliz...
cosine-similarity
Distance metrics
In machine learning the distance metrics are used as a measure of similarity between two datapoints - smaller distance means more similarity. They are utiliz...
Hamming-distance
Distance metrics
In machine learning the distance metrics are used as a measure of similarity between two datapoints - smaller distance means more similarity. They are utiliz...
relative-entropy
Distance metrics
In machine learning the distance metrics are used as a measure of similarity between two datapoints - smaller distance means more similarity. They are utiliz...
bayesian-statistics
Variational inference
Variational inference is a technique which is usually employed for approximating complex models, where computation of the exact conditional distribution is i...
evidence-lower-bound
Variational inference
Variational inference is a technique which is usually employed for approximating complex models, where computation of the exact conditional distribution is i...
ELBO
Variational inference
Variational inference is a technique which is usually employed for approximating complex models, where computation of the exact conditional distribution is i...
mean-field-variational-family
Variational inference
Variational inference is a technique which is usually employed for approximating complex models, where computation of the exact conditional distribution is i...
coordinate-ascent-variational-inference
Variational inference
Variational inference is a technique which is usually employed for approximating complex models, where computation of the exact conditional distribution is i...
CAVI
Variational inference
Variational inference is a technique which is usually employed for approximating complex models, where computation of the exact conditional distribution is i...
gaussian-distribution
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
log-marginal-likelihood
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
sparse-variational-gaussian-process
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
variational-inference
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
likelihood
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
marginal-likelihood
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
whitened-parameterization
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
parameterization-trick
Gaussian process
The Gaussian process may be viewed as a prediction technique which mainly solves regression problems by fitting a line to some given data (although it may be...
binary-classification
Logistic regression
Logistic regression is one of the simplests algorithms for binary classification, and it is based on the linear regression. In its core, it uses a linear com...
logistic-function
Logistic regression
Logistic regression is one of the simplests algorithms for binary classification, and it is based on the linear regression. In its core, it uses a linear com...
odds
Logistic regression
Logistic regression is one of the simplests algorithms for binary classification, and it is based on the linear regression. In its core, it uses a linear com...
log-odds
Logistic regression
Logistic regression is one of the simplests algorithms for binary classification, and it is based on the linear regression. In its core, it uses a linear com...
regularization
Logistic regression
Logistic regression is one of the simplests algorithms for binary classification, and it is based on the linear regression. In its core, it uses a linear com...
overfittings
Regularization
Regularization is a process of simplifying complex machine learning models in order to prevent overfitting. It may be applied explicitly by adding a penalty ...
L2-regularization
Regularization
Regularization is a process of simplifying complex machine learning models in order to prevent overfitting. It may be applied explicitly by adding a penalty ...
L1-regularization
Regularization
Regularization is a process of simplifying complex machine learning models in order to prevent overfitting. It may be applied explicitly by adding a penalty ...
Ridge-regularization
Regularization
Regularization is a process of simplifying complex machine learning models in order to prevent overfitting. It may be applied explicitly by adding a penalty ...
Lasso-regularization
Regularization
Regularization is a process of simplifying complex machine learning models in order to prevent overfitting. It may be applied explicitly by adding a penalty ...
Ridge-regression
Regularization
Regularization is a process of simplifying complex machine learning models in order to prevent overfitting. It may be applied explicitly by adding a penalty ...
Lasso-regression
Regularization
Regularization is a process of simplifying complex machine learning models in order to prevent overfitting. It may be applied explicitly by adding a penalty ...
Elastic-Net
Regularization
Regularization is a process of simplifying complex machine learning models in order to prevent overfitting. It may be applied explicitly by adding a penalty ...
logistic-regression
Neural Networks
In this article
activation-function
Neural Networks
In this article
neuron
Neural Networks
In this article
perceptron
Neural Networks
In this article
deep-neural-networks
Neural Networks
In this article
node
Neural Networks
In this article
backpropagation
Neural Networks
In this article
ReLU
Neural Networks
In this article
bias
Neural Networks
In this article
vanishing-gradient
Neural Networks
In this article
saturated-neuron
Neural Networks
In this article
dying-ReLU
Neural Networks
In this article
ELU
Neural Networks
In this article
softmax
Neural Networks
In this article
feed-forward-networks
Neural Networks
In this article
mse
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
l2
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
mean-squared-error
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
quadratic-loss
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
l1
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
mean-absolute-error
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
mae
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
m-estimator
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
huber-loss
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
bisquare-loss-function
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
entropy
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
cross-entropy
Loss functions
In machine learning the loss function (or cost, or cost function) is something that lets the machine to actually “learn”. The loss function is the function o...
Durbin–Watson-test
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
Durbin–Watson-statistic
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
white-noise
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
Breusch–Godfrey-test
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
ADF
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
KPSS
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
Kwiatkowski-Phillips-Schmidt-Shin-test
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
Dickey-Fuller-test
Time series tests
This article makes an overview of most commonly used statistical tests used in validation of the assumptions of the time series.
MA
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
ARMA
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
autoregression
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
moving-average
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
random-walk
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
differencing
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
ACF
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
autocorrelation-function
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
PACF
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
partial-autocorrelation-function
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
SARIMA
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
SARIMAX
ARIMA models
In this article we shall make a breakdown of the basic ARIMA time-series model. We shall describe each component of the model: autoregressive process (AR) an...
trend
Exponential smoothing models
In this article we shall make a breakdown of the exponential smoothing models which are used in time series forecasting.
seasonal-component
Exponential smoothing models
In this article we shall make a breakdown of the exponential smoothing models which are used in time series forecasting.
level-component
Exponential smoothing models
In this article we shall make a breakdown of the exponential smoothing models which are used in time series forecasting.
Holt-linear-trend
Exponential smoothing models
In this article we shall make a breakdown of the exponential smoothing models which are used in time series forecasting.
damped-trend
Exponential smoothing models
In this article we shall make a breakdown of the exponential smoothing models which are used in time series forecasting.
Holt-Winters-seasonal-method
Exponential smoothing models
In this article we shall make a breakdown of the exponential smoothing models which are used in time series forecasting.
Holt-linear-trend-method
Exponential smoothing models
In this article we shall make a breakdown of the exponential smoothing models which are used in time series forecasting.
state-space-model
Exponential smoothing models
In this article we shall make a breakdown of the exponential smoothing models which are used in time series forecasting.
additive-error
Exponential smoothing models
In this article we shall make a breakdown of the exponential smoothing models which are used in time series forecasting.
multiplicative-error
Exponential smoothing models
In this article we shall make a breakdown of the exponential smoothing models which are used in time series forecasting.
decision-tree
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
bagging
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
random-forest
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
bootstrap-aggregation
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
ensemble-techniques
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
extremely-randomized-trees
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
extra-trees
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
mean-decrease-in-impurity
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
MDI
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
AdaBoost
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
boosting
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
gradient-boosting
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
histogram-based-gradient-boosting
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
xgboost
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
extreme-gradient-boosting
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
gradient-boosting-machine
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
GBM
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
LightGBM
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
gradient-based-one-side-sampling
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
GOSS
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
exclusive-feature-bundling
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
EFB
Tree-like algorithms
The tree-like algorithms are a wondrous array of machine learning methods that draw inspiration from the branching structure of the natural world. They emplo...
Euler-number
Imaginary number and rotation of vectors
Introduction
real-number
Imaginary number and rotation of vectors
Introduction
trigonometric-function
Imaginary number and rotation of vectors
Introduction
Euler's-formula
Imaginary number and rotation of vectors
Introduction
vector-rotation
Imaginary number and rotation of vectors
Introduction
Euler-identity
Imaginary number and rotation of vectors
Introduction
missing-values
Interpolation
Generally speaking, interpolation is a way of generating new data points which exactly fit into some given set of points. It is used for approximation of com...
spline
Interpolation
Generally speaking, interpolation is a way of generating new data points which exactly fit into some given set of points. It is used for approximation of com...
nearest-neighbour
Interpolation
Generally speaking, interpolation is a way of generating new data points which exactly fit into some given set of points. It is used for approximation of com...
Vandermonde-matrix
Interpolation
Generally speaking, interpolation is a way of generating new data points which exactly fit into some given set of points. It is used for approximation of com...
Lagrange-multipliers
Interpolation
Generally speaking, interpolation is a way of generating new data points which exactly fit into some given set of points. It is used for approximation of com...
Newton's-divided-differences
Interpolation
Generally speaking, interpolation is a way of generating new data points which exactly fit into some given set of points. It is used for approximation of com...
cubic-spline
Interpolation
Generally speaking, interpolation is a way of generating new data points which exactly fit into some given set of points. It is used for approximation of com...
Bezier-curve
Interpolation
Generally speaking, interpolation is a way of generating new data points which exactly fit into some given set of points. It is used for approximation of com...
periodic-function
Fourier series
A Fourier series is a way to approximate a periodic function as the sum of simple sine and cosine functions which makes it useful for analyzing and synthesiz...
sine
Fourier series
A Fourier series is a way to approximate a periodic function as the sum of simple sine and cosine functions which makes it useful for analyzing and synthesiz...
fundamental-period
Fourier series
A Fourier series is a way to approximate a periodic function as the sum of simple sine and cosine functions which makes it useful for analyzing and synthesiz...
harmonic-frequency
Fourier series
A Fourier series is a way to approximate a periodic function as the sum of simple sine and cosine functions which makes it useful for analyzing and synthesiz...
RBF
Kernels overview
In this article What are kernels Kernel density estimation Kernels in Gaussian process Squared exponential kernel Matern kernel ...
radial-basis-function-kernel
Kernels overview
In this article What are kernels Kernel density estimation Kernels in Gaussian process Squared exponential kernel Matern kernel ...
squared-exponential-kernel
Kernels overview
In this article What are kernels Kernel density estimation Kernels in Gaussian process Squared exponential kernel Matern kernel ...
matern-kernel
Kernels overview
In this article What are kernels Kernel density estimation Kernels in Gaussian process Squared exponential kernel Matern kernel ...
rational-quadratic-kernel
Kernels overview
In this article What are kernels Kernel density estimation Kernels in Gaussian process Squared exponential kernel Matern kernel ...
periodic-kernel
Kernels overview
In this article What are kernels Kernel density estimation Kernels in Gaussian process Squared exponential kernel Matern kernel ...
linear-kernel
Kernels overview
In this article What are kernels Kernel density estimation Kernels in Gaussian process Squared exponential kernel Matern kernel ...
dot-product-kernel
Kernels overview
In this article What are kernels Kernel density estimation Kernels in Gaussian process Squared exponential kernel Matern kernel ...
kernel-density-estimation
Kernels overview
In this article What are kernels Kernel density estimation Kernels in Gaussian process Squared exponential kernel Matern kernel ...
KDE
Kernels overview
In this article What are kernels Kernel density estimation Kernels in Gaussian process Squared exponential kernel Matern kernel ...
kernel-bandwidth
Kernels overview
In this article What are kernels Kernel density estimation Kernels in Gaussian process Squared exponential kernel Matern kernel ...
segmentation
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
unsupervised-learning
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
hierarchical-clustering
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
agglomerative-clustering
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
dendrogram
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
k-means
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
ward-linkage
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
affinity-propagation
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
spectral-clustering
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
DBSCAN
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
HDBSCAN
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
OPTICS
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
BIRCH
Clustering overview
Clustering may be viewed as grouping of data points according to their similarity so that the most similar points end up being in the same cluster. Clusterin...
unsupervised-machine-learning
Outliers detection
This article explores machine learning models used to identify unusual or atypical observations within data sets.
semi-supervised-machine-learning
Outliers detection
This article explores machine learning models used to identify unusual or atypical observations within data sets.
anomaly-detection
Outliers detection
This article explores machine learning models used to identify unusual or atypical observations within data sets.
novelty-detection
Outliers detection
This article explores machine learning models used to identify unusual or atypical observations within data sets.
local-outlier-factor
Outliers detection
This article explores machine learning models used to identify unusual or atypical observations within data sets.
LOF
Outliers detection
This article explores machine learning models used to identify unusual or atypical observations within data sets.
GLOSH
Outliers detection
This article explores machine learning models used to identify unusual or atypical observations within data sets.
isolation-forest
Outliers detection
This article explores machine learning models used to identify unusual or atypical observations within data sets.