Statistics Rules of Thumb
A compendium of statistical rules of thumb for reference.
Correlation
Correlation is the measure of the direction and strength of a linear relationship between two quantitative variables.
r is the correlation coefficient and is bounded between -1 and 1.
r = 0 = no linear relationship
r = +1 = perfect, linear positive relationship
r = -1 = perfect, linear negative relationship
While there is no clear agreement on what constitutes a strong or weak correlation, the following rough rules apply (RR the truthful art):
0.1–0.3 is modest.
0.3–0.5 is moderate.
0.5–0.8 is strong.
0.8–0.9 is very strong.
68-95-99.7 Rule
For Gaussian, otherwise called normal distributions, the 68-95-99.7 rule applies:
68% of observations within 1 std deviations of the mean (called σ1)
95% of observations within 2 std deviations of the mean (called σ2)
99.7% of observations within 3 std deviations of the mean (called σ3)
Scientific results in particle physics, like the Large Hadron Collider, use sigma 5 (σ5).
P-Values and Significance Level
P-values can be used in two different ways, the first is as a rule of thumb to determine roughly how strong the evidence is in support of the alternative hypothesis as follows:
P-value > 10%: insufficient evidence to support HA
P-value between 5% and 10%: there is slight evidence to support HA
P-value between 1% and 5%: there is moderate evidence to support HA
P-value between 1% and 0.1%: there is strong evidence to support HA
P-value < 0.1%: there is very strong evidence to support HA
P-values can also be used as a threshold to determine whether the alternative hypothesis is rejected or not. Fisher set the trend for this with the following four most commonly used p-values:
0.10
0.05 is the most commonly used threshold
0.02
0.01
These values correspond to odds, roughly speaking. The 0.05 value means there is a 1 in 20 chance of being wrong. Basically its just setting the odds of rejecting a true null hypothesis in favour of the alternative hypothesis.
Why use a p-value of 0.05? It is typically used for historical, socio-cultural reasons. There is not a strong mathematical reason, but practically speaking it strikes a nice balance. It was basically chosen by Fisher somewhat randomly.
Homoescadicity
The rule of thumb is that the homoescadicity assumption is met if double the smallest standard deviation is greater than the largest standard deviation.
Take the example of an two groups, group A and B. If group A has a standard deviation of 0.3357 and group B has a standard deviation of 0.4601, as double the smallest is 0.6714, which is greater than 0.4601, the rule of thumb is reached.
Skewness
-0.8 to 0.8
Kurtosis
-3.0 to 3.0
Outliers
If an observation has a corresponding residual that is more than three standard deviations from the mean it is typically considered an outlier.
Multicollinearity
The rule of thumb is that if there is a correlation coefficient of 0.8 or greater between two regressors, there is evidence of collinearity.