Related Concepts

Test Statistic

ELI5: Imagine you want to know if a certain trading strategy is better than just random luck. The test statistic is like the score you get after you compare the performance of your trading strategy to what you'd expect from random luck.

Example: You have a trading strategy that says if the stock price goes up three days in a row, it will go up again on the fourth day. After testing this on 100 instances, you calculate a score (test statistic) to see if this pattern is really significant or just by chance.

P-value (p statistic)

Standard Definition: The probability of observing a test statistic as extreme as, or more extreme than, the statistic computed from the sample, given that the null hypothesis is true.

ELI5: Once you have your score from the test statistic, you want to know how surprised you should be by it. The p-value tells you the probability of seeing a score as extreme as (or more extreme than) what you observed if your trading strategy was really just random luck. If the p-value is very small, it means your results are surprising under the assumption of random luck, and you might conclude that your trading strategy has merit.

After making that basketball shot, the p-value is like asking, "What are the chances of making an even better shot if I'm just shooting randomly?".

Example: Using the previous trading strategy, let's say your score was really high. If the p-value is 0.01 (or 1%), it means that there's only a 1% chance of seeing such a strong pattern if the stock was moving purely by chance. That might make you think your strategy is onto something.

If you're testing a strategy and find it's profitable, the p-value tells you the chance of seeing this profit just by luck.

Sample Statistic

ELI5: This is a number you get from the data you collected. For instance, if you were to look at the average profit you made from 10 trades, that average would be your sample statistic.

Example: You decide to test a new trading algorithm. After 10 trades, you calculate the average profit (or loss) per trade. This average is your sample statistic. If your sample statistic is $100 profit per trade, it gives you an idea of how well your trading strategy did, but you might still want to do a statistical test (using a test statistic and p-value) to see if this result is significantly better than just random trading.

In the world of trading, these concepts are crucial. Knowing whether your strategy is truly effective or if you just got lucky can mean the difference between long-term profitability and potential bankruptcy.

Significance Level

Standard Definition:

The significance level, often denoted by the Greek letter $\alpha$ (alpha), is the probability of rejecting the null hypothesis when it is actually true. It represents the likelihood of making a Type I error. Common choices for $\alpha$ include 0.05, 0.01, and 0.10.

ELI5: Imagine you have a magic coin that you believe lands heads 60% of the time, instead of the usual 50%. To test your belief, you decide to flip the coin many times. The significance level is like your threshold of skepticism. It's how strict you are before you decide that the coin is indeed magical. If you set a significance level of 5% (or 0.05), you're saying, "I want to be 95% sure that this coin is magical before I believe it." If the test results pass this strict 5% threshold, you'd believe the coin has powers. Otherwise, you'd think the coin might just be normal, and any odd results were due to chance.

Example: Back to trading! Let's say you have a new trading strategy. You want to test if it's genuinely effective or just got lucky a few times. So, you set a significance level of 5% (0.05). After analyzing the strategy with some statistical tests, you get a p-value of 0.03. Since 0.03 is less than 0.05, you would reject the idea that your trading strategy's success was just due to luck and conclude that it's probably effective. But remember, there's still a 3% chance you might be wrong about this conclusion, which is the risk you take with a 5% significance level.

• If p value < $\alpha$, we reject $H_0$
• If p value > = $\alpha$, we fail to reject $H_0$

Cumulative Distribution Function

Standard Definition: The CDF of a random variable is a function that gives the probability that the random variable will take a value less than or equal to a given value. It's used to describe the distribution of random variables. The CDF maps a value to its percentile rank in the distribution.

ELI5: Imagine you have a big jar filled with jellybeans, and each jellybean has a number on it. The CDF is like a superhero ability that lets you instantly see how many jellybeans have numbers below a certain value. So if you pick a number, let's say 50, and use your CDF power on the jar, it would immediately tell you the fraction or percentage of jellybeans with numbers 50 or less.

Example in Trading: Consider stock prices. If you have historical data of a particular stock's closing price, the CDF could tell you the probability that the stock will close below a certain price on any given day. For example, if you check the CDF for a price of $100 and get a value of 0.8 (or 80%), it means that 80% of the time in your historical data, the stock closed at $100 or below. This can help traders get a sense of how often prices reach or fall below certain levels.

In mathematical terms, for continuous random variables: $CDF(x) = P(X \leq x)$ Where $X$ is a random variable, and $x$ is a specific value.

For discrete random variables, the CDF is a step function, jumping up at each value that the random variable can take on. For continuous random variables, the CDF is a smooth curve.

Type I & II errors

Type I and Type II errors are terms used in statistics to describe potential mistakes made in hypothesis testing.

Type I Error (False Positive):

• Standard Definition: Rejecting the null hypothesis when it is actually true.
• ELI5: It's like accusing someone of a crime they didn't commit. You thought they were guilty, but they were actually innocent.
• Probability of a Type I error: Represented by the Greek letter $\alpha$ (alpha), which is also known as the significance level of a test.

Type II Error (False Negative):

• Standard Definition: Failing to reject the null hypothesis when the alternative hypothesis is true.
• ELI5: It's like letting a guilty person go free. You thought they were innocent, but they were actually guilty.
• Probability of a Type II error: Represented by the Greek letter $\beta$ (beta).

Example:

Imagine you have a metal detector, and you're trying to find out if people are carrying metal items.

• Null Hypothesis ($H_0$): A person is not carrying any metal.
• Alternative Hypothesis ($H_1$): A person is carrying metal.

Now, consider four scenarios:

1. The metal detector beeps, and the person does have metal on them. This is a True Positive.
2. The metal detector beeps, but the person doesn't have any metal. This is a False Positive (Type I Error).
3. The metal detector doesn't beep, and the person doesn't have metal. This is a True Negative.
4. The metal detector doesn't beep, but the person does have metal. This is a False Negative (Type II Error).

In statistical tests, we aim to minimize both Type I and Type II errors, but there's often a trade-off. By making our test more strict to reduce Type I errors, we might increase the chance of making Type II errors, and vice versa.