p-value dan Confidence Interval

01Core Definitions and Conceptual Framework

The p-value is the probability of observing data as extreme as, or more extreme than, the results actually obtained, calculated under the assumption that the null hypothesis is true. It is not the probability that the null hypothesis is correct. This distinction is the single most tested conceptual point in biostatistics on any licensing exam.

When a study reports $p=0.03p\; =\; 0.03$, it means there is a 3% chance of seeing a result this extreme if the null hypothesis were true. Because this falls below the conventional threshold of $\alpha =0.05\backslash alpha\; =\; 0.05$, we reject the null hypothesis and call the result statistically significant. If $p=0.08p\; =\; 0.08$, we fail to reject the null hypothesis. Note the language: we never say we "accept" the null. We say we fail to reject it, because absence of evidence is not evidence of absence.

The confidence interval (CI) provides a range estimate rather than a single point estimate. A 95% CI means that if the study were repeated many times under identical conditions, approximately 95% of the constructed intervals would contain the true population parameter. It does not mean there is a 95% probability that the true value lies within this particular interval. The true value is fixed; it either falls inside the interval or it does not.

The width of the confidence interval reflects precision. A narrow CI suggests the study estimated the parameter with high precision, while a wide CI suggests uncertainty. Three factors drive CI width: sample size (larger samples produce narrower intervals), variability in the data (less variability narrows the interval), and the confidence level chosen (a 99% CI is wider than a 95% CI for the same data).

02Hypothesis Testing Framework

Type I error (alpha error) is the mistake of concluding a difference exists when it does not. Think of it as a false alarm. The probability of making this error is set before the study begins and equals $\alpha \backslash alpha$. If $\alpha =0.05\backslash alpha\; =\; 0.05$, you accept a 5% risk of a false positive.

Type II error (beta error) is the mistake of concluding no difference exists when there actually is one. Think of it as a missed finding. The probability of making this error is $\beta \backslash beta$. Power, defined as $1-\beta 1\; -\; \backslash beta$, is the study's ability to detect a real effect. The most commonly tested way to increase power is to increase sample size. Other methods include increasing the effect size, increasing alpha (which raises the false positive risk), or reducing measurement variability.

A helpful mnemonic: think of alpha as the probability of convicting an innocent person (false positive), and beta as the probability of letting a guilty person go free (false negative).

03P-Value: Interpretation and Common Pitfalls

The p-value answers one narrow question: how compatible is the observed data with the null hypothesis? It says nothing about the magnitude of the effect or its clinical importance. A study with 500,000 participants might find that Drug A lowers blood pressure by 0.5 mmHg more than placebo with . This is statistically significant but clinically meaningless. Conversely, a small pilot study might find a 15 mmHg reduction with $p=0.09p\; =\; 0.09$. This is not statistically significant at $\alpha =0.05\backslash alpha\; =\; 0.05$, but the effect size may be clinically important and warrant a larger trial.

Statistical significance without clinical significance is a classic exam trap. When a vignette gives you a very small p-value but the actual difference between groups is trivial, the correct answer is usually that the result is statistically significant but not clinically meaningful.

04Confidence Interval: Construction and Interpretation

The general formula for a confidence interval is:

$$CI=\text{Point Estimate}\pm (Z\times \text{Standard Error})CI\; =\; \backslash text\{Point\; Estimate\}\; \backslash pm\; (Z\; \backslash times\; \backslash text\{Standard\; Error\})$$

For a 95% CI, the Z-value is 1.96. For a 99% CI, the Z-value is 2.576.

The interpretation rule is straightforward. For any ratio measure (RR, OR, HR), the null value is 1. If the 95% CI does not cross 1, the result is statistically significant at $\alpha =0.05\backslash alpha\; =\; 0.05$. For absolute differences (mean difference, risk difference), the null value is 0. If the 95% CI does not cross 0, the result is significant.

Consider a study reporting an odds ratio of 2.4 with a 95% CI of 1.3 to 4.2. Because the interval does not include 1, this result is statistically significant. The exposed group has significantly higher odds of the outcome. Now consider a study reporting an OR of 1.8 with a 95% CI of 0.7 to 3.9. Despite the point estimate suggesting increased odds, the interval crosses 1, meaning we cannot rule out that there is no true association at the 95% confidence level.

The CI gives you two pieces of information that the p-value alone cannot: the direction and plausible magnitude of the effect, and the precision of the estimate. This is why many guidelines and journals now emphasize reporting CIs alongside or instead of p-values.

05The Relationship Between P-Value and Confidence Interval

P-value and confidence interval are mathematically linked. For a two-sided test at $\alpha =0.05\backslash alpha\; =\; 0.05$, if the p-value is less than 0.05, the 95% CI will not contain the null value. If the p-value is greater than or equal to 0.05, the 95% CI will contain the null value. They will never contradict each other when derived from the same test and the same alpha level.

However, the CI is more informative. Scenario B and Scenario C above both hover near the significance threshold. The p-values are 0.04 and 0.06, which fall on opposite sides of the cutoff. But looking at the CIs, both intervals overlap substantially and suggest a similar range of plausible effect sizes. This illustrates why the $p=0.05p\; =\; 0.05$ cutoff should not be treated as a rigid boundary between "real" and "not real." It is a convention, not a law of nature.

Exam questions frequently present a scenario where a CI barely crosses or barely excludes the null value and then ask whether the result is significant. The rule is mechanical: if the null is inside the interval, it is not significant at that confidence level. Do not overthink it.

06Factors That Affect P-Value and Confidence Interval Width

Sample size is the factor most commonly tested. A classic vignette describes a study that found no statistically significant difference, then asks what could be done to improve the study. If the effect size seems clinically meaningful but the p-value is above 0.05, the answer is almost always increase the sample size to increase power and narrow the confidence interval.

Be aware that increasing sample size makes it easier to detect even trivially small differences. This is why large epidemiological studies can produce very small p-values for clinically irrelevant findings.

07Exam Question Patterns and Distractors

08Traps and High-Yield Pearls

The most common way students lose points on p-value and CI questions is by misinterpreting the definition of the p-value. Test writers know that the intuitive reading ("probability the null is true") feels natural, and they exploit that. Every time you see a question asking what a p-value means, slow down and remember: it is the probability of the observed data (or more extreme data) under the assumption that $H_{0}H\_0$ is true. It is a statement about the data, not about the hypothesis.

The second major trap involves confusing statistical significance with clinical significance. If a vignette gives you a very large sample size and a tiny effect with a very small p-value, the question is almost certainly testing whether you recognize that statistical significance alone does not justify clinical action.

Third, know the CI rules cold. For ratios (RR, OR, HR), the null is 1. For differences, the null is 0. If the interval contains the null, the result is not significant. This is a mechanical rule and questions about it are essentially free points if you apply it consistently.

Fourth, understand that the relationship between p-value and CI is not a coincidence. They are derived from the same underlying data and test. At $\alpha =0.05\backslash alpha\; =\; 0.05$, a two-sided p-value below 0.05 will always correspond to a 95% CI that excludes the null. Questions that present both and ask you to reconcile them are testing whether you understand this linkage.

Finally, remember the hierarchy of what each concept tells you. The p-value tells you whether an effect is likely real. The CI tells you how big the effect plausibly is and how precise the estimate is. The effect size tells you whether the finding matters clinically. A complete evaluation of a study result requires all three, and the best exam answers reflect that integrated understanding.