Hypothesis Testing - Statistics

 

1. What is Hypothesis Testing and when do we use it?

Hypothesis testing is a part of statistical analysis, where we test the assumptions made regarding a population parameter.

It is generally used when we were to compare:

  • a single group with an external standard
  • two or more groups with each other

A Parameter is a number that describes the data from the population whereas, a Statistic is a number that describes the data from a sample.

2. Terminology used

Null Hypothesis: Null hypothesis is a statistical theory that suggests there is no statistical significance exists between the populations.

Alternative Hypothesis: An Alternative hypothesis suggests there is a significant difference between the population parameters. It could be greater or smaller. Basically, it is the contrast of the Null Hypothesis.

Note: H0 must always contain equality(=). Ha always contains difference(≠, >, <).

For example, if we were to test the equality of average means (µ) of two groups:

for a two-tailed test, we define H0: µ1 = µ2 and Ha: µ1≠µ2

for a one-tailed test, we define H0: µ1 = µ2 and Ha: µ1 > µ2 or Ha: µ1 < µ2

Level of significance: Denoted by alpha or α. It is a fixed probability of wrongly rejecting a True Null Hypothesis. For example, if α=5%, that means we are okay to take a 5% risk and conclude there exists a difference when there is no actual difference.

Critical Value: Denoted by C and it is a value in the distribution beyond which leads to the rejection of the Null Hypothesis. It is compared to the test statistic.

Test Statistic: It is denoted by t and is dependent on the test that we run. It is deciding factor to reject or accept Null Hypothesis.

p-value: It is the proportion of samples (assuming the Null Hypothesis is true) that would be as extreme as the test statistic. It is denoted by the letter p.

3. Steps of Hypothesis testing

  1. Start with specifying Null and Alternative Hypotheses about a population parameter
  2. Set the level of significance (α)
  3. Collect Sample data and calculate the Test Statistic and P-value by running a Hypothesis test that well suits our data
  4. Make Conclusion: Reject or Fail to Reject Null Hypothesis

4. Decision Rules

The two methods of concluding the Hypothesis test are using the Test-statistic value, p-value.

In both methods, we start assuming the Null Hypothesis to be true, and then we reject the Null hypothesis if we find enough evidence.

The decision rule for the Test-statistic method:

  • If test statistic < critical value: Fail to reject the null hypothesis.
  • If test statistic >= critical value: Reject the null hypothesis.

The decision rule for the p-value method:

  • If p-value > alpha: Fail to reject the null hypothesis (i.e. not significant result).
  • If p-value <= alpha: Reject the null hypothesis (i.e. significant result).

 

Comments

Post a Comment

Popular posts from this blog

Transformers: Self-attention

Image
 In the realm of neural networks, three primary types are commonly discussed: Artificial Neural Networks (ANNs) : These are fully connected networks comprising input, hidden, and output layers. Each neuron in a layer is connected to every neuron in the subsequent layer, enabling complex pattern recognition through weighted connections. Convolutional Neural Networks (CNNs) : CNNs incorporate convolutional layers with kernels (filters) that slide across the input data to detect features. These networks also perform pooling operations to reduce dimensionality and flatten the data before passing it to fully connected layers. CNNs are particularly effective for image and spatial data processing. Recurrent Neural Networks (RNNs) : RNNs are designed to handle sequential data by maintaining a memory of previous inputs through their hidden states. Unlike feed-forward networks, RNNs can process input sequences of variable length, making them suitable for tasks involving time series data, nat...
Read more

Retrieval Augmented Generation(RAG)

Image
 We Know core ideas like NLP, LLM. Let's look at how we might build utilizing LLM and the deployment aspect using RAG. Retrieval-Augmented Generation (RAG) is a technique in natural language processing that combines retrieval-based and generation-based approaches to create more accurate and informative responses. Let's break down each point: 1. RAG Architecture  Architecture: Retriever: Retrieves relevant documents or passages from a large corpus based on the input query. Reader/Generator: Uses the retrieved documents to generate a coherent and relevant response. Combiner: Integrates the retrieved information with the generated response. 2. Pipeline Behind the RAG The RAG pipeline is a collection of the above components working together: Query Processing: The input query is encoded into a vector. Retrieval: The encoded query is used to retrieve relevant documents from a vector database. Document Encoding: The retrieved documents are encoded into vectors. Information Inte...
Read more

Large Language Models(LLMs)

Image
  Language Model A language model is a type of artificial intelligence model designed to understand, generate, and manipulate human language. Language models are trained on large datasets of text to learn patterns in language use, including grammar, vocabulary, and context. Key functions include: Text Generation : Producing coherent and contextually relevant text. Text Completion : Predicting the next word or phrase in a given context. Machine Translation : Translating text from one language to another. Sentiment Analysis : Determining the sentiment expressed in a piece of text. Text Summarization : Creating concise summaries of longer texts. Language models can be based on various architectures, such as Recurrent Neural Networks (RNNs) and Transformer models. Large Language Model (LLM) A large language model (LLM) is a type of language model that is characterized by its large size, typically measured by the number of parameters (weights) it has. LLMs are trained on vast amounts ...
Read more