Difference Between R square and Adjusted R square

In many of the supervised machine learning problem statement basically we have two kind of use cases.

1.Regression

2.classification

For Regression type use case if we want to check accuracy usually we follow the techniques like R square and adjusted R square.

In this article we will be discuss about difference between R square and adjusted R square.

1. R square

R square formula is given by,

R^2 = 1 - \frac{RSS}{TSS}
R^2=coefficient of determination
RSS=sum of square of residuals or error
TSS=total sum of square

Here residuals are sum of square of difference between actual point and predict point. and for given data if we have only target variable we will find the best fit line by taking average of all the values. so total sum of square of actual points and average value.

So from above formula we will get value between 0 and 1, the more value near to 1 is the best fit line. 

whether can we get R2 value less than 0?

YES, only when your best fit line is worse than average best fit line.

If RSS > TSS so ratio become >1 so R2 will be negative value, thus model will be bad (best fit line worse.)

R2 is basically used to check the goodness of the best fit line.

if we have y = mx + c                         - - - - -  simple linear regression

                  y = m1x1 + m2x2 + c       - - - - -  multiple linear regression

so as we go on adding independent features to our model, R2 value usually increases. because when we are adding new features in linear regression it tries to apply some coefficient value to those features.

for example:  m3x3  - - - m3 is new coefficient such that our RSS value goes on decreasing as a result R2 score will be increase due to formula.

thus, adding new independent features to the model R2 value will also increases but here this new independent features may not be corelated with target variable that basically indicates that R2 have not doing anything with independent feature i,e it is not finding any relationship with the target variable, just depends on co-efficient assigned by linear regression.

to overcome from this type of difficulty we follow with Adjusted R square method.


2. Adjusted R square

The formula is:
r squared adjusted
where:

  • N is the number of points in your data sample.
  • K is the number of independent regressors, i.e. the number of variables in your model, excluding the constant.
Adjusted R2 is used to find the relations of the attributes that are not correlated with target variable.
so here as we increases number of predictors(k) so (n-k-1) Will become small number. due to that small value ratio becomes bigger number so that adjusted R2 value decreases.
It happen only when adding more features which are not correlated with the target variable.

If suppose new features added are correlated with the target variable than our R2 value will increases so that adjusted R2 value increases.
thus, it should make sure that when the variables are not correlated than our adjusted R2 value will slightly decreases compare to original R2 value.

Conclusion.

1. Every time you add a independent variable to a model, the R-squared increases. even if the                independent variable is insignificant. it never declines. whereas Adjusted R-squared increases     only when independent variable is significant and affects dependent variable.

2. Adjusted r-squared value always be less than or equal to r-squared value.




Comments

Post a Comment

Popular posts from this blog

Transformers: Self-attention

Image
 In the realm of neural networks, three primary types are commonly discussed: Artificial Neural Networks (ANNs) : These are fully connected networks comprising input, hidden, and output layers. Each neuron in a layer is connected to every neuron in the subsequent layer, enabling complex pattern recognition through weighted connections. Convolutional Neural Networks (CNNs) : CNNs incorporate convolutional layers with kernels (filters) that slide across the input data to detect features. These networks also perform pooling operations to reduce dimensionality and flatten the data before passing it to fully connected layers. CNNs are particularly effective for image and spatial data processing. Recurrent Neural Networks (RNNs) : RNNs are designed to handle sequential data by maintaining a memory of previous inputs through their hidden states. Unlike feed-forward networks, RNNs can process input sequences of variable length, making them suitable for tasks involving time series data, nat...
Read more

Retrieval Augmented Generation(RAG)

Image
 We Know core ideas like NLP, LLM. Let's look at how we might build utilizing LLM and the deployment aspect using RAG. Retrieval-Augmented Generation (RAG) is a technique in natural language processing that combines retrieval-based and generation-based approaches to create more accurate and informative responses. Let's break down each point: 1. RAG Architecture  Architecture: Retriever: Retrieves relevant documents or passages from a large corpus based on the input query. Reader/Generator: Uses the retrieved documents to generate a coherent and relevant response. Combiner: Integrates the retrieved information with the generated response. 2. Pipeline Behind the RAG The RAG pipeline is a collection of the above components working together: Query Processing: The input query is encoded into a vector. Retrieval: The encoded query is used to retrieve relevant documents from a vector database. Document Encoding: The retrieved documents are encoded into vectors. Information Inte...
Read more

Large Language Models(LLMs)

Image
  Language Model A language model is a type of artificial intelligence model designed to understand, generate, and manipulate human language. Language models are trained on large datasets of text to learn patterns in language use, including grammar, vocabulary, and context. Key functions include: Text Generation : Producing coherent and contextually relevant text. Text Completion : Predicting the next word or phrase in a given context. Machine Translation : Translating text from one language to another. Sentiment Analysis : Determining the sentiment expressed in a piece of text. Text Summarization : Creating concise summaries of longer texts. Language models can be based on various architectures, such as Recurrent Neural Networks (RNNs) and Transformer models. Large Language Model (LLM) A large language model (LLM) is a type of language model that is characterized by its large size, typically measured by the number of parameters (weights) it has. LLMs are trained on vast amounts ...
Read more