Naive Bayes

Page 1

Probabilities :

say total of 20 tweets

A => positive tweet : 12 tweets

B => contains word happy;

P(A) = 12/20 = .6

P(B) = 4/20 = .2

probability that : tweet is positive and contain word happy: say 3 tweets :

P(AnB) = 3/20 = .15

Page 2

Conditional probability:

Probability of event A given B happened: P(A|B)

= P(AnB)/P(B)

Probability of event B given A happened: P(B|A)

= P(AnB)/P(A)

Bayes Rule:

P(A|B) = P(B|A) *P(A) / P(B);

Suppose that in your dataset, 25% of the positive tweets contain the word ‘happy’. You also know that a total of 13% of the tweets in your dataset contain the word 'happy', and that 40% of the total number of tweets are positive. You observe the tweet: ''happy to learn NLP'. What is the probability that this tweet is positive?

P(happy) = .13

P(happy|positive) = .25

P(positive) = .4

P(positive |happy) = P(happy|positive) * P(positive)/P(happy)

= .25*.4 /.13 = .1/.13 = .77

Page 3

Naive Bayes :

Supervised Machine Learning: ?

it is called Naive , hehehe, because features you are using for classification are independent : which may not be the case

Naive -Bayes Steps:

Do the work frequency map for postive and negative tweets

sum the positive words and negative words

divide the frequency by the sum : you get the probabaility ; and if you sum all postive word probability you get 1

similarly for negative tweets

now look at word and see which of them have equal prbability in postive and negative : ex., learnign they do not add to the sentiment

look at word happy : it has more probability of being postive rather than sad

look at word because : it has negative probability zero :

next we use something called : naive-bayes inference condition rule for binary classifciation - see picture below

whats that man? oh man ? throwing jargons man ....hehehehehe

it is this : it is this : it is this:

given a new tweet : for example : I am happy today I am learning

for each word in tweet : compute p(word|positivie) / p(word|negative) ;

multiply all of them:

if outcome > 1, its more likely positive tweet

in our example, it is 1.

Page 4

problem with Naive-Bayes method : one of the words because : has zero probability;

so what can you do is :

use laplacian smoothing :

oh man another jargon here :

ok fine and take care and do well and take care and do well ;

what is thing thingy called laplacian smoothing :

see below : pics;

because of the laplacian smoothing , the word because : probability is now non zero!

Page 5

earlier we saw that : Naive Bayes inference on how to asses positive vs negative tweets:

a: get the conditional probability of word in current tweet being positive

b : get the conditional probability of word in current tweet being negative

divide a/b;

multiply all probabilities :

and then : you have to multiply : P(pos) tweets/P (negative tweets)

Page 6

Now the issues : when Vocabulary is huge : the probability numbers can be very small ;

to deal with this we use log likelihood :

and we call this lambda:

compute lambda for each word :

and sum it up ;

and then add log P(pos)/P(neg)

Page 7

in likelihood : Neutral has scale 1 : anything positive : is positive sentiment

negative sentiment !!!

in log likelihood : neutral is Log 1 = 0;

look at picture fro the algorithm for implementing : Naive Bayes algorithm: