Sunday, March 31, 2024

CountVectorizer to Extract Options from Texts in Python, in Element | by Rashida Nasrin Sucky | Oct, 2023

Must read

Picture by Towfiqu barbhuiya on Unsplash

All the pieces you must know to make use of CountVectorizer effectively in Sklearn

Essentially the most primary information processing that any Pure Language Processing (NLP) challenge requires is to transform the textual content information to the numeric information. So long as the info is in textual content kind we can’t do any sort of computation motion on it.

There are a number of strategies obtainable for this text-to-numeric information conversion. This tutorial will clarify one of the primary vectorizers, the CountVectorizer methodology within the scikit-learn library.

This methodology could be very easy. It takes the frequency of prevalence of every phrase because the numeric worth. An instance will make it clear.

Within the following code block:

  • We’ll import the CountVectorizer methodology.
  • Name the tactic.
  • Match the textual content information to the CountVectorizer methodology and, convert that to an array.
import pandas as pd 
from sklearn.feature_extraction.textual content import CountVectorizer

#That is the textual content to be vectorized
textual content = ["Hello Everyone! This is Lilly. My aunt's name is also Lilly. I love my aunt.
I am trying to learn how to use count vectorizer."]

cv= CountVectorizer()
count_matrix = cv.fit_transform(textual content)
cnt_arr = count_matrix.toarray()


array([[1, 1, 2, 1, 1, 1, 1, 2, 1, 2, 1, 2, 1, 1, 2, 1, 1, 1]],

Right here I’ve the numeric values representing the textual content information above.

How do we all know which values signify which phrases within the textual content?

To make that clear, it is going to be useful to transform the array right into a DataFrame the place column names would be the phrases themselves.

cnt_df = pd.DataFrame(information = cnt_arr, columns = cv.get_feature_names())

Now, it reveals clearly. The worth of the phrase ‘additionally’ is 1 which suggests ‘additionally’ appeared solely as soon as within the take a look at. The phrase ‘aunt’ got here twice within the textual content. So, the worth of the phrase ‘aunt’ is 2.

Within the final instance, all of the sentences had been in a single string. So, we bought just one row of knowledge for 4 sentences. Let’s rearrange the textual content and…

Supply hyperlink

More articles


Please enter your comment!
Please enter your name here

Latest article