Determining what language a string contains in a pandas DataFrame

Question

I am new to Pandas and Python.

My dataframe:

df

Text
Best tv in 2020
utilizar un servicio sms gratuito
utiliser un tv pour netflix

My desired output

Text                                    Language
Best tv in 2020                         en
utilizar un servicio sms gratuito       es
utiliser un tv pour netflix             fr

What I am using:

from textblob import TextBlob

b = TextBlob("utilizar un servicio sms gratuito")
print(b.detect_language())

>>es

I am not sure how I could integrate this method to fill my Pandas Dataframe.

I have tried:

df['Language'] = TextBlob(df['Text']).detect_language()

But I am getting an error:

TypeError: The `text` argument passed to `__init__(text)` must be a string, not <class 'pandas.core.series.Series'>

I understand what it means, that I need to pass a string rather than pandas DataFrame Series, so my question is how would I loop the entire Series to detect language per row in column text?

Thank you for your suggestions.

jezrael · Accepted Answer · 2020-01-06 10:10:54Z

3

Use Series.apply with lambda function:

df['Language'] = df['Text'].apply(lambda x: TextBlob(x).detect_language())

Or Series.map:

df['Language'] = df['Text'].map(lambda x: TextBlob(x).detect_language())

print (df)
                                Text Language
0                    Best tv in 2020       en
1  utilizar un servicio sms gratuito       es
2        utiliser un tv pour netflix       fr

edited Jan 6, 2020 at 10:10

answered Jan 6, 2020 at 10:05

jezrael

868k103 gold badges1.4k silver badges1.3k bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Jonas Palačionis Over a year ago

Hi @jezrael and thank you as always. Any comments on the performance difference between these two solutions?

jezrael Over a year ago

@JonasPalačionis - First is more common for pass functions, but not working for index values, second is used for df.index.map, also working for columns like in answer.

Jonas Palačionis Over a year ago

understood, thanks. Not sure if you have worked with textblob before, but now I am getting urllib.error.HTTPError: HTTP Error 429: Too Many Requests, any idea for workarounds?

jezrael Over a year ago

@JonasPalačionis - I working with it first time, but I guess there is some protection for avoid many requests...

Collectives™ on Stack Overflow

Determining what language a string contains in a pandas DataFrame

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related