2

I have a csv file which Im reading via pandas.Then Im trying to call a mysql query on that file which is throwing up error.

import pandas
import pandasql
fl=pandas.read_csv('file.csv')

Columns of the file are

fl.columns
Index([ u'time', u'contact', u'address'], dtype='object')

The query is

q=u"""
SELECT contact FROM fl LIMIT 50
"""

WHich im running as 

fl_solution=pandasql.sqldf(q,locals())

The error Im getting is :-

ProgrammingError: You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings.
Improve this question

2 Answers 2

Reset to default
3

Try doing this:

fl=pandas.read_csv('file.csv',encoding='utf-8')

It specifies to encode file into utf-8 encoding. Also try to encode the strings into utf-8 wherever you are using strings fetched from either file or external source.

For e.g.:

stringname.encode('utf-8')

Improve this answer
Sign up to request clarification or add additional context in comments.

5 Comments

fl=pandas.read_csv('file.csv',encoding='utf-8') ____ ERROR ::::::: UnicodeDecodeError: 'utf8' codec can't decode byte 0x81 in position 596: invalid start byte
Then I guess your file is in another encoding. Try to find out file encoding & put it while opening file.
encoding = "ISO-8859-1" worked for me. But still column names are appended with silly u. Its been a very frustrating day, wasting it entirely on this u
u appended to the string shows that string encoding is unicode (you probably might know that). You have 'latin-1' encoding for your file. This documentation might help you in further reading.
BTW are you using any editor or interpreter? See encoding of that. Might be because of that it's reading columns in unicode. Try to change the settings.
2

Tried using a different encoding as per suggestion by harshad.

encoding = "ISO-8859-1" , and thereafter the mysql queries are working fine too.

Improve this answer

1 Comment

you should likely have accepted @harshad 's answer.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.