I'm facing an issue where the same code works fine on Windows but fails on Linux with the error:
pymysql.err.ProgrammingError: nan can not be used with MySQL
In my project, I compute results, store them in a pandas DataFrame, convert it to a list, and insert the data into MySQL using pymysql. The code looks like this:
df_score = DataFrame()
temp = df_score.values.tolist()
insert(REPORT, report_cursor, insert_score.format(suffix), temp)
On Windows, this code runs perfectly. However, when running it on Linux, I encounter the error mentioned above. I have checked the DataFrame, and it does not contain any NaN values, only None. I even tried replacing NaN values with None using:
df_score = df_score.where(pd.notnull(df_score), None)
But the error persists.
Environment details:
Python version: 3.8.10
OS version: Ubuntu 18.04
pandas version: 2.0.3
pymysql version: 1.1.1
Full error traceback:
Traceback (most recent call last):
File "/root/analysis/./dataAnalysisApp/service.py", line 3872, in score
insert(REPORT, report_cursor, insert_score.format(suffix), temp)
File "/root/analysis/./utils/mysql_db_pools.py", line 207, in insert
db_pools.execute_sql(db_name, cursor, sql, params, fetch, executemany)
File "/root/analysis/./utils/mysql_db_pools.py", line 91, in execute_sql
raise e
File "/root/analysis/./utils/mysql_db_pools.py", line 82, in execute_sql
cursor.executemany(sql, params or ())
File "/root/anaconda3/envs/analysis-py38/lib/python3.8/site-packages/dbutils/steady_db.py", line 605, in tough_method
result = method(*args, **kwargs) # try to execute
File "/root/anaconda3/envs/analysis-py38/lib/python3.8/site-packages/pymysql/cursors.py", line 182, in executemany
return self._do_execute_many(
File "/root/anaconda3/envs/analysis-py38/lib/python3.8/site-packages/pymysql/cursors.py", line 211, in _do_execute_many
v = values % escape(arg, conn)
File "/root/anaconda3/envs/analysis-py38/lib/python3.8/site-packages/pymysql/cursors.py", line 102, in _escape_args
return tuple(conn.literal(arg) for arg in args)
File "/root/anaconda3/envs/analysis-py38/lib/python3.8/site-packages/pymysql/cursors.py", line 102, in <genexpr>
return tuple(conn.literal(arg) for arg in args)
File "/root/anaconda3/envs/analysis-py38/lib/python3.8/site-packages/pymysql/connections.py", line 530, in literal
return self.escape(obj, self.encoders)
File "/root/anaconda3/envs/analysis-py38/lib/python3.8/site-packages/pymysql/connections.py", line 523, in escape
return converters.escape_item(obj, self.charset, mapping=mapping)
File "/root/anaconda3/envs/xjy-data-analysis-py38/lib/python3.8/site-packages/pymysql/converters.py", line 25, in escape_item
val = encoder(val, mapping)
File "/root/anaconda3/envs/xjy-data-analysis-py38/lib/python3.8/site-packages/pymysql/converters.py", line 56, in escape_float
raise ProgrammingError("%s can not be used with MySQL" % s)
pymysql.err.ProgrammingError: nan can not be used with MySQL
I confirmed there are no
NaNvalues in the DataFrame, onlyNone.I attempted to convert
NaNtoNonewithdf_score = df_score.where(pd.notnull(df_score), None)but still got the error.
Has anyone encountered this issue before, or can anyone suggest what might be causing this on Linux?