0

I'm facing an issue where the same code works fine on Windows but fails on Linux with the error:

pymysql.err.ProgrammingError: nan can not be used with MySQL

In my project, I compute results, store them in a pandas DataFrame, convert it to a list, and insert the data into MySQL using pymysql. The code looks like this:

df_score = DataFrame()
temp = df_score.values.tolist()
insert(REPORT, report_cursor, insert_score.format(suffix), temp)

On Windows, this code runs perfectly. However, when running it on Linux, I encounter the error mentioned above. I have checked the DataFrame, and it does not contain any NaN values, only None. I even tried replacing NaN values with None using:

df_score = df_score.where(pd.notnull(df_score), None)

But the error persists.

Environment details:

  • Python version: 3.8.10

  • OS version: Ubuntu 18.04

  • pandas version: 2.0.3

  • pymysql version: 1.1.1

Full error traceback:

Traceback (most recent call last):
  File "/root/analysis/./dataAnalysisApp/service.py", line 3872, in score
    insert(REPORT, report_cursor, insert_score.format(suffix), temp)
  File "/root/analysis/./utils/mysql_db_pools.py", line 207, in insert
    db_pools.execute_sql(db_name, cursor, sql, params, fetch, executemany)
  File "/root/analysis/./utils/mysql_db_pools.py", line 91, in execute_sql
    raise e
  File "/root/analysis/./utils/mysql_db_pools.py", line 82, in execute_sql
    cursor.executemany(sql, params or ())
  File "/root/anaconda3/envs/analysis-py38/lib/python3.8/site-packages/dbutils/steady_db.py", line 605, in tough_method
    result = method(*args, **kwargs)  # try to execute
  File "/root/anaconda3/envs/analysis-py38/lib/python3.8/site-packages/pymysql/cursors.py", line 182, in executemany
    return self._do_execute_many(
  File "/root/anaconda3/envs/analysis-py38/lib/python3.8/site-packages/pymysql/cursors.py", line 211, in _do_execute_many
    v = values % escape(arg, conn)
  File "/root/anaconda3/envs/analysis-py38/lib/python3.8/site-packages/pymysql/cursors.py", line 102, in _escape_args
    return tuple(conn.literal(arg) for arg in args)
  File "/root/anaconda3/envs/analysis-py38/lib/python3.8/site-packages/pymysql/cursors.py", line 102, in <genexpr>
    return tuple(conn.literal(arg) for arg in args)
  File "/root/anaconda3/envs/analysis-py38/lib/python3.8/site-packages/pymysql/connections.py", line 530, in literal
    return self.escape(obj, self.encoders)
  File "/root/anaconda3/envs/analysis-py38/lib/python3.8/site-packages/pymysql/connections.py", line 523, in escape
    return converters.escape_item(obj, self.charset, mapping=mapping)
  File "/root/anaconda3/envs/xjy-data-analysis-py38/lib/python3.8/site-packages/pymysql/converters.py", line 25, in escape_item
    val = encoder(val, mapping)
  File "/root/anaconda3/envs/xjy-data-analysis-py38/lib/python3.8/site-packages/pymysql/converters.py", line 56, in escape_float
    raise ProgrammingError("%s can not be used with MySQL" % s)
pymysql.err.ProgrammingError: nan can not be used with MySQL
  1. I confirmed there are no NaN values in the DataFrame, only None.

  2. I attempted to convert NaN to None with df_score = df_score.where(pd.notnull(df_score), None) but still got the error.

Has anyone encountered this issue before, or can anyone suggest what might be causing this on Linux?

1 Answer 1

0

I figured out how to solve the issue. The problem can be resolved by downgrading the pandas version to 1.2.5 or lower. Any version higher than 1.2.5 seems to cause the failure. However, I am not sure why this is happening.

If anyone has insights into why versions higher than 1.2.5 of pandas might be causing this issue, I would appreciate any explanations or suggestions.

Here is how you can downgrade pandas:

pip uninstall pandas
pip install pandas==1.2.5

After downgrading, the code should work as expected on both Windows and Linux.

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.