Unicode display issues (UnicodeEncodeError) via "heroku run python"

Question

If I connect to a Heroku worker dyno with heroku run (e.g. heroku run python for an interactive Python session), any attempt to display Unicode characters via this results in a UnicodeEncodeError

Locally:

$ python
Python 2.7.1 (r271:86832, Jun 25 2011, 05:09:01) 
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> print u'\xa3'
£

Via heroku run:

$ heroku run python
Running python attached to terminal... up, run.1
Python 2.7.2 (default, Oct 31 2011, 16:22:04) 
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> print u'\xa3'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa3' in position 0: ordinal not in range(128)
>>>

Now, if I heroku run bash and use echo to try to display things, all seems fine (except my local choice of font!):

$ heroku run bash
Running bash attached to terminal... up, run.2
~ $ echo -e "\xa3"
?

I assume I'm doing something wrong / missing something, but somewhat lost as to what, or further lines of investigation.

Will Ediger · Accepted Answer · 2019-12-15 22:19:23Z

8

I believe this will fix your problem:

$ heroku config:set LANG=en_US.UTF-8

or, if after you try to run that, heroku complains that you need to supply the app parameter:

$ heroku config:set LANG=en_US.UTF-8 -a <app_name>

edited Dec 15, 2019 at 22:19

Will Ediger

90310 silver badges17 bronze badges

answered Mar 29, 2012 at 0:37

Kenneth Reitz

8,8764 gold badges34 silver badges34 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Kenneth Reitz Over a year ago

This is default on all new Heroku Python apps now :)

Kristian Glass Over a year ago

Oh, as a function of the buildpack?

ShankarG · Accepted Answer · 2012-03-28 13:42:00Z

3

I had similar issues. The problem appears to be that, when it is not outputting to a terminal, Python 2 chooses ascii as the default encoding for the print statement. There are ways around this, including the complex but accurate method described here. There are similar methods scattered around the net.

However, there is an easier, but deprecated, solution. This happens to be the one I use, which consists of inserting this at the beginning of your program:

reload(sys) 
sys.setdefaultencoding("utf-8")

This forces print to use the utf-8 default encoding rather than ascii, and should, hopefully, fix your problem.

answered Mar 28, 2012 at 13:42

ShankarG

1,16512 silver badges29 bronze badges

2 Comments

Kristian Glass Over a year ago

Alas the solution you describe doesn't seem to work for me, but the link seems highly relevant, thanks!

ShankarG Over a year ago

It may not work in the interpreter - I am away from my system and not able to check - but it ought to work if you try it in a script?

Community · Accepted Answer · 2017-05-23 12:33:00Z

3

None of the above worked but this did:

heroku config:add PYTHONIOENCODING=utf8

This is one of the answers to this question here:

UnicodeEncodeError only when running as a cron job

unfortunately it wasn't the accepted answer and that question is marked as a dup so I'm reposting here.

edited May 23, 2017 at 12:33

CommunityBot

11 silver badge

answered Jan 2, 2014 at 19:08

Andy Baker

21.6k12 gold badges61 silver badges71 bronze badges

Collectives™ on Stack Overflow

Unicode display issues (UnicodeEncodeError) via "heroku run python"

3 Answers 3

2 Comments

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related