Foreign character issue with CSV import to Heroku Postgres DB

Question

I have a rails app where my users can manually set up products via a web form. This works fine and accepts foreign characters well, words like 'Svölk' for example.

I now have a need to bulk import products and am using FasterCSV to do so. Generally this works without issue, but when the CSV contains foreign characters it stalls at that point.

Am I correct to believe the file needs to be UTF-8 in the first instance?

Also, I'm running Ruby 1.8.7 so is ICONV my only solution for converting the file? This could be an issue as the format of the original file won't be known.

Have others encountered this issue and if so, how did you overcome it?

Paulo Fidalgo · Accepted Answer · 2013-06-19 09:37:37Z

1

You have two alternatives:

Use ensure_encoding gem to find the actual encoding of the strings.
Use Ruby to determine the file encoding using:

File.open(source_file).read.encoding

I prefer the first approach as it tries to detected the encoding based on Strings, and tries to convert to your desired encoding (UTF-8) and then you can set the encoding on FasterCSV options.

answered Jun 19, 2013 at 9:37

Paulo Fidalgo

22.4k7 gold badges105 silver badges121 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Raoot Over a year ago

Thanks Paulo. I can't seem to get the ensure_encoding gem to work. It keeps giving me undefined method errors when I call it.

Raoot Over a year ago

I also get and error on approach two: NoMethodError: undefined method `encoding'

Collectives™ on Stack Overflow

Foreign character issue with CSV import to Heroku Postgres DB

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related