diff options
| author | Erik Verbruggen <erik.verbruggen@digia.com> | 2013-09-20 12:41:25 +0200 |
|---|---|---|
| committer | Erik Verbruggen <erik.verbruggen@digia.com> | 2013-09-23 14:55:28 +0200 |
| commit | 8350c4bdc2a90166db7ced2fdbba2edb6f1740fc (patch) | |
| tree | 12ab69be0befa679b41b49ba4db140a41e7a2be2 /src/libs/cplusplus/FastPreprocessor.cpp | |
| parent | 4eb633fd773263ceb9cb012611139b83bb730914 (diff) | |
C++: fix multi-byte character handling in input.
Temporary fix: if a single byte is found with the highest bit set, then
convert from utf8 to latin1. This can be removed when the lexer can
handle multi-byte characters.
Task-number: QTCREATORBUG-10141
Change-Id: I36a17aa18bd1b2378f12d0cecf4fd4957b38d8f2
Reviewed-by: Nikolai Kosjar <nikolai.kosjar@digia.com>
Reviewed-by: Orgad Shaneh <orgads@gmail.com>
Diffstat (limited to 'src/libs/cplusplus/FastPreprocessor.cpp')
| -rw-r--r-- | src/libs/cplusplus/FastPreprocessor.cpp | 16 |
1 files changed, 15 insertions, 1 deletions
diff --git a/src/libs/cplusplus/FastPreprocessor.cpp b/src/libs/cplusplus/FastPreprocessor.cpp index 95d7f1cc9fb..45f1d24d37a 100644 --- a/src/libs/cplusplus/FastPreprocessor.cpp +++ b/src/libs/cplusplus/FastPreprocessor.cpp @@ -41,6 +41,18 @@ FastPreprocessor::FastPreprocessor(const Snapshot &snapshot) , _preproc(this, &_env) { } +// This is a temporary fix to handle non-ascii characters. This can be removed when the lexer can +// handle multi-byte characters. +static QByteArray convertToLatin1(const QByteArray &contents) +{ + const char *p = contents.constData(); + while (char ch = *p++) + if (ch & 0x80) + return QString::fromUtf8(contents).toLatin1(); + + return contents; +} + QByteArray FastPreprocessor::run(Document::Ptr newDoc, const QByteArray &source) { std::swap(newDoc, _currentDoc); @@ -56,7 +68,9 @@ QByteArray FastPreprocessor::run(Document::Ptr newDoc, const QByteArray &source) mergeEnvironment(i.resolvedFileName()); } - const QByteArray preprocessed = _preproc.run(fileName, source); + QByteArray src = convertToLatin1(source); + + const QByteArray preprocessed = _preproc.run(fileName, src); // qDebug("FastPreprocessor::run for %s produced [[%s]]", fileName.toUtf8().constData(), preprocessed.constData()); std::swap(newDoc, _currentDoc); return preprocessed; |