It should start from the last '\x' position + 1 to find next '\x'
character.
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
This allows .uni input files to be encoded with UTF-8. Today, we only
support UTF-16 encoding.
The strings are still converted to UCS-2 data for use in EDK II
modules. (This is the only unicode character format supported by UEFI
and EDK II.)
Although UTF-8 would allow any UCS-4 character to be present in the
source file, we restrict the entire file to the UCS-2 range.
(Including comments.) This allows the files to be converted to UTF-16
if needed.
v2:
* Drop .utf8 extension. Use .uni file for UTF-8 data (mdkinney)
* Merge in 'BaseTools/UniClassObject: Verify string data is 16-bit'
commit
v3:
* Restrict the entire file's characters (including comments) to the
UCS-2 range in addition to string data. (mdkinney)
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Yingke Liu <yingke.d.liu@intel.com>
git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@17696 6f19259b-4bc3-4df7-8a09-765794883524
Supplementary Plane characters can exist in UTF-16 files,
but they are not valid UCS-2 characters.
For example, refer to this python interpreter code:
>>> import codecs
>>> codecs.encode(u'\U00010300', 'utf-16')
'\xff\xfe\x00\xd8\x00\xdf'
Therefore the UCS-4 0x00010300 character is encoded as two
16-bit numbers (0xd800 0xdf00) in a little endian UTF-16
file.
For more information, see:
http://en.wikipedia.org/wiki/UTF-16#U.2B10000_to_U.2B10FFFF
This means that our current BaseTools code could be allowing
unsupported UTF-16 characters be used. To fix this, we decode the file
using python's utf-16 decode support. Then we verify that each
character's code point is 0xffff or less.
v3:
* Based on Mike Kinney's feedback, we now read the whole file and
verify up-front that it contains valid UCS-2 characters. Thanks
also to Laszlo Ersek for pointing out the Supplementary Plane
characters.
v4:
* Reject code points in 0xd800-0xdfff range since they are reserved
for UTF-16 surrogate pairs. (lersek)
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Yingke Liu <yingke.d.liu@intel.com>
git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@17694 6f19259b-4bc3-4df7-8a09-765794883524
The 'include' regular expression cannot match spaces before or after this statement.
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Yingke Liu <yingke.d.liu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@17680 6f19259b-4bc3-4df7-8a09-765794883524
1. Fix !include issues
2. Fix Trim to skip the postfix 'U' for hexadecimal and decimal numbers
3. Fix building error C2733 when building C++ code.
4. Add GCC46 tool chain definition
5. Add new RVCT and RVCTLINUX tool chains
Signed-off-by: lgao4
git-svn-id: https://edk2.svn.sourceforge.net/svnroot/edk2/trunk/edk2@12782 6f19259b-4bc3-4df7-8a09-765794883524
Yonghong Zhu
Jordan Justen
Yingke Liu
Cecil Sheng
Hess Chen
Liming Gao
lgao4