diff --git a/extras/ChangeLog b/extras/ChangeLog index 5c8bbf0959..a3f95284d1 100644 --- a/extras/ChangeLog +++ b/extras/ChangeLog @@ -1,3 +1,10 @@ +012-05-05 Sancho Lerena + + * anytermd: Added anyterm to extras. Included modifications on original + anytermd project source code. Added a new spec file for centos/fedora/rhel. + Tested on FC16/i386 and centos6/x86_84. + + 2012-02-21 Ramon Novoa * pandora_update_version.sh: Update agent version in versioninfo.rc. diff --git a/extras/anytermd/ChangeLog b/extras/anytermd/ChangeLog new file mode 100644 index 0000000000..f6814d05b6 --- /dev/null +++ b/extras/anytermd/ChangeLog @@ -0,0 +1,352 @@ +2012-05-04 Sancho Lerena + + * Added to Pandora FMS repo. Some minor modifications on c files + in order to compile it without problems. + + Added README and INSTALL files. + +---- old changelog contents below ----- + +1.1.29 + + Fix for compilers that don't think they have atomic operations. + Fix brokenness on Linux 2.4. + +1.1.28 + + Re-organise directory structure with separate build directory. + Add manpage. + Strip query string before looking for static content. + Remove dependency on Boost.Thread and Boost.ProgramOptions. + +1.1.27 + + Build experimentally on OS X. + Work around FF3 scrollbar issue. + Remove unused dangling symlink. + Fix mimetype.sh when there are multiple matching entries in + /etc/mime.types, e.g. Suse 11. + +1.1.26 + + Remove apache module and anygetty. + Add control keys menu. + +1.1.25 + + Maybe make Safari keyboard input work again. + Add mozilla copy-to-clipboard from Simon Wissinger. + Set the erase character to backspace. + Add on_close_goto_url. + Tidy up various CSS things. + +1.1.24 + + OpenBSD fixes (from Patrick Augé). + Limit scrollback to 1000 lines. + +1.1.23 + + Compile on g++ 3.4.4. + Work with Firefox 3. + Try harder to make subprocesses terminate properly. + Fix compilation errors in daemon/Screen.hh. + +1.1.22 + + Handle ALT keys. + Handle ctrl-space. + Remove standalone daemon's recode dependency. + Add --name option to daemon so multiple pid files are possible. + Compile on g++ 4.0.2. + Cope better when subprocess is unkillable (e.g. sudo). + Avoid zombie processes that escape waitpid(). + +1.1.21 + + Add scrollback. + Add --local-only option to daemon. + +1.1.20 + + Remove left-over Sarrissa reference from Daemon makefile. + +1.1.19 + + Use explicit cross-browser XMLHttpRequest code in place + of Sarrissa.js. + Support shifted function keys F1-F8. + Prevent IE default behaviour for function keys. + Fixes for FreeBSD. + +1.1.18 + + Show a stacktrace if the stand-alone daemon segfaults. + Handle alt-gr better. + Map ctrl-. to ctrl-^, which browsers don't send. + Fix segfault in stand-alone daemon's scroll-up function. + +1.1.17 + + Use more CSS for colours, so it's easier to define your own + colour scheme. + Change colours for bold, since real bold messes up the layout. + Report subprocess errors in a JavaScript dialog. + Fix mutex destruction order bug in daemon. + Fix bounds error in differ. + Replace server-to-client XML with plain text. + Add --max-sessions and --max-http-connections options to daemon. + Add --diff and --nodiff options to daemon. + Possibly improved performance in Javascript and editscript + generation. + +1.1.16 + + Support arbitary character sets. + Replace ROTE with own Terminal class. + Fix the shift key in Opera. + Deprecate the WAP functionality. + Make "ssh %u@localhost" the default anyterm_command in .htaccess. + Standalone daemon has a command-line option to select + authentication method, including "null" to record but + not check the username from Apache. + Standalone daemon has better handling of subprocess termination. + +1.1.15 + + Make in daemon directory builds libpbe. + libpbe build from apachemod directory disables Recode. + Fix session closing and timeouts in backend to avoid freezes. + +1.1.14 + + Add copy and paste buttons. + +1.1.13 + + Don't build things with unwanted dependencies in libpbe. + Simplify building of common code with symlinks. + +1.1.12 + + Support serial ports in the standalone daemon. + +1.1.11 + + Support POST requests in both Apache module and standalone + daemon. + Use a new different shared memory segment name each time the + module starts. + +1.1.10 + + Add extra security to example .htaccess file. + Share more code between Apache module and standalone daemon. + Do data-plumbing to subprocess ourselves, not inside ROTE. + Use threads in backend to process process output, rather than + SIGIO. + Close sessions correctly. + Get random session ids from /dev/random. + Build two archives in common, one for shmem, one for std + containers. + +1.1.9 + + Use the candidate Boost Shared Memory library. + +1.1.8 + + Merge patch to support Solaris. + Support Apache2.2 (once they can do C++, presumably in 2.2.1). + Support F11 and F12 keys. + Fix FreeBSD build bug. + +1.1.7 + + Make compilation on FreeBSD more likely to work. + In the standalone daemon: + Use a condition variable rather than polling. + Get command to run, user to run as and port to listen + on from command-line options. + Redirect a request for / to /anyterm.html. + Use svn:external for libpbe. + Use Subversion. + Build in common/ with -fPIC. + +1.1.6 + + Make ctrl-[\] work, and try to make ctrl-^@_ work. Don't send + junk for other undefined ctrl keys. + Make cross-compilation easier. + Use common code in apachemod. + +1.1.5 + + Add experimental stand-alone Anyterm daemon. + Use apxs to find path to libtool. + +1.1.4 + + Use rote_vt_get_pty_fd from ROTE 0.2.8, removing need to + access roteprivate.h. + +1.1.3 + + Fix missing header. + Fix stylesheet comment syntax. + +1.1.2 + + Disable the status page by default. + Show only 3 digits of the session ID on the status page. + Use a 64-bit session ID. + Update to version 0.9.6 of Sarissa. + Make HTML more flexible. + Add SE-Linux configuration files to contrib directory. + Add anyterm.spec file to contrib directory. + Work around some Konqueror peculiarities. + Expand %u in anyterm_command to username from HTTP AUTH. + More 64-bit fixes. + +1.1.1 + + Add status page. + Manage shared memory properly, thereby removing the limit on + the number of sessions and terminal dimensions. + Improve error reporting. + Support 64-bit architectures. + Find Apache include directory automatically. + +1.1.0 + + Add WAP functionality. + Make terminal size adjustable per-session, up to a hard limit. + Use more C++. + Improve error reporting using exceptions. + +1.0 + + Fix build error with asm statements on platforms that use them + for APR atomic operations (again). + Support Escape key. + +0.14 + + Hack javascript to work with Opera. + Make HTTP requests POSTs rather than GETs. + Send anti-cache HTTP headers. + Update Sarissa to new version. + Fix module name so that works. + +0.13 + + Iterate after rote_vt_update() returns without processing + all pty output. + Tweak diff algorithm settings so diffs are sent for bastet. + Reduce memory used by diff algorithm when "generous" max_time + paramter is supplied. + Grey-out terminal when closed. + Put hostname and Anyterm version number in terminal title. + Improve terminal HTML appearance. + Fix potential issue in SIGCHLD handling. + +0.12 + + Fix non-random random session id. + Define character set using a macro. + +0.11 + + Close files (and sockets) inherited from Apache. + Put backend process in same process group as other Apache + processes, so hopefully it will die when Apache dies. + Possibly avoid some race conditions. + Use more C++. + Compile incrementally. + Improve error reporting. + Remove polling from backend (needs to peek inside ROTE private + data). + +0.10 + + Send child processes SIGHUP when terminal is closed or times + out. + anygetty clears utmp entry on SIGHUP. + Start work on removing polling from backend. + Fix build error with asm statements on platforms that use them + for APR atomic operations. + Convert UTF8 keypresses into Latin1 before sending them to the + terminal. + Split up module source code. + Move signal sanitising from anygetty into module, so other + applications should get a sane signal state. + +0.9 + + Remove anylogin.sh + Improve locking. + Allow client's hostname to be passed to application, so + anygetty can include it in utmp. + Allow different anyterm_command directives for each directory. + Support F1 to F10. + Avoid problems with ctrl-C and ctrl-Z by unblocking them in + anygetty. + Ignore SIGCHLD in module, but restore default behaviour in + anygetty. + Add anygetty. + +0.8 + + Attempt colours and bold. + Add some basic inter-process locking. + Re-enable SIGCHLD in sub-processes so that "man" works. + Warn when leaving page. + Fix Mozilla double-backspace bug. + +0.7 + + Officially "beta". + Work around IE blank line bug. + Handle keypresses better, including TAB and Backspace, and + prevent more browser default actions. + +0.6 + + Display cursor properly (again). + Unused sessions time out after 30 seconds. + Support multiple sessions. + +0.5 + + Send differences rather than complete screen each time. + +0.4 + + Add anyterm_command configuration command. + Add anylogin.sh login script. + Use a .htaccess file to enable module for a single URL. + Make cursor keys work in IE, and make . work in Mozilla + (. has the same code as DEL) + Add CLOSE button. + +0.3 + + Use a separate process to contain the ROTE state. + Communicate between Apache request-handling processes and ROTE + process using shared memory, + Separate keypresses and screen updates into two concurrent + channels. + Avoid polling by stalling in server until screen has changed. + Handle cursor keys, PgUp/Dn etc. + +0.2 + + Use ROTE for terminal emulation. + Try to encode control keypresses in the Javascript. + Display the cursor. + Better functionality in IE. + +0.1 + + Initial release. Alpha quality, barely works! + diff --git a/extras/anytermd/INSTALL b/extras/anytermd/INSTALL new file mode 100644 index 0000000000..7317d7f939 --- /dev/null +++ b/extras/anytermd/INSTALL @@ -0,0 +1,36 @@ +Introduction +------------ + +Pandora FMS uses a tool called "anytermd" to create a "proxy" between user browser and remote destination. This tool launches as a daemon, listeting in a port, and executing a command, forwarding all output to the user browser. + +You need to install the daemon in the server, and execute it. + +Installing anytermd +------------------- + +Sources are placed in extras/anytermd. + +Make sure you have installed gnu c++ compiler (gcc-c++), make, boost-devel and zlib-devel. + +Run: + + make + +Later, install manually the binary to /usr/bin + + cp anytermd /usr/bin + +Executing anytermd +------------------ + +To run the server daemons, you need to do manually, Pandora FMS server / console doesn't start up automatically. Pandora FMS SSH/Telnet extension will search for a different instance of anyterd running in 8023 for Telnet connections and 8022 for SSH connections. + +Execute it like: + + anytermd --port 8023 --user apache -c 'telnet %p' + anytermd --port 8022 --user apache -c 'ssh %p' + +Replace "apache" with "httpd" or any other httpd based user your system may have. This will use ports 8023 and 8022, be sure that ports are "open and clear" from the user browser to the console webserver system. No firewalls there. + +For more updated install docs, please refer official Pandora FMS documentation. + diff --git a/extras/anytermd/LICENSE b/extras/anytermd/LICENSE new file mode 100644 index 0000000000..d60c31a97a --- /dev/null +++ b/extras/anytermd/LICENSE @@ -0,0 +1,340 @@ + GNU GENERAL PUBLIC LICENSE + Version 2, June 1991 + + Copyright (C) 1989, 1991 Free Software Foundation, Inc. + 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The licenses for most software are designed to take away your +freedom to share and change it. By contrast, the GNU General Public +License is intended to guarantee your freedom to share and change free +software--to make sure the software is free for all its users. This +General Public License applies to most of the Free Software +Foundation's software and to any other program whose authors commit to +using it. (Some other Free Software Foundation software is covered by +the GNU Library General Public License instead.) You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +this service if you wish), that you receive source code or can get it +if you want it, that you can change the software or use pieces of it +in new free programs; and that you know you can do these things. + + To protect your rights, we need to make restrictions that forbid +anyone to deny you these rights or to ask you to surrender the rights. +These restrictions translate to certain responsibilities for you if you +distribute copies of the software, or if you modify it. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must give the recipients all the rights that +you have. You must make sure that they, too, receive or can get the +source code. And you must show them these terms so they know their +rights. + + We protect your rights with two steps: (1) copyright the software, and +(2) offer you this license which gives you legal permission to copy, +distribute and/or modify the software. + + Also, for each author's protection and ours, we want to make certain +that everyone understands that there is no warranty for this free +software. If the software is modified by someone else and passed on, we +want its recipients to know that what they have is not the original, so +that any problems introduced by others will not reflect on the original +authors' reputations. + + Finally, any free program is threatened constantly by software +patents. We wish to avoid the danger that redistributors of a free +program will individually obtain patent licenses, in effect making the +program proprietary. To prevent this, we have made it clear that any +patent must be licensed for everyone's free use or not licensed at all. + + The precise terms and conditions for copying, distribution and +modification follow. + + GNU GENERAL PUBLIC LICENSE + TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + + 0. This License applies to any program or other work which contains +a notice placed by the copyright holder saying it may be distributed +under the terms of this General Public License. The "Program", below, +refers to any such program or work, and a "work based on the Program" +means either the Program or any derivative work under copyright law: +that is to say, a work containing the Program or a portion of it, +either verbatim or with modifications and/or translated into another +language. (Hereinafter, translation is included without limitation in +the term "modification".) Each licensee is addressed as "you". + +Activities other than copying, distribution and modification are not +covered by this License; they are outside its scope. The act of +running the Program is not restricted, and the output from the Program +is covered only if its contents constitute a work based on the +Program (independent of having been made by running the Program). +Whether that is true depends on what the Program does. + + 1. You may copy and distribute verbatim copies of the Program's +source code as you receive it, in any medium, provided that you +conspicuously and appropriately publish on each copy an appropriate +copyright notice and disclaimer of warranty; keep intact all the +notices that refer to this License and to the absence of any warranty; +and give any other recipients of the Program a copy of this License +along with the Program. + +You may charge a fee for the physical act of transferring a copy, and +you may at your option offer warranty protection in exchange for a fee. + + 2. You may modify your copy or copies of the Program or any portion +of it, thus forming a work based on the Program, and copy and +distribute such modifications or work under the terms of Section 1 +above, provided that you also meet all of these conditions: + + a) You must cause the modified files to carry prominent notices + stating that you changed the files and the date of any change. + + b) You must cause any work that you distribute or publish, that in + whole or in part contains or is derived from the Program or any + part thereof, to be licensed as a whole at no charge to all third + parties under the terms of this License. + + c) If the modified program normally reads commands interactively + when run, you must cause it, when started running for such + interactive use in the most ordinary way, to print or display an + announcement including an appropriate copyright notice and a + notice that there is no warranty (or else, saying that you provide + a warranty) and that users may redistribute the program under + these conditions, and telling the user how to view a copy of this + License. (Exception: if the Program itself is interactive but + does not normally print such an announcement, your work based on + the Program is not required to print an announcement.) + +These requirements apply to the modified work as a whole. If +identifiable sections of that work are not derived from the Program, +and can be reasonably considered independent and separate works in +themselves, then this License, and its terms, do not apply to those +sections when you distribute them as separate works. But when you +distribute the same sections as part of a whole which is a work based +on the Program, the distribution of the whole must be on the terms of +this License, whose permissions for other licensees extend to the +entire whole, and thus to each and every part regardless of who wrote it. + +Thus, it is not the intent of this section to claim rights or contest +your rights to work written entirely by you; rather, the intent is to +exercise the right to control the distribution of derivative or +collective works based on the Program. + +In addition, mere aggregation of another work not based on the Program +with the Program (or with a work based on the Program) on a volume of +a storage or distribution medium does not bring the other work under +the scope of this License. + + 3. You may copy and distribute the Program (or a work based on it, +under Section 2) in object code or executable form under the terms of +Sections 1 and 2 above provided that you also do one of the following: + + a) Accompany it with the complete corresponding machine-readable + source code, which must be distributed under the terms of Sections + 1 and 2 above on a medium customarily used for software interchange; or, + + b) Accompany it with a written offer, valid for at least three + years, to give any third party, for a charge no more than your + cost of physically performing source distribution, a complete + machine-readable copy of the corresponding source code, to be + distributed under the terms of Sections 1 and 2 above on a medium + customarily used for software interchange; or, + + c) Accompany it with the information you received as to the offer + to distribute corresponding source code. (This alternative is + allowed only for noncommercial distribution and only if you + received the program in object code or executable form with such + an offer, in accord with Subsection b above.) + +The source code for a work means the preferred form of the work for +making modifications to it. For an executable work, complete source +code means all the source code for all modules it contains, plus any +associated interface definition files, plus the scripts used to +control compilation and installation of the executable. However, as a +special exception, the source code distributed need not include +anything that is normally distributed (in either source or binary +form) with the major components (compiler, kernel, and so on) of the +operating system on which the executable runs, unless that component +itself accompanies the executable. + +If distribution of executable or object code is made by offering +access to copy from a designated place, then offering equivalent +access to copy the source code from the same place counts as +distribution of the source code, even though third parties are not +compelled to copy the source along with the object code. + + 4. You may not copy, modify, sublicense, or distribute the Program +except as expressly provided under this License. Any attempt +otherwise to copy, modify, sublicense or distribute the Program is +void, and will automatically terminate your rights under this License. +However, parties who have received copies, or rights, from you under +this License will not have their licenses terminated so long as such +parties remain in full compliance. + + 5. You are not required to accept this License, since you have not +signed it. However, nothing else grants you permission to modify or +distribute the Program or its derivative works. These actions are +prohibited by law if you do not accept this License. Therefore, by +modifying or distributing the Program (or any work based on the +Program), you indicate your acceptance of this License to do so, and +all its terms and conditions for copying, distributing or modifying +the Program or works based on it. + + 6. Each time you redistribute the Program (or any work based on the +Program), the recipient automatically receives a license from the +original licensor to copy, distribute or modify the Program subject to +these terms and conditions. You may not impose any further +restrictions on the recipients' exercise of the rights granted herein. +You are not responsible for enforcing compliance by third parties to +this License. + + 7. If, as a consequence of a court judgment or allegation of patent +infringement or for any other reason (not limited to patent issues), +conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot +distribute so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you +may not distribute the Program at all. For example, if a patent +license would not permit royalty-free redistribution of the Program by +all those who receive copies directly or indirectly through you, then +the only way you could satisfy both it and this License would be to +refrain entirely from distribution of the Program. + +If any portion of this section is held invalid or unenforceable under +any particular circumstance, the balance of the section is intended to +apply and the section as a whole is intended to apply in other +circumstances. + +It is not the purpose of this section to induce you to infringe any +patents or other property right claims or to contest validity of any +such claims; this section has the sole purpose of protecting the +integrity of the free software distribution system, which is +implemented by public license practices. Many people have made +generous contributions to the wide range of software distributed +through that system in reliance on consistent application of that +system; it is up to the author/donor to decide if he or she is willing +to distribute software through any other system and a licensee cannot +impose that choice. + +This section is intended to make thoroughly clear what is believed to +be a consequence of the rest of this License. + + 8. If the distribution and/or use of the Program is restricted in +certain countries either by patents or by copyrighted interfaces, the +original copyright holder who places the Program under this License +may add an explicit geographical distribution limitation excluding +those countries, so that distribution is permitted only in or among +countries not thus excluded. In such case, this License incorporates +the limitation as if written in the body of this License. + + 9. The Free Software Foundation may publish revised and/or new versions +of the General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + +Each version is given a distinguishing version number. If the Program +specifies a version number of this License which applies to it and "any +later version", you have the option of following the terms and conditions +either of that version or of any later version published by the Free +Software Foundation. If the Program does not specify a version number of +this License, you may choose any version ever published by the Free Software +Foundation. + + 10. If you wish to incorporate parts of the Program into other free +programs whose distribution conditions are different, write to the author +to ask for permission. For software which is copyrighted by the Free +Software Foundation, write to the Free Software Foundation; we sometimes +make exceptions for this. Our decision will be guided by the two goals +of preserving the free status of all derivatives of our free software and +of promoting the sharing and reuse of software generally. + + NO WARRANTY + + 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY +FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN +OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES +PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED +OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS +TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE +PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, +REPAIR OR CORRECTION. + + 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR +REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, +INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING +OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED +TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY +YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER +PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE +POSSIBILITY OF SUCH DAMAGES. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +convey the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 2 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + + +Also add information on how to contact you by electronic and paper mail. + +If the program is interactive, make it output a short notice like this +when it starts in an interactive mode: + + Gnomovision version 69, Copyright (C) year name of author + Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, the commands you use may +be called something other than `show w' and `show c'; they could even be +mouse-clicks or menu items--whatever suits your program. + +You should also get your employer (if you work as a programmer) or your +school, if any, to sign a "copyright disclaimer" for the program, if +necessary. Here is a sample; alter the names: + + Yoyodyne, Inc., hereby disclaims all copyright interest in the program + `Gnomovision' (which makes passes at compilers) written by James Hacker. + + , 1 April 1989 + Ty Coon, President of Vice + +This General Public License does not permit incorporating your program into +proprietary programs. If your program is a subroutine library, you may +consider it more useful to permit linking proprietary applications with the +library. If this is what you want to do, use the GNU Library General +Public License instead of this License. diff --git a/extras/anytermd/Makefile b/extras/anytermd/Makefile new file mode 100644 index 0000000000..7705e16a86 --- /dev/null +++ b/extras/anytermd/Makefile @@ -0,0 +1,48 @@ +# Makefile +# This file is part of Anyterm; see http://anyterm.org/ +# (C) 2009 Philip Endecott + +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +# The Makefiles are organised so that you have have multiple build +# directories for different versions of the executable, i.e. debug vs. +# optimised builds, cross-compiled builds etc. This top-level +# Makefile simply invokes make in the default build directory and +# copies the resulting executable to the top level. If you want to +# build variants, create new build directories (e.g. build.debug) +# and copy build/Makefile into them. Make any necessary changes in +# the Makefile copies. These build directory Makefiles include a +# shared common.mk that does most of the work. + + +EXECUTABLE=anytermd + +default_target: ${EXECUTABLE} + +${EXECUTABLE}: FORCE + ${MAKE} -C build ${EXECUTABLE} + cp build/${EXECUTABLE} $@ + +clean: FORCE + ${MAKE} -C build clean + +veryclean: FORCE + ${MAKE} -C build veryclean + +FORCE: + +.PHONY: default_target clean veryclean + diff --git a/extras/anytermd/README b/extras/anytermd/README new file mode 100644 index 0000000000..2516b3acd6 --- /dev/null +++ b/extras/anytermd/README @@ -0,0 +1,6 @@ +This is an special bundle/package for anyterm, prepared to use with Pandora FMS as remote connection system for SSH and Telnet. You cannot use it the "default" anyterm packaged in your system, because it has some modifications. + +In order to use it, you need to install from sources or from Pandora FMS project packages. Please read INSTALL to know how to do it. + +For more information about anyterm, please check http://anyterm.org/ + diff --git a/extras/anytermd/anytermd.1 b/extras/anytermd/anytermd.1 new file mode 100644 index 0000000000..82e6f3230c --- /dev/null +++ b/extras/anytermd/anytermd.1 @@ -0,0 +1,122 @@ +.TH "anytermd" "1" "1.1.27" "Phil Endecott" "" +.SH "NAME" +.LP +anytermd \- A terminal\-emulating HTTP daemon +.SH "SYNOPSIS" +.LP +anytermd +[\fB\-\-command\fR \fIarg\fP] +[\fB\-\-device\fR \fIarg\fP] +[\fB\-\-port\fR \fIarg\fP] +[\fB\-\-user\fR \fIarg\fP] +[\fB\-\-auth\fR \fIarg\fP] +[\fB\-\-charset\fR \fIarg\fP] +[\fB\-\-foreground\fR] +[\fB\-\-diff\fR] +[\fB\-\-nodiff\fR] +[\fB\-\-max\-sessions\fR \fIarg\fR] +[\fB\-\-max\-http\-connections\fR \fIarg\fR] +[\fB\-\-local\-only\fR] +[\fB\-\-name\fR \fIarg\fR] +.LP +anytermd \-\-help +.SH "DESCRIPTION" +.LP +Anyterm provides a terminal window on a web page, so you can run command-line programs on +a machine even when the only access to it is over HTTP. +.LP +Anyterm consists of some Javascript on a web page, an XmlHttpRequest channel on standard ports +back to the server, an HTTP proxy such as Apache's mod_proxy and the Anyterm daemon. The daemon, +anytermd, uses a pseudo\-terminal to communicate with a shell or other application, and includes +terminal emulation. Key presses are picked up by the Javscript which sends them to the daemon; +changes to the emulated screen are sent from the daemon to the Javascript which updates its +display. Performance is quite reasonable and SSL can be used to secure the connection. +\fIFor security reasons, it is highly recommended that anytermd NOT be exposed directly to +the Internet!\fR See the section \fBSECURITY\fR below. +.SH "OPTIONS" +.LP +Default values are indicated in parentheses (). +.TP +\fB\-c | \-\-command\fR \fIarg\fP (=/bin/bash) +Command to run in terminal +.TP +\fB\-d | \-\-device\fR \fIarg\fP +Device to connect to (e.g. serial port) +.TP +\fB\-p | \-\-port\fR \fIarg\fP (=8080) +Port number to listen on +.TP +\fB\-u | \-\-user\fR \fIarg\fP +User to run as +.TP +\fB\-a | \-\-auth\fR \fIarg\fP (=none) +Authorisation: none|null|trivial +.TP +\fB\-s | \-\-charset\fR \fIarg\fP (=ascii) +Character set +.TP +\fB\-f | \-\-foreground\fR +run in foreground +.TP +\fB\-\-diff\fR +Send only differences to browser +.TP +\fB\-n | \-\-nodiff\fR +Send whole screen to browser each time +.TP +\fB\-m | \-\-max\-sessions\fR \fIarg\fR (=20) +Maximum number of simultaneous sessions +.TP +\fB\-\-max\-http\-connections\fR \fIarg\fR (=unlimited) +Maximum number of simultaneous HTTP connections +.TP +\fB\-\-local\-only\fR +Accept connections only from localhost +.TP +\fB\-\-name\fR \fIarg\fR (=anyterm) +Name used for logging and pid file +.TP +\fB\-\-help\fR +show help message +.SH "SECURITY" +.LP +\fIDo not run anytermd as root.\fR +.LP +Anyterm's code has not been audited for security. It is almost certainly possible to cause it to +crash by sending it invalid input, and experience suggests that this sort of weakness can also be +exploited to allow arbitary code execution. +.LP +Note that all CGI applications and similar programs are subject to this sort of problem, though +some are better written than others. The Anyterm code does not need run with a higher priviledge +level than any other web application, so weaknesses in Anyterm do not put your system at any higher +risk than weaknesses in any other web application would. +.LP +\fIAvoid exposing anytermd directly to the Internet.\fR +.LP +anytermd should be used only as a backend service, reachable only via a frontend HTTPS +proxy, such as Apache, that requires authentication. \fIhttp://anyterm.org/1.1/install.html\fR +gives examples of how to set this up. +.LP +A more complete and up\-to\-date discussion of security issues in anytermd can be found at +\fIhttp://anyterm.org/security.html\fR. +.SH "EXAMPLES" +.LP +If you just want to test Anyterm, you can do: +.LP +anytermd \-\-local\-only +.LP +and then connect to http://localhost:8080 with your Javascript\-enabled web browser. The +brower should display a terminal window containing a bash command prompt owned by the user who +started anytermd. +.LP +If you want to set up SSH\-over\-HTTP access on localhost:8022, you might run it as: +.LP +anytermd \-\-command "/usr/bin/ssh \-o your-username@localhost" \-\-port 8022 \-\-user nobody +\-\-charset utf\-8 \-\-local\-only +.SH "SEE ALSO" +.LP +Anyterm is more fully documented on its web site, \fBhttp://anyterm.org/\fP. The web site +may be more up-to-date than this man page. +.SH "AUTHOR" +.LP +Anyterm is written by Phil Endecott. This man page was contributed by Eric Rossen. diff --git a/extras/anytermd/anytermd.redhat.spec b/extras/anytermd/anytermd.redhat.spec new file mode 100644 index 0000000000..926d1fa0fe --- /dev/null +++ b/extras/anytermd/anytermd.redhat.spec @@ -0,0 +1,55 @@ +Name: anytermd +Version: 1.1.29 +Release: 1.1.29 +Summary: Pandora FMS Remote connection gateway +License: GPLv2 +Group: Applications/Communications +Vendor: ArticaST +Source0: %{name}-%{version}.tar.gz +Packager: Sancho Lerena +BuildRoot: %{_tmppath}/%{name}-%{version}-buildroot +URL: http://pandorafms.org +Requires(post): /sbin/chkconfig +Requires(preun): /sbin/chkconfig +Requires(preun): /sbin/service +Requires(postun): /sbin/service +Requires: telnet openssh-clients +%description +Pandora FMS uses a tool called "anytermd" to create a "proxy" between user browser and remote destination. This tool launches as a daemon, listeting in a port, and executing a command, forwarding all output to the user browser. That means all the connections are done FROM the pandora server and it has to be installed the telnet and ssh client + +%prep +rm -rf $RPM_BUILD_ROOT + +%setup -q -n anytermd + +%build +make + +%install +install -Dm 755 contrib/anytermd $RPM_BUILD_ROOT%{_initrddir}/anytermd +install -Dm 755 anytermd $RPM_BUILD_ROOT%{_bindir}/anytermd + +%clean +rm -rf $RPM_BUILD_ROOT + +%post +if [ $1 -eq 0 ]; then + /sbin/chkconfig --add anytermd +fi + +%preun +if [ $1 -eq 0 ]; then + /sbin/chkconfig --del anytermd +fi + +%postun +if [ $1 -ge 1 ]; then + /sbin/service anytermd stop >/dev/null 2>&1 +fi + +%files +%defattr(-,root,root,-) +%{_bindir}/* +%{_initrddir}/anytermd + + diff --git a/extras/anytermd/browser/anyterm.css b/extras/anytermd/browser/anyterm.css new file mode 100644 index 0000000000..92e1377f16 --- /dev/null +++ b/extras/anytermd/browser/anyterm.css @@ -0,0 +1,130 @@ +/* browser/anyterm.css + This file is part of Anyterm; see http://anyterm.org/ + (C) 2005-2008 Philip Endecott + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 2 of the License, or + any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +*/ + + +/* These are the background colours: */ +.a { background-color: #000000; } /* black */ +.b { background-color: #cd0000; } /* red */ +.c { background-color: #00cd00; } /* green */ +.d { background-color: #cdcd00; } /* yellow */ +.e { background-color: #0000cd; } /* blue */ +.f { background-color: #cd00cd; } /* magenta */ +.g { background-color: #00cdcd; } /* cyan */ +.h { background-color: #e5e5e5; } /* white */ + +/* These are the foreground colours used when bold mode is NOT enabled. + They're the same as the background colours. */ +.i { color: #000000; } /* black */ +.j { color: #cd0000; } /* red */ +.k { color: #00cd00; } /* green */ +.l { color: #cdcd00; } /* yellow */ +.m { color: #0000cd; } /* blue */ +.n { color: #cd00cd; } /* magenta */ +.o { color: #00cdcd; } /* cyan */ +.p { color: #e5e5e5; } /* white */ + +/* These are the brighter foreground colours used when bold mode IS enabled. + The business with !important and .p .z is because the .p default is set + on the enclosing term element; I can't see a better way to get the desired + behaviour. */ +.z.i { color: #4d4d4d !important; } /* black */ +.z.j { color: #ff0000 !important; } /* red */ +.z.k { color: #00ff00 !important; } /* green */ +.z.l { color: #ffff00 !important; } /* yellow */ +.z.m { color: #0000ff !important; } /* blue */ +.z.n { color: #ff00ff !important; } /* magenta */ +.z.o { color: #00ffff !important; } /* cyan */ +.z.p, .p .z { color: #ffffff; } /* white */ + +/* If you want a black-on-white colour scheme like xterm, rather than the + default white-on-black, you need to change the lines for black and white + above to something like the following: + .a { background-color: #ffffff; } + .h { background-color: #000000; } + .i { color: #e5e5e5; } + .p { color: #000000; } + .z.i { color: #ffffff !important; } + .z.p, .p .z { color: #000000; } +*/ + +/* If the following rule is enabled, bold mode will actually use a bold font. + This is not a good idea in general as the bold font will probably be wider + than the normal font, messing up the layout, at least for some subset of + the character set. So it's commented out; bold characters will be + distinguished only by their brighter colours (above) */ +/* .z { font-weight: bold; } */ + +/* The cursor. You can make it blink if you really want to (on some browsers). */ +.cursor { + border: 1px solid red; + margin: -1px; +/*text-decoration: blink;*/ +} + + +/* Properties for the page outside the terminal: */ + +body { + background-color: white; + /* Don't like the white background? How about this: + background-color: #222222; + */ +} + +noscript { + /* This is for the message that users see if they don't have Javascript + enabled. We want it to be visible whatever the page background colour + (above) is set to, so we give it its own background colour. */ + color: black; + background-color: white; +} + + +/* The remaining definitions determine the appearance of the frame around the + terminal, its title bar, and buttons: */ + +.termframe { + float: left; + background-color: #567953; + padding: 0.2ex; +} + +.termframe p { + margin: 0; + color: white; + font-weight: bold; + font-family: sans-serif; +} + +.termframe a { + cursor: pointer; +} + +img.button { + margin: 0 3px; + cursor: pointer; + vertical-align: top; +} + +.term { + line-height: 17px; + overflow: auto; + overflow-x: visible; +} + diff --git a/extras/anytermd/browser/anyterm.html b/extras/anytermd/browser/anyterm.html new file mode 100644 index 0000000000..004bb11dfd --- /dev/null +++ b/extras/anytermd/browser/anyterm.html @@ -0,0 +1,28 @@ + + + +Pandora FMS Remote Gateway + + + + + + + +
+ + diff --git a/extras/anytermd/browser/anyterm.js b/extras/anytermd/browser/anyterm.js new file mode 100644 index 0000000000..e073b8c5a4 --- /dev/null +++ b/extras/anytermd/browser/anyterm.js @@ -0,0 +1,789 @@ +// browser/anyterm.js +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2005-2006 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +var undefined; + +var url_prefix = ""; + +var frame; +var term; +var open=false; +var session; + +var method="POST"; +//var method="GET"; + +// Random sequence numbers are needed to prevent Opera from caching +// replies + +var is_opera = navigator.userAgent.toLowerCase().indexOf("opera") != -1; +if (is_opera) { + method="GET"; +} + +var seqnum_val=Math.round(Math.random()*100000); +function cachebust() { + if (is_opera) { + seqnum_val++; + return "&x="+seqnum_val; + } else { + return ""; + } +} + + +// Cross-platform creation of XMLHttpRequest object: + +function new_XMLHttpRequest() { + if (window.XMLHttpRequest) { + // For most browsers: + return new XMLHttpRequest(); + } else { + // For IE, it's active-X voodoo. + // There are different versions in different browsers. + // The ones we try are the ones that Sarissa tried. The disabled ones + // apparently also exist, but it seems to work OK without trying them. + + //try{ return new ActiveXObject("MSXML3.XMLHTTP"); } catch(e){} + try{ return new ActiveXObject("Msxml2.XMLHTTP.5.0"); } catch(e){} + try{ return new ActiveXObject("Msxml2.XMLHTTP.4.0"); } catch(e){} + try{ return new ActiveXObject("MSXML2.XMLHTTP.3.0"); } catch(e){} + try{ return new ActiveXObject("MSXML2.XMLHTTP"); } catch(e){} + //try{ return new ActiveXObject("Msxml2.XMLHTTP"); } catch(e){} + try{ return new ActiveXObject("Microsoft.XMLHTTP"); } catch(e){} + throw new Error("Could not find an XMLHttpRequest active-X class.") + } +} + + +// Asynchronous and Synchronous XmlHttpRequest wrappers + +// AsyncLoader is a class; an instance specifies a callback function. +// Call load to get something and the callback is invoked with the +// returned document. + +function AsyncLoader(cb) { + this.callback = cb; + this.load = function (url,query) { + var xmlhttp = new_XMLHttpRequest(); + var cbk = this.callback; + //var timeoutID = window.setTimeout("alert('No response after 20 secs')",20000); + xmlhttp.onreadystatechange = function () { + if (xmlhttp.readyState==4) { + //window.clearTimeout(timeoutID); + if (xmlhttp.status==200) { + cbk(xmlhttp.responseText); + } else { + alert("Server returned status code "+xmlhttp.status+":\n"+xmlhttp.statusText); + cbk(null); + } + } + } + if (method=="GET") { + xmlhttp.open(method, url+"?"+query, true); + xmlhttp.send(null); + } else if (method=="POST") { + xmlhttp.open(method, url, true); + xmlhttp.setRequestHeader('Content-Type', + 'application/x-www-form-urlencoded'); + xmlhttp.send(query); + } + + } +} + + +// Synchronous loader is a simple function + +function sync_load(url,query) { + var xmlhttp = new_XMLHttpRequest(); + if (method=="GET") { + xmlhttp.open(method, url+"?"+query, false); + xmlhttp.send(null); + } else if (method=="POST") { + xmlhttp.open(method, url, false); + xmlhttp.setRequestHeader('Foo','1234'); + xmlhttp.setRequestHeader('Content-Type', + 'application/x-www-form-urlencoded'); + xmlhttp.send(query); + } + if (xmlhttp.status!=200) { + alert("Server returned status code "+xmlhttp.status+":\n"+xmlhttp.statusText); + return null; + } + return xmlhttp.responseText; +} + + +// Process error message from server: + +function handle_resp_error(resp) { + if (resp.charAt(0)=="E") { + var msg = resp.substr(1); + alert(msg); + return true; + } + return false; +} + + +// Receive channel: + +var rcv_loader; + +var disp=""; + + + +function process_editscript(edscr) { + + var ndisp=""; + + var i=0; + var dp=0; + while (i=65 && kc<=90) k=String.fromCharCode(kc-64); // Ctrl-A..Z + else if (kc==219) k=String.fromCharCode(27); // Ctrl-[ + else if (kc==220) k=String.fromCharCode(28); // Ctrl-\ . + else if (kc==221) k=String.fromCharCode(29); // Ctrl-] + else if (kc==190) k=String.fromCharCode(30); // Since ctrl-^ doesn't work, map + // ctrl-. to its code. + else if (kc==32) k=String.fromCharCode(0); // Ctrl-space sends 0, like ctrl-@. + else { + key_ev_supress(ev); + return; + } + } + } + +// alert("keydown keyCode="+ev.keyCode+" which="+ev.which+ +// " shiftKey="+ev.shiftKey+" ctrlKey="+ev.ctrlKey+" altKey="+ev.altKey); + + process_key(k); + + key_ev_stop(ev); + return false; +} + + +// Open, close and initialisation: + +function open_term(rows,cols,p,charset,scrollback) { + var params = "a=open&rows="+rows+"&cols="+cols; + if (p) { + params += "&p="+p; + } + if (charset) { + params += "&ch="+charset; + } + if (scrollback) { + if (scrollback>1000) { + alert("The maximum scrollback is currently limited to 1000 lines. " + +"Please choose a smaller value and try again."); + return; + } + params += "&sb="+scrollback; + } + params += cachebust(); + var resp = sync_load(url_prefix+"anyterm-module",params); + + if (handle_resp_error(resp)) { + return; + } + + open=true; + session=resp; +} + +function close_term() { + if (!open) { + alert("Connection is not open"); + return; + } + open=false; + var resp = sync_load(url_prefix+"anyterm-module","a=close&s="+session+cachebust()); + handle_resp_error(resp); // If we get an error, we still close everything. + document.onkeypress=null; + document.onkeydown=null; + window.onbeforeunload=null; + var e; + while (e=frame.firstChild) { + frame.removeChild(e); + } + frame.className=""; + if (on_close_goto_url) { + document.location = on_close_goto_url; + } +} + + +function get_anyterm_version() { + var svn_url="$URL: file:///var/lib/svn/anyterm/tags/releases/1.1/1.1.29/browser/anyterm.js $"; + var re = /releases\/[0-9]+\.[0-9]+\/([0-9\.]+)/; + var match = re.exec(svn_url); + if (match) { + return match[1]; + } else { + return ""; + } +} + +function substitute_variables(s) { + var version = get_anyterm_version(); + if (version!="") { + version="-"+version; + } + var hostname=document.location.host; + return s.replace(/%v/g,version).replace(/%h/g,hostname); +} + + +// Copying + +function copy_ie_clipboard() { + try { + window.document.execCommand("copy",false,null); + } catch (err) { + return undefined; + } + return 1; +} + +function copy_mozilla_clipboard() { + // Thanks to Simon Wissinger for this function. + + try { + netscape.security.PrivilegeManager.enablePrivilege("UniversalXPConnect"); + } catch (err) { + return undefined; + } + + var sel=window.getSelection(); + var copytext=sel.toString(); + + var str=Components.classes["@mozilla.org/supports-string;1"] + .createInstance(Components.interfaces.nsISupportsString); + if (!str) return undefined; + + str.data=copytext; + + var trans=Components.classes["@mozilla.org/widget/transferable;1"] + .createInstance(Components.interfaces.nsITransferable); + if (!trans) return undefined; + + trans.addDataFlavor("text/unicode"); + trans.setTransferData("text/unicode", str, copytext.length * 2); + + var clipid=Components.interfaces.nsIClipboard; + + var clip=Components.classes["@mozilla.org/widget/clipboard;1"].getService(clipid); + if (!clip) return undefined; + + clip.setData(trans, null, clipid.kGlobalClipboard); + + return 1; +} + +function copy_to_clipboard() { + var r=copy_ie_clipboard(); + if (r==undefined) { + r=copy_mozilla_clipboard(); + } + if (r==undefined) { + alert("Copy seems to be disabled; maybe you need to change your security settings?" + +"\n(Copy on the Edit menu will probably work)"); + } +} + + +// Pasting + +function get_mozilla_clipboard() { + // This function is taken from + // http://www.nomorepasting.com/paste.php?action=getpaste&pasteID=41974&PHPSESSID=e6565dcf5de07256345e562b97ac9f46 + // which does not indicate any particular copyright conditions. It + // is a public forum, so one might conclude that it is public + // domain. + + // IMHO it's disgraceful that Mozilla makes us use these 30 lines of + // undocumented gobledegook to do what IE does, and documents, with + // just 'window.clipboardData.getData("Text")'. What on earth were + // they thinking? + + try { + netscape.security.PrivilegeManager.enablePrivilege("UniversalXPConnect"); + } catch (err) { + return undefined; + } + + var clip = Components.classes["@mozilla.org/widget/clipboard;1"] + .createInstance(Components.interfaces.nsIClipboard); + if (!clip) { + return undefined; + } + + var trans = Components.classes["@mozilla.org/widget/transferable;1"] + .createInstance(Components.interfaces.nsITransferable); + if (!trans) { + return undefined; + } + + trans.addDataFlavor("text/unicode"); + clip.getData(trans,clip.kGlobalClipboard); + + var str=new Object(); + var strLength=new Object(); + + try { + trans.getTransferData("text/unicode",str,strLength); + } catch(err) { + // One reason for getting here seems to be that nothing is selected + return ""; + } + + if (str) { + str=str.value.QueryInterface(Components.interfaces.nsISupportsString); + } + + if (str) { + return str.data.substring(0,strLength.value / 2); + } else { + return ""; // ? is this "clipboard empty" or "cannot access"? + } +} + +function get_ie_clipboard() { + if (window.clipboardData) { + return window.clipboardData.getData("Text"); + } + return undefined; +} + +function get_default_clipboard() { + return prompt("Paste into this box and press OK:",""); +} + +function paste_from_clipboard() { + var p = get_ie_clipboard(); + if (p==undefined) { + p = get_mozilla_clipboard(); + } + if (p==undefined) { + p = get_default_clipboard(); + if (p) { + process_key(p); + } + return; + } + + if (p=="") { + alert("The clipboard seems to be empty"); + return; + } + + if (confirm('Click OK to "type" the following into the terminal:\n'+p)) { + process_key(p); + } +} + + +function create_button(label,fn) { + var button=document.createElement("A"); + var button_t=document.createTextNode("["+label+"] "); + button.appendChild(button_t); + button.onclick=fn; + return button; +} + +function create_img_button(imgfn,label,fn) { + var button=document.createElement("A"); + var button_img=document.createElement("IMG"); + var class_attr=document.createAttribute("CLASS"); + class_attr.value="button"; + button_img.setAttributeNode(class_attr); + var src_attr=document.createAttribute("SRC"); + src_attr.value=imgfn; + button_img.setAttributeNode(src_attr); + var alt_attr=document.createAttribute("ALT"); + alt_attr.value="["+label+"] "; + button_img.setAttributeNode(alt_attr); + var title_attr=document.createAttribute("TITLE"); + title_attr.value=label; + button_img.setAttributeNode(title_attr); + button.appendChild(button_img); + button.onclick=fn; + return button; +} + +function create_term(elem_id,title,rows,cols,p,charset,scrollback) { + if (open) { + alert("Terminal is already open"); + return; + } + title=substitute_variables(title); + frame=document.getElementById(elem_id); + if (!frame) { + alert("There is no element named '"+elem_id+"' in which to build a terminal"); + return; + } + frame.className="termframe"; + var title_p=document.createElement("P"); + title_p.appendChild(create_img_button("copy.gif","Copy",copy_to_clipboard)); + title_p.appendChild(create_img_button("paste.gif","Paste",paste_from_clipboard)); + title_p.appendChild(create_ctrlkey_menu()); + var title_t=document.createTextNode(" "+title+" "); + title_p.appendChild(title_t); +// title_p.appendChild(create_button("close",close_term)); + frame.appendChild(title_p); + term=document.createElement("PRE"); + frame.appendChild(term); + term.className="term a p"; + var termbody=document.createTextNode(""); + term.appendChild(termbody); + visible_height_frac=Number(rows)/(Number(rows)+Number(scrollback)); + if (scrollback>0) { + term.style.overflowY="scroll"; + } + document.onhelp = function() { return false; }; + document.onkeypress=keypress; + document.onkeydown=keydown; + open_term(rows,cols,p,charset,scrollback); + if (open) { + window.onbeforeunload=warn_unload; + get(); + maybe_send(); + } +} + + +function warn_unload() { + if (open) { + return "Leaving this page will close the terminal."; + } +} + + +function create_ctrlkey_menu() { + var sel=document.createElement("SELECT"); + create_ctrlkey_menu_entry(sel,"Control keys...",-1); + create_ctrlkey_menu_entry(sel,"Ctrl-@",0); + for (var code=1; code<27; code++) { + var letter=String.fromCharCode(64+code); + create_ctrlkey_menu_entry(sel,"Ctrl-"+letter,code); + } + create_ctrlkey_menu_entry(sel,"Ctrl-[",27); + create_ctrlkey_menu_entry(sel,"Ctrl-\\",28); + create_ctrlkey_menu_entry(sel,"Ctrl-]",29); + create_ctrlkey_menu_entry(sel,"Ctrl-^",30); + create_ctrlkey_menu_entry(sel,"Ctrl-_",31); + sel.onchange=function() { + var code = sel.options[sel.selectedIndex].value; + if (code>=0) { + process_key(String.fromCharCode(code)); + } + }; + return sel; +} + +function create_ctrlkey_menu_entry(sel,name,code) { + var opt=document.createElement("OPTION"); + opt.appendChild(document.createTextNode(name)); + var value_attr=document.createAttribute("VALUE"); + value_attr.value=code; + opt.setAttributeNode(value_attr); + sel.appendChild(opt); +} + diff --git a/extras/anytermd/browser/copy.gif b/extras/anytermd/browser/copy.gif new file mode 100644 index 0000000000..2ab719e349 Binary files /dev/null and b/extras/anytermd/browser/copy.gif differ diff --git a/extras/anytermd/browser/copy.png b/extras/anytermd/browser/copy.png new file mode 100644 index 0000000000..1cdd8c1de2 Binary files /dev/null and b/extras/anytermd/browser/copy.png differ diff --git a/extras/anytermd/browser/paste.gif b/extras/anytermd/browser/paste.gif new file mode 100644 index 0000000000..65ddba8bac Binary files /dev/null and b/extras/anytermd/browser/paste.gif differ diff --git a/extras/anytermd/browser/paste.png b/extras/anytermd/browser/paste.png new file mode 100644 index 0000000000..034debdbf7 Binary files /dev/null and b/extras/anytermd/browser/paste.png differ diff --git a/extras/anytermd/build/Activity.d b/extras/anytermd/build/Activity.d new file mode 100644 index 0000000000..9064aba790 --- /dev/null +++ b/extras/anytermd/build/Activity.d @@ -0,0 +1,5 @@ +Activity.d ../src/Activity.o: ../src/Activity.cc ../src/Activity.hh \ + ../libpbe/include/FileDescriptor.hh ../libpbe/include/Exception.hh \ + ../libpbe/include/missing_syscalls.hh \ + ../libpbe/include/compiler_magic.hh ../libpbe/include/Thread.hh \ + ../libpbe/include/select.hh ../libpbe/include/Exception.hh diff --git a/extras/anytermd/build/Anyterm.d b/extras/anytermd/build/Anyterm.d new file mode 100644 index 0000000000..10d6ee7c94 --- /dev/null +++ b/extras/anytermd/build/Anyterm.d @@ -0,0 +1,17 @@ +Anyterm.d ../src/Anyterm.o: ../src/Anyterm.cc ../src/Anyterm.hh \ + ../libpbe/include/Locked.hh ../libpbe/include/Mutex.hh \ + ../libpbe/include/atomic.hh ../libpbe/include/Lock.hh \ + ../libpbe/include/Exception.hh ../libpbe/include/Futex.hh \ + ../libpbe/include/missing_syscalls.hh ../src/Session.hh \ + ../src/SessionId.hh ../libpbe/include/Mutex.hh \ + ../libpbe/include/Condition.hh ../libpbe/include/HPtime.hh \ + ../libpbe/include/Unlock.hh ../src/Screen.hh ../src/Cell.hh \ + ../src/Attributes.hh ../src/unicode.hh ../libpbe/include/endian.hh \ + ../src/Terminal.hh ../libpbe/include/Iconver.hh ../src/Activity.hh \ + ../libpbe/include/FileDescriptor.hh ../libpbe/include/compiler_magic.hh \ + ../libpbe/include/Thread.hh ../src/config.hh \ + ../libpbe/include/HttpRequest.hh ../libpbe/include/URI.hh \ + ../src/Error.hh ../src/auto_CgiParams.hh ../src/CgiParams.hh \ + ../src/SubProcess.hh ../src/SerialPortActivity.hh \ + ../libpbe/include/SerialPort.hh ../libpbe/include/FileDescriptor.hh \ + ../src/expand_command.hh ../libpbe/include/compiler_magic.hh diff --git a/extras/anytermd/build/AnytermDaemon.d b/extras/anytermd/build/AnytermDaemon.d new file mode 100644 index 0000000000..f58c95248a --- /dev/null +++ b/extras/anytermd/build/AnytermDaemon.d @@ -0,0 +1,21 @@ +AnytermDaemon.d ../src/AnytermDaemon.o: ../src/AnytermDaemon.cc \ + ../src/AnytermDaemon.hh ../libpbe/include/HttpDaemon.hh \ + ../libpbe/include/Daemon.hh ../libpbe/include/FileDescriptor.hh \ + ../libpbe/include/Exception.hh ../libpbe/include/missing_syscalls.hh \ + ../libpbe/include/compiler_magic.hh ../libpbe/include/Mutex.hh \ + ../libpbe/include/atomic.hh ../libpbe/include/Lock.hh \ + ../libpbe/include/Futex.hh ../libpbe/include/Condition.hh \ + ../libpbe/include/HPtime.hh ../libpbe/include/Unlock.hh \ + ../libpbe/include/HttpRequest.hh ../libpbe/include/URI.hh \ + ../libpbe/include/HttpResponse.hh ../libpbe/include/HttpAuthenticator.hh \ + ../libpbe/include/HttpRequest.hh ../libpbe/include/HttpResponse.hh \ + ../src/Anyterm.hh ../libpbe/include/Locked.hh ../src/Session.hh \ + ../src/SessionId.hh ../libpbe/include/Mutex.hh \ + ../libpbe/include/Condition.hh ../src/Screen.hh ../src/Cell.hh \ + ../src/Attributes.hh ../src/unicode.hh ../libpbe/include/endian.hh \ + ../src/Terminal.hh ../libpbe/include/Iconver.hh ../src/Activity.hh \ + ../libpbe/include/FileDescriptor.hh ../libpbe/include/Thread.hh \ + ../src/config.hh ../src/NullAuthenticator.hh \ + ../libpbe/include/HttpAuthenticator.hh \ + ../libpbe/include/compiler_magic.hh ../src/TrivialAuthenticator.hh \ + ../src/static_content.hh ../libpbe/include/segv_backtrace.hh diff --git a/extras/anytermd/build/Makefile b/extras/anytermd/build/Makefile new file mode 100644 index 0000000000..7c8ddc08ea --- /dev/null +++ b/extras/anytermd/build/Makefile @@ -0,0 +1,31 @@ +# build/Makefile +# This file is part of Anyterm; see http://anyterm.org/ +# (C) 2009 Philip Endecott + +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +# The Makefiles are aranged so that you can have multiple build +# directories for different build variants e.g. with or without +# debugging etc. Each build directory's Makefile should just set +# any necessary variables and then include ../common.mk, which does +# all of the work. + + +DEBUG_FLAGS= +OPTIMISE_FLAGS=-O + +include ../common.mk + diff --git a/extras/anytermd/build/SerialPort.d b/extras/anytermd/build/SerialPort.d new file mode 100644 index 0000000000..c0e1c01f7e --- /dev/null +++ b/extras/anytermd/build/SerialPort.d @@ -0,0 +1,7 @@ +SerialPort.d ../src/SerialPort.o: ../src/SerialPort.cc \ + ../src/SerialPortActivity.hh ../src/Activity.hh \ + ../libpbe/include/FileDescriptor.hh ../libpbe/include/Exception.hh \ + ../libpbe/include/missing_syscalls.hh \ + ../libpbe/include/compiler_magic.hh ../libpbe/include/Thread.hh \ + ../libpbe/include/SerialPort.hh ../libpbe/include/FileDescriptor.hh \ + ../libpbe/include/select.hh ../libpbe/include/Exception.hh diff --git a/extras/anytermd/build/Session.d b/extras/anytermd/build/Session.d new file mode 100644 index 0000000000..fdcfa8585f --- /dev/null +++ b/extras/anytermd/build/Session.d @@ -0,0 +1,12 @@ +Session.d ../src/Session.o: ../src/Session.cc ../src/Session.hh \ + ../src/SessionId.hh ../libpbe/include/Mutex.hh \ + ../libpbe/include/atomic.hh ../libpbe/include/Lock.hh \ + ../libpbe/include/Exception.hh ../libpbe/include/Futex.hh \ + ../libpbe/include/missing_syscalls.hh ../libpbe/include/Condition.hh \ + ../libpbe/include/HPtime.hh ../libpbe/include/Unlock.hh ../src/Screen.hh \ + ../src/Cell.hh ../src/Attributes.hh ../src/unicode.hh \ + ../libpbe/include/endian.hh ../src/Terminal.hh \ + ../libpbe/include/Iconver.hh ../src/Activity.hh \ + ../libpbe/include/FileDescriptor.hh ../libpbe/include/compiler_magic.hh \ + ../libpbe/include/Thread.hh ../libpbe/include/Lock.hh ../src/config.hh \ + ../src/html.hh ../src/editscript.hh ../src/Error.hh diff --git a/extras/anytermd/build/SessionId.d b/extras/anytermd/build/SessionId.d new file mode 100644 index 0000000000..9a302ec579 --- /dev/null +++ b/extras/anytermd/build/SessionId.d @@ -0,0 +1 @@ +SessionId.d ../src/SessionId.o: ../src/SessionId.cc ../src/SessionId.hh diff --git a/extras/anytermd/build/SubProcess.d b/extras/anytermd/build/SubProcess.d new file mode 100644 index 0000000000..59d3833b2b --- /dev/null +++ b/extras/anytermd/build/SubProcess.d @@ -0,0 +1,6 @@ +SubProcess.d ../src/SubProcess.o: ../src/SubProcess.cc \ + ../src/SubProcess.hh ../src/Activity.hh \ + ../libpbe/include/FileDescriptor.hh ../libpbe/include/Exception.hh \ + ../libpbe/include/missing_syscalls.hh \ + ../libpbe/include/compiler_magic.hh ../libpbe/include/Thread.hh \ + ../libpbe/include/select.hh ../libpbe/include/Exception.hh diff --git a/extras/anytermd/build/Terminal.d b/extras/anytermd/build/Terminal.d new file mode 100644 index 0000000000..faf0abd849 --- /dev/null +++ b/extras/anytermd/build/Terminal.d @@ -0,0 +1,5 @@ +Terminal.d ../src/Terminal.o: ../src/Terminal.cc ../src/Terminal.hh \ + ../src/Cell.hh ../src/Attributes.hh ../src/unicode.hh \ + ../libpbe/include/endian.hh ../src/Screen.hh \ + ../libpbe/include/Iconver.hh ../libpbe/include/Exception.hh \ + ../libpbe/include/Exception.hh diff --git a/extras/anytermd/build/UrlEncodedCgiParams.d b/extras/anytermd/build/UrlEncodedCgiParams.d new file mode 100644 index 0000000000..3ad294d326 --- /dev/null +++ b/extras/anytermd/build/UrlEncodedCgiParams.d @@ -0,0 +1,3 @@ +UrlEncodedCgiParams.d ../src/UrlEncodedCgiParams.o: \ + ../src/UrlEncodedCgiParams.cc ../src/UrlEncodedCgiParams.hh \ + ../src/CgiParams.hh ../libpbe/include/Exception.hh diff --git a/extras/anytermd/build/auto_CgiParams.d b/extras/anytermd/build/auto_CgiParams.d new file mode 100644 index 0000000000..8fdc89ff9f --- /dev/null +++ b/extras/anytermd/build/auto_CgiParams.d @@ -0,0 +1,4 @@ +auto_CgiParams.d ../src/auto_CgiParams.o: ../src/auto_CgiParams.cc \ + ../src/auto_CgiParams.hh ../src/CgiParams.hh \ + ../libpbe/include/HttpRequest.hh ../libpbe/include/URI.hh \ + ../src/UrlEncodedCgiParams.hh diff --git a/extras/anytermd/build/diff.d b/extras/anytermd/build/diff.d new file mode 100644 index 0000000000..dd7d4aad44 --- /dev/null +++ b/extras/anytermd/build/diff.d @@ -0,0 +1,2 @@ +diff.d ../src/diff.o: ../src/diff.cc ../src/diff.hh ../src/unicode.hh \ + ../libpbe/include/endian.hh diff --git a/extras/anytermd/build/editscript.d b/extras/anytermd/build/editscript.d new file mode 100644 index 0000000000..9be680a17f --- /dev/null +++ b/extras/anytermd/build/editscript.d @@ -0,0 +1,3 @@ +editscript.d ../src/editscript.o: ../src/editscript.cc \ + ../src/editscript.hh ../src/unicode.hh ../libpbe/include/endian.hh \ + ../src/diff.hh diff --git a/extras/anytermd/build/expand_command.d b/extras/anytermd/build/expand_command.d new file mode 100644 index 0000000000..e3b3af3ddd --- /dev/null +++ b/extras/anytermd/build/expand_command.d @@ -0,0 +1 @@ +expand_command.d ../src/expand_command.o: ../src/expand_command.cc diff --git a/extras/anytermd/build/html.d b/extras/anytermd/build/html.d new file mode 100644 index 0000000000..af4fb40175 --- /dev/null +++ b/extras/anytermd/build/html.d @@ -0,0 +1,3 @@ +html.d ../src/html.o: ../src/html.cc ../src/html.hh ../src/Screen.hh \ + ../src/Cell.hh ../src/Attributes.hh ../src/unicode.hh \ + ../libpbe/include/endian.hh diff --git a/extras/anytermd/build/main.d b/extras/anytermd/build/main.d new file mode 100644 index 0000000000..4662c52719 --- /dev/null +++ b/extras/anytermd/build/main.d @@ -0,0 +1,21 @@ +main.d ../src/main.o: ../src/main.cc ../src/AnytermDaemon.hh \ + ../libpbe/include/HttpDaemon.hh ../libpbe/include/Daemon.hh \ + ../libpbe/include/FileDescriptor.hh ../libpbe/include/Exception.hh \ + ../libpbe/include/missing_syscalls.hh \ + ../libpbe/include/compiler_magic.hh ../libpbe/include/Mutex.hh \ + ../libpbe/include/atomic.hh ../libpbe/include/Lock.hh \ + ../libpbe/include/Futex.hh ../libpbe/include/Condition.hh \ + ../libpbe/include/HPtime.hh ../libpbe/include/Unlock.hh \ + ../libpbe/include/HttpRequest.hh ../libpbe/include/URI.hh \ + ../libpbe/include/HttpResponse.hh ../libpbe/include/HttpAuthenticator.hh \ + ../libpbe/include/HttpRequest.hh ../libpbe/include/HttpResponse.hh \ + ../src/Anyterm.hh ../libpbe/include/Locked.hh ../src/Session.hh \ + ../src/SessionId.hh ../libpbe/include/Mutex.hh \ + ../libpbe/include/Condition.hh ../src/Screen.hh ../src/Cell.hh \ + ../src/Attributes.hh ../src/unicode.hh ../libpbe/include/endian.hh \ + ../src/Terminal.hh ../libpbe/include/Iconver.hh ../src/Activity.hh \ + ../libpbe/include/FileDescriptor.hh ../libpbe/include/Thread.hh \ + ../src/config.hh ../src/NullAuthenticator.hh \ + ../libpbe/include/HttpAuthenticator.hh \ + ../libpbe/include/compiler_magic.hh ../src/TrivialAuthenticator.hh \ + ../libpbe/include/Exception.hh ../libpbe/include/segv_backtrace.hh diff --git a/extras/anytermd/build/static_content.d b/extras/anytermd/build/static_content.d new file mode 100644 index 0000000000..b410742115 --- /dev/null +++ b/extras/anytermd/build/static_content.d @@ -0,0 +1 @@ +static_content.d static_content.o: static_content.cc static_content.hh diff --git a/extras/anytermd/common.mk b/extras/anytermd/common.mk new file mode 100644 index 0000000000..f8281ee5fd --- /dev/null +++ b/extras/anytermd/common.mk @@ -0,0 +1,115 @@ +# common.mk +# This file is part of Anyterm; see http://anyterm.org/ +# (C) 2005-2009 Philip Endecott + +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +# You can have multiple build directories each with their own Makefiles. +# Those Makefiles include this common code; they can set variables to +# control things like optimisation level, debugging etc first. You should +# make changes in those build Makefiles rather than here, unless your +# change shouls apply to all build variants. + +# Note that paths in here are relative to the build directory. + +default_target: anytermd + +SRC_DIR=../src + +VPATH=${SRC_DIR} . + +UNAME_S=$(shell uname -s) + +ifeq (${UNAME_S},Darwin) +else +HAVE_GNU_LD=1 +endif + +LIBPBE_DIR=../libpbe + +CPP_FLAGS= + +GCC_FLAGS=-pthread +#GCC_FLAGS=-D_REENTRANT + +COMPILE_FLAGS=$(CPP_FLAGS) $(GCC_FLAGS) -W -Wall ${OPTIMISE_FLAGS} ${DEBUG_FLAGS} + +CC_COMPILE_FLAGS=$(COMPILE_FLAGS) + +LINK_FLAGS=${GCC_FLAGS} ${DEBUG_FLAGS} \ + -lutil + +ifeq (${UNAME_S},OpenBSD) +LINK_FLAGS+=-liconv +endif + +ifeq (${UNAME_S},Darwin) +LINK_FLAGS+=-liconv +endif + +LIBPBE_MAKE_OPTIONS= +include ../libpbe.mk + +CC_SRCS=$(sort $(notdir $(wildcard ${SRC_DIR}/*.cc)) static_content.cc) + +BLOBFILES=anyterm.html anyterm.js anyterm.css copy.png paste.png copy.gif paste.gif + +BLOBS=$(addsuffix .blob.o,$(BLOBFILES)) + +OBJS=$(addsuffix .o,$(notdir $(basename $(CC_SRCS)))) + +%.o: %.cc + $(CXX) $(CC_COMPILE_FLAGS) -c $< + +ifdef HAVE_GNU_LD +%.blob.o: ../browser/% + cp $^ . ; $(LD) -r -b binary -o $@ $* ; rm $* + +else +%.blob.c: ../browser/% ./mk_blob + ./mk_blob $(subst .,_,$*) < $< > $@ + +mk_blob: mk_blob.c + $(CC) -o $@ $< +endif + + +anytermd: $(OBJS) $(BLOBS) $(LIBPBE_LIB) + $(CXX) -o $@ $(OBJS) $(BLOBS) $(LINK_FLAGS) + +%.d: %.cc + $(CXX) -MM -MG -MT $@ -MT $(<:%.cc=%.o) $(CPP_FLAGS) $(GCC_FLAGS) -o $@ $< + +DEPENDS=$(addsuffix .d,$(basename $(OBJS))) + +-include $(DEPENDS) + +install: FORCE + install anytermd /usr/local/bin + +clean: FORCE + $(RM) -f *.o *.blob.c static_content.cc + +veryclean: clean + $(RM) *.d + +.PHONY: default_target install FORCE + + +static_content.cc: ../scripts/mk_static_content.sh ../browser/* + PATH="$${PATH}:../scripts" ../scripts/mk_static_content.sh $(BLOBFILES) > $@ + +static_content.o: CPP_FLAGS+=-I../src + diff --git a/extras/anytermd/contrib/README b/extras/anytermd/contrib/README new file mode 100644 index 0000000000..2fe51bb3df --- /dev/null +++ b/extras/anytermd/contrib/README @@ -0,0 +1,15 @@ +This directory contains Anyterm-related files contributed by users. +If you have something that you think could be useful to others, do +please share it. Get in touch via the Anyterm forum at +http://anyterm.org/forums/. + +se-linux/* + Configuration files Security-Enhanced Linux (e.g. Fedora Core). + Last updated for Anyterm-1.1.1. + Contributed by Carl Roth. + +anyterm.spec + For building RPMs. + Last updated for Anyterm-1.1.1. + Contributed by Carl Roth. + diff --git a/extras/anytermd/contrib/anytermd b/extras/anytermd/contrib/anytermd new file mode 100755 index 0000000000..1aa8aaee24 --- /dev/null +++ b/extras/anytermd/contrib/anytermd @@ -0,0 +1,136 @@ +#!/bin/bash +# Copyright (c) 2005-2010 Artica ST +# +# Author: Sancho Lerena 2006-2010 +# +# /etc/init.d/anytermd +# +# System startup script for Pandora FMS's anyterm tool +# +# Comments to support chkconfig on RedHat Linux +# chkconfig: 2345 90 10 +# description: Pandora FMS Server anytermd startup scrip +# +# Comments to support LSB init script conventions +### BEGIN INIT INFO +# Provides: anytermd +# Required-Start: $network +# Should-Start: $network +# Required-Stop: $syslog +# Should-Stop: +# Default-Start: 2 3 5 +# Default-Stop: 0 1 6 +# Short-Description: Anytermd startup script +# Description: Anytermd Server startup script for being used with Pandora FMS +### END INIT INFO + +export PANDORA_DAEMON=/usr/bin/anytermd + +# Uses a wait limit before sending a KILL signal, before trying to stop +# Pandora FMS server nicely. Some big systems need some time before close +# all pending tasks / threads. + +export MAXWAIT=60 + +# Check for SUSE status scripts +if [ -f /etc/rc.status ] +then + . /etc/rc.status + rc_reset +else + # Define rc functions for non-suse systems, "void" functions. + function rc_status () { VOID=1; } + function rc_exit () { exit; } + function rc_failed () { VOID=1; } + +fi + +# This function replace pidof, not working in the same way in different linux distros + +function pidof_pandora () ( + # This sets COLUMNS to XXX chars, because if command is run + # in a "strech" term, ps aux don't report more than COLUMNS + # characters and this will not work. + COLUMNS=300 + PANDORA_PID=`ps aux | grep "$PANDORA_DAEMON" | grep -v grep | tail -1 | awk '{ print $2 }'` + echo $PANDORA_PID +) + +# Main script + +if [ ! -f $PANDORA_DAEMON ] +then + echo "Anytermd not found, please check setup and read manual" + rc_status -s + rc_exit +fi + +case "$1" in + start) + PANDORA_PID=`pidof_pandora` + if [ ! -z "$PANDORA_PID" ] + then + echo "Antermd is currently running on this machine with PID ($PANDORA_PID). Aborting now..." + rc_failed 1 + rc_exit + fi + + $PANDORA_DAEMON --port 8023 --user pandora -c 'telnet %p' + $PANDORA_DAEMON --port 8022 --user pandora -c 'ssh %p' + + sleep 1 + + PANDORA_PID=`pidof_pandora` + + if [ ! -z "$PANDORA_PID" ] + then + echo "Anyterm is now running with PID $PANDORA_PID" + rc_status -v + else + echo "Cannot start Anytermd. Aborted." + rc_status -s + fi + ;; + + stop) + PANDORA_PID=`pidof_pandora` + if [ -z "$PANDORA_PID" ] + then + echo "Anytermd is not running, cannot stop it." + rc_failed + else + echo "Killing anytermd processes $PANDORA_PID" + kill "$PANDORA_PID" > /dev/null 2>&1 + + # Let rid of the other process....shazam ! + PANDORA_PID=`pidof_pandora` + if [ ! -z "$PANDORA_PID" ] + then + kill "$PANDORA_PID" > /dev/null 2>&1 + fi + + rc_status -v + fi + ;; + status) + PANDORA_PID=`pidof_pandora` + if [ -z "$PANDORA_PID" ] + then + echo "Anytermd is not running." + rc_status + else + echo "Anytermd is running with PID $PANDORA_PID." + rc_status + fi + ;; + force-reload|restart) + $0 stop + $0 start + ;; + *) + echo "Usage: anytermd { start | stop | restart | status }" + exit 1 +esac +rc_exit + + diff --git a/extras/anytermd/libpbe.mk b/extras/anytermd/libpbe.mk new file mode 100644 index 0000000000..1e9e16e1be --- /dev/null +++ b/extras/anytermd/libpbe.mk @@ -0,0 +1,44 @@ +# libpbe.mk +# This file is part of Anyterm; see http://anyterm.org/ +# (C) 2006-2009 Philip Endecott + +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +UNAME_S=$(shell uname -s) +UNAME_R=$(shell uname -r) + +ifeq (${UNAME_S},Linux) + KVER=$(subst ., ,${UNAME_R}) + ifeq ($(word 1,${KVER}),2) + ifeq ($(word 2,${KVER}),4) + SUPPORT_LINUX_2_4=1 + endif + endif +endif + +CPP_FLAGS+=-I$(LIBPBE_DIR)/include + +ifdef SUPPORT_LINUX_2_4 +CPP_FLAGS+=-DSUPPORT_LINUX_2_4 +endif + +LINK_FLAGS+=-L$(LIBPBE_DIR) -lpbe + +LIBPBE_LIB=$(LIBPBE_DIR)/libpbe.a + +$(LIBPBE_LIB): FORCE + cd $(LIBPBE_DIR); $(MAKE) $(LIBPBE_MAKE_OPTIONS) + diff --git a/extras/anytermd/libpbe/BOOST_LICENSE b/extras/anytermd/libpbe/BOOST_LICENSE new file mode 100644 index 0000000000..ffc4a78b69 --- /dev/null +++ b/extras/anytermd/libpbe/BOOST_LICENSE @@ -0,0 +1,28 @@ +Certain files in this library are distributed under the terms of the Boost +Software License, shown below, but most are not. Please refer to each file's +header for details of the license that applies. + + +Boost Software License - Version 1.0 - August 17th, 2003 + +Permission is hereby granted, free of charge, to any person or organization +obtaining a copy of the software and accompanying documentation covered by +this license (the "Software") to use, reproduce, display, distribute, +execute, and transmit the Software, and to prepare derivative works of the +Software, and to permit third-parties to whom the Software is furnished to +do so, all subject to the following: + +The copyright notices in the Software and this entire statement, including +the above license grant, this restriction and the following disclaimer, +must be included in all copies of the Software, in whole or in part, and +all derivative works of the Software, unless such copies or derivative +works are solely in the form of machine-executable object code generated by +a source language processor. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT +SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE +FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, +ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER +DEALINGS IN THE SOFTWARE. diff --git a/extras/anytermd/libpbe/LICENSE b/extras/anytermd/libpbe/LICENSE new file mode 100644 index 0000000000..d60c31a97a --- /dev/null +++ b/extras/anytermd/libpbe/LICENSE @@ -0,0 +1,340 @@ + GNU GENERAL PUBLIC LICENSE + Version 2, June 1991 + + Copyright (C) 1989, 1991 Free Software Foundation, Inc. + 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The licenses for most software are designed to take away your +freedom to share and change it. By contrast, the GNU General Public +License is intended to guarantee your freedom to share and change free +software--to make sure the software is free for all its users. This +General Public License applies to most of the Free Software +Foundation's software and to any other program whose authors commit to +using it. (Some other Free Software Foundation software is covered by +the GNU Library General Public License instead.) You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +this service if you wish), that you receive source code or can get it +if you want it, that you can change the software or use pieces of it +in new free programs; and that you know you can do these things. + + To protect your rights, we need to make restrictions that forbid +anyone to deny you these rights or to ask you to surrender the rights. +These restrictions translate to certain responsibilities for you if you +distribute copies of the software, or if you modify it. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must give the recipients all the rights that +you have. You must make sure that they, too, receive or can get the +source code. And you must show them these terms so they know their +rights. + + We protect your rights with two steps: (1) copyright the software, and +(2) offer you this license which gives you legal permission to copy, +distribute and/or modify the software. + + Also, for each author's protection and ours, we want to make certain +that everyone understands that there is no warranty for this free +software. If the software is modified by someone else and passed on, we +want its recipients to know that what they have is not the original, so +that any problems introduced by others will not reflect on the original +authors' reputations. + + Finally, any free program is threatened constantly by software +patents. We wish to avoid the danger that redistributors of a free +program will individually obtain patent licenses, in effect making the +program proprietary. To prevent this, we have made it clear that any +patent must be licensed for everyone's free use or not licensed at all. + + The precise terms and conditions for copying, distribution and +modification follow. + + GNU GENERAL PUBLIC LICENSE + TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + + 0. This License applies to any program or other work which contains +a notice placed by the copyright holder saying it may be distributed +under the terms of this General Public License. The "Program", below, +refers to any such program or work, and a "work based on the Program" +means either the Program or any derivative work under copyright law: +that is to say, a work containing the Program or a portion of it, +either verbatim or with modifications and/or translated into another +language. (Hereinafter, translation is included without limitation in +the term "modification".) Each licensee is addressed as "you". + +Activities other than copying, distribution and modification are not +covered by this License; they are outside its scope. The act of +running the Program is not restricted, and the output from the Program +is covered only if its contents constitute a work based on the +Program (independent of having been made by running the Program). +Whether that is true depends on what the Program does. + + 1. You may copy and distribute verbatim copies of the Program's +source code as you receive it, in any medium, provided that you +conspicuously and appropriately publish on each copy an appropriate +copyright notice and disclaimer of warranty; keep intact all the +notices that refer to this License and to the absence of any warranty; +and give any other recipients of the Program a copy of this License +along with the Program. + +You may charge a fee for the physical act of transferring a copy, and +you may at your option offer warranty protection in exchange for a fee. + + 2. You may modify your copy or copies of the Program or any portion +of it, thus forming a work based on the Program, and copy and +distribute such modifications or work under the terms of Section 1 +above, provided that you also meet all of these conditions: + + a) You must cause the modified files to carry prominent notices + stating that you changed the files and the date of any change. + + b) You must cause any work that you distribute or publish, that in + whole or in part contains or is derived from the Program or any + part thereof, to be licensed as a whole at no charge to all third + parties under the terms of this License. + + c) If the modified program normally reads commands interactively + when run, you must cause it, when started running for such + interactive use in the most ordinary way, to print or display an + announcement including an appropriate copyright notice and a + notice that there is no warranty (or else, saying that you provide + a warranty) and that users may redistribute the program under + these conditions, and telling the user how to view a copy of this + License. (Exception: if the Program itself is interactive but + does not normally print such an announcement, your work based on + the Program is not required to print an announcement.) + +These requirements apply to the modified work as a whole. If +identifiable sections of that work are not derived from the Program, +and can be reasonably considered independent and separate works in +themselves, then this License, and its terms, do not apply to those +sections when you distribute them as separate works. But when you +distribute the same sections as part of a whole which is a work based +on the Program, the distribution of the whole must be on the terms of +this License, whose permissions for other licensees extend to the +entire whole, and thus to each and every part regardless of who wrote it. + +Thus, it is not the intent of this section to claim rights or contest +your rights to work written entirely by you; rather, the intent is to +exercise the right to control the distribution of derivative or +collective works based on the Program. + +In addition, mere aggregation of another work not based on the Program +with the Program (or with a work based on the Program) on a volume of +a storage or distribution medium does not bring the other work under +the scope of this License. + + 3. You may copy and distribute the Program (or a work based on it, +under Section 2) in object code or executable form under the terms of +Sections 1 and 2 above provided that you also do one of the following: + + a) Accompany it with the complete corresponding machine-readable + source code, which must be distributed under the terms of Sections + 1 and 2 above on a medium customarily used for software interchange; or, + + b) Accompany it with a written offer, valid for at least three + years, to give any third party, for a charge no more than your + cost of physically performing source distribution, a complete + machine-readable copy of the corresponding source code, to be + distributed under the terms of Sections 1 and 2 above on a medium + customarily used for software interchange; or, + + c) Accompany it with the information you received as to the offer + to distribute corresponding source code. (This alternative is + allowed only for noncommercial distribution and only if you + received the program in object code or executable form with such + an offer, in accord with Subsection b above.) + +The source code for a work means the preferred form of the work for +making modifications to it. For an executable work, complete source +code means all the source code for all modules it contains, plus any +associated interface definition files, plus the scripts used to +control compilation and installation of the executable. However, as a +special exception, the source code distributed need not include +anything that is normally distributed (in either source or binary +form) with the major components (compiler, kernel, and so on) of the +operating system on which the executable runs, unless that component +itself accompanies the executable. + +If distribution of executable or object code is made by offering +access to copy from a designated place, then offering equivalent +access to copy the source code from the same place counts as +distribution of the source code, even though third parties are not +compelled to copy the source along with the object code. + + 4. You may not copy, modify, sublicense, or distribute the Program +except as expressly provided under this License. Any attempt +otherwise to copy, modify, sublicense or distribute the Program is +void, and will automatically terminate your rights under this License. +However, parties who have received copies, or rights, from you under +this License will not have their licenses terminated so long as such +parties remain in full compliance. + + 5. You are not required to accept this License, since you have not +signed it. However, nothing else grants you permission to modify or +distribute the Program or its derivative works. These actions are +prohibited by law if you do not accept this License. Therefore, by +modifying or distributing the Program (or any work based on the +Program), you indicate your acceptance of this License to do so, and +all its terms and conditions for copying, distributing or modifying +the Program or works based on it. + + 6. Each time you redistribute the Program (or any work based on the +Program), the recipient automatically receives a license from the +original licensor to copy, distribute or modify the Program subject to +these terms and conditions. You may not impose any further +restrictions on the recipients' exercise of the rights granted herein. +You are not responsible for enforcing compliance by third parties to +this License. + + 7. If, as a consequence of a court judgment or allegation of patent +infringement or for any other reason (not limited to patent issues), +conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot +distribute so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you +may not distribute the Program at all. For example, if a patent +license would not permit royalty-free redistribution of the Program by +all those who receive copies directly or indirectly through you, then +the only way you could satisfy both it and this License would be to +refrain entirely from distribution of the Program. + +If any portion of this section is held invalid or unenforceable under +any particular circumstance, the balance of the section is intended to +apply and the section as a whole is intended to apply in other +circumstances. + +It is not the purpose of this section to induce you to infringe any +patents or other property right claims or to contest validity of any +such claims; this section has the sole purpose of protecting the +integrity of the free software distribution system, which is +implemented by public license practices. Many people have made +generous contributions to the wide range of software distributed +through that system in reliance on consistent application of that +system; it is up to the author/donor to decide if he or she is willing +to distribute software through any other system and a licensee cannot +impose that choice. + +This section is intended to make thoroughly clear what is believed to +be a consequence of the rest of this License. + + 8. If the distribution and/or use of the Program is restricted in +certain countries either by patents or by copyrighted interfaces, the +original copyright holder who places the Program under this License +may add an explicit geographical distribution limitation excluding +those countries, so that distribution is permitted only in or among +countries not thus excluded. In such case, this License incorporates +the limitation as if written in the body of this License. + + 9. The Free Software Foundation may publish revised and/or new versions +of the General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + +Each version is given a distinguishing version number. If the Program +specifies a version number of this License which applies to it and "any +later version", you have the option of following the terms and conditions +either of that version or of any later version published by the Free +Software Foundation. If the Program does not specify a version number of +this License, you may choose any version ever published by the Free Software +Foundation. + + 10. If you wish to incorporate parts of the Program into other free +programs whose distribution conditions are different, write to the author +to ask for permission. For software which is copyrighted by the Free +Software Foundation, write to the Free Software Foundation; we sometimes +make exceptions for this. Our decision will be guided by the two goals +of preserving the free status of all derivatives of our free software and +of promoting the sharing and reuse of software generally. + + NO WARRANTY + + 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY +FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN +OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES +PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED +OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS +TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE +PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, +REPAIR OR CORRECTION. + + 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR +REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, +INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING +OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED +TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY +YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER +PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE +POSSIBILITY OF SUCH DAMAGES. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +convey the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 2 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + + +Also add information on how to contact you by electronic and paper mail. + +If the program is interactive, make it output a short notice like this +when it starts in an interactive mode: + + Gnomovision version 69, Copyright (C) year name of author + Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, the commands you use may +be called something other than `show w' and `show c'; they could even be +mouse-clicks or menu items--whatever suits your program. + +You should also get your employer (if you work as a programmer) or your +school, if any, to sign a "copyright disclaimer" for the program, if +necessary. Here is a sample; alter the names: + + Yoyodyne, Inc., hereby disclaims all copyright interest in the program + `Gnomovision' (which makes passes at compilers) written by James Hacker. + + , 1 April 1989 + Ty Coon, President of Vice + +This General Public License does not permit incorporating your program into +proprietary programs. If your program is a subroutine library, you may +consider it more useful to permit linking proprietary applications with the +library. If this is what you want to do, use the GNU Library General +Public License instead of this License. diff --git a/extras/anytermd/libpbe/Makefile b/extras/anytermd/libpbe/Makefile new file mode 100644 index 0000000000..9dd11e3627 --- /dev/null +++ b/extras/anytermd/libpbe/Makefile @@ -0,0 +1,46 @@ +# Makefile +# This file is part of libpbe; see http://decimail.org +# (C) 2005-2007 Philip Endecott + +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +default_target: library + +BUILD_DIR=build + +LIBRARY=libpbe.a + +library: $(LIBRARY) + +$(LIBRARY): FORCE + cd ${BUILD_DIR}; ${MAKE} ${LIBRARY} + cp ${BUILD_DIR}/${LIBRARY} ${LIBRARY} + +libpbe%.a: FORCE + ${MAKE} -C build$* libpbe.a + cp build$*/libpbe.a $@ + +clean: FORCE + cd ${BUILD_DIR}; $(MAKE) clean + rm $(LIBRARY) + +veryclean: FORCE + cd ${BUILD_DIR}; $(MAKE) veryclean + rm $(LIBRARY) + +.PHONY: clean veryclean library default_target + +FORCE: + diff --git a/extras/anytermd/libpbe/README b/extras/anytermd/libpbe/README new file mode 100644 index 0000000000..6242b0699b --- /dev/null +++ b/extras/anytermd/libpbe/README @@ -0,0 +1,8 @@ +This is libpbe, a personal library of C++ utilities. + +See the examples directory for some example code using these functions +(not all examples are up to date.) + +For more information, see + http://decimail.org/ + http://anyterm.org/ diff --git a/extras/anytermd/libpbe/build.iphonesim/Makefile b/extras/anytermd/libpbe/build.iphonesim/Makefile new file mode 100644 index 0000000000..4ae32c8775 --- /dev/null +++ b/extras/anytermd/libpbe/build.iphonesim/Makefile @@ -0,0 +1,8 @@ +platdir=/Developer/Platforms/iPhoneSimulator.platform +CXX=${platdir}/Developer/usr/bin/g++ +AR=${platdir}/Developer/usr/bin/ar +OPTIMISE_FLAGS=-O +DEBUG_FLAGS= + +include ../common.mk + diff --git a/extras/anytermd/libpbe/build/CgiParams.d b/extras/anytermd/libpbe/build/CgiParams.d new file mode 100644 index 0000000000..be3dfbfc32 --- /dev/null +++ b/extras/anytermd/libpbe/build/CgiParams.d @@ -0,0 +1,4 @@ +CgiParams.d CgiParams.o: ../src/CgiParams.cc ../include/CgiParams.hh \ + ../include/StringSplitter.hh ../include/ci_string.hh \ + ../include/Exception.hh ../include/utils.hh \ + ../include/StringTransformer.hh diff --git a/extras/anytermd/libpbe/build/CgiVars.d b/extras/anytermd/libpbe/build/CgiVars.d new file mode 100644 index 0000000000..a9d0d4b104 --- /dev/null +++ b/extras/anytermd/libpbe/build/CgiVars.d @@ -0,0 +1 @@ +CgiVars.d CgiVars.o: ../src/CgiVars.cc ../include/CgiVars.hh diff --git a/extras/anytermd/libpbe/build/Daemon.d b/extras/anytermd/libpbe/build/Daemon.d new file mode 100644 index 0000000000..954a5b83c6 --- /dev/null +++ b/extras/anytermd/libpbe/build/Daemon.d @@ -0,0 +1,7 @@ +Daemon.d Daemon.o: ../src/Daemon.cc ../include/Daemon.hh \ + ../include/FileDescriptor.hh ../include/Exception.hh \ + ../include/missing_syscalls.hh ../include/compiler_magic.hh \ + ../include/Mutex.hh ../include/atomic.hh ../include/Lock.hh \ + ../include/Futex.hh ../include/Condition.hh ../include/HPtime.hh \ + ../include/Unlock.hh ../include/Exception.hh ../include/Lock.hh \ + ../include/Thread.hh diff --git a/extras/anytermd/libpbe/build/Date.d b/extras/anytermd/libpbe/build/Date.d new file mode 100644 index 0000000000..5f0c5c6d6a --- /dev/null +++ b/extras/anytermd/libpbe/build/Date.d @@ -0,0 +1,2 @@ +Date.d Date.o: ../src/Date.cc ../include/Date.hh ../include/DateTime.hh \ + ../include/Date.hh ../include/Time.hh ../include/Exception.hh diff --git a/extras/anytermd/libpbe/build/Directory.d b/extras/anytermd/libpbe/build/Directory.d new file mode 100644 index 0000000000..ae6e34aba2 --- /dev/null +++ b/extras/anytermd/libpbe/build/Directory.d @@ -0,0 +1,2 @@ +Directory.d Directory.o: ../src/Directory.cc ../include/Directory.hh \ + ../include/Exception.hh diff --git a/extras/anytermd/libpbe/build/Exception.d b/extras/anytermd/libpbe/build/Exception.d new file mode 100644 index 0000000000..c93e3fa7ac --- /dev/null +++ b/extras/anytermd/libpbe/build/Exception.d @@ -0,0 +1 @@ +Exception.d Exception.o: ../src/Exception.cc ../include/Exception.hh diff --git a/extras/anytermd/libpbe/build/HttpAuthenticator.d b/extras/anytermd/libpbe/build/HttpAuthenticator.d new file mode 100644 index 0000000000..69138d133c --- /dev/null +++ b/extras/anytermd/libpbe/build/HttpAuthenticator.d @@ -0,0 +1,3 @@ +HttpAuthenticator.d HttpAuthenticator.o: ../src/HttpAuthenticator.cc \ + ../include/HttpAuthenticator.hh ../include/base64.hh \ + ../include/init_array.hh ../include/compiler_magic.hh diff --git a/extras/anytermd/libpbe/build/HttpClient.d b/extras/anytermd/libpbe/build/HttpClient.d new file mode 100644 index 0000000000..981374c17e --- /dev/null +++ b/extras/anytermd/libpbe/build/HttpClient.d @@ -0,0 +1,8 @@ +HttpClient.d HttpClient.o: ../src/HttpClient.cc ../include/HttpClient.hh \ + ../include/URI.hh ../include/HttpResponse.hh \ + ../include/FileDescriptor.hh ../include/Exception.hh \ + ../include/missing_syscalls.hh ../include/compiler_magic.hh \ + ../include/HttpRequest.hh ../include/TcpClientSocket.hh ../include/ip.hh \ + ../include/Gunzipper.hh ../include/Bunzipper.hh \ + ../include/FileDescriptor.hh ../include/rfcdate.hh \ + ../include/FileType.hh ../include/atomic_ofstream.hh diff --git a/extras/anytermd/libpbe/build/HttpDaemon.d b/extras/anytermd/libpbe/build/HttpDaemon.d new file mode 100644 index 0000000000..39bf73e674 --- /dev/null +++ b/extras/anytermd/libpbe/build/HttpDaemon.d @@ -0,0 +1,10 @@ +HttpDaemon.d HttpDaemon.o: ../src/HttpDaemon.cc ../include/HttpDaemon.hh \ + ../include/Daemon.hh ../include/FileDescriptor.hh \ + ../include/Exception.hh ../include/missing_syscalls.hh \ + ../include/compiler_magic.hh ../include/Mutex.hh ../include/atomic.hh \ + ../include/Lock.hh ../include/Futex.hh ../include/Condition.hh \ + ../include/HPtime.hh ../include/Unlock.hh ../include/HttpRequest.hh \ + ../include/URI.hh ../include/HttpResponse.hh \ + ../include/HttpAuthenticator.hh ../include/HttpRequest.hh \ + ../include/parse_http_request.hh ../include/HttpResponse.hh \ + ../include/rfcdate.hh diff --git a/extras/anytermd/libpbe/build/HttpResponse.d b/extras/anytermd/libpbe/build/HttpResponse.d new file mode 100644 index 0000000000..2068bc9489 --- /dev/null +++ b/extras/anytermd/libpbe/build/HttpResponse.d @@ -0,0 +1,5 @@ +HttpResponse.d HttpResponse.o: ../src/HttpResponse.cc \ + ../include/HttpResponse.hh ../include/FileDescriptor.hh \ + ../include/Exception.hh ../include/missing_syscalls.hh \ + ../include/compiler_magic.hh ../include/FileDescriptor.hh \ + ../include/mustfind.hh diff --git a/extras/anytermd/libpbe/build/Makefile b/extras/anytermd/libpbe/build/Makefile new file mode 100644 index 0000000000..23f8a8fa29 --- /dev/null +++ b/extras/anytermd/libpbe/build/Makefile @@ -0,0 +1 @@ +include ../common.mk diff --git a/extras/anytermd/libpbe/build/SmtpClient.d b/extras/anytermd/libpbe/build/SmtpClient.d new file mode 100644 index 0000000000..23ccf65121 --- /dev/null +++ b/extras/anytermd/libpbe/build/SmtpClient.d @@ -0,0 +1 @@ +SmtpClient.d SmtpClient.o: ../src/SmtpClient.cc diff --git a/extras/anytermd/libpbe/build/URI.d b/extras/anytermd/libpbe/build/URI.d new file mode 100644 index 0000000000..62ff59c79c --- /dev/null +++ b/extras/anytermd/libpbe/build/URI.d @@ -0,0 +1 @@ +URI.d URI.o: ../src/URI.cc ../include/URI.hh diff --git a/extras/anytermd/libpbe/build/charset_names.d b/extras/anytermd/libpbe/build/charset_names.d new file mode 100644 index 0000000000..63dec8eca4 --- /dev/null +++ b/extras/anytermd/libpbe/build/charset_names.d @@ -0,0 +1,6 @@ +charset_names.d charset_names.o: ../src/charset_names.cc \ + ../include/charset/charset_names.hh ../include/charset/charset_t.hh \ + ../include/charset/generated/charset_t_enum_body.hh \ + ../src/generated/charset_iana_names_body.cc \ + ../src/generated/charset_mime_names_body.cc \ + ../src/generated/charset_lookup_body.cc diff --git a/extras/anytermd/libpbe/build/ci_string.d b/extras/anytermd/libpbe/build/ci_string.d new file mode 100644 index 0000000000..a136da789d --- /dev/null +++ b/extras/anytermd/libpbe/build/ci_string.d @@ -0,0 +1 @@ +ci_string.d ci_string.o: ../src/ci_string.cc ../include/ci_string.hh diff --git a/extras/anytermd/libpbe/build/create_dir.d b/extras/anytermd/libpbe/build/create_dir.d new file mode 100644 index 0000000000..3b9010513d --- /dev/null +++ b/extras/anytermd/libpbe/build/create_dir.d @@ -0,0 +1,2 @@ +create_dir.d create_dir.o: ../src/create_dir.cc ../include/create_dir.hh \ + ../include/Exception.hh ../include/FileType.hh diff --git a/extras/anytermd/libpbe/build/csv.d b/extras/anytermd/libpbe/build/csv.d new file mode 100644 index 0000000000..0788bb8456 --- /dev/null +++ b/extras/anytermd/libpbe/build/csv.d @@ -0,0 +1 @@ +csv.d csv.o: ../src/csv.cc ../include/csv.hh diff --git a/extras/anytermd/libpbe/build/format.d b/extras/anytermd/libpbe/build/format.d new file mode 100644 index 0000000000..32aa4af398 --- /dev/null +++ b/extras/anytermd/libpbe/build/format.d @@ -0,0 +1 @@ +format.d format.o: ../src/format.cc ../include/format.hh diff --git a/extras/anytermd/libpbe/build/geometry.d b/extras/anytermd/libpbe/build/geometry.d new file mode 100644 index 0000000000..3fb2ab3db4 --- /dev/null +++ b/extras/anytermd/libpbe/build/geometry.d @@ -0,0 +1 @@ +geometry.d geometry.o: ../src/geometry.cc ../include/geometry.hh diff --git a/extras/anytermd/libpbe/build/ip.d b/extras/anytermd/libpbe/build/ip.d new file mode 100644 index 0000000000..4b5f9c61da --- /dev/null +++ b/extras/anytermd/libpbe/build/ip.d @@ -0,0 +1 @@ +ip.d ip.o: ../src/ip.cc ../include/ip.hh ../include/Exception.hh diff --git a/extras/anytermd/libpbe/build/iso8859_tables.d b/extras/anytermd/libpbe/build/iso8859_tables.d new file mode 100644 index 0000000000..77ec8166ff --- /dev/null +++ b/extras/anytermd/libpbe/build/iso8859_tables.d @@ -0,0 +1,8 @@ +iso8859_tables.d iso8859_tables.o: ../src/iso8859_tables.cc \ + ../include/charset/conv/iso8859.hh ../include/charset/charset_t.hh \ + ../include/charset/generated/charset_t_enum_body.hh \ + ../include/charset/char_t.hh ../include/charset/charset_traits.hh \ + ../include/charset/charset_t.hh ../include/charset/char_t.hh \ + ../include/charset/char_conv.hh ../include/charset/charset_traits.hh \ + ../include/charset/error_policy.hh ../include/compiler_magic.hh \ + ../src/generated/iso8859_tables_body.cc diff --git a/extras/anytermd/libpbe/build/markup.d b/extras/anytermd/libpbe/build/markup.d new file mode 100644 index 0000000000..c8b5f9ebd5 --- /dev/null +++ b/extras/anytermd/libpbe/build/markup.d @@ -0,0 +1,2 @@ +markup.d markup.o: ../src/markup.cc ../include/markup.hh \ + ../include/StringTransformer.hh diff --git a/extras/anytermd/libpbe/build/parse_http_request.d b/extras/anytermd/libpbe/build/parse_http_request.d new file mode 100644 index 0000000000..cf90f7348b --- /dev/null +++ b/extras/anytermd/libpbe/build/parse_http_request.d @@ -0,0 +1,3 @@ +parse_http_request.d parse_http_request.o: ../src/parse_http_request.cc \ + ../include/parse_http_request.hh ../include/HttpRequest.hh \ + ../include/URI.hh ../include/Exception.hh diff --git a/extras/anytermd/libpbe/build/rfcdate.d b/extras/anytermd/libpbe/build/rfcdate.d new file mode 100644 index 0000000000..2cbaa98d95 --- /dev/null +++ b/extras/anytermd/libpbe/build/rfcdate.d @@ -0,0 +1 @@ +rfcdate.d rfcdate.o: ../src/rfcdate.cc diff --git a/extras/anytermd/libpbe/build/run_cmd.d b/extras/anytermd/libpbe/build/run_cmd.d new file mode 100644 index 0000000000..441288312e --- /dev/null +++ b/extras/anytermd/libpbe/build/run_cmd.d @@ -0,0 +1,2 @@ +run_cmd.d run_cmd.o: ../src/run_cmd.cc ../include/run_cmd.hh \ + ../include/Exception.hh diff --git a/extras/anytermd/libpbe/build/segv_backtrace.d b/extras/anytermd/libpbe/build/segv_backtrace.d new file mode 100644 index 0000000000..e8226869fd --- /dev/null +++ b/extras/anytermd/libpbe/build/segv_backtrace.d @@ -0,0 +1,2 @@ +segv_backtrace.d segv_backtrace.o: ../src/segv_backtrace.cc \ + ../include/segv_backtrace.hh ../include/compiler_magic.hh diff --git a/extras/anytermd/libpbe/build/select.d b/extras/anytermd/libpbe/build/select.d new file mode 100644 index 0000000000..45b5633df5 --- /dev/null +++ b/extras/anytermd/libpbe/build/select.d @@ -0,0 +1,2 @@ +select.d select.o: ../src/select.cc ../include/select.hh \ + ../include/Exception.hh diff --git a/extras/anytermd/libpbe/build/to_ascii_letters.d b/extras/anytermd/libpbe/build/to_ascii_letters.d new file mode 100644 index 0000000000..5e8ffff3e9 --- /dev/null +++ b/extras/anytermd/libpbe/build/to_ascii_letters.d @@ -0,0 +1,3 @@ +to_ascii_letters.d to_ascii_letters.o: ../src/to_ascii_letters.cc \ + ../include/charset/to_ascii_letters.hh ../include/charset/char_t.hh \ + ../src/generated/to_ascii_letters_tables.cc diff --git a/extras/anytermd/libpbe/build/utils.d b/extras/anytermd/libpbe/build/utils.d new file mode 100644 index 0000000000..6896b47e95 --- /dev/null +++ b/extras/anytermd/libpbe/build/utils.d @@ -0,0 +1,2 @@ +utils.d utils.o: ../src/utils.cc ../include/utils.hh \ + ../include/StringTransformer.hh ../include/Exception.hh diff --git a/extras/anytermd/libpbe/build_arm/Makefile b/extras/anytermd/libpbe/build_arm/Makefile new file mode 100644 index 0000000000..23f8a8fa29 --- /dev/null +++ b/extras/anytermd/libpbe/build_arm/Makefile @@ -0,0 +1 @@ +include ../common.mk diff --git a/extras/anytermd/libpbe/common.mk b/extras/anytermd/libpbe/common.mk new file mode 100644 index 0000000000..8baebd2010 --- /dev/null +++ b/extras/anytermd/libpbe/common.mk @@ -0,0 +1,112 @@ +# src/Makefile +# This file is part of libpbe; see http://decimail.org +# (C) 2004-2007 Philip Endecott + +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +default_target: library + +LIBRARY=libpbe.a + +library: $(LIBRARY) + +UNAME_S=$(shell uname -s) +UNAME_R=$(shell uname -r) + +ifeq (${UNAME_S},Linux) + KVER=$(subst ., ,${UNAME_R}) + ifeq ($(word 1,${KVER}),2) + ifeq ($(word 2,${KVER}),4) + SUPPORT_LINUX_2_4=1 + endif + endif +endif + +SRC_DIR=../src +INCLUDE_DIR=../include + +VPATH=${SRC_DIR} + +SKIP_SRCS:= + +ifndef ENABLE_POSTGRESQL + SKIP_SRCS+=Database.cc +endif + +ifndef ENABLE_IMAGEMAGICK + SKIP_SRCS+=jpegsize.cc +endif + +ifndef ENABLE_RECODE + SKIP_SRCS+=Recoder.cc +endif + +ifdef DISABLE_HTTP_CLIENT + SKIP_SRCS+=HttpClient.cc +endif + +CC_SRCS=$(filter-out ${SKIP_SRCS},$(notdir $(wildcard ${SRC_DIR}/*.cc))) + +CC_SRCS+= + +OBJS=$(addsuffix .o,$(notdir $(basename $(CC_SRCS)))) + +DEPENDS=$(addsuffix .d,$(notdir $(basename $(CC_SRCS)))) + +-include $(DEPENDS) + +WARN_FLAGS=-W -Wall + +OPTIMISE_FLAGS=-O + +DEBUG_FLAGS= + +ifdef ENABLE_POSTGRESQL + PG_INC_FLAGS=-I$(shell pg_config --includedir) +else + PG_INC_FLAGS= +endif + +INC_FLAGS+=$(PG_INC_FLAGS) -I${INCLUDE_DIR} + +COMPILE_FLAGS=$(WARN_FLAGS) $(OPTIMISE_FLAGS) $(DEBUG_FLAGS) $(INC_FLAGS) -pthread -fPIC + +ifdef SUPPORT_LINUX_2_4 + COMPILE_FLAGS+=-DSUPPORT_LINUX_2_4 +endif + +$(LIBRARY): $(OBJS) + $(AR) ruv $(LIBRARY) $(OBJS) + +%.o: %.cc + $(CXX) $(CXXFLAGS) $(COMPILE_FLAGS) -c $< + +%.d: %.cc + $(CXX) $(CXXFLAGS) -pthread -MM -MT $@ -MT $(notdir $(<:%.cc=%.o)) $(INC_FLAGS) -o $@ $< + +all: $(EXECUTABLE) + +clean: FORCE + $(RM) *.o $(LIBRARY) + +veryclean: clean FORCE + $(RM) *.d + +FORCE: + + +testing: + echo ${CC_SRCS} + diff --git a/extras/anytermd/libpbe/doc/Database b/extras/anytermd/libpbe/doc/Database new file mode 100644 index 0000000000..01021819eb --- /dev/null +++ b/extras/anytermd/libpbe/doc/Database @@ -0,0 +1,228 @@ +pbe::Database +============= + +pbe::Database is a C++ SQL database API. It is implemented as a wrapper around +PostgreSQL's libpq C client library. Its objective is: + +- To support a "modern" C++ style with a concise convenient syntax. +- To avoid the need to construct SQL queries in strings as far as possible. +- To perform as much compile-time type checking as possible. +- To have minimal overhead compared to raw libpq. + +The source code is somewhat commented, and you should consider the comments in +the source to be more more definitive, comprehensive and up-to-date than this +file. + + +Usage +----- + +Here's a very quick summary: + + #include + using namespace pbe; + ... + Database db("dbname=foo, username=blah"); + Query insert_thing(db,"insert into things(name,num) values ($1,$2)"); + SingletonQuery count_things(db,"select sum(num) from things where name=$1"); + Transaction t(db); + insert_thing("table",1); + insert_thing("chair",4); + int n = count_things("bed"); + t.commit(); + +Now some more details: + + +* 1. Connect to the database server. + +The pbe::Database object represents a connection to the database server. Its +constructor takes a string which defines the connection parameters in the +standard PostgreSQL format. See the PostgreSQL documentation for details. + + +* 2. Declare queries. + +Queries are declared before use. The query constructor takes a reference to the +database connection and the query's SQL string. Template parameters specify the +C++ types of the parameters. For example, in this example: + + Query insert_thing(db,"insert into things(name,num) values ($1,$2)"); + +PostgreSQL is given a query string containing two placeholders for parameters, $1 +and $2. Their C++ types are string and int respectively. Compile-time type +checking ensures that when the query is used the C++ parameters have (or are +convertible to) these types. A mapping is built in from C++ types to PostgreSQL +types. At present the mapping is as follows. It is fairly straightforward to +add more types. + + C++ Type PostgreSQL type + -------- --------------- + std::string TEXT + int INT4 + time_t TIMESTAMP WITH TIME ZONE + float FLOAT4 + +The geneeral type 'Query', as shown above, should be used for +- Queries that return multi-row, multi-column data. +- Queries that reutrn no data (e.g. update, delete, create table etc.) + +The following variants are available for other cases. They have the benefit of +additional type checking and the like. + +ColumnQuery +- Use this for a query that returns a single column of results, and give the C++ +type for the result as the first template parameter. + +SingletonQuery +- Use this for a query that returns a single value (one row, one column), e.g. +"count(*)" or "select sum(...)". The type of the result value is given as the +first template parameter. + +OptQuery +- Use this fora query that returns either a single value as above or no results +at all. Once again the result value is given as the first template parameter. + +One option for query declaration is to define a subclass of pbe::Database that +contains them. The existence of the SQL database can then be hidden from the use +of the class: + + class ThingList: public Database { + public: + Query add_thing; + SingletonQuery count_things; + Query delete_thing; + + ThingList(): + Database("dbname=things"), + add_thing(*this, "insert into things(name,num) values ($1,$2)"), + count_things(*this, "select sum(num) from things where name=$1"), + delete_thing(*this, "delete from things where name=$1") + {} + }; + + ... + ThingList tl; + tl.add_thing("pen",25); + tl.add_thing("pencil",50); + tl.delete_thing("desk"); + int num_lamps = tl.count_things("lamp"); + + +* 3. Use a transaction. + +To group a series of queries in a transaction, declare a Transaction object. +This takes a reference to the Database object as a constructor parameter. A +BEGIN statement is executed when the Transaction object is constructed. After +running your queries, call the Transaction's commit() method to execute a +COMMIT statement. + +If something goes wrong and an exception is thrown, the Transaction object's +destructor is called as it goes out of scope. At this point a ROLLBACK statement +is executed. + +It is probably best to used additional { } to bind the Transaction tightly to the +queries it is protecting, i.e. + + ... + { + Transaction t(db); + insert_thing(...); + ...more queries... + t.commit(); + } + ... + + +* 4. Run the queries. + +The Query objects are functors, i.e. they are objects that you can use as +functions. (This is achieved by defining operator().) For example: + + insert_thing("table",1); + +The parameters are passed to PostgreSQL using libpq's binary parameter format. + +The query strings are passed to PostgreSQL the first time that they are used, +where they are stored as "prepared statements" for further uses. The prepared +statements are freed ("deallocated") when the Query object goes out of scope. + +If for some reason you don't want to use prepared statements then you can use the +runonce() method: + + insert_thing.runonce("table",1); + +This should have exactly the same behaviour as the normal case. + +(Note that although PostgreSQL only supports prepared statements and binary +parameters for SELECT, UPDATE, INSERT and DELETE statements, you don't need to +worry as this library detects other statements and treats them specially.) + + +* 5. Get the results. + +The result of a general Query can be assigned to a Result object: + + Query q(db, "select a,b from t where c=$1"); + Result r = q(99); + +The size of the result is available in the rows and cols members. Individual +elements can be accessed using the get() method, which takes a row number and +either a column number or a column name, and the expected type as a template +parameter. Type checking is performed; if you want to avoid this small overhead, +use get_nocheck(). There is also an is_null method: + + string s = r.get(0,0); + int a = r.get(1,"b"); + if (!r.is_null(2,0)) { + ... + } + +The result of a ColumnQuery is a ColumnResult. ColumnResult takes the result +type as a template parameter, and you access elements using (): + + ColumnQuery q(db, "select name from t"); + ColumnResult r = q(); + string s = r(1); + +It is also a standard-library-compatible random-access container: + + typedef ColumnResult col_t; + col_t r = q(); + for ( col_t::const_iterator i = r.begin(); + i != r.end(); ++i ) { + cout << *i << "\n"; + } + +SingletonResult is used for SingletonQueries. It is convertible to the result +type itself, so you don't normally need to declare it; the exception would be if +you needed tu use is_null() first. + + SingletonQuery count_things(db,"select sum(num) from things where name=$1"); + int n = count_things("bed"); + +OptResult stores the result of an OptQuery. The presence of a value can be +tested by calling the empty() method. + + +Instalation / Compilation +------------------------- + +pbe::Database is part of libpbe, and you can build the library just by running +"make" at the top level. But libpbe probably contains lots of stuff that you +don't want, so it may be easiest just to copy the Database files, and the few +other files that they depend on, into your own build environment. + +There is one significant external dependency which is Boost. Version 1.33.1 +works; I can't comment on other versions. + + +License +------- + +This is all licensed under the GPL. I'm aware that this is different from +PostgreSQL itself; if you would like to see this use a different license, let me +know. + + + diff --git a/extras/anytermd/libpbe/doc/charsets/intro.txt b/extras/anytermd/libpbe/doc/charsets/intro.txt new file mode 100644 index 0000000000..9c8d069b76 --- /dev/null +++ b/extras/anytermd/libpbe/doc/charsets/intro.txt @@ -0,0 +1,192 @@ +Character Sets - Introduction +============================= + +This is a bottom-up file-by-file introduction to the character-set functionality in the +include/charsets/ directory. + +char_t.hh +--------- + +This defines defined-size character types char8_t, char16_t and char32_t. + +Something similar is proposed for C++0x (see N1823). That doesn't include char8_t; does +that mean that char is certain to be 8 bits? Even so, I think that char8_t is a useful +alias. + + +charset_t.hh +------------ + +This defines an enum, charset_t, to name character sets. + +It is populated with a large number of character set names and aliases automatically +generated from tha IANA character sets registrations. See the file names.txt for more +information about this automatic processing. + + +charset_names.hh +---------------- + +This file provides facilities to convert between the charset_t enum and textual +character set names (in ASCII). This is also based on automatically-generated tables +from the IANA data. + + +charset_traits.hh +----------------- + +This declares a class template, charset_traits, which can be specialised for +each character set. Each specialisation indicates: + +- The unit and character types, which differ for character sets variable-length + encodings. +- A state type, for those character sets that have a "shift state". +- Booleans to indicate characteristics such as whether the character set is an ASCII + superset, and so on. +- For variable-length encodings, various functions for encoding and decoding. (More + about this below.) + +More will probably be added, for example constants to indicate the maximum or +typical number of units per character. + + +utf8.hh +------- + +This provides a specialisation of charset_traits for utf8, including the encoding and +decoding functions. + +Similar files will be needed for UTF-16, iso-2022, GB18030, Big5, Shift-JIS etc. + + +const_character_iterator.hh +--------------------------- + +This defines an iterator adaptor, using boost::iterator_adapter, that takes an iterator +over a sequence of units and provides a const iterator over its characters. It does +this using the encoding and decoding functions from the charset_traits, which have +trivial implementations for fixed-length character sets. + +The iterator is bidirectional, not random access. + + +character_output_iterator.hh +---------------------------- + +This defines a second iterator adaptor that provides a mutable character iterator from a +mutable unit iterator. It is an output iterator, and would normally be used with a +back-insertion iterator that appends to the sequence of units. + + +charset_char_traits.hh +---------------------- + +This provides per-character-set classes compatible with std::char_traits. + + +error_policy.hh +--------------- + +When an error occurs during conversion, an error_policy indicates what should happen. +This file supplies two error policies, error_policy_throw and +error_policy_return_sentinel. + + +char_conv.hh +------------ + +Class char_conv converts a single character from one character set to another. The +input and output character sets, and an error policy, are indicated by template +parameters, and (partial) specialisation will be used to provide conversions for +particular character set pairs. + +The default class attempts to convert in two steps via UCS4. + +Static booleans could be added to indicate whether the conversion is +losslessly-reversible, and so on - "character set pair traits", in effect. + + +conv/ascii.hh +------------- + +Provides trivial conversion between ASCII and Unicode. + + +conv/unicode.hh +--------------- + +Provides trivial conversion between the different Unicode character sets. + + +conv/iso8859.hh +--------------- + +Provides conversion to and from the ISO 8859-n character sets. Conversion for 8859-1 is +trivial. For the other character sets, automatically generated tables from the Unicode +mapping data are used. Tables for 8859-to-Unicode are built in; the sparser tables +for the reverse conversion are generated from them when first needed. + + +char_converter.hh +----------------- + +This is a simple wrapper around char_conv that uses a stateful functor to track the +shift state from one invocation to the next. + + +sequence_conv.hh +---------------- + +This converts a sequence of characters using a similar interface to char_conv. Its default +implementation invokes char_conv repeatedly to do the work. + +Specialisations of this will be provided for fast conversion of contiguous-in-memory data. + + +sequence_converter.hh +--------------------- + +This is a simple wrpapper around sequence_conv that uses a stateful functor to track the +shift state from one invocation to the next. + + +string_adaptor.hh +----------------- + +This provides an adaptor that provides character-at-a-time access to a string of units. It +stores a reference to the underlying string. + +There is still a lot of work to do here. In particular, in each instance where a std::string +member takes or returns a position we have the choice to use +1. A unit poisition. +2. A character position. +3. An iterator. + +There are pros and cons to each. + +There's also the question of over-writing the middle of a string. + + +cs_string.hh +------------ + +A string tagged with its character set. This is implemented using string_adaptor plus an +owned unit-string. Again, lots still to be decided. + + + +STILL TO DO +=========== + +I still plan to add: + +- Fallback character conversion using iconv. + +- Optimisation for conversion of contiguous-in-memory data. + +- Conversion from one cs_string to another. + +- Typedefs for common string and character types, e.g. utf8_string. + +- Worry about all those places where I do comparisons on char values that may or + may not be signed.... diff --git a/extras/anytermd/libpbe/doc/charsets/names.txt b/extras/anytermd/libpbe/doc/charsets/names.txt new file mode 100644 index 0000000000..8f00ff5f8e --- /dev/null +++ b/extras/anytermd/libpbe/doc/charsets/names.txt @@ -0,0 +1,81 @@ +Character Set Names +=================== + +A typical entry from the IANA character set listing at +http://www.iana.org/assignments/character-sets is as follows: + +Name: ANSI_X3.4-1968 [RFC1345,KXS2] +MIBenum: 3 +Source: ECMA registry +Alias: iso-ir-6 +Alias: ANSI_X3.4-1986 +Alias: ISO_646.irv:1991 +Alias: ASCII +Alias: ISO646-US +Alias: US-ASCII (preferred MIME name) +Alias: us +Alias: IBM367 +Alias: cp367 +Alias: csASCII + +Based on this, three things are needed: +- A set of valid C++ identifiers that can be used in the charset_t enumeration. +- An ASCII string that should be output when the character set's name must be printed. +- A set of ASCII strings that should be recognised when the character set's name is + input. + +The following rules are applied: + + +charset_t enumeration members +----------------------------- + +The Name: and all Alias: lines are used. All punctuation characters are replaced by +underscores; all adjacent punctuation characters are compressed to one, and any trailing +punctuation characters are trimmed (?). Letters are mapped to lower case. This results +for the example above in the following: + +ansi_x3_4_1968 +iso_ir_6 +ansi_x3_4_1986 +iso_646_irv_1991 +ascii +iso646_us +us_ascii +us +ibm367 +cp367 +csascii + +Variants are then produced with underscores omitted [but not underscores that separate +numbers on both sides]: + +ansi_x3_4_1968 ansix3_4_1968 +iso_ir_6 iso_ir6 isoir_6 isoir6 +ansi_x3_4_1986 ansix3_4_1986 +iso_646_irv_1991 iso_646_irv1991 iso_646irv_1991 iso_646irv1991 + iso646_irv_1991 iso646_irv1991 iso646irv_1991 isi646irv1991 +ascii +iso646_us iso646us +us_ascii usascii +us +ibm367 +cp367 +csascii + + +ASCII Output +------------ + +Two functions convert to ASCII: charset_name uses the string from the Name: line in +the description (ANSI_X3.4-1968 in this case) and charset_mime_name uses any "preferred +MIME name" that is specified (US-ASCII in this case). + + +ASCII Input +----------- + +The name is looked up in a similar table to the one used for enumerations above; the +lookup is insensitive to case and to punctuation. + + diff --git a/extras/anytermd/libpbe/examples/Makefile b/extras/anytermd/libpbe/examples/Makefile new file mode 100644 index 0000000000..f7fa292580 --- /dev/null +++ b/extras/anytermd/libpbe/examples/Makefile @@ -0,0 +1,46 @@ +# examples/Makefile +# This file is part of libpbe; see http://decimail.org +# (C) 2007 Philip Endecott + +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +default_target: all + +all: inc_daemon mailform directory_explorer fixed_point + +DEBUG_FLAGS=-W -Wall +OPT_FLAGS=-O +INC_FLAGS=-I../include +LINK_FLAGS=-L.. -lpbe + +CPP=g++ +CPP_FLAGS=$(DEBUG_FLAGS) $(OPT_FLAGS) $(INC_FLAGS) + + +%: %.cc + $(CPP) $(CPP_FLAGS) $< -o $@ $(LINK_FLAGS) + + +inc_daemon: inc_daemon.cc + $(CPP) $(CPP_FLAGS) $< -o $@ $(LINK_FLAGS) -lomnithread + +mailform: mailform.cc + +directory_explorer: directory_explorer.cc + +bunzip http_get_bunzip: LINK_FLAGS+=-lbz2 +gunzip http_get_gunzip http_get_with_etag: LINK_FLAGS+=-lz + +.PHONY: default_target all diff --git a/extras/anytermd/libpbe/examples/backtrace.cc b/extras/anytermd/libpbe/examples/backtrace.cc new file mode 100644 index 0000000000..e800ecdb3e --- /dev/null +++ b/extras/anytermd/libpbe/examples/backtrace.cc @@ -0,0 +1,23 @@ +// Get a backtrace when we de-reference a null ponter. + +#include + +#include "segv_backtrace.hh" + + +int* ptr; + +int badstuff(int i) +{ + ptr = NULL; + return badstuff(*ptr * i); +} + + +int main() +{ + get_backtrace_on_segv(); + + return badstuff(1); +} + diff --git a/extras/anytermd/libpbe/examples/base64.cc b/extras/anytermd/libpbe/examples/base64.cc new file mode 100644 index 0000000000..990d9d6773 --- /dev/null +++ b/extras/anytermd/libpbe/examples/base64.cc @@ -0,0 +1,15 @@ +#include +#include + +#include "base64.hh" + +using namespace std; +using namespace pbe; + + +int main() +{ + string s="dGhlIHF1aWNrIGJyb3duIGZveCBqdW1wcyBvdmVyIHRoZSBsYXp5IGRvZyEKCg=="; + cout << decode_base64(s); +} + diff --git a/extras/anytermd/libpbe/examples/bunzip.cc b/extras/anytermd/libpbe/examples/bunzip.cc new file mode 100644 index 0000000000..2df160257b --- /dev/null +++ b/extras/anytermd/libpbe/examples/bunzip.cc @@ -0,0 +1,50 @@ +// examples/bunzip.cc +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#include "Bunzipper.hh" + +#include "FileDescriptor.hh" + +#include +#include + +using namespace std; +using namespace pbe; + + +int main(int argc, char* argv[]) +{ + if (argc!=2) { + cerr << "Usage: bunzip \n"; + return 1; + } + + string fn = argv[1]; + FileDescriptor fd(fn,FileDescriptor::read_only); + + string in = fd.readall(); + + Bunzipper bz; + + string out = bz(in); + + cout << out; + + return 0; +} + diff --git a/extras/anytermd/libpbe/examples/capitals.csv b/extras/anytermd/libpbe/examples/capitals.csv new file mode 100644 index 0000000000..d9de19bdba --- /dev/null +++ b/extras/anytermd/libpbe/examples/capitals.csv @@ -0,0 +1,14 @@ +England,London +Scotland,Edinburgh +Wales,Cardiff +France,Paris +Spain,Madrid +Germany,Berlin +Italy,Rome +Portugal,Lisbon +Russia,Moscow +Serbia,Belgrade +Greece,Athens +Belgium,Brussels +Denmark,Copenhagen +Norway,Oslo diff --git a/extras/anytermd/libpbe/examples/charsets.cc b/extras/anytermd/libpbe/examples/charsets.cc new file mode 100644 index 0000000000..43aa8d8b3c --- /dev/null +++ b/extras/anytermd/libpbe/examples/charsets.cc @@ -0,0 +1,262 @@ +// Example use of charset.hh in ../include/charset.hh. +// +// To compile, you need to make charset.hh accessible on the include path. +// Also, charset.hh includes my iconv wrapper, which is in ../include/Incover.hh, +// and depends on other things in that directory; those dependencies could be +// made to go away. + + +// This source file should be viewed using a UTF8 editor, and its output when run should be +// viewed on a UTF8 terminal. + + +#include "charset.hh" + +#include +#include +#include + +using namespace pbe; +using namespace std; + + +void compile_time_tagged_strings_example() +{ + // This example declares strings with compile-time-fixed character sets, converts + // them to other compile-time-fixed character sets, combines them, and checks for + // consistency: + + cout << "\ncompile_time_tagged_strings_example:\n"; + + utf8_string french = "Le traité simplifié prêt à être soumis " + "à l'approbation des gouvernements"; + latin1_string french_fixed = french.recode(); + + utf8_string icelandic = "Smjörið er brætt og hveitið smátt og smátt hrært út í það"; + latin1_string icelandic_fixed = icelandic.recode(); + + utf8_string all = french + icelandic; + + latin1_string all_fixed = french_fixed + icelandic_fixed; + + if ((all.recode() == all_fixed) + && (all == all_fixed.recode())) { + cout << "Pass, both strings are '" << all << "'\n"; + } +} + + +void utf8_const_iterator_example() +{ + // This example shows how a string with a variable-width + // character set can be iterated over character-at-a-time + // or "unit"-at-a-time. + + cout << "\nutf8_const_iterator_example:\n"; + + utf8_string s = "Théâtre"; // My editor stores UTF8. + + // Iterate "unit" (byte) at a time: + cout << "Here are the bytes of '" << s << "': " << hex; + for (utf8_string::const_iterator i = s.begin(); + i != s.end(); ++i) { + char8_t c = *i; + cout << static_cast(static_cast(c)) << " "; + } + + // Iterate character at a time: + cout << "\nHere are the characters of '" << s << "': "; + for (utf8_string::const_character_iterator i = s.begin(); + i != utf8_string::const_character_iterator( s.end() ); ++i) { + utf8_char_t c = *i; // A 32-bit decoded Unicode character + cout << static_cast(c) << " "; + } + cout << dec << "\n"; +} + + +void utf8_output_iterator_example() +{ + // This example shows how a string with a variable-width + // character set can be appended to using push_back and + // an output iterator. + + cout << "\nutf8_output_iterator_example:\n"; + + utf8_string s; + + for (utf8_char_t c=64; c<96; ++c) { + s.push_back(c); + } + + utf8_string::character_output_iterator i(s); + + for (utf8_char_t c=150; c<200; ++c) { + *i++ = c; +// s.push_back(c); + } + + cout << "Unicode characters 64 to 95 and 150 to 199:\n" + << s << "\n"; +} + + +void utf8_word_split_example() +{ + // This example demonstrates a case where a "unit" rather than a character iterator for a + // UTF8 string is useful: because bytes < 128 can only ever represent single characters in + // UTF8, we can treat a UTF8 string as a sequence of bytes when spliting at spaces. + + cout << "\nutf8_word_split_example:\n"; + + utf8_string s = "Yo también quemo la Corona española"; + utf8_string::const_iterator i = s.begin(); + utf8_string::const_iterator e = s.end(); + utf8_string::const_iterator j; + do { + j = find(i,e,' '); + utf8_string word(i,j); + cout << word << "\n"; + i = j+1; + } while (j != e); +} + + +void ucs4_line_wrap_example() +{ + // Sometimes a random-access character iterator is needed, but an iso_8859 or similar byte + // character set can't be used because the characters in the content are not restricted. + // In this case, ucs4 is normally the best choice - though its requirement for 4 bytes per + // character may be considered a disadvantage in memory-limited applications. + // This example uses random access to break a string into lines of <=40 characters each. + + cout << "\nucs4_line_wrap_example:\n"; + + utf8_string text_var = "Партия Единая Россия отказалась от формирования первой " + "тройки федерального списка - его возглавил только президент " + "Владимир Путин. Такое решение было принято на съезде Единой " + "России во вторник. Накануне президент России дал согласие " + "возглавить список Единой России на выборах в Госдуму."; + + ucs4_string text_fixed = text_var.recode(); + + for (unsigned int i=39; i() << "\n"; +} + + +// This example shows how a library-user can make a new character set available. +// The example is the KOI8 character set, a fixed-width byte character set containing +// cyrillic and latin characters. + +////// This section needs some attention from a preprocessor expert; I want to use +////// a counter of some sort to allocate new charset_t values with a macro: +////// PBE_DEFINE_CHARSET(koi8); +////// But I can't see a good way to do it. For the time being, I'll choose a value +////// manually: +const charset_t koi8 = static_cast(25); + +// Define charset_traits for KOI8: +namespace pbe { + template <> + struct charset_traits { + typedef char8_t unit_t; + typedef char8_t char_t; + }; +}; +typedef tagged_string koi8_string; + +void user_defined_charset_example() +{ + charset_names[koi8] = "koi8"; + + cout << "\nuser_defined_charset_example:\n"; + + // We'll convert a string back and forth between utf8 and koi8: + utf8_string u = "Код Обмена Информацией, 8 бит"; + koi8_string k = u.recode(); + utf8_string u2 = k.recode(); + + // KOI8 is a more compact encoiding than UTF8 for cyrillic: + cout << "Length of UTF8 string = " << u2.length() + << ", length of KOI8 string = " << k.length() << "\n"; +} + + +void runtime_tagged_example() +{ + // This example shows how character sets known only at run-time can be used. + // This is motivated by multipart MIME email, where each part can have a different + // character set. But since MIME is rather complex to parse, this example uses + // the following simpler format: the input byte sequence consists of a character + // set name (in ascii) followed by data using that character set enclosed in {}, + // followed by further content in another character set, and so on. + // This example first creates such a message and then decomposes it. + + cout << "\nruntime_tagged_example:\n"; + + // We'll store the hybrid message in a std::string. + string message = + string("utf8{") + "El catalán, moneda lingüística" + "}" + + "iso-8859-1{" + utf8_string("får årets Nobelpris i litteratur.").recode() + "}"; +// + "ucs2{" + utf8_string("Директором СВР назначен Михаил Фрадков").recode() + "}"; + + // Now parse it into a list of run-time-tagged strings: + typedef list strings_t; + strings_t strings; + string::const_iterator i = message.begin(); + string::const_iterator e = message.end(); + while (i != e) { + string::const_iterator j = find(i,e,'{'); + string charset_name(i,j); + string::const_iterator k = find(j,e,'}'); + string content(j+1,k); + rt_tagged_string s(lookup_charset(charset_name),content); + strings.push_back(s); + i = k+1; + } + + // Output the parsed strings, converting to UTF8 to do so: + for (strings_t::const_iterator a = strings.begin(); + a != strings.end(); ++a) { + utf8_string u = a->recode(); + cout << u << "\n"; + } + +} + + + +// The following examples illustrate planned functionality that's not yet implemented: + +#if 1 + +#endif + + + +int main() +{ + // These examples work: + compile_time_tagged_strings_example(); + utf8_const_iterator_example(); + utf8_output_iterator_example(); + utf8_word_split_example(); + ucs4_line_wrap_example(); + + runtime_tagged_example(); + + // These examples don't yet work: +#if 1 + user_defined_charset_example(); +#endif + + return 0; +} + diff --git a/extras/anytermd/libpbe/examples/const_string_facade.cc b/extras/anytermd/libpbe/examples/const_string_facade.cc new file mode 100644 index 0000000000..059dacee61 --- /dev/null +++ b/extras/anytermd/libpbe/examples/const_string_facade.cc @@ -0,0 +1,115 @@ +// This is an example of the const_string_facade class. +// +// We have a file containing names of countries and their capital cities: +// +// England,London +// Scotland,Edinburgh +// Wales,Cardiff +// etc. etc. +// +// We want to read this in and to store it in a std::map from country name to +// capital city name. This is simple enough to do, but the normal approach has +// the disadvantage that the data is copied out of the buffer into strings +// which are stored in the map. Avoiding this extra copy could be a useful +// thing to do. So instead of using std::strings we'll invent a new class, +// stringref, which just contains a pair of pointers; these point into the +// buffer containing the raw data which is never copied. +// +// const_string_facade is used to make the pair-of-pointers class behave like +// a const std::string: it provides implementations of all the normal member +// and free functions. So we can store them in a map output them and so on. + + +#include "const_string_facade.hh" + +#include +#include +#include +#include + +#include "FileDescriptor.hh" + +using namespace std; +using namespace pbe; + + +class stringref: public const_string_facade { +// The template parameters are: +// - The derived class (this the the "CRTP"). +// - The character type. +// - A bool indicating that the strings are not null-terminated. + +public: + stringref(): begin_ptr(NULL), end_ptr(NULL) {} + stringref(const char* b, const char* e): begin_ptr(b), end_ptr(e) {} + +private: + const char* begin_ptr; + const char* end_ptr; + + // Here are the accessors that const_string_facade needs. Since these + // are private we need to make it a friend. + friend class const_string_facade; + const char* get_begin() const { return begin_ptr; } + const char* get_end() const { return end_ptr; } +}; + + + +int main(int argc, char* argv[]) +{ + if (argc!=3) { + cerr << "Usage: " << argv[0] << " csv-file some-string\n"; + exit(1); + } + string fn = argv[1]; + string some_string = argv[2]; + + FileDescriptor f(fn, FileDescriptor::read_only); + + char buf[4096]; + char* buf_end = buf + f.readmax(buf,sizeof(buf)); + + typedef map capitals_t; + capitals_t capitals; + + char* ptr = buf; + while (ptrfirst << " is " << i->second << "\n"; + } + + const char* vowels = "aeiouAEIOU"; + + for (capitals_t::const_iterator i = capitals.begin(); + i != capitals.end(); ++i) { + const stringref country = i->first; + size_t first_vowel_pos = country.find_first_of(vowels); + size_t last_consonant_pos = country.find_last_not_of(vowels); + cout << "The first vowel in " << country << " is " << country[first_vowel_pos] + << " and the last consonant is " << country[last_consonant_pos] << "\n"; + size_t ss_pos = country.find(some_string); + if (ss_pos != stringref::npos) { + cout << " '" << country << "' contains '" << some_string << "' at position " << ss_pos << "\n"; + } + } + + + // Hopefully, a stringref has no overhead on top of the two pointers: + cout << "sizeof(stringref) = " << sizeof(stringref) << "\n"; + + return 0; +} + diff --git a/extras/anytermd/libpbe/examples/directory_explorer.cc b/extras/anytermd/libpbe/examples/directory_explorer.cc new file mode 100644 index 0000000000..0330f65df3 --- /dev/null +++ b/extras/anytermd/libpbe/examples/directory_explorer.cc @@ -0,0 +1,70 @@ +// examples/directory_explorer.cc +// This file is part of libpbe; see http://decimail.org +// (C) 2006 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +// Example for use of StringTransformer class + + +// Demonstrate the libpbe::Directory class and get_filetype. +// Main program takes a directory name and recursively prints the filenames +// below it. + +#include + +#include "Directory.hh" +#include "FileType.hh" +#include "Exception.hh" + + +using namespace std; +using namespace libpbe; + + +void explore(Directory& d) +{ + for (Directory::const_iterator i = d.begin(); + i != d.end(); ++i) { + cout << i->pathname << "\n"; + if (get_filetype(i->pathname)==directory) { + Directory subdir(i->pathname); + explore(subdir); + } + } +} + + +int main(int argc, char* argv[]) +{ + try { try { + + if (argc!=2) { + cerr << "usage:\n" + << " directory_recursor \n"; + exit(1); + } + + string dirname = argv[1]; + Directory d(dirname); + explore(d); + + exit(0); + + } RETHROW_MISC_EXCEPTIONS } + catch (Exception& E) { + E.report(cerr); + exit(1); + } +} diff --git a/extras/anytermd/libpbe/examples/fixed_point.cc b/extras/anytermd/libpbe/examples/fixed_point.cc new file mode 100644 index 0000000000..3053e35f50 --- /dev/null +++ b/extras/anytermd/libpbe/examples/fixed_point.cc @@ -0,0 +1,42 @@ +#include + +#include "fixed.hh" +#include "safe_int.hh" + +using namespace std; +using namespace pbe; + +typedef pbe::fixed<16,8> f1; +typedef pbe::fixed<16,10> f2; + +int main(int argc, char* argv[]) +{ + f1 a; + f2 b; + f1 c; + f2 d; + pbe::fixed<24,26> e; + + a = 0.25; + b = 128; + + c = a+b; + d = c * a; + e = d / a; + + cout << "a=" << static_cast(a) << " (a.val = " << a.val << ")\n"; + cout << "b=" << static_cast(b) << " (b.val = " << b.val << ")\n"; + cout << "c=" << static_cast(c) << " (c.val = " << c.val << ")\n"; + cout << "d=" << static_cast(d) << " (d.val = " << d.val << ")\n"; + cout << "e=" << static_cast(e) << " (e.val = " << e.val << ")\n"; + + typedef pbe::fixed<8,5,pbe::safe_int<13> > f_t; + f_t f; + f = a; + + while (1) { + f = f+f; + cout << "f=" << static_cast(f) << "\n"; + } +} + diff --git a/extras/anytermd/libpbe/examples/gunzip.cc b/extras/anytermd/libpbe/examples/gunzip.cc new file mode 100644 index 0000000000..9bcec85dbc --- /dev/null +++ b/extras/anytermd/libpbe/examples/gunzip.cc @@ -0,0 +1,50 @@ +// examples/gunzip.cc +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#include "Gunzipper.hh" + +#include "FileDescriptor.hh" + +#include +#include + +using namespace std; +using namespace pbe; + + +int main(int argc, char* argv[]) +{ + if (argc!=2) { + cerr << "Usage: gunzip \n"; + return 1; + } + + string fn = argv[1]; + FileDescriptor fd(fn,FileDescriptor::read_only); + + string in = fd.readall(); + + Gunzipper gz; + + string out = gz(in); + + cout << out; + + return 0; +} + diff --git a/extras/anytermd/libpbe/examples/http_get.cc b/extras/anytermd/libpbe/examples/http_get.cc new file mode 100644 index 0000000000..23a665ee64 --- /dev/null +++ b/extras/anytermd/libpbe/examples/http_get.cc @@ -0,0 +1,56 @@ +// examples/http_get.cc +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#include "HttpClient.hh" +#include "HttpResponse.hh" + +#include "Exception.hh" + +#include +#include + +using namespace pbe; +using namespace std; + + + +int main(int argc, char* argv[]) +{ + try { try { + + if (argc!=2) { + cerr << "Usage: http_get \n"; + exit(1); + } + + string uri = argv[1]; + + HttpClient http_client; + HttpResponse response = http_client.get(uri); + + cout << response.body; + + return 0; + + } RETHROW_MISC_EXCEPTIONS } + catch (Exception& E) { + E.report(cerr); + exit(E.exit_status); + } +} diff --git a/extras/anytermd/libpbe/examples/http_get_bunzip.cc b/extras/anytermd/libpbe/examples/http_get_bunzip.cc new file mode 100644 index 0000000000..75c93afbfb --- /dev/null +++ b/extras/anytermd/libpbe/examples/http_get_bunzip.cc @@ -0,0 +1,54 @@ +// examples/http_get_bunzip.cc +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#include "HttpClient.hh" + +#include "Exception.hh" + +#include +#include + +using namespace pbe; +using namespace std; + + + +int main(int argc, char* argv[]) +{ + try { try { + + if (argc!=3) { + cerr << "Usage: http_get_bunzip \n"; + exit(1); + } + + string uri = argv[1]; + string fn = argv[2]; + + HttpClient http_client; + http_client.get_bunzip_save(uri,fn); + + return 0; + + } RETHROW_MISC_EXCEPTIONS } + catch (Exception& E) { + E.report(cerr); + exit(E.exit_status); + } +} diff --git a/extras/anytermd/libpbe/examples/http_get_gunzip.cc b/extras/anytermd/libpbe/examples/http_get_gunzip.cc new file mode 100644 index 0000000000..f86c6c1cd6 --- /dev/null +++ b/extras/anytermd/libpbe/examples/http_get_gunzip.cc @@ -0,0 +1,61 @@ +// examples/http_get_gunzip.cc +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#include "HttpClient.hh" +#include "Gunzipper.hh" + +#include "Exception.hh" + +#include +#include + +using namespace pbe; +using namespace std; + + + +int main(int argc, char* argv[]) +{ + try { try { + + if (argc!=3) { + cerr << "Usage: http_get_gunzip \n"; + exit(1); + } + + string uri = argv[1]; + string fn = argv[2]; + + try { + HttpClient http_client; + http_client.get_gunzip_save(uri,fn); + } + catch (Gunzipper::InvalidData& E) { + cerr << "Gunzipper::InvalidData\n"; + exit(1); + } + + return 0; + + } RETHROW_MISC_EXCEPTIONS } + catch (Exception& E) { + E.report(cerr); + exit(E.exit_status); + } +} diff --git a/extras/anytermd/libpbe/examples/http_get_with_etag.cc b/extras/anytermd/libpbe/examples/http_get_with_etag.cc new file mode 100644 index 0000000000..87e3085abe --- /dev/null +++ b/extras/anytermd/libpbe/examples/http_get_with_etag.cc @@ -0,0 +1,55 @@ +// examples/http_get.cc +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#include "HttpClient.hh" +#include "HttpResponse.hh" + +#include "Exception.hh" + +#include +#include + +using namespace pbe; +using namespace std; + + + +int main(int argc, char* argv[]) +{ + try { try { + + if (argc!=3) { + cerr << "Usage: http_get_with_etag \n"; + exit(1); + } + + string uri = argv[1]; + string fn = argv[2]; + + HttpClient http_client; + http_client.get_save_with_etag(uri,fn); + + return 0; + + } RETHROW_MISC_EXCEPTIONS } + catch (Exception& E) { + E.report(cerr); + exit(E.exit_status); + } +} diff --git a/extras/anytermd/libpbe/examples/inc_daemon.cc b/extras/anytermd/libpbe/examples/inc_daemon.cc new file mode 100644 index 0000000000..c2ca8b70f2 --- /dev/null +++ b/extras/anytermd/libpbe/examples/inc_daemon.cc @@ -0,0 +1,134 @@ +// examples/inc_daemon.cc +// This file is part of libpbe; see http://decimail.org +// (C) 2004 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +// Demo program for Daemon class + +// The Daemon class does the mucky stuff needed to implement a +// "daemon" (i.e. a server, e.g. imapd, httpd etc) process. + +// You make a subclass of Daemon that: +// * Provides "void session(int in_fd, int out_fd)" that does the work. +// File descriptors for input and output to the connection are supplied. +// * Calls the Daemon base class constructor with port number, program +// name (used in syslog calls) and optional syslog "facility" code (see +// man syslog). +// +// Your main program then creates an instance of your Daemon subclass and +// invokes one of three run_*() methods, as shown below. + +// This example implements a trivial "inc" daemon that reads a number +// from the user, increments it, and returns the incremented value. + + +// libpbe includes: + +#include "Daemon.hh" +#include "Exception.hh" + +// Standard includes: + +#include +#include +#include +using namespace std; + + +// To demonstrate exception handling, if the user enters 666 this +// exception is thrown. +class NumberOfTheBeast: public Exception { +public: + void report(ostream& s) { + s << "The number of the beast!" << endl; + } +}; + + +class IncDaemon: public Daemon { + +public: + IncDaemon(void): Daemon(1705,"inc_daemon") {} + + + // Here is the session function that is called to handle a new + // connection. It is run in its own thread. + void session(int in_fd, int out_fd) + { + // I don't know how to make a C++ stream from a file descriptor. + FILE* inf = fdopen(in_fd,"r"); + FILE* outf = fdopen(out_fd,"w"); + + fprintf(outf,"Please enter numbers to increment.\n"); + int r=1; + // check for EOF is important - it indicates connection is closed + while(!feof(inf) && !ferror(inf) && !ferror(outf)) { + fprintf(outf,"> "); + fflush(outf); + int n; + r = fscanf(inf,"%d",&n); + if (r==0 || r==EOF) { + break; + } + if (n==666) { + throw NumberOfTheBeast(); + } + fprintf(outf,"--->%d<---\n",n+1); + } + if (r==0) { + fprintf(outf,"fscanf() didn't get a number\n"); + } + if (r==EOF) { + fprintf(outf,"fscanf() returned EOF\n"); + } + if (feof(inf)) { + fprintf(outf,"EOF on input\n"); + } + if (ferror(inf)) { + fprintf(outf,"Error on input(!)\n"); + } + if (ferror(outf)) { + fprintf(outf,"Error on output(!)\n"); + } + fflush(outf); + } +}; + + + +int main(int argc, char* argv[]) +{ + IncDaemon d; + if (argc==1) { + // Guess whether to run in interactive or daemon mode based on isatty(0) + d.run_default(); + } else if (argc==2) { + string a(argv[1]); + if (a=="-i") { + // Run interactively, i.e. not a daemon at all + d.run_interactively(); + } else if (a=="-d") { + // Run as a daemon + cout << "Daemon about to start. Telnet to port 1705." << endl; + d.run_as_daemon(); + } else { + cerr << "Unrecognised option '" << a << "'" << endl; + exit(1); + } + } else { + cerr << "Too many options" << endl; + exit(1); + } +} diff --git a/extras/anytermd/libpbe/examples/mailform.cc b/extras/anytermd/libpbe/examples/mailform.cc new file mode 100644 index 0000000000..b44fc2492c --- /dev/null +++ b/extras/anytermd/libpbe/examples/mailform.cc @@ -0,0 +1,190 @@ +// examples/mailform.cc +// This file is part of libpbe; see http://decimail.org +// (C) 2004 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +// MAILFORM +// -------- + +// This is a demonstration of the libpbe CgiParams and SmtpClient +// classes. It is a CGI program that implements the backend of a +// feedback form, or similar. The user completes a form on a web page +// and submits it to this script, which forwards its content as an +// email message. + +// The important bits of the form look like this: +//
+// +// +// +// +//
+// +// feedback_received.html is the page that will be shown when the +// message has been sucessfully sent. +// +// You can also note from which page the user clicked on the "feedback +// form" page using a tad of javascript (which will not mess up +// non-javascript browsers): +// +// (Put this inside the
element, obviously.) + + + +// libpbe includes: + +#include "CgiParams.hh" +#include "SmtpClient.hh" + + +// Standard includes: + +#include + +#include +#include +using namespace std; + + +// The values can be redefined using -D when compiling. You at least +// need to supply sensible values for SENDER and RECIPIENT. + +// From address if the user doesn't specify one: +#ifndef ANON_ADDR + #define ANON_ADDR "anonymous" +#endif + +// From address if the user doesn't specify one: +#ifndef SENDER + #define SENDER "webpage@this_machine" +#endif + +// Define the recipient of the emails here: +#ifndef RECIPIENT + #define RECIPIENT "feedback@some_domain" +#endif + +// SMTP server to connect to: +#ifndef SMTP_SERVER + #define SMTP_SERVER "localhost" +#endif + +// Our own domain name, as announced to the SMTP server when the +// connection is established: +#ifndef OWN_DOMAIN + #define OWN_DOMAIN "localhost" +#endif + + +int main (int argc, char* argv[]) +{ + // This must be called from a web server as a CGI program. Give up + // immediately if that doesn't seem to be the case. (CGI sets + // various environment variables, of which QUERY_STRING is one.) + if (!getenv("QUERY_STRING")) { + cerr << "This program must be invoked using CGI." << endl; + exit(1); + } + + // This is my standard idiom for top-level exception handling, see + // below. + try { try { + + // Create a CgiParams object and load it from the environment. We + // don't need to know the details of what this is doing, just that + // it can cope with all of the variants of CGI encoding. + CgiParams params; + params.load(); + + // Get the form values. get_default() substitutes the supplied + // default if no parameter with that name was supplied. get() + // throws an exception in this case. operator[] can be used if + // you are certain that the parameter is present (like a + // std::map). + string name = params.get_default("name","(none)"); + string email = params.get_default("email","(none)"); + string page = params.get_default("page","(none)"); + string message = params.get("message"); + string nextpage = params.get_default("nextpage","index.html"); + + // Now construct the message. This needs to be a complete message + // in RFC822 (aka RFC2822 which is easier to read) format. + + // Do PLEASE ensure that your DATE is in the correct RFC822 + // format! And, if you ever write code to parse an email, be + // prepared for dates in just about any misformat you can imagine. + // I speak from experience. + char datetime[36]; + time_t t; + time(&t); + struct tm tm; + localtime_r(&t,&tm); + strftime(datetime,sizeof(datetime),"%a, %d %b %Y %H:%M:%S %z",&tm); + + // If the user gave an email address, we can give it here as the + // from address. That makes it easy to reply to the right place. + // On the other hand it might break if you have a spam-filtering + // system like SPF in place (anyone know the magic to avoid this?) + string from_addr = (email=="" || email=="(none)") ? ANON_ADDR : email; + + string rfc822_message = + "Sender: " + string(SENDER) + "\r\n" + "From: " + from_addr + "\r\n" + "To: " + RECIPIENT + "\r\n" + "Subject: " + "Feedback form" + "\r\n" + "Date: " + datetime + "\r\n" + "\r\n" + "Name: " + name + "\r\n" + "Email: " + email + "\r\n" + "Page: " + page + "\r\n\r\n" + "Message:\r\n" + message + "\r\n"; + + // Now create the SMTP client. + // If you want to see what is going on on the SMTP connection, you + // can enable logging to syslog by defining SYSLOG in + // SmtpClient.cc. + SmtpClient smtpc; + // Make an SMTP connection to the server + smtpc.connect(SMTP_SERVER,OWN_DOMAIN); + // Send the message + smtpc.send_msg(SENDER,RECIPIENT,rfc822_message); + // Done, disconnect. + smtpc.disconnect(); + + // Final thing to do is to redirect to the "message sent" HTML page. + cout << "Location: " << nextpage << "\r\n\r\n"; + + // All done. + exit(0); + + + // My standard exception handling magic. This converts any odd + // exception types, like char* from 'throw "something bad";', into + // an Exception. + } RETHROW_MISC_EXCEPTIONS } + catch (Exception& E) { + cout << "content-type: text/plain\r\n\r\n" + << "An error has occured:\r\n"; + E.report(cout); + exit(0); + } +} + diff --git a/extras/anytermd/libpbe/examples/recode.cc b/extras/anytermd/libpbe/examples/recode.cc new file mode 100644 index 0000000000..de53c12387 --- /dev/null +++ b/extras/anytermd/libpbe/examples/recode.cc @@ -0,0 +1,64 @@ +// examples/recode.cc +// This file is part of libpbe; see http://decimail.org +// (C) 2004 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +// Demo program for libpbe Recoder class +// C++ interface to the recode library +// Convert between character sets e.g. ISO-8859-n to/from unicode +// Also CR to CRLF, do base64 coding and the like + +#include "Recoder.hh" + +// Create a converter as a global object of class Recoder. +// (There is startup overhead; doesn't have to be global, but is probably +// best in most cases. Don't create and destroy them all over the place) + +// In this case, convert LF line endings (the default) to Microsfot-style +// CR-LF line endings. + +Recoder lf_to_crlf ( "", "/CR-LF" ); + +// Parameters to constructor are "from" and "to" character sets. +// See recode documentation for details (info recode) +// Just a few examples: +// ASCII 7-bit ASCII +// ISO-8859-1 8-bit ISO-8859-1 (Latin 1; Western European languages) +// ISO-8859-15 8-bit ISO-8859-15 (ditto plus Euro symbol) +// UTF-8 8-bit Unicode +// UCS-2 16-bit Unicode +// /CR CR for end-of-line (Apple) +// /CR-LF CR-LF for end-of-line (Microsoft) +// [LF is default, use empty string] +// /Base64 Base 64 encoding (used for email attachments) +// /Decimal-1 Output decimal values for each character (for debugging) + +// Having created to Recoder object, use it as a function (on strings) to +// actually do the conversion: + +#include +#include +using namespace std; + +int main(int argc, char* argv[]) +{ + string s = "Hello\nWorld\n"; + string t = lf_to_crlf(s); + cout << s << ' ' << t; +} + +// Need to link with -lrecode + +// Pipe this program into "od -a" to see what is going on. diff --git a/extras/anytermd/libpbe/examples/safe_int.cc b/extras/anytermd/libpbe/examples/safe_int.cc new file mode 100644 index 0000000000..38389ae0c6 --- /dev/null +++ b/extras/anytermd/libpbe/examples/safe_int.cc @@ -0,0 +1,31 @@ +#include + +#include "safe_int.hh" + +#include "Exception.hh" + + +using namespace std; +using namespace pbe; + +typedef safe_int<7> safe_byte; + +int main(int argc, char* argv[]) +{ + try { try { + + safe_byte a = 1; + + for (int i=0; i<1024; ++i) { + cout << "a = " << a << "\n"; + a = a * static_cast(3); + } + + return 0; + + } RETHROW_MISC_EXCEPTIONS } + catch (Exception& E) { + E.report(cerr); + } +} + diff --git a/extras/anytermd/libpbe/examples/sort_by.cc b/extras/anytermd/libpbe/examples/sort_by.cc new file mode 100644 index 0000000000..f1122156de --- /dev/null +++ b/extras/anytermd/libpbe/examples/sort_by.cc @@ -0,0 +1,26 @@ +#include "sort_by.hh" + +#include + +using namespace std; +using namespace pbe; + + +struct A { + int x; + int y; + bool z; + A(int x_, int y_, bool z_): x(x_), y(y_), z(z_) {} +}; + + +int main() +{ + std::vector v; + v.push_back(A(1,2,true)); + v.push_back(A(3,4,false)); + v.push_back(A(9,0,true)); + + sort_by(v.begin(),v.end(),&A::y); +} + diff --git a/extras/anytermd/libpbe/examples/sort_by_pointee.cc b/extras/anytermd/libpbe/examples/sort_by_pointee.cc new file mode 100644 index 0000000000..1efda788b0 --- /dev/null +++ b/extras/anytermd/libpbe/examples/sort_by_pointee.cc @@ -0,0 +1,26 @@ +#include "sort_by_pointee.hh" + +#include + +using namespace std; +using namespace pbe; + + +struct A { + int x; + int y; + bool z; + A(int x_, int y_, bool z_): x(x_), y(y_), z(z_) {} +}; + + +int main() +{ + std::vector v; + v.push_back(new A(1,2,true)); + v.push_back(new A(3,4,false)); + v.push_back(new A(9,0,true)); + + sort_by_pointee(v.begin(),v.end(),&A::y); +} + diff --git a/extras/anytermd/libpbe/examples/sorted_ptr_vector.cc b/extras/anytermd/libpbe/examples/sorted_ptr_vector.cc new file mode 100644 index 0000000000..0a345c5f7f --- /dev/null +++ b/extras/anytermd/libpbe/examples/sorted_ptr_vector.cc @@ -0,0 +1,31 @@ +#include +#include +#include "sorted_ptr_vector.hh" + +using namespace std; +using namespace pbe; + + +struct A { + int x; + int y; + int z; + A(int x_, int y_, int z_): x(x_), y(y_), z(z_) {}; +}; + + +int main() +{ + list l; + l.push_back(new A(1,2,3)); + l.push_back(new A(5,4,3)); + l.push_back(new A(2,3,2)); + + typedef sorted_ptr_vector v_t; + v_t v(l.begin(),l.end()); + + for (v_t::const_iterator i = v.begin(); i != v.end(); ++i) { + cout << (*i)->x << "," << (*i)->y << "," << (*i)->z << "\n"; + } +} + diff --git a/extras/anytermd/libpbe/examples/string_qsort.cc b/extras/anytermd/libpbe/examples/string_qsort.cc new file mode 100644 index 0000000000..269e7e0239 --- /dev/null +++ b/extras/anytermd/libpbe/examples/string_qsort.cc @@ -0,0 +1,33 @@ +#include +#include +#include + +#include "string_qsort.hh" + +using namespace std; +using namespace pbe; + + +int main() +{ + typedef vector data_t; + data_t data; + + while (cin.good()) { + string s; + getline(cin,s); + data.push_back(s); + } + + string_qsort(data.begin(), data.end()); + +// Or try this for comparison: +// std::sort(data.begin(), data.end()); +// The result should be the same. + + for (data_t::const_iterator i = data.begin(); + i != data.end(); ++i) { + cout << *i << "\n"; + } +} + diff --git a/extras/anytermd/libpbe/examples/transform_strings.cc b/extras/anytermd/libpbe/examples/transform_strings.cc new file mode 100644 index 0000000000..5b6ad31759 --- /dev/null +++ b/extras/anytermd/libpbe/examples/transform_strings.cc @@ -0,0 +1,59 @@ +// examples/transform_strings.cc +// This file is part of libpbe; see http://decimail.org +// (C) 2004 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +// Example for use of StringTransformer class + +// Create a string transformer object to transform strings. +// Create the object once (a bit expensive) and use it many time (cheapish) +// For example: transform a string to escape quotes, make upper case etc. + + +#include "StringTransformer.hh" + +#include +#include +using namespace std; + + +bool is_ctrl(char c) { + return c<32; +} + +string to_escseq(char c) { + ostringstream s; + s << '\\' << static_cast(c); + return s.str(); +} + + +int main(int argc, char* argv[]) +{ + StringTransformer escape_quotes; + escape_quotes.add_cs_rule('\"', "\\\""); // replace " with \" + + StringTransformer escape_ctrl; + escape_ctrl.add_pf_rule(is_ctrl, to_escseq); + + while (1) { + char buf[128]; + cin.getline(buf,sizeof(buf)); + string s(buf); + + cout << "Escape quotes: " << escape_quotes(s) << endl; + cout << "Escape ctrls: " << escape_ctrl(s) << endl; + } +} diff --git a/extras/anytermd/libpbe/include/Array2d.hh b/extras/anytermd/libpbe/include/Array2d.hh new file mode 100644 index 0000000000..ef4bb729ef --- /dev/null +++ b/extras/anytermd/libpbe/include/Array2d.hh @@ -0,0 +1,59 @@ +// Array2d.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2006 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_Array2d_hh +#define libpbe_Array2d_hh + +#include + + +namespace pbe { + +template > +class Array2d { + +public: + Array2d(unsigned int w, unsigned int h): + data(w*h), width_(w), height_(h) + {} + + Array2d(unsigned int w, unsigned int h, T value): + data(w*h,value), width_(w), height_(h) + {} + + const T& operator()(unsigned int x, unsigned int y) const { + return data[y*width_ + x]; + } + + T& operator()(unsigned int x, unsigned int y) { + return data[y*width_ + x]; + } + + unsigned int width() const { return width_; } + unsigned int height() const { return height_; } + +private: + Impl data; + const unsigned int width_; + const unsigned int height_; +}; + +}; + + +#endif diff --git a/extras/anytermd/libpbe/include/AtomicCounter.hh b/extras/anytermd/libpbe/include/AtomicCounter.hh new file mode 100644 index 0000000000..7b12179276 --- /dev/null +++ b/extras/anytermd/libpbe/include/AtomicCounter.hh @@ -0,0 +1,132 @@ +// AtomicCounter.hh +// This file is part of libpbe; see http://decimail.org/ +// (C) 2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_AtomicCounter_hh +#define libpbe_AtomicCounter_hh + + +// Atomic (i.e. thread-safe) counter. Uses the gcc atomic builtins, except on ARM +// where they are not available. On ARM it uses a swap instruction, using -1 as a sentinel +// value (and has one fewer useful bits as a consequence; code should allow for the counter +// wrapping after 31 or 32 bits). It normally spins if it can't get the lock, but you can define +// YIELD_WHEN_LOCKED to make it yield in that case. Contention should be extremely rare. + +// FIXME this should use atomic.hh. It should also test whether the gcc builtins are +// available. + + +#include + + +#if defined(__arm__) && defined(YIELD_WHEN_LOCKED) +#include +#endif + + +namespace pbe { + + +#ifdef __arm__ + +static inline int32_t arm_atomic_read_and_lock(int32_t& mem) +{ + int32_t val; + do { + // Equivalent to: + // val = mem; + // mem = -1; + asm volatile ("swp\t%0, %1, [%2]" + :"=&r"(val) + :"r" (-1), + "r" (&mem) + :"memory"); + if (val != -1) { + break; + } +#ifdef YIELD_WHEN_LOCKED + sched_yield(); +#endif + } while (1); + return val; +} + +static inline int32_t arm_atomic_inc(int32_t& mem, int32_t inc) +{ + return mem = (arm_atomic_read_and_lock(mem)+inc) & 0x7fffffff; +} + +static inline int32_t arm_atomic_inc_pre(int32_t& mem, int32_t inc) +{ + int32_t r = arm_atomic_read_and_lock(mem); + mem = (r+inc) & 0x7fffffff; + return r; +} + +#endif + + + +class AtomicCounter { + + int c; + +public: + AtomicCounter(): + c(0) + {} + + AtomicCounter(int n): + c(n) + {} + + int operator++() { // ++n +#ifdef __arm__ + return arm_atomic_inc(c, 1); +#else + return __sync_add_and_fetch(&c, 1); +#endif + } + + int operator++(int) { // n++ +#ifdef __arm__ + return arm_atomic_inc_pre(c, 1); +#else + return __sync_fetch_and_add(&c, 1); +#endif + } + + int operator--() { // --n +#ifdef __arm__ + return arm_atomic_inc(c, -1); +#else + return __sync_add_and_fetch(&c, -1); +#endif + } + + int operator--(int) { // n-- +#ifdef __arm__ + return arm_atomic_inc_pre(c, -1); +#else + return __sync_fetch_and_add(&c, -1); +#endif + } +}; + +}; + +#endif diff --git a/extras/anytermd/libpbe/include/Box.hh b/extras/anytermd/libpbe/include/Box.hh new file mode 100644 index 0000000000..30bb702e5a --- /dev/null +++ b/extras/anytermd/libpbe/include/Box.hh @@ -0,0 +1,134 @@ +// include/Box.hh +// This file is part of libpbe +// (C) 2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef pbe_Box_hh +#define pbe_Box_hh + +#include "Point.hh" + +#include + + +namespace pbe { + +template +struct Box { + typedef COORD_T coord_t; + typedef AREA_T area_t; + typedef typename pbe::Point point_t; + + coord_t x0; + coord_t y0; + coord_t x1; + coord_t y1; + + Box() {} + + Box(const Box& other): + x0(other.x0), y0(other.y0), x1(other.x1), y1(other.y1) + {} + + Box(coord_t x0_, coord_t y0_, coord_t x1_, coord_t y1_): + x0(x0_), y0(y0_), x1(x1_), y1(y1_) + {} + + Box(point_t x0y0, point_t x1y1): + x0(x0y0.x), y0(x0y0.y), x1(x1y1.x), y1(x1y1.y) + {} + + Box(point_t x0y0, coord_t w, coord_t h): + x0(x0y0.x), y0(x0y0.y), x1(x0y0.x+w), y1(x0y0.y+h) + {} + + coord_t width() const { return x1-x0; } + coord_t height() const { return y1-y0; } + + point_t x0y0() const { + return point_t(x0,y0); + } + + point_t x0y1() const { + return point_t(x0,y1); + } + + point_t x1y0() const { + return point_t(x1,y0); + } + + point_t x1y1() const { + return point_t(x1,y1); + } + + bool contains(point_t p) const { + return p.x>=x0 && p.x=y0 && p.y +inline bool overlap(const Box& lhs, const Box& rhs) { + return lhs.x0<=rhs.x1 && lhs.x1>=rhs.x0 + && lhs.y0<=rhs.y1 && lhs.y1>=rhs.y0; +} + + +template +inline typename BOX_T::area_t area(BOX_T box) { + return box.width() * box.height(); +} + + +template +inline void expand_box(BOX_T& box, typename BOX_T::point_t point){ + box.x0 = min(box.x0,point.x); + box.y0 = min(box.y0,point.y); + box.x1 = max(box.x1,point.x); + box.y1 = max(box.y1,point.y); +} + + +template +inline BOX_T intersect(const BOX_T& b1, const BOX_T& b2) { + return BOX_T( std::max(b1.x0, b2.x0), + std::max(b1.y0, b2.y0), + std::min(b1.x1, b2.x1), + std::min(b1.y1, b2.y1) ); +} + +}; + + +#endif + diff --git a/extras/anytermd/libpbe/include/Bunzipper.hh b/extras/anytermd/libpbe/include/Bunzipper.hh new file mode 100644 index 0000000000..6d32b8f1c0 --- /dev/null +++ b/extras/anytermd/libpbe/include/Bunzipper.hh @@ -0,0 +1,89 @@ +// include/Bunzipper.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef pbe_Bunzipper_hh +#define pbe_Bunzipper_hh + +#ifdef HAVE_BZIP + +#include + +#include + + +namespace pbe { + + +class Bunzipper { + + bz_stream bz; + +public: + + Bunzipper(bool small_mem = false) { + bz.bzalloc = NULL; + bz.bzfree = NULL; + bz.opaque = NULL; + int r = BZ2_bzDecompressInit(&bz, 0, small_mem?1:0); + switch (r) { + case BZ_CONFIG_ERROR: throw "BZ_CONFIG_ERROR"; + case BZ_PARAM_ERROR: throw "BZ_PARAM_ERROR"; + case BZ_MEM_ERROR: throw std::bad_alloc(); + } + } + + ~Bunzipper() { + BZ2_bzDecompressEnd(&bz); + } + + struct InvalidData {}; + + std::string operator()(const char* in, size_t in_bytes) { + bz.next_in = const_cast(in); + bz.avail_in = in_bytes; + std::string out; + while (bz.avail_in>0) { + char buffer[4096]; + bz.next_out = buffer; + bz.avail_out = sizeof(buffer); + int r = BZ2_bzDecompress(&bz); + switch (r) { + case BZ_PARAM_ERROR: throw "BZ_PARAM_ERROR"; + case BZ_DATA_ERROR: throw InvalidData(); + case BZ_DATA_ERROR_MAGIC: throw InvalidData(); + case BZ_MEM_ERROR: throw std::bad_alloc(); + } + out.append(buffer,sizeof(buffer)-bz.avail_out); + } + return out; + } + + std::string operator()(std::string in) { + return operator()(in.data(),in.length()); + } +}; + + + +}; + + +#endif + +#endif + diff --git a/extras/anytermd/libpbe/include/CgiParams.hh b/extras/anytermd/libpbe/include/CgiParams.hh new file mode 100644 index 0000000000..f22b781ef5 --- /dev/null +++ b/extras/anytermd/libpbe/include/CgiParams.hh @@ -0,0 +1,77 @@ +// include/CgiParams.hh +// This file is part of libpbe; see http://decimail.org +// (C) 2004 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +#ifndef libpbe_CgiParams_hh +#define libpbe_CgiParams_hh + +#include +#include +using namespace std; + + +// CGI parameters + +// Provides a mechanism to access form parameters passed from the +// browser. + +// The CgiParams class makes the parameters available as a key-value +// map (it inherits from map. + +// CGI parameters can be passed from the browser, via the server, in +// three formats. + +// For forms with METHOD="GET" (indicated in the CGI program by the +// REQUEST_METHOD environment variable having the value GET), the +// parameters are passed over HTTP as part of the URI, and arrive at +// the CGI program in the QUERY_STRING environment variable. +// E.g. http://www.foo.com/cgi-bin/blobby.cgi?name=phil&blah=33 Note +// that & or ; can be used to separate paramters; if & is used, it +// must be escaped in HTML. Keys and values are separated by =. Keys +// and values are URI-encoded ( + is space, %nn hex for other odd +// characters). + +// For forms with METHOD="POST", the parameters are passed over HTTP +// in the "body" of the request, and arrive at the CGI program on +// stdin. They can be encoded in one of two ways, indicated by the +// CONTENT_TYPE environment variable. + +// If the content-type is "application/x-www-form-urlencoded", the data +// is in the same format as for the GET method: name=phil&blah=33. + +// If the content-type is multipart/form-data, the data is encoded in +// a MIME style. + + +// The load() method for class CgiParams will determine which +// mechanism is in use, and do whatever decoding is necessary. + + +class CgiParams: public map { +public: + CgiParams(void) {} + void load(void); + + string get(string name) const; + string get_default(string name, string def) const; + +private: + void init_from_urlencoded(string query_string); + void init_from_multipart(string input, string boundary); +}; + + +#endif diff --git a/extras/anytermd/libpbe/include/CgiVars.hh b/extras/anytermd/libpbe/include/CgiVars.hh new file mode 100644 index 0000000000..a430c8e391 --- /dev/null +++ b/extras/anytermd/libpbe/include/CgiVars.hh @@ -0,0 +1,72 @@ +// src/CgiVars.hh +// This file is part of libpbe; see http://decimail.org +// (C) 2004 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_CgiVars_hh +#define libpbe_CgiVars_hh + +#include +#include + +class CgiVars: public std::map { +private: + void setvar(const char* varname); + std::string get(std::string varname) const { + CgiVars::const_iterator i = find(varname); + if (i==end()) { + return ""; + } else { + return i->second; + } + } + +public: + void load(void); + + // Basic variables defined by CGI spec: + std::string get_server_software(void) const { return get("SERVER_SOFTWARE"); } + std::string get_server_name(void) const { return get("SERVER_NAME"); } + std::string get_gateway_interface(void) const { return get("GATEWAY_INTERFACE"); } + std::string get_server_protocol(void) const { return get("SERVER_PROTOCOL"); } + std::string get_server_port(void) const { return get("SERVER_PORT"); } + std::string get_request_method(void) const { return get("REQUEST_METHOD"); } + std::string get_path_info(void) const { return get("PATH_INFO"); } + std::string get_path_translated(void) const { return get("PATH_TRANSLATED"); } + std::string get_script_name(void) const { return get("SCRIPT_NAME"); } + std::string get_query_string(void) const { return get("QUERY_STRING"); } + std::string get_remote_host(void) const { return get("REMOTE_HOST"); } + std::string get_auth_type(void) const { return get("AUTH_TYPE"); } + std::string get_remote_user(void) const { return get("REMOTE_USER"); } + std::string get_remote_ident(void) const { return get("REMOTE_IDENT"); } + std::string get_content_type(void) const { return get("CONTENT_TYPE"); } + std::string get_content_length(void) const { return get("CONTENT_LENGTH"); } + + // Generic HTTP variables: + std::string get_http_user_agent(void) const { return get("HTTP_USER_AGENT"); } + std::string get_http_cookie(void) const { return get("HTTP_COOKIE"); } + std::string get_http_if_modified_since(void) + const { return get("HTTP_IF_MODIFIED_SINCE"); } + std::string get_http_if_none_match(void) const { return get("HTTP_IF_NONE_MATCH"); } + + // Apache specials: + bool get_https(void) const { return get("HTTPS")!=""; } + + static const CgiVars& singleton(void); +}; + + +#endif diff --git a/extras/anytermd/libpbe/include/Condition.hh b/extras/anytermd/libpbe/include/Condition.hh new file mode 100644 index 0000000000..198d66029c --- /dev/null +++ b/extras/anytermd/libpbe/include/Condition.hh @@ -0,0 +1,267 @@ +// include/Condition.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_Condition_hh +#define libpbe_Condition_hh + +// This file provides a class Condition, which implements a condition variable +// compatible with N2447's condition_variable[_any?]. + +// If we have Futex - i.e. if this is Linux - then we implement Condition directly +// on top of Futex. Otherwise we implement it using pthread_condition. + +#include "Futex.hh" +#include "atomic.hh" +#include "HPtime.hh" + +#include + +#include + +#include + + +#if defined(PBE_HAVE_FUTEX) && defined(PBE_HAVE_ATOMICS) + +#include "Unlock.hh" + + +namespace pbe { + + +class Condition: boost::noncopyable { + + uint32_t waiters; + uint32_t eventcount; + Futex fut; + +public: + Condition(): + waiters(0), // Keep track of the number of waiting threads so that + // the notify operations can be no-ops in the case where + // there are no waiters. + eventcount(0), // Incremented on each notify_all. This is needed to + // detect any notify_all that occurs between unlocking + // the mutex and waiting on the futex. + fut(eventcount) {} + + ~Condition() {} + // Note that ~Condition does not wait for any waiters to stop waiting; + // undefined behaviour results if a Condition is destructed while a thread + // is still waiting on it. + + void notify_one() { + if (atomic_read(waiters)==0) { + return; + } + unsigned int n_woken = fut.wake(1); + if (n_woken==0) { + notify_all(); // WHY??? + return; + } + atomic_dec(waiters,n_woken); + } + + void notify_all() { + if (atomic_read(waiters)==0) { + return; + } + atomic_inc(eventcount); + unsigned int n_woken = fut.wake(INT_MAX); + atomic_dec(waiters,n_woken); + } + + template + void wait(LOCK_T& lock) { + atomic_inc(waiters); + uint32_t initial_eventcount = atomic_read(eventcount); + Unlock ul(*lock.mutex()); // Must not throw. + // If this could throw, we would need to --waiters. + do { + // fut.wait can return spuriously e.g. if a signal is handled. + // We retry in this case, so Condition.wait does not return spuriously. + fut.wait(initial_eventcount); + } while (atomic_read(eventcount)==initial_eventcount); // NO broken for notify_one + } + + template + void wait(LOCK_T& lock, Predicate pred) { + // Note that the following "trivial" implementation is specified by N2447; + // if notify_one() is called, the thread that is woken may test its condition + // and sleep again. There's no guarantee that the notification is delivered + // to a thread whose condition will evaluate true. So notify_one() is not + // useful in combination with predicated wait, unless perhaps all waiters + // have the same predicate. + while (!pred()) { + wait(lock); + } + } + + + template + bool timed_wait(LOCK_T& lock, HPtime abs_time) { + atomic_inc(waiters); + uint32_t initial_eventcount = atomic_read(eventcount); + Unlock ul(*lock.mutex()); // Must not throw. + // If this could throw, we would need to --waiters. + do { + // fut.wait can return spuriously e.g. if a signal is handled. + // We retry in this case, so Condition.wait does not return spuriously. + HPtime now = HPtime::now(); + if (now>abs_time || !fut.timed_wait(initial_eventcount, abs_time-now)) { + return false; + } + } while (atomic_read(eventcount)==initial_eventcount); // NO broken for notify_one + return true; + } + + template + bool timed_wait(LOCK_T& lock, float rel_time) { + HPtime abs_time = HPtime::now() + rel_time; + return timed_wait(lock,abs_time); + } + + template + bool timed_wait(Lock& lock, ::time_t abs_time, Predicate pred) { + // See above. If the predicate becomes true and the timeout expires at + // the same time, we're supposed to return true indicating that the predicate + // was true. + while (!pred()) { + if (!timed_wait(lock,abs_time)) { + return pred(); + } + } + return true; + } + + // native_handle not provided +}; + +}; + +#else + + +#include + +#include "Exception.hh" + + +namespace pbe { + + +class Condition: boost::noncopyable { + + pthread_cond_t cond; + +public: + Condition() { + int r = pthread_cond_init(&cond, NULL); + if (r!=0) { + pbe::throw_ErrnoException("pthread_cond_init()",r); + } + } + + ~Condition() { + pthread_cond_destroy(&cond); + } + + void notify_one() { + int r = pthread_cond_signal(&cond); + if (r!=0) { + pbe::throw_ErrnoException("pthread_cond_signal()",r); + } + } + + void notify_all() { + int r = pthread_cond_broadcast(&cond); + if (r!=0) { + pbe::throw_ErrnoException("pthread_cond_broadcast()",r); + } + } + + template + void wait(LOCK_T& lock) { + int r = pthread_cond_wait(&cond, lock.mutex()->native_handle()); + if (r!=0) { + pbe::throw_ErrnoException("pthread_cond_wait()",r); + } + } + + template + void wait(LOCK_T& lock, Predicate pred) { + // Note that the following "trivial" implementation is specified by N2447; + // if notify_one() is called, the thread that is woken may test its condition + // and sleep again. There's no guarantee that the notification is delivered + // to a thread whose condition will evaluate true. So notify_one() is not + // useful in combination with predicated wait, unless perhaps all waiters + // have the same predicate. + while (!pred()) { + wait(lock); + } + } + + template + bool timed_wait(LOCK_T& lock, ::time_t abs_time) { + timespec ts; + ts.tv_sec = abs_time; + ts.tv_nsec = 0; + int r = pthread_cond_timedwait(&cond, lock.mutex()->native_handle(), &ts); + switch (r) { + case 0: return true; + case ETIMEDOUT: return false; + default: pbe::throw_ErrnoException("pthread_cond_timedwait()",r); + } + // NOT REACHED + return true; + } + + template + bool timed_wait(LOCK_T& lock, float rel_time) { + // This limits resolution to whole seconds. + return timed_wait(lock, static_cast(time(NULL)+rel_time)); + } + + template + bool timed_wait(Lock& lock, ::time_t abs_time, Predicate pred) { + // See above. If the predicate becomes true and the timeout expires at + // the same time, we're supposed to return true indicating that the predicate + // was true. + while (!pred()) { + if (!timed_wait(lock,abs_time)) { + return pred(); + } + } + return true; + } + + pthread_cond_t* native_handle() { + return &cond; + } + +}; + + +}; + + +#endif + +#endif + + diff --git a/extras/anytermd/libpbe/include/ConditionAll.hh b/extras/anytermd/libpbe/include/ConditionAll.hh new file mode 100644 index 0000000000..db23e2f383 --- /dev/null +++ b/extras/anytermd/libpbe/include/ConditionAll.hh @@ -0,0 +1,138 @@ +// include/ConditionAll.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_ConditionAll_hh +#define libpbe_ConditionAll_hh + +// This file provides a class ConditionAll, which implements a condition variable +// compatible with N2447's condition_variable_any, except that it provides +// only notify_all() and not notify_one(). + +// FIXME this should be updated to match Condition. + + +#include + +#include "atomic.hh" +#include "Futex.hh" +#include "Unlock.hh" + +#include + + +namespace pbe { + +class ConditionAll: boost::noncopyable { + + uint32_t waiters; + uint32_t eventcount; + Futex fut; + +public: + ConditionAll(): + waiters(0), // Keep track of the number of waiting threads so that + // the notify operations can be no-ops in the case where + // there are no waiters. + // In an application where there are always waiters, this + // is an unnecessary overhead. + eventcount(0), // Incremented on each notify_all. This is needed to + // detect any notify_all that occurs between unlocking + // the mutex and waiting on the futex. + fut(eventcount) {} + + ~ConditionAll() {} + // Note that ~ConditionAll does not wait for any waiters to stop waiting; + // undefined behaviour results if a ConditionAll is destructed while a thread + // is still waiting on it. + + void notify_all() { + if (atomic_read(waiters)==0) { + return; + } + atomic_inc(eventcount); + uint32_t n_woken = fut.wake(std::numeric_limits::max()); + atomic_dec(waiters,n_woken); + } + + template + void wait(LOCK_T& lock) { + atomic_inc(waiters); + uint32_t initial_eventcount = atomic_read(eventcount); + Unlock ul(*(lock.mutex())); + // Must not throw. FIXME is this a valid assumption? + // If this could throw, we would need to --waiters. + do { + // fut.wait can return spuriously e.g. if a signal is handled. + // We retry in this case, so ConditionAll.wait does not return spuriously. + fut.wait(initial_eventcount); + } while (atomic_read(eventcount)==initial_eventcount); + } + + template + void wait(LOCK_T& lock, Predicate pred) { + while (!pred()) { + wait(lock); + } + } + + // What type to use for the timeout? N2447 species that it's 'system_time', + // which doesn't mean anything to me. Note that it's an absolute time, + // in contrast to the relative timeout in Mutex::timed_lock - I'm not sure why. + // I'll use time_t, though perhaps struct timeval would be more appropriate + // as it includes a microseconds field. + template + bool timed_wait(LOCK_T& lock, ::time_t abs_time) { + ::time_t start_time = time(NULL); + atomic_inc(waiters); + uint32_t initial_eventcount = atomic_read(eventcount); + Unlock ul(*(lock.mutex())); + // Must not throw. FIXME is this a valid assumption? + // If this could throw, we would need to --waiters. + do { + // fut.timed_wait can return spuriously e.g. if a signal is handled. + // We retry in this case, so ConditionAll.wait does not return spuriously. + if (!fut.timed_wait(initial_eventcount, static_cast(abs_time-start_time))) { + // Timed out. + atomic_dec(waiters); + return false; + } + } while (atomic_read(eventcount)==initial_eventcount); + return true; + } + + template + bool timed_wait(Lock& lock, ::time_t abs_time, Predicate pred) { + // If the predicate becomes true and the timeout expires at + // the same time, we're supposed to return true indicating that the predicate + // was true. + while (!pred()) { + if (!timed_wait(lock,abs_time)) { + return pred(); + } + } + return true; + } + + // native_handle not provided +}; + +}; + +#endif + + diff --git a/extras/anytermd/libpbe/include/Daemon.hh b/extras/anytermd/libpbe/include/Daemon.hh new file mode 100644 index 0000000000..2e7596fc85 --- /dev/null +++ b/extras/anytermd/libpbe/include/Daemon.hh @@ -0,0 +1,75 @@ +// include/Daemon.hh +// This file is part of libpbe; see http://decimail.org +// (C) 2004-2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_Daemon_hh +#define libpbe_Daemon_hh + +#include +#include + +#include "FileDescriptor.hh" +#include "Mutex.hh" +#include "Condition.hh" + +// TODO: +// Sending EOF doesn't do anything (sigpipe?) +// Worry about other signals +// Spurious thread on first connection (is it syslog???) + + +class Daemon { +public: + static const int default_max_sessions = 25; + + Daemon(short p, + std::string pn, + int sf = LOG_LOCAL0, + std::string u="", + std::string d="", + int max_sessions_=0, + bool accept_local_only_=false); + virtual ~Daemon(); + + void run_interactively(void); + void run_as_daemon(bool background = true); + void run_default(void); + + virtual void startup(void) {}; + virtual void session(pbe::FileDescriptor& in_fd, pbe::FileDescriptor& out_fd) = 0; + +private: + const short port; + const std::string progname; + const int syslog_facility; + const std::string username; + const std::string dir; + const int max_sessions; + const bool accept_local_only; + + typedef pbe::Mutex<> n_sessions_mutex_t; + n_sessions_mutex_t n_sessions_mutex; + typedef pbe::Condition n_sessions_condition_t; + n_sessions_condition_t n_sessions_condition; + int n_sessions; + +public: // really private + void decrement_session_count(void); +}; + + +#endif diff --git a/extras/anytermd/libpbe/include/Database.hh b/extras/anytermd/libpbe/include/Database.hh new file mode 100644 index 0000000000..bee16d70a8 --- /dev/null +++ b/extras/anytermd/libpbe/include/Database.hh @@ -0,0 +1,678 @@ +// src/Database.hh +// This file is part of libpbe; see http://decimail.org +// (C) 2004 - 2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_Database_hh +#define libpbe_Database_hh + +#include "Exception.hh" +#include "FileDescriptor.hh" +#include "endian.hh" + +#include + +#include + +#include +#include +#include +#include +#include +#include +#include + + +namespace pbe { + +// The maximum number of query parameters is set by this macro, which user code can +// define before #including this file. +#ifndef PBE_DB_MAX_QUERY_PARAMS +#define PBE_DB_MAX_QUERY_PARAMS 7 +#endif + +// Making it too large has the disadvantage that error messages become even more +// incomprehensible, and may also increase compile times. + + +// private: + +// We support queries with variable numbers of parameters by means of +// sentinel default paramter values. The sentinel is of a special empty type +// null_t. +// ? Could we just use void for this ? + +struct null_t {}; +static const null_t nullval = null_t(); + + +// We need to map between C++ types and PostgreSQL types in query parameters. +// This enum lists all of the PostgreSQL types that we map to, plus two +// sentinels: unknown_type indicates that no PostgreSQL type corresponds to the +// supplied C++ type, and should result in a compile-time error; null_type is used +// in unused arguments slots in the varargs-like query syntax. + +enum typecode_t { /*unknown_type=0,*/ null_type, + text_type, numeric_type, timestamptz_type, bytea_type, + float_type, double_type, + typecode_t_max }; + +// Now we use template specialisation to get the typecode_t corresponding to a C++ +// type at compile time. get_typecode() is the typecode_t for type T. + +// The generic template is not implemented anywhere, so it will fail at link +// time for these unsupported types. +template inline typecode_t get_typecode(void); + +template <> inline typecode_t get_typecode(void) { return null_type; } +template <> inline typecode_t get_typecode(void) { return text_type; } +template <> inline typecode_t get_typecode(void) { return numeric_type; } +template <> inline typecode_t get_typecode(void) { return timestamptz_type; } +template <> inline typecode_t get_typecode(void) { return float_type; } +template <> inline typecode_t get_typecode(void) { return double_type; } + +// public: + +// class Database represents a connection to the database server. +// It's marked non-copyable because it's not clear what should be done with +// the underlying libpq connection object in that case. +class Database: public boost::noncopyable { +public: + + // Connect to the database server using the supplied connection info. + Database(std::string conninfo); + + // Disconnect on destruction. + ~Database(); + + // Get the file descriptor for the connction to the server. + // This is useful if you want to use select() to wait for asynchronous + // notifications from the server. You obviously mustn't read or write to + // it. + const pbe::FileDescriptor& get_fd(void) const; + + // Return any asynchronous notifications received from the server. + std::string get_any_notification(void); + +private: + // libpq data structure representing the connection. + PGconn* pgconn; + + // Connection file descriptor. + pbe::FileDescriptor conn_fd; + + // This is needed so that we can support nested transactions. + // (PostgreSQL now has built-in support for this, so it should be reviewed.) + bool transaction_in_progress; + + // Run arbitary queries + void exec_sql(std::string cmd); + + friend class QueryCore; + friend class Transaction; +}; + + +// Exceptions thrown by the Database functions: + +class DatabaseException: public Exception { +public: + DatabaseException(PGconn* pgconn, std::string doing_what_): + postgres_error(PQerrorMessage(pgconn)), + doing_what(doing_what_) {} + void report(std::ostream& s) const; +private: + std::string postgres_error; + std::string doing_what; +}; + +class QueryFailed: public DatabaseException { +public: + QueryFailed(PGconn* pgconn, std::string query): + DatabaseException(pgconn,"Executing query " + query) {} +}; + +class TypeError: public DatabaseException { +public: + TypeError(PGconn* pgconn): + DatabaseException(pgconn,"Type error") {} +}; + + +// Transactions. Typical usage: +// { +// Transaction t(db); +// ... run queries ... +// t.commit(); +// } +// If the transaction goes out of scope without commit() having been called, e.g. +// because an exception was thrown and not caught, the transaction will be rolled +// back. +// TODO: PostgreSQL now has some sort of built-in support for nested +// transactions. +class Transaction: public boost::noncopyable { +public: + Transaction(Database& database_); + ~Transaction(); + void commit(void); +private: + Database& database; + bool nested; + bool committed; +}; + + +// Convert from the representation returned by libpq to a normal C++ type. + +template +T decode_pq_res(const char* data, int length __attribute__((unused))); + +template <> +inline int decode_pq_res(const char* data, int length __attribute__((unused))) { + const int* valp = reinterpret_cast(data); + return ntohl(*valp); +} + +template <> +inline uint64_t decode_pq_res(const char* data, int length __attribute__((unused))) { + const uint64_t* valp = reinterpret_cast(data); + return ntoh64(*valp); +} + +template <> +inline int64_t decode_pq_res(const char* data, int length __attribute__((unused))) { + const int64_t* valp = reinterpret_cast(data); + return ntoh64(*valp); +} + +template <> +inline time_t decode_pq_res(const char* data, int length __attribute__((unused))) { + // Timestamp values are returned by PostgreSQL as 64-bit microsecond + // values. 946684800000000 is a magic number to convert to the Unix + // 1970 epoch. + int64_t t = decode_pq_res(data, length); + return t/1000000 + 946684800; +} + +template <> +inline std::string decode_pq_res(const char* data, int length __attribute__((unused))) { + return std::string(data,length); +} + +union float_or_int { + float f; + int i; +}; + +template <> +inline float decode_pq_res(const char* data, int length __attribute__((unused))) { + const float_or_int* uptr = reinterpret_cast(data); + float_or_int ucopy; + ucopy.i = ntohl(uptr->i); + return ucopy.f; +} + +union double_or_two_ints { + double d; + int i[2]; +}; + +template <> +inline double decode_pq_res(const char* data, int length __attribute__((unused))) { + const double_or_two_ints* uptr = reinterpret_cast(data); + double_or_two_ints ucopy; + ucopy.i[0] = ntohl(uptr->i[0]); + ucopy.i[1] = ntohl(uptr->i[1]); + return ucopy.d; +} + + +// Result of a SELECT query: +class Result { +public: + // Construct from the result object from libpq + Result(boost::shared_ptr); + + // For insert, update and delete statements the result tells us how many + // rows were inserted, updated or deleted, using the following: + int get_rows_changed(void) const; + + // Everything below here is for getting at the table returned for a select + // statement: + + // The size of the table + const int rows; + const int cols; + + // Map between column names and numbers + int column(std::string name) const ; + std::string column_name(int pos) const; + + // Get data + // The pointer returned by rawget is valid for as long as the Result object exists. + char* rawget(int row, int col) const; + + int getlength(int row, int col) const; + + template + T get_nocheck(int row, int col) const { + return decode_pq_res(rawget(row,col),getlength(row,col)); + } + + template + T get(int row, int col) const { + check_column_type(col); + return get_nocheck(row,col); + } + + template + T get(int row, std::string colname) const { + return get(row, column(colname)); + } + +#if 0 + // ... does this work? + template + T operator()(int row, int col) const { + } + + template + T operator()(int row, std::string colname) const { + return operator()(row, column(colname)); + } +#endif + + bool is_null(int row, int col) const; + + bool is_null(int row, std::string colname) const { + return is_null(row, column(colname)); + } + + // Column types + typecode_t column_typecode(int col) const; + + template + void check_column_type(int col) const { + if (column_typecode(col)!=get_typecode()) { + throw StrException("type error for column " ++boost::lexical_cast(col)+", expecting typecode " ++boost::lexical_cast(get_typecode())+" but got typecode " ++boost::lexical_cast(column_typecode(col))); + } + } + +private: + boost::shared_ptr res; +}; + + +// Alternative result for a query that generates a single column. +// Values can be accessed using (), e.g. r(i). +template +class ColumnResult: public Result { +public: + ColumnResult(Result r): Result(r) { + if (cols!=1) { + throw "Single column expected"; + } + if (rows>0) { + check_column_type(0); + } + } + T operator ()(int row) const { return get(row,0); } + bool is_null (int row) const { return Result::is_null(row,0); } + class const_iterator: + public boost::iterator_facade { + public: + const_iterator(): + res(NULL), row(0) {} + const_iterator(const const_iterator& other): + res(other.res), row(other.row) {} + private: + const ColumnResult& res; + int row; + friend class ColumnResult; + const_iterator(const ColumnResult& res_, int row_): + res(res_), row(row_) {} + friend class boost::iterator_core_access; + void increment() { ++row; } + void decrement() { --row; } + void advance(int n) { row+=n; } + int distance_to(const_iterator other) { return other.row-row; } + bool equal(const const_iterator& other) const { + return row==other.row && &res==&(other.res); + } + const T dereference() const { + return res(row); + } + }; + const_iterator begin() const { + return const_iterator(*this,0); + } + const_iterator end() const { + return const_iterator(*this,rows); + } +}; + + +// Alternative result for a query that generates a single value +// (e.g. a "count(*)" query). +// This is convertable to the type of that value. +template +class SingletonResult: public Result { +public: + SingletonResult(Result r): Result(r) { + if (rows!=1 || cols!=1) { + throw "Singleton expected"; + } + check_column_type(0); + } + operator T () const { return get(0,0); } + bool is_null () const { return Result::is_null(0,0); } +}; + + +// Alternative result for a query that generates a zero or one values. +// This is convertable to the type of that value. +template +class OptResult: public Result { +public: + OptResult(Result r): Result(r) { + if (rows>1 || cols!=1) { + throw "Zero or one values expected"; + } + if (rows==1) { + check_column_type(0); + } + } + operator T () { return get(0,0); } + bool is_null () { return Result::is_null(0,0); } + bool empty() { return rows==0; } +}; + + +// private: + +// Generate names for prepared statements +class statement_name_t: public std::string { +public: + statement_name_t(void): + std::string("stmt_"+boost::lexical_cast(counter++)) + {} +private: + static int counter; +}; + + +// Most of the work of Query is delagated to QueryCore which is not a template +// class and so need not be coded inline. +// This is "semi-private": code can use it if it needs to pass more than +// PBE_DB_MAX_QUERY_PARAMS paramters to a query, but if that's not necessary then it +// should not be used. + +class QueryCore { +public: + QueryCore(Database& database_, std::string querystr_, int nparams, + typecode_t* argtypecodes, int* lengths, int* formats); + ~QueryCore(); + Result operator()(const char* enc_args[]); + Result runonce(const char* enc_args[]); +private: + Database& database; + const std::string querystr; + const bool params_ok; + const statement_name_t statement_name; + int nparams; + int* param_lengths; + int* param_formats; + Oid* argoids; + bool prepared; + void prepare(void); +}; + + +// Convert from C++ types to the representation passed to libpq. +// Types are either binary or character. Binary is used for numeric types and +// character is used for text types. Binary types must be in network byte +// order. +// This uses template specialisation. +// It's OK for these function to modify their paramters in place. + +template +inline const char* encode_pq_arg(T& t); + +template <> +inline const char* encode_pq_arg(null_t& i __attribute__((unused)) ) { + return NULL; +} + +template <> +inline const char* encode_pq_arg(int& i) { + i = htonl(i); + return reinterpret_cast(&i); +} + +template <> +inline const char* encode_pq_arg(uint64_t& i) { + i = hton64(i); + return reinterpret_cast(&i); +} + +template <> +inline const char* encode_pq_arg(int64_t& i) { + i = hton64(i); + return reinterpret_cast(&i); +} + +template <> +inline const char* encode_pq_arg(time_t& t) { +// No! This leaks! + int64_t* iptr = new int64_t; + *iptr = static_cast(t-946684800)*1000000; + return encode_pq_arg(*iptr); +} + +template <> +inline const char* encode_pq_arg(std::string& s) { + return s.c_str(); +} + +static inline float htonfloat(float f) { + uint32_t i; + memcpy(&i,&f,4); + i = htonl(i); + float r; + memcpy(&r,&i,4); + return r; +} + +template <> +inline const char* encode_pq_arg(float& f) { + f = htonfloat(f); + return reinterpret_cast(&f); +} + +template <> +inline const char* encode_pq_arg(double& d) { + uint32_t* ptr = reinterpret_cast(&d); + ptr[0] = htonl(ptr[0]); + ptr[1] = htonl(ptr[1]); + return reinterpret_cast(&d); +} + + +// We need to know the size of the encoded data. +// In most cases this is sizeof(T), but there are exceptions when the +// encoding process changes the size. + +template +inline size_t get_enc_type_size(void) { return sizeof(T); } + +template <> +inline size_t get_enc_type_size(void) { return 8; } + + + +// The Boost preprocessor library is used to build query parameter lists (etc.) with +// multiple argumnets. + +#define PBE_DB_Q_TEMPLATE_PARAM(z,n,data) typename T##n=null_t +#define PBE_DB_Q_TEMPLATE_PARAMS BOOST_PP_ENUM(PBE_DB_MAX_QUERY_PARAMS,PBE_DB_Q_TEMPLATE_PARAM,) +// Expands to e.g. typename T0=null_t,typename T1=null_t + +#define PBE_DB_Q_OP_APPLY_PARAM(z,n,data) T##n arg##n=nullval +#define PBE_DB_Q_OP_APPLY_PARAMS BOOST_PP_ENUM(PBE_DB_MAX_QUERY_PARAMS,PBE_DB_Q_OP_APPLY_PARAM,) +// Expands to e.g. T0 arg0=nullval,T1 arg1=nullval + +#define PBE_DB_Q_BASE_TEMPL_PARAM(z,n,data) T##n +#define PBE_DB_Q_BASE_TEMPL_PARAMS BOOST_PP_ENUM(PBE_DB_MAX_QUERY_PARAMS,PBE_DB_Q_BASE_TEMPL_PARAM,) +// Expands to e.g. T0,T1 + +#define PBE_DB_Q_BASE_OP_APPLY_PARAM(z,n,data) arg##n +#define PBE_DB_Q_BASE_OP_APPLY_PARAMS BOOST_PP_ENUM(PBE_DB_MAX_QUERY_PARAMS,PBE_DB_Q_BASE_OP_APPLY_PARAM,) +// Expands to e.g. arg0,arg1 + + +// This base class is used to form the parameters passed to Query, below, into arrays +// that can be passed to QueryCore. + +template struct QueryBase { + typecode_t typecodes[PBE_DB_MAX_QUERY_PARAMS]; + int lengths[PBE_DB_MAX_QUERY_PARAMS]; + int formats[PBE_DB_MAX_QUERY_PARAMS]; + + QueryBase() { +#define PBE_DB_QB_INIT_ONE(z,n,data) \ + typecodes[n]=get_typecode(); \ + lengths[n]=get_enc_type_size();\ + formats[n]=!boost::is_same::value; +BOOST_PP_REPEAT(PBE_DB_MAX_QUERY_PARAMS,PBE_DB_QB_INIT_ONE,) + } +}; + + +// Prepare and run a query. Typical usage: +// Query insert_item(db,"insert into items(name,qty) values ($1,$2)"); +// insert_item("nut",100); +// insert_item("bolt",102); +// The query can have up to PBE_DB_MAX_QUERY_PARAMS parameters. This limit can be +// increased at the top of this file. + +template class Query: + private QueryBase { +public: + + // Create a query. SQL is supplied in querystr with parameters indicated by + // $1, $2 etc. C++ types of these parameters are the template paramters. + Query(Database& database, std::string querystr): + core(database, querystr, PBE_DB_MAX_QUERY_PARAMS, + QueryBase::typecodes, + QueryBase::lengths, + QueryBase::formats) + {} + + // Run a query. It is prepared the first time that it is run. + Result operator()(PBE_DB_Q_OP_APPLY_PARAMS) { + const char* arg_cs[PBE_DB_MAX_QUERY_PARAMS]; +#define PBE_DB_Q_OP_APPLY_ONE(z,n,data) \ + T##n arg##n##_c = arg##n; \ + arg_cs[n] = encode_pq_arg(arg##n##_c); +BOOST_PP_REPEAT(PBE_DB_MAX_QUERY_PARAMS,PBE_DB_Q_OP_APPLY_ONE,) + return core(arg_cs); + } + + // Run a query immediately, without preparation. Use this for queries that will only be + // run once. + Result runonce(PBE_DB_Q_OP_APPLY_PARAMS) { + const char* arg_cs[PBE_DB_MAX_QUERY_PARAMS]; +#define PBE_DB_Q_RUNONCE_ONE(z,n,data) \ + T##n arg##n##_c = arg##n; \ + arg_cs[n] = encode_pq_arg(arg##n##_c); +BOOST_PP_REPEAT(PBE_DB_MAX_QUERY_PARAMS,PBE_DB_Q_RUNONCE_ONE,) + return core.runonce(arg_cs); + } + + +private: + QueryCore core; +}; + + +// A query that returns a ColumnResult. +template +class ColumnQuery: public Query< PBE_DB_Q_BASE_TEMPL_PARAMS> { +public: + typedef ColumnResult result_t; + + ColumnQuery(Database& database, std::string querystr): + Query(database,querystr) + {} + + result_t operator()(PBE_DB_Q_OP_APPLY_PARAMS) { + return Query::operator()(PBE_DB_Q_BASE_OP_APPLY_PARAMS); + } + + result_t runonce(PBE_DB_Q_OP_APPLY_PARAMS) { + return Query::runonce(PBE_DB_Q_BASE_OP_APPLY_PARAMS); + } +}; + + +// A query that returns a SingeltonResult. In this case there's no need to +// delcare the result object, thanks to implicit type conversion. For +// example: int a = query(param); +template +class SingletonQuery: public Query< PBE_DB_Q_BASE_TEMPL_PARAMS> { +public: + typedef SingletonResult result_t; + + SingletonQuery(Database& database, std::string querystr): + Query(database,querystr) + {} + + result_t operator()(PBE_DB_Q_OP_APPLY_PARAMS) { + return Query::operator()(PBE_DB_Q_BASE_OP_APPLY_PARAMS); + } + + result_t runonce(PBE_DB_Q_OP_APPLY_PARAMS) { + return Query::runonce(PBE_DB_Q_BASE_OP_APPLY_PARAMS); + } +}; + + +// A query that returns a OptResult. +template +class OptQuery: public Query< PBE_DB_Q_BASE_TEMPL_PARAMS> { +public: + typedef OptResult result_t; + + OptQuery(Database& database, std::string querystr): + Query(database,querystr) + {} + + result_t operator()(PBE_DB_Q_OP_APPLY_PARAMS) { + return Query::operator()(PBE_DB_Q_BASE_OP_APPLY_PARAMS); + } + + result_t runonce(PBE_DB_Q_OP_APPLY_PARAMS) { + return Query::runonce(PBE_DB_Q_BASE_OP_APPLY_PARAMS); + } +}; + + +}; + +#endif diff --git a/extras/anytermd/libpbe/include/Date.hh b/extras/anytermd/libpbe/include/Date.hh new file mode 100644 index 0000000000..8ba20b8669 --- /dev/null +++ b/extras/anytermd/libpbe/include/Date.hh @@ -0,0 +1,126 @@ +// src/Date.hh +// This file is part of libpbe; see http://decimail.org +// (C) 2006 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_Date_hh +#define libpbe_Date_hh + +#include + +#include + +#include + + +namespace pbe { + + + static int days_in_month_[] = { 31, 0, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31 }; + + struct Date { + + uint16_t year; + uint8_t month; + uint8_t day; + + Date() {} + + Date(int year_, int month_, int day_): + year(year_), month(month_), day(day_) {} + + Date(time_t t) { + struct tm b; + localtime_r(&t, &b); + year = b.tm_year + 1900; + month = b.tm_mon + 1; + day = b.tm_mday; + } + + bool operator<(const Date& rhs) const { + return (year1) { + --day; + } else { + if (month>1) { + --month; + } else { + month=12; + --year; + } + day = days_in_month(); + } + return *this; + } + + }; + + + inline std::ostream& operator<<(std::ostream& strm, Date d) { + strm << d.year << "-" << static_cast(d.month) << "-" << static_cast(d.day); + return strm; + } + +} + + +#endif diff --git a/extras/anytermd/libpbe/include/DateTime.hh b/extras/anytermd/libpbe/include/DateTime.hh new file mode 100644 index 0000000000..ded47c09b3 --- /dev/null +++ b/extras/anytermd/libpbe/include/DateTime.hh @@ -0,0 +1,118 @@ +// src/DateTime.hh +// This file is part of libpbe; see http://decimail.org +// (C) 2006 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_DateTime_hh +#define libpbe_DateTime_hh + +#include + +#include + +#include "Date.hh" +#include "Time.hh" +#include "Exception.hh" + + +namespace pbe { + + struct DateTime { + + Date date; + Time time; + + DateTime() {} + + DateTime(int year, int month, int day, + int hour=0, int minute=0, int second=0): + date(year,month,day), + time(hour,minute,second) {} + + DateTime(time_t t) { + struct tm b; + localtime_r(&t, &b); // Or gmtime_r() ??? + date.year = b.tm_year + 1900; + date.month = b.tm_mon + 1; + date.day = b.tm_mday; + time.hour = b.tm_hour; + time.minute = b.tm_min; + time.second = b.tm_sec; + } + + DateTime(Date date_): + date(date_), time(0,0,0) {} + + DateTime(Date date_, Time time_): + date(date_), time(time_) {} + + bool operator<(const DateTime& rhs) const { + return (date +#include + +#include + +namespace pbe { + +class Directory { + +public: + Directory(std::string dirname_); + ~Directory(); + + struct Entry { + std::string leafname; + std::string pathname; + }; + + class const_iterator { + public: + ~const_iterator(); + void operator++(void); + Entry operator*(void) const {return this_entry;} + const Entry* operator->(void) const {return &this_entry;} + bool operator==(const const_iterator& r) const; + bool operator!=(const const_iterator& r) const { return !((*this)==r); } + friend class Directory; + private: + std::string dirname; + DIR* dir; + bool at_end; + Entry this_entry; + const_iterator(void); + const_iterator(std::string dirname_); + void read_next(void); + }; + + const_iterator begin(void) const; + const_iterator end(void) const; + +private: + std::string dirname; + +}; + +}; + +#endif diff --git a/extras/anytermd/libpbe/include/Exception.hh b/extras/anytermd/libpbe/include/Exception.hh new file mode 100644 index 0000000000..fa9ac9d5b7 --- /dev/null +++ b/extras/anytermd/libpbe/include/Exception.hh @@ -0,0 +1,189 @@ +// src/Exception.hh +// This file is part of libpbe; see http://decimail.org +// (C) 2004-2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_Exception_hh +#define libpbe_Exception_hh + +#include + +#include +#include + + +namespace pbe { + +class Exception +{ +public: + Exception(void): exit_status(1) {} + virtual ~Exception() {} + virtual void report(std::ostream& s) const = 0; + const int exit_status; +}; + + +class StrException: public Exception +{ +private: + const std::string msg; +public: + StrException(std::string s): msg(s) {} + void report(std::ostream& s) const { s << msg << std::endl; } +}; + + +class StdException: public Exception +{ +private: + std::string msg; +public: + StdException(std::exception& e): msg(e.what()) {} + void report(std::ostream& s) const { s << msg << std::endl; } +}; + + +class UnknownException: public Exception +{ +public: + void report(std::ostream& s) const; +}; + + +class ErrnoException: public Exception +{ +private: + int error_number; + std::string doing_what; + +public: + ErrnoException(std::string dw, int errno_=errno): error_number(errno_), doing_what(dw) {} + int get_errno(void) { return error_number; } + void report(std::ostream& s) const; +}; + + +struct NoSuchFileOrDirectory: public ErrnoException { + NoSuchFileOrDirectory(std::string dw): + ErrnoException(dw) {} +}; + +struct ConnectionRefused: public ErrnoException { + ConnectionRefused(std::string dw): + ErrnoException(dw) {} +}; + +struct NoSuchDevice: public ErrnoException { + NoSuchDevice(std::string dw): + ErrnoException(dw) {} +}; + +struct HostUnreachable: public ErrnoException { + HostUnreachable(std::string dw): + ErrnoException(dw) {} +}; + +struct NoDataAvailable: public ErrnoException { + NoDataAvailable(std::string dw): + ErrnoException(dw) {} +}; + +struct BrokenPipe: public ErrnoException { + BrokenPipe(std::string dw): + ErrnoException(dw) {} +}; + +struct Overflow: public ErrnoException { + Overflow(std::string dw): + ErrnoException(dw) {} +}; + +struct InvalidArgument: public ErrnoException { + InvalidArgument(std::string dw): + ErrnoException(dw) {} +}; + +struct WouldBlock: public ErrnoException { + WouldBlock(std::string dw): + ErrnoException(dw) {} +}; + +struct TimedOut: public ErrnoException { + TimedOut(std::string dw): + ErrnoException(dw) {} +}; + +struct IOError: public ErrnoException { + IOError(std::string dw): + ErrnoException(dw) {} +}; + +struct InterruptedSysCall: public ErrnoException { + InterruptedSysCall(std::string dw): + ErrnoException(dw) {} +}; + +struct NoSpace: public ErrnoException { + NoSpace(std::string dw): + ErrnoException(dw) {} +}; + +inline void throw_ErrnoException(std::string dw, int errno_=errno) { + switch (errno_) { + case ENOENT: throw NoSuchFileOrDirectory(dw); + case ECONNREFUSED: throw ConnectionRefused(dw); + case ENODEV: throw NoSuchDevice(dw); + case EHOSTUNREACH: throw HostUnreachable(dw); +#ifdef ENODATA +// FreeBSD doesn't have ENODATA + case ENODATA: throw NoDataAvailable(dw); +#endif + case EPIPE: throw BrokenPipe(dw); + case EOVERFLOW: throw Overflow(dw); + case EINVAL: throw InvalidArgument(dw); + case EWOULDBLOCK: throw WouldBlock(dw); + case ETIMEDOUT: throw TimedOut(dw); + case EIO: throw IOError(dw); + case EINTR: throw InterruptedSysCall(dw); + case ENOSPC: throw NoSpace(dw); + default: throw ErrnoException(dw,errno_); + } +} + + +#define RETHROW_MISC_EXCEPTIONS \ +catch(pbe::Exception& E) { \ + throw; \ +} \ +catch(std::exception& e) { \ + throw pbe::StdException(e); \ +} \ +catch(const char* s) { \ + throw pbe::StrException(s); \ +} \ +catch(std::string s) { \ + throw pbe::StrException(s); \ +} \ +catch(...) { \ + throw pbe::UnknownException();\ +} + + +}; + + +#endif diff --git a/extras/anytermd/libpbe/include/FileDescriptor.hh b/extras/anytermd/libpbe/include/FileDescriptor.hh new file mode 100644 index 0000000000..07babe7a19 --- /dev/null +++ b/extras/anytermd/libpbe/include/FileDescriptor.hh @@ -0,0 +1,864 @@ +// FileDescriptor.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2006-2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_FileDescriptor_hh +#define libpbe_FileDescriptor_hh + +#include "Exception.hh" +#include "missing_syscalls.hh" +#include "compiler_magic.hh" + +#include +#include +#include + +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + + +namespace pbe { + +class FileDescriptor: public boost::noncopyable { +// C++ wrapper around the traditional POSIX file descriptor. +// Some methods are simple wrappers around a standard POSIX library +// function. In addition there is a rich set of read/write methods and +// an easy to use wrapper for select(). +// In all cases, exceptions are used to report error conditions. + +protected: + const int fd; + const std::string fn; + const bool close_on_destr; + +public: + enum open_mode_t { read_only=O_RDONLY, read_write=O_RDWR, write_only=O_WRONLY, + create=O_WRONLY|O_CREAT }; + + FileDescriptor(std::string fn_, open_mode_t open_mode): + // Constructor that opens a file given a file name and an access mode. + // The file will be closed when the FileDescriptor is destructed. + // I set close-on-exec for all FileDescriptors, since this seems like the + // sensible default for me. + fd(open(fn_.c_str(),open_mode,0666)), + fn('"'+fn_+'"'), + close_on_destr(true) + { + if (fd==-1) { + throw_ErrnoException("open("+fn+")"); + } + // Set close-on-exec. + // There is a race condition here: if another thread has forked + // between the calls to open and fcntl, it will get the fd, which is + // bad. It may be fixable in the future if we get O_CLOEXEC. + // See http://lwn.net/Articles/236486/. + int rc = fcntl(fd,F_SETFD,FD_CLOEXEC); + if (rc==-1) { + throw_ErrnoException("fcntl("+fn+",F_SETFD,FD_CLOEXEC)"); + } + } + + FileDescriptor(int fd_, bool close_on_destr_=true): + // Constructor that takes an existing file descriptor. + // The file will be closed when the FileDescriptor is destructed + // unless the optional close_on_destr_ parameter is false. + fd(fd_), + fn("fd"+boost::lexical_cast(fd)), + close_on_destr(close_on_destr_) + { + } + + FileDescriptor(int fd_, std::string fn_, bool close_on_destr_=true): + // Constructor that takes an existing file descriptor. + // A name, for use in error messages, is supplied. + // The file will be closed when the FileDescriptor is destructed + // unless the optional close_on_destr_ parameter is false. + fd(fd_), + fn(fn_), + close_on_destr(close_on_destr_) + { + } + + ~FileDescriptor() { + if (close_on_destr) { + int rc = ::close(fd); + if (rc==-1) { + //throw_ErrnoException("close("+fn+")"); + // Don't throw exceptions from destructors, in case the destructor is being + // called during exception processing. + // TODO need a better fix for this. + } + } + } + + + void close() { + // Call close. This is only necessary if the user wants to close the fd before + // it goes out of scope for some reason; further operations on it are obviously + // not allowed. It also has the benefit of checking the return value, which + // the destructor does not do. + int rc = ::close(fd); + if (rc==-1) { + throw_ErrnoException("close("+fn+")"); + } + } + + + // + // Various implementation of READ: + // + + size_t read(char* buf, size_t max_bytes) { + // Directly provides almost the functionality of the read system call. + // At most max_bytes are read into the buffer. Fewer bytes may be + // read at EOF, when a socket has less data waiting, etc. The number + // of bytes actually read is returned. + // If EINTR is returned, the call is retried. + while (1) { + ssize_t n = ::read(fd,buf,max_bytes); + if (n==-1) { + if (errno==EINTR) { + continue; + } + throw_ErrnoException("read("+fn+")"); + } + return n; + } + } + + std::string read(size_t max_bytes) { + // Provides the functionality of the read sytem call, but returns + // the data in a std::string. At most max_bytes are read and returned. + // Fewer bytes may be read at EOF, when a socket has less data waiting, + // etc. + boost::scoped_array buf(new char[max_bytes]); + size_t bytes = read(buf.get(),max_bytes); + return std::string(buf.get(),bytes); + } + + class EndOfFile: public StrException { + public: + EndOfFile(): StrException("EOF") {}; + }; + + void readall(char* buf, size_t bytes) { + // Reads exactly bytes bytes into the buffer, making more than one + // call to read as necessary. Throws EOF if end-of-file is reached + // before the required number of bytes has been read. + while (bytes>0) { + size_t n = read(buf,bytes); + if (n==0) { + throw EndOfFile(); + } + buf += n; + bytes -= n; + } + } + + std::string readall(size_t bytes) { + // Reads exactly bytes bytes from the file descriptor and returns them + // as a std::string, making more than one call to read as necessary. + // Throws EOF if end-of-file is reached before the required number of + // bytes has been read. + boost::scoped_array buf(new char[bytes]); + readall(buf.get(),bytes); + return std::string(buf.get(),bytes); + } + + size_t readmax(char* buf, size_t bytes) { + // Reads exactly bytes bytes into the buffer, making more than one + // call to read as necessary, unless end-of-file is reached before + // the required number of bytes has been read, in which case all the + // available bytes are read. The number of bytes read is returned. + size_t bytes_read = 0; + while (bytes>0) { + size_t n = read(buf,bytes); + if (n==0) { + return bytes_read; + } + bytes_read += n; + buf += n; + bytes -= n; + } + return bytes_read; + } + + std::string readmax(size_t bytes) { + // Reads exactly bytes bytes from the file descriptor and returns them + // as a std::string, making more than one call to read as necessary, + // unless end-of-file is reached before the required number of bytes + // has been read, in which case all the available bytes are read. + boost::scoped_array buf(new char[bytes]); + size_t bytes_read = readmax(buf.get(),bytes); + return std::string(buf.get(),bytes_read); + } + + std::string readsome(void) { + // Reads an unspecified number of bytes from the file descriptor and + // returns them as a std::string. Use this call if you want to + // process the contents of the file in chunks and don't care about + // the chunk size. + // Currently returns an empty string at EOF; should it throw EndOfFile? + // (No - readall() below relies on it returning an empty string.) + char buf[BUFSIZ]; + int bytes = read(buf,BUFSIZ); + return std::string(buf,bytes); + } + + std::string readall() { + // Reads everything from the file descriptor until no more data is available + // (i.e. end of file) + std::string s; + std::string some; + while ((some=readsome()) != "") { + s += some; + } + return s; + } + + template + void binread(T& t) { + // Read and return a thing of type T in binary format. + char* ptr = reinterpret_cast(&t); + readall(ptr,sizeof(t)); + } + + template + T binread(void) { + // Read and return a thing of type T in binary format. + T t; + binread(t); + return t; + } + + template + void binread_at(off_t pos, T& t) { + // Read and return a thing of type T in binary format at position pos in the file. + seek(pos); + binread(t); + } + + std::string read_until_idle(float timeout) { + // Read something, and then keep reading until nothing more has been + // read for at least timeout. + std::string s = readsome(); + while (wait_until(readable(),timeout)) { + s += readsome(); + } + return s; + } + + void set_nonblocking() { + // Calls fcntl to make an open fd non-blocking on subsequent reads and writes. + int flags = fcntl(fd,F_GETFL); + flags |= O_NONBLOCK; + int rc = fcntl(fd,F_SETFL,flags); + if (rc==-1) { + throw_ErrnoException("fcntl("+fn+",F_SETFL,|O_NONBLOCK)"); + } + } + + void set_blocking() { + // Calls fcntl to make an open fd blocking on subsequent reads and writes. + int flags = fcntl(fd,F_GETFL); + flags &=~ O_NONBLOCK; + int rc = fcntl(fd,F_SETFL,flags); + if (rc==-1) { + throw_ErrnoException("fcntl("+fn+",F_SETFL,&~O_NONBLOCK)"); + } + } + + class scoped_nonblocking { + FileDescriptor& fd; + public: + scoped_nonblocking(FileDescriptor& fd_): fd(fd_) { + fd.set_nonblocking(); + } + ~scoped_nonblocking() { + fd.set_blocking(); // FIXME maybe it was already nonblocking! + } + }; + + size_t try_read(char* buf, size_t max_bytes, bool& readable) { + // Directly provides the functionality of the read system call. + // At most max_bytes are read into the buffer. Fewer bytes may be + // read at EOF, when a socket has less data waiting, etc. Zero bytes + // will be read if no data is waiting and the operation would block. + // The number of bytes actually read is returned, and readable is + // set to indicate whether the file was readable. (If the return + // value is zero and readable is true, we're at EOF.) + scoped_nonblocking nb(*this); + ssize_t n = ::read(fd,buf,max_bytes); + // FIXME what about EINTR? + if (n==-1) { + if (errno==EAGAIN) { + readable = false; + return 0; + } + throw_ErrnoException("read("+fn+")"); + } + readable = true; + return n; + } + + bool try_readall(char* buf, size_t bytes) { + // Try to read exactly bytes bytes into the buffer, making more than one + // call to read as necessary. Throws EOF if end-of-file is reached + // before the required number of bytes has been read. Returns false if + // the operation would block. (Hmm, data will be discarded if a partial + // read completes but a subsequent full read would block.) + while (bytes>0) { + bool readable; + size_t n = try_read(buf,bytes,readable); + if (!readable) { + return false; + } + if (n==0) { + throw EndOfFile(); + } + buf += n; + bytes -= n; + } + return true; + } + + template + bool try_binread(T& t) { + // Try to read a thing of type T in binary format. If the operation would block + // (e.g. a serial port or socket with insifficient data currently available) + // return false immediately. + char* ptr = reinterpret_cast(&t); + return try_readall(ptr,sizeof(t)); + } + + std::string try_readsome(void) { + // Tries to reads an unspecified number of bytes from the file descriptor and + // returns them as a std::string. If nothing can be read at present, returns an + // empty string. + char buf[BUFSIZ]; + bool readable; + int bytes = try_read(buf,BUFSIZ,readable); + return std::string(buf,bytes); + } + + // + // Various implementations of WRITE: + // + + size_t write(const char* buf, size_t max_bytes) PBE_WARN_RESULT_IGNORED { + // Directly provides the functionality of the write system call. + // At most max_bytes are written from the buffer. Fewer bytes may be + // written under various circumstances. The number of bytes actually + // written is returned. + ssize_t n = ::write(fd,buf,max_bytes); + if (n==-1) { + throw_ErrnoException("write("+fn+")"); + } + return n; + } + + size_t write(std::string s) PBE_WARN_RESULT_IGNORED { + // Provides the functionality of the write system call, but with the data + // coming from a std::string. Not all of the data may be written under + // various circumstances. The number of bytes actually written is + // returned. + return write(s.data(),s.length()); + } + + void writeall(const char* buf, size_t bytes) { + // Write all bytes bytes from buf, making repeated calls to write + // as necessary. + while (bytes>0) { + ssize_t n = write(buf,bytes); + if (n==-1) { + throw_ErrnoException("write("+fn+")"); + } + // What happens if 0 bytes are written? + buf += n; + bytes -= n; + } + } + + void writeall(std::string s) { + // Write all of string s, making repeated calls to write as necessary. + writeall(s.data(),s.length()); + } + + template + void binwrite(const T& t) { + // Write a thing of type T in binary format. + const char* ptr = reinterpret_cast(&t); + writeall(ptr,sizeof(T)); + } + + template + void binwrite_at(off_t pos, const T& t) { + // Write a thing of type T in binary format at position pos in the file. + seek(pos); + binwrite(t); + } + + void writeallv2(const char* buf1, size_t bytes1, const char* buf2, size_t bytes2) { + // Writes all bytes from buf1 and all bytes from buf2 using the writev system call. + iovec v[2]; + v[0].iov_base = const_cast(buf1); + v[0].iov_len = bytes1; + v[1].iov_base = const_cast(buf2); + v[1].iov_len = bytes2; + size_t bytes_written = 0; + while (1) { + int rc = writev(fd,v,2); + if (rc==-1) { + throw_ErrnoException("writev("+fn+")"); + } + bytes_written += rc; + if (bytes_written == bytes1+bytes2) { + return; + } + if (bytes_written >= bytes1) { + break; + } + v[0].iov_base = static_cast(v[0].iov_base)+rc; + v[0].iov_len -= rc; + } + writeall(buf2+bytes_written-bytes1, bytes1+bytes2-bytes_written); + } + + + int dup() { + // Directly provides the functionality of the dup() system call. + // The duplicate file descriptor is returned as an int, not as a FileDesriptor + // object. A FileDescriptor object can of course be constructed from the int. + int d = ::dup(fd); + if (d==-1) { + throw_ErrnoException("dup("+fn+")"); + } + return d; + } + +// The following code allows a C++ stream to be created from a +// file descriptor. This relies on a non-standard extension in +// GNU libstdc++. Not only is this non-standard, but it has +// changed slightly in different versions of the library. +// The following is known to work with g++ 3.4.4, and hopefully +// newer versions. If you have an earlier version that works +// with this please let me know. +#if __GLIBCXX__ >= 20050421 +#define LIBPBE_HAS_FILEDESCRIPTOR_STREAMS + + class istream: public std::istream { + private: + __gnu_cxx::stdio_filebuf fbuf; + public: + istream(FileDescriptor& fd, int bufsize=BUFSIZ): + fbuf(fd.dup(), std::ios::in, bufsize) + { + rdbuf(&fbuf); + } + }; + + class ostream: public std::ostream { + private: + __gnu_cxx::stdio_filebuf fbuf; + public: + ostream(FileDescriptor& fd, int bufsize=BUFSIZ): + fbuf(fd.dup(), std::ios::out, bufsize) + { + rdbuf(&fbuf); + } + }; + +// The following should work with gcc 3.3. +// Note that the libstdc++ version symbol has changed from __GLIBCPP__ +// to __GLIBCXX__ at some point. +// If you have an earlier or later version that works with this +// please let me know. +#elif (__GLIBCPP__ >= 20040214) && (__GLIBCPP__ <= 20050503) +#define LIBPBE_HAS_FILEDESCRIPTOR_STREAMS + + class istream: public std::istream { + private: + __gnu_cxx::stdio_filebuf fbuf; + public: + istream(FileDescriptor& fd, int bufsize=BUFSIZ): + std::istream(&fbuf), + fbuf(fd.dup(), std::ios::in, true, bufsize) + // (I'm concerned about the order of construction here) + {} + }; + + class ostream: public std::ostream { + private: + __gnu_cxx::stdio_filebuf fbuf; + public: + ostream(FileDescriptor& fd, int bufsize=BUFSIZ): + std::ostream(&fbuf), + fbuf(fd.dup(), std::ios::out, true, bufsize) + // ditto + {} + }; + +#endif +// Information about and/or patches for other versions of g++ are welcome. + + + void set_nodelay() { + // Calls setscokopt to disable Nagle's algorithm for this socket. + int flag = 1; + int rc = setsockopt(fd, IPPROTO_TCP, TCP_NODELAY, (char *) &flag, sizeof(int)); + if (rc==-1) { + throw_ErrnoException("setsockopt("+fn+",TCP_NODELAY)"); + } + } + + + struct in_addr get_peer_ip_addr() { + // For an fd that is a socket, finds the IP address of the other end of the + // connection. + // For a non-socket, returns the loopback address. + struct sockaddr_in client_addr; + socklen_t client_addr_len = sizeof(client_addr); + int rc = getpeername(fd, (struct sockaddr*)&client_addr, + &client_addr_len); + if (rc==-1) { + if (errno==ENOTSOCK) { + struct in_addr a; + a.s_addr = htonl(INADDR_LOOPBACK); + return a; + } else { + throw_ErrnoException("getpeername("+fn+")"); + } + } + if (client_addr.sin_family!=AF_INET) { + throw "socket is not an AF_INET socket"; + } + return client_addr.sin_addr; + } + + + template + int ioctl(int request, T* argp) { + // Directly provides the functionality of the ioctl system call. + int rc = ::ioctl(fd, request, reinterpret_cast(argp)); + if (rc==-1) { + throw_ErrnoException("ioctl("+boost::lexical_cast(request)+","+fn+")"); + } + return rc; + } + + template + int ioctl(int request, T& arg) { + return ioctl(request,&arg); + } + + int ioctl(int request) { + // ioctl with no data. + return ioctl(request,NULL); + } + + + template + int try_ioctl(int request, T* argp, bool& done) { + // Directly provides the functionality of the ioctl system call, non-blocking; + // done is false if the ioctl would block. + int rc = ::ioctl(fd, request, reinterpret_cast(argp)); + if (rc==-1) { + if (errno==EAGAIN) { + done = false; + return 0; + } + throw_ErrnoException("ioctl("+boost::lexical_cast(request)+","+fn+")"); + } + done = true; + return rc; + } + + template + int try_ioctl(int request, T& arg, bool& done) { + return try_ioctl(request,&arg,done); + } + + int try_ioctl(int request, bool& done) { + // try_ioctl with no data. + return try_ioctl(request,NULL,done); + } + + + void get_sigio(void) + // Ask the kernel to send this process or thread SIGIO when this fd becomes + // readable or writable. + { + int r = fcntl(fd,F_SETOWN,getpid()); // should be gettid + if (r==-1) { + throw_ErrnoException("fcntl("+fn+",F_SETOWN)"); + } + r = fcntl(fd,F_SETFL,O_ASYNC); + if (r==-1) { + throw_ErrnoException("fnctl("+fn+",F_SETFL,O_ASYNC)"); + } + } + + + enum whence_t {seek_set=SEEK_SET, seek_cur=SEEK_CUR, seek_end=SEEK_END}; + + off_t seek(off_t offset, whence_t whence = seek_set) { + // Directly provides the functionality of the lseek() system call. + // By default the offset is interpretted relative to the start of the file. + off_t r = lseek(fd,offset,whence); + if (r==(off_t)-1) { + throw_ErrnoException("lseek("+fn+")"); + } + return r; + } + + + off_t getpos() { + return seek(0,seek_cur); + } + + off_t file_length() { + // Returns the length of the file. This is done by seeking to the end. + off_t pos = getpos(); + off_t len = seek(0,seek_end); + seek(pos); + return len; + } + + + void* mmap(size_t length, open_mode_t open_mode, off_t offset = 0, bool copy_on_write=false) { + int prot = 0; + if (open_mode == read_only || open_mode == read_write) { + prot |= PROT_READ; + } + if (open_mode == write_only || open_mode == read_write) { + prot |= PROT_WRITE; + } + void* ptr = ::mmap(0, length, prot, copy_on_write ? MAP_PRIVATE : MAP_SHARED, fd, offset); + if (ptr==MAP_FAILED) { + throw_ErrnoException("mmap("+fn+")"); + } + return ptr; + } + + + void truncate(off_t length) { + int rc = ftruncate(fd,length); + if (rc==-1) { + throw_ErrnoException("truncate("+fn+")"); + } + } + + + void sync() { + int rc = ::fsync(fd); + if (rc==-1) { + throw_ErrnoException("fsync("+fn+")"); + } + } + +#if ! (defined __FreeBSD__ || defined __OpenBSD__ || defined __APPLE__) +// These systems don't have fdatasync + void datasync() { + int rc = ::fdatasync(fd); + if (rc==-1) { + throw_ErrnoException("fdatasync("+fn+")"); + } + } +#endif + + +private: + + struct select_item_readable { + const int fd; + select_item_readable(int fd_): fd(fd_) {} + }; + + struct select_item_writeable { + const int fd; + select_item_writeable(int fd_): fd(fd_) {} + }; + + struct select_item_exception { + const int fd; + select_item_exception(int fd_): fd(fd_) {} + }; + + class select_info { + private: + fd_set readfds; + fd_set writefds; + fd_set exceptfds; + int max_fd; + + public: + void clear() { + FD_ZERO(&readfds); + FD_ZERO(&writefds); + FD_ZERO(&exceptfds); + max_fd=0; + } + + void set_readable(int fd) { + FD_SET(fd,&readfds); + max_fd=std::max(max_fd,fd); + } + + void set_writeable(int fd) { + FD_SET(fd,&writefds); + max_fd=std::max(max_fd,fd); + } + + void set_exception(int fd) { + FD_SET(fd,&exceptfds); + max_fd=std::max(max_fd,fd); + } + + select_info(select_item_readable i) { + clear(); + set_readable(i.fd); + } + + select_info(select_item_writeable i) { + clear(); + set_writeable(i.fd); + } + + select_info(select_item_exception i) { + clear(); + set_exception(i.fd); + } + + friend int wait_until_(FileDescriptor::select_info, struct timeval*); + }; + + +public: + + select_item_readable readable() const { + return select_item_readable(fd); + } + select_item_writeable writeable() const { + return select_item_writeable(fd); + } + select_item_exception exception() const { + return select_item_exception(fd); + } + + friend select_info operator||(select_info, select_item_readable); + friend select_info operator||(select_info, select_item_writeable); + friend select_info operator||(select_info, select_item_exception); + friend int wait_until_(FileDescriptor::select_info, struct timeval*); + friend int wait_until(FileDescriptor::select_info); + friend int wait_until(FileDescriptor::select_info, float); + + + bool operator==(int rhs) const { + return fd==rhs; + } + + bool operator==(const FileDescriptor& rhs) const { + return fd==rhs.fd; + } + + +}; + + +inline FileDescriptor::select_info +operator||(FileDescriptor::select_info lhs, + FileDescriptor::select_item_readable rhs) { + FileDescriptor::select_info i = lhs; + i.set_readable(rhs.fd); + return i; +} + +inline FileDescriptor::select_info +operator||(FileDescriptor::select_info lhs, + FileDescriptor::select_item_writeable rhs) { + FileDescriptor::select_info i = lhs; + i.set_writeable(rhs.fd); + return i; +} + +inline FileDescriptor::select_info +operator||(FileDescriptor::select_info lhs, + FileDescriptor::select_item_exception rhs) { + FileDescriptor::select_info i = lhs; + i.set_exception(rhs.fd); + return i; +} + + +inline int wait_until_(FileDescriptor::select_info i, struct timeval* tv) { + int rc; + do { + rc = select(i.max_fd+1, &i.readfds, &i.writefds, &i.exceptfds, tv); + if (rc==-1) { + if (errno==EINTR) { + continue; + } + throw_ErrnoException("select()"); + } + } while (0); + + if (rc==0) { + return -1; + } + for (int n=0; n<=i.max_fd; n++) { + if (FD_ISSET(n,&i.readfds) || FD_ISSET(n,&i.writefds) + || FD_ISSET(n,&i.exceptfds)) { + return n; + } + } + throw "not reached"; +} + +inline int wait_until(FileDescriptor::select_info i) { + // Returns a file descriptor number that changed. + return wait_until_(i,NULL); +} + +inline int wait_until(FileDescriptor::select_info i, + float timeout) { + // Returns -1 if timed out, else a file descriptor number that changed. + struct timeval tv; + float timeout_whole; + float timeout_frac; + timeout_frac = modff(timeout, &timeout_whole); + tv.tv_sec = static_cast(timeout_whole); + tv.tv_usec = static_cast(1000000.0*timeout_frac); + return wait_until_(i,&tv); +} + +}; + + +#endif diff --git a/extras/anytermd/libpbe/include/FileMonitor.hh b/extras/anytermd/libpbe/include/FileMonitor.hh new file mode 100644 index 0000000000..a7bc788993 --- /dev/null +++ b/extras/anytermd/libpbe/include/FileMonitor.hh @@ -0,0 +1,81 @@ +// include/FileMonitor.cc +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef pbe_FileMonitor_hh +#define pbe_FileMonitor_hh + +#include "FileDescriptor.hh" +#include "Exception.hh" +#include "missing_syscalls.hh" + +#include + + +namespace pbe { + +#if defined(PBE_HAVE_INOTIFY_INIT) && defined(PBE_HAVE_INOTIFY_ADD_WATCH) + +static inline int check_inotify_init() { + int rc = ::inotify_init(); + if (rc==-1) { + pbe::throw_ErrnoException("inotify_init()"); + } + return rc; +} + +class FileMonitor { + + int fdnum; + pbe::FileDescriptor fd; + +public: + FileMonitor(std::string fn): + fdnum(check_inotify_init()), + fd(fdnum,"inotify handle") + { + int rc = ::inotify_add_watch(fdnum,fn.c_str(),IN_MODIFY); + if (rc==-1) { + pbe::throw_ErrnoException("inotify_add_watch()"); + } + } + + void wait_until_modified() { + struct inotify_event ev; + fd.binread(ev); + } + + bool wait_until_modified_or_timeout(float timeout) { + bool readable = fd.wait_until_readable_or_timeout(timeout); + if (!readable) { + return false; + } + struct inotify_event ev; + fd.binread(ev); + return true; + } + + +}; + +#endif + +}; + + +#endif + diff --git a/extras/anytermd/libpbe/include/FileType.hh b/extras/anytermd/libpbe/include/FileType.hh new file mode 100644 index 0000000000..3711c06c69 --- /dev/null +++ b/extras/anytermd/libpbe/include/FileType.hh @@ -0,0 +1,104 @@ +// src/FileType.hh +// This file is part of libpbe; see http://decimail.org +// (C) 2006 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +// Determine the type of a pathname: regular file, directory, symlink, etc. + + +#ifndef libpbe_FileType_hh +#define libpbe_FileType_hh + +#include +#include +#include + +#include + +#include "Exception.hh" + + +namespace pbe { + + enum FileType { file, directory, symlink, does_not_exist, other }; + + static inline FileType filetype_from_stat(struct stat& stat_buf) + { + if (S_ISDIR(stat_buf.st_mode)) { + return directory; + } else if (S_ISREG(stat_buf.st_mode)) { + return file; + } else if (S_ISLNK(stat_buf.st_mode)) { + return symlink; + } else { + return other; + } + } + + inline FileType get_filetype(std::string pathname, bool report_noexist=false) + { + struct stat stat_buf; + int ret = stat(pathname.c_str(),&stat_buf); + if (ret==-1) { + if (report_noexist && errno==ENOENT) { + return does_not_exist; + } + throw_ErrnoException("stat("+pathname+")"); + } + return filetype_from_stat(stat_buf); + } + + inline FileType get_link_filetype(std::string pathname, bool report_noexist=false) + { + struct stat stat_buf; + int ret = lstat(pathname.c_str(),&stat_buf); + if (ret==-1) { + if (report_noexist && errno==ENOENT) { + return does_not_exist; + } + throw_ErrnoException("lstat("+pathname+")"); + } + return filetype_from_stat(stat_buf); + } + + inline FileType get_fd_filetype(int fd) + { + struct stat stat_buf; + int ret = fstat(fd,&stat_buf); + if (ret==-1) { + throw_ErrnoException("fstat()"); + } + return filetype_from_stat(stat_buf); + } + + inline bool file_exists(std::string pathname) + { + struct stat stat_buf; + int ret = stat(pathname.c_str(),&stat_buf); + if (ret==-1) { + if (errno==ENOENT) { + return false; + } + throw_ErrnoException("stat("+pathname+")"); + } + return true; + } + +}; + +#endif + diff --git a/extras/anytermd/libpbe/include/Futex.hh b/extras/anytermd/libpbe/include/Futex.hh new file mode 100644 index 0000000000..a907c2b8eb --- /dev/null +++ b/extras/anytermd/libpbe/include/Futex.hh @@ -0,0 +1,117 @@ +// include/Futex.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_Futex_hh +#define libpbe_Futex_hh + +// The file implements a Futex class, which is a simple wrapper around +// the futex syscall. +// Futex is supported on Linux 2.6 but not older kernels. If you want to +// run on a 2.4 kernel you need to compile with SUPPORT_LINUX_2_4 defined. +// This will supress the content of this file and the things that use it +// will fall back to something else (as they do on non-Linux platforms). + +#if defined(__linux__) && !defined(SUPPORT_LINUX_2_4) + +#define PBE_HAVE_FUTEX + +#include +#include +#include + +#include + +#include "missing_syscalls.hh" +#include "Exception.hh" + +namespace pbe { + + +template +struct Futex { + + BOOST_STATIC_ASSERT(sizeof(INT_T)==sizeof(int)); + + int* const valptr; + + Futex(INT_T& val): valptr(reinterpret_cast(&val)) {} + + ~Futex() {} + + void wait(INT_T expected) { + int r = futex(valptr, FUTEX_WAIT, + static_cast(expected), NULL, NULL, 0); + if (r==-1) { + // There are various legitimate errors that we'll throw an exception for. + // But there are also these, which are expected in normal operations: + // EWOULDBLOCK - valptr != expected + // EINTR - signal or spurious wakeup + // We might want to return something indicating that one of these conditions + // had occurred, but it seems that the mutex algorithm doesn't need this. + switch (errno) { + case EWOULDBLOCK: return; + case EINTR: return; + default: throw_ErrnoException("futex(FUTEX_WAIT)"); + } + } + } + + bool timed_wait(INT_T expected, const timespec& timeout) { + // Returns false if timeout reached. + int r = futex(valptr, FUTEX_WAIT, + static_cast(expected), &timeout, NULL, 0); + if (r==-1) { + // As above. + switch (errno) { + case EWOULDBLOCK: return true; + case EINTR: return true; + case ETIMEDOUT: return false; + default: throw_ErrnoException("futex(FUTEX_WAIT)"); + } + } + return true; + } + + bool timed_wait(INT_T expected, float timeout) { + // Returns false if timeout reached. + struct timespec ts; + float timeout_whole; + float timeout_frac; + timeout_frac = modff(timeout, &timeout_whole); + ts.tv_sec = static_cast(timeout_whole); + ts.tv_nsec = static_cast(1e9*timeout_frac); + return timed_wait(expected,ts); + } + + // Wake up to n waiters; returns the number woken. + unsigned int wake(int n=1) { + int r = futex(valptr, FUTEX_WAKE, n, NULL, NULL, 0); + if (r==-1) { + throw_ErrnoException("futex(FUTEX_WAIT)"); + } + return r; + } +}; + + +}; + + +#endif +#endif + diff --git a/extras/anytermd/libpbe/include/Gunzipper.hh b/extras/anytermd/libpbe/include/Gunzipper.hh new file mode 100644 index 0000000000..3658031813 --- /dev/null +++ b/extras/anytermd/libpbe/include/Gunzipper.hh @@ -0,0 +1,92 @@ +// include/Gunzipper.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef pbe_Gunzipper_hh +#define pbe_Gunzipper_hh + +#include + +#include + + +namespace pbe { + + +class Gunzipper { + + z_stream_s gz; + +public: + + Gunzipper() { + gz.zalloc = Z_NULL; + gz.zfree = Z_NULL; + gz.opaque = Z_NULL; + gz.next_in = Z_NULL; + gz.avail_in = 0; + int r = inflateInit2(&gz, 15+32); // 15 = windowbits; this is the default; + // adding 32 (ARGH!) magically makes it + // recognise both zlib and gzip formats + // (you didn't even know they differed, right?) + switch (r) { + case Z_MEM_ERROR: throw std::bad_alloc(); + case Z_VERSION_ERROR: throw "Z_VERSION_ERROR"; + } + } + + ~Gunzipper() { + inflateEnd(&gz); + } + + struct InvalidData {}; + + std::string operator()(const char* in, size_t in_bytes) { + gz.next_in = reinterpret_cast(const_cast(in)); + gz.avail_in = in_bytes; + std::string out; + while (gz.avail_in>0 || gz.avail_out==0) { + char buffer[4096]; + gz.next_out = reinterpret_cast(buffer); + gz.avail_out = sizeof(buffer); + int r = inflate(&gz,Z_SYNC_FLUSH); + switch (r) { + case Z_NEED_DICT: throw "Z_NEED_DICT"; + case Z_DATA_ERROR: throw InvalidData(); + case Z_STREAM_ERROR: throw InvalidData(); + case Z_MEM_ERROR: throw std::bad_alloc(); + } + out.append(buffer,sizeof(buffer)-gz.avail_out); + if (r==Z_STREAM_END) { + break; + } + } + return out; + } + + std::string operator()(std::string in) { + return operator()(in.data(),in.length()); + } +}; + + + +}; + + +#endif + diff --git a/extras/anytermd/libpbe/include/HPtime.hh b/extras/anytermd/libpbe/include/HPtime.hh new file mode 100644 index 0000000000..c25f15971c --- /dev/null +++ b/extras/anytermd/libpbe/include/HPtime.hh @@ -0,0 +1,61 @@ +// include/HPtime.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2007-2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +// High-precision time. +// Convertible to/from a double, representing the time in seconds since the epoch. +// Resolution is whatever gettimeofday provides, which is at best microseconds. + + +#ifndef pbe_HPtime_hh +#define pbe_HPtime_hh + +#include +#include + + +namespace pbe { + +class HPtime { + +private: + double dt; + +public: + HPtime(): dt(0) {} + + HPtime(const HPtime& t): dt(t.dt) {} + + HPtime(double t): dt(t) {} + + operator double() const { return dt; } + + static HPtime now() { + struct timeval tv; + gettimeofday(&tv,NULL); + return HPtime(tv.tv_sec + tv.tv_usec/1e6); + } + +}; + + +}; + + +#endif + diff --git a/extras/anytermd/libpbe/include/HttpAuthenticator.hh b/extras/anytermd/libpbe/include/HttpAuthenticator.hh new file mode 100644 index 0000000000..713e73cba7 --- /dev/null +++ b/extras/anytermd/libpbe/include/HttpAuthenticator.hh @@ -0,0 +1,39 @@ +// HttpAuthenticator.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2005 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#ifndef HttpAuthenticator_hh +#define HttpAuthenticator_hh + +#include + + +class HttpAuthenticator { + +public: + class NotAuthenticated {}; + + virtual ~HttpAuthenticator() {} + + virtual std::string operator()(std::string credentials) const; + virtual void basic_auth(std::string username, std::string password) const = 0; + +}; + + +#endif diff --git a/extras/anytermd/libpbe/include/HttpClient.hh b/extras/anytermd/libpbe/include/HttpClient.hh new file mode 100644 index 0000000000..bc4bd9d1ca --- /dev/null +++ b/extras/anytermd/libpbe/include/HttpClient.hh @@ -0,0 +1,63 @@ +#ifndef libpbe_HttpClient_hh +#define libpbe_HttpClient_hh + +// include/HttpClient.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#include "URI.hh" +#include "HttpResponse.hh" + +#include + + +namespace pbe { + + +class HttpClient { + + std::string user_agent; + + template + void get_process_save(const URI& uri, std::string fn, int redirect_hops); + +public: + HttpClient(std::string user_agent_="libpbe::HttpClient"): + user_agent(user_agent_) + {} + + HttpResponse get(const URI& uri, int redirect_hops=10); + + void get_save(const URI& uri, std::string fn, int redirect_hops=10); + void get_save_with_etag(const URI& uri, std::string fn, int redirect_hops=10); + + void get_gunzip_save(const URI& uri, std::string fn, int redirect_hops=10); + void get_gunzip_save_with_etag(const URI& uri, std::string fn, int redirect_hops=10); + +#ifdef HAVE_BZIP + void get_bunzip_save(const URI& uri, std::string fn, int redirect_hops=10); + void get_bunzip_save_with_etag(const URI& uri, std::string fn, int redirect_hops=10); +#endif +}; + + +}; + + +#endif + diff --git a/extras/anytermd/libpbe/include/HttpDaemon.hh b/extras/anytermd/libpbe/include/HttpDaemon.hh new file mode 100644 index 0000000000..da302ecacd --- /dev/null +++ b/extras/anytermd/libpbe/include/HttpDaemon.hh @@ -0,0 +1,65 @@ +// HttpDaemon.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2005-2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef HttpDaemon_hh +#define HttpDaemon_hh + +#include "Daemon.hh" +#include "HttpRequest.hh" +#include "HttpResponse.hh" +#include "HttpAuthenticator.hh" +#include "FileDescriptor.hh" + +#include + + +namespace pbe { + + +class HttpDaemon: public Daemon { + +private: + HttpAuthenticator* const authenticator; + const bool keepalive; + +public: + HttpDaemon(short port=80, std::string progname="httpd", std::string user="", + bool keepalive_=true, int max_connections=0, bool accept_local_only=false): + Daemon(port, progname, LOG_LOCAL0, user, "", max_connections, accept_local_only), + authenticator(NULL), + keepalive(keepalive_) {} + + HttpDaemon(HttpAuthenticator& a, short port=80, std::string progname="httpd", + std::string user="", bool keepalive_=true, int max_connections=0, + bool accept_local_only=false): + Daemon(port, progname, LOG_LOCAL0, user, "", max_connections, accept_local_only), + authenticator(&a), keepalive(keepalive_) {} + + void session(pbe::FileDescriptor& in_fd, pbe::FileDescriptor& out_fd); + + virtual void session_start() {} + virtual void handle(const HttpRequest& req, HttpResponse& resp) = 0; + + void authenticate(HttpRequest& req); +}; + + +}; + + +#endif diff --git a/extras/anytermd/libpbe/include/HttpRequest.hh b/extras/anytermd/libpbe/include/HttpRequest.hh new file mode 100644 index 0000000000..9376dbd769 --- /dev/null +++ b/extras/anytermd/libpbe/include/HttpRequest.hh @@ -0,0 +1,79 @@ +// include/HttpRequest.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2006-2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#ifndef libpbe_HttpRequest_hh +#define libpbe_HttpRequest_hh + +#include "URI.hh" + +#include +#include + + +namespace pbe { + + +struct HttpRequest { + std::string method; + std::string host; + std::string abs_path; + std::string query; + std::string http_version; + std::string userinfo; + typedef std::map headers_t; + headers_t headers; + std::string body; + + HttpRequest() {} + + HttpRequest(const URI& uri, std::string method_="GET", std::string http_version_="HTTP/1.1"): + method(method_), + host(uri.host), + abs_path(uri.abs_path), + query(uri.query), + http_version(http_version_), + userinfo(uri.userinfo) + { + headers["Host"]=host; + } + + std::string request_line() const { + return method + +" " + +abs_path + (query.empty() ? std::string() : "?"+query) + +" " + +http_version+"\r\n"; + } + + std::string headers_str() const { + std::string s; + for (headers_t::const_iterator i = headers.begin(); + i != headers.end(); ++i) { + s += i->first + ": " + i->second + "\r\n"; + } + return s; + } + +}; + + +}; + + +#endif diff --git a/extras/anytermd/libpbe/include/HttpResponse.hh b/extras/anytermd/libpbe/include/HttpResponse.hh new file mode 100644 index 0000000000..c0a652e90e --- /dev/null +++ b/extras/anytermd/libpbe/include/HttpResponse.hh @@ -0,0 +1,53 @@ +// HttpResponse.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2005-2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#ifndef libpbe_HttpResponse_hh +#define libpbe_HttpResponse_hh + +#include +#include + +#include "FileDescriptor.hh" + + +namespace pbe { + + +class HttpResponse { + +public: + HttpResponse(): + http_version("HTTP/1.1"), status_code(200), reason_phrase("OK") {} + + HttpResponse(std::string s); + + void send(pbe::FileDescriptor& fd); + + std::string http_version; + int status_code; + std::string reason_phrase; + std::map headers; + std::string body; +}; + + +}; + + +#endif diff --git a/extras/anytermd/libpbe/include/Iconver.hh b/extras/anytermd/libpbe/include/Iconver.hh new file mode 100644 index 0000000000..5dd3c1e8fa --- /dev/null +++ b/extras/anytermd/libpbe/include/Iconver.hh @@ -0,0 +1,192 @@ +// src/Iconvert.hh +// This file is part of libpbe; see http://decimail.org and http://anyterm.org +// (C) 2006 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +// C++ wrapper around iconv. + +// This is not thread-safe in the sense that an Iconver object cannot +// be used safely from multiple threads; each thread must have its own +// object. + +#ifndef libpbe_Iconver_hh +#define libpbe_Iconver_hh + +#include + +#include + +#include + +#include + +#include "Exception.hh" + + +#if defined(__FreeBSD__) || defined(__OpenBSD__) || defined(__sun__) +// Previously __APPLE__ was included in this list; presumably they have +// changed their headers. If you have an older system you may need to put +// it back. +#define ICONV_ARG2_IS_CONST +#endif + + +namespace pbe { + + +enum iconv_errmode { reversible, // Throw if an input character cannot be reversibly converted. + complete, // Throw if the input is not complete. + valid, // Throw if the input is not valid. + permissive // Don't throw. + }; // TODO there are combinations of these.... + +// The template parameter char types don't need to be the actual types of the character sets, +// since iconv just deals with lumps of bytes; they just need to correspond to the types of the +// input and output to operator(). +template +class Iconver { + +public: + Iconver(std::string from_charset, std::string to_charset) { + iconverter = iconv_open(to_charset.c_str(), from_charset.c_str()); + if (iconverter==reinterpret_cast(-1)) { + throw InvalidCharset(); + } + } + + ~Iconver() { + int rc = iconv_close(iconverter); + if (rc==-1) { + // pbe::throw_ErrnoException("iconv_close()"); + // Don't throw an exception from a destructor, in case it has been invoked + // during exception processing. + // (TODO is there a better solution to this?) + } + } + + std::basic_string operator()(std::basic_string i) { + return operator()(i.data(),i.size()); + } + + std::basic_string operator()(const from_char* i, size_t l) { + if (carry.size()) { + std::basic_string s = carry; + s.append(i,l); + carry.clear(); + i = s.data(); + l = s.size(); + } + const size_t bytes_in = sizeof(from_char) * l; + const size_t obuf_sz = l * 2; // do multiple chunks if necessary + const size_t buf_bytes = obuf_sz * sizeof(to_char); + boost::scoped_array obuf (new to_char[obuf_sz]); + std::basic_string o; + +#ifdef ICONV_ARG2_IS_CONST + const char* ip = reinterpret_cast(i); +#else + char* ip = reinterpret_cast(const_cast(i)); +#endif + + size_t in = bytes_in; + + do { + char* op = reinterpret_cast(obuf.get()); + size_t on = buf_bytes; + + int rc = iconv(iconverter, &ip, &in, &op, &on); + if (rc==-1) { + if (errno==E2BIG) { + // Output buffer is full. We'll go around the loop again. + } else if (errno==EILSEQ) { + // An invalid multibyte sequence has been found. + if (errmode==permissive) { + // Skip the offending character and continue. + // (iconv stores any valid converted data from before the error and updates + // the pointers correctly in this case.) + ip += sizeof(from_char); + in -= sizeof(from_char); + } else { + throw InvalidInput(); + } + } else if (errno==EINVAL) { + // An incomplete multibyte sequence has been found at the end of the input. + if (errmode==complete) { + throw IncompleteInput(); + } else { + carry = std::basic_string(reinterpret_cast(ip),in/sizeof(from_char)); + in = 0; + } + } else { + pbe::throw_ErrnoException("iconv()"); + } + } else if (rc>0) { + if (errmode==reversible) { + throw NotReversible(); + } + } + o += std::basic_string(obuf.get(), (buf_bytes - on)/sizeof(to_char)); + } while (in>0); + + return o; + } + + + void flush() { + // Caller believes that the input is complete; throws if a multi-byte character is outstanding. + if (errmode==permissive) { + reset(); + } else if (carry.size()) { + throw IncompleteInput(); + } + } + + + void reset() { + // Clear any outstanding partial multi-byte character. + carry.clear(); + } + + + class InvalidCharset: public pbe::StrException { + public: + InvalidCharset(): pbe::StrException("Invalid character set or unsupported conversion") {} + }; + + class InvalidInput: public pbe::StrException { + public: + InvalidInput(): pbe::StrException("Invalid input to Iconv") {} + }; + + class IncompleteInput: public pbe::StrException { + public: + IncompleteInput(): pbe::StrException("Incomplete multi-byte input to Iconv") {} + }; + + class NotReversible: public pbe::StrException { + public: + NotReversible(): pbe::StrException("Non-reversible input to Iconv") {} + }; + +private: + iconv_t iconverter; + std::basic_string carry; +}; + + +}; + +#endif diff --git a/extras/anytermd/libpbe/include/Line.hh b/extras/anytermd/libpbe/include/Line.hh new file mode 100644 index 0000000000..8467970deb --- /dev/null +++ b/extras/anytermd/libpbe/include/Line.hh @@ -0,0 +1,192 @@ +// include/Line.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_Line_hh +#define libpbe_Line_hh + +#include "Point.hh" +#include "Vector.hh" +#include "Box.hh" + + +namespace pbe { + +template +struct Line { + typedef COORD_T coord_t; + typedef Point point_t; + + point_t a; + point_t b; + + Line() {} + + Line(const Line& other): + a(other.a), b(other.b) + {} + + Line(point_t a_, point_t b_): + a(a_), b(b_) + {} + + bool operator==(const Line& other) const { + return (a==other.a && b==other.b); + } + + bool operator!=(const Line& other) const { + return ! operator==(other); + } +}; + + + +template +bool line_crosses_line(Line u, Line v, Point& cross) +{ + // Do lines u and v cross? If so, find the point where they cross. + // Precondition: the lines must not be points. + // + // Method: + // U = u.a + alpha(u.b-u.a) + // V = v.a + beta(v.b-v.a) + // Solve for U=V + // If 0<=alpha<=1 und 0<=beta<=1 then the lines cross. + // + // u.x = u.a.x + alpha(u.b.x-u.a.x) + // u.y = u.a.y + alpha(u.b.y-u.a.y) + // v.x = v.a.x + beta(v.b.x-v.a.x) + // v.y = v.a.y + beta(v.b.y-v.a.y) + // + // u.a.x + alpha(u.b.x-u.a.x) = v.a.x + beta(v.b.x-v.a.x) + // u.a.y + alpha(u.b.y-u.a.y) = v.a.y + beta(v.b.y-v.a.y) + // + // alpha = ( v.a.x + beta(v.b.x-v.a.x) - u.a.x ) / (u.b.x-u.a.x) provided u.b.x!=u.a.x + // alpha = ( v.a.y + beta(v.b.y-v.a.y) - u.a.y ) / (u.b.y-u.a.y) provided u.b.y!=u.a.y + // ( v.a.x + beta(v.b.x-v.a.x) - u.a.x ) / (u.b.x-u.a.x) = ( v.a.y + beta(v.b.y-v.a.y) - u.a.y ) / (u.b.y-u.a.y) + // ( v.a.x + beta(v.b.x-v.a.x) - u.a.x ) (u.b.y-u.a.y) = ( v.a.y + beta(v.b.y-v.a.y) - u.a.y ) (u.b.x-u.a.x) + // beta.(v.b.x-v.a.x).(u.b.y-u.a.y) + (v.a.x-u.a.x).(u.b.y-u.a.y) = beta.(v.b.y-v.a.y).(u.b.x-u.a.x) + (v.a.y-u.a.y).(u.b.x-u.a.x) + // beta.(v.b.x-v.a.x).(u.b.y-u.a.y) - beta.(v.b.y-v.a.y).(u.b.x-u.a.x) = (v.a.y-u.a.y).(u.b.x-u.a.x) - (v.a.x-u.a.x).(u.b.y-u.a.y) + // beta = ( (v.a.y-u.a.y).(u.b.x-u.a.x) - (v.a.x-u.a.x).(u.b.y-u.a.y) ) / ( (v.b.x-v.a.x).(u.b.y-u.a.y) - (v.b.y-v.a.y).(u.b.x-u.a.x) ) + // + // beta = ( u.a.x + alpha(u.b.x-u.a.x) - v.a.x ) / (v.b.x-v.a.x) + // beta = ( u.a.y + alpha(u.b.y-u.a.y) - v.a.y ) / (v.b.y-v.a.y) + // ( u.a.x + alpha(u.b.x-u.a.x) - v.a.x ) / (v.b.x-v.a.x) = ( u.a.y + alpha(u.b.y-u.a.y) - v.a.y ) / (v.b.y-v.a.y) + // ( u.a.x + alpha(u.b.x-u.a.x) - v.a.x ) (v.b.y-v.a.y) = ( u.a.y + alpha(u.b.y-u.a.y) - v.a.y ) (v.b.x-v.a.x) + // alpha.(u.b.x-u.a.x).(v.b.y-v.a.y) + (u.a.x-v.a.x).(v.b.y-v.a.y) = alpha.(u.b.y-u.a.y).(v.b.x-v.a.x) + (u.a.y-v.a.y).(v.b.x-v.a.x) + // alpha.(u.b.x-u.a.x).(v.b.y-v.a.y) - alpha.(u.b.y-u.a.y).(v.b.x-v.a.x) = (u.a.y-v.a.y).(v.b.x-v.a.x) - (u.a.x-v.a.x).(v.b.y-v.a.y) + // alpha = ( (u.a.y-v.a.y).(v.b.x-v.a.x) - (u.a.x-v.a.x).(v.b.y-v.a.y) ) / ( (u.b.x-u.a.x).(v.b.y-v.a.y) - (u.b.y-u.a.y).(v.b.x-v.a.x) ) + // + // Note that the denominators of the expressions for alpha and beta are almost the + // same, differing only in sign. + // If this expression is zero it indicates that that lines do not cross because they + // are parallel, or that they are co-linear. + + double alpha_denom = (u.b.x-u.a.x)*(v.b.y-v.a.y) - (u.b.y-u.a.y)*(v.b.x-v.a.x); + if (alpha_denom==0) { + // In the case of co-linear lines, do we consider them to cross if they overlap? + // It's simplest to say "no", and always return false here. + return false; + + } else { + double alpha_num = (u.a.y-v.a.y)*(v.b.x-v.a.x) - (u.a.x-v.a.x)*(v.b.y-v.a.y); + double alpha = alpha_num / alpha_denom; + if (alpha<0 || alpha>1) { + return false; + } + double beta_denom = -alpha_denom; + double beta_num = (v.a.y-u.a.y)*(u.b.x-u.a.x) - (v.a.x-u.a.x)*(u.b.y-u.a.y); + double beta = beta_num / beta_denom; + if (beta<0 || beta>1) { + return false; + } + Vector uvec = u.b-u.a; + cross = u.a + uvec * alpha; + return true; + } +} + + +template +bool line_crosses_box(Box b, Line& l) +{ + // Does box b contain any part of line l? + // If line l crosses the boundary of the box, it is modified in place to + // clip at the boundary. + + if (b.contains(l.a) && b.contains(l.b)) { + // Both ends inside the box - easy. + return true; + } + + if ( (l.a.x < b.x0 && l.b.x < b.x0) + || (l.a.x > b.x1 && l.b.x > b.x1) + || (l.a.y < b.y0 && l.b.y < b.y0) + || (l.a.y > b.y1 && l.b.y > b.y1) ) { + // Line can't cross the box - easy. + return false; + } + + // There's a chance that the line crosses the box, but at least one + // end is outside it and will need to be clipped. + Point cross; + + bool crosses_left = line_crosses_line(l,Line(b.x0y0(),b.x0y1()),cross); + if (crosses_left) { + if (l.a.x<=b.x0) { + l.a = cross; + } else { + l.b = cross; + } + } + + bool crosses_top = line_crosses_line(l,Line(b.x0y1(),b.x1y1()),cross); + if (crosses_top) { + if (l.a.y>=b.y1) { + l.a = cross; + } else { + l.b = cross; + } + } + + bool crosses_right = line_crosses_line(l,Line(b.x1y1(),b.x1y0()),cross); + if (crosses_right) { + if (l.a.x>=b.x1) { + l.a = cross; + } else { + l.b = cross; + } + } + + bool crosses_bottom = line_crosses_line(l,Line(b.x1y0(),b.x0y0()),cross); + if (crosses_bottom) { + if (l.a.y<=b.y0) { + l.a = cross; + } else { + l.b = cross; + } + } + + return crosses_left || crosses_top || crosses_right || crosses_bottom; +} + + +}; + + +#endif + diff --git a/extras/anytermd/libpbe/include/Lock.hh b/extras/anytermd/libpbe/include/Lock.hh new file mode 100644 index 0000000000..807ba6297a --- /dev/null +++ b/extras/anytermd/libpbe/include/Lock.hh @@ -0,0 +1,55 @@ +// include/Lock.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_Lock_hh +#define libpbe_Lock_hh + +// The file implements a Lock class, which is based on the unique_lock from N2447 +// without many of that lock's features. + +#include + + +namespace pbe { + + +template +struct Lock: boost::noncopyable { + + typedef MUTEX_T mutex_t; + + mutex_t& m; + + explicit Lock(mutex_t& m_): m(m_) { + m.lock(); + } + + ~Lock() { + m.unlock(); + } + + mutex_t* mutex(void) { + return &m; + } +}; + + +}; + +#endif + diff --git a/extras/anytermd/libpbe/include/Lockable.hh b/extras/anytermd/libpbe/include/Lockable.hh new file mode 100644 index 0000000000..e8ee51cef2 --- /dev/null +++ b/extras/anytermd/libpbe/include/Lockable.hh @@ -0,0 +1,42 @@ +// src/Lockable.hh +// This file is part of libpbe; see http://decimail.org +// (C) 2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_Lockable_hh +#define libpbe_Lockable_hh + +#include "Mutex.hh" +#include "Lock.hh" + + +namespace pbe { + + template + struct Lockable: public T { + Lockable() {} + Lockable(const T& t): T(t) {} + Lockable& operator=(const Lockable& rhs) { return T::operator=(rhs); } + Lockable& operator=(const T& rhs) { T::operator=(rhs); return *this; } + typedef pbe::Mutex mutex_t; + typedef pbe::Lock scoped_lock_t; + }; + + +} + +#endif + diff --git a/extras/anytermd/libpbe/include/Locked.hh b/extras/anytermd/libpbe/include/Locked.hh new file mode 100644 index 0000000000..a5d91d7ec8 --- /dev/null +++ b/extras/anytermd/libpbe/include/Locked.hh @@ -0,0 +1,97 @@ +// Locked.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +// This provides a class template that augments a variable with a mutex, and +// allows access only when the mutex is locked. Example: +// +// typedef Locked< vector > x_t; +// x_t x; +// +// // We can only modify x via an x_t::writer: +// x_t::writer xw; +// // Creating xw locks the mutex. +// // xw behaves like a pointer to the underlying vector: +// xw->push_back(123); +// xw->push_back(321); +// // The lock is released when xw goes out of scope. +// +// To read, use an x_t::reader. It behaves like a const pointer to the +// underlying data. +// +// The mutex and lock types can be specified with template parameters, but +// they default to the boost versions. +// +// I had hoped to allow conversion-to-reference for the writer and reader, so +// that you could write xw.push_back(123) rather than xw->push_back(123). +// But this didn't work, presumably because I don't really understand how +// implict type conversion is supposed to work. My attempt is still present, +// commented out. + + +#ifndef libpbe_Locked_hh +#define libpbe_Locked_hh + +#include "Mutex.hh" + + +namespace pbe { + +template , + typename LOCK_T = pbe::Lock > +class Locked { + +private: + T data; + MUTEX_T mutex; + +public: + class writer { + Locked& locked; + LOCK_T l; + public: + writer(Locked& locked_): + locked(locked_), + l(locked.mutex) + {} +// operator T&() { return locked.data; } + T& operator*() { return locked.data; } + T* operator->() { return &locked.data; } + }; + + class reader { + Locked& locked; + LOCK_T l; + public: + reader(Locked& locked_): + locked(locked_), + l(locked.mutex) + {} +// operator const T&() const { return locked.data; } + const T& operator*() const { return locked.data; } + const T* operator->() const { return &locked.data; } + }; + +}; + + +}; + + +#endif diff --git a/extras/anytermd/libpbe/include/MappedFile.hh b/extras/anytermd/libpbe/include/MappedFile.hh new file mode 100644 index 0000000000..c2d11f7e5e --- /dev/null +++ b/extras/anytermd/libpbe/include/MappedFile.hh @@ -0,0 +1,73 @@ +// include/MappedFile.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_MappedFile_hh +#define libpbe_MappedFile_hh + +#include "FileDescriptor.hh" +#include "Exception.hh" + +#include + + +namespace pbe { + +class MappedFile { + +public: + const size_t length; + const off_t offset; + char* const addr; + + MappedFile(pbe::FileDescriptor& fd, size_t length_, pbe::FileDescriptor::open_mode_t open_mode, + off_t offset_ = 0, bool copy_on_write = false): + length(length_), offset(offset_), + addr(reinterpret_cast(fd.mmap(length,open_mode,offset,copy_on_write))) + {} + + ~MappedFile() { + int rc = munmap(addr,length); + if (rc==-1) { + pbe::throw_ErrnoException("munmap()"); + } + } + + operator char* () const { return addr; } + + template + T* ptr(int offset) const { + return reinterpret_cast(reinterpret_cast(addr)+offset); + } + + template + T& ref(int offset) const { + return *ptr(offset); + } + + void sync() { + int rc = ::msync(addr,length,MS_SYNC|MS_INVALIDATE); + if (rc==-1) { + throw_ErrnoException("msync()"); + } + } + +}; + +}; + +#endif diff --git a/extras/anytermd/libpbe/include/Mutex.hh b/extras/anytermd/libpbe/include/Mutex.hh new file mode 100644 index 0000000000..c4d8ebd2cd --- /dev/null +++ b/extras/anytermd/libpbe/include/Mutex.hh @@ -0,0 +1,214 @@ +// include/Mutex.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_Mutex_hh +#define libpbe_Mutex_hh + + +// This file provides class Mutex, which is a model of a non-recursive +// timed Mutex as defined in N2447. +// It's implemented on top of a user-supplied Futex-like class, defaulting +// to Futex. +// If we don't have Futex (i.e. this isn't Linux), or if we don't have atomic +// operations (i.e. is this an old gcc), the default is a specialisation +// that uses pthread_mutex_t. + +#include "atomic.hh" +#include "Lock.hh" +#include "Exception.hh" + +#include +#include + +#include +#include + + +#include "Futex.hh" +#if defined(PBE_HAVE_FUTEX) && defined(PBE_HAVE_ATOMICS) +#define FUTEX_T Futex<> +#else +#include +#define FUTEX_T pthread_mutex_t +#endif + + +namespace pbe { + +template +class Mutex: boost::noncopyable { + +#ifdef PBE_HAVE_ATOMICS + + int state; + FUT_T fut; + +public: + Mutex(): state(0), fut(state) {} + + ~Mutex() {} + + // Lock, waiting if necessary. + void lock() { + if (!try_lock()) { + lock_body(); + } + } + + // Try to lock, but don't wait if it's not possible. + // Return indicating whether lock is now held. + bool try_lock() { + return atomic_compare_and_swap(state,0,1); + } + + // Spend up to rel_time trying to lock. + // Return indicating whether lock is now held. + // N2447 templates this on the TimeDuration type, but we can't do specialisation + // of member functions; is that a new C++0x feature? So we'll use overloading. + + // C++0x will have various time types that can be used here. + typedef uint32_t microseconds_t; + + bool timed_lock(microseconds_t rel_time) { + return try_lock() || timed_lock_body(rel_time); + } + + bool timed_lock(float rel_time) { + // Careful! This will fail for durations that overflow microseconds_t, i.e. + // about an hour. + // We could postpone the float conversion until after the try_lock(). + return timed_lock(static_cast(rel_time*1e6)); + } + + void unlock() { + if (atomic_post_dec(state) != 1) { + unlock_body(); + } + } + + + // native_handle not provided. + + // This is not in N2447: + typedef Lock lock_t; + +private: + void lock_body() { + while (atomic_swap(state,2) != 0) { + fut.wait(2); + } + } + + bool timed_lock_body(microseconds_t rel_time) { + ::timeval start; + // Can gettimeofday() return an error? The only errors that the joint man + // page lists apply only to settimeofday(), or to gettimeofday() with the + // obsolete timezone parameter. + gettimeofday(&start,NULL); + while (atomic_swap(state,2) != 0) { + ::timeval now; + gettimeofday(&now,NULL); + microseconds_t elapsed = now.tv_usec - start.tv_usec + + 1000000 * (now.tv_sec - start.tv_sec); + // Hmm, this would be faster if we pre-computed the end time + // and just did a comparison in here. But it's not trivial + // to add rel_time to start without overlow concerns. + if (elapsed>rel_time) { + return false; + } + fut.wait(2); + } + } + + void unlock_body() { + atomic_write(state,0); + fut.wake(1); + } + +#endif + +}; + + + +#if !(defined(PBE_HAVE_FUTEX) && defined(PBE_HAVE_ATOMICS)) + +template <> +class Mutex: boost::noncopyable { + + pthread_mutex_t mut; + +public: + Mutex() { + int r = pthread_mutex_init(&mut, NULL); + if (r!=0) { + pbe::throw_ErrnoException("pthread_mutex_init()",r); + } + } + + ~Mutex() { + pthread_mutex_destroy(&mut); + } + + // Lock, waiting if necessary. + void lock() { + int r = pthread_mutex_lock(&mut); + if (r!=0) { + pbe::throw_ErrnoException("pthread_mutex_lock()",r); + } + } + + // Try to lock, but don't wait if it's not possible. + // Return indicating whether lock is now held. + bool try_lock() { + int r = pthread_mutex_trylock(&mut); + switch (r) { + case 0: return true; + case EBUSY: return false; + default: pbe::throw_ErrnoException("pthread_mutex_trylock()",r); + } + } + + // FIXME I'm too lazy to implement timed_lock. + + void unlock() { + int r = pthread_mutex_unlock(&mut); + if (r!=0) { + pbe::throw_ErrnoException("pthread_mutex_unlock()",r); + } + } + + + pthread_mutex_t* native_handle() { + return &mut; + } + + // This is not in N2447: + typedef Lock lock_t; + +}; + +#endif + + +}; + +#undef FUTEX_T + +#endif + diff --git a/extras/anytermd/libpbe/include/Point.hh b/extras/anytermd/libpbe/include/Point.hh new file mode 100644 index 0000000000..a7614d7729 --- /dev/null +++ b/extras/anytermd/libpbe/include/Point.hh @@ -0,0 +1,67 @@ +// include/Point.hh +// This file is part of libpbe +// (C) 2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef pbe_Point_hh +#define pbe_Point_hh + +namespace pbe { + + +template +struct Point { + typedef COORD_T coord_t; + + coord_t x; + coord_t y; + + Point() {} + + Point(const Point& other): + x(other.x), y(other.y) + {} + + Point(coord_t x_, coord_t y_): + x(x_), y(y_) + {} + + bool operator<(const Point& other) const { + if (x + +#include + +#include + + +// ReadTiffFile is a wrapper around libtiff's facilities for reading +// a TIFF file. The ctor takes a filename; you can then access properties +// such as the dimensions of the image. +// +// libtiff provides various methods to access the pixel data, some of +// which work with all files and others work with files with particular +// organisations [tiff allows data to be in raster order or tiled, with +// the colour channels packed or separate, etc]. This wrapper currently +// only supports reading in "ARGB strips", i.e. the colours are decoded +// to 32-bits-per-pixel and the data is in strips the full width of the +// image and some some number of pixels (possibly 1) high. +// +// Boost.GIL also has a libtiff wrapper, but I believe that it only +// supports reading whole images. + + +namespace pbe { + + +struct ReadTiffFile: boost::noncopyable { + TIFF* const tiff; + ReadTiffFile(const char* fn): + tiff(TIFFOpen(fn,"r")) + { + if (!tiff) { + throw "TIFFOpen() failed"; + } + } + ~ReadTiffFile() { + TIFFClose(tiff); + } + + uint32_t width() const { + uint32_t w; + int r = TIFFGetField(tiff,TIFFTAG_IMAGEWIDTH,&w); + if (r==0) { + throw "TIFFGetField(TIFFTAG_IMAGEWIDTH); failed"; + } + return w; + } + uint32_t height() const { + uint32_t h; + int r = TIFFGetField(tiff,TIFFTAG_IMAGELENGTH,&h); + if (r==0) { + throw "TIFFGetField(TIFFTAG_IMAGLENGTH); failed"; + } + return h; + } + uint32_t rows_per_strip() const { + uint32_t n; + int r = TIFFGetField(tiff,TIFFTAG_ROWSPERSTRIP,&n); + if (r==0) { + throw "TIFFGetField(TIFFTAG_ROWSPERSTRIP); failed"; + } + return n; + } + size_t number_of_strips() const { + return TIFFNumberOfStrips(tiff); + } + + void read_rgba_strip(uint32_t row, uint32_t* data) const { + int r = TIFFReadRGBAStrip(tiff,row,data); + if (r==0) { + throw "TIFFReadRGBAStrip failed"; + } + } +}; + + +}; + + +#endif + diff --git a/extras/anytermd/libpbe/include/Recoder.hh b/extras/anytermd/libpbe/include/Recoder.hh new file mode 100644 index 0000000000..63d5f81365 --- /dev/null +++ b/extras/anytermd/libpbe/include/Recoder.hh @@ -0,0 +1,55 @@ +// src/Recoder.hh +// This file is part of libpbe; see http://decimail.org +// (C) 2004 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +#ifndef libpbe_Recoder_hh +#define libpbe_Recoder_hh + +#include +#include +#include + +#include + +using namespace std; + + +class Recoder { + +public: + + enum ErrorLevel { not_canonical=RECODE_NOT_CANONICAL, + ambiguous_output=RECODE_AMBIGUOUS_OUTPUT, + untranslatable=RECODE_UNTRANSLATABLE, + invalid_input=RECODE_INVALID_INPUT, + system_error=RECODE_SYSTEM_ERROR }; + + Recoder(string from_charset, string to_charset, + ErrorLevel e=ambiguous_output); + ~Recoder(); + string operator()(string i); + string operator()(const char* i, int l); + +private: + + ErrorLevel error_level; + static RECODE_OUTER outer; + RECODE_REQUEST request; + +}; + + +#endif diff --git a/extras/anytermd/libpbe/include/SerialPort.hh b/extras/anytermd/libpbe/include/SerialPort.hh new file mode 100644 index 0000000000..7fd707808e --- /dev/null +++ b/extras/anytermd/libpbe/include/SerialPort.hh @@ -0,0 +1,133 @@ +// SerialPort.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2006-2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#ifndef libpbe_SerialPort_hh +#define libpbe_SerialPort_hh + +#include "FileDescriptor.hh" +#include "Exception.hh" + +#include +#include + +#include +#include + + +namespace pbe { + +class SerialPort: public FileDescriptor { + +public: + SerialPort(std::string fn, open_mode_t open_mode, int baudrate, bool raw=true): + FileDescriptor(fn,open_mode) + { + set_serial_mode(baudrate,raw); + } + + SerialPort(std::string fn, open_mode_t open_mode): + FileDescriptor(fn,open_mode) + {} + + SerialPort(int fd): + FileDescriptor(fd) + {} + + void set_serial_mode(int baudrate, bool raw=true) { + struct termios settings; + int rc = tcgetattr(fd, &settings); + if (rc<0) { + throw_ErrnoException("tcgetattr("+fn+")"); + } + if (raw) { + cfmakeraw(&settings); + } + rc = cfsetspeed(&settings,baudrate); + if (rc!=0) { + throw_ErrnoException("cfsetspeed("+fn+"," + +boost::lexical_cast(baudrate)+")"); + } + rc = tcsetattr(fd, TCSANOW, &settings); + if (rc<0) { + throw_ErrnoException("tcsetattr("+fn+")"); + } + } + + void discard_pending_input(void) { + int rc = tcflush(fd, TCIFLUSH); + if (rc!=0) { + throw_ErrnoException("tcflush(TCIFLUSH)"); + } + } + + + typedef int modem_bits_t; + + modem_bits_t get_modem_bits(void) { + modem_bits_t bits; + ioctl(TIOCMGET,&bits); + return bits; + } + + bool is_cd(void) { + return get_modem_bits()&TIOCM_CD; + } + + bool is_ri(void) { + return get_modem_bits()&TIOCM_RI; + } + + bool is_dsr(void) { + return get_modem_bits()&TIOCM_DSR; + } + + bool is_cts(void) { + return get_modem_bits()&TIOCM_CTS; + } + + + void set_these_modem_bits(modem_bits_t bits) { + ioctl(TIOCMBIS,&bits); + } + + void clear_these_modem_bits(modem_bits_t bits) { + ioctl(TIOCMBIC,&bits); + } + + void define_modem_bits(modem_bits_t bits, bool v) { + if (v) { + set_these_modem_bits(bits); + } else { + clear_these_modem_bits(bits); + } + } + + void define_rts(bool v) { + define_modem_bits(TIOCM_RTS,v); + } + + void define_dtr(bool v) { + define_modem_bits(TIOCM_DTR,v); + } + +}; + +}; + +#endif diff --git a/extras/anytermd/libpbe/include/SmtpClient.hh b/extras/anytermd/libpbe/include/SmtpClient.hh new file mode 100644 index 0000000000..16446d6c2e --- /dev/null +++ b/extras/anytermd/libpbe/include/SmtpClient.hh @@ -0,0 +1,55 @@ +// src/SmtpClient.hh +// This file is part of libpbe; see http://decimail.org +// (C) 2004 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +#ifndef libpbe_SmtpClient_hh +#define libpbe_SmtpClient_hh + +#include "Exception.hh" + +#include +#include +using namespace std; + + + class SmtpError: public pbe::Exception { + private: + string msg; + public: + SmtpError(string m): msg(m) {} + void report(ostream& s) const; + }; + + + class SmtpClient { + public: + SmtpClient(bool log=false); + void connect(string server_name, string domain, int port=25); + bool is_connected(void) { return connected; } + void send_msg(string sender, string recipient, string msg); + void send_msg(string sender, const list& recipients, string msg); + void disconnect(void); + + private: + const bool enable_log; + bool connected; + int fd; + + void wait_for_reply(int expected_code, int timeout); + void send(string d); + }; + +#endif diff --git a/extras/anytermd/libpbe/include/Spin.hh b/extras/anytermd/libpbe/include/Spin.hh new file mode 100644 index 0000000000..eec93ac45e --- /dev/null +++ b/extras/anytermd/libpbe/include/Spin.hh @@ -0,0 +1,62 @@ +// include/Spin.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_Spin_hh +#define libpbe_Spin_hh + +// The file implements a Futex-like class which simply spins. + +#include "compiler_magic.hh" + +#include + + +namespace pbe { + + +struct Spin { + + Spin(PBE_UNUSED_ARG(int& val)) {} + ~Spin() {} + + void wait(PBE_UNUSED_ARG(int expected)) { + } + + bool timed_wait(PBE_UNUSED_ARG(int expected), const timespec& timeout) { + // TODO detect timeout + return true; + } + + bool timed_wait(PBE_UNUSED_ARG(int expected), float timeout) { + // TODO detect timeout + return true; + } + + // Wake up to n waiters; returns the number woken. + int wake(int n=1) { + // Hmm, how much does the return value matter? + return n; + } +}; + + +}; + + +#endif + diff --git a/extras/anytermd/libpbe/include/StringSplitter.hh b/extras/anytermd/libpbe/include/StringSplitter.hh new file mode 100644 index 0000000000..f48fcc3471 --- /dev/null +++ b/extras/anytermd/libpbe/include/StringSplitter.hh @@ -0,0 +1,122 @@ +// src/StringSplitter.hh +// This file is part of libpbe; see http://decimail.org +// (C) 2004 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +#ifndef libpbe_StringSplitter_hh +#define libpbe_StringSplitter_hh + +#include +using namespace std; + + +class StringSplitter { +protected: + const string& str; + + std::string::size_type start; + std::string::size_type end; + bool init; + bool last; + bool past_end; + +public: + StringSplitter ( const string& s ): + str(s), start(0), init(false), last(false), past_end(false) {} + + virtual ~StringSplitter() {} + + bool exhausted(void) const { + return past_end; + } + + string operator*(void) /*const*/ { + if (!init) { + find_end(); + init=true; + } + return str.substr(start,end-start); + } + + void operator++(void) { + if (last) { + past_end=true; + } else { + if (!init) { + find_end(); + init=true; + } + find_next_start(); + find_end(); + } + } + +private: + virtual void find_next_start(void) = 0; + virtual void find_end(void) = 0; + +}; + + + +class StringSplitterSeq: public StringSplitter { +private: + const string split_seq; + +public: + StringSplitterSeq ( const string& s, const string ss ): + StringSplitter(s), split_seq(ss) {} + +private: + void find_next_start(void) { + start = end+split_seq.size(); + } + + void find_end(void) { + end = str.find(split_seq,start); + if (end==str.npos) { + end=str.size(); + last=true; + } + } +}; + + + +class StringSplitterAny: public StringSplitter { +private: + const string split_chars; + +public: + StringSplitterAny ( const string& s, const string sc ): + StringSplitter(s), split_chars(sc) {} + +private: + void find_next_start(void) { + start = end+1; + } + +private: + void find_end(void) { + end = str.find_first_of(split_chars,start); + if (end==str.npos) { + end=str.size(); + last=true; + } + } +}; + + +#endif diff --git a/extras/anytermd/libpbe/include/StringTransformer.hh b/extras/anytermd/libpbe/include/StringTransformer.hh new file mode 100644 index 0000000000..76e2da1c2d --- /dev/null +++ b/extras/anytermd/libpbe/include/StringTransformer.hh @@ -0,0 +1,116 @@ +// src/StringTransformer.hh +// This file is part of libpbe; see http://decimail.org +// (C) 2004 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +#ifndef libpbe_StringTransformer_hh +#define libpbe_StringTransformer_hh + +#include +#include +using namespace std; + +class StringTransformer { +private: + int char_idx(char c) const { return (unsigned char)(c); } + +public: + StringTransformer(void): + transformation(256) + { + for(int c=0; c<256; ++c) { + transformation[c] = c; + } + } + + void add_cc_rule(char from, char to) + { + transformation[char_idx(from)]=to; + } + + void add_cs_rule(char from, string to) + { + transformation[char_idx(from)]=to; + } + + template + void add_cf_rule(char from, const CharToStringFunc xform) + { + transformation[char_idx(from)]=xform(from); + } + + template + void add_pc_rule(const CharPredicate from, char to) + { + for(int c=0; c<256; ++c) { + if (from(c)) { + transformation[c] = to; + } + } + } + + template + void add_ps_rule(const CharPredicate from, string to) + { + for(int c=0; c<256; ++c) { + if (from(c)) { + transformation[c] = to; + } + } + } + + template + void add_pf_rule(const CharPredicate from, const CharToStringFunc xform) + { + for(int c=0; c<256; ++c) { + if (from(c)) { + transformation[c] = xform(c); + } + } + } + + string operator()(const string& s) const + { + string r; + for(string::const_iterator i=s.begin(); + i!=s.end(); ++i) { + r.append(transformation[char_idx(*i)]); + } + return r; + } + +private: + vector transformation; +}; + + + +class EscapeInserter: public StringTransformer { +public: + EscapeInserter(string chars_to_escape, char escape_char = '\\') + { + string escaped(" "); + escaped[0]=escape_char; + for (string::const_iterator i=chars_to_escape.begin(); + i!=chars_to_escape.end(); ++i) { + escaped[1]=*i; + add_cs_rule(*i,escaped); + } + } +}; + + + +#endif diff --git a/extras/anytermd/libpbe/include/TcpClientSocket.hh b/extras/anytermd/libpbe/include/TcpClientSocket.hh new file mode 100644 index 0000000000..0eb15c7d3a --- /dev/null +++ b/extras/anytermd/libpbe/include/TcpClientSocket.hh @@ -0,0 +1,45 @@ +// TcpClientSocket.hh +// This file is part of libpbe; see http://decimail.org/ +// (C) 2006-2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_TcpClientSocket_hh +#define libpbe_TcpClientSocket_hh + +#include + +#include "FileDescriptor.hh" +#include "ip.hh" + + +namespace pbe { + +class TcpClientSocket: public FileDescriptor { + +private: + const std::string host; + const short port; + +public: + TcpClientSocket(std::string host_, short port_): + FileDescriptor(tcp_client_connect(host_,port_)), + host(host_), port(port_) + {} +}; + +}; + +#endif diff --git a/extras/anytermd/libpbe/include/TcpListenSocket.hh b/extras/anytermd/libpbe/include/TcpListenSocket.hh new file mode 100644 index 0000000000..f0e2e25b42 --- /dev/null +++ b/extras/anytermd/libpbe/include/TcpListenSocket.hh @@ -0,0 +1,102 @@ +// TcpListenSocket.hh +// This file is part of libpbe; see http://decimail.org/ +// (C) 2006-2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_TcpListenSocket_hh +#define libpbe_TcpListenSocket_hh + +#include + +#include "FileDescriptor.hh" + +#include +#include +#include +#include + + +namespace pbe { + +class TcpListenSocket: public FileDescriptor { + + static int create_tcp_listen_socket(short port) { + int listenfd = socket(PF_INET,SOCK_STREAM,0); + if (listenfd==-1) { + throw_ErrnoException("socket()"); + } + try { + // race condition here + int rc = fcntl(listenfd,F_SETFD,FD_CLOEXEC); + if (rc==-1) { + throw_ErrnoException("fcntl(listenfd,F_SETFD,FD_CLOEXEC)"); + } + + // Not sure what this does + const int t=1; + setsockopt(listenfd, SOL_SOCKET, SO_REUSEADDR, &t, sizeof(t)); + + struct sockaddr_in server_addr; + memset(&server_addr,0,sizeof(server_addr)); + server_addr.sin_family=AF_INET; + server_addr.sin_addr.s_addr=htonl(INADDR_ANY); + server_addr.sin_port=htons(port); + int r = bind(listenfd,reinterpret_cast(&server_addr),sizeof(server_addr)); + if (r==-1) { + throw_ErrnoException("bind()"); + } + + // 128 is the "backlog" parameter. + r = listen(listenfd,128); + if (r==-1) { + throw_ErrnoException("listen()"); + } + } catch(...) { + ::close(listenfd); + throw; + } + return listenfd; + } + +public: + TcpListenSocket(short port): + FileDescriptor(create_tcp_listen_socket(port)) + {} + + int accept() { + struct sockaddr_in client_addr; + socklen_t client_size=sizeof(client_addr); + int connfd = ::accept(fd,reinterpret_cast(&client_addr),&client_size); + if (connfd==-1) { + throw_ErrnoException("accept()"); + } + try { + // race condition here + int rc = fcntl(connfd,F_SETFD,FD_CLOEXEC); + if (rc==-1) { + throw_ErrnoException("fcntl(connfd,F_SETFD,FD_CLOEXEC)"); + } + } catch(...) { + ::close(connfd); + throw; + } + return connfd; + } +}; + +}; + +#endif diff --git a/extras/anytermd/libpbe/include/TcpServerSocket.hh b/extras/anytermd/libpbe/include/TcpServerSocket.hh new file mode 100644 index 0000000000..37862d34cc --- /dev/null +++ b/extras/anytermd/libpbe/include/TcpServerSocket.hh @@ -0,0 +1,42 @@ +// TcpServerSocket.hh +// This file is part of libpbe; see http://decimail.org/ +// (C) 2006-2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_TcpServerSocket_hh +#define libpbe_TcpServerSocket_hh + +#include "FileDescriptor.hh" +#include "TcpListenSocket.hh" + + +namespace pbe { + +class TcpServerSocket: public FileDescriptor { + +public: + TcpServerSocket(TcpListenSocket& listen_socket): + FileDescriptor(listen_socket.accept()) + {} + + TcpServerSocket(int fd): + FileDescriptor(fd) + {} +}; + +}; + +#endif diff --git a/extras/anytermd/libpbe/include/Thread.hh b/extras/anytermd/libpbe/include/Thread.hh new file mode 100644 index 0000000000..d9a2a9d4b2 --- /dev/null +++ b/extras/anytermd/libpbe/include/Thread.hh @@ -0,0 +1,314 @@ +// include/Thread.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_thread_hh +#define libpbe_thread_hh + + +// This file defines a class Thread, which has a similar interface to +// Boost.Thread and the proposed std::thread described in N2447, but +// with various extensions including cancellation and priorities. +// It's implemented on top of the POSIX thread primitives, as implemented +// in linux by NPTL; that's included in glibc and sufficiently-new +// versions of uClibc. + +#include +#include + +#include +#include +#include + +#include "Exception.hh" + + +namespace pbe { + +class Thread; + +namespace this_thread { + // Operations on the current thread: +// Thread::id get_id(); + void yield(); + template void sleep(const TimeDuration& rel_t); +}; + + +// This is a helper used in thread creation, see below: +typedef boost::function thread_fn_t; +// Hmm, maybe this has to be in a .cc file??? +static void* start_thread(void* voidstar_fn_ptr) { + thread_fn_t* fn_ptr = static_cast(voidstar_fn_ptr); + thread_fn_t fn = *fn_ptr; + delete fn_ptr; + fn(); + return NULL; +} + + +class Thread: boost::noncopyable { + + // Note that the lifetime of the thread is not coupled to the lifetime of + // the Thread object; i.e. destroying the Thread object does not terminate + // the thread. It's more of a "thread handle" really. + + + class pthread_attr_wrapper: boost::noncopyable { + // The POSIX pthread_attr_t type needs be initialised and destroyed + // before and after use. This wrapper makes that automatic. + ::pthread_attr_t attrs; + public: + pthread_attr_wrapper() { + int r = ::pthread_attr_init(&attrs); + if (r!=0) { + // Apparently error code is in r, not errno. This is the case for many + // of the pthread_* functions. + throw_ErrnoException("pthread_attr_init()",r); + } + } + ~pthread_attr_wrapper() { + ::pthread_attr_destroy(&attrs); + } + // Make it possible to use a pthread_attr_wrapper anywhere that a + // pthread_attr_t* is called for: + operator ::pthread_attr_t* () { + return &attrs; + } + }; + + +public: + //typedef pid_t id; // TODO this should probably be the thread's tid, but pthread_create + // doesn't give a way to get this easily. + typedef ::pthread_t native_handle_type; + + // POSIX defines the scheduling parameters for a thread as a (priority, policy) pair. + // Higher-priority threads are always run in preference to lower-priority threads, + // irrespective of scheduling policy. Scheduling policy determines how equal-priority + // threads are scheduled, with the two choices SCHED_FIFO and SCHED_RR. RR, "Round + // Robin", is somewhat fair in the sense that threads are preempted after a timeslice + // has elapsed and don't run again until all other threads (at that priority) have + // had a chance to run. FIFO, in contrast, runs the same thread until it blocks + // (e.g. a system call). + // I'm not totally convinced that Linux necessarily implements all of this: you + // can find documentation saying that it definitely doesn't, but that may be out of + // date by now. + + // Neither std::thread not Boost.Threads provides a way to set these parameters, + // but we do, when the thread is created, using this struct to store the parameters: + + struct scheduling_parameters { + int priority; + int policy; + bool inherit; // This causes the parameters to be inheritted from the calling thread. + + scheduling_parameters(): inherit(true) {} + scheduling_parameters(int priority_, int policy_=SCHED_RR): + priority(priority_), policy(policy_), inherit(false) {} + + // Legitimate values for priority are rather inconveniently defined dynamically + // and depend on the policy. There's a guarantee that at least 32 levels exist, + // but there's no guarantee about any overlap between the ranges for the different + // policies. We ignore this: users should assume that the priorities range 0 to 31 + // and that there's an undefined relationship between the policies. + + void set_attrs(::pthread_attr_t* attrs_p) const { // Store these parameters in this + // pthread_attrs_t + int r = ::pthread_attr_setinheritsched(attrs_p, inherit ? PTHREAD_INHERIT_SCHED + : PTHREAD_EXPLICIT_SCHED); + if (r!=0) { + throw_ErrnoException("pthread_attr_setinheritsched()",r); + } + if (!inherit) { + r = ::pthread_attr_setschedpolicy(attrs_p, policy); + if (r!=0) { + throw_ErrnoException("pthread_attr_setschedpolicy()",r); + } + struct ::sched_param p; + p.sched_priority = priority + ::sched_get_priority_min(policy); + r = ::pthread_attr_setschedparam(attrs_p, &p); + if (r!=0) { + throw_ErrnoException("pthread_attr_setschedparam()",r); + } + } + } + }; + + + // The default constructor is supposed to return an object not attached to any + // thread with an id that compares equal to all detached or joined threads. + Thread(): + //id(), + joinable_(false) + {} + + + // This is the main constructor that actually starts a thread. + // std::thread has this constructor: + // template explicit Thread(F f, Args&&... args); + // The Args&&... stuff is magic C++0x varargs voodoo that we can't use. We'll + // use Boost.Function. Note that there's a thread-safety bug in Boost.Function + // in 1.34.*; it's fixed in 1.35, but I think that we're OK as gcc provides + // thread-safe statics by default. + // We also supply initial scheduling parameters, with a default of inheritting the + // caller's. std::thread only allows this via the native_handle hack. + // We also take a stack size, with a default provided by pthreads (maybe 2 MBytes, + // or 10 MBytes, or something like that). There's no way to specify this + // in std::thread or Boost.Threads. The kernel will reserve all of this memory, + // which doesn't matter on a system with lots of swap (though it can make some + // memory-usage numbers look odd). On a system with no swap, it's still OK as + // long as the kernel is happy to over-commit; see /proc/sys/vm/overcommit*. + + explicit Thread(thread_fn_t fn, + scheduling_parameters sp = scheduling_parameters(), + size_t stacksize=0) { + pthread_attr_wrapper attrs; + sp.set_attrs(attrs); + if (stacksize!=0) { + int r = ::pthread_attr_setstacksize(attrs,stacksize); + if (r!=0) { + throw_ErrnoException("pthread_attr_setstacksize()",r); + } + } + // pthread_create takes a C function pointer, so we can't pass the + // boost::function to it directly (though we could perhaps specialise + // this constructor for the simple case where the thread-function is + // a C function pointer). Instead we'll use a helper function and pass + // the boost::function as the arg. But the pthread_create arg is a + // void*, and the boost::function is larger than a pointer, and we + // can't pass a pointer to fn since this function may have terminated + // and fn gone out of scope before the new thread starts. Solution: + // dynamically allocate a copy of fn, and have the new thread delete + // it. + + thread_fn_t* fn_ptr = new thread_fn_t(fn); + + int r = pthread_create(&pt, attrs, &start_thread, static_cast(fn_ptr)); + if (r!=0) { + delete fn_ptr; // The thread has not been created in any case reporting + // an error, right? + throw_ErrnoException("pthread_create()",r); + } + joinable_ = true; + } + + + // Destroying this Thread object does not terminate the thread. + // It does, however, detach it (see below) if it is still running. + ~Thread() { + // We can't unconditionally call detach() as pthread_detach is undefined + // if called more than once, or if called when join() has been called (I think). + if (joinable()) { + detach(); + } + } + + // std::thread defines C++0x "move" copy constructor and assignment operators. + // We don't because we don't have the necessary C++0x magic yet. + // Thread(Thread&&); + // Thread& operator=(Thread&&); + // void swap(Thread&&); + + // A thread that has not yet been joined or detached is "joinable". + // Note that "joinable" doesn't mean that join will return immediately. + bool joinable() const { + return joinable_; + } + + // "joining" with a thread means waiting for it to terminate if it has not + // yet done so, and then releasing its resources. It's "wait() for threads". + void join() { + int r = ::pthread_join(pt,NULL); + if (r!=0) { + throw_ErrnoException("pthread_join()",r); + } + joinable_ = false; + } + + // Note that there is no version of join that takes a timeout. There's no + // such functionality provided by POSIX. If needed, it could be emulated + // using a condition variable that's signalled to indicate termination. + + // "detaching" a thread indicates that it will not be "joined"; when it + // terminates its resources will be released automatically. + void detach() { + int r = ::pthread_detach(pt); + if (r!=0) { + throw_ErrnoException("pthread_detach()",r); + } + joinable_ = false; + } + + + // std::threads doesn't provide any way to cancel a thread. Boost.Threads + // does provide a way, but it only provides cancellation points in + // synchronisation primitives (i.e. lock mutex, wait for condition), not + // in blocking IO operations. POSIX does provide cancellation that works + // for blocking IO, but the POSIX spec doesn't provide for interoperation + // with C++: destructors and catch-blocks are not executed. However, the + // glibc implementation of pthread_cancel() _does_ invoke destructors. + // But it might not do the right thing with catch blocks; this needs to + // be understood. I'm not yet sure if the uClibc NPTL implementation has + // this C++ integration. + // Anyway, we provide thread cancellation based on the POSIX primitives. + // Calling cancel() will ask the thread to cancel and return immediately; + // it does not wait for the thread to actually terminate. The thread will + // start to terminate either immediately, at the next cancellation point, + // or not at all depending on its current cancellation mode (see + // cancellation.hh). It will then run its destructors and terminate. + // For a list of functions that must or may be cancellation points, see + // http://www.opengroup.org/onlinepubs/009695399/functions/xsh_chap02_09.html#tag_02_09_05 + + void cancel() { + int r = ::pthread_cancel(pt); + if (r!=0) { + throw_ErrnoException("pthread_cancel()",r); + } + } + + + //id get_id() const; // TODO + + native_handle_type native_handle() { + return pt; + } + + // This static function indicates the number of processors that the system + // has. + // There are actually two measures, for the total number of processors and + // the total number of "online" processors, e.g. if the OS can power them + // up and down dynamically. This returns the former. + static unsigned hardware_concurrency() { + int n = sysconf(_SC_NPROCESSORS_ONLN); + if (n==-1) { + throw_ErrnoException("sysconf(_SC_NPROCESSORS_ONLN)"); + } + return n; + } + +private: + //id tid; + pthread_t pt; + bool joinable_; +}; + + +}; + +#endif + diff --git a/extras/anytermd/libpbe/include/Time.hh b/extras/anytermd/libpbe/include/Time.hh new file mode 100644 index 0000000000..110ab7dabf --- /dev/null +++ b/extras/anytermd/libpbe/include/Time.hh @@ -0,0 +1,62 @@ +// src/Time.hh +// This file is part of libpbe; see http://decimail.org +// (C) 2006 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_Time_hh +#define libpbe_Time_hh + +#include + +#include + + +namespace pbe { + + struct Time { + + uint8_t hour; + uint8_t minute; + uint8_t second; + + Time() {} + + Time(int hour_, int minute_, int second_): + hour(hour_), minute(minute_), second(second_) {} + + bool operator<(const Time& rhs) const { + return (hour(t.hour) << ":" << static_cast(t.minute) + << ":" << static_cast(t.second); + return strm; + } + +} + + +#endif diff --git a/extras/anytermd/libpbe/include/Timer.hh b/extras/anytermd/libpbe/include/Timer.hh new file mode 100644 index 0000000000..9eb9700afb --- /dev/null +++ b/extras/anytermd/libpbe/include/Timer.hh @@ -0,0 +1,53 @@ +// include/Timer.cc +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +// A Timer. +// Constructor takes a period (a double, in seconds), and the expired() method indicates +// whether that much time has elapsed since construction. + + +#ifndef pbe_Timer_hh +#define pbe_Timer_hh + +#include "HPtime.hh" + + +namespace pbe { + +class Timer { + +private: + HPtime end_time; + +public: + Timer(double dt): + end_time(HPtime::now() + dt) + {} + + bool expired() { + return end_time + +#include +#include + +// URIs per RFC2396. +// All URIs have a scheme - well, except for relative URIs outside their +// context - but the form of the rest is scheme-dependent. However, +// schemes that are hierarchical have a common syntax for hierarchy, and +// schemes that involve the network have a common syntax for indicating +// the remote host. This URI class is intended primarily for these schemes +// i.e. HTTP, HTTPS, and FTP. + + +namespace pbe { + + +class URI { + +public: + URI(): port(0) {} + URI(std::string absolute_uri); + URI(std::string scheme_, std::string host_, std::string abs_path_, + std::string query_="", uint16_t port_=0, std::string userinfo_=""): + scheme(scheme_), userinfo(userinfo_), host(host_), port(port_), + abs_path(abs_path_), query(query_) + {} + + // The names of the following fields match the rules in the RFC2396 EBNF. + std::string scheme; + std::string userinfo; + std::string host; + uint16_t port; // port is set to zero if it's not specified; this class doesn't + // know about protocol-specific default port numbers. + std::string abs_path; + std::string query; + // Example: http://joe@example.com:8080/path/to/foo.cgi?a=1 + // scheme=http, userinfo=joe, host=example.com, port=8080, + // abs_path=/path/to/foo.cgi, quary=a=1 + + struct SyntaxError {}; + + std::string str() const { + std::string s = scheme + "://"; + if (!userinfo.empty()) { + s += userinfo + "@"; + } + s += host; + if (port!=0) { + s += ":" + boost::lexical_cast(port); + } + s += abs_path; + if (!query.empty()) { + s += "?" + query; + } + return s; + } + +}; + + +}; + + +#endif + diff --git a/extras/anytermd/libpbe/include/Unlock.hh b/extras/anytermd/libpbe/include/Unlock.hh new file mode 100644 index 0000000000..be02892e56 --- /dev/null +++ b/extras/anytermd/libpbe/include/Unlock.hh @@ -0,0 +1,55 @@ +// include/Unlock.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_Unlock_hh +#define libpbe_Unlock_hh + +// The file implements a Unlock class, which unlocks a mutex while it is +// in scope. + +#include + + +namespace pbe { + + +template +struct Unlock: boost::noncopyable { + + typedef MUTEX_T mutex_t; + + mutex_t& m; + + explicit Unlock(mutex_t& m_): m(m_) { + m.unlock(); + } + + ~Unlock() { + m.lock(); + } + + mutex_t* mutex(void) { + return &m; + } +}; + + +}; + +#endif + diff --git a/extras/anytermd/libpbe/include/Vector.hh b/extras/anytermd/libpbe/include/Vector.hh new file mode 100644 index 0000000000..ebe4d99e5f --- /dev/null +++ b/extras/anytermd/libpbe/include/Vector.hh @@ -0,0 +1,105 @@ +// include/Vector.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef pbe_Vector_hh +#define pbe_Vector_hh + +#include + +#include "Point.hh" + + +namespace pbe { + + +template +struct Vector { + typedef COORD_T coord_t; + + coord_t x; + coord_t y; + + Vector() {} + + Vector(const Vector& other): + x(other.x), y(other.y) + {} + + Vector(coord_t x_, coord_t y_): + x(x_), y(y_) + {} + + bool operator<(const Vector& other) const { + if (x + Vector operator/=(DIV_T divisor) { x /= divisor; y /= divisor; return *this; } + + template + Vector operator/(DIV_T divisor) const { Vector v = *this; v /= divisor; return v; } + + template + Vector operator*=(MUL_T multiplier) { x *= multiplier; y *= multiplier; return *this; } + + template + Vector operator*(MUL_T multiplier) const { Vector v = *this; v *= multiplier; return v; } + +}; + + +template +Vector operator-(const Point& a, const Point& b) { + return Vector(a.x-b.x, a.y-b.y); +} + +template +Point operator+(const Point& a, const Vector& b) { + return Point(a.x+b.x, a.y+b.y); +} + +template +Point operator-(const Point& a, const Vector& b) { + return Point(a.x-b.x, a.y-b.y); +} + + + +}; + + +#endif + diff --git a/extras/anytermd/libpbe/include/WritePngFile.hh b/extras/anytermd/libpbe/include/WritePngFile.hh new file mode 100644 index 0000000000..feeb3422fc --- /dev/null +++ b/extras/anytermd/libpbe/include/WritePngFile.hh @@ -0,0 +1,103 @@ +// include/WritePngFile.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2009 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_WritePngFile_hh +#define libpbe_WritePngFile_hh + +#include + +#include + + +// WritePngFile is a wrapper around libpng's facilities for writing +// a PNG file. The ctor takes a filename, image dimensions and so on; +// you then supply pixel data row-at-a-time. This is a small subsit +// of libpng's facilities. +// +// Boost.GIL also has a libpng wrapper, but I believe that it only +// supports writing whole images. + + +namespace pbe { + + +struct WritePngFile { + + FILE* f; + png_structp png_p; + png_infop info_p; + + static FILE* check_fopen(const char* fn, const char* mode) { + FILE* f = fopen(fn,mode); + if (!f) { + throw "fopen() failed"; + } + return f; + } + + static png_structp check_png_create_write_struct(png_const_charp user_png_ver, png_voidp error_ptr, + png_error_ptr error_fn, png_error_ptr warn_fn) { + png_structp p = png_create_write_struct(user_png_ver, error_ptr, error_fn, warn_fn); + if (!p) { + throw "png_create_write_struct() failed"; + } + return p; + } + + static png_infop check_png_create_info_struct(png_structp png_ptr) { + png_infop i = png_create_info_struct(png_ptr); + if (!i) { + throw "png_create_info_struct() failed"; + } + return i; + } + + static void error_fn(png_structp, png_const_charp error_msg) { + throw error_msg; + } + + WritePngFile(const char* fn, uint32_t width, uint32_t height, + int bit_depth=8, int colour_type=PNG_COLOR_TYPE_RGB_ALPHA): + f(check_fopen(fn,"wb")), + png_p(check_png_create_write_struct(PNG_LIBPNG_VER_STRING, NULL, &error_fn, 0)), + info_p(check_png_create_info_struct(png_p)) + { + png_init_io(png_p,f); + png_set_IHDR(png_p, info_p, width, height, bit_depth, colour_type, + PNG_INTERLACE_NONE, PNG_COMPRESSION_TYPE_DEFAULT, PNG_FILTER_TYPE_DEFAULT); + png_write_info(png_p, info_p); + } + + ~WritePngFile() { + png_write_end(png_p, info_p); + png_destroy_write_struct(&png_p,&info_p); // frees both + fclose(f); + } + + void write_row(const uint32_t* data) { + png_write_row(png_p, const_cast(reinterpret_cast(data))); + } + +}; + + +}; + + +#endif + diff --git a/extras/anytermd/libpbe/include/Yield.hh b/extras/anytermd/libpbe/include/Yield.hh new file mode 100644 index 0000000000..e04aa53ea1 --- /dev/null +++ b/extras/anytermd/libpbe/include/Yield.hh @@ -0,0 +1,68 @@ +// include/Yield.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_Yield_hh +#define libpbe_Yield_hh + +// The file implements a Futex-like class which simply calls sched_yield(). + +#include + +#include "Exception.hh" +#include "compiler_magic.hh" + +namespace pbe { + + +struct Yield { + + Yield(PBE_UNUSED_ARG(int& val)) {} + ~Yield() {} + + void wait(PBE_UNUSED_ARG(int expected)) { + int r = ::sched_yield(); + if (r==-1) { + throw_ErrnoException("futex(FUTEX_WAIT)"); + } + } + + bool timed_wait(PBE_UNUSED_ARG(int expected), const timespec& timeout) { + // TODO detect timeout + wait(expected); + return true; + } + + bool timed_wait(PBE_UNUSED_ARG(int expected), float timeout) { + // TODO detect timeout + wait(expected); + return true; + } + + // Wake up to n waiters; returns the number woken. + int wake(int n=1) { + // Hmm, how much does the return value matter? + return n; + } +}; + + +}; + + +#endif + diff --git a/extras/anytermd/libpbe/include/atomic.hh b/extras/anytermd/libpbe/include/atomic.hh new file mode 100644 index 0000000000..58c80ed2f7 --- /dev/null +++ b/extras/anytermd/libpbe/include/atomic.hh @@ -0,0 +1,124 @@ +// include/atomic.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_atomic_hh +#define libpbe_atomic_hh + + +// This file provides some basic atomic operations using the gcc builtins. +// It doesn't attempt to provide an N2427-compatiable interface. +// Memory ordering sematics are those provided by the gcc builtins, +// which basically means that they all provide full memory barriers (I think). + +// We have a special version for ARM: + +#ifdef __arm__ +#include "atomic_arm.hh" +#else + +// gcc provides __sync atomic primatives from version 4.1. However, +// they didn't bother to add the feature-test-macros until version 4.2 or 4.3. +// Ideally we would ignore the gcc versions that have the primitives +// but don't tell us about them, but unfortunately that covers a few systems +// that we might be interested in. So for the time being we'll fake it: + +#if __GNUC__ == 4 && __GNUC_MINOR__ == 1 +// Maybe need to check for 4.2 too. +// This assumes that all types <= 32 bits are atomic, which is a good +// bet but not 100% certain. +#define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_1 +#define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_2 +#define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_4 +#endif + + +#if defined(__GCC_HAVE_SYNC_COMPARE_AND_SWAP_1) || defined(__GCC_HAVE_SYNC_COMPARE_AND_SWAP_2) \ + || defined(__GCC_HAVE_SYNC_COMPARE_AND_SWAP_4) + +#define PBE_HAVE_ATOMICS + +namespace pbe { + +// The N2427 proposed CAS is: +// if ( *object == *expected ) +// *object = desired; +// else +// *expected = *object; +// The boolean result of the comparison is returned. +// Note that *expected is updated if the comparison fails. This seems +// odd to me; it isn't what the gcc builtin does. I'm going to provide +// something closer to gcc. + +template +// T must of course be a type that's atomic on the hardware in use. +// I hope that some sort of error will occur if it isn't. +bool atomic_compare_and_swap(volatile T& var, T expected, T newval) { + // The return value indicates the result of the compare. + return __sync_bool_compare_and_swap(&var, expected, newval); +} + + +template +T atomic_post_dec(volatile T& var, T amt = 1) { + return __sync_fetch_and_sub(&var, amt); +} + + +template +T atomic_pre_inc(volatile T& var, T amt = 1) { + return __sync_add_and_fetch(&var, amt); +} + + +template +void atomic_inc(volatile T& var, T amt = 1) { + __sync_fetch_and_add(&var, amt); +} + + +template +void atomic_dec(volatile T& var, T amt = 1) { + __sync_fetch_and_sub(&var, amt); +} + + +template +T atomic_swap(volatile T& var, T newval) { + return __sync_lock_test_and_set(&var,newval); // misnamed +} + + +template +T atomic_read(volatile T& var) { + return var; +}; + + +template +void atomic_write(volatile T& var, T newval) { + var = newval; +}; + + +}; + +#endif + +#endif +#endif + diff --git a/extras/anytermd/libpbe/include/atomic_arm.hh b/extras/anytermd/libpbe/include/atomic_arm.hh new file mode 100644 index 0000000000..e82fe0c0d8 --- /dev/null +++ b/extras/anytermd/libpbe/include/atomic_arm.hh @@ -0,0 +1,141 @@ +// include/atomic_arm.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_atomic_arm_hh +#define libpbe_atomic_arm_hh + + +// This file provides some basic atomic operations for ARM processors +// where the only primitive is swap. + + +#define PBE_HAVE_ATOMICS + + +namespace pbe { + + +// First some asm to do a swap: + +template +static inline T simple_atomic_swap(volatile T& mem, T newval) +{ + T oldval; + asm volatile ("swp\t%0, %1, [%2]" + :"=&r"(oldval) + :"r" (newval), + "r" (&mem) + :"memory"); + return oldval; +} + +// The other atomic operations are emulated by means of a sentinel value, -1: +// the variable is first swapped with the sentinel, the operation is performed and +// the modified value is written back. Any thread that wants to read the +// variable must re-try if it reads the sentinel. + +template +static inline T atomic_read_and_lock(T& var) +{ + do { + int val = simple_atomic_swap(var,static_cast(-1)); + if (static_cast(val) != -1) { + return val; + } + } while (1); +} + + +template +// T must of course be a type that's atomic on the hardware in use. +// I hope that some sort of error will occur if it isn't. +static inline bool atomic_compare_and_swap(volatile T& var, T expected, T newval) { + // The return value indicates the result of the compare. + T oldval = atomic_read_and_lock(var); + if (oldval==expected) { + var = newval; + return true; + } else { + var = oldval; + return false; + } +} + + +template +static inline T atomic_post_dec(volatile T& var, T amt = 1) { + T val = atomic_read_and_lock(var); + var = val - amt; + return val; +} + + +template +static inline T atomic_pre_inc(volatile T& var, T amt = 1) { + T val = atomic_read_and_lock(var); + T newval = val + amt; + var = newval; + return newval; +} + + +template +static inline void atomic_inc(volatile T& var, T amt = 1) { + T val = atomic_read_and_lock(var); + T newval = val + amt; + var = newval; +} + + +template +static inline void atomic_dec(volatile T& var, T amt = 1) { + T val = atomic_read_and_lock(var); + T newval = val - amt; + var = newval; +} + + +template +static inline T atomic_swap(volatile T& var, T newval) { + T oldval = atomic_read_and_lock(var); + var = newval; + return oldval; +} + + +template +static inline T atomic_read(volatile T& var) { + do { + T val = var; + if (static_cast(val) != -1) { + return val; + } + } while (1); +}; + + +template +static inline void atomic_write(volatile T& var, T newval) { + atomic_read_and_lock(var); + var = newval; +}; + + +}; + +#endif diff --git a/extras/anytermd/libpbe/include/atomic_ofstream.hh b/extras/anytermd/libpbe/include/atomic_ofstream.hh new file mode 100644 index 0000000000..f287e44436 --- /dev/null +++ b/extras/anytermd/libpbe/include/atomic_ofstream.hh @@ -0,0 +1,92 @@ +// include/atomic_ofstream.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_atomic_ofstream_hh +#define libpbe_atomic_ofstream_hh + +#include "Exception.hh" + +#include +#include + +#include +#include + + +namespace pbe { + + +// atomic_ofstream is like std::ofstream except that the stream writes to +// a temporary file. When the file has been completely written the client +// code should call commit() which removes any existing file and moves the +// new one into its place; this should happen somewhat atomically. If +// commit() has not been called by the time the destructor is reached +// the temporary file is removed and the original file is left unchanged. + + +struct atomic_ofstream_base { + const std::string orig_fn; + const std::string tmp_fn; + atomic_ofstream_base(std::string orig_fn_): + orig_fn(orig_fn_), + tmp_fn(orig_fn+".tmp") + {} +}; + + +class atomic_ofstream: + private atomic_ofstream_base, + public std::ofstream { + + bool committed; + +public: + atomic_ofstream(std::string orig_fn_): + atomic_ofstream_base(orig_fn_), + std::ofstream(tmp_fn.c_str()), + committed(false) + {} + + void commit() { + if (!committed) { + std::ofstream::close(); + // We don't need to remove the old file; rename() does this. + int r = ::rename(tmp_fn.c_str(),orig_fn.c_str()); + if (r==-1) { + throw_ErrnoException("rename()"); + } + committed = true; + } + } + + ~atomic_ofstream() { + if (!committed) { + std::ofstream::close(); + ::unlink(tmp_fn.c_str()); + } + } + +}; + + +}; + + +#endif + + diff --git a/extras/anytermd/libpbe/include/base64.hh b/extras/anytermd/libpbe/include/base64.hh new file mode 100644 index 0000000000..8d1ef9b97a --- /dev/null +++ b/extras/anytermd/libpbe/include/base64.hh @@ -0,0 +1,202 @@ +// include/base64.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_base64_hh +#define libpbe_base64_hh + +#include +#include +#include +#include + +#include + +#include "init_array.hh" +#include "compiler_magic.hh" + + +// Base64 decoding +// =============== +// +// As specified by RFC2045 section 6.8. + + +namespace pbe { + + +// Thrown if invalid base64 data is encountered while decoding: +struct InvalidBase64 {}; + + +// An iterator adaptor that does base64 decoding. +// It wraps an underlying iterator that iterates over the bytes of the +// base64-coded data, providing an iterator over the bytes of the +// decoded data. Both iterators are input iterators. + + +template +class base64_iter +{ + +// The implementation stores 3 decoded bytes internally and iterates +// through them until they're all used; it then reads and decodes +// another 4 raw bytes from the input. + +// Note that this iterator expects you to alternately dereference and +// increment it. If you increment without dereferencing, bad things +// will happen. (Multiple dereferencing without incrementing is OK.) + +public: + typedef std::input_iterator_tag iterator_category; + typedef boost::uint8_t value_type; + typedef std::ptrdiff_t difference_type; + typedef boost::uint8_t& reference; + typedef boost::uint8_t* pointer; + + base64_iter() {} + + // Construct using the underlying iterator. + // This takes a second iterator pointing to the end of the data; + // this could be made optional for cases where it's not feasible to + // provide it (e.g. an istream_iterator), but in this case it's not + // clear how to detect and report end-of-data. + base64_iter(Iter i_, Iter i_end_): + i(i_), + i_end(i_end_), + pos(3), + bytes(0) + {} + + boost::uint8_t operator*() { + maybe_get_more_bytes(); + return bytes[pos]; + } + + base64_iter& operator++() { + ++pos; + return *this; + } + + void operator++(int) { + ++pos; + } + + bool operator==(const base64_iter& other) const { + return i==other.i && pos==other.pos; + } + + bool operator!=(const base64_iter& other) const { + return !operator==(other); + } + +private: + Iter i; + Iter i_end; + int pos; + init_array bytes; + + void maybe_get_more_bytes() { + if (pos > 2) { + get_3bytes(); + } + } + + void get_3bytes() { + int h0 = get_sixbits(); + int h1 = get_sixbits(); + int h2 = get_sixbits(); + int h3 = get_sixbits(); + // The format of each 4-character group can be: + // XXXX : 3 bytes + // XXX= : 2 bytes + // XX== : 1 byte + // There will always be 4 characters in the group. + // get_sixbits returns -1 if it sees an =. + IF_LIKELY(h0>=0 && h1>=0 && h2>=0 && h3>=0) { + bytes[0] = (h0<<2) | (h1>>4); + bytes[1] = ((h1&15)<<4) | (h2>>2); + bytes[2] = ((h2&3)<<6) | (h3); + pos -= 3; + return; + } + // In the "short" cases we put the bytes that we have received at + // the end of the bytes array. This means that once they've been + // read, end-of-data will be readed with i==i_end && pos==3. This + // should equal the base64_iter constructed from the underlying end(). + if (h0>=0 && h1>=0 && h2>=0 && h3==-1) { + bytes[1] = (h0<<2) | (h1>>4); + bytes[2] = ((h1&15)<<4) | (h2>>2); + pos -= 2; + return; + } + if (h0>=0 && h1>=0 && h2==-1 && h3==-1) { + bytes[2] = (h0<<2) | (h1>>4); + pos -= 1; + return; + } + throw InvalidBase64(); + } + + int get_sixbits() { + char c; + do { + if (i==i_end) { + return 0; + } + c = *i; + ++i; + } while (std::isspace(c)); + if (c>='a' && c<='z') return c-'a'+26; + if (c>='A' && c<='Z') return c-'A'; + if (c>='0' && c<='9') return c-'0'+52; + if (c=='+') return 62; + if (c=='/') return 63; + if (c=='=') return -1; + throw InvalidBase64(); + // The RFC says that we "must ignore" other characters, but a + // couple of lines later says that they "probably indicate a + // transmission error" ... "rejection might be appropriate". + } +}; + + + + +// Decode a std::string + +inline std::string decode_base64(const std::string& data) +{ + std::string result; + result.reserve(data.length()*3/4); + typedef base64_iter in_iter_t; + in_iter_t data_begin(data.begin(), data.end()); + in_iter_t data_end(data.end(), data.end()); + typedef std::back_insert_iterator out_iter_t; + out_iter_t result_inserter(result); + std::copy(data_begin, data_end, result_inserter); +//for (; data_begin != data_end; ++data_begin) { *(result_inserter++) = *data_begin; } + return result; +} + + + +}; + + +#endif + diff --git a/extras/anytermd/libpbe/include/bistrector.hh b/extras/anytermd/libpbe/include/bistrector.hh new file mode 100644 index 0000000000..6d3242b9a9 --- /dev/null +++ b/extras/anytermd/libpbe/include/bistrector.hh @@ -0,0 +1,274 @@ +// include/bistrector.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_bistrector_hh +#define libpbe_bistrector_hh + +// bistrector: bidirectionally-stretchy-vector +// ------------------------------------------- +// +// Provides a vector which stretches when new elements are accessed +// using operator[]. It stretches in both directions, i.e. only enough +// space for elements between the minimum and maximem indices is +// allocated. In contrast, a strector allocates space between 0 and +// the maximum index. +// +// This is actually implemented using a pair of strectors, one for elements +// above the first one added and a reversed one for those below it. A +// consequence is that if elements are erased the bistrector cannot shrink +// beyond the first element added. In fact, erasing elements in general +// is problematic because of this and is not implemented, except for clear(). +// +// The rules for invalidation etc. are the same as for strector. +// +// An additional optional constructor parameter allows you to specify a value +// used for new elements when the vector is resized during operator[] and +// when a beyond-the-end element is read as above. + + +#include "strector.hh" + +#include + + +namespace pbe { + + +template < typename T, typename ALLOC=std::allocator > +class bistrector: public std::vector { + typedef strector strector_t; + strector_t bottom; + strector_t top; + size_t mid_index; + +public: + typedef T value_type; + typedef T* pointer; + typedef T& reference; + typedef const T& const_reference; + typedef size_t size_type; + typedef ssize_t difference_type; + + + // Most of the vector ctors don't make much sense and aren't implemented. + + bistrector(): mid_index(0) {} + explicit bistrector(const T& default_t): bottom(default_t), top(default_t), mid_index(0) {} + bistrector(const bistrector& other): bottom(other.bottom), top(other.top), mid_index(other.mid_index) {} + + template + void advance_iterator(iter_t& i, bool& top_half, size_t n) { + if (n>0) { + if (top_half) { + i += n; + } else if (i-bottom.begin() >= n) { + i -= n; + } else { + i = top.begin() + (i-bottom.begin()) + (n-1); + top_half = true; + } + } else { + if (!top_half) { + i -= n; + } else if (i-top.begin() >= -n) { + i += n; + } else { + i = bottom.begin() + (i-top.begin()) + (-n-1); + top_half = false; + } + } + } + + + class iterator: + public boost::iterator_facade< iterator, T, std::random_access_iterator_tag > { + + friend class boost::iterator_core_access; + friend class bistrector; + + const bistrector& b; + bool top_half; + typename strector_t::iterator i; + + iterator(const bistrector& b_, bool top_half_, typename strector_t::iterator i_): + b(b_), top_half(top_half_), i(i_) + {} + + T& dereference() { + return *i; + } + + bool equal(const iterator& other) { + return i == other.i; + } + + void advance(size_t n) { + b.advance_iterator(i,top_half,n); + } + + void increment() { + advance(1); + } + + void decrement() { + advance(-1); + } + + }; + + + class const_iterator: + public boost::iterator_facade< const_iterator, T, std::random_access_iterator_tag, const T& > { + + friend class boost::iterator_core_access; + friend class bistrector; + + const bistrector& b; + bool top_half; + typename strector_t::const_iterator i; + + const_iterator(const bistrector& b_, bool top_half_, typename strector_t::const_iterator i_): + b(b_), top_half(top_half_), i(i_) + {} + + const T& dereference() { + return *i; + } + + bool equal(const const_iterator& other) { + return i == other.i; + } + + void advance(size_t n) { + bistrector::advance_iterator(i,top_half,n); + } + + void increment() { + advance(1); + } + + void decrement() { + advance(-1); + } + + }; + + + iterator begin() { + if (bottom.empty()) { + return iterator(*this,true,top.begin()); + } else { + return iterator(*this,false,bottom.end()-1); + } + } + + iterator end() { + return iterator(*this,true,top.end()); + } + + const_iterator begin() const { + if (bottom.empty()) { + return const_iterator(*this,true,top.begin()); + } else { + return const_iterator(*this,false,bottom.end()-1); + } + } + + const_iterator end() const { + return const_iterator(*this,true,top.end()); + } + + size_type size() const { + return top.size() + bottom.size(); + } + + size_type max_size() const { + return top.max_size() + bottom.max_size(); + } + + bool empty() const { + return top.empty() && bottom.empty(); + } + + reference operator[](size_type n) { + if (top.empty() && bottom.empty()) { + mid_index = n; + } + if (n>=mid_index) { + return top[n-mid_index]; + } else { + return bottom[mid_index-n-1]; + } + } + + const_reference operator[](size_type n) const { + if (n>=mid_index) { + return top[n-mid_index]; + } else { + return bottom[mid_index-n-1]; + } + } + + reference front() { + if (bottom.empty()) { + return top.front(); + } else { + return bottom.back(); + } + } + + const_reference front() const { + if (bottom.empty()) { + return top.front(); + } else { + return bottom.back(); + } + } + + reference back() { + if (top.empty()) { + return bottom.front(); + } else { + return top.back(); + } + } + + const_reference back() const { + if (top.empty()) { + return bottom.front(); + } else { + return top.back(); + } + } + + void push_back(const T& t) { + top.push_back(t); + } + + void clear() { + top.clear(); + bottom.clear(); + } +}; + + +}; + + +#endif + diff --git a/extras/anytermd/libpbe/include/bloom_filter.hh b/extras/anytermd/libpbe/include/bloom_filter.hh new file mode 100644 index 0000000000..6c70203440 --- /dev/null +++ b/extras/anytermd/libpbe/include/bloom_filter.hh @@ -0,0 +1,129 @@ +// include/bloom_filter.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#ifndef bloom_filter_hh +#define bloom_filter_hh + +#include + +// This class implements an approximation to set using a Bloom Filter. +// It's approximate in the senses that: +// - It sometimes returns false positives. (It never returns false negatives.) +// - It provides only a subset of the features of a real set, in particular it does +// not support iteration or erasure of elements. Only insertion and testing are +// provided. +// +// Insertion and testing are both O(1) in the number of keys stored. +// +// See e.g. the Wikipedia Bloom Filter page for the theory. +// +// There are a few template parameters that you need to set: +// Choose n_bits based on the acceptable probability of false positives (f) and the +// number of stored keys (n_keys) to satisfy: +// f = 0.618 ^ (n_bits/n_keys) +// => n_bits = n_keys ln(f)/-0.48 +// It's best to round n_bits to a power of 2; if you don't, mod is needed +// at run-time. In any case it must be a multiple of 32. +// Space is O(n_bits). +// +// Choose n_hashes using +// n_hashes = 0.7 n_bits / n_keys +// Insertion and testing are O(n_hashes). Using fewer hashes will make these operations +// faster at the expense of more false positives; this can be offset by increasing +// n_bits i.e. trading off time vs. space. Using more hashes will NOT reduce the rate of +// false positivies. +// +// (Hmm, I now wonder if the better approach is to choose n_hashes based on f, and then +// to choose n_bits.) +// +// So for example, to store the 56840 words in /usr/share/dict/words with a probability +// of a false positive of 1% (i.e. 1% of your mis-spellings are identified as correct), +// we get n_bits = 545329 which rounds down to 512*1024 or up to 1024*1024. n_hashes +// is 9 or 18 respectively. +// +// These values are theoretical optimums; in practice you may need to experiment a bit +// to find good values if you're fussy about the space required and the false positive rate. +// +// You need to supply a hash function. It takes the key as an argument and returns a tr1::array +// of n_hashes hash values, which much each have at least log2(n_bits) bits. +// Possible sources of hash functions are: +// - The standard library includes them, returning size_t, for integer types and std::string. +// - Boost.CRC. +// +// In cases where you want to do the same insert or lookup on multiple filters, it makes +// sense to do the hash calculation only once. To allow this the insert and check methods +// are overloaded with versions that take hash results. + + +template < typename KEY_TYPE, + int n_bits, + int n_hashes, + typename HASH + > +class bloom_filter { +public: + typedef KEY_TYPE key_type; + typedef std::tr1::array< unsigned int, n_hashes > hash_return_type; + + bloom_filter() { + bits_t b_ = {{0}}; + bits = b_; + } + + void insert(const hash_return_type& h) { + for (int k=0; k bits_t; + bits_t bits; + + void set_bit(int n) { + bits[n>>5] |= 1<<(n&31); + } + + bool test_bit(int n) const { + return bits[n>>5] & (1<<(n&31)); + } +}; + + +#endif + diff --git a/extras/anytermd/libpbe/include/charset/char_conv.hh b/extras/anytermd/libpbe/include/charset/char_conv.hh new file mode 100644 index 0000000000..e4ed7a670e --- /dev/null +++ b/extras/anytermd/libpbe/include/charset/char_conv.hh @@ -0,0 +1,116 @@ +// char_conv.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2008 Philip Endecott + +// Distributed under the Boost Software License, Version 1.0: +// +// Permission is hereby granted, free of charge, to any person or organization +// obtaining a copy of the software and accompanying documentation covered by +// this license (the "Software") to use, reproduce, display, distribute, +// execute, and transmit the Software, and to prepare derivative works of the +// Software, and to permit third-parties to whom the Software is furnished to +// do so, all subject to the following: +// +// The copyright notices in the Software and this entire statement, including +// the above license grant, this restriction and the following disclaimer, +// must be included in all copies of the Software, in whole or in part, and +// all derivative works of the Software, unless such copies or derivative +// works are solely in the form of machine-executable object code generated by +// a source language processor. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT +// SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE +// FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, +// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER +// DEALINGS IN THE SOFTWARE. + +#ifndef libpbe_charset_char_conv_hh +#define libpbe_charset_char_conv_hh + +#include "charset_t.hh" +#include "charset_traits.hh" +#include "error_policy.hh" + +#include + +#include "compiler_magic.hh" + + +namespace pbe { + +// Conversion of characters between character sets +// ----------------------------------------------- +// +// This file provides facilities to convert a single character from +// one character set to another. It's implemented using specialisations +// for character set pairs in other files. +// A template parameter specifies a policy for error handling. +// Because function templates can't be partially specialised, and +// the character-set-pair specialisation will not want to be specialised +// on the error handling polocy, we need to wrap the conversion in a +// class. +// The default implementation here converts to a UCS4 character as an +// intermediate step, which will work as long as the necessary supporting +// conversions are implemented somehow. +// There's also a specialisation here for the null conversion where source +// and destination are the same. + + +// Base template, with default implementation via ucs4: + +template +struct char_conv { + // We shouldn't be here if from_cs or to_cs is ucs4; that indicates recursion. + // A specialisation should have been invoked. Maybe the appropriate + // specialisation hasn't been #included, or hasn't been written? + BOOST_STATIC_ASSERT(from_cs!=cs::ucs4 && to_cs!=cs::ucs4); + + // FIXME shouldn't this be a static member function? + typename charset_traits::char_t + operator() ( typename charset_traits::char_t c, + typename charset_traits::state_t& from_state + = charset_traits::state_t(), + typename charset_traits::state_t& to_state + = charset_traits::state_t() ) + { + charset_traits::state_t ucs4_state; + char32_t tmp = char_conv() (c,from_state,ucs4_state); + return char_conv() (tmp,ucs4_state,to_state); + } +}; + + +// Specialisation for trivial conversion to the same character set: +template +struct char_conv { + typename charset_traits::char_t + operator() ( typename charset_traits::char_t c, + PBE_UNUSED_ARG(typename charset_traits::state_t& from_state) + = charset_traits::state_t(), + PBE_UNUSED_ARG(typename charset_traits::state_t& to_state) + = charset_traits::state_t() ) + { + return c; + } +}; + + +// This macro is used elsewhere to define trivial conversions. + +#define IDENTITY_CHAR_CONV(FROM_CS,TO_CS) \ +template \ +struct char_conv { \ + charset_traits::char_t \ + operator() ( charset_traits::char_t c, \ + charset_traits::state_t&, \ + charset_traits::state_t& ) { \ + return static_cast::char_t>(c); \ + } \ +}; + + +}; + +#endif diff --git a/extras/anytermd/libpbe/include/charset/char_converter.hh b/extras/anytermd/libpbe/include/charset/char_converter.hh new file mode 100644 index 0000000000..98b992c118 --- /dev/null +++ b/extras/anytermd/libpbe/include/charset/char_converter.hh @@ -0,0 +1,67 @@ +// char_converter.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2008 Philip Endecott + +// Distributed under the Boost Software License, Version 1.0: +// +// Permission is hereby granted, free of charge, to any person or organization +// obtaining a copy of the software and accompanying documentation covered by +// this license (the "Software") to use, reproduce, display, distribute, +// execute, and transmit the Software, and to prepare derivative works of the +// Software, and to permit third-parties to whom the Software is furnished to +// do so, all subject to the following: +// +// The copyright notices in the Software and this entire statement, including +// the above license grant, this restriction and the following disclaimer, +// must be included in all copies of the Software, in whole or in part, and +// all derivative works of the Software, unless such copies or derivative +// works are solely in the form of machine-executable object code generated by +// a source language processor. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT +// SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE +// FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, +// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER +// DEALINGS IN THE SOFTWARE. + +#ifndef libpbe_charset_char_converter_hh +#define libpbe_charset_char_converter_hh + +#include "charset_t.hh" +#include "charset_traits.hh" +#include "char_conv.hh" +#include "error_policy.hh" + + +namespace pbe { + +// Stateful functor for character conversion +// ----------------------------------------- +// +// The function template char_conv takes state variables for the input and output, +// in case they are stateful. +// A simplified interface is provided by this functor, which keeps the state +// internally from one call to the next. + +template +class char_converter { + typedef charset_traits from_traits; + typedef charset_traits to_traits; + + typename from_traits::state_t from_state; + typename to_traits::state_t to_state; + +public: + char_conv cc; + typename to_traits::char_t operator()(typename from_traits::char_t c) { + return cc ( c, from_state, to_state ); + } + + // TODO flush, perhaps reset. +}; + +}; + +#endif diff --git a/extras/anytermd/libpbe/include/charset/char_t.hh b/extras/anytermd/libpbe/include/charset/char_t.hh new file mode 100644 index 0000000000..4f5d50dd90 --- /dev/null +++ b/extras/anytermd/libpbe/include/charset/char_t.hh @@ -0,0 +1,72 @@ +// char_t.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2007-2008 Philip Endecott + +// Distributed under the Boost Software License, Version 1.0: +// +// Permission is hereby granted, free of charge, to any person or organization +// obtaining a copy of the software and accompanying documentation covered by +// this license (the "Software") to use, reproduce, display, distribute, +// execute, and transmit the Software, and to prepare derivative works of the +// Software, and to permit third-parties to whom the Software is furnished to +// do so, all subject to the following: +// +// The copyright notices in the Software and this entire statement, including +// the above license grant, this restriction and the following disclaimer, +// must be included in all copies of the Software, in whole or in part, and +// all derivative works of the Software, unless such copies or derivative +// works are solely in the form of machine-executable object code generated by +// a source language processor. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT +// SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE +// FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, +// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER +// DEALINGS IN THE SOFTWARE. + +#ifndef libpbe_charset_char_t_hh +#define libpbe_charset_char_t_hh + +#include + +namespace pbe { + +// Defined-size character types +// ---------------------------- +// +// The built-in types char and wchar_t have different numbers of bits on +// different platforms. C++0x will probably have new char types of determined +// sizes; this file provides three typedefs, char8_t, char16_t and char32_t +// that do the same sort of thing. These map to char and wchar_t when +// appropriate, and to an unsigned integer of the appropriate size (using +// Boost.Integer) otherwise. +// +// The specialised template class char_t provides a typedef 'type' for an +// N-bit character. + + +template +struct char_t { + typedef typename boost::uint_t::least type; +}; + +template <> +struct char_t<8*sizeof(char)> { + typedef char type; +}; + +template <> +struct char_t<8*sizeof(wchar_t)> { + typedef wchar_t type; +}; + +typedef char_t<8>::type char8_t; +typedef char_t<16>::type char16_t; +typedef char_t<32>::type char32_t; + + +}; + +#endif diff --git a/extras/anytermd/libpbe/include/charset/character_output_iterator.hh b/extras/anytermd/libpbe/include/charset/character_output_iterator.hh new file mode 100644 index 0000000000..2948f3db1d --- /dev/null +++ b/extras/anytermd/libpbe/include/charset/character_output_iterator.hh @@ -0,0 +1,84 @@ +// character_output_iterator.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2007-2008 Philip Endecott + +// Distributed under the Boost Software License, Version 1.0: +// +// Permission is hereby granted, free of charge, to any person or organization +// obtaining a copy of the software and accompanying documentation covered by +// this license (the "Software") to use, reproduce, display, distribute, +// execute, and transmit the Software, and to prepare derivative works of the +// Software, and to permit third-parties to whom the Software is furnished to +// do so, all subject to the following: +// +// The copyright notices in the Software and this entire statement, including +// the above license grant, this restriction and the following disclaimer, +// must be included in all copies of the Software, in whole or in part, and +// all derivative works of the Software, unless such copies or derivative +// works are solely in the form of machine-executable object code generated by +// a source language processor. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT +// SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE +// FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, +// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER +// DEALINGS IN THE SOFTWARE. + +#ifndef libpbe_charset_character_output_iterator_hh +#define libpbe_charset_character_output_iterator_hh + +#include "charset_t.hh" +#include "charset_traits.hh" + +namespace pbe { + +// Character Output Iterator Adapter +// --------------------------------- +// +// This iterator adapter takes an output iterator over a string's units +// and supplies an output iterator that stores characters. This is of +// course trivial for fixed-length character sets; for variable-length +// character sets the encoding is performed by a function supplied by the +// character set's traits class. +// The supplied iterator is only an output iterator: the only permitted +// pattern of use is to alternately store characters and increment the +// iterator. Typically it will be used with an appending ("back insert") +// iterator: +// +// typedef std::container ctr_t; +// ctr_t ctr; +// typedef std::back_insert_iterator unit_iter_t; +// unit_iter_t ui(ctr); // ui inserts units +// typedef character_output_iterator char_iter_t; +// char_iter_t ci(ui); // ci inserts characters +// *(ci++) = 0x1234; // some random non-ASCII character + + +template +class character_output_iterator { + unit_iterator_t i; + + struct assignment_proxy { + unit_iterator_t& i; + assignment_proxy(unit_iterator_t& i_): i(i_) {} + void operator=(typename charset_traits::char_t c) { + charset_traits::encode(i,c); + } + }; + +public: + character_output_iterator() {} + character_output_iterator(const unit_iterator_t& i_): i(i_) {} + assignment_proxy operator*() { return assignment_proxy(i); } + character_output_iterator& operator++() { return *this; } + character_output_iterator& operator++(int) { return *this; } + unit_iterator_t& base() { return i; } +}; + + + +}; + +#endif diff --git a/extras/anytermd/libpbe/include/charset/charset_char_traits.hh b/extras/anytermd/libpbe/include/charset/charset_char_traits.hh new file mode 100644 index 0000000000..01ffd3e56c --- /dev/null +++ b/extras/anytermd/libpbe/include/charset/charset_char_traits.hh @@ -0,0 +1,80 @@ +// charset_char_traits.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2007-2008 Philip Endecott + +// Distributed under the Boost Software License, Version 1.0: +// +// Permission is hereby granted, free of charge, to any person or organization +// obtaining a copy of the software and accompanying documentation covered by +// this license (the "Software") to use, reproduce, display, distribute, +// execute, and transmit the Software, and to prepare derivative works of the +// Software, and to permit third-parties to whom the Software is furnished to +// do so, all subject to the following: +// +// The copyright notices in the Software and this entire statement, including +// the above license grant, this restriction and the following disclaimer, +// must be included in all copies of the Software, in whole or in part, and +// all derivative works of the Software, unless such copies or derivative +// works are solely in the form of machine-executable object code generated by +// a source language processor. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT +// SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE +// FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, +// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER +// DEALINGS IN THE SOFTWARE. + +#ifndef libpbe_charset_charset_char_traits_hh +#define libpbe_charset_charset_char_traits_hh + +#include "charset_t.hh" +#include "charset_traits.hh" + +#include + + +namespace pbe { + +// Character Set Character Traits +// ------------------------------ +// +// It would be great if we could write +// std::char_traits +// but unfortunately we can't as iso8859_1_char is only a typedef, and +// std::char_traits can't be specialised differently for it than for char, +// iso8859_2_char, and all the others. +// +// So instead we can write +// charset_char_traits +// which is a traits class compatible with std::char_traits, for iso-8859-1 +// characters. We also have +// charset_unit_traits +// which gives the character traits for the character set's unit type, rather +// than its character type, which will be different in the case of variable +// length character sets like utf8. +// They are all defined in terms of charset_traits and std::char_traits. + +template +struct charset_char_traits { + typedef charset_traits::char_t char_type; + typedef std::char_traits::int_t int_type; + typedef std::char_traits::pos_t pos_type; + typedef std::char_traits::off_t off_type; + typedef charset_traits::state_t state_type; +}; + +template +struct charset_unit_traits { + typedef charset_traits::unit_t char_type; + typedef std::char_traits::int_t int_type; + typedef std::char_traits::pos_t pos_type; + typedef std::char_traits::off_t off_type; + typedef charset_traits::state_t state_type; +}; + +// FIXME the state_types in the two versions should be DIFFERENT, right??? + + +#endif diff --git a/extras/anytermd/libpbe/include/charset/charset_names.hh b/extras/anytermd/libpbe/include/charset/charset_names.hh new file mode 100644 index 0000000000..1d743ba59f --- /dev/null +++ b/extras/anytermd/libpbe/include/charset/charset_names.hh @@ -0,0 +1,74 @@ +// charset_names.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2008 Philip Endecott + +// Distributed under the Boost Software License, Version 1.0: +// +// Permission is hereby granted, free of charge, to any person or organization +// obtaining a copy of the software and accompanying documentation covered by +// this license (the "Software") to use, reproduce, display, distribute, +// execute, and transmit the Software, and to prepare derivative works of the +// Software, and to permit third-parties to whom the Software is furnished to +// do so, all subject to the following: +// +// The copyright notices in the Software and this entire statement, including +// the above license grant, this restriction and the following disclaimer, +// must be included in all copies of the Software, in whole or in part, and +// all derivative works of the Software, unless such copies or derivative +// works are solely in the form of machine-executable object code generated by +// a source language processor. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT +// SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE +// FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, +// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER +// DEALINGS IN THE SOFTWARE. + +#ifndef libpbe_charset_charset_names_hh +#define libpbe_charset_charset_names_hh + +#include + +#include "charset_t.hh" + +namespace pbe { + +// Character Set Names +// ------------------- +// +// This file provides functions for converting between character set names +// in ASCII strings and charset_t values. +// +// Several names may be recorded for each character set. The primary +// IANA-registered name is returned by charset_name(). charset_mime_name() +// returns the same as charset_name(), unless a different name is preferred +// for MIME purposes. (Typically the preferred MIME name is more concise, +// while the primary name is less ambiguous.) lookup_charset() recognises +// both of these names plus a large number of IANA-defined aliases and +// many variations in case and punctuation. + + +extern const char* const charset_iana_names[]; + +inline const char* charset_name(charset_t cs) { + return charset_iana_names[cs]; +} + + +extern const char* const charset_mime_names[]; + +inline const char* charset_mime_name(charset_t cs) { + return charset_mime_names[cs]; +} + + +struct NoSuchCharacterSet {}; // FIXME maybe we need a no-throw version too? + +charset_t lookup_charset(const char* name); +charset_t lookup_charset(std::string name); + +}; + +#endif diff --git a/extras/anytermd/libpbe/include/charset/charset_t.hh b/extras/anytermd/libpbe/include/charset/charset_t.hh new file mode 100644 index 0000000000..cceb6ac715 --- /dev/null +++ b/extras/anytermd/libpbe/include/charset/charset_t.hh @@ -0,0 +1,55 @@ +// charset_t.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2008 Philip Endecott + +// Distributed under the Boost Software License, Version 1.0: +// +// Permission is hereby granted, free of charge, to any person or organization +// obtaining a copy of the software and accompanying documentation covered by +// this license (the "Software") to use, reproduce, display, distribute, +// execute, and transmit the Software, and to prepare derivative works of the +// Software, and to permit third-parties to whom the Software is furnished to +// do so, all subject to the following: +// +// The copyright notices in the Software and this entire statement, including +// the above license grant, this restriction and the following disclaimer, +// must be included in all copies of the Software, in whole or in part, and +// all derivative works of the Software, unless such copies or derivative +// works are solely in the form of machine-executable object code generated by +// a source language processor. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT +// SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE +// FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, +// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER +// DEALINGS IN THE SOFTWARE. + +#ifndef libpbe_charset_charset_t_hh +#define libpbe_charset_charset_t_hh + +namespace pbe { + +// Definition of charset_t +// ----------------------- +// +// This file defines an enum, charset_t, which names character sets. +// The body of the enum is populated by an automatically-generated file +// generated from the IANA list of character sets. + +// The character set names include some rather terse ones, e.g. "l1" and "no", +// so to avoid collisions they're kept in their own sub-namespace, pbe::cs. +// The type charset_t itself is brought up to the main pbe:: namesapce. + +namespace cs { +#include "generated/charset_t_enum_body.hh" +}; + +using cs::charset_t; + + +}; + +#endif + diff --git a/extras/anytermd/libpbe/include/charset/charset_traits.hh b/extras/anytermd/libpbe/include/charset/charset_traits.hh new file mode 100644 index 0000000000..27342e7f88 --- /dev/null +++ b/extras/anytermd/libpbe/include/charset/charset_traits.hh @@ -0,0 +1,416 @@ +// charset_traits.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2007-2008 Philip Endecott + +// Distributed under the Boost Software License, Version 1.0: +// +// Permission is hereby granted, free of charge, to any person or organization +// obtaining a copy of the software and accompanying documentation covered by +// this license (the "Software") to use, reproduce, display, distribute, +// execute, and transmit the Software, and to prepare derivative works of the +// Software, and to permit third-parties to whom the Software is furnished to +// do so, all subject to the following: +// +// The copyright notices in the Software and this entire statement, including +// the above license grant, this restriction and the following disclaimer, +// must be included in all copies of the Software, in whole or in part, and +// all derivative works of the Software, unless such copies or derivative +// works are solely in the form of machine-executable object code generated by +// a source language processor. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT +// SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE +// FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, +// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER +// DEALINGS IN THE SOFTWARE. + +#ifndef libpbe_charset_charset_traits_hh +#define libpbe_charset_charset_traits_hh + +#include "charset_t.hh" +#include "char_t.hh" + + +namespace pbe { + +// Character Set Traits +// -------------------- +// +// charset_traits provides information about a character set cs, identified +// by a member of the charset_t enumeration. This information comprises: +// - The character type, char_t. +// - The unit type, unit_t. +// - The state type (for character sets like Shift-JIS), state_t. +// The character and unit types differ for variable-length encodings. +// Since the majority of the character sets in charset_t are fixed-length +// byte character sets with no shift-state, this is defined as a default. +// Note that different character sets' state types are distinct, even if they +// are all stateless character sets. +// - The boolean constants is_ascii_superset and is_iso8859_common_superset, +// which indicate whether the character set shares codes 0-127 with ASCII +// and whether it shares code 0-160 (inclusive) with the iso-8859 character +// sets, respectively. (The iso-8859 character sets are all ASCII supersets, +// and they also have in common with each other characters 128 to 160 +// inclusive; 128 to 159 are control characters and 160 is non-breaking space. +// - The following functions, for conversion between the character type and +// a sequence of units: +// template char_t decode(const_unit_ptr_t& p); +// Decodes and returns one character pointed to by p, and advances p +// to point to the next character. +// template void encode(unit_ptr_t& p, char_t c); +// Encodes the character c and stores it starting at *p, and advances +// p to point after the character that it has written. +// - The following functions, for per-character iteration though units: +// template void skip_forward_char(const_unit_ptr_t& p); +// Advance p to point to the next character. +// template void skip_backward_char(const_unit_ptr_t& p); +// Move p back to point to the previous character. +// template int char_length(const_unit_ptr_t& p); +// Returns the length in units of the character pointed to by p. +// - The following to help client code allocate buffer space: +// size_t max_characters(size_t n_units) +// size_t typ_characters(size_t n_units) +// size_t max_units(size_t n_characters) +// size_t typ_units(size_t n_characters) + +// Maybe add: check whether units pointed to are valid. + +// For fixed-length character sets, most of these things are trivial; the +// only variable is whether they have the common ASCII or ISO-8859 subset. +// Specialisations for such cases can inherit from one of these: + +// cset must be a template parameter so that the state_t for each character +// set is distinct. +template +struct fixed_charset_traits { + typedef C char_t; + typedef C unit_t; + struct state_t {}; + static const bool is_ascii_superset = false; + static const bool is_ascii_plus_c1_superset = false; + template static char_t decode(const_unit_ptr_t& p) { return *(p++); } + template static void encode(unit_ptr_t& p, char_t c) { *(p++) = c; } + template static void skip_forward_char(const_unit_ptr_t& p) { ++p; } + template static void skip_backward_char(const_unit_ptr_t& p) { --p; } + template static int char_length(const_unit_ptr_t& p) { return 1; } + static size_t max_characters(size_t n_units) { return n_units; } + static size_t typ_characters(size_t n_units) { return n_units; } + static size_t max_units(size_t n_characters) { return n_characters; } + static size_t typ_units(size_t n_characters) { return n_characters; } +}; + +template +struct fixed_byte_charset_traits: fixed_charset_traits {}; + +template +struct fixed_byte_ascii_superset_charset_traits: fixed_byte_charset_traits { + static const bool is_ascii_superset = true; + static const bool is_ascii_plus_c1_superset = false; +}; + +template +struct fixed_byte_iso8859_common_superset_charset_traits: fixed_byte_charset_traits { + static const bool is_ascii_superset = true; + static const bool is_ascii_plus_c1_superset = true; +}; + + +// Here is the base charset traits class, for which specialisations are made: + +template +struct charset_traits {}; + + +// For many of the fixed-length character sets, we define specialisations here: + +template <> struct charset_traits: fixed_byte_ascii_superset_charset_traits {}; + +// The ISO-8859 character sets: +template <> struct charset_traits: fixed_byte_iso8859_common_superset_charset_traits {}; +template <> struct charset_traits: fixed_byte_iso8859_common_superset_charset_traits {}; +template <> struct charset_traits: fixed_byte_iso8859_common_superset_charset_traits {}; +template <> struct charset_traits: fixed_byte_iso8859_common_superset_charset_traits {}; +template <> struct charset_traits: fixed_byte_iso8859_common_superset_charset_traits {}; +template <> struct charset_traits: fixed_byte_iso8859_common_superset_charset_traits {}; +template <> struct charset_traits: fixed_byte_iso8859_common_superset_charset_traits {}; +template <> struct charset_traits: fixed_byte_iso8859_common_superset_charset_traits {}; +template <> struct charset_traits: fixed_byte_iso8859_common_superset_charset_traits {}; +template <> struct charset_traits: fixed_byte_iso8859_common_superset_charset_traits {}; +template <> struct charset_traits: fixed_byte_iso8859_common_superset_charset_traits {}; +template <> struct charset_traits: fixed_byte_iso8859_common_superset_charset_traits {}; +template <> struct charset_traits: fixed_byte_iso8859_common_superset_charset_traits {}; +template <> struct charset_traits: fixed_byte_iso8859_common_superset_charset_traits {}; +// These sound related, but I don't know what they are: +// iso8859_6_e +// iso8859_6_i +// iso8859_supp + +// The ISO-646 character sets. These are 7-bit character sets with most characters shared +// with ASCII, but with some punctuation characters replaced by national accented characters +// and symbols. +// It might be useful to have a trait indicating that they're "approximately" ASCII. +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +// This sounds related, but I don't know what it is: +// ISO_646.basic:1983 + +// These "code pages" are all 8-bit fixed-length encodings. +// Some/many of them will be ASCII supersets but this isn't identified here. +// (In some/most cases a definition is provided in the Unicode mapping tables.) +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; +template <> struct charset_traits: fixed_byte_charset_traits {}; + +// The cyrillic KOI8 character sets are ASCII supersets. +template <> struct charset_traits: fixed_byte_ascii_superset_charset_traits {}; +template <> struct charset_traits: fixed_byte_ascii_superset_charset_traits {}; +template <> struct charset_traits: fixed_byte_ascii_superset_charset_traits {}; + + +// The fixed Unicode formats: +template <> struct charset_traits: fixed_charset_traits {}; +template <> struct charset_traits: fixed_charset_traits {}; + + +// For the variable-length, specialisations are +// provided in separate files. These are: +// utf8 +// utf16 +// iso_10646_utf_1 +// unicode_1_1_utf_7 +// UTF-7 +// UTF16-BE +// UTF16-LE +// UTF32 +// UTF32-BE +// UTF32-LE +// iso_2022_kr +// iso_2022_jp +// iso_2022_jp_2 +// iso_2022_cn +// iso_2022_cn_ext +// shift_jis + + +// I don't yet know enough about the following character sets to supply traits +// for them. I do not aim to do so for every single character set. + +// iso_ir_142 +// jis_x0201 +// jis_encoding +// euc_jp +// Extended_UNIX_Code_Fixed_Width_for_Japanese +// INVARIANT +// iso_ir_2 +// iso_ir_8_1 +// iso_ir_8_2 +// iso_ir_9_1 +// iso_ir_9_2 +// iso_ir_149 +// euc_kr +// iso_ir_13 +// iso_ir_14 +// iso_ir_18 +// iso_ir_19 +// iso_ir_27 +// iso_ir_37 +// iso_ir_42 +// iso_ir_47 +// iso_ir_49 +// iso_ir_50 +// iso_ir_51 +// iso_ir_52 +// iso_ir_53 +// iso_ir_54 +// iso_ir_55 +// iso_ir_57 +// iso_ir_58 +// iso_ir_70 +// iso_ir_87 +// iso_ir_88 +// iso_ir_89 +// iso_ir_90 +// iso_ir_91 +// iso_ir_93 +// iso_ir_94 +// iso_ir_95 +// iso_ir_96 +// iso_ir_98 +// iso_ir_99 +// iso_ir_102 +// iso_ir_103 +// iso_ir_103 +// iso_ir_123 +// iso_ir_139 +// iso_ir_143 +// iso_ir_146 +// iso_ir_147 +// iso_ir_150 +// iso_ir_152 +// iso_ir_153 +// iso_ir_155 +// iso_ir_158 +// iso_ir_159 +// us-dk +// dk-us +// gb18030 +// OSD_EBCDIC_DF04_15 +// OSD_EBCDIC_DF03_IRV +// OSD_EBCDIC_DF04_1 +// ISO-11548-1 +// KZ-1048 +// ISO-10646-UCS-Basic +// ISO-10646-Unicode-Latin1 +// ISO-10646-J-1 +// ISO-Unicode-IBM-1261 +// ISO-Unicode-IBM-1268 +// ISO-Unicode-IBM-1276 +// ISO-Unicode-IBM-1264 +// ISO-Unicode-IBM-1265 +// UNICODE-1-1 +// SCSU +// CESU-8 +// BOCU-1 +// ISO-8859-1-Windows-3.0-Latin-1 +// ISO-8859-1-Windows-3.1-Latin-1 +// ISO-8859-2-Windows-Latin-2 +// ISO-8859-9-Windows-Latin-5 +// hp_roman8 +// Adobe-Standard-Encoding +// Ventura-US +// Ventura-International +// DEC-MCS +// PC8-Danish-Norwegian +// PC8-Turkish +// IBM-Symbols +// IBM-Thai +// HP-Legal +// HP-Pi-font +// HP-Math8 +// Adobe-Symbol-Encoding +// HP-DeskTop +// Ventura-Math +// Microsoft-Publishing +// Windows-31J +// GB2312 +// Big5 +// macintosh +// EBCDIC-AT-DE +// EBCDIC-AT-DE-A +// EBCDIC-CA-FR +// EBCDIC-DK-NO +// EBCDIC-DK-NO-A +// EBCDIC-FI-SE +// EBCDIC-FI-SE-A +// EBCDIC-FR +// EBCDIC-IT +// EBCDIC-PT +// EBCDIC-ES +// EBCDIC-ES-A +// EBCDIC-ES-S +// EBCDIC-UK +// EBCDIC-US +// UNKNOWN-8BIT +// MNEMONIC +// MNEM +// VISCII +// VIQR +// HZ-GB-2312 +// Big5-HKSCS +// Amiga1251 +// KOI7-switched +// BRF +// TSCII +// TIS-620 + + + +}; + + + +#endif diff --git a/extras/anytermd/libpbe/include/charset/const_character_iterator.hh b/extras/anytermd/libpbe/include/charset/const_character_iterator.hh new file mode 100644 index 0000000000..6fd7117c68 --- /dev/null +++ b/extras/anytermd/libpbe/include/charset/const_character_iterator.hh @@ -0,0 +1,102 @@ +// const_character_iterator.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2007-2008 Philip Endecott + +// Distributed under the Boost Software License, Version 1.0: +// +// Permission is hereby granted, free of charge, to any person or organization +// obtaining a copy of the software and accompanying documentation covered by +// this license (the "Software") to use, reproduce, display, distribute, +// execute, and transmit the Software, and to prepare derivative works of the +// Software, and to permit third-parties to whom the Software is furnished to +// do so, all subject to the following: +// +// The copyright notices in the Software and this entire statement, including +// the above license grant, this restriction and the following disclaimer, +// must be included in all copies of the Software, in whole or in part, and +// all derivative works of the Software, unless such copies or derivative +// works are solely in the form of machine-executable object code generated by +// a source language processor. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT +// SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE +// FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, +// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER +// DEALINGS IN THE SOFTWARE. + + +#ifndef libpbe_charset_const_character_iterator_hh +#define libpbe_charset_const_character_iterator_hh + +#include "charset_t.hh" +#include "charset_traits.hh" + +#include + + +namespace pbe { + + +// const_character_iterator +// ------------------------ +// +// This iterator adaptor converts an iterator over the units of a string +// to an iterator over the characters. +// This is of course a trivial operation for fixed-length character sets, +// and non-trivial for variable-length ones. The character-set-specific +// details are provided by various functions in the character set's traits +// class. +// The iterator is implemented using Boost's iterator_adaptor class. +// +// Typical use is as follows: +// std::container ctr; // some sort of container containing bytes, +// // in UTF-8. +// std::container::const_iterator i = ctr.begin(); +// // i iterates over the bytes. +// const_character_iterator j = i; +// // j iterates over the decoded characters. +// +// The const_character_iterator is, as the name suggests, a const iterator: +// it's not possible to change the string using it since that could require +// that the string grows or shrinks. It's also not a random-access iterator, +// even if the underlying iterator is random-access, since random access +// is O(n) rather than O(1). + + +template +class const_character_iterator: + public boost::iterator_adaptor, + unit_iterator_t, + typename charset_traits::char_t const, + boost::bidirectional_traversal_tag, + // FIXME should be min(unit_iterator_t,bidirectional) + typename charset_traits::char_t const> +{ + typedef charset_traits traits; + typename traits::char_t dereference() const { + unit_iterator_t i = const_character_iterator::iterator_adaptor_::base(); + return traits::decode(i); + } + void increment(void) { + traits::skip_forward_char(const_character_iterator::iterator_adaptor_::base_reference()); + } + void decrement(void) { + traits::skip_backward_char(const_character_iterator::iterator_adaptor_::base_reference()); + } + + friend class boost::iterator_core_access; + +public: + const_character_iterator() {} + const_character_iterator(const unit_iterator_t& iter): + const_character_iterator::iterator_adaptor_(iter) + {} +}; + + +}; + + +#endif diff --git a/extras/anytermd/libpbe/include/charset/contig_sequence_conv.hh b/extras/anytermd/libpbe/include/charset/contig_sequence_conv.hh new file mode 100644 index 0000000000..5871f02e09 --- /dev/null +++ b/extras/anytermd/libpbe/include/charset/contig_sequence_conv.hh @@ -0,0 +1,148 @@ +// contig_sequence_conv.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2008 Philip Endecott + +// Distributed under the Boost Software License, Version 1.0: +// +// Permission is hereby granted, free of charge, to any person or organization +// obtaining a copy of the software and accompanying documentation covered by +// this license (the "Software") to use, reproduce, display, distribute, +// execute, and transmit the Software, and to prepare derivative works of the +// Software, and to permit third-parties to whom the Software is furnished to +// do so, all subject to the following: +// +// The copyright notices in the Software and this entire statement, including +// the above license grant, this restriction and the following disclaimer, +// must be included in all copies of the Software, in whole or in part, and +// all derivative works of the Software, unless such copies or derivative +// works are solely in the form of machine-executable object code generated by +// a source language processor. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT +// SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE +// FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, +// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER +// DEALINGS IN THE SOFTWARE. + +#ifndef libpbe_charset_contig_sequence_conv_hh +#define libpbe_charset_contig_sequence_conv_hh + +#include "charset_t.hh" +#include "sequence_conv.hh" +#include "const_character_iterator.hh" +#include "charset_traits.hh" + +#include + + +namespace pbe { + +// Conversion of strings in contiguous memory +// ------------------------------------------ +// +// This file provides a specialisation of sequence_conv for data in +// contiguous memory, for which an optimised algorithm is possible. + +// Some background can be found at +// http://gcc.gnu.org/ml/libstdc++/2005-11/msg00275.html + +// There are at least these cases where the elements are contiguous: +// - raw memory, i.e. the iterator is a pointer (guaranteed) +// - std::vector (guaranteed) +// - std::valarray (guaranteed, I think) +// - tr1::array / boost::array (guaranteed, I think) +// - std::basic_string (not formally guaranteed, but true in practice) +// It would be good to be able to automatically detect other cases, +// e.g. user-defined array-like classes, but I don't know how that can be +// done. +// This file includes the optimisations for raw memory (pointers), vectors +// and strings. Adding the others would be straightforward; the only +// issue is that it would bring in their headers. + + +template +static uint8_t byte_n(unsigned int i) { + return (i>>(8*n))&0xff; +} + + +// First we specialise for a character_iterator over a unit_t*: + +template< charset_t from_cs, charset_t to_cs, typename error_policy, typename OutputIterator > +struct sequence_conv< from_cs, to_cs, error_policy, + const_character_iterator::unit_t*>, + OutputIterator, + typename boost::enable_if_c< charset_traits::is_ascii_superset + && charset_traits::is_ascii_superset >::type + > +{ + typedef const_character_iterator::unit_t*> InputIterator; + void + operator() ( InputIterator first, const InputIterator& last, + OutputIterator result, // ?? modify in place, return new version, or what? + typename charset_traits::state_t& from_state, + typename charset_traits::state_t& to_state) + { +std::cout << "specialisation!\n"; + uintptr_t last_addr = reinterpret_cast(last.base()); + while (first != last) { + uintptr_t first_addr = reinterpret_cast(first.base()); + if (((first_addr & (sizeof(int)-1))==0) + && ((last_addr-first_addr) >= sizeof(int))) { + const unsigned* uptr = static_cast(static_cast(first.base())); + unsigned u = *uptr; + if (!(u&0x80808080)) { // FIXME 64 bits + *(result++) = byte_n<0>(u); // FIXME enable_if both cs ASCII supersets // FIXME result is ptr too + *(result++) = byte_n<1>(u); + *(result++) = byte_n<2>(u); + *(result++) = byte_n<3>(u); + first = InputIterator(first.base() + 4); + continue; + } + } + *(result++) = char_conv()(*first,from_state,to_state); + ++first; + } + } +}; + + +// Now specialise for std::basic_string and std::vector by calling the above +// pointer specialisation: + +template +struct sequence_conv< from_cs, to_cs, error_policy, + const_character_iterator::unit_t> + ::const_iterator>, + OutputIterator, + typename boost::enable_if_c< charset_traits::is_ascii_superset + && charset_traits::is_ascii_superset >::type + > +{ + typedef const_character_iterator::unit_t> + ::const_iterator> InputIterator; + typedef const_character_iterator::unit_t*> InputPointer; + void + operator() ( InputIterator first, const InputIterator& last, + OutputIterator result, // ?? modify in place, return new version, or what? + typename charset_traits::state_t& from_state, + typename charset_traits::state_t& to_state) + { + sequence_conv< from_cs,to_cs,error_policy, + InputPointer, OutputIterator > sc; + InputPointer first_p(&(*(first.base()))); + InputPointer last_p(&(*(last.base()))); + sc(first_p,last_p,result,from_state,to_state); + } +}; + + + + +}; + +#endif diff --git a/extras/anytermd/libpbe/include/charset/conv/ascii.hh b/extras/anytermd/libpbe/include/charset/conv/ascii.hh new file mode 100644 index 0000000000..74d2e4c2e6 --- /dev/null +++ b/extras/anytermd/libpbe/include/charset/conv/ascii.hh @@ -0,0 +1,77 @@ +// ascii.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2008 Philip Endecott + +// Distributed under the Boost Software License, Version 1.0: +// +// Permission is hereby granted, free of charge, to any person or organization +// obtaining a copy of the software and accompanying documentation covered by +// this license (the "Software") to use, reproduce, display, distribute, +// execute, and transmit the Software, and to prepare derivative works of the +// Software, and to permit third-parties to whom the Software is furnished to +// do so, all subject to the following: +// +// The copyright notices in the Software and this entire statement, including +// the above license grant, this restriction and the following disclaimer, +// must be included in all copies of the Software, in whole or in part, and +// all derivative works of the Software, unless such copies or derivative +// works are solely in the form of machine-executable object code generated by +// a source language processor. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT +// SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE +// FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, +// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER +// DEALINGS IN THE SOFTWARE. + +#ifndef libpbe_charset_conv_ascii_hh +#define libpbe_charset_conv_ascii_hh + +#include "charset/charset_t.hh" +#include "charset/char_conv.hh" + + +namespace pbe { + +// ASCII-to-Unicode character conversion +// -------------------------------------- +// +// This file provides specialisations of char_conv to convert between ASCII +// and ucs2 and ucs4. + + +IDENTITY_CHAR_CONV(cs::ascii,cs::ucs4) +IDENTITY_CHAR_CONV(cs::ascii,cs::ucs2) + +template +struct char_conv { + char8_t + operator() ( char16_t c, + charset_traits::state_t&, + charset_traits::state_t& ) { + if (c>0x7f) { + return error_policy::no_equivalent(); + } + return c; + } +}; + +template +struct char_conv { + char8_t + operator() ( char32_t c, + charset_traits::state_t&, + charset_traits::state_t& ) { + if (c>0x7f) { + return error_policy::no_equivalent(); + } + return c; + } +}; + + +}; + +#endif diff --git a/extras/anytermd/libpbe/include/charset/conv/iso8859.hh b/extras/anytermd/libpbe/include/charset/conv/iso8859.hh new file mode 100644 index 0000000000..6443d0132c --- /dev/null +++ b/extras/anytermd/libpbe/include/charset/conv/iso8859.hh @@ -0,0 +1,255 @@ +// iso8859.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2008 Philip Endecott + +// Distributed under the Boost Software License, Version 1.0: +// +// Permission is hereby granted, free of charge, to any person or organization +// obtaining a copy of the software and accompanying documentation covered by +// this license (the "Software") to use, reproduce, display, distribute, +// execute, and transmit the Software, and to prepare derivative works of the +// Software, and to permit third-parties to whom the Software is furnished to +// do so, all subject to the following: +// +// The copyright notices in the Software and this entire statement, including +// the above license grant, this restriction and the following disclaimer, +// must be included in all copies of the Software, in whole or in part, and +// all derivative works of the Software, unless such copies or derivative +// works are solely in the form of machine-executable object code generated by +// a source language processor. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT +// SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE +// FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, +// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER +// DEALINGS IN THE SOFTWARE. + +#ifndef libpbe_charset_conv_iso8859_hh +#define libpbe_charset_conv_iso8859_hh + +#include "charset/charset_t.hh" +#include "charset/char_t.hh" +#include "charset/charset_traits.hh" +#include "charset/char_conv.hh" + +#include + +#include + + +namespace pbe { + +// Conversion of iso8859 characters +// -------------------------------- +// +// This file implements character conversions to and from the iso8859 character +// sets. + + +// Conversions to Unicode characters +// --------------------------------- + +// All iso8859 characters correspond to "basic multilingual plane" unicode +// characters, i.e. they fit in a 16-bit ucs2 character. + +// iso8859-1 is a special case: the first 256 Unicode characters are +// the iso8859-1 characters. + +IDENTITY_CHAR_CONV(cs::iso8859_1,cs::ucs2) +IDENTITY_CHAR_CONV(cs::iso8859_1,cs::ucs4) + + +// For the other character sets, characters 0 to 160 inclusivve map directly to +// Unicode. Tables are used to map characters 161 to 255: + +typedef char16_t iso8859_table_t[95]; // character n in [n-161]. +extern iso8859_table_t iso8859_tables[15]; // iso8859-n in [n-2]. + +// These tables are automatically generated from data from unicode.org. + +// Some character sets don't use some codes. The sentinel value -1 is stored +// in the tables for these cases. (0xffff is not a valid ucs2 character.) + +// In the following, template parameter n is the character set number 2-16. + +template +static inline int iso8859_to_ucs ( char8_t c ) +{ + int i = static_cast(static_cast(c)); + return (i<161) ? i : iso8859_tables[n-2][i-161]; +} + +#define CONV_ISO8859_TO_UCS(N) \ +template \ +struct char_conv { \ + char16_t operator() ( char8_t c, \ + charset_traits::state_t&, \ + charset_traits::state_t& ) \ + { \ + return iso8859_to_ucs(c); \ + } \ +}; \ +template \ +struct char_conv { \ + char32_t operator() ( char8_t c, \ + charset_traits::state_t&, \ + charset_traits::state_t& ) \ + { \ + return iso8859_to_ucs(c); \ + } \ +}; + +CONV_ISO8859_TO_UCS(2) +CONV_ISO8859_TO_UCS(3) +CONV_ISO8859_TO_UCS(4) +CONV_ISO8859_TO_UCS(5) +CONV_ISO8859_TO_UCS(6) +CONV_ISO8859_TO_UCS(7) +CONV_ISO8859_TO_UCS(8) +CONV_ISO8859_TO_UCS(9) +CONV_ISO8859_TO_UCS(10) +//CONV_ISO8859_TO_UCS(11) // This is missing from the IANA file + // -11 should be Thai. + // -12 is supposed to be missing; it's the abandoned Devanagari +CONV_ISO8859_TO_UCS(13) +CONV_ISO8859_TO_UCS(14) +CONV_ISO8859_TO_UCS(15) +CONV_ISO8859_TO_UCS(16) + +#undef CONV_ISO8859_TO_UCS + + +// Conversion from Unicode characters +// ---------------------------------- + +// iso8859-1 is a special case again: + +template +struct char_conv { + char8_t operator() ( char16_t c, + charset_traits::state_t&, + charset_traits::state_t& ) + { + if (c>0xff) { + error_policy::no_equivalent(); + } + return c; + } +}; + +template +struct char_conv { + char8_t operator() ( char32_t c, + charset_traits::state_t&, + charset_traits::state_t& ) + { + if (c>0xff) { + error_policy::no_equivalent(); + } + return c; + } +}; + + +// For the other cases we use tables. +// A single-level table-driven conversion would require large, sparse +// tables; instead we break the unicode space into pages and have one table +// for each combination of ucs page and iso8859 character set. +// These tables are generated dynamically only as needed by invoking +// the reverse functions above. + + +template +static inline const char8_t* mk_ucs_to_iso8859_page_table() { + char8_t* table = new char8_t[256]; // never deleted + std::fill(table,table+256,0); // 0 = no equivalent + for (int c=161; c<256; ++c) { + int unichar = iso8859_to_ucs(c); + if ((unichar>>8) == page) { + table[unichar&0xff] = c; + } + } + return table; +} + +template +static inline char8_t char_conv_ucs_to_iso8859_lookup ( uint8_t point ) +{ + /*FIXME THREAD SAFE*/ static const char8_t* table_p = mk_ucs_to_iso8859_page_table(); + char8_t c = table_p[point]; + if (c==0) { + return error_policy::no_equivalent(); + } + return c; +} + +template +static inline char8_t ucs_to_iso8859 ( int c ) +{ + if (c<=160) { + return c; + } else { + int page = c>>8; + uint8_t point = c&0xff; + switch (page) { + // These are the only pages that have any characters in any iso8859 character sets. + // FIXME we could use specialisation to consider only those pages that apply to + // a partiuclar character set. + case 0x00: return char_conv_ucs_to_iso8859_lookup(point); + case 0x01: return char_conv_ucs_to_iso8859_lookup(point); + case 0x02: return char_conv_ucs_to_iso8859_lookup(point); + case 0x03: return char_conv_ucs_to_iso8859_lookup(point); + case 0x04: return char_conv_ucs_to_iso8859_lookup(point); + case 0x05: return char_conv_ucs_to_iso8859_lookup(point); + case 0x06: return char_conv_ucs_to_iso8859_lookup(point); + case 0x0e: return char_conv_ucs_to_iso8859_lookup(point); + case 0x1e: return char_conv_ucs_to_iso8859_lookup(point); + case 0x20: return char_conv_ucs_to_iso8859_lookup(point); + case 0x21: return char_conv_ucs_to_iso8859_lookup(point); + default: return error_policy::no_equivalent(); + } + } +} + + +#define CONV_UCS_TO_ISO8859(N) \ +template \ +struct char_conv { \ + char8_t operator() ( char16_t c, \ + charset_traits::state_t&, \ + charset_traits::state_t& ) \ + { \ + return ucs_to_iso8859(c); \ + } \ +}; \ +template \ +struct char_conv { \ + char8_t operator() ( char32_t c, \ + charset_traits::state_t&, \ + charset_traits::state_t& ) \ + { \ + return ucs_to_iso8859(c); \ + } \ +}; + +CONV_UCS_TO_ISO8859(2) +CONV_UCS_TO_ISO8859(3) +CONV_UCS_TO_ISO8859(4) +CONV_UCS_TO_ISO8859(5) +CONV_UCS_TO_ISO8859(6) +CONV_UCS_TO_ISO8859(7) +CONV_UCS_TO_ISO8859(8) +CONV_UCS_TO_ISO8859(9) +CONV_UCS_TO_ISO8859(10) +CONV_UCS_TO_ISO8859(13) +CONV_UCS_TO_ISO8859(14) +CONV_UCS_TO_ISO8859(15) +CONV_UCS_TO_ISO8859(16) + +#undef CONV_UCS_TO_ISO8859 + +}; + +#endif diff --git a/extras/anytermd/libpbe/include/charset/conv/unicode.hh b/extras/anytermd/libpbe/include/charset/conv/unicode.hh new file mode 100644 index 0000000000..f5c0fbd37c --- /dev/null +++ b/extras/anytermd/libpbe/include/charset/conv/unicode.hh @@ -0,0 +1,88 @@ +// unicode.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2008 Philip Endecott + +// Distributed under the Boost Software License, Version 1.0: +// +// Permission is hereby granted, free of charge, to any person or organization +// obtaining a copy of the software and accompanying documentation covered by +// this license (the "Software") to use, reproduce, display, distribute, +// execute, and transmit the Software, and to prepare derivative works of the +// Software, and to permit third-parties to whom the Software is furnished to +// do so, all subject to the following: +// +// The copyright notices in the Software and this entire statement, including +// the above license grant, this restriction and the following disclaimer, +// must be included in all copies of the Software, in whole or in part, and +// all derivative works of the Software, unless such copies or derivative +// works are solely in the form of machine-executable object code generated by +// a source language processor. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT +// SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE +// FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, +// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER +// DEALINGS IN THE SOFTWARE. + +#ifndef libpbe_charset_conv_unicode_hh +#define libpbe_charset_conv_unicode_hh + +#include "charset/charset_t.hh" +#include "charset/char_conv.hh" +#include "charset/char_t.hh" + + +namespace pbe { + +// Unicode character conversion +// ---------------------------- +// +// This file provides specialisation of char_conv for the Unicode +// character types. +// Note that UTF-8 and UTF-16 characters are 32-bit characters here, +// not groups of units. +// Everything is trivial except conversion to ucs2. + +// UTF16 has not been implemented yet. + + +IDENTITY_CHAR_CONV(cs::ucs2,cs::ucs4) +IDENTITY_CHAR_CONV(cs::ucs2,cs::utf8) +//IDENTITY_CHAR_CONV(cs::ucs2,cs::utf16) + +IDENTITY_CHAR_CONV(cs::ucs4,cs::utf8) +//IDENTITY_CHAR_CONV(cs::ucs4,cs::utf16) + +IDENTITY_CHAR_CONV(cs::utf8,cs::ucs4) +//IDENTITY_CHAR_CONV(cs::utf8,cs::utf16) + +//IDENTITY_CHAR_CONV(cs::utf16,cs::ucs4) +//IDENTITY_CHAR_CONV(cs::utf16,cs::utf8) + + +#define CONV_TO_UCS2(FROM_CS) \ +template \ +struct char_conv { \ + char16_t \ + operator() ( charset_traits::char_t c, \ + charset_traits::state_t&, \ + charset_traits::state_t& ) { \ + if (c>0xffff) { \ + return error_policy::no_equivalent(); \ + } \ + return c; \ + } \ +}; + +CONV_TO_UCS2(cs::ucs4) +CONV_TO_UCS2(cs::utf8) +//CONV_TO_UCS2(cs::utf16) + +#undef CONV_TO_UCS2 + + +}; + +#endif diff --git a/extras/anytermd/libpbe/include/charset/cs_string.hh b/extras/anytermd/libpbe/include/charset/cs_string.hh new file mode 100644 index 0000000000..483b70f984 --- /dev/null +++ b/extras/anytermd/libpbe/include/charset/cs_string.hh @@ -0,0 +1,114 @@ +// cs_string.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2007-2008 Philip Endecott + +// Distributed under the Boost Software License, Version 1.0: +// +// Permission is hereby granted, free of charge, to any person or organization +// obtaining a copy of the software and accompanying documentation covered by +// this license (the "Software") to use, reproduce, display, distribute, +// execute, and transmit the Software, and to prepare derivative works of the +// Software, and to permit third-parties to whom the Software is furnished to +// do so, all subject to the following: +// +// The copyright notices in the Software and this entire statement, including +// the above license grant, this restriction and the following disclaimer, +// must be included in all copies of the Software, in whole or in part, and +// all derivative works of the Software, unless such copies or derivative +// works are solely in the form of machine-executable object code generated by +// a source language processor. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT +// SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE +// FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, +// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER +// DEALINGS IN THE SOFTWARE. + +#ifndef libpbe_charset_cs_string_hh +#define libpbe_charset_cs_string_hh + +#include "charset/charset_t.hh" +#include "charset/string_adaptor.hh" + +#include +#include + + +namespace pbe { + +// String tagged with character set +// -------------------------------- +// +// This class provides a string tagged with its character set. It is +// implemented using a std::basic_string of the character set's unit_t +// and a string_adaptor. + + +// This base class is used so that ustr is constructed before it is +// passed by reference to string_adaptor. +template +struct cs_string_base { + unit_string_t ustr_; + cs_string_base() {} + cs_string_base(unit_string_t u): ustr_(u) {} + cs_string_base(const typename unit_string_t::pointer u): ustr_(u) {} +}; + + +template< charset_t cset, + typename Alloc = std::allocator< typename charset_traits::unit_t > + > +class cs_string: + private cs_string_base< typename string_adaptor::unit_string_t >, + public string_adaptor +{ + typedef string_adaptor adaptor; + typedef cs_string_base< typename adaptor::unit_string_t > base; + typedef typename charset_traits::unit_t unit_t; + +public: + cs_string(): adaptor(base::ustr_) {} + cs_string(const unit_t* s): base(s), adaptor(base::ustr_) {} // ??? + cs_string(const cs_string& other): base(other.unit_str()), adaptor(base::ustr_) {} + cs_string(typename adaptor::size_type n, typename adaptor::value_type c): + adaptor(base::ustr_) { + append(n,c); + } + template + cs_string(InputIterator first, InputIterator last): + adaptor(base::ustr_) { + append(first,last); + } +}; + + +template +cs_string operator+(const cs_string& s1, + const cs_string& s2) { + cs_string s = s1; + s.append(s2); + return s; +} + +template +cs_string operator+(typename charset_traits::char_t c, + const cs_string& s2) { + cs_string s = c; + s.append(s2); + return s; +} + +template +cs_string operator+(const cs_string& s1, + typename charset_traits::char_t c) { + cs_string s = s1; + s.append(c); + return s; +} + + +}; + +#endif diff --git a/extras/anytermd/libpbe/include/charset/error_policy.hh b/extras/anytermd/libpbe/include/charset/error_policy.hh new file mode 100644 index 0000000000..562b519567 --- /dev/null +++ b/extras/anytermd/libpbe/include/charset/error_policy.hh @@ -0,0 +1,93 @@ +// error_policy.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2008 Philip Endecott + +// Distributed under the Boost Software License, Version 1.0: +// +// Permission is hereby granted, free of charge, to any person or organization +// obtaining a copy of the software and accompanying documentation covered by +// this license (the "Software") to use, reproduce, display, distribute, +// execute, and transmit the Software, and to prepare derivative works of the +// Software, and to permit third-parties to whom the Software is furnished to +// do so, all subject to the following: +// +// The copyright notices in the Software and this entire statement, including +// the above license grant, this restriction and the following disclaimer, +// must be included in all copies of the Software, in whole or in part, and +// all derivative works of the Software, unless such copies or derivative +// works are solely in the form of machine-executable object code generated by +// a source language processor. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT +// SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE +// FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, +// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER +// DEALINGS IN THE SOFTWARE. + +#ifndef libpbe_charset_error_policy_hh +#define libpbe_charset_error_policy_hh + + +namespace pbe { + +// Error Policies +// -------------- +// +// The function template char_conv takes a template parameter that specifies +// its behaviour on error. This file supplies implementations of the +// error_policy concept, though the user is free to supply +// their own. +// +// An error_policy class should provide the following static +// member functions, corresponding to the different errors that char_conv +// may detect: +// +// static int no_equivalent() +// Called when the output character set does not have an equivalent +// character. +// +// static int invalid_input() +// Called when the input is not valid for the input character set. +// +// TODO - perhaps something for approximate equivalents? +// +// If the function returns, the value that it returns is used as the return +// value of char_conv. + +// In cases where these error policies throw an exception, by default it +// is the following: +struct conversion_error {}; + +// Throw an exception on error: +template +struct error_policy_throw { + static int no_equivalent() { + throw exception(); + } + static int invalid_input() { + throw exception(); + } +}; + +// Return a sentinel value on error: +template +struct error_policy_return_sentinel { + static int no_equivalent() { + return sentinel; + } + static int invalid_input() { + return sentinel; + } +}; + + +// The default error policy is error_policy_throw: + +typedef error_policy_throw<> default_error_policy; + + +}; + +#endif diff --git a/extras/anytermd/libpbe/include/charset/generated/charset_t_enum_body.hh b/extras/anytermd/libpbe/include/charset/generated/charset_t_enum_body.hh new file mode 100644 index 0000000000..8191f02462 --- /dev/null +++ b/extras/anytermd/libpbe/include/charset/generated/charset_t_enum_body.hh @@ -0,0 +1,1949 @@ +enum charset_t { + ansi_x3_4_1968, + iso646_us = ansi_x3_4_1968, + iso_646irv_1991 = ansi_x3_4_1968, + iso_ir_6 = ansi_x3_4_1968, + cp367 = ansi_x3_4_1968, + ansi_x3_4_1986 = ansi_x3_4_1968, + usascii = ansi_x3_4_1968, + iso646irv1991 = ansi_x3_4_1968, + iso_646_irv_1991 = ansi_x3_4_1968, + isoir6 = ansi_x3_4_1968, + csascii = ansi_x3_4_1968, + iso646irv_1991 = ansi_x3_4_1968, + ansix3_4_1986 = ansi_x3_4_1968, + ansix3_4_1968 = ansi_x3_4_1968, + us_ascii = ansi_x3_4_1968, + iso_ir6 = ansi_x3_4_1968, + ibm367 = ansi_x3_4_1968, + iso646us = ansi_x3_4_1968, + iso646_irv1991 = ansi_x3_4_1968, + iso646_irv_1991 = ansi_x3_4_1968, + isoir_6 = ansi_x3_4_1968, + us = ansi_x3_4_1968, + ascii = ansi_x3_4_1968, + iso_646irv1991 = ansi_x3_4_1968, + iso_646_irv1991 = ansi_x3_4_1968, + iso_8859_1_1987, + iso_ir_100 = iso_8859_1_1987, + l1 = iso_8859_1_1987, + isoir100 = iso_8859_1_1987, + iso8859_1_1987 = iso_8859_1_1987, + iso8859_1 = iso_8859_1_1987, + csisolatin1 = iso_8859_1_1987, + ibm819 = iso_8859_1_1987, + isoir_100 = iso_8859_1_1987, + cp819 = iso_8859_1_1987, + iso_8859_1 = iso_8859_1_1987, + latin1 = iso_8859_1_1987, + iso_ir100 = iso_8859_1_1987, + iso_8859_2_1987, + iso_ir_101 = iso_8859_2_1987, + iso8859_2_1987 = iso_8859_2_1987, + l2 = iso_8859_2_1987, + isoir101 = iso_8859_2_1987, + iso8859_2 = iso_8859_2_1987, + csisolatin2 = iso_8859_2_1987, + isoir_101 = iso_8859_2_1987, + iso_8859_2 = iso_8859_2_1987, + latin2 = iso_8859_2_1987, + iso_ir101 = iso_8859_2_1987, + iso_8859_3_1988, + latin3 = iso_8859_3_1988, + iso8859_3_1988 = iso_8859_3_1988, + l3 = iso_8859_3_1988, + isoir_109 = iso_8859_3_1988, + iso8859_3 = iso_8859_3_1988, + csisolatin3 = iso_8859_3_1988, + iso_ir109 = iso_8859_3_1988, + iso_ir_109 = iso_8859_3_1988, + isoir109 = iso_8859_3_1988, + iso_8859_3 = iso_8859_3_1988, + iso_8859_4_1988, + iso_8859_4 = iso_8859_4_1988, + latin4 = iso_8859_4_1988, + iso_ir_110 = iso_8859_4_1988, + isoir110 = iso_8859_4_1988, + l4 = iso_8859_4_1988, + iso8859_4 = iso_8859_4_1988, + csisolatin4 = iso_8859_4_1988, + isoir_110 = iso_8859_4_1988, + iso8859_4_1988 = iso_8859_4_1988, + iso_ir110 = iso_8859_4_1988, + iso_8859_5_1988, + csisolatincyrillic = iso_8859_5_1988, + iso8859_5_1988 = iso_8859_5_1988, + iso_8859_5 = iso_8859_5_1988, + isoir_144 = iso_8859_5_1988, + iso_ir144 = iso_8859_5_1988, + iso8859_5 = iso_8859_5_1988, + iso_ir_144 = iso_8859_5_1988, + cyrillic = iso_8859_5_1988, + isoir144 = iso_8859_5_1988, + iso_8859_6_1987, + asmo708 = iso_8859_6_1987, + ecma_114 = iso_8859_6_1987, + iso_8859_6 = iso_8859_6_1987, + arabic = iso_8859_6_1987, + isoir_127 = iso_8859_6_1987, + asmo_708 = iso_8859_6_1987, + iso_ir127 = iso_8859_6_1987, + iso8859_6 = iso_8859_6_1987, + iso_ir_127 = iso_8859_6_1987, + iso8859_6_1987 = iso_8859_6_1987, + csisolatinarabic = iso_8859_6_1987, + ecma114 = iso_8859_6_1987, + isoir127 = iso_8859_6_1987, + iso_8859_7_1987, + ecma118 = iso_8859_7_1987, + isoir_126 = iso_8859_7_1987, + greek8 = iso_8859_7_1987, + iso_8859_7 = iso_8859_7_1987, + elot928 = iso_8859_7_1987, + ecma_118 = iso_8859_7_1987, + iso_ir126 = iso_8859_7_1987, + csisolatingreek = iso_8859_7_1987, + iso_ir_126 = iso_8859_7_1987, + iso8859_7_1987 = iso_8859_7_1987, + greek = iso_8859_7_1987, + elot_928 = iso_8859_7_1987, + iso8859_7 = iso_8859_7_1987, + isoir126 = iso_8859_7_1987, + iso_8859_8_1988, + isoir138 = iso_8859_8_1988, + iso_8859_8 = iso_8859_8_1988, + csisolatinhebrew = iso_8859_8_1988, + isoir_138 = iso_8859_8_1988, + iso8859_8_1988 = iso_8859_8_1988, + iso_ir138 = iso_8859_8_1988, + hebrew = iso_8859_8_1988, + iso8859_8 = iso_8859_8_1988, + iso_ir_138 = iso_8859_8_1988, + iso_8859_9_1989, + isoir148 = iso_8859_9_1989, + latin5 = iso_8859_9_1989, + l5 = iso_8859_9_1989, + iso_8859_9 = iso_8859_9_1989, + isoir_148 = iso_8859_9_1989, + iso8859_9_1989 = iso_8859_9_1989, + csisolatin5 = iso_8859_9_1989, + iso_ir148 = iso_8859_9_1989, + iso8859_9 = iso_8859_9_1989, + iso_ir_148 = iso_8859_9_1989, + iso_8859_10, + isoir157 = iso_8859_10, + latin6 = iso_8859_10, + l6 = iso_8859_10, + isoir_157 = iso_8859_10, + iso_8859_10_1992 = iso_8859_10, + csisolatin6 = iso_8859_10, + iso_ir157 = iso_8859_10, + iso8859_10 = iso_8859_10, + iso8859_10_1992 = iso_8859_10, + iso_ir_157 = iso_8859_10, + iso_6937_2_add, + csisotextcomm = iso_6937_2_add, + isoir_142 = iso_6937_2_add, + iso_6937_2add = iso_6937_2_add, + iso_ir142 = iso_6937_2_add, + iso_ir_142 = iso_6937_2_add, + isoir142 = iso_6937_2_add, + iso6937_2add = iso_6937_2_add, + iso6937_2_add = iso_6937_2_add, + jis_x0201, + cshalfwidthkatakana = jis_x0201, + jisx0201 = jis_x0201, + x0201 = jis_x0201, + jis_encoding, + csjisencoding = jis_encoding, + jisencoding = jis_encoding, + shift_jis, + csshiftjis = shift_jis, + mskanji = shift_jis, + ms_kanji = shift_jis, + shiftjis = shift_jis, + extended_unix_code_packed_format_for_japanese, + extended_unixcode_packed_format_for_japanese= extended_unix_code_packed_format_for_japanese, + extendedunix_code_packed_formatfor_japanese= extended_unix_code_packed_format_for_japanese, + extended_unixcodepackedformat_for_japanese= extended_unix_code_packed_format_for_japanese, + extendedunixcode_packed_formatforjapanese= extended_unix_code_packed_format_for_japanese, + extendedunix_code_packedformatforjapanese= extended_unix_code_packed_format_for_japanese, + extended_unixcodepackedformat_forjapanese= extended_unix_code_packed_format_for_japanese, + extendedunix_codepacked_format_forjapanese= extended_unix_code_packed_format_for_japanese, + extendedunixcodepacked_formatforjapanese= extended_unix_code_packed_format_for_japanese, + extendedunix_code_packed_formatforjapanese= extended_unix_code_packed_format_for_japanese, + extendedunixcode_packed_format_forjapanese= extended_unix_code_packed_format_for_japanese, + extended_unixcodepacked_format_forjapanese= extended_unix_code_packed_format_for_japanese, + extended_unix_codepacked_formatforjapanese= extended_unix_code_packed_format_for_japanese, + extended_unixcode_packed_formatforjapanese= extended_unix_code_packed_format_for_japanese, + euc_jp = extended_unix_code_packed_format_for_japanese, + extended_unix_codepacked_format_forjapanese= extended_unix_code_packed_format_for_japanese, + extended_unix_code_packed_formatforjapanese= extended_unix_code_packed_format_for_japanese, + extended_unixcode_packed_format_forjapanese= extended_unix_code_packed_format_for_japanese, + extendedunix_codepacked_format_for_japanese= extended_unix_code_packed_format_for_japanese, + extendedunixcode_packedformat_forjapanese= extended_unix_code_packed_format_for_japanese, + extendedunix_codepackedformat_forjapanese= extended_unix_code_packed_format_for_japanese, + extendedunixcode_packedformatforjapanese= extended_unix_code_packed_format_for_japanese, + extendedunix_code_packed_format_for_japanese= extended_unix_code_packed_format_for_japanese, + extended_unix_code_packedformat_for_japanese= extended_unix_code_packed_format_for_japanese, + extendedunix_codepackedformatforjapanese= extended_unix_code_packed_format_for_japanese, + extended_unix_code_packedformatforjapanese= extended_unix_code_packed_format_for_japanese, + extendedunixcodepacked_formatfor_japanese= extended_unix_code_packed_format_for_japanese, + extended_unix_code_packedformat_forjapanese= extended_unix_code_packed_format_for_japanese, + extendedunixcode_packed_format_for_japanese= extended_unix_code_packed_format_for_japanese, + cseucpkdfmtjapanese = extended_unix_code_packed_format_for_japanese, + extended_unixcodepackedformatfor_japanese= extended_unix_code_packed_format_for_japanese, + extended_unix_code_packed_format_forjapanese= extended_unix_code_packed_format_for_japanese, + extendedunixcodepackedformatforjapanese= extended_unix_code_packed_format_for_japanese, + extendedunixcodepackedformat_forjapanese= extended_unix_code_packed_format_for_japanese, + extendedunix_code_packedformat_forjapanese= extended_unix_code_packed_format_for_japanese, + extended_unix_codepacked_formatfor_japanese= extended_unix_code_packed_format_for_japanese, + extended_unix_codepackedformat_forjapanese= extended_unix_code_packed_format_for_japanese, + extended_unixcode_packedformat_forjapanese= extended_unix_code_packed_format_for_japanese, + eucjp = extended_unix_code_packed_format_for_japanese, + extendedunix_codepacked_formatfor_japanese= extended_unix_code_packed_format_for_japanese, + extendedunix_code_packedformat_for_japanese= extended_unix_code_packed_format_for_japanese, + extendedunixcode_packed_formatfor_japanese= extended_unix_code_packed_format_for_japanese, + extended_unixcodepacked_formatfor_japanese= extended_unix_code_packed_format_for_japanese, + extended_unixcode_packed_formatfor_japanese= extended_unix_code_packed_format_for_japanese, + extendedunixcode_packedformatfor_japanese= extended_unix_code_packed_format_for_japanese, + extendedunix_codepackedformatfor_japanese= extended_unix_code_packed_format_for_japanese, + extendedunixcodepacked_format_for_japanese= extended_unix_code_packed_format_for_japanese, + extendedunixcodepackedformatfor_japanese= extended_unix_code_packed_format_for_japanese, + extendedunix_code_packedformatfor_japanese= extended_unix_code_packed_format_for_japanese, + extended_unixcodepacked_format_for_japanese= extended_unix_code_packed_format_for_japanese, + extended_unix_codepackedformatfor_japanese= extended_unix_code_packed_format_for_japanese, + extendedunix_code_packed_format_forjapanese= extended_unix_code_packed_format_for_japanese, + extended_unix_code_packedformatfor_japanese= extended_unix_code_packed_format_for_japanese, + extended_unixcodepacked_formatforjapanese= extended_unix_code_packed_format_for_japanese, + extended_unix_code_packed_formatfor_japanese= extended_unix_code_packed_format_for_japanese, + extended_unixcode_packedformatforjapanese= extended_unix_code_packed_format_for_japanese, + extendedunixcodepackedformat_for_japanese= extended_unix_code_packed_format_for_japanese, + extended_unixcode_packedformatfor_japanese= extended_unix_code_packed_format_for_japanese, + extendedunix_codepackedformat_for_japanese= extended_unix_code_packed_format_for_japanese, + extendedunixcode_packedformat_for_japanese= extended_unix_code_packed_format_for_japanese, + extendedunixcodepacked_format_forjapanese= extended_unix_code_packed_format_for_japanese, + extendedunix_codepacked_formatforjapanese= extended_unix_code_packed_format_for_japanese, + extended_unix_codepackedformat_for_japanese= extended_unix_code_packed_format_for_japanese, + extended_unix_codepacked_format_for_japanese= extended_unix_code_packed_format_for_japanese, + extended_unixcode_packedformat_for_japanese= extended_unix_code_packed_format_for_japanese, + extended_unix_codepackedformatforjapanese= extended_unix_code_packed_format_for_japanese, + extended_unixcodepackedformatforjapanese= extended_unix_code_packed_format_for_japanese, + extended_unix_code_fixed_width_for_japanese, + extended_unixcode_fixedwidth_for_japanese= extended_unix_code_fixed_width_for_japanese, + extendedunixcodefixed_widthfor_japanese= extended_unix_code_fixed_width_for_japanese, + extendedunixcode_fixedwidthfor_japanese= extended_unix_code_fixed_width_for_japanese, + extendedunix_codefixed_widthfor_japanese= extended_unix_code_fixed_width_for_japanese, + extended_unixcodefixed_widthforjapanese= extended_unix_code_fixed_width_for_japanese, + extendedunix_codefixed_widthforjapanese= extended_unix_code_fixed_width_for_japanese, + extended_unixcodefixedwidth_forjapanese= extended_unix_code_fixed_width_for_japanese, + extended_unix_code_fixed_widthforjapanese= extended_unix_code_fixed_width_for_japanese, + extended_unix_codefixedwidth_for_japanese= extended_unix_code_fixed_width_for_japanese, + extended_unixcode_fixed_width_forjapanese= extended_unix_code_fixed_width_for_japanese, + extended_unixcode_fixed_widthforjapanese= extended_unix_code_fixed_width_for_japanese, + extended_unixcodefixedwidth_for_japanese= extended_unix_code_fixed_width_for_japanese, + extended_unixcodefixed_width_for_japanese= extended_unix_code_fixed_width_for_japanese, + extended_unix_code_fixedwidth_forjapanese= extended_unix_code_fixed_width_for_japanese, + extended_unixcode_fixedwidth_forjapanese= extended_unix_code_fixed_width_for_japanese, + extendedunix_codefixedwidthforjapanese= extended_unix_code_fixed_width_for_japanese, + extendedunixcodefixed_widthforjapanese= extended_unix_code_fixed_width_for_japanese, + extendedunix_code_fixedwidth_for_japanese= extended_unix_code_fixed_width_for_japanese, + extended_unix_codefixedwidthfor_japanese= extended_unix_code_fixed_width_for_japanese, + extendedunix_codefixedwidth_forjapanese= extended_unix_code_fixed_width_for_japanese, + extended_unix_codefixed_width_forjapanese= extended_unix_code_fixed_width_for_japanese, + extendedunixcode_fixedwidth_for_japanese= extended_unix_code_fixed_width_for_japanese, + extended_unixcodefixed_width_forjapanese= extended_unix_code_fixed_width_for_japanese, + extended_unix_code_fixedwidthforjapanese= extended_unix_code_fixed_width_for_japanese, + extended_unix_code_fixedwidth_for_japanese= extended_unix_code_fixed_width_for_japanese, + extendedunix_code_fixed_width_forjapanese= extended_unix_code_fixed_width_for_japanese, + extendedunixcodefixedwidth_forjapanese= extended_unix_code_fixed_width_for_japanese, + extendedunixcode_fixed_widthforjapanese= extended_unix_code_fixed_width_for_japanese, + extendedunixcode_fixed_width_for_japanese= extended_unix_code_fixed_width_for_japanese, + extendedunix_codefixed_width_for_japanese= extended_unix_code_fixed_width_for_japanese, + extendedunixcodefixedwidth_for_japanese= extended_unix_code_fixed_width_for_japanese, + extendedunix_code_fixed_widthforjapanese= extended_unix_code_fixed_width_for_japanese, + extendedunix_codefixedwidth_for_japanese= extended_unix_code_fixed_width_for_japanese, + extendedunix_code_fixed_width_for_japanese= extended_unix_code_fixed_width_for_japanese, + extended_unixcodefixedwidthfor_japanese= extended_unix_code_fixed_width_for_japanese, + extended_unixcode_fixed_widthfor_japanese= extended_unix_code_fixed_width_for_japanese, + extended_unixcode_fixedwidthforjapanese= extended_unix_code_fixed_width_for_japanese, + extendedunix_codefixedwidthfor_japanese= extended_unix_code_fixed_width_for_japanese, + extended_unix_code_fixedwidthfor_japanese= extended_unix_code_fixed_width_for_japanese, + extendedunixcode_fixed_width_forjapanese= extended_unix_code_fixed_width_for_japanese, + extended_unixcodefixedwidthforjapanese= extended_unix_code_fixed_width_for_japanese, + extended_unixcode_fixedwidthfor_japanese= extended_unix_code_fixed_width_for_japanese, + cseucfixwidjapanese = extended_unix_code_fixed_width_for_japanese, + extendedunixcode_fixedwidth_forjapanese= extended_unix_code_fixed_width_for_japanese, + extendedunixcodefixed_width_for_japanese= extended_unix_code_fixed_width_for_japanese, + extendedunix_codefixed_width_forjapanese= extended_unix_code_fixed_width_for_japanese, + extended_unix_code_fixed_width_forjapanese= extended_unix_code_fixed_width_for_japanese, + extendedunix_code_fixedwidth_forjapanese= extended_unix_code_fixed_width_for_japanese, + extended_unix_codefixed_width_for_japanese= extended_unix_code_fixed_width_for_japanese, + extended_unixcode_fixed_width_for_japanese= extended_unix_code_fixed_width_for_japanese, + extended_unix_codefixed_widthforjapanese= extended_unix_code_fixed_width_for_japanese, + extendedunixcodefixedwidthforjapanese= extended_unix_code_fixed_width_for_japanese, + extended_unix_codefixed_widthfor_japanese= extended_unix_code_fixed_width_for_japanese, + extended_unixcodefixed_widthfor_japanese= extended_unix_code_fixed_width_for_japanese, + extendedunixcodefixed_width_forjapanese= extended_unix_code_fixed_width_for_japanese, + extendedunix_code_fixedwidthforjapanese= extended_unix_code_fixed_width_for_japanese, + extendedunix_code_fixed_widthfor_japanese= extended_unix_code_fixed_width_for_japanese, + extendedunixcodefixedwidthfor_japanese= extended_unix_code_fixed_width_for_japanese, + extended_unix_codefixedwidth_forjapanese= extended_unix_code_fixed_width_for_japanese, + extendedunixcode_fixed_widthfor_japanese= extended_unix_code_fixed_width_for_japanese, + extendedunixcode_fixedwidthforjapanese= extended_unix_code_fixed_width_for_japanese, + extended_unix_codefixedwidthforjapanese= extended_unix_code_fixed_width_for_japanese, + extended_unix_code_fixed_widthfor_japanese= extended_unix_code_fixed_width_for_japanese, + extendedunix_code_fixedwidthfor_japanese= extended_unix_code_fixed_width_for_japanese, + bs_4730, + uk = bs_4730, + isoir4 = bs_4730, + iso646gb = bs_4730, + gb = bs_4730, + iso_ir4 = bs_4730, + bs4730 = bs_4730, + iso646_gb = bs_4730, + isoir_4 = bs_4730, + iso_ir_4 = bs_4730, + csiso4unitedkingdom = bs_4730, + sen_850200_c, + isoir_11 = sen_850200_c, + iso_ir11 = sen_850200_c, + sen850200c = sen_850200_c, + iso646se2 = sen_850200_c, + iso646_se2 = sen_850200_c, + sen_850200c = sen_850200_c, + csiso11swedishfornames = sen_850200_c, + iso_ir_11 = sen_850200_c, + sen850200_c = sen_850200_c, + se2 = sen_850200_c, + isoir11 = sen_850200_c, + it, + iso_ir_15 = it, + iso646_it = it, + isoir15 = it, + csiso15italian = it, + isoir_15 = it, + iso_ir15 = it, + iso646it = it, + es, + iso_ir_17 = es, + isoir17 = es, + isoir_17 = es, + iso646es = es, + iso_ir17 = es, + csiso17spanish = es, + iso646_es = es, + din_66003, + isoir_21 = din_66003, + iso_ir21 = din_66003, + csiso21german = din_66003, + iso646de = din_66003, + din66003 = din_66003, + de = din_66003, + iso646_de = din_66003, + iso_ir_21 = din_66003, + isoir21 = din_66003, + ns_4551_1, + no = ns_4551_1, + iso_ir_60 = ns_4551_1, + csiso60norwegian1 = ns_4551_1, + iso646_no = ns_4551_1, + isoir60 = ns_4551_1, + csiso60danishnorwegian = ns_4551_1, + isoir_60 = ns_4551_1, + iso_ir60 = ns_4551_1, + ns4551_1 = ns_4551_1, + iso646no = ns_4551_1, + nf_z_62_010, + isoir_69 = nf_z_62_010, + nfz62_010 = nf_z_62_010, + iso_ir69 = nf_z_62_010, + csiso69french = nf_z_62_010, + iso646fr = nf_z_62_010, + fr = nf_z_62_010, + iso_ir_69 = nf_z_62_010, + iso646_fr = nf_z_62_010, + isoir69 = nf_z_62_010, + nf_z62_010 = nf_z_62_010, + nfz_62_010 = nf_z_62_010, + iso_10646_utf_1, + iso10646_utf1 = iso_10646_utf_1, + iso_10646utf_1 = iso_10646_utf_1, + iso_10646_utf1 = iso_10646_utf_1, + iso10646utf1 = iso_10646_utf_1, + iso10646_utf_1 = iso_10646_utf_1, + iso10646utf_1 = iso_10646_utf_1, + csiso10646utf1 = iso_10646_utf_1, + iso_10646utf1 = iso_10646_utf_1, + iso_646_basic_1983, + iso646_basic_1983 = iso_646_basic_1983, + iso_646basic1983 = iso_646_basic_1983, + iso646basic1983 = iso_646_basic_1983, + iso646_basic1983 = iso_646_basic_1983, + ref = iso_646_basic_1983, + iso646basic_1983 = iso_646_basic_1983, + csiso646basic1983 = iso_646_basic_1983, + iso_646basic_1983 = iso_646_basic_1983, + iso_646_basic1983 = iso_646_basic_1983, + invariant, + csinvariant = invariant, + iso_646_irv_1983, + iso_646_irv1983 = iso_646_irv_1983, + iso_646irv_1983 = iso_646_irv_1983, + isoir2 = iso_646_irv_1983, + irv = iso_646_irv_1983, + iso_ir2 = iso_646_irv_1983, + isoir_2 = iso_646_irv_1983, + iso646irv1983 = iso_646_irv_1983, + csiso2intlrefversion = iso_646_irv_1983, + iso646irv_1983 = iso_646_irv_1983, + iso_ir_2 = iso_646_irv_1983, + iso646_irv1983 = iso_646_irv_1983, + iso646_irv_1983 = iso_646_irv_1983, + iso_646irv1983 = iso_646_irv_1983, + nats_sefi, + natssefi = nats_sefi, + iso_ir8_1 = nats_sefi, + csnatssefi = nats_sefi, + iso_ir_8_1 = nats_sefi, + isoir8_1 = nats_sefi, + isoir_8_1 = nats_sefi, + nats_sefi_add, + isoir_8_2 = nats_sefi_add, + iso_ir8_2 = nats_sefi_add, + iso_ir_8_2 = nats_sefi_add, + natssefi_add = nats_sefi_add, + csnatssefiadd = nats_sefi_add, + isoir8_2 = nats_sefi_add, + natssefiadd = nats_sefi_add, + nats_sefiadd = nats_sefi_add, + nats_dano, + isoir_9_1 = nats_dano, + csnatsdano = nats_dano, + iso_ir9_1 = nats_dano, + iso_ir_9_1 = nats_dano, + natsdano = nats_dano, + isoir9_1 = nats_dano, + nats_dano_add, + isoir9_2 = nats_dano_add, + csnatsdanoadd = nats_dano_add, + natsdanoadd = nats_dano_add, + nats_danoadd = nats_dano_add, + isoir_9_2 = nats_dano_add, + iso_ir9_2 = nats_dano_add, + iso_ir_9_2 = nats_dano_add, + natsdano_add = nats_dano_add, + sen_850200_b, + iso_ir10 = sen_850200_b, + sen850200b = sen_850200_b, + iso646_fi = sen_850200_b, + sen_850200b = sen_850200_b, + iso646se = sen_850200_b, + se = sen_850200_b, + iso_ir_10 = sen_850200_b, + sen850200_b = sen_850200_b, + iso646_se = sen_850200_b, + isoir10 = sen_850200_b, + csiso10swedish = sen_850200_b, + iso646fi = sen_850200_b, + fi = sen_850200_b, + isoir_10 = sen_850200_b, + ks_c_5601_1987, + ksc_5601_1989 = ks_c_5601_1987, + iso_ir_149 = ks_c_5601_1987, + ksc_5601 = ks_c_5601_1987, + isoir149 = ks_c_5601_1987, + korean = ks_c_5601_1987, + ksc5601_1987 = ks_c_5601_1987, + ks_c_5601_1989 = ks_c_5601_1987, + ksc5601_1989 = ks_c_5601_1987, + ksc5601 = ks_c_5601_1987, + isoir_149 = ks_c_5601_1987, + ks_c5601_1987 = ks_c_5601_1987, + csksc56011987 = ks_c_5601_1987, + ks_c5601_1989 = ks_c_5601_1987, + iso_ir149 = ks_c_5601_1987, + ksc_5601_1987 = ks_c_5601_1987, + iso_2022_kr, + iso2022kr = iso_2022_kr, + iso2022_kr = iso_2022_kr, + iso_2022kr = iso_2022_kr, + csiso2022kr = iso_2022_kr, + euc_kr, + cseuckr = euc_kr, + euckr = euc_kr, + iso_2022_jp, + iso2022_jp = iso_2022_jp, + iso_2022jp = iso_2022_jp, + csiso2022jp = iso_2022_jp, + iso2022jp = iso_2022_jp, + iso_2022_jp_2, + iso_2022_jp2 = iso_2022_jp_2, + iso2022jp_2 = iso_2022_jp_2, + iso_2022jp_2 = iso_2022_jp_2, + iso2022_jp_2 = iso_2022_jp_2, + iso2022jp2 = iso_2022_jp_2, + csiso2022jp2 = iso_2022_jp_2, + iso_2022jp2 = iso_2022_jp_2, + iso2022_jp2 = iso_2022_jp_2, + jis_c6220_1969_jp, + isoir13 = jis_c6220_1969_jp, + isoir_13 = jis_c6220_1969_jp, + iso_ir13 = jis_c6220_1969_jp, + csiso13jisc6220jp = jis_c6220_1969_jp, + katakana = jis_c6220_1969_jp, + jisc6220_1969 = jis_c6220_1969_jp, + jis_c6220_1969jp = jis_c6220_1969_jp, + jis_c6220_1969 = jis_c6220_1969_jp, + x0201_7 = jis_c6220_1969_jp, + iso_ir_13 = jis_c6220_1969_jp, + jisc6220_1969_jp = jis_c6220_1969_jp, + jisc6220_1969jp = jis_c6220_1969_jp, + jis_c6220_1969_ro, + csiso14jisc6220ro = jis_c6220_1969_ro, + isoir14 = jis_c6220_1969_ro, + jis_c6220_1969ro = jis_c6220_1969_ro, + isoir_14 = jis_c6220_1969_ro, + iso_ir14 = jis_c6220_1969_ro, + jisc6220_1969_ro = jis_c6220_1969_ro, + jisc6220_1969ro = jis_c6220_1969_ro, + iso646jp = jis_c6220_1969_ro, + jp = jis_c6220_1969_ro, + iso646_jp = jis_c6220_1969_ro, + iso_ir_14 = jis_c6220_1969_ro, + pt, + iso_ir_16 = pt, + isoir16 = pt, + csiso16portuguese = pt, + isoir_16 = pt, + iso646pt = pt, + iso_ir16 = pt, + iso646_pt = pt, + greek7_old, + csiso18greek7old = greek7_old, + iso_ir_18 = greek7_old, + isoir18 = greek7_old, + isoir_18 = greek7_old, + iso_ir18 = greek7_old, + greek7old = greek7_old, + latin_greek, + iso_ir_19 = latin_greek, + latingreek = latin_greek, + isoir19 = latin_greek, + isoir_19 = latin_greek, + iso_ir19 = latin_greek, + csiso19latingreek = latin_greek, + nf_z_62_010__1973_, + nfz62_010__1973_ = nf_z_62_010__1973_, + iso_ir_25 = nf_z_62_010__1973_, + isoir25 = nf_z_62_010__1973_, + nf_z62_010__1973_ = nf_z_62_010__1973_, + iso646fr1 = nf_z_62_010__1973_, + iso646_fr1 = nf_z_62_010__1973_, + isoir_25 = nf_z_62_010__1973_, + nf_z_62_010_1973 = nf_z_62_010__1973_, + nf_z62_010_1973 = nf_z_62_010__1973_, + csiso25french = nf_z_62_010__1973_, + iso_ir25 = nf_z_62_010__1973_, + nf_z62_010_1973_ = nf_z_62_010__1973_, + nf_z_62_010__1973 = nf_z_62_010__1973_, + nf_z62_010__1973 = nf_z_62_010__1973_, + nfz_62_010_1973_ = nf_z_62_010__1973_, + nfz_62_010__1973 = nf_z_62_010__1973_, + nfz62_010__1973 = nf_z_62_010__1973_, + nfz62_010_1973 = nf_z_62_010__1973_, + nfz_62_010__1973_ = nf_z_62_010__1973_, + nf_z_62_010_1973_ = nf_z_62_010__1973_, + nfz_62_010_1973 = nf_z_62_010__1973_, + nfz62_010_1973_ = nf_z_62_010__1973_, + latin_greek_1, + iso_ir_27 = latin_greek_1, + isoir27 = latin_greek_1, + isoir_27 = latin_greek_1, + iso_ir27 = latin_greek_1, + latingreek_1 = latin_greek_1, + csiso27latingreek1 = latin_greek_1, + latin_greek1 = latin_greek_1, + latingreek1 = latin_greek_1, + iso_5427, + iso_ir_37 = iso_5427, + isoir37 = iso_5427, + iso5427 = iso_5427, + isoir_37 = iso_5427, + iso_ir37 = iso_5427, + csiso5427cyrillic = iso_5427, + jis_c6226_1978, + iso_ir_42 = jis_c6226_1978, + isoir42 = jis_c6226_1978, + isoir_42 = jis_c6226_1978, + iso_ir42 = jis_c6226_1978, + jisc6226_1978 = jis_c6226_1978, + csiso42jisc62261978 = jis_c6226_1978, + bs_viewdata, + bsviewdata = bs_viewdata, + iso_ir_47 = bs_viewdata, + isoir47 = bs_viewdata, + isoir_47 = bs_viewdata, + iso_ir47 = bs_viewdata, + csiso47bsviewdata = bs_viewdata, + inis, + iso_ir_49 = inis, + isoir49 = inis, + isoir_49 = inis, + csiso49inis = inis, + iso_ir49 = inis, + inis_8, + isoir50 = inis_8, + isoir_50 = inis_8, + iso_ir50 = inis_8, + csiso50inis8 = inis_8, + inis8 = inis_8, + iso_ir_50 = inis_8, + inis_cyrillic, + csiso51iniscyrillic = inis_cyrillic, + iso_ir_51 = inis_cyrillic, + isoir51 = inis_cyrillic, + isoir_51 = inis_cyrillic, + iso_ir51 = inis_cyrillic, + iniscyrillic = inis_cyrillic, + iso_5427_1981, + iso_ir_54 = iso_5427_1981, + isoir54 = iso_5427_1981, + isoir_54 = iso_5427_1981, + iso_ir54 = iso_5427_1981, + iso5427_1981 = iso_5427_1981, + iso5427cyrillic1981 = iso_5427_1981, + iso_5428_1980, + iso_ir_55 = iso_5428_1980, + iso5428_1980 = iso_5428_1980, + isoir55 = iso_5428_1980, + isoir_55 = iso_5428_1980, + iso_ir55 = iso_5428_1980, + csiso5428greek = iso_5428_1980, + gb_1988_80, + iso646cn = gb_1988_80, + gb1988_80 = gb_1988_80, + cn = gb_1988_80, + iso646_cn = gb_1988_80, + iso_ir_57 = gb_1988_80, + isoir57 = gb_1988_80, + isoir_57 = gb_1988_80, + iso_ir57 = gb_1988_80, + csiso57gb1988 = gb_1988_80, + gb_2312_80, + csiso58gb231280 = gb_2312_80, + chinese = gb_2312_80, + iso_ir_58 = gb_2312_80, + isoir58 = gb_2312_80, + gb2312_80 = gb_2312_80, + isoir_58 = gb_2312_80, + iso_ir58 = gb_2312_80, + ns_4551_2, + iso_ir_61 = ns_4551_2, + isoir61 = ns_4551_2, + csiso61norwegian2 = ns_4551_2, + iso646no2 = ns_4551_2, + iso646_no2 = ns_4551_2, + isoir_61 = ns_4551_2, + iso_ir61 = ns_4551_2, + no2 = ns_4551_2, + ns4551_2 = ns_4551_2, + videotex_suppl, + iso_ir_70 = videotex_suppl, + isoir70 = videotex_suppl, + csiso70videotexsupp1 = videotex_suppl, + isoir_70 = videotex_suppl, + iso_ir70 = videotex_suppl, + videotexsuppl = videotex_suppl, + pt2, + csiso84portuguese2 = pt2, + iso646pt2 = pt2, + iso646_pt2 = pt2, + iso_ir_84 = pt2, + isoir84 = pt2, + isoir_84 = pt2, + iso_ir84 = pt2, + es2, + iso646es2 = es2, + iso646_es2 = es2, + iso_ir_85 = es2, + isoir85 = es2, + isoir_85 = es2, + csiso85spanish2 = es2, + iso_ir85 = es2, + msz_7795_3, + hu = msz_7795_3, + iso_ir86 = msz_7795_3, + msz7795_3 = msz_7795_3, + iso646_hu = msz_7795_3, + csiso86hungarian = msz_7795_3, + iso_ir_86 = msz_7795_3, + isoir86 = msz_7795_3, + iso646hu = msz_7795_3, + isoir_86 = msz_7795_3, + jis_c6226_1983, + jisx0208_1983 = jis_c6226_1983, + isoir_87 = jis_c6226_1983, + iso_ir87 = jis_c6226_1983, + csiso87jisx0208 = jis_c6226_1983, + x0208 = jis_c6226_1983, + jisc6226_1983 = jis_c6226_1983, + jis_x0208_1983 = jis_c6226_1983, + iso_ir_87 = jis_c6226_1983, + isoir87 = jis_c6226_1983, + greek7, + isoir_88 = greek7, + iso_ir88 = greek7, + csiso88greek7 = greek7, + iso_ir_88 = greek7, + isoir88 = greek7, + asmo_449, + isoir89 = asmo_449, + iso_9036 = asmo_449, + isoir_89 = asmo_449, + iso_ir89 = asmo_449, + asmo449 = asmo_449, + csiso89asmo449 = asmo_449, + iso_ir_89 = asmo_449, + arabic7 = asmo_449, + iso9036 = asmo_449, + iso_ir_90, + csiso90 = iso_ir_90, + isoir90 = iso_ir_90, + isoir_90 = iso_ir_90, + iso_ir90 = iso_ir_90, + jis_c6229_1984_a, + csiso91jisc62291984a = jis_c6229_1984_a, + iso_ir_91 = jis_c6229_1984_a, + jp_ocr_a = jis_c6229_1984_a, + jisc6229_1984a = jis_c6229_1984_a, + jisc6229_1984_a = jis_c6229_1984_a, + jpocr_a = jis_c6229_1984_a, + isoir91 = jis_c6229_1984_a, + isoir_91 = jis_c6229_1984_a, + jp_ocra = jis_c6229_1984_a, + iso_ir91 = jis_c6229_1984_a, + jis_c6229_1984a = jis_c6229_1984_a, + jpocra = jis_c6229_1984_a, + jis_c6229_1984_b, + jpocrb = jis_c6229_1984_b, + iso646_jp_ocrb = jis_c6229_1984_b, + iso646_jp_ocr_b = jis_c6229_1984_b, + iso646jpocr_b = jis_c6229_1984_b, + csiso92jisc62991984b = jis_c6229_1984_b, + iso_ir_92 = jis_c6229_1984_b, + jp_ocr_b = jis_c6229_1984_b, + iso646_jpocr_b = jis_c6229_1984_b, + jisc6229_1984b = jis_c6229_1984_b, + jisc6229_1984_b = jis_c6229_1984_b, + jpocr_b = jis_c6229_1984_b, + iso646_jpocrb = jis_c6229_1984_b, + isoir92 = jis_c6229_1984_b, + iso646jp_ocrb = jis_c6229_1984_b, + isoir_92 = jis_c6229_1984_b, + jp_ocrb = jis_c6229_1984_b, + iso_ir92 = jis_c6229_1984_b, + iso646jp_ocr_b = jis_c6229_1984_b, + jis_c6229_1984b = jis_c6229_1984_b, + iso646jpocrb = jis_c6229_1984_b, + jis_c6229_1984_b_add, + csiso93jis62291984badd = jis_c6229_1984_b_add, + jp_ocr_b_add = jis_c6229_1984_b_add, + jp_ocrb_add = jis_c6229_1984_b_add, + jp_ocr_badd = jis_c6229_1984_b_add, + jpocr_b_add = jis_c6229_1984_b_add, + jpocrb_add = jis_c6229_1984_b_add, + jpocr_badd = jis_c6229_1984_b_add, + jisc6229_1984b_add = jis_c6229_1984_b_add, + jp_ocrbadd = jis_c6229_1984_b_add, + jisc6229_1984_badd = jis_c6229_1984_b_add, + jisc6229_1984_b_add = jis_c6229_1984_b_add, + iso_ir_93 = jis_c6229_1984_b_add, + jpocrbadd = jis_c6229_1984_b_add, + isoir93 = jis_c6229_1984_b_add, + jis_c6229_1984badd = jis_c6229_1984_b_add, + jis_c6229_1984b_add = jis_c6229_1984_b_add, + jis_c6229_1984_badd = jis_c6229_1984_b_add, + jisc6229_1984badd = jis_c6229_1984_b_add, + isoir_93 = jis_c6229_1984_b_add, + iso_ir93 = jis_c6229_1984_b_add, + jis_c6229_1984_hand, + csiso94jis62291984hand = jis_c6229_1984_hand, + jp_ocr_hand = jis_c6229_1984_hand, + jpocr_hand = jis_c6229_1984_hand, + jisc6229_1984_hand = jis_c6229_1984_hand, + iso_ir_94 = jis_c6229_1984_hand, + jpocrhand = jis_c6229_1984_hand, + jis_c6229_1984hand = jis_c6229_1984_hand, + jp_ocrhand = jis_c6229_1984_hand, + isoir94 = jis_c6229_1984_hand, + jisc6229_1984hand = jis_c6229_1984_hand, + isoir_94 = jis_c6229_1984_hand, + iso_ir94 = jis_c6229_1984_hand, + jis_c6229_1984_hand_add, + iso_ir95 = jis_c6229_1984_hand_add, + jp_ocrhandadd = jis_c6229_1984_hand_add, + jpocr_hand_add = jis_c6229_1984_hand_add, + jpocr_handadd = jis_c6229_1984_hand_add, + jisc6229_1984_hand_add = jis_c6229_1984_hand_add, + jis_c6229_1984_handadd = jis_c6229_1984_hand_add, + jis_c6229_1984handadd = jis_c6229_1984_hand_add, + jp_ocr_handadd = jis_c6229_1984_hand_add, + jp_ocrhand_add = jis_c6229_1984_hand_add, + iso_ir_95 = jis_c6229_1984_hand_add, + csiso95jis62291984handadd = jis_c6229_1984_hand_add, + jpocrhandadd = jis_c6229_1984_hand_add, + jp_ocr_hand_add = jis_c6229_1984_hand_add, + jpocrhand_add = jis_c6229_1984_hand_add, + isoir95 = jis_c6229_1984_hand_add, + jisc6229_1984_handadd = jis_c6229_1984_hand_add, + jisc6229_1984hand_add = jis_c6229_1984_hand_add, + jis_c6229_1984hand_add = jis_c6229_1984_hand_add, + isoir_95 = jis_c6229_1984_hand_add, + jisc6229_1984handadd = jis_c6229_1984_hand_add, + jis_c6229_1984_kana, + csiso96jisc62291984kana = jis_c6229_1984_kana, + isoir_96 = jis_c6229_1984_kana, + iso_ir96 = jis_c6229_1984_kana, + jisc6229_1984kana = jis_c6229_1984_kana, + iso_ir_96 = jis_c6229_1984_kana, + isoir96 = jis_c6229_1984_kana, + jis_c6229_1984kana = jis_c6229_1984_kana, + jisc6229_1984_kana = jis_c6229_1984_kana, + iso_2033_1983, + isoir98 = iso_2033_1983, + iso2033_1983 = iso_2033_1983, + isoir_98 = iso_2033_1983, + iso_ir98 = iso_2033_1983, + e13b = iso_2033_1983, + csiso2033 = iso_2033_1983, + iso_ir_98 = iso_2033_1983, + ansi_x3_110_1983, + naplps = ansi_x3_110_1983, + isoir99 = ansi_x3_110_1983, + ansix3_110_1983 = ansi_x3_110_1983, + csiso99naplps = ansi_x3_110_1983, + isoir_99 = ansi_x3_110_1983, + csat500_1983 = ansi_x3_110_1983, + iso_ir99 = ansi_x3_110_1983, + csa_t500_1983 = ansi_x3_110_1983, + iso_ir_99 = ansi_x3_110_1983, + t_61_7bit, + iso_ir102 = t_61_7bit, + t61_7bit = t_61_7bit, + iso_ir_102 = t_61_7bit, + isoir102 = t_61_7bit, + isoir_102 = t_61_7bit, + csiso102t617bit = t_61_7bit, + t_61_8bit, + csiso103t618bit = t_61_8bit, + iso_ir103 = t_61_8bit, + iso_ir_103 = t_61_8bit, + t61_8bit = t_61_8bit, + isoir103 = t_61_8bit, + t61 = t_61_8bit, + isoir_103 = t_61_8bit, + t_61 = t_61_8bit, + ecma_cyrillic, + iso_ir111 = ecma_cyrillic, + koi8_e = ecma_cyrillic, + iso_ir_111 = ecma_cyrillic, + isoir111 = ecma_cyrillic, + ecmacyrillic = ecma_cyrillic, + csiso111ecmacyrillic = ecma_cyrillic, + koi8e = ecma_cyrillic, + isoir_111 = ecma_cyrillic, + csa_z243_4_1985_1, + iso646ca = csa_z243_4_1985_1, + ca = csa_z243_4_1985_1, + iso_ir121 = csa_z243_4_1985_1, + csa7_1 = csa_z243_4_1985_1, + iso646_ca = csa_z243_4_1985_1, + iso_ir_121 = csa_z243_4_1985_1, + isoir121 = csa_z243_4_1985_1, + csaz243_4_1985_1 = csa_z243_4_1985_1, + csiso121canadian1 = csa_z243_4_1985_1, + isoir_121 = csa_z243_4_1985_1, + csa_z243_4_1985_2, + iso646ca2 = csa_z243_4_1985_2, + iso646_ca2 = csa_z243_4_1985_2, + iso_ir122 = csa_z243_4_1985_2, + csa7_2 = csa_z243_4_1985_2, + iso_ir_122 = csa_z243_4_1985_2, + isoir122 = csa_z243_4_1985_2, + csaz243_4_1985_2 = csa_z243_4_1985_2, + csiso122canadian2 = csa_z243_4_1985_2, + isoir_122 = csa_z243_4_1985_2, + csa_z243_4_1985_gr, + csaz243_4_1985_gr = csa_z243_4_1985_gr, + iso_ir123 = csa_z243_4_1985_gr, + csaz243_4_1985gr = csa_z243_4_1985_gr, + iso_ir_123 = csa_z243_4_1985_gr, + isoir123 = csa_z243_4_1985_gr, + csa_z243_4_1985gr = csa_z243_4_1985_gr, + csiso123csaz24341985gr = csa_z243_4_1985_gr, + isoir_123 = csa_z243_4_1985_gr, + iso_8859_6_e, + iso8859_6e = iso_8859_6_e, + csiso88596e = iso_8859_6_e, + iso_8859_6e = iso_8859_6_e, + iso8859_6_e = iso_8859_6_e, + iso_8859_6_i, + csiso88596i = iso_8859_6_i, + iso_8859_6i = iso_8859_6_i, + iso8859_6_i = iso_8859_6_i, + iso8859_6i = iso_8859_6_i, + t_101_g2, + csiso128t101g2 = t_101_g2, + t_101g2 = t_101_g2, + isoir_128 = t_101_g2, + t101g2 = t_101_g2, + t101_g2 = t_101_g2, + iso_ir128 = t_101_g2, + iso_ir_128 = t_101_g2, + isoir128 = t_101_g2, + iso_8859_8_e, + iso8859_8e = iso_8859_8_e, + iso8859_8_e = iso_8859_8_e, + csiso88598e = iso_8859_8_e, + iso_8859_8e = iso_8859_8_e, + iso_8859_8_i, + csiso88598i = iso_8859_8_i, + iso_8859_8i = iso_8859_8_i, + iso8859_8i = iso_8859_8_i, + iso8859_8_i = iso_8859_8_i, + csn_369103, + isoir139 = csn_369103, + csiso139csn369103 = csn_369103, + isoir_139 = csn_369103, + iso_ir139 = csn_369103, + csn369103 = csn_369103, + iso_ir_139 = csn_369103, + jus_i_b1_002, + js = jus_i_b1_002, + iso646yu = jus_i_b1_002, + iso_ir141 = jus_i_b1_002, + yu = jus_i_b1_002, + iso_ir_141 = jus_i_b1_002, + jusib1_002 = jus_i_b1_002, + iso646_yu = jus_i_b1_002, + isoir141 = jus_i_b1_002, + jusi_b1_002 = jus_i_b1_002, + csiso141jusib1002 = jus_i_b1_002, + jus_ib1_002 = jus_i_b1_002, + isoir_141 = jus_i_b1_002, + iec_p27_1, + isoir_143 = iec_p27_1, + iso_ir143 = iec_p27_1, + iecp27_1 = iec_p27_1, + iso_ir_143 = iec_p27_1, + csiso143iecp271 = iec_p27_1, + isoir143 = iec_p27_1, + jus_i_b1_003_serb, + jus_ib1_003_serb = jus_i_b1_003_serb, + jus_i_b1_003serb = jus_i_b1_003_serb, + isoir_146 = jus_i_b1_003_serb, + jusi_b1_003_serb = jus_i_b1_003_serb, + jus_ib1_003serb = jus_i_b1_003_serb, + jusib1_003serb = jus_i_b1_003_serb, + iso_ir146 = jus_i_b1_003_serb, + jusi_b1_003serb = jus_i_b1_003_serb, + jusib1_003_serb = jus_i_b1_003_serb, + csiso146serbian = jus_i_b1_003_serb, + iso_ir_146 = jus_i_b1_003_serb, + serbian = jus_i_b1_003_serb, + isoir146 = jus_i_b1_003_serb, + jus_i_b1_003_mac, + isoir147 = jus_i_b1_003_mac, + jus_ib1_003_mac = jus_i_b1_003_mac, + jusib1_003_mac = jus_i_b1_003_mac, + csiso147macedonian = jus_i_b1_003_mac, + jusib1_003mac = jus_i_b1_003_mac, + isoir_147 = jus_i_b1_003_mac, + jus_i_b1_003mac = jus_i_b1_003_mac, + jusi_b1_003mac = jus_i_b1_003_mac, + jusi_b1_003_mac = jus_i_b1_003_mac, + iso_ir147 = jus_i_b1_003_mac, + iso_ir_147 = jus_i_b1_003_mac, + jus_ib1_003mac = jus_i_b1_003_mac, + macedonian = jus_i_b1_003_mac, + greek_ccitt, + iso_ir150 = greek_ccitt, + iso_ir_150 = greek_ccitt, + isoir150 = greek_ccitt, + greekccitt = greek_ccitt, + csiso150greekccitt = greek_ccitt, + csiso150 = greek_ccitt, + isoir_150 = greek_ccitt, + nc_nc00_10_81, + isoir_151 = nc_nc00_10_81, + ncnc00_10_81 = nc_nc00_10_81, + iso_ir151 = nc_nc00_10_81, + csiso151cuba = nc_nc00_10_81, + iso_ir_151 = nc_nc00_10_81, + iso646cu = nc_nc00_10_81, + isoir151 = nc_nc00_10_81, + iso646_cu = nc_nc00_10_81, + cuba = nc_nc00_10_81, + iso_6937_2_25, + csiso6937add = iso_6937_2_25, + isoir_152 = iso_6937_2_25, + iso_ir152 = iso_6937_2_25, + iso_ir_152 = iso_6937_2_25, + isoir152 = iso_6937_2_25, + iso6937_2_25 = iso_6937_2_25, + gost_19768_74, + gost19768_74 = gost_19768_74, + stsev_358_88 = gost_19768_74, + isoir_153 = gost_19768_74, + st_sev_358_88 = gost_19768_74, + csiso153gost1976874 = gost_19768_74, + iso_ir153 = gost_19768_74, + iso_ir_153 = gost_19768_74, + isoir153 = gost_19768_74, + stsev358_88 = gost_19768_74, + st_sev358_88 = gost_19768_74, + iso_8859_supp, + iso_8859supp = iso_8859_supp, + isoir_154 = iso_8859_supp, + csiso8859supp = iso_8859_supp, + iso8859supp = iso_8859_supp, + iso_ir154 = iso_8859_supp, + iso_ir_154 = iso_8859_supp, + iso8859_supp = iso_8859_supp, + latin1_2_5 = iso_8859_supp, + isoir154 = iso_8859_supp, + iso_10367_box, + iso10367_box = iso_10367_box, + isoir_155 = iso_10367_box, + iso_ir155 = iso_10367_box, + csiso10367box = iso_10367_box, + iso10367box = iso_10367_box, + iso_ir_155 = iso_10367_box, + isoir155 = iso_10367_box, + iso_10367box = iso_10367_box, + latin_lap, + iso_ir_158 = latin_lap, + latinlap = latin_lap, + isoir158 = latin_lap, + csiso158lap = latin_lap, + isoir_158 = latin_lap, + lap = latin_lap, + iso_ir158 = latin_lap, + jis_x0212_1990, + csiso159jisx02121990 = jis_x0212_1990, + iso_ir_159 = jis_x0212_1990, + isoir159 = jis_x0212_1990, + isoir_159 = jis_x0212_1990, + iso_ir159 = jis_x0212_1990, + x0212 = jis_x0212_1990, + jisx0212_1990 = jis_x0212_1990, + ds_2089, + iso646_dk = ds_2089, + ds2089 = ds_2089, + csiso646danish = ds_2089, + iso646dk = ds_2089, + dk = ds_2089, + us_dk, + csusdk = us_dk, + usdk = us_dk, + dk_us, + csdkus = dk_us, + dkus = dk_us, + ksc5636, + iso646_kr = ksc5636, + csksc5636 = ksc5636, + iso646kr = ksc5636, + unicode_1_1_utf_7, + unicode1_1_utf_7 = unicode_1_1_utf_7, + unicode_1_1_utf7 = unicode_1_1_utf_7, + unicode_1_1utf7 = unicode_1_1_utf_7, + csunicode11utf7 = unicode_1_1_utf_7, + unicode1_1utf7 = unicode_1_1_utf_7, + unicode1_1utf_7 = unicode_1_1_utf_7, + unicode_1_1utf_7 = unicode_1_1_utf_7, + unicode1_1_utf7 = unicode_1_1_utf_7, + iso_2022_cn, + iso2022cn = iso_2022_cn, + iso2022_cn = iso_2022_cn, + iso_2022cn = iso_2022_cn, + iso_2022_cn_ext, + iso2022_cnext = iso_2022_cn_ext, + iso_2022cnext = iso_2022_cn_ext, + iso2022_cn_ext = iso_2022_cn_ext, + iso_2022_cnext = iso_2022_cn_ext, + iso2022cn_ext = iso_2022_cn_ext, + iso_2022cn_ext = iso_2022_cn_ext, + iso2022cnext = iso_2022_cn_ext, + utf_8, + utf8 = utf_8, + iso_8859_13, + iso8859_13 = iso_8859_13, + iso_8859_14, + iso8859_14 = iso_8859_14, + iso_ir199 = iso_8859_14, + iso_celtic = iso_8859_14, + latin8 = iso_8859_14, + iso_ir_199 = iso_8859_14, + isoir199 = iso_8859_14, + l8 = iso_8859_14, + iso_8859_14_1998 = iso_8859_14, + iso8859_14_1998 = iso_8859_14, + isoceltic = iso_8859_14, + isoir_199 = iso_8859_14, + iso_8859_15, + iso8859_15 = iso_8859_15, + latin9 = iso_8859_15, + latin_9 = iso_8859_15, + iso_8859_16, + l10 = iso_8859_16, + iso_ir_226 = iso_8859_16, + isoir226 = iso_8859_16, + iso8859_16 = iso_8859_16, + latin10 = iso_8859_16, + isoir_226 = iso_8859_16, + iso_ir226 = iso_8859_16, + iso_8859_16_2001 = iso_8859_16, + iso8859_16_2001 = iso_8859_16, + gbk, + ms936 = gbk, + cp936 = gbk, + windows_936 = gbk, + windows936 = gbk, + gb18030, + osd_ebcdic_df04_15, + osdebcdic_df04_15 = osd_ebcdic_df04_15, + osd_ebcdicdf04_15 = osd_ebcdic_df04_15, + osdebcdicdf04_15 = osd_ebcdic_df04_15, + osd_ebcdic_df03_irv, + osd_ebcdic_df03irv = osd_ebcdic_df03_irv, + osdebcdic_df03irv = osd_ebcdic_df03_irv, + osd_ebcdicdf03_irv = osd_ebcdic_df03_irv, + osdebcdic_df03_irv = osd_ebcdic_df03_irv, + osd_ebcdicdf03irv = osd_ebcdic_df03_irv, + osdebcdicdf03irv = osd_ebcdic_df03_irv, + osdebcdicdf03_irv = osd_ebcdic_df03_irv, + osd_ebcdic_df04_1, + osdebcdicdf04_1 = osd_ebcdic_df04_1, + osdebcdic_df04_1 = osd_ebcdic_df04_1, + osd_ebcdicdf04_1 = osd_ebcdic_df04_1, + iso_11548_1, + csiso115481 = iso_11548_1, + isotr_11548_1 = iso_11548_1, + isotr11548_1 = iso_11548_1, + iso11548_1 = iso_11548_1, + iso_tr_11548_1 = iso_11548_1, + iso_tr11548_1 = iso_11548_1, + kz_1048, + cskz1048 = kz_1048, + strk1048_2002 = kz_1048, + kz1048 = kz_1048, + rk1048 = kz_1048, + iso_10646_ucs_2, + iso10646_ucs_2 = iso_10646_ucs_2, + csunicode = iso_10646_ucs_2, + ucs2 = iso_10646_ucs_2, + iso10646ucs_2 = iso_10646_ucs_2, + iso_10646ucs2 = iso_10646_ucs_2, + iso_10646ucs_2 = iso_10646_ucs_2, + ucs_2 = iso_10646_ucs_2, + iso10646_ucs2 = iso_10646_ucs_2, + iso_10646_ucs2 = iso_10646_ucs_2, + iso10646ucs2 = iso_10646_ucs_2, + iso_10646_ucs_4, + iso_10646_ucs4 = iso_10646_ucs_4, + iso10646ucs4 = iso_10646_ucs_4, + csucs4 = iso_10646_ucs_4, + iso10646_ucs_4 = iso_10646_ucs_4, + ucs4 = iso_10646_ucs_4, + iso10646ucs_4 = iso_10646_ucs_4, + iso_10646ucs4 = iso_10646_ucs_4, + iso_10646ucs_4 = iso_10646_ucs_4, + ucs_4 = iso_10646_ucs_4, + iso10646_ucs4 = iso_10646_ucs_4, + iso_10646_ucs_basic, + iso10646ucsbasic = iso_10646_ucs_basic, + iso10646ucs_basic = iso_10646_ucs_basic, + iso_10646ucs_basic = iso_10646_ucs_basic, + iso10646_ucs_basic = iso_10646_ucs_basic, + iso_10646ucsbasic = iso_10646_ucs_basic, + iso10646_ucsbasic = iso_10646_ucs_basic, + csunicodeascii = iso_10646_ucs_basic, + iso_10646_ucsbasic = iso_10646_ucs_basic, + iso_10646_unicode_latin1, + csunicodelatin1 = iso_10646_unicode_latin1, + iso10646_unicode_latin1 = iso_10646_unicode_latin1, + iso_10646 = iso_10646_unicode_latin1, + iso_10646unicode_latin1 = iso_10646_unicode_latin1, + iso10646unicodelatin1 = iso_10646_unicode_latin1, + iso10646 = iso_10646_unicode_latin1, + iso10646_unicodelatin1 = iso_10646_unicode_latin1, + iso_10646_unicodelatin1 = iso_10646_unicode_latin1, + iso_10646unicodelatin1 = iso_10646_unicode_latin1, + iso10646unicode_latin1 = iso_10646_unicode_latin1, + iso_10646_j_1, + iso10646_j1 = iso_10646_j_1, + iso_10646_j1 = iso_10646_j_1, + iso10646_j_1 = iso_10646_j_1, + iso10646j1 = iso_10646_j_1, + iso_10646j1 = iso_10646_j_1, + iso10646j_1 = iso_10646_j_1, + iso_10646j_1 = iso_10646_j_1, + iso_unicode_ibm_1261, + csunicodeibm1261 = iso_unicode_ibm_1261, + iso_unicode_ibm1261 = iso_unicode_ibm_1261, + iso_unicodeibm_1261 = iso_unicode_ibm_1261, + isounicode_ibm_1261 = iso_unicode_ibm_1261, + isounicode_ibm1261 = iso_unicode_ibm_1261, + isounicodeibm1261 = iso_unicode_ibm_1261, + iso_unicodeibm1261 = iso_unicode_ibm_1261, + isounicodeibm_1261 = iso_unicode_ibm_1261, + iso_unicode_ibm_1268, + isounicode_ibm1268 = iso_unicode_ibm_1268, + isounicodeibm1268 = iso_unicode_ibm_1268, + iso_unicodeibm1268 = iso_unicode_ibm_1268, + isounicodeibm_1268 = iso_unicode_ibm_1268, + csunicodeibm1268 = iso_unicode_ibm_1268, + iso_unicode_ibm1268 = iso_unicode_ibm_1268, + iso_unicodeibm_1268 = iso_unicode_ibm_1268, + isounicode_ibm_1268 = iso_unicode_ibm_1268, + iso_unicode_ibm_1276, + isounicode_ibm1276 = iso_unicode_ibm_1276, + isounicodeibm1276 = iso_unicode_ibm_1276, + iso_unicodeibm1276 = iso_unicode_ibm_1276, + isounicodeibm_1276 = iso_unicode_ibm_1276, + csunicodeibm1276 = iso_unicode_ibm_1276, + iso_unicode_ibm1276 = iso_unicode_ibm_1276, + iso_unicodeibm_1276 = iso_unicode_ibm_1276, + isounicode_ibm_1276 = iso_unicode_ibm_1276, + iso_unicode_ibm_1264, + iso_unicodeibm1264 = iso_unicode_ibm_1264, + isounicodeibm_1264 = iso_unicode_ibm_1264, + csunicodeibm1264 = iso_unicode_ibm_1264, + iso_unicode_ibm1264 = iso_unicode_ibm_1264, + iso_unicodeibm_1264 = iso_unicode_ibm_1264, + isounicode_ibm_1264 = iso_unicode_ibm_1264, + isounicode_ibm1264 = iso_unicode_ibm_1264, + isounicodeibm1264 = iso_unicode_ibm_1264, + iso_unicode_ibm_1265, + isounicode_ibm1265 = iso_unicode_ibm_1265, + isounicodeibm1265 = iso_unicode_ibm_1265, + iso_unicodeibm1265 = iso_unicode_ibm_1265, + isounicodeibm_1265 = iso_unicode_ibm_1265, + csunicodeibm1265 = iso_unicode_ibm_1265, + iso_unicode_ibm1265 = iso_unicode_ibm_1265, + iso_unicodeibm_1265 = iso_unicode_ibm_1265, + isounicode_ibm_1265 = iso_unicode_ibm_1265, + unicode_1_1, + csunicode11 = unicode_1_1, + unicode1_1 = unicode_1_1, + scsu, + utf_7, + utf7 = utf_7, + utf_16be, + utf16be = utf_16be, + utf_16le, + utf16le = utf_16le, + utf_16, + utf16 = utf_16, + cesu_8, + cscesu8 = cesu_8, + cesu8 = cesu_8, + cscesu_8 = cesu_8, + utf_32, + utf32 = utf_32, + utf_32be, + utf32be = utf_32be, + utf_32le, + utf32le = utf_32le, + bocu_1, + csbocu1 = bocu_1, + bocu1 = bocu_1, + csbocu_1 = bocu_1, + iso_8859_1_windows_3_0_latin_1, + iso_8859_1_windows3_0_latin_1= iso_8859_1_windows_3_0_latin_1, + iso_8859_1windows3_0_latin1= iso_8859_1_windows_3_0_latin_1, + iso8859_1_windows3_0_latin_1= iso_8859_1_windows_3_0_latin_1, + iso8859_1windows3_0latin_1= iso_8859_1_windows_3_0_latin_1, + iso_8859_1_windows3_0_latin1= iso_8859_1_windows_3_0_latin_1, + iso_8859_1windows_3_0latin1= iso_8859_1_windows_3_0_latin_1, + iso8859_1_windows3_0latin1= iso_8859_1_windows_3_0_latin_1, + iso8859_1windows3_0latin1 = iso_8859_1_windows_3_0_latin_1, + iso_8859_1windows_3_0latin_1= iso_8859_1_windows_3_0_latin_1, + iso_8859_1_windows_3_0latin1= iso_8859_1_windows_3_0_latin_1, + iso_8859_1windows3_0_latin_1= iso_8859_1_windows_3_0_latin_1, + iso8859_1_windows3_0_latin1= iso_8859_1_windows_3_0_latin_1, + iso_8859_1_windows_3_0_latin1= iso_8859_1_windows_3_0_latin_1, + iso8859_1_windows_3_0_latin1= iso_8859_1_windows_3_0_latin_1, + iso8859_1windows_3_0latin_1= iso_8859_1_windows_3_0_latin_1, + iso_8859_1windows3_0latin1= iso_8859_1_windows_3_0_latin_1, + iso8859_1_windows_3_0latin1= iso_8859_1_windows_3_0_latin_1, + iso8859_1windows3_0_latin_1= iso_8859_1_windows_3_0_latin_1, + iso_8859_1windows_3_0_latin_1= iso_8859_1_windows_3_0_latin_1, + iso_8859_1windows3_0latin_1= iso_8859_1_windows_3_0_latin_1, + iso8859_1windows_3_0_latin_1= iso_8859_1_windows_3_0_latin_1, + iso8859_1windows3_0_latin1= iso_8859_1_windows_3_0_latin_1, + iso_8859_1_windows3_0latin1= iso_8859_1_windows_3_0_latin_1, + iso_8859_1_windows3_0latin_1= iso_8859_1_windows_3_0_latin_1, + iso_8859_1windows_3_0_latin1= iso_8859_1_windows_3_0_latin_1, + iso8859_1windows_3_0latin1= iso_8859_1_windows_3_0_latin_1, + cswindows30latin1 = iso_8859_1_windows_3_0_latin_1, + iso8859_1_windows_3_0_latin_1= iso_8859_1_windows_3_0_latin_1, + iso8859_1_windows3_0latin_1= iso_8859_1_windows_3_0_latin_1, + iso8859_1windows_3_0_latin1= iso_8859_1_windows_3_0_latin_1, + iso_8859_1_windows_3_0latin_1= iso_8859_1_windows_3_0_latin_1, + iso8859_1_windows_3_0latin_1= iso_8859_1_windows_3_0_latin_1, + iso_8859_1_windows_3_1_latin_1, + iso_8859_1windows3_1_latin1= iso_8859_1_windows_3_1_latin_1, + iso_8859_1_windows_3_1_latin1= iso_8859_1_windows_3_1_latin_1, + iso_8859_1_windows3_1_latin1= iso_8859_1_windows_3_1_latin_1, + iso_8859_1windows_3_1_latin_1= iso_8859_1_windows_3_1_latin_1, + iso_8859_1windows_3_1latin_1= iso_8859_1_windows_3_1_latin_1, + iso_8859_1windows3_1_latin_1= iso_8859_1_windows_3_1_latin_1, + iso8859_1windows3_1_latin1= iso_8859_1_windows_3_1_latin_1, + iso_8859_1windows_3_1latin1= iso_8859_1_windows_3_1_latin_1, + iso8859_1_windows3_1_latin1= iso_8859_1_windows_3_1_latin_1, + iso8859_1_windows3_1latin1= iso_8859_1_windows_3_1_latin_1, + iso8859_1windows3_1latin1 = iso_8859_1_windows_3_1_latin_1, + iso8859_1_windows_3_1_latin1= iso_8859_1_windows_3_1_latin_1, + iso8859_1windows_3_1latin_1= iso_8859_1_windows_3_1_latin_1, + iso8859_1windows3_1_latin_1= iso_8859_1_windows_3_1_latin_1, + iso_8859_1_windows_3_1latin1= iso_8859_1_windows_3_1_latin_1, + iso8859_1windows_3_1_latin_1= iso_8859_1_windows_3_1_latin_1, + iso_8859_1windows3_1latin_1= iso_8859_1_windows_3_1_latin_1, + iso_8859_1_windows_3_1latin_1= iso_8859_1_windows_3_1_latin_1, + iso_8859_1_windows3_1_latin_1= iso_8859_1_windows_3_1_latin_1, + iso_8859_1windows3_1latin1= iso_8859_1_windows_3_1_latin_1, + iso8859_1_windows_3_1latin1= iso_8859_1_windows_3_1_latin_1, + iso_8859_1_windows3_1latin_1= iso_8859_1_windows_3_1_latin_1, + iso_8859_1windows_3_1_latin1= iso_8859_1_windows_3_1_latin_1, + iso8859_1_windows_3_1_latin_1= iso_8859_1_windows_3_1_latin_1, + iso_8859_1_windows3_1latin1= iso_8859_1_windows_3_1_latin_1, + iso8859_1windows3_1latin_1= iso_8859_1_windows_3_1_latin_1, + iso8859_1_windows3_1latin_1= iso_8859_1_windows_3_1_latin_1, + iso8859_1windows_3_1_latin1= iso_8859_1_windows_3_1_latin_1, + iso8859_1windows_3_1latin1= iso_8859_1_windows_3_1_latin_1, + cswindows31latin1 = iso_8859_1_windows_3_1_latin_1, + iso8859_1_windows_3_1latin_1= iso_8859_1_windows_3_1_latin_1, + iso8859_1_windows3_1_latin_1= iso_8859_1_windows_3_1_latin_1, + iso_8859_2_windows_latin_2, + cswindows31latin2 = iso_8859_2_windows_latin_2, + iso8859_2_windowslatin_2 = iso_8859_2_windows_latin_2, + iso_8859_2_windowslatin_2 = iso_8859_2_windows_latin_2, + iso_8859_2windows_latin2 = iso_8859_2_windows_latin_2, + iso8859_2_windowslatin2 = iso_8859_2_windows_latin_2, + iso8859_2windows_latin_2 = iso_8859_2_windows_latin_2, + iso_8859_2windows_latin_2 = iso_8859_2_windows_latin_2, + iso8859_2_windows_latin2 = iso_8859_2_windows_latin_2, + iso8859_2_windows_latin_2 = iso_8859_2_windows_latin_2, + iso8859_2windowslatin_2 = iso_8859_2_windows_latin_2, + iso_8859_2_windows_latin2 = iso_8859_2_windows_latin_2, + iso8859_2windowslatin2 = iso_8859_2_windows_latin_2, + iso_8859_2_windowslatin2 = iso_8859_2_windows_latin_2, + iso_8859_2windowslatin2 = iso_8859_2_windows_latin_2, + iso_8859_2windowslatin_2 = iso_8859_2_windows_latin_2, + iso8859_2windows_latin2 = iso_8859_2_windows_latin_2, + iso_8859_9_windows_latin_5, + iso_8859_9_windows_latin5 = iso_8859_9_windows_latin_5, + iso_8859_9windowslatin5 = iso_8859_9_windows_latin_5, + iso_8859_9_windowslatin5 = iso_8859_9_windows_latin_5, + iso8859_9_windows_latin_5 = iso_8859_9_windows_latin_5, + cswindows31latin5 = iso_8859_9_windows_latin_5, + iso_8859_9windowslatin_5 = iso_8859_9_windows_latin_5, + iso8859_9windows_latin5 = iso_8859_9_windows_latin_5, + iso8859_9_windowslatin_5 = iso_8859_9_windows_latin_5, + iso_8859_9_windowslatin_5 = iso_8859_9_windows_latin_5, + iso8859_9windowslatin5 = iso_8859_9_windows_latin_5, + iso_8859_9windows_latin_5 = iso_8859_9_windows_latin_5, + iso_8859_9windows_latin5 = iso_8859_9_windows_latin_5, + iso8859_9_windowslatin5 = iso_8859_9_windows_latin_5, + iso8859_9windows_latin_5 = iso_8859_9_windows_latin_5, + iso8859_9windowslatin_5 = iso_8859_9_windows_latin_5, + iso8859_9_windows_latin5 = iso_8859_9_windows_latin_5, + hp_roman8, + r8 = hp_roman8, + cshproman8 = hp_roman8, + roman8 = hp_roman8, + hproman8 = hp_roman8, + adobe_standard_encoding, + adobestandardencoding = adobe_standard_encoding, + adobe_standardencoding = adobe_standard_encoding, + adobestandard_encoding = adobe_standard_encoding, + csadobestandardencoding = adobe_standard_encoding, + ventura_us, + csventuraus = ventura_us, + venturaus = ventura_us, + ventura_international, + csventurainternational = ventura_international, + venturainternational = ventura_international, + dec_mcs, + decmcs = dec_mcs, + dec = dec_mcs, + csdecmcs = dec_mcs, + ibm850, + cp850 = ibm850, + cspc850multilingual = ibm850, + pc8_danish_norwegian, + cspc8danishnorwegian = pc8_danish_norwegian, + pc8danishnorwegian = pc8_danish_norwegian, + pc8danish_norwegian = pc8_danish_norwegian, + pc8_danishnorwegian = pc8_danish_norwegian, + ibm862, + cspc862latinhebrew = ibm862, + cp862 = ibm862, + pc8_turkish, + pc8turkish = pc8_turkish, + cspc8turkish = pc8_turkish, + ibm_symbols, + ibmsymbols = ibm_symbols, + csibmsymbols = ibm_symbols, + ibm_thai, + ibmthai = ibm_thai, + csibmthai = ibm_thai, + hp_legal, + hplegal = hp_legal, + cshplegal = hp_legal, + hp_pi_font, + hppi_font = hp_pi_font, + cshppifont = hp_pi_font, + hppifont = hp_pi_font, + hp_pifont = hp_pi_font, + hp_math8, + cshpmath8 = hp_math8, + hpmath8 = hp_math8, + adobe_symbol_encoding, + cshppsmath = adobe_symbol_encoding, + adobe_symbolencoding = adobe_symbol_encoding, + adobesymbolencoding = adobe_symbol_encoding, + adobesymbol_encoding = adobe_symbol_encoding, + hp_desktop, + cshpdesktop = hp_desktop, + hpdesktop = hp_desktop, + ventura_math, + csventuramath = ventura_math, + venturamath = ventura_math, + microsoft_publishing, + microsoftpublishing = microsoft_publishing, + csmicrosoftpublishing = microsoft_publishing, + windows_31j, + cswindows31j = windows_31j, + windows31j = windows_31j, + gb2312, + csgb2312 = gb2312, + big5, + csbig5 = big5, + macintosh, + csmacintosh = macintosh, + mac = macintosh, + ibm037, + ebcdic_cp_nl = ibm037, + ebcdic_cp_us = ibm037, + csibm037 = ibm037, + ebcdiccp_wt = ibm037, + ebcdiccp_ca = ibm037, + ebcdiccpnl = ibm037, + ebcdic_cp_wt = ibm037, + ebcdic_cp_ca = ibm037, + ebcdiccpus = ibm037, + ebcdiccpwt = ibm037, + ebcdiccpca = ibm037, + ebcdic_cpnl = ibm037, + ebcdic_cpus = ibm037, + cp037 = ibm037, + ebcdiccp_nl = ibm037, + ebcdic_cpwt = ibm037, + ebcdic_cpca = ibm037, + ebcdiccp_us = ibm037, + ibm038, + csibm038 = ibm038, + ebcdic_int = ibm038, + cp038 = ibm038, + ebcdicint = ibm038, + ibm273, + cp273 = ibm273, + csibm273 = ibm273, + ibm274, + csibm274 = ibm274, + ebcdic_be = ibm274, + cp274 = ibm274, + ebcdicbe = ibm274, + ibm275, + ebcdic_br = ibm275, + csibm275 = ibm275, + cp275 = ibm275, + ebcdicbr = ibm275, + ibm277, + ebcdiccp_dk = ibm277, + ebcdiccp_no = ibm277, + ebcdic_cp_dk = ibm277, + csibm277 = ibm277, + ebcdic_cp_no = ibm277, + ebcdiccpdk = ibm277, + ebcdiccpno = ibm277, + ebcdic_cpdk = ibm277, + ebcdic_cpno = ibm277, + ibm278, + ebcdiccp_fi = ibm278, + ebcdiccpse = ibm278, + ebcdic_cp_fi = ibm278, + csibm278 = ibm278, + ebcdiccpfi = ibm278, + ebcdic_cpse = ibm278, + ebcdiccp_se = ibm278, + ebcdic_cpfi = ibm278, + ebcdic_cp_se = ibm278, + cp278 = ibm278, + ibm280, + ebcdiccp_it = ibm280, + ebcdic_cp_it = ibm280, + cp280 = ibm280, + ebcdiccpit = ibm280, + csibm280 = ibm280, + ebcdic_cpit = ibm280, + ibm281, + ebcdic_jp_e = ibm281, + ebcdicjpe = ibm281, + cp281 = ibm281, + ebcdic_jpe = ibm281, + csibm281 = ibm281, + ebcdicjp_e = ibm281, + ibm284, + csibm284 = ibm284, + ebcdiccpes = ibm284, + ebcdic_cpes = ibm284, + cp284 = ibm284, + ebcdiccp_es = ibm284, + ebcdic_cp_es = ibm284, + ibm285, + csibm285 = ibm285, + ebcdic_cpgb = ibm285, + ebcdiccp_gb = ibm285, + ebcdic_cp_gb = ibm285, + cp285 = ibm285, + ebcdiccpgb = ibm285, + ibm290, + ebcdicjp_kana = ibm290, + ebcdicjpkana = ibm290, + cp290 = ibm290, + ebcdic_jp_kana = ibm290, + csibm290 = ibm290, + ebcdic_jpkana = ibm290, + ibm297, + ebcdiccpfr = ibm297, + csibm297 = ibm297, + ebcdic_cpfr = ibm297, + ebcdiccp_fr = ibm297, + ebcdic_cp_fr = ibm297, + cp297 = ibm297, + ibm420, + csibm420 = ibm420, + ebcdiccp_ar1 = ibm420, + ebcdiccpar1 = ibm420, + ebcdic_cpar1 = ibm420, + cp420 = ibm420, + ebcdic_cp_ar1 = ibm420, + ibm423, + ebcdiccpgr = ibm423, + csibm423 = ibm423, + ebcdic_cpgr = ibm423, + ebcdiccp_gr = ibm423, + cp423 = ibm423, + ebcdic_cp_gr = ibm423, + ibm424, + ebcdic_cp_he = ibm424, + ebcdiccphe = ibm424, + csibm424 = ibm424, + ebcdic_cphe = ibm424, + ebcdiccp_he = ibm424, + cp424 = ibm424, + ibm437, + cp437 = ibm437, + cspc8codepage437 = ibm437, + ibm500, + ebcdiccpch = ibm500, + ebcdic_cpbe = ibm500, + csibm500 = ibm500, + ebcdiccp_be = ibm500, + ebcdic_cpch = ibm500, + ebcdic_cp_be = ibm500, + ebcdiccp_ch = ibm500, + ebcdiccpbe = ibm500, + ebcdic_cp_ch = ibm500, + cp500 = ibm500, + ibm851, + cp851 = ibm851, + csibm851 = ibm851, + ibm852, + cspcp852 = ibm852, + cp852 = ibm852, + ibm855, + csibm855 = ibm855, + cp855 = ibm855, + ibm857, + csibm857 = ibm857, + cp857 = ibm857, + ibm860, + cp860 = ibm860, + csibm860 = ibm860, + ibm861, + csibm861 = ibm861, + cp_is = ibm861, + cp861 = ibm861, + cpis = ibm861, + ibm863, + csibm863 = ibm863, + cp863 = ibm863, + ibm864, + csibm864 = ibm864, + cp864 = ibm864, + ibm865, + csibm865 = ibm865, + cp865 = ibm865, + ibm868, + cpar = ibm868, + cp868 = ibm868, + csibm868 = ibm868, + cp_ar = ibm868, + ibm869, + cp869 = ibm869, + cp_gr = ibm869, + cpgr = ibm869, + csibm869 = ibm869, + ibm870, + csibm870 = ibm870, + ebcdic_cproece = ibm870, + ebcdiccp_roece = ibm870, + ebcdic_cpyu = ibm870, + ebcdiccp_yu = ibm870, + ebcdiccproece = ibm870, + ebcdic_cp_yu = ibm870, + cp870 = ibm870, + ebcdic_cp_roece = ibm870, + ebcdiccpyu = ibm870, + ibm871, + ebcdiccp_is = ibm871, + csibm871 = ibm871, + ebcdic_cp_is = ibm871, + ebcdiccpis = ibm871, + cp871 = ibm871, + ebcdic_cpis = ibm871, + ibm880, + ebcdiccyrillic = ibm880, + csibm880 = ibm880, + ebcdic_cyrillic = ibm880, + cp880 = ibm880, + ibm891, + csibm891 = ibm891, + cp891 = ibm891, + ibm903, + csibm903 = ibm903, + cp903 = ibm903, + ibm904, + csibbm904 = ibm904, + cp904 = ibm904, + ibm905, + ebcdiccptr = ibm905, + csibm905 = ibm905, + ebcdic_cptr = ibm905, + ebcdiccp_tr = ibm905, + cp905 = ibm905, + ebcdic_cp_tr = ibm905, + ibm918, + cp918 = ibm918, + ebcdiccp_ar2 = ibm918, + ebcdiccpar2 = ibm918, + csibm918 = ibm918, + ebcdic_cpar2 = ibm918, + ebcdic_cp_ar2 = ibm918, + ibm1026, + csibm1026 = ibm1026, + cp1026 = ibm1026, + ebcdic_at_de, + ebcdic_atde = ebcdic_at_de, + ebcdicat_de = ebcdic_at_de, + ebcdicatde = ebcdic_at_de, + csibmebcdicatde = ebcdic_at_de, + ebcdic_at_de_a, + ebcdic_atdea = ebcdic_at_de_a, + ebcdic_at_dea = ebcdic_at_de_a, + ebcdic_atde_a = ebcdic_at_de_a, + csebcdicatdea = ebcdic_at_de_a, + ebcdicatdea = ebcdic_at_de_a, + ebcdicat_dea = ebcdic_at_de_a, + ebcdicat_de_a = ebcdic_at_de_a, + ebcdicatde_a = ebcdic_at_de_a, + ebcdic_ca_fr, + ebcdiccafr = ebcdic_ca_fr, + csebcdiccafr = ebcdic_ca_fr, + ebcdic_cafr = ebcdic_ca_fr, + ebcdicca_fr = ebcdic_ca_fr, + ebcdic_dk_no, + ebcdic_dkno = ebcdic_dk_no, + ebcdicdkno = ebcdic_dk_no, + ebcdicdk_no = ebcdic_dk_no, + csebcdicdkno = ebcdic_dk_no, + ebcdic_dk_no_a, + ebcdicdk_noa = ebcdic_dk_no_a, + ebcdic_dknoa = ebcdic_dk_no_a, + ebcdicdkno_a = ebcdic_dk_no_a, + csebcdicdknoa = ebcdic_dk_no_a, + ebcdic_dk_noa = ebcdic_dk_no_a, + ebcdicdknoa = ebcdic_dk_no_a, + ebcdicdk_no_a = ebcdic_dk_no_a, + ebcdic_dkno_a = ebcdic_dk_no_a, + ebcdic_fi_se, + csebcdicfise = ebcdic_fi_se, + ebcdicfi_se = ebcdic_fi_se, + ebcdic_fise = ebcdic_fi_se, + ebcdicfise = ebcdic_fi_se, + ebcdic_fi_se_a, + ebcdicfise_a = ebcdic_fi_se_a, + ebcdic_fi_sea = ebcdic_fi_se_a, + ebcdic_fisea = ebcdic_fi_se_a, + csebcdicfisea = ebcdic_fi_se_a, + ebcdicfisea = ebcdic_fi_se_a, + ebcdic_fise_a = ebcdic_fi_se_a, + ebcdicfi_se_a = ebcdic_fi_se_a, + ebcdicfi_sea = ebcdic_fi_se_a, + ebcdic_fr, + ebcdicfr = ebcdic_fr, + csebcdicfr = ebcdic_fr, + ebcdic_it, + ebcdicit = ebcdic_it, + csebcdicit = ebcdic_it, + ebcdic_pt, + ebcdicpt = ebcdic_pt, + csebcdicpt = ebcdic_pt, + ebcdic_es, + ebcdices = ebcdic_es, + csebcdices = ebcdic_es, + ebcdic_es_a, + ebcdicesa = ebcdic_es_a, + ebcdic_esa = ebcdic_es_a, + ebcdices_a = ebcdic_es_a, + csebcdicesa = ebcdic_es_a, + ebcdic_es_s, + ebcdic_ess = ebcdic_es_s, + ebcdices_s = ebcdic_es_s, + csebcdicess = ebcdic_es_s, + ebcdicess = ebcdic_es_s, + ebcdic_uk, + ebcdicuk = ebcdic_uk, + csebcdicuk = ebcdic_uk, + ebcdic_us, + ebcdicus = ebcdic_us, + csebcdicus = ebcdic_us, + unknown_8bit, + csunknown8bit = unknown_8bit, + unknown8bit = unknown_8bit, + mnemonic, + csmnemonic = mnemonic, + mnem, + csmnem = mnem, + viscii, + csviscii = viscii, + viqr, + csviqr = viqr, + koi8_r, + cskoi8r = koi8_r, + koi8r = koi8_r, + hz_gb_2312, + hzgb_2312 = hz_gb_2312, + hz_gb2312 = hz_gb_2312, + hzgb2312 = hz_gb_2312, + ibm866, + csibm866 = ibm866, + cp866 = ibm866, + ibm775, + cspc775baltic = ibm775, + cp775 = ibm775, + koi8_u, + koi8u = koi8_u, + ibm00858, + cp00858 = ibm00858, + pc_multilingual_850euro = ibm00858, + pc_multilingual850_euro = ibm00858, + ccsid00858 = ibm00858, + pcmultilingual_850euro = ibm00858, + pcmultilingual_850_euro = ibm00858, + pcmultilingual850euro = ibm00858, + pc_multilingual850euro = ibm00858, + pc_multilingual_850_euro = ibm00858, + pcmultilingual850_euro = ibm00858, + ibm00924, + ccsid00924 = ibm00924, + ebcdic_latin9euro = ibm00924, + ebcdiclatin9__euro = ibm00924, + ebcdiclatin9_euro = ibm00924, + ebcdiclatin9euro = ibm00924, + ebcdic_latin9_euro = ibm00924, + ebcdic_latin9__euro = ibm00924, + cp00924 = ibm00924, + ibm01140, + ebcdicus37euro = ibm01140, + ccsid01140 = ibm01140, + ebcdicus_37_euro = ibm01140, + ebcdic_us37_euro = ibm01140, + ebcdicus37_euro = ibm01140, + ebcdic_us37euro = ibm01140, + ebcdic_us_37euro = ibm01140, + ebcdic_us_37_euro = ibm01140, + ebcdicus_37euro = ibm01140, + cp01140 = ibm01140, + ibm01141, + ccsid01141 = ibm01141, + ebcdic_de273euro = ibm01141, + ebcdicde_273_euro = ibm01141, + ebcdicde273euro = ibm01141, + ebcdic_de_273euro = ibm01141, + ebcdic_de273_euro = ibm01141, + ebcdic_de_273_euro = ibm01141, + ebcdicde_273euro = ibm01141, + ebcdicde273_euro = ibm01141, + cp01141 = ibm01141, + ibm01142, + cp01142 = ibm01142, + ebcdic_dk277_euro = ibm01142, + ebcdic_no277euro = ibm01142, + ebcdicno_277euro = ibm01142, + ebcdicno_277_euro = ibm01142, + ebcdicdk_277euro = ibm01142, + ccsid01142 = ibm01142, + ebcdic_dk_277_euro = ibm01142, + ebcdicno277_euro = ibm01142, + ebcdicdk277_euro = ibm01142, + ebcdic_no_277euro = ibm01142, + ebcdicdk277euro = ibm01142, + ebcdic_no_277_euro = ibm01142, + ebcdic_no277_euro = ibm01142, + ebcdicno277euro = ibm01142, + ebcdic_dk_277euro = ibm01142, + ebcdic_dk277euro = ibm01142, + ebcdicdk_277_euro = ibm01142, + ibm01143, + ebcdic_se278_euro = ibm01143, + ebcdicse_278euro = ibm01143, + cp01143 = ibm01143, + ebcdic_se278euro = ibm01143, + ebcdic_fi_278_euro = ibm01143, + ebcdic_fi278euro = ibm01143, + ebcdicfi278euro = ibm01143, + ebcdicse_278_euro = ibm01143, + ebcdic_fi_278euro = ibm01143, + ebcdicfi278_euro = ibm01143, + ccsid01143 = ibm01143, + ebcdic_se_278euro = ibm01143, + ebcdic_se_278_euro = ibm01143, + ebcdicse278_euro = ibm01143, + ebcdicfi_278euro = ibm01143, + ebcdicfi_278_euro = ibm01143, + ebcdicse278euro = ibm01143, + ebcdic_fi278_euro = ibm01143, + ibm01144, + ebcdic_it280_euro = ibm01144, + ebcdic_it_280_euro = ibm01144, + cp01144 = ibm01144, + ebcdic_it280euro = ibm01144, + ebcdicit_280euro = ibm01144, + ebcdicit280euro = ibm01144, + ccsid01144 = ibm01144, + ebcdicit_280_euro = ibm01144, + ebcdicit280_euro = ibm01144, + ebcdic_it_280euro = ibm01144, + ibm01145, + ebcdic_es284euro = ibm01145, + ebcdices284euro = ibm01145, + ebcdic_es_284_euro = ibm01145, + cp01145 = ibm01145, + ebcdic_es284_euro = ibm01145, + ebcdices_284euro = ibm01145, + ccsid01145 = ibm01145, + ebcdic_es_284euro = ibm01145, + ebcdices_284_euro = ibm01145, + ebcdices284_euro = ibm01145, + ibm01146, + ebcdic_gb_285_euro = ibm01146, + ebcdic_gb285_euro = ibm01146, + ebcdic_gb285euro = ibm01146, + ebcdicgb285euro = ibm01146, + ebcdic_gb_285euro = ibm01146, + ebcdicgb_285_euro = ibm01146, + cp01146 = ibm01146, + ccsid01146 = ibm01146, + ebcdicgb285_euro = ibm01146, + ebcdicgb_285euro = ibm01146, + ibm01147, + ebcdic_fr_297_euro = ibm01147, + ebcdic_fr297_euro = ibm01147, + ebcdic_fr_297euro = ibm01147, + cp01147 = ibm01147, + ebcdic_fr297euro = ibm01147, + ebcdicfr297euro = ibm01147, + ebcdicfr297_euro = ibm01147, + ebcdicfr_297_euro = ibm01147, + ccsid01147 = ibm01147, + ebcdicfr_297euro = ibm01147, + ibm01148, + ebcdic_international500euro= ibm01148, + ebcdicinternational500_euro= ibm01148, + ebcdic_international500_euro= ibm01148, + ebcdic_international_500euro= ibm01148, + ebcdic_international_500_euro= ibm01148, + cp01148 = ibm01148, + ebcdicinternational_500euro= ibm01148, + ccsid01148 = ibm01148, + ebcdicinternational_500_euro= ibm01148, + ebcdicinternational500euro= ibm01148, + ibm01149, + ebcdic_is_871euro = ibm01149, + ebcdic_is871_euro = ibm01149, + ebcdicis_871_euro = ibm01149, + ebcdic_is871euro = ibm01149, + ebcdicis871euro = ibm01149, + ebcdic_is_871_euro = ibm01149, + cp01149 = ibm01149, + ccsid01149 = ibm01149, + ebcdicis_871euro = ibm01149, + ebcdicis871_euro = ibm01149, + big5_hkscs, + big5hkscs = big5_hkscs, + ibm1047, + ibm_1047 = ibm1047, + ptcp154, + cp154 = ptcp154, + csptcp154 = ptcp154, + cyrillic_asian = ptcp154, + cyrillicasian = ptcp154, + pt154 = ptcp154, + amiga_1251, + ami_1251 = amiga_1251, + ami1251 = amiga_1251, + amiga1251 = amiga_1251, + koi7_switched, + koi7switched = koi7_switched, + brf, + csbrf = brf, + tscii, + cstscii = tscii, + windows_1250, + windows1250 = windows_1250, + windows_1251, + windows1251 = windows_1251, + windows_1252, + windows1252 = windows_1252, + windows_1253, + windows1253 = windows_1253, + windows_1254, + windows1254 = windows_1254, + windows_1255, + windows1255 = windows_1255, + windows_1256, + windows1256 = windows_1256, + windows_1257, + windows1257 = windows_1257, + windows_1258, + windows1258 = windows_1258, + tis_620, + tis620 = tis_620, + max_charset +}; diff --git a/extras/anytermd/libpbe/include/charset/sequence_conv.hh b/extras/anytermd/libpbe/include/charset/sequence_conv.hh new file mode 100644 index 0000000000..51780a456a --- /dev/null +++ b/extras/anytermd/libpbe/include/charset/sequence_conv.hh @@ -0,0 +1,74 @@ +// sequence_conv.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2008 Philip Endecott + +// Distributed under the Boost Software License, Version 1.0: +// +// Permission is hereby granted, free of charge, to any person or organization +// obtaining a copy of the software and accompanying documentation covered by +// this license (the "Software") to use, reproduce, display, distribute, +// execute, and transmit the Software, and to prepare derivative works of the +// Software, and to permit third-parties to whom the Software is furnished to +// do so, all subject to the following: +// +// The copyright notices in the Software and this entire statement, including +// the above license grant, this restriction and the following disclaimer, +// must be included in all copies of the Software, in whole or in part, and +// all derivative works of the Software, unless such copies or derivative +// works are solely in the form of machine-executable object code generated by +// a source language processor. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT +// SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE +// FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, +// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER +// DEALINGS IN THE SOFTWARE. + +#ifndef libpbe_charset_sequence_conv_hh +#define libpbe_charset_sequence_conv_hh + +#include "charset_t.hh" +#include "charset_traits.hh" +#include "char_conv.hh" +#include "error_policy.hh" + + +namespace pbe { + +// Conversion of strings between character sets +// ----------------------------------------------- +// +// This file provides facilites to convert sequences of characters, including +// strings and other sequences, from one character set to another. Essentially +// this just involves repeated application of the character-at-a-time +// char_conv template, and that is the default strategy implemented below. +// However for some iterators and some character set pairs, optimised +// implementations are possible and these can be provided by specialisation +// of sequence_conv. + +// Base template, with default implementation using char_conv: + +template< charset_t from_cs, charset_t to_cs, typename error_policy, + typename InputIterator, typename OutputIterator, typename Enable = void > +struct sequence_conv { + OutputIterator + operator() ( InputIterator first, const InputIterator& last, + OutputIterator result, + typename charset_traits::state_t& from_state, + typename charset_traits::state_t& to_state) + { + for (; first != last; ++first) { + *(result++) = char_conv()(*first,from_state,to_state); + } + return result; + } +}; + + + + +}; + +#endif diff --git a/extras/anytermd/libpbe/include/charset/sequence_converter.hh b/extras/anytermd/libpbe/include/charset/sequence_converter.hh new file mode 100644 index 0000000000..9b3b057ae1 --- /dev/null +++ b/extras/anytermd/libpbe/include/charset/sequence_converter.hh @@ -0,0 +1,69 @@ +// sequence_converter.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2008 Philip Endecott + +// Distributed under the Boost Software License, Version 1.0: +// +// Permission is hereby granted, free of charge, to any person or organization +// obtaining a copy of the software and accompanying documentation covered by +// this license (the "Software") to use, reproduce, display, distribute, +// execute, and transmit the Software, and to prepare derivative works of the +// Software, and to permit third-parties to whom the Software is furnished to +// do so, all subject to the following: +// +// The copyright notices in the Software and this entire statement, including +// the above license grant, this restriction and the following disclaimer, +// must be included in all copies of the Software, in whole or in part, and +// all derivative works of the Software, unless such copies or derivative +// works are solely in the form of machine-executable object code generated by +// a source language processor. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT +// SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE +// FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, +// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER +// DEALINGS IN THE SOFTWARE. + +#ifndef libpbe_charset_sequence_converter_hh +#define libpbe_charset_sequence_converter_hh + +#include "charset_t.hh" +#include "charset_traits.hh" +#include "sequence_conv.hh" +#include "error_policy.hh" + + +namespace pbe { + +// Stateful functor for sequence conversion +// ----------------------------------------- +// +// The function template sequence_conv takes state variables for the input and output, +// in case they are stateful. +// A simplified interface is provided by this functor, which keeps the state +// internally from one call to the next. + +template< charset_t from_cs, charset_t to_cs, typename error_policy /*= default_error_policy*/, + typename InputIterator, typename OutputIterator > +class sequence_converter { + typedef charset_traits from_traits; + typedef charset_traits to_traits; + + typename from_traits::state_t from_state; + typename to_traits::state_t to_state; + +public: + sequence_conv sc; + OutputIterator operator()(const InputIterator& first, const InputIterator& last, + OutputIterator result) { + return sc ( first, last, result, from_state, to_state ); + } + + // TODO flush, perhaps reset. +}; + +}; + +#endif diff --git a/extras/anytermd/libpbe/include/charset/string_adaptor.hh b/extras/anytermd/libpbe/include/charset/string_adaptor.hh new file mode 100644 index 0000000000..338e6dc1c6 --- /dev/null +++ b/extras/anytermd/libpbe/include/charset/string_adaptor.hh @@ -0,0 +1,196 @@ +// string_adaptor.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2007-2008 Philip Endecott + +// Distributed under the Boost Software License, Version 1.0: +// +// Permission is hereby granted, free of charge, to any person or organization +// obtaining a copy of the software and accompanying documentation covered by +// this license (the "Software") to use, reproduce, display, distribute, +// execute, and transmit the Software, and to prepare derivative works of the +// Software, and to permit third-parties to whom the Software is furnished to +// do so, all subject to the following: +// +// The copyright notices in the Software and this entire statement, including +// the above license grant, this restriction and the following disclaimer, +// must be included in all copies of the Software, in whole or in part, and +// all derivative works of the Software, unless such copies or derivative +// works are solely in the form of machine-executable object code generated by +// a source language processor. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT +// SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE +// FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, +// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER +// DEALINGS IN THE SOFTWARE. + +#ifndef libpbe_charset_string_adaptor_hh +#define libpbe_charset_string_adaptor_hh + +#include "charset/charset_t.hh" +#include "charset/charset_traits.hh" +#include "charset/const_character_iterator.hh" + +#include + +#include +#include + + +namespace pbe { + +// String adaptor +// -------------- +// +// This class adapts a std::string containing a sequence of units into a +// std::string-like object containing a sequence of characters. It stores +// a reference to the underlying unit-string. + + +template +class string_adaptor { + + typedef charset_traits traits_t; + typedef typename traits_t::char_t char_t; + typedef typename traits_t::unit_t unit_t; + +public: + typedef std::basic_string, Alloc> unit_string_t; + +private: + unit_string_t& ustr; + +public: + string_adaptor(unit_string_t& ustr_): + ustr(ustr_) + {} + + unit_string_t& unit_str() { return ustr; } + const unit_string_t& unit_str() const { return ustr; } + + typedef char_t value_type; + typedef char_t* pointer; + typedef char_t& reference; + typedef const char_t& const_reference; + typedef size_t size_type; + typedef ptrdiff_t difference_type; // ?? + static const size_type npos = -1; + // iterator ?? + typedef const_character_iterator const_iterator; + // reverse_iterator ?? + typedef boost::reverse_iterator const_reverse_iterator; + + // begin + // end + const_iterator begin() const { return const_iterator(ustr.begin()); } + const_iterator end() const { return const_iterator(ustr.end()); } + // rbegin + // rend + const_reverse_iterator rbegin() const { return const_reverse_iterator(end()); } + const_reverse_iterator rend() const { return const_reverse_iterator(begin()); } + + // size // ?? could do this in O(n) + // length + size_type max_size() const { return ustr.max_size(); } // ??? + size_type capacity() const { return ustr.capacity(); } // ??? + bool empty() const { return ustr.empty(); } + // operator[] + // operator[] const + // c_str() ??? + // data() ??? + + string_adaptor& operator=(const string_adaptor& other) { ustr=other.ustr; return *this; } + string_adaptor& operator=(char_t c) { clear(); push_back(c); } + void reserve(size_t sz) { ustr.reserve(sz); } // *typ_units_per_char + void swap(string_adaptor& other) { ustr.swap(other.ustr); } + + // insert + + string_adaptor& append(const string_adaptor& s) { ustr.append(s.unit_str()); return *this; } + string_adaptor& append(size_type n, char_t c) { + for (size_type i=0; i + string_adaptor& append(InputIterator first, InputIterator last) { + for (; first!=last; ++first) { + push_back(*first); + } + } + + void push_back(char_t c) { + std::back_insert_iterator i(ustr); + traits_t::encode(i,c); + } + + string_adaptor& operator+=(const string_adaptor& s) { append(s); } + string_adaptor& operator+=(char_t c) { push_back(c); } + +#if 0 + iterator erase(iterator p) { + ustr.erase(p.base(), p.base()+traits_t::char_length(p.base())-1); + } + iterator erase(iterator first, iterator last) { + ustr.erase(first.base(), last.base()+traits_t::char_length(last.base())-1); + } +#endif + + void clear() { ustr.clear(); } + + // resize + + string_adaptor& assign(const string_adaptor& other) { operator=(other); } + string_adaptor& assign(size_type n, char_t c) { clear(); append(n,c); } + template + string_adaptor& assign(InputIterator first, InputIterator last) { clear(); append(first,last); } + + // replace + + // copy + + // find + // find variants + + // substr + + int compare(const string_adaptor& s) const { + // FIXME apparently this is an SGI thing that's not in the standard + // return std::lexicographical_compare_3way(s.begin(), s.end(), begin(), end()); + } + // Note that for UTF8 we can do the comparison directly on the units, which + // should be faster. + +}; + + +template +bool operator==(const string_adaptor& s1, + const string_adaptor& s2) { + return s1.ustr == s2.ustr; +} + +template +bool operator!=(const string_adaptor& s1, + const string_adaptor& s2) { + return s1.ustr != s2.ustr; +} + +template +bool operator<(const string_adaptor& s1, + const string_adaptor& s2) { + return s1.compare(s2)<0; +} + +template +void swap(string_adaptor& s1, + string_adaptor& s2) { + s1.swap(s2); +} + + +}; + +#endif diff --git a/extras/anytermd/libpbe/include/charset/to_ascii_letters.hh b/extras/anytermd/libpbe/include/charset/to_ascii_letters.hh new file mode 100644 index 0000000000..5f2c4b6d4a --- /dev/null +++ b/extras/anytermd/libpbe/include/charset/to_ascii_letters.hh @@ -0,0 +1,210 @@ +// include/charset/to_ascii_letters.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef pbe_charset_to_ascii_letters_hh +#define pbe_charset_to_ascii_letters_hh + +#include "charset/char_t.hh" + +#include + + +namespace pbe { + + +typedef char char_expansion_page_00_t [3]; +typedef char char_expansion_page_01_t [3]; +typedef char char_expansion_page_02_t [3]; +typedef char char_expansion_page_1D_t [2]; +typedef char char_expansion_page_1E_t [2]; +typedef char char_expansion_page_20_t [3]; +typedef char char_expansion_page_21_t [5]; +typedef char char_expansion_page_24_t [2]; +typedef char char_expansion_page_2C_t [2]; +typedef char char_expansion_page_32_t [4]; +typedef char char_expansion_page_33_t [5]; +typedef char char_expansion_page_FB_t [4]; +typedef char char_expansion_page_FF_t [2]; +typedef char char_expansion_page_1D4_t [2]; +typedef char char_expansion_page_1D5_t [2]; +typedef char char_expansion_page_1D6_t [2]; + +extern char_expansion_page_00_t to_ascii_letters_page_00 [256]; +extern char_expansion_page_01_t to_ascii_letters_page_01 [256]; +extern char_expansion_page_02_t to_ascii_letters_page_02 [256]; +extern char_expansion_page_1D_t to_ascii_letters_page_1D [256]; +extern char_expansion_page_1E_t to_ascii_letters_page_1E [256]; +extern char_expansion_page_20_t to_ascii_letters_page_20 [256]; +extern char_expansion_page_21_t to_ascii_letters_page_21 [256]; +extern char_expansion_page_24_t to_ascii_letters_page_24 [256]; +extern char_expansion_page_2C_t to_ascii_letters_page_2C [256]; +extern char_expansion_page_32_t to_ascii_letters_page_32 [256]; +extern char_expansion_page_33_t to_ascii_letters_page_33 [256]; +extern char_expansion_page_FB_t to_ascii_letters_page_FB [256]; +extern char_expansion_page_FF_t to_ascii_letters_page_FF [256]; +extern char_expansion_page_1D4_t to_ascii_letters_page_1D4 [256]; +extern char_expansion_page_1D5_t to_ascii_letters_page_1D5 [256]; +extern char_expansion_page_1D6_t to_ascii_letters_page_1D6 [256]; + + +inline const char* to_ascii_letters(char32_t c) { + // Given a Unicode (UCS4) character c, return a pointer to a sequence of ASCII + // lower-case letters (a-z) that are equivalent in the following sense: + // - Upper case is mapped to lower case. + // - Accents (etc) are stripped. + // - "Compound" letters are decomposed into multiple individial letters (e.g. ae). + // (This includes various mathemtical symbols and oddities like VIII, for which there + // is a single unicode character.) + // If the character has no corresponding letters, an empty sequence is returned + // (e.g. punctuation symbols and letters in other scripts. Letters from e.g. cyrillic + // and greek that are homoglyphs to latin letters are not considered equivalent.) + // The aim of this conversion is to convert a string to something that can be used + // as a search key for user-supplied search terms. + // This is based on data extracted from the unicode character database, using the + // "NFKD" rules. One oddity is that the German ezsett is not expanded to ss; I don't + // know why not, or what other oddities there are. All of the useful conversions are + // in the various "Latin" pages (see below). + // The returned pointer points to static data. + + // FIXME it would probably be better to distinguish between NULL, space and other + // word-breaking punctuation, non-word-breaking punctuation, and non-latin-convertible + // characters in some way. + + int page = c>>8; + int point = c&0xff; + switch (page) { + case 0x000: return to_ascii_letters_page_00[point]; // Basic Latin & Latin-1 Supplement + case 0x001: return to_ascii_letters_page_01[point]; // Latin Extended-A & Latin Extended-B + case 0x002: return to_ascii_letters_page_02[point]; // Latin Extended-B etc. + case 0x01D: return to_ascii_letters_page_1D[point]; // Phonetic Extensions etc. + case 0x01E: return to_ascii_letters_page_1E[point]; // Latin Extended Additional. + case 0x020: return to_ascii_letters_page_20[point]; // General Punctuation etc. + case 0x021: return to_ascii_letters_page_21[point]; // Letterlike symbols etc. + case 0x024: return to_ascii_letters_page_24[point]; // Enclosed alphanumerics etc. + case 0x02C: return to_ascii_letters_page_2C[point]; // Latin Extended-C etc. + case 0x032: return to_ascii_letters_page_32[point]; // Enclosed CJK Letters and Months. + case 0x033: return to_ascii_letters_page_33[point]; // CJK Compatibility. + case 0x0FB: return to_ascii_letters_page_FB[point]; // Alphabetic Presentation Forms etc. + case 0x0FF: return to_ascii_letters_page_FF[point]; // Halfwidth and Fullwidth Forms etc. + case 0x1D4: return to_ascii_letters_page_1D4[point]; // Mathematical Alphanumeric Symbols. + case 0x1D5: return to_ascii_letters_page_1D5[point]; // (cont.) + case 0x1D6: return to_ascii_letters_page_1D6[point]; // (cont.) + default: return ""; + } +} + + +template +inline OutputIter to_ascii_letters(InputIter first, InputIter last, OutputIter result) { + // Copy Unicode (UCS4) characters in the range first to last to result, converting to ASCII + // letters as abive, and return an iterator for the end of the result. Input characters that + // don't correspond to any ASCII letters are replaced with spaces, except that 0 remains 0. + + for (InputIter i = first; i!=last; ++i) { + char32_t c = *i; + if (!c) { + *(result++) = 0; + } else { + const char* l = to_ascii_letters(c); + if (!*l) { + *(result++) = ' '; + } else { + do { + *(result++) = *(l++); + } while (*l); + } + } + } + return result; +} + + +template +class ascii_letter_iterator: public boost::iterator_facade< ascii_letter_iterator, + char, + boost::forward_traversal_tag, + char > +{ + // This is an immutable forward iterator that steps through the ascii letters (as above) + // that come from the contained iterator. Punctuation is replaced with spaces; multiple + // punctuation yeilds multiple spaces. A null in the input results in a null in the + // output (but don't try to increment past it - FIXME this is a bit broken). + // The end-of-input iterator must be supplied to the constructor. You can get away with + // passing a fake end-of-input iterator (e.g. NULL) if you can be certain that + // dereferencing end() is harmless. + + Iter i; + const char* decomp_ptr; + Iter end; + + friend class boost::iterator_core_access; + + void increment() { + if (*decomp_ptr) { + ++decomp_ptr; + } + if (!*decomp_ptr) { + ++i; + if (i==end || !*i) { + decomp_ptr = NULL; + } else { + decomp_ptr = to_ascii_letters(*i); + } + } + } + + bool equal(const ascii_letter_iterator& other) const { + return (i == other.i) && (decomp_ptr == other.decomp_ptr); + } + + char dereference() const { + if (!decomp_ptr) { + return 0; + } + char c = *decomp_ptr; + if (!c) { + return ' '; + } else { + return c; + } + } + +public: + ascii_letter_iterator(Iter i_, Iter end_): + i(i_), + decomp_ptr(NULL), + end(end_) + { + if (i!=end && *i) { + decomp_ptr = to_ascii_letters(*i); + } + } + + + Iter base() const { + return i; + } + +}; + + +}; + + +#endif + diff --git a/extras/anytermd/libpbe/include/charset/utf8.hh b/extras/anytermd/libpbe/include/charset/utf8.hh new file mode 100644 index 0000000000..1d33a3db21 --- /dev/null +++ b/extras/anytermd/libpbe/include/charset/utf8.hh @@ -0,0 +1,210 @@ +// utf8.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2007-2008 Philip Endecott + +// Distributed under the Boost Software License, Version 1.0: +// +// Permission is hereby granted, free of charge, to any person or organization +// obtaining a copy of the software and accompanying documentation covered by +// this license (the "Software") to use, reproduce, display, distribute, +// execute, and transmit the Software, and to prepare derivative works of the +// Software, and to permit third-parties to whom the Software is furnished to +// do so, all subject to the following: +// +// The copyright notices in the Software and this entire statement, including +// the above license grant, this restriction and the following disclaimer, +// must be included in all copies of the Software, in whole or in part, and +// all derivative works of the Software, unless such copies or derivative +// works are solely in the form of machine-executable object code generated by +// a source language processor. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT +// SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE +// FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, +// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER +// DEALINGS IN THE SOFTWARE. + +#ifndef libpbe_charset_utf8_hh +#define libpbe_charset_utf8_hh + +#include "char_t.hh" +#include "charset_t.hh" +#include "charset_traits.hh" + +#include "compiler_magic.hh" + + +// UTF-8 Encoding and Decoding +// --------------------------- +// +// This file provides functions that do the bit-shuffling needed for UTF-8 +// encoding and decoding in the form of a charset_traits specialisation. +// +// TODO there should be an error_policy template parameter to the decoding +// function so that the behaviour when invalid input is encountered can be +// defined. +// +// These functions perform significantly better than the alternaitves that +// I benchmarked: Ken Thompson's originals, glibc's mbsrtowcs etc. and +// glibc's iconv. Performance is somewhat sensitive to optimisation, however; +// the branch prediction hints that I've included make this less of a problem. + + +namespace pbe { + +template <> struct charset_traits { + + typedef char32_t char_t; + typedef char8_t unit_t; + struct state_t {}; + static const bool is_ascii_superset = true; + static const bool is_ascii_plus_c1_superset = true; + + struct InvalidUTF8 {}; // FIXME see above error_policy needed + +private: + static void check(bool condition) { + IF_UNLIKELY(!condition) throw InvalidUTF8(); + } + +public: + template + static char32_t decode(const_char8_ptr_t& p) { + char8_t b0 = *(p++); + IF_LIKELY((b0&0x80)==0) { + return b0; + } + char8_t b1 = *(p++); + check((b1&0xc0)==0x80); + IF_LIKELY((b0&0xe0)==0xc0) { + char32_t r = (b1&0x3f) | ((b0&0x1f)<<6); + check(r>=0x80); + return r; + } + char8_t b2 = *(p++); + check((b2&0xc0)==0x80); + IF_LIKELY((b0&0xf0)==0xe0) { + char32_t r = (b2&0x3f) | ((b1&0x3f)<<6) | ((b0&0x0f)<<12); + check(r>=0x800); + return r; + } + char8_t b3 = *(p++); + check((b3&0xc0)==0x80); + IF_LIKELY((b0&0xf8)==0xf0) { + char32_t r = (b3&0x3f) | ((b2&0x3f)<<6) | ((b1&0x3f)<<12) | ((b0&0x07)<<18); + check(r>=0x10000); + return r; + } + // The original definition of UTF-8 includes 5- and 6-byte encodings. + // But these correspond to values that are not valid Unicode characters, + // and they are not included in the more recent spec. Included here + // for interest: + // char8_t b4 = *(p++); + // check((b4&0xc0)==0x80); + // IF_LIKELY((b0&0xfc)==0xf8) { + // char32_t r = (b4&0x3f) | ((b3&0x3f)<<6) | ((b2&0x3f)<<12) | ((b1&0x3f)<<18) | ((b0&0x03)<<24); + // check(r>=0x200000); + // return r; + // } + // char8_t b5 = *(p++); + // check((b5&0xc0)==0x80); + // IF_LIKELY((b0&0xfe)==0xfc) { + // char32_t r = (b5&0x3f) | ((b4&0x3f)<<6) | ((b3&0x3f)<<12) | ((b2&0x3f)<<18) | ((b1&0x3f)<<24) | ((b0&0x01)<<30); + // check(r>=0x4000000); + // return r; + // } + check(false); + return 0; // not reached + } + + + template + static void encode(char8_ptr_t& p, char32_t c) { + IF_LIKELY(c<=0x7f) { + *(p++) = c; + } else { + IF_LIKELY(c<=0x7ff) { + *(p++) = 0xc0 | (c>>6); + } else { + IF_LIKELY(c<=0xffff) { + *(p++) = 0xe0 | (c>>12); + } else { + IF_LIKELY(c<=0x1fffff) { + *(p++) = 0xf0 | (c>>18); + } else { + // As above, disable 5- and 6-byte forms: + // IF_LIKELY(c<=0x3ffffff) { + // *(p++) = 0xf8 | (c>>24); + // } else { + // if (c&0x80000000) { + throw "can't represent this value in UTF8"; // needs error_policy + // } + // *(p++) = 0xfc | (c>>30); + // *(p++) = 0x80 | ((c>>24)&0x3f); + // } + // *(p++) = 0x80 | ((c>>18)&0x3f); + } + *(p++) = 0x80 | ((c>>12)&0x3f); + } + *(p++) = 0x80 | ((c>>6)&0x3f); + } + *(p++) = 0x80 | (c&0x3f); + } + } + + + // Skip forward and backward to the start of the next character. + // We know the length of a character from its first byte, so in principle we + // need only look at that to skip forward. But I guess that it's actually + // quicker to look at all bytes until a character-starting-byte is found, + // because the bit manipulation is simpler and less code is needed. + +private: + static bool char_start_byte(char8_t b) { + // All non-first bytes of a UTF8 character are 10xxxxxx. + return (b&0xc0) != 0x80; + } + +public: + template + static void skip_forward_char(char8_ptr_t& i) { + do { + ++i; + } while (!char_start_byte(*i)); // Maybe hint this? + } + + template + static void skip_backward_char(char8_ptr_t& i) { + do { + --i; + } while (!char_start_byte(*i)); // Maybe hint this? + } + + template + static int char_length(const_char8_ptr_t& p) { + char8_t b = *p; + if ((b&0x80)==0) return 1; + if ((b&0xe0)==0xc0) return 2; + if ((b&0xf0)==0xe0) return 3; + if ((b&0xf8)==0xf0) return 4; + // As above, disable 5- and 6-byte forms: + // if ((b&0xfc)==0xf8) return 5; + // if ((b&0xfe)==0xfc) return 6; + return 0; // not reached for valid input + } + + + static size_t max_characters(size_t n_units) { return n_units; } + static size_t typ_characters(size_t n_units) { return n_units; } + static size_t max_units(size_t n_characters) { return 4*n_characters; } + static size_t typ_units(size_t n_characters) { return 2*n_characters; } + + +}; + + +}; + +#endif diff --git a/extras/anytermd/libpbe/include/ci_string.hh b/extras/anytermd/libpbe/include/ci_string.hh new file mode 100644 index 0000000000..c8dc82e653 --- /dev/null +++ b/extras/anytermd/libpbe/include/ci_string.hh @@ -0,0 +1,68 @@ +// src/ci_string.hh +// This file is part of libpbe; see http://decimail.org +// (C) 2004 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +#ifndef libpbe_ci_string_hh +#define libpbe_ci_string_hh + +#include +#include +#include + +using namespace std; + + +// This code is from http://www.gotw.ca/gotw/029.htm +// It seems to be in the public domain (usenet posting) + +struct ci_char_traits : public char_traits { + static bool eq( char c1, char c2 ) { + return toupper(c1) == toupper(c2); + } + + static bool ne( char c1, char c2 ) { + return toupper(c1) != toupper(c2); + } + + static bool lt( char c1, char c2 ) { + return toupper(c1) < toupper(c2); + } + + static int compare( const char* s1, const char* s2, size_t n ) { + return strncasecmp( s1, s2, n ); + } + + static const char* find( const char* s, int n, char a ) { + while( n-- > 0 && toupper(*s) != toupper(a) ) { + ++s; + } + return s; + } +}; + + +typedef basic_string ci_string; + +// End of code from gowt.ca + + +string to_string(const ci_string& s); + +ci_string to_ci_string(const string& s); + + + +#endif diff --git a/extras/anytermd/libpbe/include/compiler_magic.hh b/extras/anytermd/libpbe/include/compiler_magic.hh new file mode 100644 index 0000000000..dbf7261ac6 --- /dev/null +++ b/extras/anytermd/libpbe/include/compiler_magic.hh @@ -0,0 +1,47 @@ +// libpbe/include/compiler_magic.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2007-2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +// Wrappers around various gcc thingies. + + +#ifndef compiler_magic_hh +#define compiler_magic_hh + +#ifdef __GNUC__ + +// Supress unused-argument warning: +#define PBE_UNUSED_ARG(a) a __attribute__((unused)) + +// Give status branch prediction hints: +#define IF_LIKELY(c) if(__builtin_expect(c,1)) +#define IF_UNLIKELY(c) if(__builtin_expect(c,0)) + +// Warn if function result is ignored: +#define PBE_WARN_RESULT_IGNORED __attribute__ ((warn_unused_result)) + +#else + +#define PBE_UNUSED_ARG(a) a +#define IF_LIKELY(c) if(c) +#define IF_UNLIKELY(c) if(c) +#define PBE_WARN_RESULT_IGNORED + +#endif + +#endif diff --git a/extras/anytermd/libpbe/include/const_string_facade.hh b/extras/anytermd/libpbe/include/const_string_facade.hh new file mode 100644 index 0000000000..c57a845f97 --- /dev/null +++ b/extras/anytermd/libpbe/include/const_string_facade.hh @@ -0,0 +1,355 @@ +// include/const_string_facade.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef pbe_const_string_facade_hh +#define pbe_const_string_facade_hh + +#include +#include +#include +#include + +#include + +// const_string_facade makes it simpler to implement a class with +// an interface similar to a const std::string. The implementation +// simply needs to provide access to pointers to the beginning and +// end of the raw data. +// If the raw data is null-terminated then c_str() will work. +// See ../examples/const_string_facade.cc for example usage. + + +namespace pbe { + + +template +inline size_t strlen(const CHAR* s) { return std::find(s,s-1,0) - s; } + +template<> +inline size_t strlen(const char* s) { return ::strlen(s); } + +template<> +inline size_t strlen(const unsigned char* s) { return ::strlen(reinterpret_cast(s)); } + +template<> +inline size_t strlen(const signed char* s) { return ::strlen(reinterpret_cast(s)); } + +template +inline bool contains(ITER b, ITER e, CHAR c) { return std::find(b,e,c)!=e; } + +template +inline bool not_contains(ITER b, ITER e, CHAR c) { return std::find(b,e,c)==e; } + +template +inline int strcmp(CHAR* abeg, CHAR* aend, CHAR* bbeg, CHAR* bend) { + CHAR* a = abeg; + CHAR* b = bbeg; + while (a *b) { + return 1; + } + ++a; + ++b; + } + if (a +class const_string_facade {}; + + +template +class const_string_facade { + + DERIVED& derived() { return *static_cast(this); } + const DERIVED& derived() const { return *static_cast(this); } + +public: + + typedef CHAR value_type; + typedef CHAR* pointer; + typedef CHAR& reference; + typedef const CHAR& const_reference; + typedef size_t size_type; + typedef ptrdiff_t difference_type; + + static const size_type npos = -1; + + typedef CHAR* iterator; + typedef const CHAR* const_iterator; + + typedef boost::reverse_iterator reverse_iterator; + typedef boost::reverse_iterator const_reverse_iterator; + + iterator begin() { return derived().get_begin(); } + iterator end() { return derived().get_end(); } + const_iterator begin() const { return derived().get_begin(); } + const_iterator end() const { return derived().get_end(); } + + reverse_iterator rbegin() { return reverse_iterator(derived().get_end()); } + reverse_iterator rend() { return reverse_iterator(derived().get_begin()); } + const_reverse_iterator rbegin() const { return const_reverse_iterator(derived().get_end()); } + const_reverse_iterator rend() const { return const_reverse_iterator(derived().get_begin()); } + + size_type size() const { return end() - begin(); } + size_type length() const { return end() - begin(); } + size_type max_size() const { return end() - begin(); } // ??? + size_type capacity() const { return end() - begin(); } // ??? + bool empty() const { return end() == begin(); } + + reference operator[](size_type n) { return *(begin()+n); } + const_reference operator[](size_type n) const { return *(begin()+n); } + + const CHAR* data() const { return begin(); } + + size_type copy(CHAR* buf, size_type n, size_type pos=0) const { + // Is n allowed to be larger than size() ? + return std::copy(begin()+pos, std::min(end(),begin()+pos+n), buf) - buf; + } + + template + size_type find(const STR& s, size_type pos=0) const { + const_iterator i = std::search(begin()+pos, end(), s.begin(), s.end()); + return (i==end()) ? npos : (i-begin()); + } + size_type find(const CHAR* s, size_type pos, size_type n) const { + const_iterator i = std::search(begin()+pos, end(), s, s+n); + return (i==end()) ? npos : (i-begin()); + } + size_type find(const CHAR* s, size_type pos=0) const { + size_t len = strlen(s); + return find(s,pos,len); + } + size_type find(CHAR c, size_type pos=0) const { + const_iterator i = std::find(begin()+pos, end(), c); + return (i==end()) ? npos : (i-begin()); + } + + template + size_type rfind(const STR& s, size_type pos=npos) const { + const_reverse_iterator start = rend() - std::min(pos,size()); + const_reverse_iterator i = std::search(start, rend(), s.rbegin(), s.rend()); + return (i==rend()) ? npos : (rend()-i-s.size()); + } + size_type rfind(const CHAR* s, size_type pos, size_type n) const { + const_reverse_iterator start = rend() - std::min(pos,size()); + const_reverse_iterator i = std::search(start, rend(), const_reverse_iterator(s+n), const_reverse_iterator(s)); + return (i==rend()) ? npos : (rend()-i-n); + } + size_type rfind(const CHAR* s, size_type pos=npos) const { + size_t len = strlen(s); + return rfind(s,pos,len); + } + size_type rfind(CHAR c, size_type pos=npos) const { + const_reverse_iterator start = rend() - std::min(pos,size()); + const_reverse_iterator i = std::find(start, rend(), c); + return (i==rend()) ? npos : (rend()-i-1); + } + + template + size_type find_first_of(const STR& s, size_type pos=0) const { + const_iterator i = std::find_if(begin()+pos, end(), boost::bind(&contains,s.begin(),s.end(),_1)); + return (i==end()) ? npos : (i-begin()); + } + size_type find_first_of(const CHAR* s, size_type pos, size_type n) const { + const_iterator i = std::find_if(begin()+pos, end(), boost::bind(&contains,s,s+n,_1)); + return (i==end()) ? npos : (i-begin()); + } + size_type find_first_of(const CHAR* s, size_type pos=0) const { + size_t len = strlen(s); + return find_first_of(s,pos,len); + } + size_type find_first_of(CHAR c, size_type pos=0) const { + return find(c,pos); + } + + template + size_type find_first_not_of(const STR& s, size_type pos=0) const { + const_iterator i = std::find_if(begin()+pos, end(), boost::bind(¬_contains,s.begin(),s.end(),_1)); + return (i==end()) ? npos : (i-begin()); + } + size_type find_first_not_of(const CHAR* s, size_type pos, size_type n) const { + const_iterator i = std::find_if(begin()+pos, end(), boost::bind(¬_contains,s,s+n,_1)); + return (i==end()) ? npos : (i-begin()); + } + size_type find_first_not_of(const CHAR* s, size_type pos=0) const { + size_t len = strlen(s); + return find_first_not_of(s,pos,len); + } + size_type find_first_not_of(CHAR c, size_type pos=0) const { + return find_first_not_of(&c, pos, 1); // FIXME should use find_if + } + + template + size_type find_last_of(const STR& s, size_type pos=npos) const { + const_reverse_iterator start = rend() - std::min(pos,size()); + const_reverse_iterator i = std::find_if(start, rend(), boost::bind(&contains,s.begin(),s.end(),_1)); + return (i==rend()) ? npos : (rend()-i-1); + } + size_type find_last_of(const CHAR* s, size_type pos, size_type n) const { + const_reverse_iterator start = rend() - std::min(pos,size()); + const_reverse_iterator i = std::find_if(start, rend(), boost::bind(&contains,s,s+n,_1)); + return (i==rend()) ? npos : (rend()-i-1); + } + size_type find_last_of(const CHAR* s, size_type pos=npos) const { + size_t len = strlen(s); + return find_last_of(s,pos,len); + } + size_type find_last_of(CHAR c, size_type pos=npos) const { + return rfind(c,pos); + } + + template + size_type find_last_not_of(const STR& s, size_type pos=npos) const { + const_reverse_iterator start = rend() - std::min(pos,size()); + const_reverse_iterator i = std::find_if(start, rend(), boost::bind(¬_contains,s.begin(),s.end(),_1)); + return (i==rend()) ? npos : (rend()-i-1); + } + size_type find_last_not_of(const CHAR* s, size_type pos, size_type n) const { + const_reverse_iterator start = rend() - std::min(pos,size()); + const_reverse_iterator i = std::find_if(start, rend(), boost::bind(¬_contains,s,s+n,_1)); + return (i==rend()) ? npos : (rend()-i-1); + } + size_type find_last_not_of(const CHAR* s, size_type pos=npos) const { + size_t len = strlen(s); + return find_last_not_of(s,pos,len); + } + size_type find_last_not_of(CHAR c, size_type pos=npos) const { + return find_last_not_of(&c, pos, 1); // FIXME should use find_if + } + + std::basic_string substr(size_type pos=0, size_type n=npos) const { + return std::basic_string(begin()+pos,begin()+std::min(n,size())); + } + + template + int compare(const STR& s) const { + return strcmp(begin(), end(), s.begin(), s.end()); + } + template + int compare(size_type pos, size_type n, const STR& s) const { + return strcmp(begin()+pos, begin()+pos+n, s.begin(), s.end()); + } + template + int compare(size_type pos, size_type n, const STR& s, size_type pos1, size_type n1) const { + return strcmp(begin()+pos, begin()+pos+n, s.begin()+pos1, s.begin()+pos1+n1); + } + int compare(const CHAR* s) const { + return strcmp(begin(), end, s, s+strlen(s)); + } + int compare(size_type pos, size_type n, const CHAR* s, size_type len=npos) const { + return strcmp(begin()+pos, begin()+pos+n, s, s+std::min(len,strlen(s))); // traits::length??? + } + +}; + + +#if 0 +template +class const_string_facade: public const_string_facade { +public: + const CHAR* c_str() const { return begin(); } +}; +#endif + + + +template +inline bool operator==(const const_string_facade& s1, + const const_string_facade& s2) { + return s1.compare(s2) == 0; +} + +template +inline bool operator==(const CHAR* s1, + const const_string_facade& s2) { + return s2.compare(s1) == 0; +} + +template +inline bool operator==(const const_string_facade& s1, + const CHAR* s2) { + return s1.compare(s2) == 0; +} + + +template +inline bool operator!=(const const_string_facade& s1, + const const_string_facade& s2) { + return s1.compare(s2) != 0; +} + +template +inline bool operator!=(const CHAR* s1, + const const_string_facade& s2) { + return s2.compare(s1) != 0; +} + +template +inline bool operator!=(const const_string_facade& s1, + const CHAR* s2) { + return s1.compare(s2) != 0; +} + + +template +inline bool operator<(const const_string_facade& s1, + const const_string_facade& s2) { + return s1.compare(s2) < 0; +} + +template +inline bool operator<(const CHAR* s1, + const const_string_facade& s2) { + return s2.compare(s1) < 0; +} + +template +inline bool operator<(const const_string_facade& s1, + const CHAR* s2) { + return s1.compare(s2) < 0; +} + + +template +inline std::basic_ostream& operator<<(std::basic_ostream& os, + const const_string_facade& s) { + std::ostream_iterator osi(os); + copy(s.begin(),s.end(),osi); // something about os.width() ??? + return os; +} + + +}; + +#endif + diff --git a/extras/anytermd/libpbe/include/create_dir.hh b/extras/anytermd/libpbe/include/create_dir.hh new file mode 100644 index 0000000000..735b9cddfa --- /dev/null +++ b/extras/anytermd/libpbe/include/create_dir.hh @@ -0,0 +1,56 @@ +// include/create_dir.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe +// (C) 2009 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +// Create directories. + + +#ifndef libpbe_create_dir_hh +#define libpbe_create_dir_hh + +#include "Exception.hh" + +#include +#include + +#include + +namespace pbe { + + +// Simple wrapper for mkdir(). Doesn't create parent directories; fails +// if the directory already exists or is a file. + +inline void create_dir(std::string path, mode_t mode = 0777) { + int r = ::mkdir(path.c_str(),mode); + if (r==-1) { + pbe::throw_ErrnoException("mkdir()"); + } +} + + +// Check for and create necessary parent directories. Succeeds if the +// directory already exists; fails if it is a file. + +extern void create_dir_with_parents(const std::string path, mode_t mode = 0777); + + +}; + +#endif + diff --git a/extras/anytermd/libpbe/include/csv.hh b/extras/anytermd/libpbe/include/csv.hh new file mode 100644 index 0000000000..bd4c7542bf --- /dev/null +++ b/extras/anytermd/libpbe/include/csv.hh @@ -0,0 +1,29 @@ +// csv.hh +// (C) 2008 Philip Endecott +// This file is part of libpbe. See http://svn.chezphil.org/libpbe/ +// This file is distributed under the terms of the Boost Software License v1.0. +// Please see http://www.boost.org/LICENSE_1_0.txt or the accompanying file BOOST_LICENSE. + +#ifndef pbe_csv_hh +#define pbe_csv_hh + +#include +#include + +namespace pbe { + +// Parse a line from a comma-separated-variables (CSV) file. +// The fields are used to populate a vector. +// +// Each field may be quoted with "" or not quoted. If it's quoted it may +// contain unescaped commas; if it's not quoted it can't. In either case +// it may contain escaped characters using \. +// +// The code makes some attempt to handle invalid input sanely, but +// it is not designed to be secure against malicious input. + +extern void parse_csv_line(std::string l, std::vector& v); + +}; + +#endif diff --git a/extras/anytermd/libpbe/include/endian.hh b/extras/anytermd/libpbe/include/endian.hh new file mode 100644 index 0000000000..5883842ae7 --- /dev/null +++ b/extras/anytermd/libpbe/include/endian.hh @@ -0,0 +1,78 @@ +// src/endian.hh +// This file is part of libpbe; see http://decimail.org +// (C) 2006 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_endian_hh +#define libpbe_endian_hh + +#include + +#ifdef __GLIBC__ +#include +#define PBE_BYTE_ORDER __BYTE_ORDER +#define PBE_BIG_ENDIAN __BIG_ENDIAN +#define PBE_LITTLE_ENDIAN __LITTLE_ENDIAN + +#elif defined(__FreeBSD__) +#include +#define PBE_BYTE_ORDER _BYTE_ORDER +#define PBE_BIG_ENDIAN _BIG_ENDIAN +#define PBE_LITTLE_ENDIAN _LITTLE_ENDIAN + +#endif + + +namespace pbe { + +inline uint32_t swap_end_32(uint32_t data) +{ + uint32_t r = data>>24; + r |= (data>>8)&0x0000ff00; + r |= (data<<8)&0x00ff0000; + r |= data<<24; + return r; +} + +inline uint64_t swap_end_64(uint64_t data) +{ + uint64_t a = swap_end_32(data>>32); + uint64_t b = swap_end_32(data&0xffffffff); + return a | b<<32; +} + + +inline uint64_t hton64(uint64_t i) { +#if PBE_BYTE_ORDER == PBE_BIG_ENDIAN + return i; +#else + return swap_end_64(i); +#endif +} + + +inline uint64_t ntoh64(uint64_t i) { +#if PBE_BYTE_ORDER == PBE_BIG_ENDIAN + return i; +#else + return swap_end_64(i); +#endif +} + + +}; + +#endif diff --git a/extras/anytermd/libpbe/include/ffs.hh b/extras/anytermd/libpbe/include/ffs.hh new file mode 100644 index 0000000000..1e57e16425 --- /dev/null +++ b/extras/anytermd/libpbe/include/ffs.hh @@ -0,0 +1,61 @@ +// include/ffs.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef pbe_ffs_hh +#define pbe_ffs_hh + +// ffs - Find First Set +// This is overloading of the libc ffs* functions for different int types. +// Find the least significant bit set in a value and return its index. +// Bits are numbered starting from 1. +// If no bits are set, return 0. + +#ifndef _GNU_SOURCE +#define _GNU_SOURCE +#include +#undef _GNU_SOURCE +#else +#include +#endif + +#include + + +namespace pbe { + +inline int ffs(::uint32_t i) { + return ::ffsl(i); +} + +inline int ffs(::int32_t i) { + return ::ffsl(i); +} + +inline int ffs(::uint64_t i) { + return ::ffsll(i); +} + +inline int ffs(::int64_t i) { + return ::ffsll(i); +} + +}; + + +#endif + diff --git a/extras/anytermd/libpbe/include/file_length.hh b/extras/anytermd/libpbe/include/file_length.hh new file mode 100644 index 0000000000..65d60f33fd --- /dev/null +++ b/extras/anytermd/libpbe/include/file_length.hh @@ -0,0 +1,52 @@ +// include/file_length.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef pbe_file_length_hh +#define pbe_file_length_hh + +#include "Exception.hh" + +#include + +#include +#include +#include + + +// Find the length of a file, given its pathname + + +namespace pbe { + + ::off_t file_length(const char* pathname) { + struct ::stat s; + int rc = ::stat(pathname,&s); + if (rc==-1) { + throw_ErrnoException("stat()"); + } + return s.st_size; + } + + ::off_t file_length(std::string pathname) { + return file_length(pathname.c_str()); + } + +}; + +#endif + diff --git a/extras/anytermd/libpbe/include/fixed.hh b/extras/anytermd/libpbe/include/fixed.hh new file mode 100644 index 0000000000..3c0f574798 --- /dev/null +++ b/extras/anytermd/libpbe/include/fixed.hh @@ -0,0 +1,397 @@ +// include/fixed.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// It is also offered as a candidate for inclusion in Boost, http://boost.org/ +// (C) 2007 Philip Endecott +// This file is distributed under the terms of the Boost Software License v. 1.0, +// see http://www.boost.org/LICENSE_1_0.txt. + +// Introduction +// ------------ + +// This file provides class templates 'fixed' and 'ufixed' which implement signed and +// unsigned fixed-point arithmetic. For background information about fixed-point +// arithmetic, see for example this Wikipedia article: +// http://en.wikipedia.org/wiki/Fixed-point_arithmetic. Note in particular that this +// is binary arithmetic, not binary coded decimal. +// +// The author's requirement for fixed-point arithmetic is based on these +// characteristics: +// +// - Comparing a 32-bit fixed-point value and a 32-bit single-precision +// floating-point value, the fixed-point value has about 8 bits more precision +// because it does not "waste" bits representing the exponent. In the case of +// latitude and longitude values this makes the difference between a precision of +// hundreds of metres and ones of metres. +// +// - Certain operations that directly operate on the bit patterns of values are much +// simpler to carry out on fixed-point values than on floating-point values. (For +// example, the bit interleaving required for Z-curves.) +// +// Other characteristics that are often beneficial include: +// +// - Computation is faster on systems that have no floating-point hardware, e.g. +// embedded systems. +// +// - The precision of the values is well-defined and predictable. +// +// +// Quick Start +// ----------- +// +// Here's a very quick illustration of how the class can be used: +// +// #include +// +// void test() { +// fixed<15,16> f1, f2; // Signed fixed-point values with 15 bits of integer part +// // and 16 bits of fractional part. +// f1 = 1.2345; // Conversion from floating point. +// f2 = f1 - 2; // Mixed arithmetic with integers. +// f2 = f1 / f2; // Arithmetic on fixed-point values. +// } +// +// +// Primary Objective +// ----------------- +// +// The primary objective of this implementation is zero performance overhead compared +// to using integer arithmetic with manually-inserted scaling. This is because +// performance is important in many applications of fixed-point arithmetic and it +// really isn't very hard to do ad-hoc fixed-point arithmetic using integers and +// shifts. Users will therefore not be attracted to this library unless it can offer +// this feature. +// +// +// Secondary Objectives +// -------------------- +// +// Secondary objectives are: +// +// - To behave in as many respects as possible in the same was as the built-in +// numeric types (integer and floating-point). The motivation for this is to give +// the user the behaviour that they are already familiar with from those types. +// +// +// Hardware Support +// ---------------- +// +// The objective of this implementation is to work efficiently with conventional +// processors' integer arithmetic instructions. It is worth noting, however, that +// some processors have hardware support for fixed-point operations. These include: +// +// [Hmm, no MMX doesn't seem to have any explicit fixed-point support] +// +// - DSP processors. +// - Intel's MMX instruction set (also implemented in other x86-compatible +// processors). +// - The PowerPC's AltiVec instruction set. +// - The ARM Piccolo instruction set. +// +// Support for any of these instruction sets would require either use of +// architecture-specific assembly language or a very smart compiler. These are +// considered outside the scope of the current work; however, it is worthwhile to see +// what these instruction sets offer in order to avoid unintentionally implementing +// something that is incompatible. +// +// For example, it seems that these instruction sets could more easily support +// fixed<16,15> than fixed<15,16>. It may be that the user would be happy with +// either of these formats. So we should consider offering a way to get the "best" +// format with at least a certain number of bits of integer and fractional parts for +// a particular instruction set. +// +// +// Related Work +// ------------ +// +// "Embedded C" (see ISO/IEC [draft] technical report 18037:2004) proposes fixed +// point types for C. It defines +// new keywords _Fract and _Accum which modify the built-in short, int and long +// types; _Fract makes the value entirely fractional while _Accum gives it 4 +// integer bits. +// +// It requires saturating arithmetic (controlled by an _Sat keyword or a #pragma), +// and does not require wrap-around behaviour. +// +// Fixed-point constants can be written using letter suffixes, e.g. 3.1415uhk. +// +// It looks very much as if this proposal is motivated by one particular category of +// applications (i.e. digital signal processing). The requirement for saturating +// arithmetic imposes a significant overhead on processors that do not have support +// for it, and the lack of support for values with more than 4 integer bits makes it +// useless for the author's latitude/longitude application. +// +// +// Getting the High part of a multiply result +// ------------------------------------------ +// +// MMX has "packed multiply high" instruction. +// +// +// Meaning of ++ and -- +// -------------------- +// +// It could be argued that the meaning of ++ and -- is unclear: do they add/subtract +// 1, or do they add/subtract the type's delta? (...there is an example of something +// where they do delta...). I note, however, that the built-in floating point types +// add and subtract 1, as does the proposed "Embedded C" fixed-point type. +// Therefore, ++ and -- are defined to do that for fixed and ufixed. +// +// +// Fixed Point Constants +// --------------------- +// +// It would be ideal if code such as +// +// const ufixed<2,14> pi = 3.1415926; +// +// could have no run-time overhead. To be investigated... +// +// +// Result Types +// ------------ +// +// +// Free Functions +// -------------- +// +// embedded C has... +// ...names are crytic due to absence of template/overloading in C. +// +// +// Overflow and Saturation +// ----------------------- +// +// Users of fixed-point arithmetic sometimes want to detect overflow, or for +// operations to saturate when overflow would occur. However, users of +// floating-point and integer arithmetic also sometimes want these features, but they +// are not provided by the built-in C/C++ types. The approach taken in this +// implementation is to follow the behaviour of the built-in types, i.e. to not +// handle overflow; values will wrap-around. +// +// Hardware overhead of saturation; MMX has saturating instructions. +// +// +// Division By Zero +// ---------------- +// +// compare with int and float... +// +// embedded C fixed is undefined +// +// +// Formatted input/output +// ---------------------- +// +// ... implemented by conversion to floating point? ... +// +// +// 'Math library' functions +// ------------------------ +// +// A set of functions similar to those provided by the standard library for integer +// and floating-point types is provided. The skeleton for these functions was +// provided by ...... +// +// In the case of ....., tables of constants are required. Different tables are +// required for each fixed point type. By default, the approach taken is to compute +// these tables when the functions are first used [note that the use of static +// variables may not be thread safe]. This could impose an unacceptable delay in +// some applications; in this case, the preprocessor symbol ... should be defined, +// and the included program ... should be run once for each type to generate static +// tables. Simply include the output of this program in your build. +// +// +// bigint +// ------ +// +// +// Benchmarks +// ---------- +// +// The following results are based on the benchmark programs in the ... directory. +// There are: +// +// - sort: generates a std::vector of xxxx pseudo-random values and sorts them +// using std::sort. This measures comparison performance, which should be of the +// same order as addition and subtraction. +// +// - filt: generates a vector of pseudo-random values and filters them using an FIR +// (finite impulse response) filter. The filter coefficients are also generated +// pseudo-randomly. This measures primarily multiply performance. +// +// In each case the performance is measured for fixed-point, floating-point and +// integer types with manually-inserted shifts. +// +// The following platforms have been tested: +// +// - NSLU2. (See http://nslu2-linux.org/). This system has a 266 MHz Intel IXP420 +// XScale ARM processor, which has no floating point hardware. The compiler used was +// g++ verson 4.1.2. +// +// - The author's desktop computer, which as a 1 GHz VIA C3 (x86-compatible) +// processor. The compiler used was also g++ version 4.1.2. +// + + + +// Note to Boost people: I will change all the 'libpbe' identifiers to 'boost' +// when this is formally proposed for review. + +#ifndef libpbe_fixed_hh +#define libpbe_fixed_hh + +#include // for ldexp* +#include "./integer.hpp" // Copy of boost::integer to which I've added 64-bit support. + // I expect this functionality to be added to the official + // version before this code is reviewed. +#include +#include + + +namespace pbe { + + +// This helper function exists mainly to avoid warnings about negative shifts. + +template +T shift(T val) { + if (amt>0) { + unsigned int u_amt = amt; + return val>>u_amt; + } else { + unsigned int u_amt = -amt; + return val<::least> +// WB = Whole Bits +// FB = Fraction Bits +// VAL_T = type to use for implementation +class fixed: + + boost::totally_ordered >, + boost::additive >, + boost::bitwise >, + boost::unit_steppable > +{ + +// Question: do we want to expose the implementation type and value? +// In my application I have found it useful to do so. +//private: +public: + typedef VAL_T val_t; + val_t val; + +public: + fixed() {} + fixed(const fixed& x): val(x.val) {} + + template + fixed(const fixed& x): + val(shift(x.val)) + //(X_FB>(X_FB-FB))) + {} + + fixed(int x): val(x<(ldexpf(x,FB))) {} + fixed(double x): val(static_cast(ldexp (x,FB))) {} + + operator float() const { return ldexpf(static_cast(val),-FB); } + operator double() const { return ldexp (static_cast(val),-FB); } + + + bool operator<(const fixed& x) const { return val + fixed& operator*=(const fixed& x) { (*this) = (*this) * x; return *this; } + + template + fixed& operator/=(const fixed& x) { (*this) = (*this) / x; return *this; } + + fixed& operator*=(float x) { (*this) = static_cast(*this) * x; return *this; } + + fixed& operator/=(float x) { (*this) = static_cast(*this) / x; return *this; } +}; + + +// I don't know how to choose the VAL_T for the results of these operations. +// I.e. if the operands use safe_int then the result should also use safe_int. + +template +pbe::fixed::value, boost::static_unsigned_max::value> +operator+(pbe::fixed x, pbe::fixed y) { + typedef pbe::fixed::value, boost::static_unsigned_max::value> result_t; + result_t x_(x); + result_t y_(y); + return x_ + y_; +} + + +template +pbe::fixed::value, boost::static_unsigned_max::value> +operator-(pbe::fixed x, pbe::fixed y) { + typedef pbe::fixed::value, boost::static_unsigned_max::value> result_t; + result_t x_(x); + result_t y_(y); + return x_ - y_; +} + + +template +pbe::fixed +operator*(pbe::fixed x, pbe::fixed y) { + typedef pbe::fixed result_t; + result_t r; + r.val = static_cast(x.val) * y.val; + return r; +} + + +template +pbe::fixed +operator/(pbe::fixed x, pbe::fixed y) { + pbe::fixed r; + r.val = x.val; + r.val <<= (YWB+YFB); + r.val /= y.val; + return r; +} + + +template +float operator*(pbe::fixed x, float y) { + // fixed * float is done using floating-point. Right choice? + return static_cast(x) * y; +} + +template +float operator/(pbe::fixed x, float y) { + // Ditto. + return static_cast(x) / y; +} + + + +// Also planned: an unsigned version. + + +}; + + +#endif + diff --git a/extras/anytermd/libpbe/include/format.hh b/extras/anytermd/libpbe/include/format.hh new file mode 100644 index 0000000000..cfa5c027df --- /dev/null +++ b/extras/anytermd/libpbe/include/format.hh @@ -0,0 +1,38 @@ +// include/format.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2009 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_format_hh +#define libpbe_format_hh + +#include + + +// pbe::format is an sprintf-like function that returns a std::string. + + +namespace pbe { + + +extern std::string format(const char* fmt,...) __attribute__ ((format (printf, 1, 2))); + + +}; + + +#endif + diff --git a/extras/anytermd/libpbe/include/geometry.hh b/extras/anytermd/libpbe/include/geometry.hh new file mode 100644 index 0000000000..99b1ab1a42 --- /dev/null +++ b/extras/anytermd/libpbe/include/geometry.hh @@ -0,0 +1,100 @@ +// include/geometry.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_geometry_hh +#define libpbe_geometry_hh + +#include + + +namespace pbe { + +const float PI = 3.1415927F; +const double PId = 3.1415927; +const float earth_radius = 6372795.0F; + +template +inline T sqr(T x) { return x*x; } + +inline float deg2rad(float a) { return PI/180.0F * a; } +inline float rad2deg(float a) { return 180.0F/PI * a; } + +inline float degsin(float a) { return sin(deg2rad(a)); } +inline float degcos(float a) { return cos(deg2rad(a)); } +inline float degtan(float a) { return tan(deg2rad(a)); } + +inline float degasin(float x) { return rad2deg(asin(x)); } +inline float degacos(float x) { return rad2deg(acos(x)); } +inline float degatan(float x) { return rad2deg(atan(x)); } +inline float degatan2(float y, float x) { return rad2deg(atan2(y,x)); } + +inline double deg2rad(double a) { return PId/180.0 * a; } +inline double rad2deg(double a) { return 180.0/PId * a; } + +inline double degsin(double a) { return sin(deg2rad(a)); } +inline double degcos(double a) { return cos(deg2rad(a)); } +inline double degtan(double a) { return tan(deg2rad(a)); } + +inline double degasin(double x) { return rad2deg(asin(x)); } +inline double degacos(double x) { return rad2deg(acos(x)); } +inline double degatan(double x) { return rad2deg(atan(x)); } +inline double degatan2(double y, double x) { return rad2deg(atan2(y,x)); } + +inline float normalise_unsigned_angle(float a) { return a>0 ? fmod(a,360.0F) : 360.0F+fmod(a,360.0F); } +inline float normalise_signed_angle(float a) { return normalise_unsigned_angle(a+180.0F)-180.0F; } + + +struct position { + float lng; + float lat; + float alt; + + position() {} + position(float lng_, float lat_, float alt_): + lng(lng_), lat(lat_), alt(alt_) + {} +}; + +float distance(const position& a, const position& b); // Proper great circle calculation +float bearing(const position& a, const position& b); +float elevation(const position& a, const position& b); + +inline float cheap_distance(const position& a, const position& b) { + return earth_radius / 360.0F * 2*PI + * sqrt( sqr(a.lat-b.lat) + sqr((a.lng-b.lng)*degcos(a.lat)) ); +} + +inline float cheap_bearing(const position& a, const position& b) { + float d = degatan2( (b.lng-a.lng) * degcos(a.lat), + b.lat-a.lat ); + return normalise_unsigned_angle(d); +} + +inline float elevation(const position& a, const position& b) { + return degatan2(b.alt-a.alt,cheap_distance(a,b)); +} + +inline float rectilinear_elevation(const position& a, const position& b) { + return rad2deg((b.alt-a.alt) / cheap_distance(a,b)); +} + +}; + + +#endif + diff --git a/extras/anytermd/libpbe/include/inc_init_array.hh b/extras/anytermd/libpbe/include/inc_init_array.hh new file mode 100644 index 0000000000..ae404fc725 --- /dev/null +++ b/extras/anytermd/libpbe/include/inc_init_array.hh @@ -0,0 +1,240 @@ +// include/inc_init_array.hh +// This file is part of libpbe; see http://decimail.org +// (C) 2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +// This work is based on Boost.Array, which is: +// (C) Copyright Nicolai M. Josuttis 2001. +// Distributed under the Boost Software License, Version 1.0. (See +// accompanying file LICENSE_1_0.txt or copy at +// http://www.boost.org/LICENSE_1_0.txt) + + +#ifndef libpbe_inc_init_array_hh +#define libpbe_inc_init_array_hh + +// pbe::inc_init_array is like boost::array, except that T must have +// a constructor that takes an integer, and when inc_init_array is +// constructed it will make N instances of T passing values 0 to N-1 to T's +// constructor. When given an extra template parameter M it will make N +// instances passing values M to M+N-1 + +// You can also pass an additional, constant, arbitrary-typed value to the elements' +// constructors by passing it as a final parameter to the array's constructor, i.e. +// if T has a constructor T(int,X) then pbe::inc_init_array has a +// constructor inc_init_array(X). + +#include + +#include +#include +#include + + +namespace pbe { + +template +class inc_init_array { + + char mem[sizeof(T)*N]; // worry about alignment + +public: + + // Unlike Boost.Array, elems is a function: + T* elems() { return reinterpret_cast(&mem); } + const T* elems() const { return reinterpret_cast(&mem); } + + inc_init_array() { + int i; + try { + for (i=0; i0; --i) { + (elems()+i) -> ~T(); + } + throw; + } + } + + inc_init_array(X x) { + int i; + try { + for (i=0; i0; --i) { + (elems()+i) -> ~T(); + } + throw; + } + } + + ~inc_init_array() { + for (int i=N-1; i>=0; --i) { + (elems()+i) -> ~T(); + } + } + + // type definitions + typedef T value_type; + typedef T* iterator; + typedef const T* const_iterator; + typedef T& reference; + typedef const T& const_reference; + typedef std::size_t size_type; + typedef std::ptrdiff_t difference_type; + + // iterator support + iterator begin() { return elems(); } + const_iterator begin() const { return elems(); } + iterator end() { return elems()+N; } + const_iterator end() const { return elems()+N; } + + // reverse iterator support + typedef std::reverse_iterator reverse_iterator; + typedef std::reverse_iterator const_reverse_iterator; + + reverse_iterator rbegin() { return reverse_iterator(end()); } + const_reverse_iterator rbegin() const { + return const_reverse_iterator(end()); + } + reverse_iterator rend() { return reverse_iterator(begin()); } + const_reverse_iterator rend() const { + return const_reverse_iterator(begin()); + } + + // operator[] + reference operator[](size_type i) + { + BOOST_ASSERT( i < N && "out of range" ); + return elems()[i]; + } + + const_reference operator[](size_type i) const + { + BOOST_ASSERT( i < N && "out of range" ); + return elems()[i]; + } + + // at() with range check + reference at(size_type i) { rangecheck(i); return elems()[i]; } + const_reference at(size_type i) const { rangecheck(i); return elems()[i]; } + + // front() and back() + reference front() + { + return elems()[0]; + } + + const_reference front() const + { + return elems()[0]; + } + + reference back() + { + return elems()[N-1]; + } + + const_reference back() const + { + return elems()[N-1]; + } + + // size is constant + static size_type size() { return N; } + static bool empty() { return false; } + static size_type max_size() { return N; } + enum { static_size = N }; + + // swap (note: linear complexity) + void swap (inc_init_array& y) { + std::swap_ranges(begin(),end(),y.begin()); + } + + // direct access to data (read-only) + const T* data() const { return elems(); } + + // use array as C array (direct read/write access to data) + T* c_array() { return elems(); } + + // assignment with type conversion + template + inc_init_array& operator= (const inc_init_array& rhs) { + std::copy(rhs.begin(),rhs.end(), begin()); + return *this; + } + + // assign one value to all elements + void assign (const T& value) + { + std::fill_n(begin(),size(),value); + } + + // check range (may be private because it is static) + static void rangecheck (size_type i) { + if (i >= size()) { + throw std::range_error("inc_init_array<>: index out of range"); + } + } + +}; + + +// comparisons +template +bool operator== (const inc_init_array& x, const inc_init_array& y) { + return std::equal(x.begin(), x.end(), y.begin()); +} +template +bool operator< (const inc_init_array& x, const inc_init_array& y) { + return std::lexicographical_compare(x.begin(),x.end(),y.begin(),y.end()); +} +template +bool operator!= (const inc_init_array& x, const inc_init_array& y) { + return !(x==y); +} +template +bool operator> (const inc_init_array& x, const inc_init_array& y) { + return y +bool operator<= (const inc_init_array& x, const inc_init_array& y) { + return !(y +bool operator>= (const inc_init_array& x, const inc_init_array& y) { + return !(x +inline void swap (inc_init_array& x, inc_init_array& y) { + x.swap(y); +} + + +}; + + +#endif diff --git a/extras/anytermd/libpbe/include/inc_init_vector.hh b/extras/anytermd/libpbe/include/inc_init_vector.hh new file mode 100644 index 0000000000..95f404dfb5 --- /dev/null +++ b/extras/anytermd/libpbe/include/inc_init_vector.hh @@ -0,0 +1,51 @@ +// include/inc_init_vector.hh +// This file is part of libpbe; see http://decimail.org +// (C) 2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_inc_init_vector_hh +#define libpbe_inc_init_vector_hh + + +// pbe::inc_init_vector is like std::vector, except that T must have +// a constructor that takes an integer, and when inc_init_vector is +// constructed with a parameter n it will make n instances of T, passing +// values 0 to n-1 to T's constructor, or when constructed with parameters +// n and m it will make n instances passing values m to m+n-1 + +#include + + +namespace pbe { + +template +class inc_init_vector: public std::vector { + +public: + inc_init_vector() {} + inc_init_vector(const inc_init_vector& other): std::vector(other) {} + inc_init_vector(int n, int m=0) { + for (int i=m; i is like boost::array, except that it provides +// a constructor that takes a T and copy-constructs each of its elements +// using it. + +#include + +#include + + +namespace pbe { + +template +class init_array { + + char mem[sizeof(T)*N]; // worry about alignment + +public: + + // Unlike Boost.Array, elems is a function: + T* elems() { return reinterpret_cast(&mem); } + const T* elems() const { return reinterpret_cast(&mem); } + + init_array() { + int i; + try { + for (i=0; i0; --i) { + (elems()+i) -> ~T(); + } + throw; + } + } + + init_array(T t) { + int i; + try { + for (i=0; i0; --i) { + (elems()+i) -> ~T(); + } + throw; + } + } + + ~init_array() { + for (int i=N-1; i>=0; --i) { + (elems()+i) -> ~T(); + } + } + + // type definitions + typedef T value_type; + typedef T* iterator; + typedef const T* const_iterator; + typedef T& reference; + typedef const T& const_reference; + typedef std::size_t size_type; + typedef std::ptrdiff_t difference_type; + + // iterator support + iterator begin() { return elems(); } + const_iterator begin() const { return elems(); } + iterator end() { return elems()+N; } + const_iterator end() const { return elems()+N; } + + // reverse iterator support + typedef std::reverse_iterator reverse_iterator; + typedef std::reverse_iterator const_reverse_iterator; + + reverse_iterator rbegin() { return reverse_iterator(end()); } + const_reverse_iterator rbegin() const { + return const_reverse_iterator(end()); + } + reverse_iterator rend() { return reverse_iterator(begin()); } + const_reverse_iterator rend() const { + return const_reverse_iterator(begin()); + } + + // operator[] + reference operator[](size_type i) + { + BOOST_ASSERT( i < N && "out of range" ); + return elems()[i]; + } + + const_reference operator[](size_type i) const + { + BOOST_ASSERT( i < N && "out of range" ); + return elems()[i]; + } + + // at() with range check + reference at(size_type i) { rangecheck(i); return elems()[i]; } + const_reference at(size_type i) const { rangecheck(i); return elems()[i]; } + + // front() and back() + reference front() + { + return elems()[0]; + } + + const_reference front() const + { + return elems()[0]; + } + + reference back() + { + return elems()[N-1]; + } + + const_reference back() const + { + return elems()[N-1]; + } + + // size is constant + static size_type size() { return N; } + static bool empty() { return false; } + static size_type max_size() { return N; } + enum { static_size = N }; + + // swap (note: linear complexity) + void swap (init_array& y) { + std::swap_ranges(begin(),end(),y.begin()); + } + + // direct access to data (read-only) + const T* data() const { return elems(); } + + // use array as C array (direct read/write access to data) + T* c_array() { return elems(); } + + // assignment with type conversion + template + init_array& operator= (const init_array& rhs) { + std::copy(rhs.begin(),rhs.end(), begin()); + return *this; + } + + // assign one value to all elements + void assign (const T& value) + { + std::fill_n(begin(),size(),value); + } + + // check range (may be private because it is static) + static void rangecheck (size_type i) { + if (i >= size()) { + throw std::range_error("init_array<>: index out of range"); + } + } + +}; + + +// comparisons +template +bool operator== (const init_array& x, const init_array& y) { + return std::equal(x.begin(), x.end(), y.begin()); +} +template +bool operator< (const init_array& x, const init_array& y) { + return std::lexicographical_compare(x.begin(),x.end(),y.begin(),y.end()); +} +template +bool operator!= (const init_array& x, const init_array& y) { + return !(x==y); +} +template +bool operator> (const init_array& x, const init_array& y) { + return y +bool operator<= (const init_array& x, const init_array& y) { + return !(y +bool operator>= (const init_array& x, const init_array& y) { + return !(x +inline void swap (init_array& x, init_array& y) { + x.swap(y); +} + + +}; + + +#endif diff --git a/extras/anytermd/libpbe/include/integer.hpp b/extras/anytermd/libpbe/include/integer.hpp new file mode 100644 index 0000000000..dae64f2c94 --- /dev/null +++ b/extras/anytermd/libpbe/include/integer.hpp @@ -0,0 +1,136 @@ +// boost integer.hpp header file -------------------------------------------// + +// Copyright Beman Dawes and Daryle Walker 1999. Distributed under the Boost +// Software License, Version 1.0. (See accompanying file +// LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) + +// See http://www.boost.org/libs/integer for documentation. + +// Hacked by Phil Endecott to add 64 bits + +// Revision History +// 22 Sep 01 Added value-based integer templates. (Daryle Walker) +// 01 Apr 01 Modified to use new header. (John Maddock) +// 30 Jul 00 Add typename syntax fix (Jens Maurer) +// 28 Aug 99 Initial version + +#ifndef BOOST_INTEGER_HPP +#define BOOST_INTEGER_HPP + +#include "./integer_fwd.hpp" // self include + +#include // for boost::integer_traits +#include // for std::numeric_limits +#include + +namespace boost +{ + + // Helper templates ------------------------------------------------------// + + // fast integers from least integers + // int_fast_t<> works correctly for unsigned too, in spite of the name. + template< typename LeastInt > + struct int_fast_t { typedef LeastInt fast; }; // imps may specialize + + // convert category to type + template< int Category > struct int_least_helper {}; // default is empty + + // specializatons: 1=int64_t 2=long, 3=int, 4=short, 5=signed char, + // 7=uint64_t 8=unsigned long, 9=unsigned int, 10=unsigned short, 11=unsigned char + // no specializations for 0 and 6: requests for a type > 64 bits are in error + template<> struct int_least_helper<1> { typedef int64_t least; }; + template<> struct int_least_helper<2> { typedef long least; }; + template<> struct int_least_helper<3> { typedef int least; }; + template<> struct int_least_helper<4> { typedef short least; }; + template<> struct int_least_helper<5> { typedef signed char least; }; + template<> struct int_least_helper<7> { typedef uint64_t least; }; + template<> struct int_least_helper<8> { typedef unsigned long least; }; + template<> struct int_least_helper<9> { typedef unsigned int least; }; + template<> struct int_least_helper<10>{ typedef unsigned short least; }; + template<> struct int_least_helper<11>{ typedef unsigned char least; }; + + // integer templates specifying number of bits ---------------------------// + + // signed + template< int Bits > // bits (including sign) required + struct int_t + { + typedef typename int_least_helper + < + (Bits-1 <= std::numeric_limits::digits) + + (Bits-1 <= std::numeric_limits::digits) + + (Bits-1 <= std::numeric_limits::digits) + + (Bits-1 <= std::numeric_limits::digits) + + (Bits-1 <= std::numeric_limits::digits) + >::least least; + typedef typename int_fast_t::fast fast; + }; + + // unsigned + template< int Bits > // bits required + struct uint_t + { + typedef typename int_least_helper + < + 6 + + (Bits <= std::numeric_limits::digits) + + (Bits <= std::numeric_limits::digits) + + (Bits <= std::numeric_limits::digits) + + (Bits <= std::numeric_limits::digits) + >::least least; + typedef typename int_fast_t::fast fast; + // int_fast_t<> works correctly for unsigned too, in spite of the name. + }; + + // integer templates specifying extreme value ----------------------------// + + // signed + template< int64_t MaxValue > // maximum value to require support + struct int_max_value_t + { + typedef typename int_least_helper + < + (MaxValue <= integer_traits::const_max) + + (MaxValue <= integer_traits::const_max) + + (MaxValue <= integer_traits::const_max) + + (MaxValue <= integer_traits::const_max) + + (MaxValue <= integer_traits::const_max) + >::least least; + typedef typename int_fast_t::fast fast; + }; + + template< int64_t MinValue > // minimum value to require support + struct int_min_value_t + { + typedef typename int_least_helper + < + (MinValue >= integer_traits::const_min) + + (MinValue >= integer_traits::const_min) + + (MinValue >= integer_traits::const_min) + + (MinValue >= integer_traits::const_min) + + (MinValue >= integer_traits::const_min) + >::least least; + typedef typename int_fast_t::fast fast; + }; + + // unsigned + template< uint64_t Value > // maximum value to require support + struct uint_value_t + { + typedef typename int_least_helper + < + 6 + + (Value <= integer_traits::const_max) + + (Value <= integer_traits::const_max) + + (Value <= integer_traits::const_max) + + (Value <= integer_traits::const_max) + + (Value <= integer_traits::const_max) + >::least least; + typedef typename int_fast_t::fast fast; + }; + + +} // namespace boost + +#endif // BOOST_INTEGER_HPP diff --git a/extras/anytermd/libpbe/include/integer_fwd.hpp b/extras/anytermd/libpbe/include/integer_fwd.hpp new file mode 100644 index 0000000000..c37d141bc7 --- /dev/null +++ b/extras/anytermd/libpbe/include/integer_fwd.hpp @@ -0,0 +1,153 @@ +// Boost integer_fwd.hpp header file ---------------------------------------// + +// (C) Copyright Dave Abrahams and Daryle Walker 2001. Distributed under the Boost +// Software License, Version 1.0. (See accompanying file +// LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) + +// See http://www.boost.org/libs/integer for documentation. + +#ifndef BOOST_INTEGER_FWD_HPP +#define BOOST_INTEGER_FWD_HPP + +#include // for UCHAR_MAX, etc. +#include // for std::size_t +#include + +#include // for BOOST_NO_INTRINSIC_WCHAR_T +#include // for std::numeric_limits + + +namespace boost +{ + + +// From ------------------------------------------------// + +// Only has typedefs or using statements, with #conditionals + + +// From -----------------------------------------// + +template < class T > + class integer_traits; + +template < > + class integer_traits< bool >; + +template < > + class integer_traits< char >; + +template < > + class integer_traits< signed char >; + +template < > + class integer_traits< unsigned char >; + +#ifndef BOOST_NO_INTRINSIC_WCHAR_T +template < > + class integer_traits< wchar_t >; +#endif + +template < > + class integer_traits< short >; + +template < > + class integer_traits< unsigned short >; + +template < > + class integer_traits< int >; + +template < > + class integer_traits< unsigned int >; + +template < > + class integer_traits< long >; + +template < > + class integer_traits< unsigned long >; + +#ifdef ULLONG_MAX +template < > + class integer_traits< ::boost::long_long_type>; + +template < > + class integer_traits< ::boost::ulong_long_type >; +#endif + + +// From ------------------------------------------------// + +template < typename LeastInt > + struct int_fast_t; + +template< int Bits > + struct int_t; + +template< int Bits > + struct uint_t; + +template< int64_t MaxValue > + struct int_max_value_t; + +template< int64_t MinValue > + struct int_min_value_t; + +template< uint64_t Value > + struct uint_value_t; + + +// From -----------------------------------// + +template < std::size_t Bit > + struct high_bit_mask_t; + +template < std::size_t Bits > + struct low_bits_mask_t; + +template < > + struct low_bits_mask_t< ::std::numeric_limits::digits >; + +#if USHRT_MAX > UCHAR_MAX +template < > + struct low_bits_mask_t< ::std::numeric_limits::digits >; +#endif + +#if UINT_MAX > USHRT_MAX +template < > + struct low_bits_mask_t< ::std::numeric_limits::digits >; +#endif + +#if ULONG_MAX > UINT_MAX +template < > + struct low_bits_mask_t< ::std::numeric_limits::digits >; +#endif + + +// From ------------------------------------// + +template < unsigned long Value > + struct static_log2; + +template < > + struct static_log2< 0ul >; + + +// From ---------------------------------// + +template < long Value1, long Value2 > + struct static_signed_min; + +template < long Value1, long Value2 > + struct static_signed_max; + +template < unsigned long Value1, unsigned long Value2 > + struct static_unsigned_min; + +template < unsigned long Value1, unsigned long Value2 > + struct static_unsigned_max; + + +} // namespace boost + + +#endif // BOOST_INTEGER_FWD_HPP diff --git a/extras/anytermd/libpbe/include/ip.hh b/extras/anytermd/libpbe/include/ip.hh new file mode 100644 index 0000000000..48587edfb5 --- /dev/null +++ b/extras/anytermd/libpbe/include/ip.hh @@ -0,0 +1,60 @@ +// src/ip.hh +// This file is part of libpbe; see http://decimail.org +// (C) 2004-2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_ip_hh +#define libpbe_ip_hh + +#include "Exception.hh" + +#include +#include + +#include +#include + + +namespace pbe { + + +std::ostream& operator<<(std::ostream& s, struct in_addr ip); + + +class HostLookupError: public Exception +{ +private: + std::string hostname; + int error_code; +public: + HostLookupError(std::string h, int e): hostname(h), error_code(e) {} + void report(std::ostream& s) const; +}; + + +struct in_addr get_ip_address(std::string hostname); + +std::string get_own_hostname(void); + +int tcp_client_connect(std::string server_name, int port); + +int local_client_connect(std::string address); + + +}; + + +#endif diff --git a/extras/anytermd/libpbe/include/map_iterators.hh b/extras/anytermd/libpbe/include/map_iterators.hh new file mode 100644 index 0000000000..118fe9747c --- /dev/null +++ b/extras/anytermd/libpbe/include/map_iterators.hh @@ -0,0 +1,69 @@ +// include/map_iterators.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_map_iterators_hh +#define libpbe_map_iterators_hh + +// Provides map_key_iterator and map_value_iterator, which adapt a +// std::map::iterator to iterate over only the keys or values respectively. + +#include +#include + + +namespace pbe { + +template +struct map_key_iterator: + public boost::transform_iterator< __gnu_cxx::select1st, + iter_t > +{ + map_key_iterator() {} + map_key_iterator(iter_t i): + boost::transform_iterator< __gnu_cxx::select1st, iter_t >(i) + {} +}; + +template +static map_key_iterator make_map_key_iterator(const iter_t& i) { + return map_key_iterator(i); +} + + + +template +struct map_value_iterator: + public boost::transform_iterator< __gnu_cxx::select2nd, + iter_t > +{ + map_value_iterator() {} + map_value_iterator(iter_t i): + boost::transform_iterator< __gnu_cxx::select2nd, iter_t >(i) + {} +}; + +template +static map_value_iterator make_map_value_iterator(const iter_t& i) { + return map_value_iterator(i); +} + + +}; + +#endif + diff --git a/extras/anytermd/libpbe/include/markup.hh b/extras/anytermd/libpbe/include/markup.hh new file mode 100644 index 0000000000..adfc37042e --- /dev/null +++ b/extras/anytermd/libpbe/include/markup.hh @@ -0,0 +1,28 @@ +// src/markup.hh +// This file is part of libpbe; see http://decimail.org +// (C) 2004 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +#ifndef libpbe_markup_hh +#define libpbe_markup_hh + +#include +using namespace std; + + +void markup_uris_emails(string& s); + + +#endif diff --git a/extras/anytermd/libpbe/include/missing_syscalls.hh b/extras/anytermd/libpbe/include/missing_syscalls.hh new file mode 100644 index 0000000000..3e3507db8c --- /dev/null +++ b/extras/anytermd/libpbe/include/missing_syscalls.hh @@ -0,0 +1,78 @@ +// include/missing_syscalls.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef pbe_missing_syscalls_hh +#define pbe_missing_syscalls_hh + +#include +#include +#include +#include + + +#ifdef __NR_futex +inline int futex(int *uaddr, int op, int val, const struct timespec *timeout, int *uaddr2, int val3) +{ + return syscall(__NR_futex,uaddr,op,val,timeout,uaddr2,val3); +} +#endif + +#ifdef __NR_gettid +inline int gettid() +{ + return syscall(__NR_gettid); +} +#endif + +#ifdef __NR_tkill +inline int tkill(int tid, int sig) +{ + return syscall(__NR_tkill,tid,sig); +} +#endif + +#ifdef __NR_delete_module +inline int delete_module(const char* name) +{ + return syscall(__NR_delete_module,name); +} +#endif + + +// These are missing from my glibc, but they may be present in newer versions: + +#ifdef __NR_inotify_init +#define PBE_HAVE_INOTIFY_INIT 1 +inline int inotify_init(void) +{ + return syscall(__NR_inotify_init); +} +#endif + +#ifdef __NR_inotify_add_watch +#define PBE_HAVE_INOTIFY_ADD_WATCH +inline int inotify_add_watch(int fd, const char* pathname, uint32_t mask) +{ + return syscall(__NR_inotify_add_watch,fd,pathname,mask); +} +#endif + + + +#endif + diff --git a/extras/anytermd/libpbe/include/mustfind.hh b/extras/anytermd/libpbe/include/mustfind.hh new file mode 100644 index 0000000000..b42979651d --- /dev/null +++ b/extras/anytermd/libpbe/include/mustfind.hh @@ -0,0 +1,86 @@ +// include/mustfind.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_mustfind_hh +#define libpbe_mustfind_hh + + +// pbe::mustfind is an an algorithm like std::find except that it +// throws an exception, pbe::notfound, if it fails to find a match. +// So calling code can rely on the returned iterator pointing to a +// valid element. + +// pbe::mustfind_if does the same thing for std::find_if and pbe::mustsearch +// does the same thing for std::search. + + +#include + + +namespace pbe { + + +struct notfound {}; + + +template +Iter mustfind(Iter first, Iter last, const Comp& val) { + Iter i = std::find(first,last,val); + if (i==last) { + throw notfound(); + } + return i; +} + + +template +Iter mustfind_if(Iter first, Iter last, Pred pred) { + Iter i = std::find_if(first,last,pred); + if (i==last) { + throw notfound(); + } + return i; +} + + +template +Iter1 mustsearch(Iter1 first1, Iter1 last1, Iter2 first2, Iter2 last2) { + Iter1 i = std::search(first1,last1,first2,last2); + if (i==last1) { + throw notfound(); + } + return i; +} + + +template +Iter1 mustsearch(Iter1 first1, Iter1 last1, Iter2 first2, Iter2 last2, Pred pred) { + Iter1 i = std::search(first1,last1,first2,last2,pred); + if (i==last1) { + throw notfound(); + } + return i; +} + + + +}; + + +#endif + diff --git a/extras/anytermd/libpbe/include/parse_http_request.hh b/extras/anytermd/libpbe/include/parse_http_request.hh new file mode 100644 index 0000000000..34b4b2c053 --- /dev/null +++ b/extras/anytermd/libpbe/include/parse_http_request.hh @@ -0,0 +1,44 @@ +// include/parse_http_request.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2005-2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_parse_http_request_hh +#define libpbe_parse_http_request_hh + +#include + +#include "HttpRequest.hh" +#include "Exception.hh" + + +namespace pbe { + + +HttpRequest parse_http_request(std::istream& strm); + + +struct HttpRequestSyntaxError: StrException { + HttpRequestSyntaxError(void): + StrException("Syntax error in http request") + {} +}; + + +}; + + +#endif diff --git a/extras/anytermd/libpbe/include/rfcdate.hh b/extras/anytermd/libpbe/include/rfcdate.hh new file mode 100644 index 0000000000..7970e6fa00 --- /dev/null +++ b/extras/anytermd/libpbe/include/rfcdate.hh @@ -0,0 +1,36 @@ +// rfcdate.hh +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2006 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_rfcdate_hh +#define libpbe_rfcdate_hh + + +#include + + +namespace pbe { + +std::string rfc_date(void); +// Return a date in the format required by RFC2616, RFC1123, +// RFC822 and RFC2822 + +}; + + +#endif + diff --git a/extras/anytermd/libpbe/include/run_cmd.hh b/extras/anytermd/libpbe/include/run_cmd.hh new file mode 100644 index 0000000000..15c2080ea0 --- /dev/null +++ b/extras/anytermd/libpbe/include/run_cmd.hh @@ -0,0 +1,34 @@ +// src/run_cmd.hh +// This file is part of libpbe; see http://decimail.org +// (C) 2005, 2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_run_cmd_hh +#define libpbe_run_cmd_hh + +#include + + +namespace pbe { + +std::string run_cmd(std::string cmd, bool& exit_ok); + +std::string run_cmd(std::string cmd); + +}; + + +#endif diff --git a/extras/anytermd/libpbe/include/safe_int.hh b/extras/anytermd/libpbe/include/safe_int.hh new file mode 100644 index 0000000000..18ca41613e --- /dev/null +++ b/extras/anytermd/libpbe/include/safe_int.hh @@ -0,0 +1,151 @@ +// src/safe_int.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +// safe_int: provide an N-bit signed integer (N excludes the sign bit) with overflow +// checking. An exception is thrown if the value overflows. + +// The implementation uses Boost.Integer to supply an integer with at least one more bit +// than we need. This extra guard bit is tested to detect overflow. +// This approach is reasonably efficient, except in the case where the need for the extra +// bit means that we have to use a larger underlying value; for example, safe_int<31> needs +// an int64_t. There is another approach that could work in that case, which is to compare +// the operands before performing the operation; that could be added as a specialisation of +// this template for safe_int<31>, safe_int<15> and safe_int<7>. + +// One of the better resources describing this subject is: +// http://msdn2.microsoft.com/en-us/library/ms972705.aspx + + +#ifndef libpbe_safe_int_hh +#define libpbe_safe_int_hh + +#include "./integer.hpp" +#include + +namespace pbe { + +template // NBITS excludes the sign bit +class safe_int: + boost::operators > +{ +public: + struct overflow: public std::exception { + const char* what() const throw() { return "safe_int overflow"; } + }; + +private: + typedef typename boost::int_t::least val_t; + val_t val; + + void check() { + // If guard bits != sign bit, value has overflowed. + val_t x = val>>NBITS; + if (x==0 || x==-1) { + return; + } + throw overflow(); + } + +public: + safe_int() {} + safe_int(const safe_int& x): val(x.val) {} + + // This can convert from integer or other safe_int types. + // It would be good if we could do a non-checking version when X_T is guaranteed to be + // smaller than NBITS. Is there any template-magic that can do that? (enable_if?) + template + safe_int(X_T x): val(static_cast(x)) { check(); } + + // Implicitly convert to val_t. (Is this a safe thing to do?) + operator val_t() { return val; } + + // These don't need any checking: + bool operator<(const safe_int& x) const { return val>=(const safe_int& x) { val>>=x.val; return *this; } + + // These can be checked using the guard bit: + safe_int& operator+=(const safe_int& x) { val+=x.val; check(); return *this; } + safe_int& operator-=(const safe_int& x) { val-=x.val; check(); return *this; } + + // These need more complex checking: + + // This only works when val_t is <= int32_t. + safe_int& operator*=(const safe_int& x) { + // Do a multiplication that is certain to have enough bits for the result: + typedef typename boost::int_t<2*NBITS+1>::least twice_val_t; + twice_val_t tw = static_cast(val) * static_cast(x.val); + // Overflow if any too-significant bits are set: + twice_val_t r = tw >> NBITS; + if (r!=0 && r!=-1) { + throw overflow(); + } + val = static_cast(tw); + return *this; + } + + // The only case where division can overflow is -MAX/-1. + // We could also detect division by zero here if we wanted to. + safe_int& operator/=(const safe_int& x) { + if (x.val==-1 && val==(1<>(NBITS-x.val); + if (r!=0 && r!=-1) { + throw overflow(); + } + val <<= x.val; + return *this; + } + + // These are implemented in terms of other operators, which do checking: + safe_int& operator++() { return (*this) += 1; } + safe_int& operator--() { return (*this) -= 1; } + + // (Does Boost.Operators provide unary operator- ?) + // safe_int operator-() { ... } + + + // It would probably be reasonable to allow a different type for the RHS of most of these + // operators. Boost.Integer has some code to do this, but I have not investigated it. + +}; + +}; + + +#endif + diff --git a/extras/anytermd/libpbe/include/scoped_true.hh b/extras/anytermd/libpbe/include/scoped_true.hh new file mode 100644 index 0000000000..b6e1716ecd --- /dev/null +++ b/extras/anytermd/libpbe/include/scoped_true.hh @@ -0,0 +1,44 @@ +// include/scoped_true.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef pbe_scoped_true_hh +#define pbe_scoped_true_hh + +// This file provides a simple class that sets a boolean to true when it +// is constructed and to false when it is destructed. + + +namespace pbe { + +class scoped_true { + bool& var; +public: + scoped_true(bool& var_): + var(var_) + { + var = true; + } + ~scoped_true() { + var = false; + } +}; + +}; + +#endif + diff --git a/extras/anytermd/libpbe/include/segv_backtrace.hh b/extras/anytermd/libpbe/include/segv_backtrace.hh new file mode 100644 index 0000000000..e49a167d9f --- /dev/null +++ b/extras/anytermd/libpbe/include/segv_backtrace.hh @@ -0,0 +1,30 @@ +// include/segv_backtrace.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/trunk +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_segv_backtrace_hh +#define libpbe_segv_backtrace_hh + + +// Call this to install a signal handler that will report a +// backtrace to stderr on segfault, bus error, illegal instruction +// and FP exception. + +void get_backtrace_on_segv(); + + +#endif diff --git a/extras/anytermd/libpbe/include/select.hh b/extras/anytermd/libpbe/include/select.hh new file mode 100644 index 0000000000..d0964281ea --- /dev/null +++ b/extras/anytermd/libpbe/include/select.hh @@ -0,0 +1,34 @@ +// src/select.hh +// This file is part of libpbe; see http://decimail.org +// (C) 2004 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +#ifndef libpbe_select_hh +#define libpbe_select_hh + +// User-friendly wrappers around select(). +// Take various numbers of file descriptors and an optional timeout. +// Naming scheme is "r" for "readable", "t" for "timeout". +// i.e. select_rr waits for either of two fds to be readable. +// The return value is the file descriptor that has become useable, +// i.e. select_rr(4,9) can return 4 or 9. If the timeout expires, +// -1 is returned. -2 can also be returned but I can't recall when. + +int select_r (int fd1); +int select_rr(int fd1, int fd2); +int select_rt(int fd1, float timeout); + + +#endif diff --git a/extras/anytermd/libpbe/include/sleep.hh b/extras/anytermd/libpbe/include/sleep.hh new file mode 100644 index 0000000000..8bae55de25 --- /dev/null +++ b/extras/anytermd/libpbe/include/sleep.hh @@ -0,0 +1,58 @@ +// include/sleep.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_sleep_hh +#define libpbe_sleep_hh + +#include + +#include + +#include "Exception.hh" + + +namespace pbe { + +inline void sleep(double t) +{ + double whole_secs; + double frac_secs = modf(t, &whole_secs); + struct timespec ts; + ts.tv_sec = static_cast(whole_secs); + ts.tv_nsec = static_cast(frac_secs*1e9); + struct timespec rem; + while (1) { + int rc = nanosleep(&ts,&rem); + if (rc==0) { + return; + } else { + if (errno==EINTR) { + ts = rem; + continue; + } + throw_ErrnoException("nanosleep"); + } + } +} + + +}; + + +#endif + diff --git a/extras/anytermd/libpbe/include/sort_by.hh b/extras/anytermd/libpbe/include/sort_by.hh new file mode 100644 index 0000000000..e9856379f5 --- /dev/null +++ b/extras/anytermd/libpbe/include/sort_by.hh @@ -0,0 +1,60 @@ +// include/sort_by.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_sort_by_hh +#define libpbe_sort_by_hh + +// Provides a function sort_by, which is like std::sort except that it +// takes a pointer to member to indicate which field of a struct it should +// use for sorting. +// Typical usage: +// struct T { int a; ..... }; +// std::container c; +// sort_by(c.begin(),c.end(),&T::a); + + +#include + + +namespace pbe { + +template +struct compare_data_member { + data_member_ptr_t data_member_ptr; + compare_data_member(data_member_ptr_t data_member_ptr_): + data_member_ptr(data_member_ptr_) + {} + bool operator()(const obj_t& a, const obj_t& b) + { + return a.*data_member_ptr < b.*data_member_ptr; + } +}; + +template +void sort_by(iter_t begin, iter_t end, data_member_ptr_t data_member_ptr) { + std::sort(begin,end, + compare_data_member (data_member_ptr)); +} + + +}; + + +#endif + diff --git a/extras/anytermd/libpbe/include/sort_by_pointee.hh b/extras/anytermd/libpbe/include/sort_by_pointee.hh new file mode 100644 index 0000000000..97f3202f9f --- /dev/null +++ b/extras/anytermd/libpbe/include/sort_by_pointee.hh @@ -0,0 +1,60 @@ +// include/sort_by_pointee.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_sort_by_pointee_hh +#define libpbe_sort_by_pointee_hh + +// Provides a function sort_by_pointee, which is like std::sort except that it +// takes a pointer to member to indicate which field of a struct it should +// use for sorting. +// Typical usage: +// struct T { int a; ..... }; +// std::container c; +// sort_by_pointee(c.begin(),c.end(),&T::a); + + +#include + + +namespace pbe { + +template +struct compare_ptr_data_member { + data_member_ptr_t data_member_ptr; + compare_ptr_data_member(data_member_ptr_t data_member_ptr_): + data_member_ptr(data_member_ptr_) + {} + bool operator()(ptr_t a, ptr_t b) + { + return a->*data_member_ptr < b->*data_member_ptr; + } +}; + +template +void sort_by_pointee(iter_t begin, iter_t end, data_member_ptr_t data_member_ptr) { + std::sort(begin,end, + compare_ptr_data_member (data_member_ptr)); +} + + +}; + + +#endif + diff --git a/extras/anytermd/libpbe/include/sorted_ptr_vector.hh b/extras/anytermd/libpbe/include/sorted_ptr_vector.hh new file mode 100644 index 0000000000..16f8090aec --- /dev/null +++ b/extras/anytermd/libpbe/include/sorted_ptr_vector.hh @@ -0,0 +1,65 @@ +// include/sorted_ptr_vector.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_sorted_ptr_vector_hh +#define libpbe_sorted_ptr_vector_hh + +// Provides a vector of pointers, which is ordered according to one of +// the fields in the pointed-to struct. It is sorted after construction. + +#include +#include + +#include "sort_by_pointee.hh" + + +namespace pbe { + + +template +class sorted_ptr_vector { + + typedef std::vector vec_t; + vec_t vec; + +public: + // default ctor is not very useful: + sorted_ptr_vector() {} + // default copy ctor is OK + + // This is the useful one: + template + sorted_ptr_vector(iter_t begin, iter_t end): + vec(begin,end) + { + sort_by_pointee(vec.begin(),vec.end(),data_member_ptr); + } + + typedef typename vec_t::const_iterator const_iterator; + + const_iterator begin() const { return vec.begin(); } + const_iterator end() const { return vec.end(); } + +}; + + +}; + + +#endif + diff --git a/extras/anytermd/libpbe/include/sorted_vector.hh b/extras/anytermd/libpbe/include/sorted_vector.hh new file mode 100644 index 0000000000..2f3b150a99 --- /dev/null +++ b/extras/anytermd/libpbe/include/sorted_vector.hh @@ -0,0 +1,63 @@ +// include/sorted_vector.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_sorted_vector_hh +#define libpbe_sorted_vector_hh + +// Provides a vector, which is sorted immediately after construction +// according to a supplied less-than predicate, and is then const. + +#include +#include + + +namespace pbe { + + +template +class sorted_vector { + + typedef std::vector vec_t; + vec_t vec; + +public: + // default ctor is not very useful: + sorted_vector() {} + // default copy ctor is OK + + // This is the useful one: + template + sorted_vector(iter_t begin, iter_t end, COMP comp = COMP()): + vec(begin,end) + { + std::sort(vec.begin(),vec.end(),comp); + } + + typedef typename vec_t::const_iterator const_iterator; + + const_iterator begin() const { return vec.begin(); } + const_iterator end() const { return vec.end(); } + +}; + + +}; + + +#endif + diff --git a/extras/anytermd/libpbe/include/strector.hh b/extras/anytermd/libpbe/include/strector.hh new file mode 100644 index 0000000000..cfebf78897 --- /dev/null +++ b/extras/anytermd/libpbe/include/strector.hh @@ -0,0 +1,93 @@ +// include/strector.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_strector_hh +#define libpbe_strector_hh + +// strector: stretchy-vector +// ------------------------- +// +// Provides a vector which stretches when new elements are accessed +// using operator[]. +// +// Since the stretch may cause the vector to re-allocate, operator[] +// on a mutable strector invalidates iterators, references and pointers +// to its elements unless the index is less than capacity(). +// +// What should operator[] do for a const strector when you access a +// beyond-the-end element? It can't stretch the vector. It could be +// undefined (like vector). It could throw. It could return a default +// T by value - but then it would have to return by value in all cases. +// The option that I've chosen is to return a const reference to a static +// T. This is broken if you try to compare addresses of elements. It +// is also broken if you expected the reference to change when you wrote +// to that element via a mutable reference to the strector, but that would +// be broken anyway unless size() < index < capacity() due to the +// iterator-invalidation rules. The use of a static may also have +// thread-safety problems. +// +// An additional optional constructor parameter allows you to specify a value +// used for new elements when the vector is resized during operator[] and +// when a beyond-the-end element is read as above. + + +#include + + +namespace pbe { + + +template < typename T, typename ALLOC=std::allocator > +class strector: public std::vector { + typedef std::vector base_t; + const T default_t; + +public: + // Same ctors as std::vector: + strector() {} + strector(typename base_t::size_type n): base_t(n) {} + strector(typename base_t::size_type n, const T& t): base_t(n,t), default_t(t) {} + // The copy-ctor takes a base_t, so we can copy from another strector or + // from a compatible std::vector: + strector(const base_t& other): base_t(other) {} + template + strector(iter first, iter last): base_t(first,last) {} + + // Additional ctor to specify default_t: + explicit strector(const T& default_t_): default_t(default_t_) {} + + typename base_t::reference operator[](typename base_t::size_type n) { + if (n>=base_t::size()) { + base_t::resize(n+1,default_t); + } + return base_t::operator[](n); + } + typename base_t::const_reference operator[](typename base_t::size_type n) const { + if (n>=base_t::size()) { + return default_t; + } + return base_t::operator[](n); + } +}; + + +}; + + +#endif + diff --git a/extras/anytermd/libpbe/include/string_qsort.hh b/extras/anytermd/libpbe/include/string_qsort.hh new file mode 100644 index 0000000000..c797c017a8 --- /dev/null +++ b/extras/anytermd/libpbe/include/string_qsort.hh @@ -0,0 +1,213 @@ +// include/string_qsort.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott +// This file is distributed under the terms of the Boost Software License v1.0. +// Please see http://www.boost.org/LICENSE_1_0.txt or the accompanying file BOOST_LICENSE. + +// string_qsort is a function template that sorts a range of strings using the +// quicksort algorithm, and avoids unnecessary comparisons of string prefixes that +// are already known to be equal while doing so. The range iterators must be +// random-access and the "strings" they refer to can be any random-access containers. +// +// When the input consists of strings with frequent long common prefixes this can +// give a performance advantage. However in other cases there is no advantage and +// this implementation's defects then make it slower than std::sort. Those +// defects are dumb pivot selection, dumb character comparison, no introsort-style +// fallback mode, and general lack of optimisation. + + +#ifndef libpbe_string_qsort_hh +#define libpbe_string_qsort_hh + +#include +#include +#include +#include + + +namespace pbe { + + +template +ITER choose_pivot(ITER begin, ITER end, std::size_t /*equal_prefix_length*/) +{ + // We just choose the middle of the range which gives good results for an + // already-sorted range. + // A smart version would do something clever especially when end-begin is large, + // e.g. choose a set of candidate pivots and use the middle one. + // (It's a shame we can't just use std::sort's version.) + // The desire to maximise the prefix length could also be taken into account. + return begin + (end-begin)/2; +} + + +template +bool chars_equal_at_offset(ITER begin, ITER end, std::size_t offset) +{ + // Is the character at [offset] equal for all strings in the range begin to end? + + if (begin->size() <= offset) { + return false; + } + + typedef typename ITER::value_type STRING; + typedef typename STRING::value_type CHAR; + CHAR c = (*begin)[offset]; + for (ITER i = begin; i != end; ++i) { + if (i->size() <= offset) { + return false; + } + if ((*i)[offset] != c) { + return false; + } + } + return true; +} + + +template +bool less_with_prefix(const STRING& a, const STRING& b, std::size_t equal_prefix_length) +{ + // Is a < b ? We know that the fist equal_prefix_length characters of a and b are equal. + + typename STRING::const_iterator ai = a.begin()+equal_prefix_length; + typename STRING::const_iterator bi = b.begin()+equal_prefix_length; + + // We can make this code a bit faster by avoiding the need to check for end-of-input + // on both strings on every iteration; instead we determine which is shorter up front + // and check only that one. + + if (a.size() >= b.size()) { + + while (1) { + if (bi==b.end()) { + return false; + } + if (*ai != *bi) { + return *ai < *bi; + } + ++ai; + ++bi; + } + + } else { + + while (1) { + if (ai==a.end()) { + return true; + } + if (*ai != *bi) { + return *ai < *bi; + } + ++ai; + ++bi; + } + + } +} + + +bool less_with_prefix(const std::string& a, const std::string& b, std::size_t equal_prefix_length) +{ + // We know that the characters are in contiguous memory for std::string so we can use memcmp(). + if (a.size() >= b.size()) { + return memcmp(&(a[equal_prefix_length]), &(b[equal_prefix_length]), b.size()-equal_prefix_length) == -1; + } else { + return memcmp(&(a[equal_prefix_length]), &(b[equal_prefix_length]), a.size()-equal_prefix_length) != 1; + } +} + +bool less_with_prefix(const std::vector& a, const std::vector& b, std::size_t equal_prefix_length) +{ + // We know that the characters are in contiguous memory for std::vector so we can use memcmp(). + if (a.size() >= b.size()) { + return memcmp(&(a[equal_prefix_length]), &(b[equal_prefix_length]), b.size()-equal_prefix_length) == -1; + } else { + return memcmp(&(a[equal_prefix_length]), &(b[equal_prefix_length]), a.size()-equal_prefix_length) != 1; + } +} + + +template +inline void string_qsort(ITER begin, ITER end, std::size_t equal_prefix_length=0) +{ + if (end-begin < 2) { + // Nothing to do if there are 0 or 1 elements. + return; + } + + if (end-begin == 2) { + // We can save some effort in this case. + if (less_with_prefix(*(begin+1), *begin, equal_prefix_length)) { + std::swap(*(begin+1), *begin); + } + return; + } + + if (end-begin == 3) { + /// The benefit from this is barely measureable. + if (less_with_prefix(*(begin+1), *begin, equal_prefix_length)) { + // BAC CAB CBA + if (less_with_prefix(*(begin+2), *(begin+1), equal_prefix_length)) { + // CBA + std::swap(*begin, *(begin+2)); + } else { + // BAC CAB + if (less_with_prefix(*(begin+2), *begin, equal_prefix_length)) { + // CAB + std::swap(*begin, *(begin+1)); + std::swap(*(begin+1), *(begin+2)); + } else { + // BAC + std::swap(*begin, *(begin+1)); + } + } + } else { + // ABC ACB BCA + if (less_with_prefix(*(begin+2), *(begin+1), equal_prefix_length)) { + // ACB BCA + if (less_with_prefix(*(begin+2), *begin, equal_prefix_length)) { + // BCA + std::swap(*begin, *(begin+1)); + std::swap(*begin, *(begin+2)); + } else { + // ACB + std::swap(*(begin+1), *(begin+2)); + } + } else { + // ABC + } + } + return; + } + + // Increase the equal prefix length if possible. + // This is a breadth-first method. Depth-first is an alternative that + // would have better locality but could waste effort for some inputs. + // (Is there a way to do this while partitioning in the previous pass?) + while (chars_equal_at_offset(begin,end,equal_prefix_length)) { + ++equal_prefix_length; + } + + // This is standard quicksort. + ITER pivot_i = choose_pivot(begin,end,equal_prefix_length); + std::swap(*pivot_i, *(end-1)); + pivot_i = end-1; + ITER s = begin; + for (ITER i = begin; i != pivot_i; ++i) { + if (less_with_prefix(*i,*pivot_i,equal_prefix_length)) { + std::swap(*i, *s); + ++s; + } + } + swap(*s, *pivot_i); + + string_qsort(begin, s, equal_prefix_length); + string_qsort(s+1, end, equal_prefix_length); +} + + +}; + +#endif + diff --git a/extras/anytermd/libpbe/include/utils.hh b/extras/anytermd/libpbe/include/utils.hh new file mode 100644 index 0000000000..3fd15e8dfd --- /dev/null +++ b/extras/anytermd/libpbe/include/utils.hh @@ -0,0 +1,81 @@ +// src/utils.hh +// This file is part of libpbe; see http://decimail.org +// (C) 2004-2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef libpbe_utils_hh +#define libpbe_utils_hh + +#include "StringTransformer.hh" + +#include +//#include + + +namespace pbe { + +//string int_to_string(int i); +//string float_to_string(float f); +//string double_to_string(double f); + +//int string_to_int(string s); +//float string_to_float(string s); +//double string_to_double(string s); + +int hex_string_to_int(string s); + +int maybe_hex_string_to_int(string s); + +extern std::string escape_for_dquoted_string(std::string s); +extern std::string escape_for_squoted_string(std::string s); +extern std::string escape_for_regexp(std::string s); + +//string escape_for_xpath_string(string s); + +//extern const StringTransformer& to_upper; + +//extern const StringTransformer& to_lower; + +//extern const StringTransformer& lf_to_crlf; + +template // contaier +std::string join(const T& strs, std::string joiner) { + if (strs.empty()) { + return ""; + } + typename T::const_iterator i = strs.begin(); + std::string s = *i; + ++i; + while (i != strs.end()) { + s += joiner; + s += *i; + ++i; + } + return s; +} + +//string trim_whitespace (string s); + +//string normalise_whitespace (string s); + +//bool starts_with ( string s, string prefix ); + +//void check_alphanumeric ( string s ); +//void check_numeric ( string s ); + +}; + +#endif diff --git a/extras/anytermd/libpbe/scripts/iana/Makefile b/extras/anytermd/libpbe/scripts/iana/Makefile new file mode 100644 index 0000000000..ecb180769d --- /dev/null +++ b/extras/anytermd/libpbe/scripts/iana/Makefile @@ -0,0 +1,27 @@ +default_target: all + +iana_character_sets: + wget -O $@ http://www.iana.org/assignments/character-sets + +charsets.dat: iana_character_sets mk_charsets_dat.awk + awk -f mk_charsets_dat.awk iana_character_sets > charsets.dat + +charset_t_enum_body.hh: charsets.dat mk_charset_t_enum_body.awk + awk -f mk_charset_t_enum_body.awk charsets.dat > charset_t_enum_body.hh + +charset_iana_names_body.cc: charsets.dat mk_charset_iana_names_body.awk + awk -f mk_charset_iana_names_body.awk charsets.dat > charset_iana_names_body.cc + +charset_mime_names_body.cc: charsets.dat mk_charset_mime_names_body.awk + awk -f mk_charset_mime_names_body.awk charsets.dat > charset_mime_names_body.cc + +charset_lookup_body.cc: charsets.dat mk_charset_lookup_body.sh + ./mk_charset_lookup_body.sh < charsets.dat > charset_lookup_body.cc + +all: iana_character_sets charsets.dat charset_t_enum_body.hh \ + charset_iana_names_body.cc charset_mime_names_body.cc charset_lookup_body.cc + +FORCE: + +.PHONY: FORCE default_target all + diff --git a/extras/anytermd/libpbe/scripts/iana/iana-diffs.patch b/extras/anytermd/libpbe/scripts/iana/iana-diffs.patch new file mode 100644 index 0000000000..02f8cec3f6 --- /dev/null +++ b/extras/anytermd/libpbe/scripts/iana/iana-diffs.patch @@ -0,0 +1,19 @@ +*** iana_character_sets.orig 2008-02-14 22:42:58.000000000 +0000 +--- iana_character_sets 2008-02-14 22:43:36.000000000 +0000 +*************** +*** 899,910 **** +--- 899,912 ---- + Source: the 2-octet Basic Multilingual Plane, aka Unicode + this needs to specify network byte order: the standard + does not specify (it is a 16-bit integer space) ++ Alias: UCS-2 + Alias: csUnicode + + Name: ISO-10646-UCS-4 + MIBenum: 1001 + Source: the full code space. (same comment about byte order, + these are 31-bit numbers. ++ Alias: UCS-4 + Alias: csUCS4 + + Name: ISO-10646-UCS-Basic diff --git a/extras/anytermd/libpbe/scripts/iana/mk_charset_iana_names_body.awk b/extras/anytermd/libpbe/scripts/iana/mk_charset_iana_names_body.awk new file mode 100644 index 0000000000..c7a051c9e1 --- /dev/null +++ b/extras/anytermd/libpbe/scripts/iana/mk_charset_iana_names_body.awk @@ -0,0 +1,5 @@ +BEGIN { print "const char * const charset_iana_names[] = {"; } +NF>0 { printf(" \"%s\",\n",$1); } +END { print " NULL"; + print "};"; } + diff --git a/extras/anytermd/libpbe/scripts/iana/mk_charset_lookup_body.sh b/extras/anytermd/libpbe/scripts/iana/mk_charset_lookup_body.sh new file mode 100755 index 0000000000..6e914014d5 --- /dev/null +++ b/extras/anytermd/libpbe/scripts/iana/mk_charset_lookup_body.sh @@ -0,0 +1,31 @@ +#!/bin/sh + +awk ' +function to_ident(n) { + n = tolower(n); + gsub("[-:.()+_]","_",n); + return n; +} +function to_key(n) { + n = tolower(n); + n = gensub("([[:digit:]])[-:.()+_]+([[:digit:]])","\\1@\\2","g",n); + n = gensub("([[:digit:]])[-:.()+_]+([[:digit:]])","\\1@\\2","g",n); + gsub("[-:.()+_]","",n); + gsub("@","_",n); + return n; +} +NF>0 { ident = to_ident($1); + for (i=1; i<=NF; i++) { + if ($i != "*") { + aliases[to_key($i)] = 1; + } + } + for (A in aliases) { + printf(" {\"%s\", cs::%s},\n", A, ident); + } + delete aliases; + } +' | +LC_ALL=C sort -t '"' + + diff --git a/extras/anytermd/libpbe/scripts/iana/mk_charset_mime_names_body.awk b/extras/anytermd/libpbe/scripts/iana/mk_charset_mime_names_body.awk new file mode 100644 index 0000000000..017dacff46 --- /dev/null +++ b/extras/anytermd/libpbe/scripts/iana/mk_charset_mime_names_body.awk @@ -0,0 +1,14 @@ +BEGIN { print "const char * const charset_mime_names[] = {"; } +NF>0 { N=$1; + for (i=2; i<=NF; i++) { + if ($i=="*") { + i++; + N=$i; + break; + } + } + printf(" \"%s\",\n",N); + } +END { print " NULL"; + print "};"; } + diff --git a/extras/anytermd/libpbe/scripts/iana/mk_charset_t_enum_body.awk b/extras/anytermd/libpbe/scripts/iana/mk_charset_t_enum_body.awk new file mode 100644 index 0000000000..e3f768c875 --- /dev/null +++ b/extras/anytermd/libpbe/scripts/iana/mk_charset_t_enum_body.awk @@ -0,0 +1,46 @@ +function to_ident(n) { + n = tolower(n); + gsub("[-:.()+]","_",n); + return n; +} +function add_aliases_rec(pref,suff, pos,chunk,newsuff) { + pos = index(suff,"_"); + if (pos>0) { + chunk = substr(suff,0,pos-1); + newsuff = substr(suff,pos+1); + if (pref !~ "[[:digit:]]$" || suff !~ "^[[:digit:]]") { + add_aliases_rec(pref chunk, newsuff); + } + if (pref!="") { + add_aliases_rec(pref "_" chunk, newsuff); + } + } else if (pref=="") { + aliases[suff]=1; + } else { + aliases[pref "_" suff]=1; + if (pref !~ "[[:digit:]]$" || suff !~ "^[[:digit:]]") { + aliases[pref suff]=1; + } + } +} +function add_aliases(A) { + add_aliases_rec("",A); +} +BEGIN { print "enum charset_t {"; } +NF>0 { CS=to_ident($1); + printf(" %s,\n",CS); + add_aliases(CS); + for (i=2; i<=NF; i++) { + if ($i != "*") { + A=to_ident($i); + add_aliases(A); + } + } + for (A in aliases) { + if (A!=CS && A !~ /^[[:digit:]]/) { + printf(" %-26s= %s,\n",A,CS); + } + } + delete aliases; + } +END { print " max_charset"; print "};"; } diff --git a/extras/anytermd/libpbe/scripts/iana/mk_charsets_dat.awk b/extras/anytermd/libpbe/scripts/iana/mk_charsets_dat.awk new file mode 100644 index 0000000000..34734a20eb --- /dev/null +++ b/extras/anytermd/libpbe/scripts/iana/mk_charsets_dat.awk @@ -0,0 +1,4 @@ +/^Name:/ { printf("\n%s",$2); } +/^Alias:/ && $2!="None" && /preferred MIME name/ { printf("\t* %s",$2); } +/^Alias:/ && $2!="None" && !/preferred MIME name/ { printf("\t%s",$2); } + diff --git a/extras/anytermd/libpbe/scripts/unicode/Makefile b/extras/anytermd/libpbe/scripts/unicode/Makefile new file mode 100644 index 0000000000..28cd6dddbf --- /dev/null +++ b/extras/anytermd/libpbe/scripts/unicode/Makefile @@ -0,0 +1,7 @@ +iso8859_tables_body.cc: FORCE + ./mk_c_tables.sh > $@ + + +FORCE: + +.PHONY: FORCE diff --git a/extras/anytermd/libpbe/scripts/unicode/download_tables.sh b/extras/anytermd/libpbe/scripts/unicode/download_tables.sh new file mode 100755 index 0000000000..f6832d265e --- /dev/null +++ b/extras/anytermd/libpbe/scripts/unicode/download_tables.sh @@ -0,0 +1,20 @@ +for i in \ +ISO8859/8859-2.TXT \ +ISO8859/8859-3.TXT \ +ISO8859/8859-4.TXT \ +ISO8859/8859-5.TXT \ +ISO8859/8859-6.TXT \ +ISO8859/8859-7.TXT \ +ISO8859/8859-8.TXT \ +ISO8859/8859-9.TXT \ +ISO8859/8859-10.TXT \ +ISO8859/8859-11.TXT \ +ISO8859/8859-13.TXT \ +ISO8859/8859-14.TXT \ +ISO8859/8859-15.TXT \ +ISO8859/8859-16.TXT +do + f="tables/$i" + mkdir -p `dirname $f` + wget "http://unicode.org/Public/MAPPINGS/$i" -O $f +done diff --git a/extras/anytermd/libpbe/scripts/unicode/make_to_ascii_letters_tables.sh b/extras/anytermd/libpbe/scripts/unicode/make_to_ascii_letters_tables.sh new file mode 100755 index 0000000000..7480d047fd --- /dev/null +++ b/extras/anytermd/libpbe/scripts/unicode/make_to_ascii_letters_tables.sh @@ -0,0 +1,280 @@ +# scripts/unicode/make_to_ascii_letters_tables.sh +# This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +# (C) 2008 Philip Endecott + +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +# Download the Unicode character database: +# wget http://www.unicode.org/Public/5.1.0/ucd/UnicodeData.txt + +# Extract the decompositions: +cat UnicodeData.txt | +awk -F ';' '$6!="" {print $1,$6}' | +awk '/ decompositions + +# Each line in decompositions starts with the source character and is followed by the +# sequence of characters that it may be replaced by. This replacement needs to be +# applied recursively: +while true +do + +awk '{if (ARGIND==1) {d=""; for(i=2;i<=NF;i++) {d = d $i " "}; dec[$1]=d;} + else {printf("%s ",$1); + for(i=2;i<=NF;i++) { + if ($i in dec) {printf("%s",dec[$i]);} else {printf("%s ",$i);} + } + printf("\n");} + }' decompositions decompositions > new_decompositions + +if cmp decompositions new_decompositions +then + break +fi + +mv new_decompositions decompositions + +done + +mv new_decompositions full_decompositions + + +# For the puposes of converting strings to text searchable ASCII equivalents, we're only +# interested in determining the ASCII letter base characters. Strip all the other characters +# in the expansion, and if none are left remove the line. + +awk '{printf("%s ",$1); + for(i=2;i<=NF;i++) { + n=strtonum("0x" $i); + if (n>=65 && n<=90 || n>=97 && n<=122) + printf("%s ",$i) + } + printf("\n"); + }' full_decompositions | +awk '{if (NF>1) print}' | + + +# Now convert the letter codes to actual letters, and convert to lower case. + +awk '{printf("%s \"",$1); + for(i=2;i<=NF;i++) { + v=strtonum("0x" $i); + c=sprintf("%c",v); + printf("%c",tolower(c)); + } + printf("\"\n"); + }' | + +# Add 'identity' conversions for a-z and 'tolower' conversions for A-Z. + +awk 'BEGIN {for(i=65;i<=90;i++) printf("%04X \"%c\"\n",i,i+32); + for(i=97;i<=122;i++) printf("%04X \"%c\"\n",i,i);} + {print}' > all_decompositions + + +# Add some more local decompositions. + +cat >>all_decompositions <(page ".d0")}' all_decompositions + + +# Reformat them as struct initialisers: + +for i in *.d0 +do + +n=`basename $i .d0` + +echo -n "char_expansion_page_${n}_t to_ascii_letters_page_${n} [] = {"; + +awk '{n=strtonum("0x" $1); a[n%256]=$2;} + END {for(i=0;i<256;i++) { + if (i in a) printf("%s, ",a[i]); else printf("\"\", "); + } + printf("};\n") + }' $i + +done > to_ascii_letters_tables.cc + + diff --git a/extras/anytermd/libpbe/scripts/unicode/mk_c_tables.sh b/extras/anytermd/libpbe/scripts/unicode/mk_c_tables.sh new file mode 100755 index 0000000000..475d0b4289 --- /dev/null +++ b/extras/anytermd/libpbe/scripts/unicode/mk_c_tables.sh @@ -0,0 +1,33 @@ +#!/bin/sh + +echo "iso8859_table_t iso8859_tables[] = {" + +for n in 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 +do + if [ $n == 12 ] + then + echo " // ISO-8859-12 does not exist" + echo " {}," + else + echo " // ISO-8859-${n}" + echo -n " {" + awk 'BEGIN { L = -1; } + !/^#/ && !/^ *$/ { + N = strtonum($1); U = $2; + if (N < L) { print "Table is out of order"; exit(1); } + while (L + +#include +#include + +#ifdef __OpenBSD__ +// Why is this needed? +#include +#endif + +using namespace std; +using namespace pbe; + + +class CgiError: public Exception { +private: + string msg; +public: + CgiError(string m): msg(m) {} + void report(ostream& s) const { + s << "Error while decoding CGI input: " << msg << endl; + } +}; + + +static string uri_unescape(string esc) +{ + string unesc; + string::size_type start=0; + while(start(hex_string_to_int(esc.substr(end+1,2))); + start=end+3; + } + } + return unesc; +} + + +class MimeHeader { +private: + ci_string name; + string value; + typedef map params_t; + params_t params; +public: + + MimeHeader(string s) + { + const string::size_type colon_pos = s.find(':'); + if (colon_pos==s.npos) { + throw CgiError("Malformed MIME header; no colon"); + } + name = s.substr(0,colon_pos).c_str(); + const string value_and_params = s.substr(colon_pos+1); + StringSplitterSeq splitter ( value_and_params, ";" ); + value = boost::algorithm::trim_copy(*splitter); + ++splitter; + while (!splitter.exhausted()) { + const string param = *splitter; + const unsigned int equals_pos = param.find('='); + const ci_string pname = boost::algorithm::trim_copy(param.substr(0,equals_pos)).c_str(); + string pvalue = boost::algorithm::trim_copy(param.substr(equals_pos+1)); + if (pvalue[0]=='\"' && pvalue[pvalue.size()-1]=='\"') { + pvalue=pvalue.substr(1,pvalue.size()-2); + } + params.insert(make_pair(pname,pvalue)); + ++splitter; + } + } + + ci_string get_name(void) const { return name; } + string get_value(void) const { return value; } + string get_param(ci_string name) const + { + params_t::const_iterator i = params.find(name); + if (i==params.end()) { + throw CgiError("No such header \""+string(name.c_str())+"\""); + } else { + return i->second; + } + } +}; + + + +void CgiParams::load(void) +{ + const char* method_c = getenv("REQUEST_METHOD"); + if (!method_c) { + throw CgiError("REQUEST_METHOD not set"); + } + const string method = method_c; + + // Normally you can assume that GET is accompanied with parameters + // in the QUERY_STRING, while POST is accompanied with parameters in + // the request body. But I now sometimes use POST requests with + // GET-style parameters and no body, so this code looks at the + // QUERY_STRING in all cases, and does not reject an empty body. If + // it finds more than set of parameters they will be merged. + + const char* query_c = getenv("QUERY_STRING"); + if (query_c) { + const string query = query_c; + init_from_urlencoded(query); + } else { + if (method=="GET") { + throw CgiError("QUERY_STRING not set in GET request"); + } + } + + if (method=="POST") { + + // N.B. The CGI spec says that the server is not required to send EOF + // after the POST data. We should really look at the CONTENT_LENGTH + // environment variable to find how much we should read. + // (This does seem to work with Apache however.) + string input; + char buf[1024]; + int nr; + do { + nr = read(0,buf,sizeof(buf)); + input.append(buf,nr); + } while (nr>0); + + const char* enctype_c = getenv("CONTENT_TYPE"); + if (!enctype_c) { + return; + } + const string enctype = enctype_c; + + const MimeHeader enctype_hdr("content-type: "+enctype); + + if (enctype_hdr.get_value()=="application/x-www-form-urlencoded") { + init_from_urlencoded(input); + } else if (enctype_hdr.get_value()=="multipart/form-data") { + const string boundary = enctype_hdr.get_param("boundary"); + init_from_multipart("\r\n"+input,boundary); + } else { + throw CgiError("Unknown content-type \""+enctype_hdr.get_value()+"\""); + } + + } +} + + +void CgiParams::init_from_urlencoded(string query_string) +{ + if (query_string=="") { + return; + } + for ( StringSplitterAny splitter ( query_string, "&;" ); + !splitter.exhausted(); ++splitter ) { + string key_value = *splitter; + string::size_type equals_pos = key_value.find('='); + if (equals_pos==key_value.npos) { + throw CgiError("Query string \""+query_string+"\" malformed"); + } + string key_esc = key_value.substr(0,equals_pos); + string key = uri_unescape(key_esc); + string value_esc = key_value.substr(equals_pos+1); + string value = uri_unescape(value_esc); + insert(make_pair(key,value)); + } +} + + +void CgiParams::init_from_multipart(string input, string boundary) +{ + StringSplitterSeq splitter(input,"\r\n--"+boundary); + ++splitter; + + while(!splitter.exhausted()) { + const string pre_section = *splitter; + if (pre_section.substr(0,2)=="--") { + break; + } + + unsigned int nl_pos = pre_section.find_first_not_of(" \t"); + if (pre_section.substr(nl_pos,2)!="\r\n") { + throw CgiError("MIME boundary not followed by newline"); + } + const string section = pre_section.substr(nl_pos+2); + const string::size_type blank_line_pos = section.find("\r\n\r\n"); + if (blank_line_pos==section.npos) { + throw CgiError("MIME section has no blank line"); + } + const string header = section.substr(0,blank_line_pos); + const string body = section.substr(blank_line_pos+4); + + string name=""; + + for (StringSplitterSeq headersplit(header,"\r\n"); + !headersplit.exhausted(); ++headersplit) { + const string headerline = *headersplit; + MimeHeader hdr(headerline); + if (hdr.get_name()=="content-disposition") { + if (hdr.get_value()=="form-data") { + name = hdr.get_param("name"); + } + } + if (hdr.get_name()=="content-transfer-encoding") { + throw CgiError("Content transfer encodings are not supported"); + } + } + + if (name=="") { + throw CgiError("No name found"); + } + + insert(make_pair(name,body)); + ++splitter; + } +} + + +string CgiParams::get(string name) const +{ + const_iterator i = find(name); + if (i!=end()) { + return i->second; + } else { + throw CgiError("Parameter "+name+" not found"); + } +} + + +string CgiParams::get_default(string name, string def) const +{ + const_iterator i = find(name); + if (i!=end()) { + return i->second; + } else { + return def; + } +} + diff --git a/extras/anytermd/libpbe/src/CgiVars.cc b/extras/anytermd/libpbe/src/CgiVars.cc new file mode 100644 index 0000000000..f89c417808 --- /dev/null +++ b/extras/anytermd/libpbe/src/CgiVars.cc @@ -0,0 +1,76 @@ +// src/CgiVars.hh +// This file is part of libpbe; see http://decimail.org +// (C) 2004 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#include "CgiVars.hh" + +#include + +using namespace std; + + +void CgiVars::setvar(const char* varname) +{ + const char* envvar = getenv(varname); + if (envvar) { + (*this)[varname]=envvar; + } else { + (*this)[varname]=""; + } +} + + +void CgiVars::load(void) +{ + setvar("SERVER_SOFTWARE"); + setvar("SERVER_NAME"); + setvar("GATEWAY_INTERFACE"); + setvar("SERVER_PROTOCOL"); + setvar("SERVER_PORT"); + setvar("REQUEST_METHOD"); + setvar("PATH_INFO"); + setvar("PATH_TRANSLATED"); + setvar("SCRIPT_NAME"); + setvar("QUERY_STRING"); + setvar("REMOTE_HOST"); + setvar("AUTH_TYPE"); + setvar("REMOTE_USER"); + setvar("REMOTE_IDENT"); + setvar("CONTENT_TYPE"); + setvar("CONTENT_LENGTH"); + + setvar("HTTP_USER_AGENT"); + setvar("HTTP_COOKIE"); + setvar("HTTP_IF_MODIFIED_SINCE"); + setvar("HTTP_IF_NONE_MATCH"); + + setvar("HTTPS"); +} + + +/*static*/ const CgiVars& CgiVars::singleton(void) +{ + static CgiVars cgivars; + static bool loaded=false; + + if (!loaded) { + cgivars.load(); + loaded=true; + } + return cgivars; +} diff --git a/extras/anytermd/libpbe/src/Daemon.cc b/extras/anytermd/libpbe/src/Daemon.cc new file mode 100644 index 0000000000..e94103bbee --- /dev/null +++ b/extras/anytermd/libpbe/src/Daemon.cc @@ -0,0 +1,329 @@ +// src/Daemon.cc +// This file is part of libpbe; see http://decimail.org +// (C) 2004-2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#include "Daemon.hh" + +#include "Exception.hh" +#include "Lock.hh" +#include "Thread.hh" + +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +using namespace std; +using namespace pbe; + + +static void handle_foreground_exception(Exception& E) +{ + E.report(cerr); + exit(E.exit_status); +} + + +static void handle_background_exception(Exception& E) +{ + ostringstream s; + E.report(s); + syslog(LOG_ERR,"Unhandled exception: %s",s.str().c_str()); +} + + +Daemon::Daemon(short p, string n, int sf, string u, string d, int max_sessions_, + bool accept_local_only_): + port(p), progname(n), syslog_facility(sf), username(u), dir(d), + max_sessions(max_sessions_ ? max_sessions_ : default_max_sessions), + accept_local_only(accept_local_only_), + n_sessions(0) +{} + + +Daemon::~Daemon() +{} + + +void Daemon::run_interactively(void) +{ + FileDescriptor fdin(0); + FileDescriptor fdout(1); + try { + try { + startup(); + session(fdin,fdout); + } + RETHROW_MISC_EXCEPTIONS; + } + catch(Exception& E) { + handle_foreground_exception(E); + } +} + + +void Daemon::decrement_session_count() +{ + { + Lock l(n_sessions_mutex); + --n_sessions; + } + n_sessions_condition.notify_all(); +} + +static void run_session(Daemon* daemon, int connfd_n) +{ + try { + try { + //syslog(LOG_INFO,"new session starting"); + FileDescriptor connfd(connfd_n); + daemon->session(connfd,connfd); + //syslog(LOG_INFO,"session finished"); + daemon->decrement_session_count(); + return; + } + RETHROW_MISC_EXCEPTIONS; + } + catch(Exception& E) { + handle_background_exception(E); + // what do we want to do after an exception in a session? + // presumably continue? alternative is to quit the entire process. + // exit(E.exit_status); + daemon->decrement_session_count(); + } +} + + +void handle_signal(int signo) +{ + syslog(LOG_CRIT,"got signal %s",strsignal(signo)); + + switch(signo) { + + case SIGHUP: + // Hangup - often means "restart" for daemons, but we ignore it. + signal (SIGHUP, handle_signal); + return; + + case SIGPIPE: + // Wrote to a closed pipe. + // This signal is now ignored, so this case is never reached. + // We must detect closed sockets by checking the return values + // of system calls. + // The difficulty with SIGPIPE is that it is unclear which thread + // receives it - it may not be the one doing the write(), according + // to some web references - and even then, I don't know how the + // signal handler could then cause an exception (or whatever) in + // the corresponding session code. Horrible. + return; + + default: + // Other signals e.g. INT, SEGV etc. + // Terminate entire process. + // Could consider terminating a single thread. + syslog(LOG_CRIT,"terminating"); + exit(1); + } + + // remember signal need to be re-enabled if the handler is going to + // return +} + + +void Daemon::run_as_daemon(bool background) +{ + signal (SIGHUP, handle_signal); + signal (SIGINT, handle_signal); + //signal (SIGQUIT, SIG_DFL); // This means we can kill -QUIT and dump core + signal (SIGILL, handle_signal); + signal (SIGBUS, handle_signal); + signal (SIGFPE, handle_signal); + signal (SIGSEGV, handle_signal); + signal (SIGTERM, handle_signal); + signal (SIGPIPE, SIG_IGN); + signal (SIGABRT, handle_signal); + + openlog(progname.c_str(),LOG_PID,syslog_facility); + + try { + try { + + // set umask? + + int listenfd = socket(PF_INET,SOCK_STREAM,0); + // Add SO_KEEPALIVE so we get SIGPIPE if connection fails? + if(listenfd==-1) { + throw_ErrnoException("socket()"); + } + // race condition here + int rc = fcntl(listenfd,F_SETFD,FD_CLOEXEC); + if (rc==-1) { + throw_ErrnoException("fcntl(listenfd,F_SETFD,FD_CLOEXEC)"); + } + + // Not sure what this does + const int t=1; + setsockopt(listenfd, SOL_SOCKET, SO_REUSEADDR, &t, sizeof(t)); + + struct sockaddr_in server_addr; + memset(&server_addr,0,sizeof(server_addr)); + server_addr.sin_family=AF_INET; + server_addr.sin_addr.s_addr=htonl( accept_local_only ? INADDR_LOOPBACK : INADDR_ANY ); + server_addr.sin_port=htons(port); + int r = bind(listenfd,(struct sockaddr*)&server_addr,sizeof(server_addr)); + if (r==-1) { + throw_ErrnoException("bind()"); + } + + // The parameter to listen is the "backlog" parameter, the number un-accepted + // connections allowed before the connection is refused. + r = listen(listenfd,8); + if (r==-1) { + throw_ErrnoException("listen()"); + } + + if (background) { + // cd to / + // disconnect std[in|out|err]; + // fork; parent process exits. + r = daemon(0,0); + if(r) { + throw_ErrnoException("daemon()"); + } + } + + // Errors now go to syslog + try { + try { + + // Create PID file + // Need to do this after daemon() else we get the wrong pid + if (getuid()==0) { + string pidfilename = "/var/run/"+progname+".pid"; + ofstream pidfile(pidfilename.c_str()); + pidfile << getpid() << endl; + pidfile.close(); + } + // Can't delete this file at termination as we don't have + // the neccessary permissions then - does this matter? + + syslog(LOG_INFO,"%s starting",progname.c_str()); + + // Start as root in order to bind to a reserved port + // Having bound, give up root and run as the specified daemon user + if (getuid()==0 && username!="") { + struct passwd* pw = getpwnam(username.c_str()); + if (!pw) { + throw_ErrnoException("getpwnam(\""+username+"\")"); + } + int rc = setuid(pw->pw_uid); + if (rc==-1) { + throw_ErrnoException("seteuid()"); + } + } + + // Change working directory, if requested. + // This was added so that core files can go somewhere sensible. + if (dir!="") { + int rc = chdir(dir.c_str()); + if (rc==-1) { + throw_ErrnoException("chdir("+dir+")"); + } + } + + // If we dump core, it may be a large core file if we have many threads + // running, and some systems set an rlimit that breaks this. If this is + // a soft rlimit we can increase it here. If for some reason we can't + // increase it we won't treat it as an error. Note that we're doing this + // having changed user to the run-as user. + struct rlimit unlimited_cores; + unlimited_cores.rlim_cur = RLIM_INFINITY; + unlimited_cores.rlim_max = RLIM_INFINITY; + setrlimit(RLIMIT_CORE,&unlimited_cores); + + startup(); + + while(1) { + + { + Lock l(n_sessions_mutex); + while (n_sessions >= max_sessions) { + n_sessions_condition.wait(l); + } + } + + struct sockaddr_in client_addr; + socklen_t client_size=sizeof(client_addr); + int connfd_n; + do { + connfd_n = accept(listenfd,(struct sockaddr*)&client_addr, + &client_size); + } while (connfd_n==-1 && errno==EINTR); + if (connfd_n==-1) { + throw_ErrnoException("accept()"); + } + // race condition here + int rc = fcntl(connfd_n,F_SETFD,FD_CLOEXEC); + if (rc==-1) { + throw_ErrnoException("fcntl(connfd_n,F_SETFD,FD_CLOEXEC)"); + } + + { + Lock l(n_sessions_mutex); + ++n_sessions; + } + + Thread t(boost::bind(&run_session,this,connfd_n)); + } + } + RETHROW_MISC_EXCEPTIONS; + } + catch(Exception& E) { + if (background) { + handle_background_exception(E); + } else { + handle_foreground_exception(E); + } + exit(E.exit_status); + } + + } + RETHROW_MISC_EXCEPTIONS; + } + catch(Exception& E) { + handle_foreground_exception(E); + } +} + + +void Daemon::run_default(void) +{ + if (isatty(0)) { + run_interactively(); + } else { + run_as_daemon(); + } +} + diff --git a/extras/anytermd/libpbe/src/Database.cc b/extras/anytermd/libpbe/src/Database.cc new file mode 100644 index 0000000000..67026df722 --- /dev/null +++ b/extras/anytermd/libpbe/src/Database.cc @@ -0,0 +1,422 @@ +// src/Database.cc +// This file is part of libpbe; see http://decimail.org +// (C) 2004 - 2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#include "Database.hh" + +#include "Exception.hh" +#include "utils.hh" +#include "StringTransformer.hh" + +#include +#include + +#include +#include +#include +#include +#include + +#include +#include +#include + +using namespace std; + + +namespace pbe { + + +static typecode_t oid_to_typecode(Oid oid) +{ + switch(oid) { + case TEXTOID: return text_type; + case BYTEAOID: return text_type; + case INT4OID: return numeric_type; + case INT8OID: return numeric_type; + case TIMESTAMPTZOID: return timestamptz_type; + case TIMESTAMPOID: return timestamptz_type; // Hmm, should distinguish it is local time + case FLOAT4OID: return float_type; + case FLOAT8OID: return double_type; + default: throw StrException("type error, unrecognised oid "+boost::lexical_cast(oid)); + } +} + + +static Oid typecode_to_oid(typecode_t typecode) +{ + switch(typecode) { + case null_type: return 0; + case text_type: return TEXTOID; + case numeric_type: return INT4OID; + case timestamptz_type: return TIMESTAMPTZOID; + case bytea_type: return BYTEAOID; + case float_type: return FLOAT4OID; + case double_type: return FLOAT8OID; + default: throw "type error, unrecognised typecode"; + } +} + + +class DatabaseConnectionFailed: public DatabaseException { +public: + DatabaseConnectionFailed(PGconn* pgconn): + DatabaseException(pgconn,"Connecting to database") {} +}; + + +Database::Database(string conninfo): + pgconn(PQconnectdb(conninfo.c_str())), + conn_fd(PQsocket(pgconn),false), + transaction_in_progress(false) +{ + if (PQstatus(pgconn)!=CONNECTION_OK) { + throw DatabaseConnectionFailed(pgconn); + } +} + + +Database::~Database() +{ + PQfinish(pgconn); +} + + +const FileDescriptor& Database::get_fd(void) const +{ + return conn_fd; +} + + +string Database::get_any_notification(void) +{ + int rc = PQconsumeInput(pgconn); + if (rc==0) { + throw QueryFailed(pgconn,"checking for notifications"); + } + boost::shared_ptr p(PQnotifies(pgconn), PQfreemem); + if (!p) { + return ""; + } + return p.get()->relname; +} + + +void Database::exec_sql(string cmd) +{ + boost::shared_ptr res(PQexec(pgconn, cmd.c_str()), PQclear); + if (!res || PQresultStatus(res.get()) !=PGRES_COMMAND_OK) { + throw QueryFailed(pgconn, cmd.c_str()); + } +} + + +void DatabaseException::report(ostream& s) const +{ + s << "Database exception: " << postgres_error; + if (doing_what!="") { + s << " while " << doing_what; + } + s << "\n"; +} + + +Transaction::Transaction(Database& database_): + database(database_), + committed(false) +{ + if (database.transaction_in_progress) { + nested=true; + } else { + nested=false; + database.transaction_in_progress=true; + database.exec_sql("begin"); + } +} + + +Transaction::~Transaction() +{ + if (!nested) { + if (!committed) { + database.transaction_in_progress=false; + try { + database.exec_sql("rollback;"); + } + catch(...) { + // Mustn't throw an exception from inside a destructor, in case it is being + // invoked during exception processing. + // (TODO is there a better fix for this?) + } + } + } +} + + +void Transaction::commit(void) +{ + if (!nested) { + database.transaction_in_progress=false; + database.exec_sql("commit"); + committed=true; + } +} + + +Result::Result(boost::shared_ptr res_): + rows(PQntuples(res_.get())), + cols(PQnfields(res_.get())), + res(res_) +{} + + +int Result::get_rows_changed(void) const +{ + return boost::lexical_cast(PQcmdTuples(res.get())); +} + + +class ColumnNotFound: public StrException { +public: + ColumnNotFound(string colname): + StrException("Table has no column named " + colname) + {} +}; + + +int Result::column(std::string name) const +{ + int n = PQfnumber(res.get(),name.c_str()); + if (n==-1) { + throw ColumnNotFound(name); + } + return n; +} + + +std::string Result::column_name(int col) const +{ + return PQfname(res.get(),col); +} + +char* Result::rawget(int row, int col) const +{ + return PQgetvalue(res.get(),row,col); +} + +int Result::getlength(int row, int col) const +{ + return PQgetlength(res.get(),row,col); +} + + +bool Result::is_null(int row, int col) const +{ + return PQgetisnull(res.get(),row,col); +} + + +typecode_t Result::column_typecode(int col) const +{ + Oid oid = PQftype(res.get(),col); + return oid_to_typecode(oid); +} + + +int statement_name_t::counter = 0; + + +static bool can_use_pqexecparams(string querystr) +{ + boost::algorithm::trim_left_if(querystr,boost::algorithm::is_any_of(" (")); + return boost::algorithm::istarts_with(querystr,"select") + || boost::algorithm::istarts_with(querystr,"update") + || boost::algorithm::istarts_with(querystr,"insert") + || boost::algorithm::istarts_with(querystr,"delete"); +} + + +QueryCore::QueryCore(Database& database_, std::string querystr_, int nparams_, + typecode_t* argtypecodes, int* lengths, int* formats): + database(database_), + querystr(querystr_), + params_ok(can_use_pqexecparams(querystr_)), + nparams(nparams_), + param_lengths(lengths), + param_formats(formats), + prepared(false) +{ + while (nparams>0 && argtypecodes[nparams-1]==null_type) { + nparams--; + } + argoids = new Oid[nparams]; // hmm, use something smart + for (int i=0; i + result(PQexecPrepared(database.pgconn, statement_name.c_str(), nparams, + enc_args, param_lengths, param_formats, 1), + PQclear); + if (result) { + ExecStatusType status = PQresultStatus(result.get()); + if (status==PGRES_TUPLES_OK || status==PGRES_COMMAND_OK) { + return Result(result); + } + } + throw QueryFailed(database.pgconn, querystr); +} + + +static PGresult* wrap_PQexecParams(PGconn* conn, string command, int nparams, + const Oid* paramTypes, const char* const * paramValues, + const int* paramLengths, const int* paramFormats) +{ + string new_command; + string::size_type p=0; + while (p(command.substr(q+1,(r-q-1))); + if (n==0) { + throw "$0 not allowed"; + } + if (n>nparams) { + throw "Not enough parameters"; + } + --n; + Oid o = paramTypes[n]; + switch (o) { + case TEXTOID: { boost::scoped_array buf(new char[paramLengths[n]*2+1]); + PQescapeStringConn(conn,buf.get(),paramValues[n], + paramLengths[n],NULL); + new_command += string("\'") + buf.get() + "\'"; + break; } + case BYTEAOID: { boost::shared_ptr buf ( + PQescapeByteaConn(conn, + reinterpret_cast(paramValues[n]), + paramLengths[n],NULL), + PQfreemem); + new_command += string("\'") + + reinterpret_cast(buf.get()) + "\'"; + break; } + case INT4OID: { int32_t i = ntohl(*reinterpret_cast(paramValues[n])); + new_command += boost::lexical_cast(i); + break; } + case INT8OID: { int64_t i = ntoh64(*reinterpret_cast(paramValues[n])); + new_command += boost::lexical_cast(i); + break; } + case TIMESTAMPTZOID: throw "timestamptz not implemented"; + break; + default: throw "unrecognised oid"; + } + p = r; + } + //cout << "converted '" << command << "' to '" << new_command << "'\n"; + return PQexec(conn, new_command.c_str()); +} + + +Result QueryCore::runonce(const char* enc_args[]) +{ + if (params_ok) { + boost::shared_ptr + result(PQexecParams(database.pgconn, querystr.c_str(), nparams, + argoids, enc_args, param_lengths, param_formats, 1), + PQclear); + if (result) { + ExecStatusType status = PQresultStatus(result.get()); + if (status==PGRES_TUPLES_OK || status==PGRES_COMMAND_OK) { + return Result(result); + } + } + } else { + boost::shared_ptr + result(wrap_PQexecParams(database.pgconn, querystr, nparams, + argoids, enc_args, param_lengths, param_formats), + PQclear); + if (result) { + ExecStatusType status = PQresultStatus(result.get()); + if (status==PGRES_TUPLES_OK) { + throw StrException("Not expecting tuples in result from " + "non-pqexecparams query '"+querystr+"'"); + } + if (status==PGRES_COMMAND_OK) { + return Result(result); + } + } + } + throw QueryFailed(database.pgconn, querystr); +} + + +void QueryCore::prepare(void) +{ +//cout << "Preparing query with nparams=" << nparams << "\n"; + boost::shared_ptr + result(PQprepare(database.pgconn, statement_name.c_str(), + querystr.c_str(), nparams, argoids), + PQclear); + if (result) { + ExecStatusType status = PQresultStatus(result.get()); + if (status==PGRES_COMMAND_OK) { + prepared=true; + return; + } + } + throw QueryFailed(database.pgconn, querystr); +} + + + +}; + diff --git a/extras/anytermd/libpbe/src/Date.cc b/extras/anytermd/libpbe/src/Date.cc new file mode 100644 index 0000000000..fd1f4d1eec --- /dev/null +++ b/extras/anytermd/libpbe/src/Date.cc @@ -0,0 +1,34 @@ +// src/Date.cc +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#include "Date.hh" + +#include "DateTime.hh" + + +namespace pbe { + +int Date::day_of_week(void) const +{ + DateTime dt(*this); + return dt.day_of_week(); +} + + +}; + diff --git a/extras/anytermd/libpbe/src/Directory.cc b/extras/anytermd/libpbe/src/Directory.cc new file mode 100644 index 0000000000..b3f5e1e593 --- /dev/null +++ b/extras/anytermd/libpbe/src/Directory.cc @@ -0,0 +1,112 @@ +// src/Directory.cc +// This file is part of libpbe; see http://decimail.org +// (C) 2004 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#include "Directory.hh" + +#include "Exception.hh" + +#include + +using namespace std; + +namespace pbe { + + +Directory::Directory(string dirname_): + dirname(dirname_) +{} + + +Directory::~Directory() +{} + + +void Directory::const_iterator::operator++(void) +{ + read_next(); +} + + +bool Directory::const_iterator::operator==(const const_iterator& rhs) const +{ + if (!at_end && !rhs.at_end) { + throw "Cannot compare these Directory::const_iterators"; + } + return at_end==rhs.at_end; +} + + +Directory::const_iterator::const_iterator(void): + dir(NULL), at_end(true) +{} + + +Directory::const_iterator::const_iterator(string dirname_): + dirname(dirname_), at_end(false) +{ + dir = opendir(dirname.c_str()); + if (!dir) { + throw_ErrnoException("opendir("+dirname+")"); + } + read_next(); +} + + +Directory::const_iterator::~const_iterator() +{ + if (dir) { + int ret = closedir(dir); + if (ret==-1) { + // throw_ErrnoException("closing a directory"); + // Don't throw an exception from a destructor, in case it is being invoked + // during exception processing. + // (TODO is there a better fix for this?) + } + } +} + + +void Directory::const_iterator::read_next(void) +{ + struct dirent* ent_p = readdir(dir); + if (!ent_p) { + at_end=true; + } else { + this_entry.leafname = ent_p->d_name; + if (this_entry.leafname=="." || this_entry.leafname=="..") { + read_next(); + return; + } + this_entry.pathname = dirname + '/' + this_entry.leafname; + } +} + + +Directory::const_iterator Directory::begin(void) const +{ + return const_iterator(dirname); +} + + +Directory::const_iterator Directory::end(void) const +{ + return const_iterator(); +} + +}; + diff --git a/extras/anytermd/libpbe/src/Exception.cc b/extras/anytermd/libpbe/src/Exception.cc new file mode 100644 index 0000000000..9495ab8bdb --- /dev/null +++ b/extras/anytermd/libpbe/src/Exception.cc @@ -0,0 +1,43 @@ +// src/Exception.cc +// This file is part of libpbe; see http://decimail.org +// (C) 2004-2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#include "Exception.hh" + +#include + +using namespace std; +//using namespace pbe; + + +namespace pbe { + + +void UnknownException::report(ostream& s) const +{ + s << "Unknown Exception." << endl; +} + + +void ErrnoException::report(ostream& s) const +{ + s << strerror(error_number) << " while " << doing_what << endl; +} + + +}; + diff --git a/extras/anytermd/libpbe/src/HttpAuthenticator.cc b/extras/anytermd/libpbe/src/HttpAuthenticator.cc new file mode 100644 index 0000000000..191990afe1 --- /dev/null +++ b/extras/anytermd/libpbe/src/HttpAuthenticator.cc @@ -0,0 +1,44 @@ +// HttpAuthenticator.cc +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2005 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#include "HttpAuthenticator.hh" + +#include "base64.hh" + +using namespace std; +using namespace pbe; + + +string HttpAuthenticator::operator()(string credentials) const +{ + if (credentials.substr(0,6)!="Basic ") { + // should be case-insensitive + throw NotAuthenticated(); + } + string user_pass_b64 = credentials.substr(6); + + string user_pass = decode_base64(user_pass_b64); + unsigned int colon_pos = user_pass.find(':'); + string username = user_pass.substr(0,colon_pos); + string password = user_pass.substr(colon_pos+1); + + basic_auth(username,password); + + return username; +} diff --git a/extras/anytermd/libpbe/src/HttpClient.cc b/extras/anytermd/libpbe/src/HttpClient.cc new file mode 100644 index 0000000000..a4b9e5a38e --- /dev/null +++ b/extras/anytermd/libpbe/src/HttpClient.cc @@ -0,0 +1,202 @@ +// src/HttpClient.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#include "HttpClient.hh" + +#include "HttpRequest.hh" +#include "TcpClientSocket.hh" +#include "Gunzipper.hh" +#include "Bunzipper.hh" +#include "FileDescriptor.hh" +#include "rfcdate.hh" +#include "FileType.hh" +#include "atomic_ofstream.hh" + +#include +#include + +using namespace std; + + +namespace pbe { + + +HttpResponse HttpClient::get(const URI& uri, int redirect_hops) +{ + if (uri.scheme != "http") { + throw "Not an HTTP URI"; + } + HttpRequest req(uri); + req.headers["Date"] = rfc_date(); + req.headers["Connection"] = "close"; + req.headers["User-Agent"] = user_agent; + TcpClientSocket sock(uri.host, uri.port ? uri.port : 80); + sock.writeall(req.request_line() + req.headers_str() + "\r\n"); + HttpResponse response(sock.readall()); + if (response.status_code==301 || response.status_code==302 + || response.status_code==303 || response.status_code==307) { + if (redirect_hops<=0) { + throw "Redirection limit reached"; + } + return get(response.headers["Location"], redirect_hops-1); + } + return response; +} + + +template +void HttpClient::get_process_save(const URI& uri, std::string fn, int redirect_hops) +{ + if (uri.scheme != "http") { + throw "Not an HTTP URI"; + } + const string etag_fn = fn+".etag"; + HttpRequest req(uri); + req.headers["Date"] = rfc_date(); + req.headers["Connection"] = "close"; + req.headers["User-Agent"] = user_agent; + if (use_etag && file_exists(etag_fn) && file_exists(fn)) { + ifstream etagf(etag_fn.c_str()); + string etag; + getline(etagf,etag); + req.headers["If-None-Match"] = etag; + } + TcpClientSocket sock(uri.host, uri.port ? uri.port : 80); + sock.writeall(req.request_line() + req.headers_str() + "\r\n"); + + string resp_start; + string::iterator crlf2pos; + do { + bool timed_out = wait_until(sock.readable(), 30)==-1; + if (timed_out) { + throw TimedOut("read()"); + } + resp_start.append(sock.readsome()); + string crlf2 = "\r\n\r\n"; + crlf2pos = search(resp_start.begin(),resp_start.end(), crlf2.begin(),crlf2.end()); + } while (crlf2pos==resp_start.end()); + HttpResponse response(string(resp_start.begin(),crlf2pos+4)); + + if (use_etag && response.status_code==304) { + return; + } + + if (response.status_code==301 || response.status_code==302 + || response.status_code==303 || response.status_code==307) { + if (redirect_hops<=0) { + throw "Redirection limit reached"; + } + get_process_save(response.headers["Location"], fn, redirect_hops-1); + return; + } + + if (response.status_code != 200) { + throw response; + } + + string tmp_fn = fn+".part"; + + { + FileDescriptor fd(tmp_fn,FileDescriptor::create); + try { + Processor proc; + fd.writeall(proc(string(crlf2pos+4,resp_start.end()))); + + while (1) { + bool timed_out = wait_until(sock.readable(), 30)==-1; + if (timed_out) { + throw TimedOut("read()"); + } + string s = sock.readsome(); + if (s.empty()) { + break; + } + fd.writeall(proc(s)); + } + } + catch (...) { + unlink(tmp_fn.c_str()); + throw; + } + } + + // Ideally we should rename both the data file and the etag file atomically, but + // we can't do that. A safe alternative is to delete the old etag file first; in this + // case the worst that can happen is that we end up with a valid data file and a + // missing etag file. + + if (use_etag) { + unlink(etag_fn.c_str()); + } + + int rc = rename(tmp_fn.c_str(),fn.c_str()); + if (rc==-1) { + throw_ErrnoException("rename("+tmp_fn+","+fn+")"); + } + + if (use_etag && response.headers.find("ETag")!=response.headers.end()) { + atomic_ofstream etagf(etag_fn); + etagf << response.headers["ETag"]; + etagf.commit(); + } +} + + +struct identity_processor { + std::string operator()(string s) const { return s; } +}; + + +void HttpClient::get_save(const URI& uri, std::string fn, int redirect_hops) +{ + return get_process_save(uri,fn,redirect_hops); +} + +void HttpClient::get_save_with_etag(const URI& uri, std::string fn, int redirect_hops) +{ + return get_process_save(uri,fn,redirect_hops); +} + + +void HttpClient::get_gunzip_save(const URI& uri, std::string fn, int redirect_hops) +{ + return get_process_save(uri,fn,redirect_hops); +} + +void HttpClient::get_gunzip_save_with_etag(const URI& uri, std::string fn, int redirect_hops) +{ + return get_process_save(uri,fn,redirect_hops); +} + + +#ifdef HAVE_BZIP + +void HttpClient::get_bunzip_save(const URI& uri, std::string fn, int redirect_hops) +{ + return get_process_save(uri,fn,redirect_hops); +} + +void HttpClient::get_bunzip_save_with_etag(const URI& uri, std::string fn, int redirect_hops) +{ + return get_process_save(uri,fn,redirect_hops); +} + + +#endif + +}; diff --git a/extras/anytermd/libpbe/src/HttpDaemon.cc b/extras/anytermd/libpbe/src/HttpDaemon.cc new file mode 100644 index 0000000000..05c91fa256 --- /dev/null +++ b/extras/anytermd/libpbe/src/HttpDaemon.cc @@ -0,0 +1,124 @@ +// HttpDaemon.cc +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2005 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#include "HttpDaemon.hh" + +#include "HttpRequest.hh" +#include "parse_http_request.hh" +#include "HttpResponse.hh" +#include "rfcdate.hh" + +#include +#include + +using namespace std; + + +namespace pbe { + + +void HttpDaemon::session(FileDescriptor& in_fd, FileDescriptor& out_fd) +{ + session_start(); + +#ifndef LIBPBE_HAS_FILEDESCRIPTOR_STREAMS +#error "Sorry, there is a problem with your compiler / C++ library. Please ask for assistance" +#endif + FileDescriptor::istream in_strm(in_fd); + + bool close_connection=!keepalive; + do { + HttpRequest req; + HttpResponse resp; + try { + try { + req = parse_http_request(in_strm); + // Should look at Host: header and consider complete URIs in + // request line + if (req.http_version!="HTTP/1.1") { + // We should send a 1.0 response if the request was for 1.0; i.e. + // we should send a content-length header and not use chunked encoding. + close_connection=true; + } + if (req.headers["Connection"]=="close") { + // could be other tokens in the line. Should be case-insensitive. + close_connection=true; + } + if (req.headers.find("Host")==req.headers.end()) { + resp.status_code=400; + resp.reason_phrase="Bad Request (missing Host: header)"; + + } else if (req.method!="GET" && req.method!="POST") { + resp.status_code=405; + resp.reason_phrase="Method not allowed"; + resp.headers["Allow"]="GET POST"; + // should check for Expect: header and reject with 417 response. + + } else { + handle(req,resp); + } + } + catch (HttpRequestSyntaxError& E) { + resp.status_code=400; + resp.reason_phrase="Malformed request"; + close_connection=true; + } + catch (HttpAuthenticator::NotAuthenticated& NA) { + resp.status_code=401; + resp.reason_phrase="Unauthorised"; + resp.headers["WWW-Authenticate"]="Basic realm=\"Anyterm\""; + } + RETHROW_MISC_EXCEPTIONS; + } + catch (Exception& E) { + resp.status_code=500; + ostringstream s; + s << "Server error: "; + E.report(s); + resp.reason_phrase=s.str(); + } + if (resp.status_code!=200) { + close_connection=true; + // Actually we don't need to do this, but maybe it is safer + } + if (close_connection) { + resp.headers["Connection"]="close"; + } + resp.headers["Date"]=rfc_date(); + resp.send(out_fd); + } while (!close_connection); +} + + + +void HttpDaemon::authenticate(HttpRequest& req) +{ + if (authenticator) { + HttpRequest::headers_t::const_iterator i = req.headers.find("Authorization"); + if (i==req.headers.end()) { + throw HttpAuthenticator::NotAuthenticated(); + } + string credentials = i->second; + req.userinfo = (*authenticator)(credentials); + } +} + + +}; + diff --git a/extras/anytermd/libpbe/src/HttpResponse.cc b/extras/anytermd/libpbe/src/HttpResponse.cc new file mode 100644 index 0000000000..7398383192 --- /dev/null +++ b/extras/anytermd/libpbe/src/HttpResponse.cc @@ -0,0 +1,103 @@ +// src/HttpResponse.cc +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2005-2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#include "HttpResponse.hh" + +#include "FileDescriptor.hh" +#include "mustfind.hh" + +#include + +#include +#include + +using namespace std; + + +namespace pbe { + + +static void send_chunked(ostream& strm, string data) +{ + strm << hex << data.length() << "\r\n" + << data << "\r\n" + << "0\r\n" + << "\r\n"; +} + + +void HttpResponse::send(FileDescriptor& fd) +{ + ostringstream s; + s << http_version << ' ' << status_code << ' ' << reason_phrase << "\r\n"; + for (map::const_iterator i = headers.begin(); + i != headers.end(); ++i) { + s << i->first << ": " << i->second << "\r\n"; + } + s << "Transfer-Encoding: chunked\r\n" + << "\r\n"; + if (body.length()==0) { + s << "0\r\n\r\n"; + } else { + send_chunked(s,body); + } + + fd.writeall(s.str()); +} + + +static string get_line(string& s) +{ + string crlf = "\r\n"; + string::iterator i = mustsearch(s.begin(),s.end(),crlf.begin(),crlf.end()); + string l(s.begin(),i); + s = string(i+2,s.end()); + return l; +} + + +HttpResponse::HttpResponse(string s) +{ + const string response_line = get_line(s); + string::const_iterator sp1 = mustfind(response_line.begin(),response_line.end(),' '); + http_version = string(response_line.begin(),sp1); + string::const_iterator sp2 = mustfind(sp1+1,response_line.end(),' '); + status_code = boost::lexical_cast(string(sp1+1,sp2)); + reason_phrase = string(sp2+2,response_line.end()); + + while (1) { + const string hdr = get_line(s); + if (hdr.empty()) { + break; + } + string::const_iterator colon_pos = mustfind(hdr.begin(),hdr.end(),':'); + const string hdr_name(hdr.begin(),colon_pos); + const string hdr_val(colon_pos+1,hdr.end()); + const string trimmed_hdr_val(mustfind_if(hdr_val.begin(),hdr_val.end(), + bind2nd(not_equal_to(),' ')), + hdr_val.end()); + headers[hdr_name] = trimmed_hdr_val; + } + + body = s; +} + + +}; + diff --git a/extras/anytermd/libpbe/src/Recoder.cc b/extras/anytermd/libpbe/src/Recoder.cc new file mode 100644 index 0000000000..ce166e6466 --- /dev/null +++ b/extras/anytermd/libpbe/src/Recoder.cc @@ -0,0 +1,80 @@ +// src/Recoder.cc +// This file is part of libpbe; see http://decimail.org +// (C) 2004 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +#include "Recoder.hh" + +#include +#include + +#include + + +RECODE_OUTER Recoder::outer = NULL; + +Recoder::Recoder(string from_charset, string to_charset, ErrorLevel e): + error_level(e) +{ + if (!outer) { + outer = recode_new_outer(false); + assert(outer); + } + request = recode_new_request(outer); + assert(request); + assert(recode_scan_request(request, + string(from_charset+".."+to_charset).c_str())); +}; + + +Recoder::~Recoder() +{ + recode_delete_request(request); +} + + +string Recoder::operator()(string i) +{ + return operator()(i.data(),i.size()); +} + + +string Recoder::operator()(const char* i, int l) +{ + RECODE_TASK task = recode_new_task(request); + + task->input.name = NULL; + task->input.file = NULL; + task->input.buffer = i; + task->input.cursor = i; + task->input.limit = i+l; + + task->output.name = NULL; + task->output.file = NULL; + task->output.buffer = NULL; + task->output.cursor = NULL; + task->output.limit = NULL; + + task->fail_level = static_cast(error_level); + + assert(recode_perform_task(task)); + + string o(task->output.buffer,task->output.cursor - task->output.buffer); + free(task->output.buffer); + + recode_delete_task(task); + + return o; +} diff --git a/extras/anytermd/libpbe/src/SmtpClient.cc b/extras/anytermd/libpbe/src/SmtpClient.cc new file mode 100644 index 0000000000..e69de29bb2 diff --git a/extras/anytermd/libpbe/src/SmtpClient.cc.caca b/extras/anytermd/libpbe/src/SmtpClient.cc.caca new file mode 100644 index 0000000000..0a4933d0d1 --- /dev/null +++ b/extras/anytermd/libpbe/src/SmtpClient.cc.caca @@ -0,0 +1,147 @@ +// src/SmtpClient.cc +// This file is part of libpbe; see http://decimail.org +// (C) 2004 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +#include "SmtpClient.hh" + +#include "ip.hh" +#include "select.hh" + +#include +#include + +#ifdef __OpenBSD__ +// Is this really needed? +#include +#endif + +using namespace std; +using namespace pbe; + + + void SmtpError::report(ostream& s) const + { + s << msg; + } + + + SmtpClient::SmtpClient(bool log): enable_log(log), connected(false) + {} + + + void SmtpClient::connect(string server_name, string domain, int port) + { + fd = tcp_client_connect(server_name, port); + + wait_for_reply(220,300); + send("EHLO "+domain); + wait_for_reply(250,300); + + connected=true; + } + + + void SmtpClient::wait_for_reply(int expected_code, int timeout) + { + if (select_rt(fd,timeout)==-1) { + throw SmtpError("Timeout"); + } + char buf[513]; + int n=0; + while (1) { + int c = read(fd,buf+n,sizeof(buf)-n); + buf[n+c+1]='\0'; + if (enable_log) { + syslog(LOG_MAIL|LOG_DEBUG,"SmtpClient:S: %s",buf); + } + if (c==-1) { + throw_ErrnoException("read()"); + } + n += c; + if (buf[n-1]=='\n') { + break; + } + if (n==sizeof(buf)) { + throw SmtpError("Command line did not terminate"); + } + } + int code; + int rc = sscanf(buf,"%d",&code); + if (rc!=1) { + throw SmtpError("No reply code at start of line"); + } + if (code != expected_code) { + throw SmtpError("Unexpected reply: \'"+string(buf)+"\'"); + } + } + + + void SmtpClient::send(string d) + { + if (enable_log) { + syslog(LOG_MAIL|LOG_DEBUG,"SmtpClient:C: %s",d.c_str()); + } + d.append("\r\n"); + // ought to impose a timeout on these writes + const char* p = d.data(); + int n = d.length(); + int c = 0; + while(c recipients; + recipients.push_back(recipient); + send_msg(sender, recipients, msg); + } + + + void SmtpClient::send_msg(string sender, const list& recipients, + string msg) + { + send("MAIL FROM:<"+sender+">"); + wait_for_reply(250,300); + for (list::const_iterator i = recipients.begin(); + i != recipients.end(); ++i) { + send("RCPT TO:<"+*i+">"); + wait_for_reply(250,300); // could get 251 as well + } + send("DATA"); + wait_for_reply(354,120); + send(msg); + send("."); + wait_for_reply(250,600); + } + + + void SmtpClient::disconnect(void) + { + send("QUIT"); + wait_for_reply(221,300); + int rc = close(fd); + if (rc==-1) { + throw_ErrnoException("close()"); + } + connected=false; + } diff --git a/extras/anytermd/libpbe/src/URI.cc b/extras/anytermd/libpbe/src/URI.cc new file mode 100644 index 0000000000..f2cfd77d30 --- /dev/null +++ b/extras/anytermd/libpbe/src/URI.cc @@ -0,0 +1,120 @@ +// src/URI.cc +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2005-2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#include "URI.hh" + +#include +#include +#include +#include +#include + +#include +#include + +#include + +using namespace std; +using namespace boost::spirit; + + +namespace pbe { + + +URI::URI(string absolute_uri): + port(0) +{ + typedef rule<> rule_t; + + // URI parsing EBNF based on + // RFC2616 + // RFC2396 + // HTTP/1.1 Errata (http://skrb.org/ietf/http_errata.html) + + rule_t mark = ch_p('-') | '_' | '.' | '!' | '~' | '*' | '\'' | '(' | ')'; + + rule_t unreserved = alnum_p | mark; + + rule_t escaped = ch_p('%') >> xdigit_p >> xdigit_p; + + rule_t reserved = ch_p(';') | '/' | '?' | ':' | '@' | '&' | '=' | '+' | '$' | ','; + + rule_t pchar = unreserved | escaped | ':' | '@' | '&' | '=' | '+' | '$' | ','; + + rule_t param = *pchar; + + rule_t segment = *pchar >> *(';' >> param); + + rule_t path_segments = segment >> *('/' >> segment); + + rule_t abs_path = ( ch_p('/') >> path_segments )[assign_a(URI::abs_path)]; + + rule_t scheme = alpha_p >> *(alpha_p | digit_p | '+' | '-' | '.' ); + + rule_t userinfo = *(unreserved | escaped | ';' | ':' | '&' | '=' | '+' | '$' | ',' ); + + //rule_t domainlabel = alnum_p | alnum_p >> *(alnum_p | '-') >> alnum_p; + rule_t domainlabel = *(alnum_p | '-'); + + //rule_t toplabel = alpha_p | alpha_p >> *(alnum_p | '-') >> alnum_p; + + //rule_t hostname = *(domainlabel >> '.') >> toplabel >> !ch_p('.'); + rule_t hostname = domainlabel % ch_p('.'); + + uint_parser decimal_byte; + + rule_t ipv4address = decimal_byte >> '.' >> decimal_byte >> '.' >> + decimal_byte >> '.' >> decimal_byte; + + rule_t host = hostname | ipv4address; + + rule_t port = uint_p[assign_a(URI::port)]; + + rule_t hostport = host[assign_a(URI::host)] + >> !(':' >> port); + + rule_t server = !( !(userinfo[assign_a(URI::userinfo)] >> '@') >> hostport ); + + rule_t reg_name = +(unreserved | escaped | '$' | ',' | ';' | ':' | '@' | + '&' | '=' | '+'); + + rule_t authority = server | reg_name; + + rule_t net_path = str_p("//") >> authority >> !abs_path; + + rule_t uric = reserved | unreserved | escaped; + + rule_t query = (*uric) [assign_a(URI::query)]; + + rule_t hier_part = (net_path | abs_path) >> !('?' >> query); + + rule_t uric_no_slash = unreserved | escaped | ';' | '?' | ':' | '@' | + '&' | '=' | '+' | '$' | ','; + + rule_t opaque_part = uric_no_slash >> *uric; + + rule_t absoluteURI = scheme[assign_a(URI::scheme)] >> ':' >> (hier_part | opaque_part); + + if (!parse(absolute_uri.c_str(), absoluteURI).full) { + throw SyntaxError(); + } +} + + +}; + diff --git a/extras/anytermd/libpbe/src/charset_names.cc b/extras/anytermd/libpbe/src/charset_names.cc new file mode 100644 index 0000000000..06651b1561 --- /dev/null +++ b/extras/anytermd/libpbe/src/charset_names.cc @@ -0,0 +1,97 @@ +// charset_names.cc +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2008 Philip Endecott + +// Distributed under the Boost Software License, Version 1.0: +// +// Permission is hereby granted, free of charge, to any person or organization +// obtaining a copy of the software and accompanying documentation covered by +// this license (the "Software") to use, reproduce, display, distribute, +// execute, and transmit the Software, and to prepare derivative works of the +// Software, and to permit third-parties to whom the Software is furnished to +// do so, all subject to the following: +// +// The copyright notices in the Software and this entire statement, including +// the above license grant, this restriction and the following disclaimer, +// must be included in all copies of the Software, in whole or in part, and +// all derivative works of the Software, unless such copies or derivative +// works are solely in the form of machine-executable object code generated by +// a source language processor. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT +// SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE +// FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, +// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER +// DEALINGS IN THE SOFTWARE. + +#include "charset/charset_names.hh" + +#include +#include +#include +#include + +#include + +using namespace std; + + +namespace pbe { + +#include "generated/charset_iana_names_body.cc" +#include "generated/charset_mime_names_body.cc" + + +struct charset_lookup { + const char* name; + charset_t cs; +}; + +static const charset_lookup lookup_table[] = { +#include "generated/charset_lookup_body.cc" +// {NULL, 0} +}; + + +static bool charset_name_less_than(const charset_lookup& a, string b) +{ + return ::strcmp(a.name,b.c_str())<0; +} + +charset_t lookup_charset(const char* name) +{ + string stripped_lc_name; + bool was_digit=false; + bool uscore_pending=false; + for (const char* p = name; *p; ++p) { + if (isalnum(*p)) { + if (uscore_pending && isdigit(*p)) { + stripped_lc_name += '_'; + } + stripped_lc_name += tolower(*p); + was_digit = isdigit(*p); + uscore_pending = false; + } else if (was_digit) { + uscore_pending=true; + } + } + + const charset_lookup* lookup_table_end = + lookup_table+sizeof(lookup_table)/sizeof(charset_lookup); + const charset_lookup* l = + lower_bound(&lookup_table[0], lookup_table_end, stripped_lc_name, charset_name_less_than); + if (l==lookup_table_end || string(l->name)!=stripped_lc_name) { + throw NoSuchCharacterSet(); + } + return l->cs; +} + + +charset_t lookup_charset(string name) { + return lookup_charset(name.c_str()); +} + +}; + diff --git a/extras/anytermd/libpbe/src/ci_string.cc b/extras/anytermd/libpbe/src/ci_string.cc new file mode 100644 index 0000000000..4b89dc5998 --- /dev/null +++ b/extras/anytermd/libpbe/src/ci_string.cc @@ -0,0 +1,26 @@ +// src/ci_string.cc +// This file is part of libpbe; see http://decimail.org +// (C) 2004 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +#include "ci_string.hh" + +string to_string(const ci_string& s) { + return string(s.data(),s.size()); +} + +ci_string to_ci_string(const string& s) { + return ci_string(s.data(),s.size()); +} diff --git a/extras/anytermd/libpbe/src/create_dir.cc b/extras/anytermd/libpbe/src/create_dir.cc new file mode 100644 index 0000000000..1d6f24a2bf --- /dev/null +++ b/extras/anytermd/libpbe/src/create_dir.cc @@ -0,0 +1,68 @@ +// src/create_dir.hh +// This file is part of libpbe; see http://svn.chezphil.org/libpbe +// (C) 2009 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +// Create directories. + + +#include "create_dir.hh" + +#include "FileType.hh" + +#include + +using namespace std; + +namespace pbe { + +// Check for and create necessary parent directories. Succeeds if the +// directory already exists; fails if it is a file. + +void create_dir_with_parents(const string path, mode_t mode) +{ + string::const_iterator i = path.begin(); + while (1) { + string::const_iterator j = find(i,path.end(),'/'); + string elem(i,j); + string to_here(path.begin(),j); + if (elem=="" || elem==".") { + } else { + switch (get_link_filetype(to_here,true)) { + case does_not_exist: + create_dir(to_here,mode); + break; + case directory: + break; + default: + throw_ErrnoException("create_dir_with_parents()",EEXIST); + break; + } + } + + if (j==path.end()) { + break; + } + + i = j; + ++i; + } +} + + +}; + diff --git a/extras/anytermd/libpbe/src/csv.cc b/extras/anytermd/libpbe/src/csv.cc new file mode 100644 index 0000000000..7ad49176c8 --- /dev/null +++ b/extras/anytermd/libpbe/src/csv.cc @@ -0,0 +1,75 @@ +// csv.cc +// (C) 2008 Philip Endecott +// This file is part of libpbe. See http://svn.chezphil.org/libpbe/ +// This file is distributed under the terms of the Boost Software License v1.0. +// Please see http://www.boost.org/LICENSE_1_0.txt or the accompanying file BOOST_LICENSE. + +#include "csv.hh" + +#include +#include +#include + +using namespace std; + + +namespace pbe { + +template +static string parse_csv_field(iter& i, iter j) +{ + if (i==j) { + return ""; + } + string s; + if (*i == '"') { + // Quoted: + ++i; // initial quote + while (i != j && *i != '"') { + if (*i == '\\') { + ++i; + if (i==j) { + break; + } + } + s += *i; + ++i; + } + if (i != j) { + ++i; // final quote + } + } else { + // Unquoted: + while (i != j && *i != ',') { + if (*i == '\\') { + ++i; + if (i==j) { + break; + } + } + s += *i; + ++i; + } + } + return s; +} + + +void parse_csv_line(const string l, vector& v) +{ + v.clear(); + string::const_iterator i = l.begin(); + while (1) { + v.push_back(parse_csv_field(i, l.end())); + if (i==l.end()) { + break; + } + if (*i != ',') { + throw "expecting a comma"; + } + ++i; + } +} + + +}; diff --git a/extras/anytermd/libpbe/src/format.cc b/extras/anytermd/libpbe/src/format.cc new file mode 100644 index 0000000000..769a435201 --- /dev/null +++ b/extras/anytermd/libpbe/src/format.cc @@ -0,0 +1,54 @@ +// src/format.cc +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2009 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#include "format.hh" + +#include +#include +#include +#include + + +namespace pbe { + + +// vasprintf is a GNUism. We could achieve the same thing using vsnprintf +// in a loop for other C libraries. + +#ifdef __GLIBC__ + +std::string format(const char* fmt,...) +{ + va_list args; + va_start(args,fmt); + char* p; + int r = vasprintf(&p,fmt,args); + if (r<0) { + throw "vasprintf failed"; + } + va_end(args); + std::string s(p); + free(p); + return s; +} + +#endif + + +}; + diff --git a/extras/anytermd/libpbe/src/generated/charset_iana_names_body.cc b/extras/anytermd/libpbe/src/generated/charset_iana_names_body.cc new file mode 100644 index 0000000000..4130db46de --- /dev/null +++ b/extras/anytermd/libpbe/src/generated/charset_iana_names_body.cc @@ -0,0 +1,257 @@ +const char * const charset_iana_names[] = { + "ANSI_X3.4-1968", + "ISO_8859-1:1987", + "ISO_8859-2:1987", + "ISO_8859-3:1988", + "ISO_8859-4:1988", + "ISO_8859-5:1988", + "ISO_8859-6:1987", + "ISO_8859-7:1987", + "ISO_8859-8:1988", + "ISO_8859-9:1989", + "ISO-8859-10", + "ISO_6937-2-add", + "JIS_X0201", + "JIS_Encoding", + "Shift_JIS", + "Extended_UNIX_Code_Packed_Format_for_Japanese", + "Extended_UNIX_Code_Fixed_Width_for_Japanese", + "BS_4730", + "SEN_850200_C", + "IT", + "ES", + "DIN_66003", + "NS_4551-1", + "NF_Z_62-010", + "ISO-10646-UTF-1", + "ISO_646.basic:1983", + "INVARIANT", + "ISO_646.irv:1983", + "NATS-SEFI", + "NATS-SEFI-ADD", + "NATS-DANO", + "NATS-DANO-ADD", + "SEN_850200_B", + "KS_C_5601-1987", + "ISO-2022-KR", + "EUC-KR", + "ISO-2022-JP", + "ISO-2022-JP-2", + "JIS_C6220-1969-jp", + "JIS_C6220-1969-ro", + "PT", + "greek7-old", + "latin-greek", + "NF_Z_62-010_(1973)", + "Latin-greek-1", + "ISO_5427", + "JIS_C6226-1978", + "BS_viewdata", + "INIS", + "INIS-8", + "INIS-cyrillic", + "ISO_5427:1981", + "ISO_5428:1980", + "GB_1988-80", + "GB_2312-80", + "NS_4551-2", + "videotex-suppl", + "PT2", + "ES2", + "MSZ_7795.3", + "JIS_C6226-1983", + "greek7", + "ASMO_449", + "iso-ir-90", + "JIS_C6229-1984-a", + "JIS_C6229-1984-b", + "JIS_C6229-1984-b-add", + "JIS_C6229-1984-hand", + "JIS_C6229-1984-hand-add", + "JIS_C6229-1984-kana", + "ISO_2033-1983", + "ANSI_X3.110-1983", + "T.61-7bit", + "T.61-8bit", + "ECMA-cyrillic", + "CSA_Z243.4-1985-1", + "CSA_Z243.4-1985-2", + "CSA_Z243.4-1985-gr", + "ISO_8859-6-E", + "ISO_8859-6-I", + "T.101-G2", + "ISO_8859-8-E", + "ISO_8859-8-I", + "CSN_369103", + "JUS_I.B1.002", + "IEC_P27-1", + "JUS_I.B1.003-serb", + "JUS_I.B1.003-mac", + "greek-ccitt", + "NC_NC00-10:81", + "ISO_6937-2-25", + "GOST_19768-74", + "ISO_8859-supp", + "ISO_10367-box", + "latin-lap", + "JIS_X0212-1990", + "DS_2089", + "us-dk", + "dk-us", + "KSC5636", + "UNICODE-1-1-UTF-7", + "ISO-2022-CN", + "ISO-2022-CN-EXT", + "UTF-8", + "ISO-8859-13", + "ISO-8859-14", + "ISO-8859-15", + "ISO-8859-16", + "GBK", + "GB18030", + "OSD_EBCDIC_DF04_15", + "OSD_EBCDIC_DF03_IRV", + "OSD_EBCDIC_DF04_1", + "ISO-11548-1", + "KZ-1048", + "ISO-10646-UCS-2", + "ISO-10646-UCS-4", + "ISO-10646-UCS-Basic", + "ISO-10646-Unicode-Latin1", + "ISO-10646-J-1", + "ISO-Unicode-IBM-1261", + "ISO-Unicode-IBM-1268", + "ISO-Unicode-IBM-1276", + "ISO-Unicode-IBM-1264", + "ISO-Unicode-IBM-1265", + "UNICODE-1-1", + "SCSU", + "UTF-7", + "UTF-16BE", + "UTF-16LE", + "UTF-16", + "CESU-8", + "UTF-32", + "UTF-32BE", + "UTF-32LE", + "BOCU-1", + "ISO-8859-1-Windows-3.0-Latin-1", + "ISO-8859-1-Windows-3.1-Latin-1", + "ISO-8859-2-Windows-Latin-2", + "ISO-8859-9-Windows-Latin-5", + "hp-roman8", + "Adobe-Standard-Encoding", + "Ventura-US", + "Ventura-International", + "DEC-MCS", + "IBM850", + "PC8-Danish-Norwegian", + "IBM862", + "PC8-Turkish", + "IBM-Symbols", + "IBM-Thai", + "HP-Legal", + "HP-Pi-font", + "HP-Math8", + "Adobe-Symbol-Encoding", + "HP-DeskTop", + "Ventura-Math", + "Microsoft-Publishing", + "Windows-31J", + "GB2312", + "Big5", + "macintosh", + "IBM037", + "IBM038", + "IBM273", + "IBM274", + "IBM275", + "IBM277", + "IBM278", + "IBM280", + "IBM281", + "IBM284", + "IBM285", + "IBM290", + "IBM297", + "IBM420", + "IBM423", + "IBM424", + "IBM437", + "IBM500", + "IBM851", + "IBM852", + "IBM855", + "IBM857", + "IBM860", + "IBM861", + "IBM863", + "IBM864", + "IBM865", + "IBM868", + "IBM869", + "IBM870", + "IBM871", + "IBM880", + "IBM891", + "IBM903", + "IBM904", + "IBM905", + "IBM918", + "IBM1026", + "EBCDIC-AT-DE", + "EBCDIC-AT-DE-A", + "EBCDIC-CA-FR", + "EBCDIC-DK-NO", + "EBCDIC-DK-NO-A", + "EBCDIC-FI-SE", + "EBCDIC-FI-SE-A", + "EBCDIC-FR", + "EBCDIC-IT", + "EBCDIC-PT", + "EBCDIC-ES", + "EBCDIC-ES-A", + "EBCDIC-ES-S", + "EBCDIC-UK", + "EBCDIC-US", + "UNKNOWN-8BIT", + "MNEMONIC", + "MNEM", + "VISCII", + "VIQR", + "KOI8-R", + "HZ-GB-2312", + "IBM866", + "IBM775", + "KOI8-U", + "IBM00858", + "IBM00924", + "IBM01140", + "IBM01141", + "IBM01142", + "IBM01143", + "IBM01144", + "IBM01145", + "IBM01146", + "IBM01147", + "IBM01148", + "IBM01149", + "Big5-HKSCS", + "IBM1047", + "PTCP154", + "Amiga-1251", + "KOI7-switched", + "BRF", + "TSCII", + "windows-1250", + "windows-1251", + "windows-1252", + "windows-1253", + "windows-1254", + "windows-1255", + "windows-1256", + "windows-1257", + "windows-1258", + "TIS-620", + NULL +}; diff --git a/extras/anytermd/libpbe/src/generated/charset_lookup_body.cc b/extras/anytermd/libpbe/src/generated/charset_lookup_body.cc new file mode 100644 index 0000000000..70e61f2957 --- /dev/null +++ b/extras/anytermd/libpbe/src/generated/charset_lookup_body.cc @@ -0,0 +1,807 @@ + {"437", cs::ibm437}, + {"850", cs::ibm850}, + {"851", cs::ibm851}, + {"852", cs::ibm852}, + {"855", cs::ibm855}, + {"857", cs::ibm857}, + {"860", cs::ibm860}, + {"861", cs::ibm861}, + {"862", cs::ibm862}, + {"863", cs::ibm863}, + {"865", cs::ibm865}, + {"866", cs::ibm866}, + {"869", cs::ibm869}, + {"904", cs::ibm904}, + {"adobestandardencoding", cs::adobe_standard_encoding}, + {"adobesymbolencoding", cs::adobe_symbol_encoding}, + {"ami1251", cs::amiga_1251}, + {"amiga1251", cs::amiga_1251}, + {"ansix3_110_1983", cs::ansi_x3_110_1983}, + {"ansix3_4_1968", cs::ansi_x3_4_1968}, + {"ansix3_4_1986", cs::ansi_x3_4_1968}, + {"arabic", cs::iso_8859_6_1987}, + {"arabic7", cs::asmo_449}, + {"ascii", cs::ansi_x3_4_1968}, + {"asmo449", cs::asmo_449}, + {"asmo708", cs::iso_8859_6_1987}, + {"big5", cs::big5}, + {"big5hkscs", cs::big5_hkscs}, + {"bocu1", cs::bocu_1}, + {"brf", cs::brf}, + {"bs4730", cs::bs_4730}, + {"bsviewdata", cs::bs_viewdata}, + {"ca", cs::csa_z243_4_1985_1}, + {"ccsid00858", cs::ibm00858}, + {"ccsid00924", cs::ibm00924}, + {"ccsid01140", cs::ibm01140}, + {"ccsid01141", cs::ibm01141}, + {"ccsid01142", cs::ibm01142}, + {"ccsid01143", cs::ibm01143}, + {"ccsid01144", cs::ibm01144}, + {"ccsid01145", cs::ibm01145}, + {"ccsid01146", cs::ibm01146}, + {"ccsid01147", cs::ibm01147}, + {"ccsid01148", cs::ibm01148}, + {"ccsid01149", cs::ibm01149}, + {"cesu8", cs::cesu_8}, + {"chinese", cs::gb_2312_80}, + {"cn", cs::gb_1988_80}, + {"cp00858", cs::ibm00858}, + {"cp00924", cs::ibm00924}, + {"cp01140", cs::ibm01140}, + {"cp01141", cs::ibm01141}, + {"cp01142", cs::ibm01142}, + {"cp01143", cs::ibm01143}, + {"cp01144", cs::ibm01144}, + {"cp01145", cs::ibm01145}, + {"cp01146", cs::ibm01146}, + {"cp01147", cs::ibm01147}, + {"cp01148", cs::ibm01148}, + {"cp01149", cs::ibm01149}, + {"cp037", cs::ibm037}, + {"cp038", cs::ibm038}, + {"cp1026", cs::ibm1026}, + {"cp154", cs::ptcp154}, + {"cp273", cs::ibm273}, + {"cp274", cs::ibm274}, + {"cp275", cs::ibm275}, + {"cp278", cs::ibm278}, + {"cp280", cs::ibm280}, + {"cp281", cs::ibm281}, + {"cp284", cs::ibm284}, + {"cp285", cs::ibm285}, + {"cp290", cs::ibm290}, + {"cp297", cs::ibm297}, + {"cp367", cs::ansi_x3_4_1968}, + {"cp420", cs::ibm420}, + {"cp423", cs::ibm423}, + {"cp424", cs::ibm424}, + {"cp437", cs::ibm437}, + {"cp500", cs::ibm500}, + {"cp775", cs::ibm775}, + {"cp819", cs::iso_8859_1_1987}, + {"cp850", cs::ibm850}, + {"cp851", cs::ibm851}, + {"cp852", cs::ibm852}, + {"cp855", cs::ibm855}, + {"cp857", cs::ibm857}, + {"cp860", cs::ibm860}, + {"cp861", cs::ibm861}, + {"cp862", cs::ibm862}, + {"cp863", cs::ibm863}, + {"cp864", cs::ibm864}, + {"cp865", cs::ibm865}, + {"cp866", cs::ibm866}, + {"cp868", cs::ibm868}, + {"cp869", cs::ibm869}, + {"cp870", cs::ibm870}, + {"cp871", cs::ibm871}, + {"cp880", cs::ibm880}, + {"cp891", cs::ibm891}, + {"cp903", cs::ibm903}, + {"cp904", cs::ibm904}, + {"cp905", cs::ibm905}, + {"cp918", cs::ibm918}, + {"cp936", cs::gbk}, + {"cpar", cs::ibm868}, + {"cpgr", cs::ibm869}, + {"cpis", cs::ibm861}, + {"csa7_1", cs::csa_z243_4_1985_1}, + {"csa7_2", cs::csa_z243_4_1985_2}, + {"csadobestandardencoding", cs::adobe_standard_encoding}, + {"csascii", cs::ansi_x3_4_1968}, + {"csat500_1983", cs::ansi_x3_110_1983}, + {"csaz243_4_1985_1", cs::csa_z243_4_1985_1}, + {"csaz243_4_1985_2", cs::csa_z243_4_1985_2}, + {"csaz243_4_1985gr", cs::csa_z243_4_1985_gr}, + {"csbig5", cs::big5}, + {"csbocu1", cs::bocu_1}, + {"csbrf", cs::brf}, + {"cscesu8", cs::cesu_8}, + {"csdecmcs", cs::dec_mcs}, + {"csdkus", cs::dk_us}, + {"csebcdicatdea", cs::ebcdic_at_de_a}, + {"csebcdiccafr", cs::ebcdic_ca_fr}, + {"csebcdicdkno", cs::ebcdic_dk_no}, + {"csebcdicdknoa", cs::ebcdic_dk_no_a}, + {"csebcdices", cs::ebcdic_es}, + {"csebcdicesa", cs::ebcdic_es_a}, + {"csebcdicess", cs::ebcdic_es_s}, + {"csebcdicfise", cs::ebcdic_fi_se}, + {"csebcdicfisea", cs::ebcdic_fi_se_a}, + {"csebcdicfr", cs::ebcdic_fr}, + {"csebcdicit", cs::ebcdic_it}, + {"csebcdicpt", cs::ebcdic_pt}, + {"csebcdicuk", cs::ebcdic_uk}, + {"csebcdicus", cs::ebcdic_us}, + {"cseucfixwidjapanese", cs::extended_unix_code_fixed_width_for_japanese}, + {"cseuckr", cs::euc_kr}, + {"cseucpkdfmtjapanese", cs::extended_unix_code_packed_format_for_japanese}, + {"csgb2312", cs::gb2312}, + {"cshalfwidthkatakana", cs::jis_x0201}, + {"cshpdesktop", cs::hp_desktop}, + {"cshplegal", cs::hp_legal}, + {"cshpmath8", cs::hp_math8}, + {"cshppifont", cs::hp_pi_font}, + {"cshppsmath", cs::adobe_symbol_encoding}, + {"cshproman8", cs::hp_roman8}, + {"csibbm904", cs::ibm904}, + {"csibm037", cs::ibm037}, + {"csibm038", cs::ibm038}, + {"csibm1026", cs::ibm1026}, + {"csibm273", cs::ibm273}, + {"csibm274", cs::ibm274}, + {"csibm275", cs::ibm275}, + {"csibm277", cs::ibm277}, + {"csibm278", cs::ibm278}, + {"csibm280", cs::ibm280}, + {"csibm281", cs::ibm281}, + {"csibm284", cs::ibm284}, + {"csibm285", cs::ibm285}, + {"csibm290", cs::ibm290}, + {"csibm297", cs::ibm297}, + {"csibm420", cs::ibm420}, + {"csibm423", cs::ibm423}, + {"csibm424", cs::ibm424}, + {"csibm500", cs::ibm500}, + {"csibm851", cs::ibm851}, + {"csibm855", cs::ibm855}, + {"csibm857", cs::ibm857}, + {"csibm860", cs::ibm860}, + {"csibm861", cs::ibm861}, + {"csibm863", cs::ibm863}, + {"csibm864", cs::ibm864}, + {"csibm865", cs::ibm865}, + {"csibm866", cs::ibm866}, + {"csibm868", cs::ibm868}, + {"csibm869", cs::ibm869}, + {"csibm870", cs::ibm870}, + {"csibm871", cs::ibm871}, + {"csibm880", cs::ibm880}, + {"csibm891", cs::ibm891}, + {"csibm903", cs::ibm903}, + {"csibm905", cs::ibm905}, + {"csibm918", cs::ibm918}, + {"csibmebcdicatde", cs::ebcdic_at_de}, + {"csibmsymbols", cs::ibm_symbols}, + {"csibmthai", cs::ibm_thai}, + {"csinvariant", cs::invariant}, + {"csiso102t617bit", cs::t_61_7bit}, + {"csiso10367box", cs::iso_10367_box}, + {"csiso103t618bit", cs::t_61_8bit}, + {"csiso10646utf1", cs::iso_10646_utf_1}, + {"csiso10swedish", cs::sen_850200_b}, + {"csiso111ecmacyrillic", cs::ecma_cyrillic}, + {"csiso115481", cs::iso_11548_1}, + {"csiso11swedishfornames", cs::sen_850200_c}, + {"csiso121canadian1", cs::csa_z243_4_1985_1}, + {"csiso122canadian2", cs::csa_z243_4_1985_2}, + {"csiso123csaz24341985gr", cs::csa_z243_4_1985_gr}, + {"csiso128t101g2", cs::t_101_g2}, + {"csiso139csn369103", cs::csn_369103}, + {"csiso13jisc6220jp", cs::jis_c6220_1969_jp}, + {"csiso141jusib1002", cs::jus_i_b1_002}, + {"csiso143iecp271", cs::iec_p27_1}, + {"csiso146serbian", cs::jus_i_b1_003_serb}, + {"csiso147macedonian", cs::jus_i_b1_003_mac}, + {"csiso14jisc6220ro", cs::jis_c6220_1969_ro}, + {"csiso150", cs::greek_ccitt}, + {"csiso150greekccitt", cs::greek_ccitt}, + {"csiso151cuba", cs::nc_nc00_10_81}, + {"csiso153gost1976874", cs::gost_19768_74}, + {"csiso158lap", cs::latin_lap}, + {"csiso159jisx02121990", cs::jis_x0212_1990}, + {"csiso15italian", cs::it}, + {"csiso16portuguese", cs::pt}, + {"csiso17spanish", cs::es}, + {"csiso18greek7old", cs::greek7_old}, + {"csiso19latingreek", cs::latin_greek}, + {"csiso2022jp", cs::iso_2022_jp}, + {"csiso2022jp2", cs::iso_2022_jp_2}, + {"csiso2022kr", cs::iso_2022_kr}, + {"csiso2033", cs::iso_2033_1983}, + {"csiso21german", cs::din_66003}, + {"csiso25french", cs::nf_z_62_010__1973_}, + {"csiso27latingreek1", cs::latin_greek_1}, + {"csiso2intlrefversion", cs::iso_646_irv_1983}, + {"csiso42jisc62261978", cs::jis_c6226_1978}, + {"csiso47bsviewdata", cs::bs_viewdata}, + {"csiso49inis", cs::inis}, + {"csiso4unitedkingdom", cs::bs_4730}, + {"csiso50inis8", cs::inis_8}, + {"csiso51iniscyrillic", cs::inis_cyrillic}, + {"csiso5427cyrillic", cs::iso_5427}, + {"csiso5428greek", cs::iso_5428_1980}, + {"csiso57gb1988", cs::gb_1988_80}, + {"csiso58gb231280", cs::gb_2312_80}, + {"csiso60danishnorwegian", cs::ns_4551_1}, + {"csiso60norwegian1", cs::ns_4551_1}, + {"csiso61norwegian2", cs::ns_4551_2}, + {"csiso646basic1983", cs::iso_646_basic_1983}, + {"csiso646danish", cs::ds_2089}, + {"csiso6937add", cs::iso_6937_2_25}, + {"csiso69french", cs::nf_z_62_010}, + {"csiso70videotexsupp1", cs::videotex_suppl}, + {"csiso84portuguese2", cs::pt2}, + {"csiso85spanish2", cs::es2}, + {"csiso86hungarian", cs::msz_7795_3}, + {"csiso87jisx0208", cs::jis_c6226_1983}, + {"csiso88596e", cs::iso_8859_6_e}, + {"csiso88596i", cs::iso_8859_6_i}, + {"csiso88598e", cs::iso_8859_8_e}, + {"csiso88598i", cs::iso_8859_8_i}, + {"csiso8859supp", cs::iso_8859_supp}, + {"csiso88greek7", cs::greek7}, + {"csiso89asmo449", cs::asmo_449}, + {"csiso90", cs::iso_ir_90}, + {"csiso91jisc62291984a", cs::jis_c6229_1984_a}, + {"csiso92jisc62991984b", cs::jis_c6229_1984_b}, + {"csiso93jis62291984badd", cs::jis_c6229_1984_b_add}, + {"csiso94jis62291984hand", cs::jis_c6229_1984_hand}, + {"csiso95jis62291984handadd", cs::jis_c6229_1984_hand_add}, + {"csiso96jisc62291984kana", cs::jis_c6229_1984_kana}, + {"csiso99naplps", cs::ansi_x3_110_1983}, + {"csisolatin1", cs::iso_8859_1_1987}, + {"csisolatin2", cs::iso_8859_2_1987}, + {"csisolatin3", cs::iso_8859_3_1988}, + {"csisolatin4", cs::iso_8859_4_1988}, + {"csisolatin5", cs::iso_8859_9_1989}, + {"csisolatin6", cs::iso_8859_10}, + {"csisolatinarabic", cs::iso_8859_6_1987}, + {"csisolatincyrillic", cs::iso_8859_5_1988}, + {"csisolatingreek", cs::iso_8859_7_1987}, + {"csisolatinhebrew", cs::iso_8859_8_1988}, + {"csisotextcomm", cs::iso_6937_2_add}, + {"csjisencoding", cs::jis_encoding}, + {"cskoi8r", cs::koi8_r}, + {"csksc56011987", cs::ks_c_5601_1987}, + {"csksc5636", cs::ksc5636}, + {"cskz1048", cs::kz_1048}, + {"csmacintosh", cs::macintosh}, + {"csmicrosoftpublishing", cs::microsoft_publishing}, + {"csmnem", cs::mnem}, + {"csmnemonic", cs::mnemonic}, + {"csn369103", cs::csn_369103}, + {"csnatsdano", cs::nats_dano}, + {"csnatsdanoadd", cs::nats_dano_add}, + {"csnatssefi", cs::nats_sefi}, + {"csnatssefiadd", cs::nats_sefi_add}, + {"cspc775baltic", cs::ibm775}, + {"cspc850multilingual", cs::ibm850}, + {"cspc862latinhebrew", cs::ibm862}, + {"cspc8codepage437", cs::ibm437}, + {"cspc8danishnorwegian", cs::pc8_danish_norwegian}, + {"cspc8turkish", cs::pc8_turkish}, + {"cspcp852", cs::ibm852}, + {"csptcp154", cs::ptcp154}, + {"csshiftjis", cs::shift_jis}, + {"cstscii", cs::tscii}, + {"csucs4", cs::iso_10646_ucs_4}, + {"csunicode", cs::iso_10646_ucs_2}, + {"csunicode11", cs::unicode_1_1}, + {"csunicode11utf7", cs::unicode_1_1_utf_7}, + {"csunicodeascii", cs::iso_10646_ucs_basic}, + {"csunicodeibm1261", cs::iso_unicode_ibm_1261}, + {"csunicodeibm1264", cs::iso_unicode_ibm_1264}, + {"csunicodeibm1265", cs::iso_unicode_ibm_1265}, + {"csunicodeibm1268", cs::iso_unicode_ibm_1268}, + {"csunicodeibm1276", cs::iso_unicode_ibm_1276}, + {"csunicodelatin1", cs::iso_10646_unicode_latin1}, + {"csunknown8bit", cs::unknown_8bit}, + {"csusdk", cs::us_dk}, + {"csventurainternational", cs::ventura_international}, + {"csventuramath", cs::ventura_math}, + {"csventuraus", cs::ventura_us}, + {"csviqr", cs::viqr}, + {"csviscii", cs::viscii}, + {"cswindows30latin1", cs::iso_8859_1_windows_3_0_latin_1}, + {"cswindows31j", cs::windows_31j}, + {"cswindows31latin1", cs::iso_8859_1_windows_3_1_latin_1}, + {"cswindows31latin2", cs::iso_8859_2_windows_latin_2}, + {"cswindows31latin5", cs::iso_8859_9_windows_latin_5}, + {"cuba", cs::nc_nc00_10_81}, + {"cyrillic", cs::iso_8859_5_1988}, + {"cyrillicasian", cs::ptcp154}, + {"de", cs::din_66003}, + {"dec", cs::dec_mcs}, + {"decmcs", cs::dec_mcs}, + {"din66003", cs::din_66003}, + {"dk", cs::ds_2089}, + {"dkus", cs::dk_us}, + {"ds2089", cs::ds_2089}, + {"e13b", cs::iso_2033_1983}, + {"ebcdicatde", cs::ebcdic_at_de}, + {"ebcdicatdea", cs::ebcdic_at_de_a}, + {"ebcdicbe", cs::ibm274}, + {"ebcdicbr", cs::ibm275}, + {"ebcdiccafr", cs::ebcdic_ca_fr}, + {"ebcdiccpar1", cs::ibm420}, + {"ebcdiccpar2", cs::ibm918}, + {"ebcdiccpbe", cs::ibm500}, + {"ebcdiccpca", cs::ibm037}, + {"ebcdiccpch", cs::ibm500}, + {"ebcdiccpdk", cs::ibm277}, + {"ebcdiccpes", cs::ibm284}, + {"ebcdiccpfi", cs::ibm278}, + {"ebcdiccpfr", cs::ibm297}, + {"ebcdiccpgb", cs::ibm285}, + {"ebcdiccpgr", cs::ibm423}, + {"ebcdiccphe", cs::ibm424}, + {"ebcdiccpis", cs::ibm871}, + {"ebcdiccpit", cs::ibm280}, + {"ebcdiccpnl", cs::ibm037}, + {"ebcdiccpno", cs::ibm277}, + {"ebcdiccproece", cs::ibm870}, + {"ebcdiccpse", cs::ibm278}, + {"ebcdiccptr", cs::ibm905}, + {"ebcdiccpus", cs::ibm037}, + {"ebcdiccpwt", cs::ibm037}, + {"ebcdiccpyu", cs::ibm870}, + {"ebcdiccyrillic", cs::ibm880}, + {"ebcdicde273euro", cs::ibm01141}, + {"ebcdicdk277euro", cs::ibm01142}, + {"ebcdicdkno", cs::ebcdic_dk_no}, + {"ebcdicdknoa", cs::ebcdic_dk_no_a}, + {"ebcdices", cs::ebcdic_es}, + {"ebcdices284euro", cs::ibm01145}, + {"ebcdicesa", cs::ebcdic_es_a}, + {"ebcdicess", cs::ebcdic_es_s}, + {"ebcdicfi278euro", cs::ibm01143}, + {"ebcdicfise", cs::ebcdic_fi_se}, + {"ebcdicfisea", cs::ebcdic_fi_se_a}, + {"ebcdicfr", cs::ebcdic_fr}, + {"ebcdicfr297euro", cs::ibm01147}, + {"ebcdicgb285euro", cs::ibm01146}, + {"ebcdicint", cs::ibm038}, + {"ebcdicinternational500euro", cs::ibm01148}, + {"ebcdicis871euro", cs::ibm01149}, + {"ebcdicit", cs::ebcdic_it}, + {"ebcdicit280euro", cs::ibm01144}, + {"ebcdicjpe", cs::ibm281}, + {"ebcdicjpkana", cs::ibm290}, + {"ebcdiclatin9euro", cs::ibm00924}, + {"ebcdicno277euro", cs::ibm01142}, + {"ebcdicpt", cs::ebcdic_pt}, + {"ebcdicse278euro", cs::ibm01143}, + {"ebcdicuk", cs::ebcdic_uk}, + {"ebcdicus", cs::ebcdic_us}, + {"ebcdicus37euro", cs::ibm01140}, + {"ecma114", cs::iso_8859_6_1987}, + {"ecma118", cs::iso_8859_7_1987}, + {"ecmacyrillic", cs::ecma_cyrillic}, + {"elot928", cs::iso_8859_7_1987}, + {"es", cs::es}, + {"es2", cs::es2}, + {"eucjp", cs::extended_unix_code_packed_format_for_japanese}, + {"euckr", cs::euc_kr}, + {"extendedunixcodefixedwidthforjapanese", cs::extended_unix_code_fixed_width_for_japanese}, + {"extendedunixcodepackedformatforjapanese", cs::extended_unix_code_packed_format_for_japanese}, + {"fi", cs::sen_850200_b}, + {"fr", cs::nf_z_62_010}, + {"gb", cs::bs_4730}, + {"gb18030", cs::gb18030}, + {"gb1988_80", cs::gb_1988_80}, + {"gb2312", cs::gb2312}, + {"gb2312_80", cs::gb_2312_80}, + {"gbk", cs::gbk}, + {"gost19768_74", cs::gost_19768_74}, + {"greek", cs::iso_8859_7_1987}, + {"greek7", cs::greek7}, + {"greek7old", cs::greek7_old}, + {"greek8", cs::iso_8859_7_1987}, + {"greekccitt", cs::greek_ccitt}, + {"hebrew", cs::iso_8859_8_1988}, + {"hpdesktop", cs::hp_desktop}, + {"hplegal", cs::hp_legal}, + {"hpmath8", cs::hp_math8}, + {"hppifont", cs::hp_pi_font}, + {"hproman8", cs::hp_roman8}, + {"hu", cs::msz_7795_3}, + {"hzgb2312", cs::hz_gb_2312}, + {"ibm00858", cs::ibm00858}, + {"ibm00924", cs::ibm00924}, + {"ibm01140", cs::ibm01140}, + {"ibm01141", cs::ibm01141}, + {"ibm01142", cs::ibm01142}, + {"ibm01143", cs::ibm01143}, + {"ibm01144", cs::ibm01144}, + {"ibm01145", cs::ibm01145}, + {"ibm01146", cs::ibm01146}, + {"ibm01147", cs::ibm01147}, + {"ibm01148", cs::ibm01148}, + {"ibm01149", cs::ibm01149}, + {"ibm037", cs::ibm037}, + {"ibm038", cs::ibm038}, + {"ibm1026", cs::ibm1026}, + {"ibm1047", cs::ibm1047}, + {"ibm273", cs::ibm273}, + {"ibm274", cs::ibm274}, + {"ibm275", cs::ibm275}, + {"ibm277", cs::ibm277}, + {"ibm278", cs::ibm278}, + {"ibm280", cs::ibm280}, + {"ibm281", cs::ibm281}, + {"ibm284", cs::ibm284}, + {"ibm285", cs::ibm285}, + {"ibm290", cs::ibm290}, + {"ibm297", cs::ibm297}, + {"ibm367", cs::ansi_x3_4_1968}, + {"ibm420", cs::ibm420}, + {"ibm423", cs::ibm423}, + {"ibm424", cs::ibm424}, + {"ibm437", cs::ibm437}, + {"ibm500", cs::ibm500}, + {"ibm775", cs::ibm775}, + {"ibm819", cs::iso_8859_1_1987}, + {"ibm850", cs::ibm850}, + {"ibm851", cs::ibm851}, + {"ibm852", cs::ibm852}, + {"ibm855", cs::ibm855}, + {"ibm857", cs::ibm857}, + {"ibm860", cs::ibm860}, + {"ibm861", cs::ibm861}, + {"ibm862", cs::ibm862}, + {"ibm863", cs::ibm863}, + {"ibm864", cs::ibm864}, + {"ibm865", cs::ibm865}, + {"ibm866", cs::ibm866}, + {"ibm868", cs::ibm868}, + {"ibm869", cs::ibm869}, + {"ibm870", cs::ibm870}, + {"ibm871", cs::ibm871}, + {"ibm880", cs::ibm880}, + {"ibm891", cs::ibm891}, + {"ibm903", cs::ibm903}, + {"ibm904", cs::ibm904}, + {"ibm905", cs::ibm905}, + {"ibm918", cs::ibm918}, + {"ibmsymbols", cs::ibm_symbols}, + {"ibmthai", cs::ibm_thai}, + {"iecp27_1", cs::iec_p27_1}, + {"inis", cs::inis}, + {"inis8", cs::inis_8}, + {"iniscyrillic", cs::inis_cyrillic}, + {"invariant", cs::invariant}, + {"irv", cs::iso_646_irv_1983}, + {"iso10367box", cs::iso_10367_box}, + {"iso10646", cs::iso_10646_unicode_latin1}, + {"iso10646j1", cs::iso_10646_j_1}, + {"iso10646ucs2", cs::iso_10646_ucs_2}, + {"iso10646ucs4", cs::iso_10646_ucs_4}, + {"iso10646ucsbasic", cs::iso_10646_ucs_basic}, + {"iso10646unicodelatin1", cs::iso_10646_unicode_latin1}, + {"iso10646utf1", cs::iso_10646_utf_1}, + {"iso11548_1", cs::iso_11548_1}, + {"iso2022cn", cs::iso_2022_cn}, + {"iso2022cnext", cs::iso_2022_cn_ext}, + {"iso2022jp", cs::iso_2022_jp}, + {"iso2022jp2", cs::iso_2022_jp_2}, + {"iso2022kr", cs::iso_2022_kr}, + {"iso2033_1983", cs::iso_2033_1983}, + {"iso5427", cs::iso_5427}, + {"iso5427_1981", cs::iso_5427_1981}, + {"iso5427cyrillic1981", cs::iso_5427_1981}, + {"iso5428_1980", cs::iso_5428_1980}, + {"iso646basic1983", cs::iso_646_basic_1983}, + {"iso646ca", cs::csa_z243_4_1985_1}, + {"iso646ca2", cs::csa_z243_4_1985_2}, + {"iso646cn", cs::gb_1988_80}, + {"iso646cu", cs::nc_nc00_10_81}, + {"iso646de", cs::din_66003}, + {"iso646dk", cs::ds_2089}, + {"iso646es", cs::es}, + {"iso646es2", cs::es2}, + {"iso646fi", cs::sen_850200_b}, + {"iso646fr", cs::nf_z_62_010}, + {"iso646fr1", cs::nf_z_62_010__1973_}, + {"iso646gb", cs::bs_4730}, + {"iso646hu", cs::msz_7795_3}, + {"iso646irv1983", cs::iso_646_irv_1983}, + {"iso646irv1991", cs::ansi_x3_4_1968}, + {"iso646it", cs::it}, + {"iso646jp", cs::jis_c6220_1969_ro}, + {"iso646jpocrb", cs::jis_c6229_1984_b}, + {"iso646kr", cs::ksc5636}, + {"iso646no", cs::ns_4551_1}, + {"iso646no2", cs::ns_4551_2}, + {"iso646pt", cs::pt}, + {"iso646pt2", cs::pt2}, + {"iso646se", cs::sen_850200_b}, + {"iso646se2", cs::sen_850200_c}, + {"iso646us", cs::ansi_x3_4_1968}, + {"iso646yu", cs::jus_i_b1_002}, + {"iso6937_2_25", cs::iso_6937_2_25}, + {"iso6937_2add", cs::iso_6937_2_add}, + {"iso8859_1", cs::iso_8859_1_1987}, + {"iso8859_10", cs::iso_8859_10}, + {"iso8859_10_1992", cs::iso_8859_10}, + {"iso8859_13", cs::iso_8859_13}, + {"iso8859_14", cs::iso_8859_14}, + {"iso8859_14_1998", cs::iso_8859_14}, + {"iso8859_15", cs::iso_8859_15}, + {"iso8859_16", cs::iso_8859_16}, + {"iso8859_16_2001", cs::iso_8859_16}, + {"iso8859_1_1987", cs::iso_8859_1_1987}, + {"iso8859_1windows3_0latin1", cs::iso_8859_1_windows_3_0_latin_1}, + {"iso8859_1windows3_1latin1", cs::iso_8859_1_windows_3_1_latin_1}, + {"iso8859_2", cs::iso_8859_2_1987}, + {"iso8859_2_1987", cs::iso_8859_2_1987}, + {"iso8859_2windowslatin2", cs::iso_8859_2_windows_latin_2}, + {"iso8859_3", cs::iso_8859_3_1988}, + {"iso8859_3_1988", cs::iso_8859_3_1988}, + {"iso8859_4", cs::iso_8859_4_1988}, + {"iso8859_4_1988", cs::iso_8859_4_1988}, + {"iso8859_5", cs::iso_8859_5_1988}, + {"iso8859_5_1988", cs::iso_8859_5_1988}, + {"iso8859_6", cs::iso_8859_6_1987}, + {"iso8859_6_1987", cs::iso_8859_6_1987}, + {"iso8859_6e", cs::iso_8859_6_e}, + {"iso8859_6i", cs::iso_8859_6_i}, + {"iso8859_7", cs::iso_8859_7_1987}, + {"iso8859_7_1987", cs::iso_8859_7_1987}, + {"iso8859_8", cs::iso_8859_8_1988}, + {"iso8859_8_1988", cs::iso_8859_8_1988}, + {"iso8859_8e", cs::iso_8859_8_e}, + {"iso8859_8i", cs::iso_8859_8_i}, + {"iso8859_9", cs::iso_8859_9_1989}, + {"iso8859_9_1989", cs::iso_8859_9_1989}, + {"iso8859_9windowslatin5", cs::iso_8859_9_windows_latin_5}, + {"iso8859supp", cs::iso_8859_supp}, + {"iso9036", cs::asmo_449}, + {"isoceltic", cs::iso_8859_14}, + {"isoir10", cs::sen_850200_b}, + {"isoir100", cs::iso_8859_1_1987}, + {"isoir101", cs::iso_8859_2_1987}, + {"isoir102", cs::t_61_7bit}, + {"isoir103", cs::t_61_8bit}, + {"isoir109", cs::iso_8859_3_1988}, + {"isoir11", cs::sen_850200_c}, + {"isoir110", cs::iso_8859_4_1988}, + {"isoir111", cs::ecma_cyrillic}, + {"isoir121", cs::csa_z243_4_1985_1}, + {"isoir122", cs::csa_z243_4_1985_2}, + {"isoir123", cs::csa_z243_4_1985_gr}, + {"isoir126", cs::iso_8859_7_1987}, + {"isoir127", cs::iso_8859_6_1987}, + {"isoir128", cs::t_101_g2}, + {"isoir13", cs::jis_c6220_1969_jp}, + {"isoir138", cs::iso_8859_8_1988}, + {"isoir139", cs::csn_369103}, + {"isoir14", cs::jis_c6220_1969_ro}, + {"isoir141", cs::jus_i_b1_002}, + {"isoir142", cs::iso_6937_2_add}, + {"isoir143", cs::iec_p27_1}, + {"isoir144", cs::iso_8859_5_1988}, + {"isoir146", cs::jus_i_b1_003_serb}, + {"isoir147", cs::jus_i_b1_003_mac}, + {"isoir148", cs::iso_8859_9_1989}, + {"isoir149", cs::ks_c_5601_1987}, + {"isoir15", cs::it}, + {"isoir150", cs::greek_ccitt}, + {"isoir151", cs::nc_nc00_10_81}, + {"isoir152", cs::iso_6937_2_25}, + {"isoir153", cs::gost_19768_74}, + {"isoir154", cs::iso_8859_supp}, + {"isoir155", cs::iso_10367_box}, + {"isoir157", cs::iso_8859_10}, + {"isoir158", cs::latin_lap}, + {"isoir159", cs::jis_x0212_1990}, + {"isoir16", cs::pt}, + {"isoir17", cs::es}, + {"isoir18", cs::greek7_old}, + {"isoir19", cs::latin_greek}, + {"isoir199", cs::iso_8859_14}, + {"isoir2", cs::iso_646_irv_1983}, + {"isoir21", cs::din_66003}, + {"isoir226", cs::iso_8859_16}, + {"isoir25", cs::nf_z_62_010__1973_}, + {"isoir27", cs::latin_greek_1}, + {"isoir37", cs::iso_5427}, + {"isoir4", cs::bs_4730}, + {"isoir42", cs::jis_c6226_1978}, + {"isoir47", cs::bs_viewdata}, + {"isoir49", cs::inis}, + {"isoir50", cs::inis_8}, + {"isoir51", cs::inis_cyrillic}, + {"isoir54", cs::iso_5427_1981}, + {"isoir55", cs::iso_5428_1980}, + {"isoir57", cs::gb_1988_80}, + {"isoir58", cs::gb_2312_80}, + {"isoir6", cs::ansi_x3_4_1968}, + {"isoir60", cs::ns_4551_1}, + {"isoir61", cs::ns_4551_2}, + {"isoir69", cs::nf_z_62_010}, + {"isoir70", cs::videotex_suppl}, + {"isoir84", cs::pt2}, + {"isoir85", cs::es2}, + {"isoir86", cs::msz_7795_3}, + {"isoir87", cs::jis_c6226_1983}, + {"isoir88", cs::greek7}, + {"isoir89", cs::asmo_449}, + {"isoir8_1", cs::nats_sefi}, + {"isoir8_2", cs::nats_sefi_add}, + {"isoir90", cs::iso_ir_90}, + {"isoir91", cs::jis_c6229_1984_a}, + {"isoir92", cs::jis_c6229_1984_b}, + {"isoir93", cs::jis_c6229_1984_b_add}, + {"isoir94", cs::jis_c6229_1984_hand}, + {"isoir95", cs::jis_c6229_1984_hand_add}, + {"isoir96", cs::jis_c6229_1984_kana}, + {"isoir98", cs::iso_2033_1983}, + {"isoir99", cs::ansi_x3_110_1983}, + {"isoir9_1", cs::nats_dano}, + {"isoir9_2", cs::nats_dano_add}, + {"isotr11548_1", cs::iso_11548_1}, + {"isounicodeibm1261", cs::iso_unicode_ibm_1261}, + {"isounicodeibm1264", cs::iso_unicode_ibm_1264}, + {"isounicodeibm1265", cs::iso_unicode_ibm_1265}, + {"isounicodeibm1268", cs::iso_unicode_ibm_1268}, + {"isounicodeibm1276", cs::iso_unicode_ibm_1276}, + {"it", cs::it}, + {"jisc6220_1969", cs::jis_c6220_1969_jp}, + {"jisc6220_1969jp", cs::jis_c6220_1969_jp}, + {"jisc6220_1969ro", cs::jis_c6220_1969_ro}, + {"jisc6226_1978", cs::jis_c6226_1978}, + {"jisc6226_1983", cs::jis_c6226_1983}, + {"jisc6229_1984a", cs::jis_c6229_1984_a}, + {"jisc6229_1984b", cs::jis_c6229_1984_b}, + {"jisc6229_1984badd", cs::jis_c6229_1984_b_add}, + {"jisc6229_1984hand", cs::jis_c6229_1984_hand}, + {"jisc6229_1984handadd", cs::jis_c6229_1984_hand_add}, + {"jisc6229_1984kana", cs::jis_c6229_1984_kana}, + {"jisencoding", cs::jis_encoding}, + {"jisx0201", cs::jis_x0201}, + {"jisx0208_1983", cs::jis_c6226_1983}, + {"jisx0212_1990", cs::jis_x0212_1990}, + {"jp", cs::jis_c6220_1969_ro}, + {"jpocra", cs::jis_c6229_1984_a}, + {"jpocrb", cs::jis_c6229_1984_b}, + {"jpocrbadd", cs::jis_c6229_1984_b_add}, + {"jpocrhand", cs::jis_c6229_1984_hand}, + {"jpocrhandadd", cs::jis_c6229_1984_hand_add}, + {"js", cs::jus_i_b1_002}, + {"jusib1_002", cs::jus_i_b1_002}, + {"jusib1_003mac", cs::jus_i_b1_003_mac}, + {"jusib1_003serb", cs::jus_i_b1_003_serb}, + {"katakana", cs::jis_c6220_1969_jp}, + {"koi7switched", cs::koi7_switched}, + {"koi8e", cs::ecma_cyrillic}, + {"koi8r", cs::koi8_r}, + {"koi8u", cs::koi8_u}, + {"korean", cs::ks_c_5601_1987}, + {"ksc5601", cs::ks_c_5601_1987}, + {"ksc5601_1987", cs::ks_c_5601_1987}, + {"ksc5601_1989", cs::ks_c_5601_1987}, + {"ksc5636", cs::ksc5636}, + {"kz1048", cs::kz_1048}, + {"l1", cs::iso_8859_1_1987}, + {"l10", cs::iso_8859_16}, + {"l2", cs::iso_8859_2_1987}, + {"l3", cs::iso_8859_3_1988}, + {"l4", cs::iso_8859_4_1988}, + {"l5", cs::iso_8859_9_1989}, + {"l6", cs::iso_8859_10}, + {"l8", cs::iso_8859_14}, + {"lap", cs::latin_lap}, + {"latin1", cs::iso_8859_1_1987}, + {"latin10", cs::iso_8859_16}, + {"latin1_2_5", cs::iso_8859_supp}, + {"latin2", cs::iso_8859_2_1987}, + {"latin3", cs::iso_8859_3_1988}, + {"latin4", cs::iso_8859_4_1988}, + {"latin5", cs::iso_8859_9_1989}, + {"latin6", cs::iso_8859_10}, + {"latin8", cs::iso_8859_14}, + {"latin9", cs::iso_8859_15}, + {"latingreek", cs::latin_greek}, + {"latingreek1", cs::latin_greek_1}, + {"latinlap", cs::latin_lap}, + {"mac", cs::macintosh}, + {"macedonian", cs::jus_i_b1_003_mac}, + {"macintosh", cs::macintosh}, + {"microsoftpublishing", cs::microsoft_publishing}, + {"mnem", cs::mnem}, + {"mnemonic", cs::mnemonic}, + {"ms936", cs::gbk}, + {"mskanji", cs::shift_jis}, + {"msz7795_3", cs::msz_7795_3}, + {"naplps", cs::ansi_x3_110_1983}, + {"natsdano", cs::nats_dano}, + {"natsdanoadd", cs::nats_dano_add}, + {"natssefi", cs::nats_sefi}, + {"natssefiadd", cs::nats_sefi_add}, + {"ncnc00_10_81", cs::nc_nc00_10_81}, + {"nfz62_010", cs::nf_z_62_010}, + {"nfz62_010_1973", cs::nf_z_62_010__1973_}, + {"no", cs::ns_4551_1}, + {"no2", cs::ns_4551_2}, + {"ns4551_1", cs::ns_4551_1}, + {"ns4551_2", cs::ns_4551_2}, + {"osdebcdicdf03irv", cs::osd_ebcdic_df03_irv}, + {"osdebcdicdf04_1", cs::osd_ebcdic_df04_1}, + {"osdebcdicdf04_15", cs::osd_ebcdic_df04_15}, + {"pc8danishnorwegian", cs::pc8_danish_norwegian}, + {"pc8turkish", cs::pc8_turkish}, + {"pcmultilingual850euro", cs::ibm00858}, + {"pt", cs::pt}, + {"pt154", cs::ptcp154}, + {"pt2", cs::pt2}, + {"ptcp154", cs::ptcp154}, + {"r8", cs::hp_roman8}, + {"ref", cs::iso_646_basic_1983}, + {"rk1048", cs::kz_1048}, + {"roman8", cs::hp_roman8}, + {"scsu", cs::scsu}, + {"se", cs::sen_850200_b}, + {"se2", cs::sen_850200_c}, + {"sen850200b", cs::sen_850200_b}, + {"sen850200c", cs::sen_850200_c}, + {"serbian", cs::jus_i_b1_003_serb}, + {"shiftjis", cs::shift_jis}, + {"strk1048_2002", cs::kz_1048}, + {"stsev358_88", cs::gost_19768_74}, + {"t101g2", cs::t_101_g2}, + {"t61", cs::t_61_8bit}, + {"t61_7bit", cs::t_61_7bit}, + {"t61_8bit", cs::t_61_8bit}, + {"tis620", cs::tis_620}, + {"tscii", cs::tscii}, + {"ucs2", cs::iso_10646_ucs_2}, + {"ucs4", cs::iso_10646_ucs_4}, + {"uk", cs::bs_4730}, + {"unicode1_1", cs::unicode_1_1}, + {"unicode1_1utf7", cs::unicode_1_1_utf_7}, + {"unknown8bit", cs::unknown_8bit}, + {"us", cs::ansi_x3_4_1968}, + {"usascii", cs::ansi_x3_4_1968}, + {"usdk", cs::us_dk}, + {"utf16", cs::utf_16}, + {"utf16be", cs::utf_16be}, + {"utf16le", cs::utf_16le}, + {"utf32", cs::utf_32}, + {"utf32be", cs::utf_32be}, + {"utf32le", cs::utf_32le}, + {"utf7", cs::utf_7}, + {"utf8", cs::utf_8}, + {"venturainternational", cs::ventura_international}, + {"venturamath", cs::ventura_math}, + {"venturaus", cs::ventura_us}, + {"videotexsuppl", cs::videotex_suppl}, + {"viqr", cs::viqr}, + {"viscii", cs::viscii}, + {"windows1250", cs::windows_1250}, + {"windows1251", cs::windows_1251}, + {"windows1252", cs::windows_1252}, + {"windows1253", cs::windows_1253}, + {"windows1254", cs::windows_1254}, + {"windows1255", cs::windows_1255}, + {"windows1256", cs::windows_1256}, + {"windows1257", cs::windows_1257}, + {"windows1258", cs::windows_1258}, + {"windows31j", cs::windows_31j}, + {"windows936", cs::gbk}, + {"x0201", cs::jis_x0201}, + {"x0201_7", cs::jis_c6220_1969_jp}, + {"x0208", cs::jis_c6226_1983}, + {"x0212", cs::jis_x0212_1990}, + {"yu", cs::jus_i_b1_002}, diff --git a/extras/anytermd/libpbe/src/generated/charset_mime_names_body.cc b/extras/anytermd/libpbe/src/generated/charset_mime_names_body.cc new file mode 100644 index 0000000000..d12b79d806 --- /dev/null +++ b/extras/anytermd/libpbe/src/generated/charset_mime_names_body.cc @@ -0,0 +1,257 @@ +const char * const charset_mime_names[] = { + "US-ASCII", + "ISO-8859-1", + "ISO-8859-2", + "ISO-8859-3", + "ISO-8859-4", + "ISO-8859-5", + "ISO-8859-6", + "ISO-8859-7", + "ISO-8859-8", + "ISO-8859-9", + "ISO-8859-10", + "ISO_6937-2-add", + "JIS_X0201", + "JIS_Encoding", + "Shift_JIS", + "EUC-JP", + "Extended_UNIX_Code_Fixed_Width_for_Japanese", + "BS_4730", + "SEN_850200_C", + "IT", + "ES", + "DIN_66003", + "NS_4551-1", + "NF_Z_62-010", + "ISO-10646-UTF-1", + "ISO_646.basic:1983", + "INVARIANT", + "ISO_646.irv:1983", + "NATS-SEFI", + "NATS-SEFI-ADD", + "NATS-DANO", + "NATS-DANO-ADD", + "SEN_850200_B", + "KS_C_5601-1987", + "ISO-2022-KR", + "EUC-KR", + "ISO-2022-JP", + "ISO-2022-JP-2", + "JIS_C6220-1969-jp", + "JIS_C6220-1969-ro", + "PT", + "greek7-old", + "latin-greek", + "NF_Z_62-010_(1973)", + "Latin-greek-1", + "ISO_5427", + "JIS_C6226-1978", + "BS_viewdata", + "INIS", + "INIS-8", + "INIS-cyrillic", + "ISO_5427:1981", + "ISO_5428:1980", + "GB_1988-80", + "GB_2312-80", + "NS_4551-2", + "videotex-suppl", + "PT2", + "ES2", + "MSZ_7795.3", + "JIS_C6226-1983", + "greek7", + "ASMO_449", + "iso-ir-90", + "JIS_C6229-1984-a", + "JIS_C6229-1984-b", + "JIS_C6229-1984-b-add", + "JIS_C6229-1984-hand", + "JIS_C6229-1984-hand-add", + "JIS_C6229-1984-kana", + "ISO_2033-1983", + "ANSI_X3.110-1983", + "T.61-7bit", + "T.61-8bit", + "ECMA-cyrillic", + "CSA_Z243.4-1985-1", + "CSA_Z243.4-1985-2", + "CSA_Z243.4-1985-gr", + "ISO-8859-6-E", + "ISO-8859-6-I", + "T.101-G2", + "ISO-8859-8-E", + "ISO-8859-8-I", + "CSN_369103", + "JUS_I.B1.002", + "IEC_P27-1", + "JUS_I.B1.003-serb", + "JUS_I.B1.003-mac", + "greek-ccitt", + "NC_NC00-10:81", + "ISO_6937-2-25", + "GOST_19768-74", + "ISO_8859-supp", + "ISO_10367-box", + "latin-lap", + "JIS_X0212-1990", + "DS_2089", + "us-dk", + "dk-us", + "KSC5636", + "UNICODE-1-1-UTF-7", + "ISO-2022-CN", + "ISO-2022-CN-EXT", + "UTF-8", + "ISO-8859-13", + "ISO-8859-14", + "ISO-8859-15", + "ISO-8859-16", + "GBK", + "GB18030", + "OSD_EBCDIC_DF04_15", + "OSD_EBCDIC_DF03_IRV", + "OSD_EBCDIC_DF04_1", + "ISO-11548-1", + "KZ-1048", + "ISO-10646-UCS-2", + "ISO-10646-UCS-4", + "ISO-10646-UCS-Basic", + "ISO-10646-Unicode-Latin1", + "ISO-10646-J-1", + "ISO-Unicode-IBM-1261", + "ISO-Unicode-IBM-1268", + "ISO-Unicode-IBM-1276", + "ISO-Unicode-IBM-1264", + "ISO-Unicode-IBM-1265", + "UNICODE-1-1", + "SCSU", + "UTF-7", + "UTF-16BE", + "UTF-16LE", + "UTF-16", + "CESU-8", + "UTF-32", + "UTF-32BE", + "UTF-32LE", + "BOCU-1", + "ISO-8859-1-Windows-3.0-Latin-1", + "ISO-8859-1-Windows-3.1-Latin-1", + "ISO-8859-2-Windows-Latin-2", + "ISO-8859-9-Windows-Latin-5", + "hp-roman8", + "Adobe-Standard-Encoding", + "Ventura-US", + "Ventura-International", + "DEC-MCS", + "IBM850", + "PC8-Danish-Norwegian", + "IBM862", + "PC8-Turkish", + "IBM-Symbols", + "IBM-Thai", + "HP-Legal", + "HP-Pi-font", + "HP-Math8", + "Adobe-Symbol-Encoding", + "HP-DeskTop", + "Ventura-Math", + "Microsoft-Publishing", + "Windows-31J", + "GB2312", + "Big5", + "macintosh", + "IBM037", + "IBM038", + "IBM273", + "IBM274", + "IBM275", + "IBM277", + "IBM278", + "IBM280", + "IBM281", + "IBM284", + "IBM285", + "IBM290", + "IBM297", + "IBM420", + "IBM423", + "IBM424", + "IBM437", + "IBM500", + "IBM851", + "IBM852", + "IBM855", + "IBM857", + "IBM860", + "IBM861", + "IBM863", + "IBM864", + "IBM865", + "IBM868", + "IBM869", + "IBM870", + "IBM871", + "IBM880", + "IBM891", + "IBM903", + "IBM904", + "IBM905", + "IBM918", + "IBM1026", + "EBCDIC-AT-DE", + "EBCDIC-AT-DE-A", + "EBCDIC-CA-FR", + "EBCDIC-DK-NO", + "EBCDIC-DK-NO-A", + "EBCDIC-FI-SE", + "EBCDIC-FI-SE-A", + "EBCDIC-FR", + "EBCDIC-IT", + "EBCDIC-PT", + "EBCDIC-ES", + "EBCDIC-ES-A", + "EBCDIC-ES-S", + "EBCDIC-UK", + "EBCDIC-US", + "UNKNOWN-8BIT", + "MNEMONIC", + "MNEM", + "VISCII", + "VIQR", + "KOI8-R", + "HZ-GB-2312", + "IBM866", + "IBM775", + "KOI8-U", + "IBM00858", + "IBM00924", + "IBM01140", + "IBM01141", + "IBM01142", + "IBM01143", + "IBM01144", + "IBM01145", + "IBM01146", + "IBM01147", + "IBM01148", + "IBM01149", + "Big5-HKSCS", + "IBM1047", + "PTCP154", + "Amiga-1251", + "KOI7-switched", + "BRF", + "TSCII", + "windows-1250", + "windows-1251", + "windows-1252", + "windows-1253", + "windows-1254", + "windows-1255", + "windows-1256", + "windows-1257", + "windows-1258", + "TIS-620", + NULL +}; diff --git a/extras/anytermd/libpbe/src/generated/iso8859_tables_body.cc b/extras/anytermd/libpbe/src/generated/iso8859_tables_body.cc new file mode 100644 index 0000000000..bb141a0f06 --- /dev/null +++ b/extras/anytermd/libpbe/src/generated/iso8859_tables_body.cc @@ -0,0 +1,32 @@ +iso8859_table_t iso8859_tables[] = { + // ISO-8859-2 + { 0x0104, 0x02D8, 0x0141, 0x00A4, 0x013D, 0x015A, 0x00A7, 0x00A8, 0x0160, 0x015E, 0x0164, 0x0179, 0x00AD, 0x017D, 0x017B, 0x00B0, 0x0105, 0x02DB, 0x0142, 0x00B4, 0x013E, 0x015B, 0x02C7, 0x00B8, 0x0161, 0x015F, 0x0165, 0x017A, 0x02DD, 0x017E, 0x017C, 0x0154, 0x00C1, 0x00C2, 0x0102, 0x00C4, 0x0139, 0x0106, 0x00C7, 0x010C, 0x00C9, 0x0118, 0x00CB, 0x011A, 0x00CD, 0x00CE, 0x010E, 0x0110, 0x0143, 0x0147, 0x00D3, 0x00D4, 0x0150, 0x00D6, 0x00D7, 0x0158, 0x016E, 0x00DA, 0x0170, 0x00DC, 0x00DD, 0x0162, 0x00DF, 0x0155, 0x00E1, 0x00E2, 0x0103, 0x00E4, 0x013A, 0x0107, 0x00E7, 0x010D, 0x00E9, 0x0119, 0x00EB, 0x011B, 0x00ED, 0x00EE, 0x010F, 0x0111, 0x0144, 0x0148, 0x00F3, 0x00F4, 0x0151, 0x00F6, 0x00F7, 0x0159, 0x016F, 0x00FA, 0x0171, 0x00FC, 0x00FD, 0x0163, 0x02D9,}, + // ISO-8859-3 + { 0x0126, 0x02D8, 0x00A3, 0x00A4, -1, 0x0124, 0x00A7, 0x00A8, 0x0130, 0x015E, 0x011E, 0x0134, 0x00AD, -1, 0x017B, 0x00B0, 0x0127, 0x00B2, 0x00B3, 0x00B4, 0x00B5, 0x0125, 0x00B7, 0x00B8, 0x0131, 0x015F, 0x011F, 0x0135, 0x00BD, -1, 0x017C, 0x00C0, 0x00C1, 0x00C2, -1, 0x00C4, 0x010A, 0x0108, 0x00C7, 0x00C8, 0x00C9, 0x00CA, 0x00CB, 0x00CC, 0x00CD, 0x00CE, 0x00CF, -1, 0x00D1, 0x00D2, 0x00D3, 0x00D4, 0x0120, 0x00D6, 0x00D7, 0x011C, 0x00D9, 0x00DA, 0x00DB, 0x00DC, 0x016C, 0x015C, 0x00DF, 0x00E0, 0x00E1, 0x00E2, -1, 0x00E4, 0x010B, 0x0109, 0x00E7, 0x00E8, 0x00E9, 0x00EA, 0x00EB, 0x00EC, 0x00ED, 0x00EE, 0x00EF, -1, 0x00F1, 0x00F2, 0x00F3, 0x00F4, 0x0121, 0x00F6, 0x00F7, 0x011D, 0x00F9, 0x00FA, 0x00FB, 0x00FC, 0x016D, 0x015D, 0x02D9,}, + // ISO-8859-4 + { 0x0104, 0x0138, 0x0156, 0x00A4, 0x0128, 0x013B, 0x00A7, 0x00A8, 0x0160, 0x0112, 0x0122, 0x0166, 0x00AD, 0x017D, 0x00AF, 0x00B0, 0x0105, 0x02DB, 0x0157, 0x00B4, 0x0129, 0x013C, 0x02C7, 0x00B8, 0x0161, 0x0113, 0x0123, 0x0167, 0x014A, 0x017E, 0x014B, 0x0100, 0x00C1, 0x00C2, 0x00C3, 0x00C4, 0x00C5, 0x00C6, 0x012E, 0x010C, 0x00C9, 0x0118, 0x00CB, 0x0116, 0x00CD, 0x00CE, 0x012A, 0x0110, 0x0145, 0x014C, 0x0136, 0x00D4, 0x00D5, 0x00D6, 0x00D7, 0x00D8, 0x0172, 0x00DA, 0x00DB, 0x00DC, 0x0168, 0x016A, 0x00DF, 0x0101, 0x00E1, 0x00E2, 0x00E3, 0x00E4, 0x00E5, 0x00E6, 0x012F, 0x010D, 0x00E9, 0x0119, 0x00EB, 0x0117, 0x00ED, 0x00EE, 0x012B, 0x0111, 0x0146, 0x014D, 0x0137, 0x00F4, 0x00F5, 0x00F6, 0x00F7, 0x00F8, 0x0173, 0x00FA, 0x00FB, 0x00FC, 0x0169, 0x016B, 0x02D9,}, + // ISO-8859-5 + { 0x0401, 0x0402, 0x0403, 0x0404, 0x0405, 0x0406, 0x0407, 0x0408, 0x0409, 0x040A, 0x040B, 0x040C, 0x00AD, 0x040E, 0x040F, 0x0410, 0x0411, 0x0412, 0x0413, 0x0414, 0x0415, 0x0416, 0x0417, 0x0418, 0x0419, 0x041A, 0x041B, 0x041C, 0x041D, 0x041E, 0x041F, 0x0420, 0x0421, 0x0422, 0x0423, 0x0424, 0x0425, 0x0426, 0x0427, 0x0428, 0x0429, 0x042A, 0x042B, 0x042C, 0x042D, 0x042E, 0x042F, 0x0430, 0x0431, 0x0432, 0x0433, 0x0434, 0x0435, 0x0436, 0x0437, 0x0438, 0x0439, 0x043A, 0x043B, 0x043C, 0x043D, 0x043E, 0x043F, 0x0440, 0x0441, 0x0442, 0x0443, 0x0444, 0x0445, 0x0446, 0x0447, 0x0448, 0x0449, 0x044A, 0x044B, 0x044C, 0x044D, 0x044E, 0x044F, 0x2116, 0x0451, 0x0452, 0x0453, 0x0454, 0x0455, 0x0456, 0x0457, 0x0458, 0x0459, 0x045A, 0x045B, 0x045C, 0x00A7, 0x045E, 0x045F,}, + // ISO-8859-6 + { -1, -1, -1, 0x00A4, -1, -1, -1, -1, -1, -1, -1, 0x060C, 0x00AD, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 0x061B, -1, -1, -1, 0x061F, -1, 0x0621, 0x0622, 0x0623, 0x0624, 0x0625, 0x0626, 0x0627, 0x0628, 0x0629, 0x062A, 0x062B, 0x062C, 0x062D, 0x062E, 0x062F, 0x0630, 0x0631, 0x0632, 0x0633, 0x0634, 0x0635, 0x0636, 0x0637, 0x0638, 0x0639, 0x063A, -1, -1, -1, -1, -1, 0x0640, 0x0641, 0x0642, 0x0643, 0x0644, 0x0645, 0x0646, 0x0647, 0x0648, 0x0649, 0x064A, 0x064B, 0x064C, 0x064D, 0x064E, 0x064F, 0x0650, 0x0651, 0x0652,}, + // ISO-8859-7 + { 0x2018, 0x2019, 0x00A3, 0x20AC, 0x20AF, 0x00A6, 0x00A7, 0x00A8, 0x00A9, 0x037A, 0x00AB, 0x00AC, 0x00AD, -1, 0x2015, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x0384, 0x0385, 0x0386, 0x00B7, 0x0388, 0x0389, 0x038A, 0x00BB, 0x038C, 0x00BD, 0x038E, 0x038F, 0x0390, 0x0391, 0x0392, 0x0393, 0x0394, 0x0395, 0x0396, 0x0397, 0x0398, 0x0399, 0x039A, 0x039B, 0x039C, 0x039D, 0x039E, 0x039F, 0x03A0, 0x03A1, -1, 0x03A3, 0x03A4, 0x03A5, 0x03A6, 0x03A7, 0x03A8, 0x03A9, 0x03AA, 0x03AB, 0x03AC, 0x03AD, 0x03AE, 0x03AF, 0x03B0, 0x03B1, 0x03B2, 0x03B3, 0x03B4, 0x03B5, 0x03B6, 0x03B7, 0x03B8, 0x03B9, 0x03BA, 0x03BB, 0x03BC, 0x03BD, 0x03BE, 0x03BF, 0x03C0, 0x03C1, 0x03C2, 0x03C3, 0x03C4, 0x03C5, 0x03C6, 0x03C7, 0x03C8, 0x03C9, 0x03CA, 0x03CB, 0x03CC, 0x03CD, 0x03CE,}, + // ISO-8859-8 + { -1, 0x00A2, 0x00A3, 0x00A4, 0x00A5, 0x00A6, 0x00A7, 0x00A8, 0x00A9, 0x00D7, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x00AF, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x00B4, 0x00B5, 0x00B6, 0x00B7, 0x00B8, 0x00B9, 0x00F7, 0x00BB, 0x00BC, 0x00BD, 0x00BE, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 0x2017, 0x05D0, 0x05D1, 0x05D2, 0x05D3, 0x05D4, 0x05D5, 0x05D6, 0x05D7, 0x05D8, 0x05D9, 0x05DA, 0x05DB, 0x05DC, 0x05DD, 0x05DE, 0x05DF, 0x05E0, 0x05E1, 0x05E2, 0x05E3, 0x05E4, 0x05E5, 0x05E6, 0x05E7, 0x05E8, 0x05E9, 0x05EA, -1, -1, 0x200E, 0x200F,}, + // ISO-8859-9 + { 0x00A1, 0x00A2, 0x00A3, 0x00A4, 0x00A5, 0x00A6, 0x00A7, 0x00A8, 0x00A9, 0x00AA, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x00AF, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x00B4, 0x00B5, 0x00B6, 0x00B7, 0x00B8, 0x00B9, 0x00BA, 0x00BB, 0x00BC, 0x00BD, 0x00BE, 0x00BF, 0x00C0, 0x00C1, 0x00C2, 0x00C3, 0x00C4, 0x00C5, 0x00C6, 0x00C7, 0x00C8, 0x00C9, 0x00CA, 0x00CB, 0x00CC, 0x00CD, 0x00CE, 0x00CF, 0x011E, 0x00D1, 0x00D2, 0x00D3, 0x00D4, 0x00D5, 0x00D6, 0x00D7, 0x00D8, 0x00D9, 0x00DA, 0x00DB, 0x00DC, 0x0130, 0x015E, 0x00DF, 0x00E0, 0x00E1, 0x00E2, 0x00E3, 0x00E4, 0x00E5, 0x00E6, 0x00E7, 0x00E8, 0x00E9, 0x00EA, 0x00EB, 0x00EC, 0x00ED, 0x00EE, 0x00EF, 0x011F, 0x00F1, 0x00F2, 0x00F3, 0x00F4, 0x00F5, 0x00F6, 0x00F7, 0x00F8, 0x00F9, 0x00FA, 0x00FB, 0x00FC, 0x0131, 0x015F, 0x00FF,}, + // ISO-8859-10 + { 0x0104, 0x0112, 0x0122, 0x012A, 0x0128, 0x0136, 0x00A7, 0x013B, 0x0110, 0x0160, 0x0166, 0x017D, 0x00AD, 0x016A, 0x014A, 0x00B0, 0x0105, 0x0113, 0x0123, 0x012B, 0x0129, 0x0137, 0x00B7, 0x013C, 0x0111, 0x0161, 0x0167, 0x017E, 0x2015, 0x016B, 0x014B, 0x0100, 0x00C1, 0x00C2, 0x00C3, 0x00C4, 0x00C5, 0x00C6, 0x012E, 0x010C, 0x00C9, 0x0118, 0x00CB, 0x0116, 0x00CD, 0x00CE, 0x00CF, 0x00D0, 0x0145, 0x014C, 0x00D3, 0x00D4, 0x00D5, 0x00D6, 0x0168, 0x00D8, 0x0172, 0x00DA, 0x00DB, 0x00DC, 0x00DD, 0x00DE, 0x00DF, 0x0101, 0x00E1, 0x00E2, 0x00E3, 0x00E4, 0x00E5, 0x00E6, 0x012F, 0x010D, 0x00E9, 0x0119, 0x00EB, 0x0117, 0x00ED, 0x00EE, 0x00EF, 0x00F0, 0x0146, 0x014D, 0x00F3, 0x00F4, 0x00F5, 0x00F6, 0x0169, 0x00F8, 0x0173, 0x00FA, 0x00FB, 0x00FC, 0x00FD, 0x00FE, 0x0138,}, + // ISO-8859-11 + { 0x0E01, 0x0E02, 0x0E03, 0x0E04, 0x0E05, 0x0E06, 0x0E07, 0x0E08, 0x0E09, 0x0E0A, 0x0E0B, 0x0E0C, 0x0E0D, 0x0E0E, 0x0E0F, 0x0E10, 0x0E11, 0x0E12, 0x0E13, 0x0E14, 0x0E15, 0x0E16, 0x0E17, 0x0E18, 0x0E19, 0x0E1A, 0x0E1B, 0x0E1C, 0x0E1D, 0x0E1E, 0x0E1F, 0x0E20, 0x0E21, 0x0E22, 0x0E23, 0x0E24, 0x0E25, 0x0E26, 0x0E27, 0x0E28, 0x0E29, 0x0E2A, 0x0E2B, 0x0E2C, 0x0E2D, 0x0E2E, 0x0E2F, 0x0E30, 0x0E31, 0x0E32, 0x0E33, 0x0E34, 0x0E35, 0x0E36, 0x0E37, 0x0E38, 0x0E39, 0x0E3A, -1, -1, -1, -1, 0x0E3F, 0x0E40, 0x0E41, 0x0E42, 0x0E43, 0x0E44, 0x0E45, 0x0E46, 0x0E47, 0x0E48, 0x0E49, 0x0E4A, 0x0E4B, 0x0E4C, 0x0E4D, 0x0E4E, 0x0E4F, 0x0E50, 0x0E51, 0x0E52, 0x0E53, 0x0E54, 0x0E55, 0x0E56, 0x0E57, 0x0E58, 0x0E59, 0x0E5A, 0x0E5B,}, + // ISO-8859-12 does not exist + {}, + // ISO-8859-13 + { 0x201D, 0x00A2, 0x00A3, 0x00A4, 0x201E, 0x00A6, 0x00A7, 0x00D8, 0x00A9, 0x0156, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x00C6, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x201C, 0x00B5, 0x00B6, 0x00B7, 0x00F8, 0x00B9, 0x0157, 0x00BB, 0x00BC, 0x00BD, 0x00BE, 0x00E6, 0x0104, 0x012E, 0x0100, 0x0106, 0x00C4, 0x00C5, 0x0118, 0x0112, 0x010C, 0x00C9, 0x0179, 0x0116, 0x0122, 0x0136, 0x012A, 0x013B, 0x0160, 0x0143, 0x0145, 0x00D3, 0x014C, 0x00D5, 0x00D6, 0x00D7, 0x0172, 0x0141, 0x015A, 0x016A, 0x00DC, 0x017B, 0x017D, 0x00DF, 0x0105, 0x012F, 0x0101, 0x0107, 0x00E4, 0x00E5, 0x0119, 0x0113, 0x010D, 0x00E9, 0x017A, 0x0117, 0x0123, 0x0137, 0x012B, 0x013C, 0x0161, 0x0144, 0x0146, 0x00F3, 0x014D, 0x00F5, 0x00F6, 0x00F7, 0x0173, 0x0142, 0x015B, 0x016B, 0x00FC, 0x017C, 0x017E, 0x2019,}, + // ISO-8859-14 + { 0x1E02, 0x1E03, 0x00A3, 0x010A, 0x010B, 0x1E0A, 0x00A7, 0x1E80, 0x00A9, 0x1E82, 0x1E0B, 0x1EF2, 0x00AD, 0x00AE, 0x0178, 0x1E1E, 0x1E1F, 0x0120, 0x0121, 0x1E40, 0x1E41, 0x00B6, 0x1E56, 0x1E81, 0x1E57, 0x1E83, 0x1E60, 0x1EF3, 0x1E84, 0x1E85, 0x1E61, 0x00C0, 0x00C1, 0x00C2, 0x00C3, 0x00C4, 0x00C5, 0x00C6, 0x00C7, 0x00C8, 0x00C9, 0x00CA, 0x00CB, 0x00CC, 0x00CD, 0x00CE, 0x00CF, 0x0174, 0x00D1, 0x00D2, 0x00D3, 0x00D4, 0x00D5, 0x00D6, 0x1E6A, 0x00D8, 0x00D9, 0x00DA, 0x00DB, 0x00DC, 0x00DD, 0x0176, 0x00DF, 0x00E0, 0x00E1, 0x00E2, 0x00E3, 0x00E4, 0x00E5, 0x00E6, 0x00E7, 0x00E8, 0x00E9, 0x00EA, 0x00EB, 0x00EC, 0x00ED, 0x00EE, 0x00EF, 0x0175, 0x00F1, 0x00F2, 0x00F3, 0x00F4, 0x00F5, 0x00F6, 0x1E6B, 0x00F8, 0x00F9, 0x00FA, 0x00FB, 0x00FC, 0x00FD, 0x0177, 0x00FF,}, + // ISO-8859-15 + { 0x00A1, 0x00A2, 0x00A3, 0x20AC, 0x00A5, 0x0160, 0x00A7, 0x0161, 0x00A9, 0x00AA, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x00AF, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x017D, 0x00B5, 0x00B6, 0x00B7, 0x017E, 0x00B9, 0x00BA, 0x00BB, 0x0152, 0x0153, 0x0178, 0x00BF, 0x00C0, 0x00C1, 0x00C2, 0x00C3, 0x00C4, 0x00C5, 0x00C6, 0x00C7, 0x00C8, 0x00C9, 0x00CA, 0x00CB, 0x00CC, 0x00CD, 0x00CE, 0x00CF, 0x00D0, 0x00D1, 0x00D2, 0x00D3, 0x00D4, 0x00D5, 0x00D6, 0x00D7, 0x00D8, 0x00D9, 0x00DA, 0x00DB, 0x00DC, 0x00DD, 0x00DE, 0x00DF, 0x00E0, 0x00E1, 0x00E2, 0x00E3, 0x00E4, 0x00E5, 0x00E6, 0x00E7, 0x00E8, 0x00E9, 0x00EA, 0x00EB, 0x00EC, 0x00ED, 0x00EE, 0x00EF, 0x00F0, 0x00F1, 0x00F2, 0x00F3, 0x00F4, 0x00F5, 0x00F6, 0x00F7, 0x00F8, 0x00F9, 0x00FA, 0x00FB, 0x00FC, 0x00FD, 0x00FE, 0x00FF,}, + // ISO-8859-16 + { 0x0104, 0x0105, 0x0141, 0x20AC, 0x201E, 0x0160, 0x00A7, 0x0161, 0x00A9, 0x0218, 0x00AB, 0x0179, 0x00AD, 0x017A, 0x017B, 0x00B0, 0x00B1, 0x010C, 0x0142, 0x017D, 0x201D, 0x00B6, 0x00B7, 0x017E, 0x010D, 0x0219, 0x00BB, 0x0152, 0x0153, 0x0178, 0x017C, 0x00C0, 0x00C1, 0x00C2, 0x0102, 0x00C4, 0x0106, 0x00C6, 0x00C7, 0x00C8, 0x00C9, 0x00CA, 0x00CB, 0x00CC, 0x00CD, 0x00CE, 0x00CF, 0x0110, 0x0143, 0x00D2, 0x00D3, 0x00D4, 0x0150, 0x00D6, 0x015A, 0x0170, 0x00D9, 0x00DA, 0x00DB, 0x00DC, 0x0118, 0x021A, 0x00DF, 0x00E0, 0x00E1, 0x00E2, 0x0103, 0x00E4, 0x0107, 0x00E6, 0x00E7, 0x00E8, 0x00E9, 0x00EA, 0x00EB, 0x00EC, 0x00ED, 0x00EE, 0x00EF, 0x0111, 0x0144, 0x00F2, 0x00F3, 0x00F4, 0x0151, 0x00F6, 0x015B, 0x0171, 0x00F9, 0x00FA, 0x00FB, 0x00FC, 0x0119, 0x021B, 0x00FF,}, +}; diff --git a/extras/anytermd/libpbe/src/generated/to_ascii_letters_tables.cc b/extras/anytermd/libpbe/src/generated/to_ascii_letters_tables.cc new file mode 100644 index 0000000000..76077f91a5 --- /dev/null +++ b/extras/anytermd/libpbe/src/generated/to_ascii_letters_tables.cc @@ -0,0 +1,16 @@ +char_expansion_page_00_t to_ascii_letters_page_00 [] = {"", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "", "", "", "", "", "", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "a", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "o", "", "", "", "", "", "a", "a", "a", "a", "a", "a", "ae", "c", "e", "e", "e", "e", "i", "i", "i", "i", "d", "n", "o", "o", "o", "o", "o", "", "o", "u", "u", "u", "u", "y", "th", "ss", "a", "a", "a", "a", "a", "a", "ae", "c", "e", "e", "e", "e", "i", "i", "i", "i", "d", "n", "o", "o", "o", "o", "o", "", "o", "u", "u", "u", "u", "y", "th", "y", }; +char_expansion_page_01_t to_ascii_letters_page_01 [] = {"a", "a", "a", "a", "a", "a", "c", "c", "c", "c", "c", "c", "c", "c", "d", "d", "d", "d", "e", "e", "e", "e", "e", "e", "e", "e", "e", "e", "g", "g", "g", "g", "g", "g", "g", "g", "h", "h", "h", "h", "i", "i", "i", "i", "i", "i", "i", "i", "i", "i", "ij", "ij", "j", "j", "k", "k", "", "l", "l", "l", "l", "l", "l", "l", "l", "l", "l", "n", "n", "n", "n", "n", "n", "n", "", "", "o", "o", "o", "o", "o", "o", "oe", "oe", "r", "r", "r", "r", "r", "r", "s", "s", "s", "s", "s", "s", "s", "s", "t", "t", "t", "t", "t", "t", "u", "u", "u", "u", "u", "u", "u", "u", "u", "u", "u", "u", "w", "w", "y", "y", "y", "z", "z", "z", "z", "z", "z", "s", "b", "b", "b", "b", "", "", "o", "c", "c", "d", "d", "d", "d", "", "e", "", "e", "f", "f", "g", "", "hv", "", "i", "k", "k", "l", "", "m", "n", "n", "o", "o", "o", "oi", "oi", "p", "p", "yr", "", "", "", "", "t", "t", "t", "t", "u", "u", "", "v", "y", "y", "z", "z", "", "", "", "", "", "", "", "", "", "", "", "", "", "dz", "dz", "dz", "lj", "lj", "lj", "nj", "nj", "nj", "a", "a", "i", "i", "o", "o", "u", "u", "u", "u", "u", "u", "u", "u", "u", "u", "e", "a", "a", "a", "a", "", "", "g", "g", "g", "g", "k", "k", "o", "o", "o", "o", "", "", "j", "dz", "dz", "dz", "g", "g", "", "", "n", "n", "a", "a", "", "", "", "", }; +char_expansion_page_02_t to_ascii_letters_page_02 [] = {"a", "a", "a", "a", "e", "e", "e", "e", "i", "i", "i", "i", "o", "o", "o", "o", "r", "r", "r", "r", "u", "u", "u", "u", "s", "s", "t", "t", "", "", "h", "h", "n", "d", "", "ou", "z", "z", "a", "a", "e", "e", "o", "o", "o", "o", "o", "o", "o", "o", "y", "y", "l", "n", "t", "j", "db", "qp", "a", "", "c", "l", "t", "s", "z", "", "", "b", "u", "v", "e", "e", "j", "j", "q", "q", "r", "r", "y", "y", "a", "", "", "b", "o", "c", "d", "d", "e", "", "", "e", "e", "e", "e", "j", "g", "g", "g", "", "", "h", "h", "", "i", "", "i", "l", "l", "l", "", "m", "m", "m", "m", "n", "n", "o", "oe", "", "", "r", "r", "r", "r", "r", "r", "r", "r", "r", "s", "", "j", "", "", "t", "t", "u", "", "v", "v", "w", "y", "y", "z", "z", "", "", "", "", "", "c", "", "b", "e", "g", "h", "j", "k", "l", "q", "", "", "dz", "", "dz", "ts", "", "tc", "", "ls", "lz", "", "", "h", "h", "h", "", "j", "r", "", "", "", "w", "y", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "l", "s", "x", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", }; +char_expansion_page_1D4_t to_ascii_letters_page_1D4 [] = {"a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "a", "b", "c", "d", "e", "f", "g", "", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "a", "", "c", "d", "", "", "g", "", "", "j", "k", "", "", "n", "o", "p", "q", "", "s", "t", "u", "v", "w", "x", "y", "z", "a", "b", "c", "d", "", "f", "", "h", "i", "j", "k", "l", "m", "n", "", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", }; +char_expansion_page_1D5_t to_ascii_letters_page_1D5 [] = {"w", "x", "y", "z", "a", "b", "", "d", "e", "f", "g", "", "", "j", "k", "l", "m", "n", "o", "p", "q", "", "s", "t", "u", "v", "w", "x", "y", "", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "a", "b", "", "d", "e", "f", "g", "", "i", "j", "k", "l", "m", "", "o", "", "", "", "s", "t", "u", "v", "w", "x", "y", "", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", }; +char_expansion_page_1D6_t to_ascii_letters_page_1D6 [] = {"s", "t", "u", "v", "w", "x", "y", "z", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", }; +char_expansion_page_1D_t to_ascii_letters_page_1D [] = {"", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "a", "", "b", "", "d", "e", "", "g", "h", "i", "j", "k", "l", "m", "n", "", "o", "", "p", "r", "t", "u", "w", "a", "", "", "", "b", "d", "e", "", "", "", "g", "", "k", "m", "", "o", "", "", "", "p", "t", "u", "", "", "v", "", "", "", "", "", "", "i", "r", "u", "v", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "c", "", "", "", "f", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "z", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", }; +char_expansion_page_1E_t to_ascii_letters_page_1E [] = {"a", "a", "b", "b", "b", "b", "b", "b", "c", "c", "d", "d", "d", "d", "d", "d", "d", "d", "d", "d", "e", "e", "e", "e", "e", "e", "e", "e", "e", "e", "f", "f", "g", "g", "h", "h", "h", "h", "h", "h", "h", "h", "h", "h", "i", "i", "i", "i", "k", "k", "k", "k", "k", "k", "l", "l", "l", "l", "l", "l", "l", "l", "m", "m", "m", "m", "m", "m", "n", "n", "n", "n", "n", "n", "n", "n", "o", "o", "o", "o", "o", "o", "o", "o", "p", "p", "p", "p", "r", "r", "r", "r", "r", "r", "r", "r", "s", "s", "s", "s", "s", "s", "s", "s", "s", "s", "t", "t", "t", "t", "t", "t", "t", "t", "u", "u", "u", "u", "u", "u", "u", "u", "u", "u", "v", "v", "v", "v", "w", "w", "w", "w", "w", "w", "w", "w", "w", "w", "x", "x", "x", "x", "y", "y", "z", "z", "z", "z", "z", "z", "h", "t", "w", "y", "a", "s", "", "", "", "", "a", "a", "a", "a", "a", "a", "a", "a", "a", "a", "a", "a", "a", "a", "a", "a", "a", "a", "a", "a", "a", "a", "a", "a", "e", "e", "e", "e", "e", "e", "e", "e", "e", "e", "e", "e", "e", "e", "e", "e", "i", "i", "i", "i", "o", "o", "o", "o", "o", "o", "o", "o", "o", "o", "o", "o", "o", "o", "o", "o", "o", "o", "o", "o", "o", "o", "o", "o", "u", "u", "u", "u", "u", "u", "u", "u", "u", "u", "u", "u", "u", "u", "y", "y", "y", "y", "y", "y", "y", "y", "", "", "", "", "", "", }; +char_expansion_page_20_t to_ascii_letters_page_20 [] = {"", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "i", "", "", "", "", "", "", "", "", "", "", "", "", "", "n", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "a", "e", "o", "x", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "rs", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", }; +char_expansion_page_21_t to_ascii_letters_page_21 [] = {"ac", "as", "c", "c", "", "co", "cu", "", "", "f", "g", "h", "h", "h", "h", "", "i", "i", "l", "l", "", "n", "no", "", "", "p", "q", "r", "r", "r", "", "", "sm", "tel", "tm", "", "z", "", "", "", "z", "", "k", "a", "b", "c", "", "e", "e", "f", "", "m", "o", "", "", "", "", "i", "", "fax", "", "", "", "", "", "", "", "", "", "d", "d", "e", "i", "j", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "i", "ii", "iii", "iv", "v", "vi", "vii", "viii", "ix", "x", "xi", "xii", "l", "c", "d", "m", "i", "ii", "iii", "iv", "v", "vi", "vii", "viii", "ix", "x", "xi", "xii", "l", "c", "d", "m", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", }; +char_expansion_page_24_t to_ascii_letters_page_24 [] = {"", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", }; +char_expansion_page_2C_t to_ascii_letters_page_2C [] = {"", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "j", "v", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", }; +char_expansion_page_32_t to_ascii_letters_page_32 [] = {"", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "pte", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "hg", "erg", "ev", "ltd", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", }; +char_expansion_page_33_t to_ascii_letters_page_33 [] = {"", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "hpa", "da", "au", "bar", "ov", "pc", "dm", "dm", "dm", "iu", "", "", "", "", "", "pa", "na", "a", "ma", "ka", "kb", "mb", "gb", "cal", "kcal", "pf", "nf", "f", "g", "mg", "kg", "hz", "khz", "mhz", "ghz", "thz", "l", "ml", "dl", "kl", "fm", "nm", "m", "mm", "cm", "km", "mm", "cm", "m", "km", "mm", "cm", "m", "km", "ms", "ms", "pa", "kpa", "mpa", "gpa", "rad", "rads", "rads", "ps", "ns", "s", "ms", "pv", "nv", "v", "mv", "kv", "mv", "pw", "nw", "w", "mw", "kw", "mw", "k", "m", "am", "bq", "cc", "cd", "ckg", "co", "db", "gy", "ha", "hp", "in", "kk", "km", "kt", "lm", "ln", "log", "lx", "mb", "mil", "mol", "ph", "pm", "ppm", "pr", "sr", "sv", "wb", "vm", "am", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "gal", }; +char_expansion_page_FB_t to_ascii_letters_page_FB [] = {"ff", "fi", "fl", "ffi", "ffl", "st", "st", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", }; +char_expansion_page_FF_t to_ascii_letters_page_FF [] = {"", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "", "", "", "", "", "", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", }; diff --git a/extras/anytermd/libpbe/src/geometry.cc b/extras/anytermd/libpbe/src/geometry.cc new file mode 100644 index 0000000000..96d22c5f5a --- /dev/null +++ b/extras/anytermd/libpbe/src/geometry.cc @@ -0,0 +1,49 @@ +// src/geometry.cc +// (C) 2008 Philip Endecott +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#include "geometry.hh" + +#include + +using namespace std; + + +namespace pbe { + + +float distance(const position& a, const position& b) +{ + return earth_radius * 2.0F * asin( + sqrt( + sqr(degsin((a.lat-b.lat)/2.0F)) + + degcos(a.lat)*degcos(b.lat)*sqr(degsin((a.lng-b.lng)/2.0F)) + ) + ); +} + + +float bearing(const position& a, const position& b) +{ + float d = degatan2( degsin(b.lng-a.lng) * degcos(b.lat), + degcos(a.lat) * degsin(b.lat) + - degsin(a.lat) * degcos(b.lat) * degcos(b.lng-a.lng) + ); + return normalise_unsigned_angle(d); +} + + +}; diff --git a/extras/anytermd/libpbe/src/ip.cc b/extras/anytermd/libpbe/src/ip.cc new file mode 100644 index 0000000000..3e4ddf1074 --- /dev/null +++ b/extras/anytermd/libpbe/src/ip.cc @@ -0,0 +1,141 @@ +// src/ip.cc +// This file is part of libpbe; see http://decimail.org +// (C) 2004-2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#include "ip.hh" + +#include +#include +#include +#include +#include + +using namespace std; + + +namespace pbe { + + +ostream& operator<<(ostream& s, struct in_addr ip) +{ + uint32_t i = ntohl(ip.s_addr); + s << ((i&0xff000000)>>24) + << '.' << ((i&0x00ff0000)>>16) + << '.' << ((i&0x0000ff00)>>8) + << '.' << (i&0x000000ff); + return s; +} + + +void HostLookupError::report(ostream& s) const { + s << "host name lookup failed for " << hostname << ": " + << gai_strerror(error_code) << endl; +} + + +struct in_addr get_ip_address(string hostname) +{ + struct addrinfo ai; + memset(&ai, 0, sizeof(ai)); + ai.ai_family = AF_INET; + + struct addrinfo* res; + int rc = getaddrinfo(hostname.c_str(), "", &ai, &res); + if (rc) { + throw HostLookupError(hostname,rc); + } + + struct in_addr result = ((sockaddr_in *)(res->ai_addr))->sin_addr; + freeaddrinfo(res); + + return result; +} + + +string get_own_hostname(void) +{ + char name[256]; + int rc = gethostname(name,sizeof(name)); + if (rc!=0) { + throw_ErrnoException("gethostname()"); + } + + struct addrinfo ai; + memset(&ai, 0, sizeof(ai)); + ai. ai_flags = AI_CANONNAME; + ai.ai_family = AF_INET; + + struct addrinfo* res; + // FIXME Is there any way to specify a timeout? + rc = getaddrinfo(name, "", &ai, &res); + if (rc) { + throw HostLookupError(name,rc); + } + + string result(res->ai_canonname); + freeaddrinfo(res); + + return result; +} + + + +int tcp_client_connect(string server_name, int port) +{ + struct sockaddr_in addr; + addr.sin_family = AF_INET; + addr.sin_addr = get_ip_address(server_name); + addr.sin_port = htons(port); + + int fd = socket(PF_INET,SOCK_STREAM,0); + if (fd==-1) { + throw_ErrnoException("socket()"); + } + // FIXME if we wanted to detect a timeout here we should make fd nonblocking; + // this will cause connect() to return with errno==EINPROGRESS; we then select() + // until fd is writeable (or times out), at which point we call getsockopt(SO_ERROR) + // to get any connect error. + int rc = connect(fd, (struct sockaddr*)&addr, sizeof(addr)); + if (rc==-1) { + throw_ErrnoException("connect()"); + } + + return fd; +} + + +int local_client_connect(string address) +{ + struct sockaddr_un addr; + addr.sun_family = AF_LOCAL; + strcpy(addr.sun_path,address.c_str()); + + int fd = socket(PF_LOCAL,SOCK_STREAM,0); + if (fd==-1) { + throw_ErrnoException("socket()"); + } + int rc = connect(fd, (struct sockaddr*)&addr, SUN_LEN(&addr)); + if (rc==-1) { + throw_ErrnoException("connect()"); + } + + return fd; +} + + +}; + diff --git a/extras/anytermd/libpbe/src/iso8859_tables.cc b/extras/anytermd/libpbe/src/iso8859_tables.cc new file mode 100644 index 0000000000..e487057ec7 --- /dev/null +++ b/extras/anytermd/libpbe/src/iso8859_tables.cc @@ -0,0 +1,37 @@ +// iso8859_tables.cc +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2008 Philip Endecott + +// Distributed under the Boost Software License, Version 1.0: +// +// Permission is hereby granted, free of charge, to any person or organization +// obtaining a copy of the software and accompanying documentation covered by +// this license (the "Software") to use, reproduce, display, distribute, +// execute, and transmit the Software, and to prepare derivative works of the +// Software, and to permit third-parties to whom the Software is furnished to +// do so, all subject to the following: +// +// The copyright notices in the Software and this entire statement, including +// the above license grant, this restriction and the following disclaimer, +// must be included in all copies of the Software, in whole or in part, and +// all derivative works of the Software, unless such copies or derivative +// works are solely in the form of machine-executable object code generated by +// a source language processor. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT +// SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE +// FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, +// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER +// DEALINGS IN THE SOFTWARE. + + +#include "charset/conv/iso8859.hh" + +namespace pbe { + +#include "generated/iso8859_tables_body.cc" + +}; + diff --git a/extras/anytermd/libpbe/src/markup.cc b/extras/anytermd/libpbe/src/markup.cc new file mode 100644 index 0000000000..53effae99c --- /dev/null +++ b/extras/anytermd/libpbe/src/markup.cc @@ -0,0 +1,138 @@ +// src/markup.cc +// This file is part of libpbe; see http://decimail.org +// (C) 2004 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +#include "markup.hh" + +#include "StringTransformer.hh" + +#include +#include + + +class AtDotToWords: public StringTransformer { +public: + AtDotToWords() { + add_cs_rule('@'," AT "); + add_cs_rule('.'," DOT "); + } +}; + +static AtDotToWords at_dot_to_words; + + +static string render_email(string e) +{ + string h = ""+at_dot_to_words(e)+""; + return h; +} + + +static string render_uri(string u) +{ + string uri; + if (u.substr(0,4)!="http") { + uri = "http://" + u; + } else { + uri=u; + } + string h = ""; + h += u; + h += ""; + return h; +} + + +// See RFC2396 for URI syntax. + +// We make the protocol prefix optional, so www.foo.com works. + +// We require that the last component of the domain name is purely +// alphabetic and has at least two characters. This is true in +// practice (.com, .uk) and helps to avoid false positives ("e.g."). + +// We require that the last character of a path segment is not a +// common punctuation character: .!):;, This is because +// these rarely fall at the end of a true URI but are often placed +// after a URI when it is written in text. + +// We apply this after escaping & to &. This is only an issue in +// the path where ; normally has a special meaning which we have to +// ignore. + +// We reuse the URI domain rules for emails. + +// For email local parts we use the list of characters allowed by +// RFC2822, but allow adjacent .s. + +// We disallow @ in URIs so that we can use it to distinguish emails +// from URIs. They should be allowed in paths, queries and +// fragment-ids. + + +const char* uri_regexp = +"(https?://)?" // optional protocol +"[-a-zA-Z0-9]+(\\.[-a-zA-Z0-9]+)*\\.[a-zA-Z][a-zA-Z]+" // hostname +"(:[0-9]+)?" // optional port number +"(/[-a-zA-Z0-9_.!~*'()%:&;=+$,]*[-a-zA-Z0-9_~*'(%&=+$])*" // path, may be empty +"/?" // path to directory may be terminated with a / +"(\\?[-;/?:&=+$,a-zA-Z0-9_.!~*'()]*)?" // optional queery +"(#[-;/?:&=+$,a-zA-Z0-9_.!~*'()]*)?" // optional fragment-id +; + +const char* email_regexp = +"[a-zA-Z0-9!#$%&'*+-/=?^_`{|}~.]+" // local-part +"@" +"[-a-zA-Z0-9]+(\\.[-a-zA-Z0-9]+)*\\.[a-zA-Z][a-zA-Z]+" // hostname +; + + +void markup_uris_emails ( string& text ) +{ + static bool re_compiled = false; + static regex_t re; + if (!re_compiled) { + string regexp = string("(") + uri_regexp + ")|(" + email_regexp +")"; + int rc = regcomp(&re, regexp.c_str(), REG_EXTENDED); + assert(rc==0); + re_compiled=true; + } + + string r; + regmatch_t match; + + int pos=0; + while (1) { + int rc = regexec(&re, text.c_str()+pos, 1, &match, 0); + assert((rc==0) || (rc==REG_NOMATCH)); + if (rc==REG_NOMATCH) { + r.append(text.substr(pos)); + break; + } + r += text.substr(pos,match.rm_so); + string p = text.substr(pos+match.rm_so,match.rm_eo-match.rm_so); + if (p.find('@')!=p.npos) { + r += render_email(p); + } else { + r += render_uri(p); + } + pos += match.rm_eo; + } + + text = r; +} diff --git a/extras/anytermd/libpbe/src/parse_http_request.cc b/extras/anytermd/libpbe/src/parse_http_request.cc new file mode 100644 index 0000000000..873232320b --- /dev/null +++ b/extras/anytermd/libpbe/src/parse_http_request.cc @@ -0,0 +1,173 @@ +// src/parse_http_request.cc +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2005-2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#include "parse_http_request.hh" + +#include +#include +#include +#include +#include + +#include +#include + +#include + + +using namespace std; +using namespace boost::spirit; + + +namespace pbe { + + +HttpRequest parse_http_request(istream& strm) +{ + HttpRequest request; + + typedef multi_pass > iterator_t; + typedef scanner scanner_t; + typedef rule rule_t; + + string tmp_hn; + string tmp_hv; + + // HTTP/1.1 request parsing, based on: + // RFC2616 + // RFC2396 + // HTTP/1.1 Errata (http://skrb.org/ietf/http_errata.html) + + rule_t CRLF = str_p("\r\n"); + + rule_t extension_method = nothing_p; + + rule_t method = str_p("OPTIONS") | "GET" | "HEAD" | "POST" | "PUT" + | "DELETE" | "TRACE" | "CONNECT" | extension_method; + + rule_t mark = ch_p('-') | '_' | '.' | '!' | '~' | '*' | '\'' | '(' | ')'; + + rule_t unreserved = alnum_p | mark; + + rule_t escaped = ch_p('%') >> xdigit_p >> xdigit_p; + + rule_t reserved = ch_p(';') | '/' | '?' | ':' | '@' | '&' | '=' | '+' | '$' | ','; + + rule_t pchar = unreserved | escaped | ':' | '@' | '&' | '=' | '+' | '$' | ','; + + rule_t param = *pchar; + + rule_t segment = *pchar >> *(';' >> param); + + rule_t path_segments = segment >> *('/' >> segment); + + rule_t abs_path = ( ch_p('/') >> path_segments )[assign_a(request.abs_path)]; + + rule_t scheme = alpha_p >> *(alpha_p | digit_p | '+' | '-' | '.' ); + + rule_t userinfo = *(unreserved | escaped | ';' | ':' | '&' | '=' | '+' | '$' | ',' ); + + rule_t domainlabel = alnum_p | alnum_p >> *(alnum_p | '-') >> alnum_p; + + rule_t toplabel = alpha_p | alpha_p >> *(alnum_p | '-') >> alnum_p; + + rule_t hostname = *(domainlabel >> '.') >> toplabel >> !ch_p('.'); + + uint_parser decimal_byte; + + rule_t ipv4address = decimal_byte >> '.' >> decimal_byte >> '.' >> + decimal_byte >> '.' >> decimal_byte; + + rule_t host = hostname | ipv4address; + + rule_t port = uint_p; + + rule_t hostport = host >> !(':' >> port); + + rule_t server = !( !(userinfo >> '@') >> hostport ); + + rule_t reg_name = +(unreserved | escaped | '$' | ',' | ';' | ':' | '@' | + '&' | '=' | '+'); + + rule_t authority = server | reg_name; + + rule_t net_path = str_p("//") >> authority >> !abs_path; + + rule_t uric = reserved | unreserved | escaped; + + rule_t query = (*uric) [assign_a(request.query)]; + + rule_t hier_part = (net_path | abs_path) >> !('?' >> query); + + rule_t uric_no_slash = unreserved | escaped | ';' | '?' | ':' | '@' | + '&' | '=' | '+' | '$' | ','; + + rule_t opaque_part = uric_no_slash >> *uric; + + rule_t absolute_uri = scheme >> ':' >> (hier_part | opaque_part); + + rule_t request_uri = ch_p('*') + | absolute_uri + | (abs_path >> !(ch_p('?') >> query)) + | authority; + + rule_t http_version = str_p("HTTP/") >> uint_p >> '.' >> uint_p; + + rule_t request_line = + method [assign_a(request.method)] + >> ' ' >> request_uri + >> ' ' >> http_version [assign_a(request.http_version)] + >> CRLF ; + + rule_t header_name = +(alnum_p|'-'); + + rule_t header_value = *(print_p|' '|'\t'); + + rule_t header = ( header_name [assign_a(tmp_hn)] >> + ':' >> *(ch_p(' ')) >> header_value [assign_a(tmp_hv)] ) + [insert_at_a(request.headers,tmp_hn,tmp_hv)]; + + rule_t request_r = + request_line + >> *(header >> CRLF) + >> CRLF; + + iterator_t first(make_multi_pass(std::istreambuf_iterator(strm))); + iterator_t last(make_multi_pass(std::istreambuf_iterator())); + + scanner_t scanner(first,last); + + if (!request_r.parse(scanner)) { + throw HttpRequestSyntaxError(); + } + + HttpRequest::headers_t::const_iterator i = request.headers.find("Content-Length"); + if (i==request.headers.end()) { + return request; + } + size_t content_length = boost::lexical_cast(i->second); + boost::scoped_array buf(new char[content_length]); + strm.read(buf.get(),content_length); + request.body = string(buf.get(),content_length); + + return request; +} + + +}; + diff --git a/extras/anytermd/libpbe/src/rfcdate.cc b/extras/anytermd/libpbe/src/rfcdate.cc new file mode 100644 index 0000000000..80097a94fc --- /dev/null +++ b/extras/anytermd/libpbe/src/rfcdate.cc @@ -0,0 +1,44 @@ +// rfcdate.cc +// This file is part of libpbe; see http://anyterm.org/ +// (C) 2006 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#include + +#include + +using namespace std; + + +namespace pbe { + + +string rfc_date(void) +// RFC2616 cites RFC1123 for date/time format, which cites RFC822 with +// clarifications. +{ + char s[36]; + time_t t; + time(&t); + struct tm tm; + localtime_r(&t,&tm); + strftime(s,sizeof(s),"%a, %d %b %Y %H:%M:%S %z",&tm); + return s; +} + + +}; diff --git a/extras/anytermd/libpbe/src/run_cmd.cc b/extras/anytermd/libpbe/src/run_cmd.cc new file mode 100644 index 0000000000..b6efe0968a --- /dev/null +++ b/extras/anytermd/libpbe/src/run_cmd.cc @@ -0,0 +1,72 @@ +// src/run_cmd.cc +// This file is part of libpbe; see http://decimail.org +// (C) 2005, 2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#include "run_cmd.hh" + +#include "Exception.hh" + +#include +#include + +using namespace std; +using namespace pbe; + +namespace pbe { + +string run_cmd( string cmd, bool& exit_ok ) +{ + cmd+=" 2>&1"; + FILE* p = popen(cmd.c_str(),"r"); + if (!p) { + throw_ErrnoException("popen("+cmd+")"); + } + + string result; + while (1) { + const int bfsz=1024; + char buf[bfsz]; + int n = fread(buf,sizeof(char),bfsz,p); + result.append(buf,n); + if (n +#include +#include +#include +#include + +#include "compiler_magic.hh" + +#define max_backtrace_size 200 + +static void print_backtrace(int signum) +{ + signal(SIGSEGV,SIG_DFL); + signal(SIGBUS,SIG_DFL); + signal(SIGILL,SIG_DFL); + signal(SIGFPE,SIG_DFL); + const char* signame; + switch(signum) { + case SIGSEGV: signame="Segmentation fault"; break; + case SIGBUS: signame="Bus error"; break; + case SIGILL: signame="Illegal instruction"; break; + case SIGFPE: signame="FP exception"; break; + default: signame="unexpected signal"; break; + } + ::write(2,signame,::strlen(signame)); + const char* msg=" detected; Backtrace:\n"; + ::write(2,msg,::strlen(msg)); + void* return_addrs[max_backtrace_size]; + size_t n_return_addrs = ::backtrace(return_addrs,max_backtrace_size); + ::backtrace_symbols_fd(return_addrs,n_return_addrs,2); + raise(signum); +} + + +void get_backtrace_on_segv() +{ + struct sigaction s; + s.sa_handler = &print_backtrace; + sigfillset(&s.sa_mask); + s.sa_flags=0; + sigaction(SIGSEGV,&s,NULL); + sigaction(SIGBUS,&s,NULL); + sigaction(SIGILL,&s,NULL); + sigaction(SIGFPE,&s,NULL); +} + +#else +// Platform doesn't have these functions, so no-op. + +void get_backtrace_on_segv() +{ +} + +#endif + diff --git a/extras/anytermd/libpbe/src/select.cc b/extras/anytermd/libpbe/src/select.cc new file mode 100644 index 0000000000..4f5e2bfa14 --- /dev/null +++ b/extras/anytermd/libpbe/src/select.cc @@ -0,0 +1,132 @@ +// src/select.cc +// This file is part of libpbe; see http://decimail.org +// (C) 2004-2005 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#include "select.hh" + +#include "Exception.hh" + +#include + +#include +#include + +using namespace std; +using namespace pbe; + + +int select_r(int fd1) +{ + fd_set rd_fds; + FD_ZERO(&rd_fds); + fd_set wr_fds; + FD_ZERO(&wr_fds); + fd_set ex_fds; + FD_ZERO(&ex_fds); + + FD_SET(fd1,&rd_fds); + + int rc; + do { + rc = select(fd1+1,&rd_fds,&wr_fds,&ex_fds,NULL); + if (rc==-1) { + if (errno==EINTR) { + continue; + } + throw_ErrnoException("select()"); + } + break; + } while (1); + + if (FD_ISSET(fd1,&rd_fds)) { + return fd1; + } + + return -2; +} + + +int select_rr(int fd1, int fd2) +{ + fd_set rd_fds; + FD_ZERO(&rd_fds); + fd_set wr_fds; + FD_ZERO(&wr_fds); + fd_set ex_fds; + FD_ZERO(&ex_fds); + + FD_SET(fd1,&rd_fds); + FD_SET(fd2,&rd_fds); + + int rc; + do { + rc = select(max(fd1,fd2)+1,&rd_fds,&wr_fds,&ex_fds,NULL); + if (rc==-1) { + if (errno==EINTR) { + continue; + } + throw_ErrnoException("select()"); + } + } while (0); + + if (FD_ISSET(fd1,&rd_fds)) { + return fd1; + } else if (FD_ISSET(fd2,&rd_fds)) { + return fd2; + } + + return -2; +} + + +int select_rt(int fd1, float timeout) +{ + fd_set rd_fds; + FD_ZERO(&rd_fds); + fd_set wr_fds; + FD_ZERO(&wr_fds); + fd_set ex_fds; + FD_ZERO(&ex_fds); + + FD_SET(fd1,&rd_fds); + + struct timeval tv; + float timeout_whole; + float timeout_frac; + timeout_frac = modff(timeout, &timeout_whole); + tv.tv_sec = (int)timeout_whole; + tv.tv_usec = (int)(1000000.0*timeout_frac); + + int rc; + do { + rc = select(fd1+1,&rd_fds,&wr_fds,&ex_fds,&tv); + if (rc==-1) { + if (errno==EINTR) { + continue; + } + throw_ErrnoException("select()"); + } + } while (0); + + if (rc==0) { + return -1; + } else if (FD_ISSET(fd1,&rd_fds)) { + return fd1; + } + + return -2; +} diff --git a/extras/anytermd/libpbe/src/to_ascii_letters.cc b/extras/anytermd/libpbe/src/to_ascii_letters.cc new file mode 100644 index 0000000000..6a8e3d3540 --- /dev/null +++ b/extras/anytermd/libpbe/src/to_ascii_letters.cc @@ -0,0 +1,27 @@ +// src/to_ascii_letters.cc +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#include "charset/to_ascii_letters.hh" + + +namespace pbe { + +#include "generated/to_ascii_letters_tables.cc" + +}; + diff --git a/extras/anytermd/libpbe/src/utils.cc b/extras/anytermd/libpbe/src/utils.cc new file mode 100644 index 0000000000..6014ef4158 --- /dev/null +++ b/extras/anytermd/libpbe/src/utils.cc @@ -0,0 +1,339 @@ +// src/utils.cc +// This file is part of libpbe; see http://decimail.org +// (C) 2004-2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#include "utils.hh" + +#include "Exception.hh" + +#if 0 +#include +#include +#endif + +#include +#include +#include + +using namespace std; + + +namespace pbe { + +#if 0 +string int_to_string(int i) +{ + ostringstream s; + s << i; + return s.str(); +} + +string float_to_string(float f) +{ + char buf[20]; + unsigned int nchars = snprintf(buf,sizeof(buf),"%g",f); + if (nchars>=sizeof(buf)) { + throw "float doesn't fit in buffer"; + } + return buf; +} + +string double_to_string(double f) +{ + char buf[20]; + unsigned int nchars = snprintf(buf,sizeof(buf),"%g",f); + if (nchars>=sizeof(buf)) { + throw "float doesn't fit in buffer"; + } + return buf; +} +#endif + +class NotANumber: public Exception { +private: + string numstr; + string base; +public: + NotANumber(string ns, string b): numstr(ns), base(b) {} + void report(ostream& s) const { + s << '"' << numstr << "\" is not a " << base << " number" << endl; + } +}; + + +#if 0 +int string_to_int(string s) +{ + if (s.size()==0) { + throw NotANumber(s,"decimal"); + } + char* endptr; + int i=strtol(s.c_str(),&endptr,10); + if (*endptr) { + throw NotANumber(s,"decimal"); + } + return i; +} + + +float string_to_float(string s) +{ + if (s.size()==0) { + throw NotANumber(s,"floating point"); + } + char* endptr; + float f=strtof(s.c_str(),&endptr); + if (*endptr) { + throw NotANumber(s,"floating point"); + } + return f; +} + + +double string_to_double(string s) +{ + if (s.size()==0) { + throw NotANumber(s,"floating point"); + } + char* endptr; + double d=strtod(s.c_str(),&endptr); + if (*endptr) { + throw NotANumber(s,"floating point"); + } + return d; +} +#endif + +int hex_string_to_int(string s) +{ + if (s.size()==0) { + throw NotANumber(s,"hex"); + } + char* endptr; + int i=strtol(s.c_str(),&endptr,16); + if (*endptr) { + throw NotANumber(s,"hex"); + } + return i; +} + + +int maybe_hex_string_to_int(string s) +{ + if (s.substr(0,2)=="0x") { + return hex_string_to_int(s.substr(2)); + } else { + return boost::lexical_cast(s); + } +} + + +static string +prefix_backslash(const boost::iterator_range& Match ) +{ + return "\\"+string(Match.begin(),Match.end()); +} + + +string escape_for_dquoted_string(string s) +{ + return boost::find_format_all_copy(s, + boost::token_finder(boost::is_any_of("\\\"")), + prefix_backslash); +} + +string escape_for_squoted_string(string s) +{ + return boost::find_format_all_copy(s, + boost::token_finder(boost::is_any_of("\\\'")), + prefix_backslash); +} + +string escape_for_regexp(string s) +{ + return boost::find_format_all_copy(s, + boost::token_finder(boost::is_any_of("/*+?{().^$\\[")), + prefix_backslash); +} + + +#if 0 +class EscapeForQuotedString: public EscapeInserter { +public: + EscapeForQuotedString(void): EscapeInserter("\\\"") {} +}; + + + +class EscapeForSQuotedString: public EscapeInserter { +public: + EscapeForSQuotedString(void): EscapeInserter("\\\'") {} +}; + + + +class EscapeForRegexp: public EscapeInserter { +public: + EscapeForRegexp(void): EscapeInserter("/*+?{().^$\\[") {} +}; + +const StringTransformer& escape_for_regexp = EscapeForRegexp(); + + +class ReplaceSquoteWithHack: public StringTransformer { +public: + ReplaceSquoteWithHack(void) { + add_cs_rule('\'',"',\"'\",'"); + } +}; + +string escape_for_xpath_string(string s) +{ + if (s.find('\'')==s.npos) { + return "\'"+s+"\'"; + } + if (s.find('\"')==s.npos) { + return "\""+s+"\""; + } + { + static ReplaceSquoteWithHack replace_squote_with_hack; + return "concat('" + replace_squote_with_hack(s) + "')"; + } +} + + + +class ToUpper: public StringTransformer { +public: + ToUpper(void) { + for(int c=0; c<256; ++c) { + add_cc_rule(c,toupper(c)); + } + } +}; + +const StringTransformer& to_upper = ToUpper(); + + +class ToLower: public StringTransformer { +public: + ToLower(void) { + for(int c=0; c<256; ++c) { + add_cc_rule(c,tolower(c)); + } + } +}; + +const StringTransformer& to_lower = ToLower(); + + +class LfToCrlf: public StringTransformer { +public: + LfToCrlf(void) { + add_cs_rule('\n',"\r\n"); + } +}; + +const StringTransformer& lf_to_crlf = LfToCrlf(); + + +string join(const list& strs, string joiner) +{ + if (strs.size()==0) { + return ""; + } + string j; + list::const_iterator i=strs.begin(); + while(1) { + j+=(*i); + ++i; + if (i==strs.end()) { + break; + } else { + j+=joiner; + } + } + return j; +} + + +string trim_whitespace (string s) +{ + string::size_type start = s.find_first_not_of("\r\n \t"); + if (start==s.npos) { + return ""; + } + unsigned int end = s.find_last_not_of("\r\n \t"); + return s.substr(start,(end-start)+1); +} + + +string normalise_whitespace (string s) +{ + string r; + bool skip_spaces=true; + bool add_space=false; + + for(unsigned int i = 0; i /dev/stderr + exit 1 + fi + if [ -f ${mime_types} ] + then + break + fi +done + + +awk '!/^#/ '"{for (i=2; i<=NF; i++) {if (\$i==\"$EXT\") {print \$1; exit;}}}" ${mime_types} + + diff --git a/extras/anytermd/scripts/mk_static_content.sh b/extras/anytermd/scripts/mk_static_content.sh new file mode 100755 index 0000000000..bd23274598 --- /dev/null +++ b/extras/anytermd/scripts/mk_static_content.sh @@ -0,0 +1,53 @@ +# daemon/mk_static_content.sh +# This file is part of Anyterm; see http://anyterm.org/ +# (C) 2005-2007 Philip Endecott + +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#!/bin/sh + +FILES="$@" + +echo "#include \"static_content.hh\"" +echo "#include " +echo "using namespace std;" + +echo 'extern "C" {' +for i in $FILES +do + sym=`echo $i | sed 's/\./_/g'` + echo "extern char _binary_${sym}_start[];" + echo "extern char _binary_${sym}_end[];" +done +echo "};" + +echo "bool get_static_content(string fn, string& mime_type, string& body) {" + +for i in $FILES +do + sym=`echo $i | sed 's/\./_/g'` + FILE="../browser/$i" + TYPE=`mimetype.sh $FILE` + echo " if (fn==\"/$i\") {" + echo " mime_type=\"$TYPE\";" + echo " body=string(_binary_${sym}_start, _binary_${sym}_end);" + echo " return true;" + echo " };" + +done + +echo " return false;" +echo "}" diff --git a/extras/anytermd/src/Activity.cc b/extras/anytermd/src/Activity.cc new file mode 100644 index 0000000000..12823364b0 --- /dev/null +++ b/extras/anytermd/src/Activity.cc @@ -0,0 +1,77 @@ +// common/Activity.cc +// This file is part of SlugTerm; see http://chezphil.org/slugterm +// (C) 2006 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#include "Activity.hh" + +#include +#include +#include +#include + +#include + +#include "select.hh" +#include "Exception.hh" + +#include + +using namespace std; +using namespace pbe; + + +Activity::Activity(onOutput_t onOutput_, onError_t onError_, pbe::FileDescriptor& fd_): + fd(fd_), + terminating(false), + onOutput(onOutput_), + onError(onError_), + output_processor(new Thread(boost::bind(&Activity::process_output,this))) +{} + + +Activity::~Activity() +{ + output_processor->join(); +} + + +void Activity::send(string s) +{ + fd.writeall(s); +} + + +void Activity::process_output(void) +{ + try { try { + while(!terminating) { + string s = fd.readsome(); + onOutput(s); + } + } RETHROW_MISC_EXCEPTIONS } + catch (IOError) { + onError("Subprocess terminated"); + return; + } + catch (Exception& e) { + ostringstream s; + s << "Exception: "; + e.report(s); + onError(s.str()); + return; + } +} diff --git a/extras/anytermd/src/Activity.hh b/extras/anytermd/src/Activity.hh new file mode 100644 index 0000000000..c23e02eb51 --- /dev/null +++ b/extras/anytermd/src/Activity.hh @@ -0,0 +1,56 @@ +// common/Activity.hh +// This file is part of AnyTerm; see http://anyterm.org/ +// (C) 2006-2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef Activity_hh +#define Activity_hh + +#include + +#include +#include + +#include "FileDescriptor.hh" +#include "Thread.hh" + + +class Activity { + +public: + typedef boost::function onOutput_t; + typedef boost::function onError_t; + +protected: + pbe::FileDescriptor& fd; + volatile bool terminating; + +private: + const onOutput_t onOutput; + const onError_t onError; + boost::scoped_ptr output_processor; + void process_output(void); + +public: + Activity(onOutput_t onOutput_, onError_t onError_, pbe::FileDescriptor& fd_); + + virtual ~Activity(); + + void send(std::string s); +}; + + +#endif diff --git a/extras/anytermd/src/Anyterm.cc b/extras/anytermd/src/Anyterm.cc new file mode 100644 index 0000000000..b958b7ac52 --- /dev/null +++ b/extras/anytermd/src/Anyterm.cc @@ -0,0 +1,259 @@ +// daemon/Anyterm.cc +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2005-2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#include "Anyterm.hh" + +#include +#include + +#include +#if defined(__OpenBSD__) || defined(__APPLE__) +// sigemptyset is provided as macros, but actual function is available +// if their names are undefined (with #undef name) +#undef sigemptyset +#endif + +#include +#include + +#include "Error.hh" + +#include "auto_CgiParams.hh" + +#include "SubProcess.hh" +#include "SerialPortActivity.hh" +#include "expand_command.hh" +#include "unicode.hh" + +#include "compiler_magic.hh" + +#include + + +using namespace std; +using namespace pbe; + + +struct SubProcessFactory { + string command; + SubProcessFactory(string command_): command(command_) {} + Activity* operator()(Activity::onOutput_t onOutput, + Activity::onError_t onError, + string host, string user, + string param, int rows=25, int cols=80) { + string exp_cmd = expand_command(command,host,user,param); + return new SubProcess(onOutput,onError,exp_cmd,rows,cols); + } +}; + +struct SerialPortFactory { + string device; + int baudrate; + SerialPortFactory(string device_, int baudrate_): + device(device_), baudrate(baudrate_) {} + Activity* operator()(Activity::onOutput_t onOutput, + Activity::onError_t onError, + PBE_UNUSED_ARG(string host), PBE_UNUSED_ARG(string user), + PBE_UNUSED_ARG(string param), PBE_UNUSED_ARG(int rows), PBE_UNUSED_ARG(int cols)) { + return new SerialPortActivity(onOutput,onError,device,baudrate); + } +}; + + +static void install_sigchld_handler(void); + +static void reap_child(PBE_UNUSED_ARG(int sig)) +{ + // Handler function for SIGCHLD. + // Note that apparently you might get only one SIGCHLD for multiple + // subprocesses if they terminate at about the same time. So you + // can't just have + // wait(NULL); + + ::pid_t rc; + do { + rc = ::waitpid(-1, NULL, WNOHANG); + } while (rc>0); + + install_sigchld_handler(); +} + +static void install_sigchld_handler(void) +{ + struct sigaction sa; + sa.sa_handler=&reap_child; + ::sigemptyset(&sa.sa_mask); + sa.sa_flags=SA_NOCLDSTOP; + ::sigaction(SIGCHLD,&sa,NULL); // should check return value +} + + +Anyterm::Anyterm(std::string command, std::string device, std::string charset, bool diff_, + int max_sessions_): + def_charset(charset), + diff(diff_), + max_sessions(max_sessions_), + reaper_running(false) +{ + if (command!="") { + activityfactory=SubProcessFactory(command); + } else if (device!="") { + activityfactory=SerialPortFactory(device,9600); + } else { + throw "Neither command nor device specified"; + } + + try { + Iconver utf8_to_charset("UTF-8",def_charset); + Iconver charset_to_ucs4(def_charset,UCS4_NATIVE); + } + catch(...) { + throw "Character set not supported by Iconv. Try running iconv -l to find supported " + "character sets. It must be possible to convert from UTF-8 to CHARSET and from " + "CHARSET to UCS-4."; + } + + // We don't want child processes to become zombies when they terminate. + // (It's not OK to signal(SIGCHLD,SIG_IGN) here because [posix exec] + // "If the SIGCHLD signal is set to be ignored by the calling process image, + // it is unspecified whether the SIGCHLD signal is set to be ignored or to + // the default action in the new process image." Installing a handler + // that calls wait is safe because "Signals set to be caught by the calling + // process image shall be set to the default action in the new process image." + install_sigchld_handler(); +} + + +static Anyterm::response_t text_resp(string s) +{ + return Anyterm::response_t("text/plain; charset=\"UTF-8\"", s); +} + +Anyterm::response_t Anyterm::process_request(const HttpRequest& request) +{ + if (!reaper_running) { + // We can't start the reaper thread from the constructor, because this Anyterm + // object is constructed before the daemon forks itself into the background; child + // threads don't survive the fork. So we do it on the first call to process_request. + // There's a race condition here if the first two request processing threads start + // simultaneously. + reaper_running = true; + Thread timed_out_session_reaper + (boost::bind(&Anyterm::run_reaper_thread,this)); + } + + // Parse the arguments and call the appropriate function + + CgiParams params = auto_CgiParams(request); + + try { + + string action = params.get("a"); + + if (action=="open") { + reap_timed_out_sessions(); + + { + locked_sessions_t::reader sessions_rd(sessions); + int n_sessions = distance(sessions_rd->begin(),sessions_rd->end()); + if (n_sessions>=max_sessions) { + throw Error("The maximum number of concurrent sessions has been reached"); + } + } + + session_ptr_t ses(new Session(params.get_as("rows",25), + params.get_as("cols",80), + params.get_as("sb",0), + "[host unknown]", + request.userinfo, + params.get("p"), + ANYTERM_TIMEOUT, + activityfactory, + params.get("ch",def_charset), + diff)); + { + locked_sessions_t::writer sessions_wr(sessions); + (*sessions_wr)[ses->id]=ses; + } + return text_resp(ses->id.str()); + } + + string idstr = params.get("s"); + SessionId id(idstr); + session_ptr_t ses; + { + locked_sessions_t::reader sessions_rd(sessions); + sessions_t::const_iterator s = sessions_rd->find(id); + if (s==sessions_rd->end()) { + throw Error("no such session '"+idstr+"'"); + } + ses = s->second; + } + + if (action=="rcv") { + return text_resp(ses->rcv()); + + } else if (action=="send") { + string k = params.get("k"); + ses->send(k); + return text_resp(""); + + } else if (action=="close") { + { + locked_sessions_t::writer sessions_wr(sessions); + sessions_wr->erase(id); + } + return text_resp(""); + + } else { + throw Error("invalid query string '"+request.query+"'"); + } + } + + catch (Error& E) { + return text_resp("E"+E.get_msg()); + } +} + + +void Anyterm::reap_timed_out_sessions(void) +{ + list timed_out_sessions; // The timed out sessions are transfered to this list + // which is destroyed after the lock on session has been released. + locked_sessions_t::writer sessions_wr(sessions); + for (sessions_t::iterator i = sessions_wr->begin(); + i != sessions_wr->end();) { + sessions_t::iterator next = i; ++next; + if (i->second->timed_out()) { + timed_out_sessions.push_back(i->second); + sessions_wr->erase(i); + } + i = next; + } +} + + +void Anyterm::run_reaper_thread(void) +{ + while (1) { + sleep(30); + reap_timed_out_sessions(); + } +} + diff --git a/extras/anytermd/src/Anyterm.hh b/extras/anytermd/src/Anyterm.hh new file mode 100644 index 0000000000..9da27c330b --- /dev/null +++ b/extras/anytermd/src/Anyterm.hh @@ -0,0 +1,71 @@ +// daemon/Anyterm.hh +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2005-2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + + + +#ifndef Anyterm_hh +#define Anyterm_hh + +#include +#include + +#include + +#include "Locked.hh" + +#include "Session.hh" +#include "config.hh" +#include "HttpRequest.hh" + + +class Anyterm { + +private: + const std::string def_charset; + const bool diff; + const int max_sessions; + typedef boost::shared_ptr session_ptr_t; + typedef std::map sessions_t; + typedef pbe::Locked locked_sessions_t; + locked_sessions_t sessions; + Session::activityfactory_t activityfactory; + bool reaper_running; + +public: + + Anyterm(std::string command, std::string device, std::string charset, bool diff, + int max_sessions_); + + struct response_t { + std::string type; + std::string body; + response_t(std::string t, std::string b): type(t), body(b) {}; + }; + + response_t process_request(const pbe::HttpRequest& request); + + void reap_timed_out_sessions(void); + void run_reaper_thread(void); + +}; + + + + +#endif diff --git a/extras/anytermd/src/AnytermDaemon.cc b/extras/anytermd/src/AnytermDaemon.cc new file mode 100644 index 0000000000..bde3efcf3b --- /dev/null +++ b/extras/anytermd/src/AnytermDaemon.cc @@ -0,0 +1,74 @@ +// daemon/AnytermDaemon.cc +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2005-2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#include "AnytermDaemon.hh" + +#include "Anyterm.hh" +#include "static_content.hh" + +#include +#include + +#include "segv_backtrace.hh" + + +using namespace std; +using namespace pbe; + + +/*virtual*/ void AnytermDaemon::session_start() +{ + get_backtrace_on_segv(); +} + +void AnytermDaemon::handle(const HttpRequest& req0, HttpResponse& resp) +{ + HttpRequest req = req0; + + //syslog(LOG_NOTICE,"anytermd handling %s",req.uri.c_str()); + + resp.headers["Server"]="anytermd"; + + authenticate(req); + + // Redirect a request for '/' to '/anyterm.html'. + if (req.abs_path=="/") { + string host = req.headers.find("Host")->second; + // This is tricky if we're being proxied to because "host" is the post-proxy + // hostname (e.g. locahost) while the browser needs to see the pre-proxy hostname. + // Apache can fix this up for us if we use something like this: + // ProxyPassReverse http://localhost:8080 + resp.headers["Location"]="http://"+host+"/anyterm.html"; + resp.status_code=301; + return; + } + + if (get_static_content(req.abs_path, resp.headers["Content-Type"], resp.body)) { + return; + } + + Anyterm::response_t r = anyterm.process_request(req); + resp.headers["Content-Type"] = r.type; + resp.headers["Cache-Control"] = "no-cache, no-store"; + resp.headers["Pragma"] = "no-cache"; + resp.body = r.body; +} + + + diff --git a/extras/anytermd/src/AnytermDaemon.hh b/extras/anytermd/src/AnytermDaemon.hh new file mode 100644 index 0000000000..1cf7cc21ca --- /dev/null +++ b/extras/anytermd/src/AnytermDaemon.hh @@ -0,0 +1,82 @@ +// daemon/AnytermDaemon.hh +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2005-2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#ifndef AnytermDaemon_hh +#define AnytermDaemon_hh + +#include "HttpDaemon.hh" +#include "HttpRequest.hh" +#include "HttpResponse.hh" +#include "Anyterm.hh" +#include "NullAuthenticator.hh" +#include "TrivialAuthenticator.hh" + + +static inline HttpAuthenticator* choose_authenticator(std::string authname) { + if (authname=="none") { + return NULL; + } else if (authname=="null") { + return new NullAuthenticator; + } else if (authname=="trivial") { + return new TrivialAuthenticator; + } else { + throw "Unrecognised auth type"; + } +} + + +class AnytermDaemon_base { +protected: + boost::scoped_ptr auth_p; + +public: + AnytermDaemon_base(std::string authname): + auth_p(choose_authenticator(authname)) + {} +}; + + +class AnytermDaemon: public AnytermDaemon_base, public pbe::HttpDaemon { +private: + Anyterm anyterm; + +public: + AnytermDaemon(short port=80, std::string user="", + std::string command="", + std::string device="", + std::string name="", + std::string authname="none", + std::string charset="ascii", + bool diff=true, + int max_sessions=20, + int max_http_connections=60, + bool accept_local_only=false): + AnytermDaemon_base(authname), + HttpDaemon(*auth_p.get(), port, (name=="") ? "anyterm" : name, + user, true, max_http_connections, accept_local_only), + anyterm(command, device, charset, diff, max_sessions) + {} + + void session_start(); + void handle(const pbe::HttpRequest& req, pbe::HttpResponse& resp); + +}; + + +#endif diff --git a/extras/anytermd/src/Attributes.hh b/extras/anytermd/src/Attributes.hh new file mode 100644 index 0000000000..e6100a2364 --- /dev/null +++ b/extras/anytermd/src/Attributes.hh @@ -0,0 +1,49 @@ +// common/Attributes.hh +// This file is part of AnyTerm; see http://anyterm.org/ +// (C) 2006-2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef Attributes_hh +#define Attributes_hh + + +struct Attributes { + unsigned fg : 3; + unsigned bg : 3; + unsigned halfbright : 1; + unsigned bold : 1; + unsigned underline : 1; + unsigned blink : 1; + unsigned inverse : 1; + + Attributes(): + fg(7), bg(0), + halfbright(false), bold(false), underline(false), blink(false), inverse(false) + {} + + bool operator==(const Attributes& other) const { + return fg==other.fg && bg==other.bg + && halfbright==other.halfbright && bold==other.bold + && underline==other.underline && blink==other.blink + && inverse==other.inverse; + } + bool operator!=(const Attributes& other) const { + return ! operator==(other); + } +}; + + +#endif diff --git a/extras/anytermd/src/Cell.hh b/extras/anytermd/src/Cell.hh new file mode 100644 index 0000000000..f82390163f --- /dev/null +++ b/extras/anytermd/src/Cell.hh @@ -0,0 +1,36 @@ +// common/Cell.hh +// This file is part of AnyTerm; see http://anyterm.org/ +// (C) 2006-2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef Cell_hh +#define Cell_hh + +#include "Attributes.hh" + +#include "unicode.hh" + + +struct Cell { + ucs4_char c; + Attributes attrs; + + Cell(ucs4_char c_=' '): c(c_) {} + Cell(ucs4_char c_, Attributes attrs_): c(c_), attrs(attrs_) {} +}; + + +#endif diff --git a/extras/anytermd/src/CgiParams.hh b/extras/anytermd/src/CgiParams.hh new file mode 100644 index 0000000000..9e58f5a72d --- /dev/null +++ b/extras/anytermd/src/CgiParams.hh @@ -0,0 +1,56 @@ +// common/CgiParams.hh +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2006 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#ifndef CgiParams_hh +#define CgiParams_hh + +#include +#include +#include + + +class CgiParams: public std::map { + +protected: + typedef std::map basetype; + +public: + std::string get(std::string name, std::string def="") const { + basetype::const_iterator i = find(name); + if (i==end()) { + return def; + } else { + return i->second; + } + } + + template + T get_as(std::string name, T def = T()) const { + basetype::const_iterator i = find(name); + if (i==end()) { + return def; + } else { + return boost::lexical_cast(i->second); + } + } + +}; + + +#endif diff --git a/extras/anytermd/src/Error.hh b/extras/anytermd/src/Error.hh new file mode 100644 index 0000000000..f183951b97 --- /dev/null +++ b/extras/anytermd/src/Error.hh @@ -0,0 +1,49 @@ +// common/Error.hh +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2005 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +// Exception classes to indicate errors + +#ifndef Error_hh +#define Error_hh + +#include + +#include + + +class Error { + private: + std::string msg; + + public: + Error(std::string m): msg(m) {}; + std::string get_msg(void) const {return msg;} +}; + + +class SysError: public Error { + public: + SysError(std::string m): Error(m+": "+strerror(errno)) {}; +}; + + + + + +#endif diff --git a/extras/anytermd/src/NullAuthenticator.hh b/extras/anytermd/src/NullAuthenticator.hh new file mode 100644 index 0000000000..dc4fb5720d --- /dev/null +++ b/extras/anytermd/src/NullAuthenticator.hh @@ -0,0 +1,40 @@ +// daemon/NullAuthenticator.hh +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#ifndef NullAuthenticator_hh +#define NullAuthenticator_hh + +#include "HttpAuthenticator.hh" + +#include "compiler_magic.hh" + + +class NullAuthenticator: public HttpAuthenticator { + +public: + + void basic_auth(PBE_UNUSED_ARG(std::string username), + PBE_UNUSED_ARG(std::string password)) const { + return; + } + +}; + + +#endif diff --git a/extras/anytermd/src/Screen.hh b/extras/anytermd/src/Screen.hh new file mode 100644 index 0000000000..764c96653a --- /dev/null +++ b/extras/anytermd/src/Screen.hh @@ -0,0 +1,140 @@ +// daemon/Screen.hh +// This file is part of AnyTerm; see http://anyterm.org/ +// (C) 2006-2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef Screen_hh +#define Screen_hh + +#include +#include + +#include "Cell.hh" + + +class Screen { + int rows_; + int cols_; + int scrollback_; + typedef std::vector row_t; + typedef std::vector cells_t; + cells_t cells; + unsigned int wrap; + +public: + Screen(int rows, int cols, int scrollback=0): + rows_(rows), cols_(cols), scrollback_(scrollback), + wrap(0), + cursor_row(0), cursor_col(0), cursor_visible(true) + { + for (int r=0; r +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#include "select.hh" +#include "Exception.hh" + + +using namespace std; +using namespace pbe; + + +SerialPortActivity::SerialPortActivity(Activity::onOutput_t onOutput, + Activity::onError_t onError, + string fn, int baudrate): + SerialPortActivity_base(fn,FileDescriptor::read_write,baudrate), + Activity(onOutput, onError, sp) +{} + + + + diff --git a/extras/anytermd/src/SerialPortActivity.hh b/extras/anytermd/src/SerialPortActivity.hh new file mode 100644 index 0000000000..ee04f3837a --- /dev/null +++ b/extras/anytermd/src/SerialPortActivity.hh @@ -0,0 +1,49 @@ +// common/SerialPort.hh +// This file is part of AnyTerm; see http://anyterm.org/ +// (C) 2006 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef SerialPort_hh +#define SerialPort_hh + +#include "Activity.hh" + +#include + +#include "SerialPort.hh" + + +// This base class is just so that sp can be initialised before being Activity. +struct SerialPortActivity_base { + pbe::SerialPort sp; + SerialPortActivity_base(std::string fn, pbe::FileDescriptor::open_mode_t open_mode, + int baudrate, bool raw=true): + sp(fn,open_mode,baudrate,raw) + {} +}; + + +class SerialPortActivity: private SerialPortActivity_base, public Activity { +public: + SerialPortActivity(Activity::onOutput_t onOutput, + Activity::onError_t onError, + std::string fn, int baudrate); + + ~SerialPortActivity() {}; +}; + + +#endif diff --git a/extras/anytermd/src/Session.cc b/extras/anytermd/src/Session.cc new file mode 100644 index 0000000000..3af9b1026c --- /dev/null +++ b/extras/anytermd/src/Session.cc @@ -0,0 +1,196 @@ +// daemon/Session.cc +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2005-2006 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#include "Session.hh" + +#include +#include + +#include +#include + +#include + +#include "Lock.hh" + +#include "config.hh" +#include "html.hh" +#include "editscript.hh" +#include "Error.hh" +#include "Iconver.hh" +#include "unicode.hh" + + +using namespace std; +using namespace pbe; + + +Session::Session(int r, int c, int sb, + string host, string user, string param, + int t, + activityfactory_t af, + string charset_, + bool diff_): + rows(r), + cols(c), + scrollback(sb), + time_out(t), + charset(charset_), + diff(diff_), + utf8_to_charset("UTF-8",charset), + charset_to_ucs4(charset,UCS4_NATIVE), + ucs4_to_utf8(UCS4_NATIVE,"UTF-8"), + screen(rows,cols,scrollback), + error(false), + term(rows,cols,screen), + activity(af(boost::bind(&Session::process_output,this,_1), + boost::bind(&Session::process_error,this,_1), + host, user, param, r, c)) +{ + dirty=true; + touch(); +} + + +Session::~Session() +{} + + +// Timeout unused sessions: + +void Session::touch(void) +{ + last_access = time(NULL); +} + + +// Check if session has a backend error to report + +void Session::report_any_backend_error(void) +{ + if (error) { + error=false; + throw Error(error_msg); + } +} + + +// Handle Apache requests: + +string int_to_string(int i) +{ + char b[32]; + snprintf(b,sizeof(b),"%d",i); + return b; +} + + +void Session::send(string k) +{ + if (!k.empty()) { + string dk = utf8_to_charset(k); + activity->send(dk); + } + + touch(); +} + + + +string escape_html(string s) +{ + string t; + for(string::size_type i=0; i': t+=">"; break; + case '&': t+="&"; break; + default: t+=c; break; + } + } + return t; +} + + +string Session::rcv(void) +{ + { + Lock l(screen_lock); + if (!dirty && !error) { + dirty_condition.timed_wait(l,10.0F); + } + } + touch(); + + report_any_backend_error(); + + bool was_dirty; + ucs4_string html_screen; + { + Lock l(screen_lock); + was_dirty = dirty; + dirty=false; + if (!was_dirty) { + return "n"; + } + html_screen = htmlify_screen(screen); + } + + string utf8_editscript; + if (diff) { + ucs4_string editscript = make_editscript(old_html_screen,html_screen); + old_html_screen=html_screen; + utf8_editscript = ucs4_to_utf8(editscript); + } else { + utf8_editscript = "R"; + utf8_editscript.append(ucs4_to_utf8(html_screen)); + } + + return utf8_editscript; +} + + +void Session::process_output(string s) +{ + ucs4_string ucs4_s = charset_to_ucs4(s); + { + Lock l(screen_lock); + term.write(ucs4_s); + dirty = true; + } + dirty_condition.notify_all(); +} + + +void Session::process_error(string s) +{ + // We could have a lock here, but maybe we can trust that the assignment + // to error is atomic. + error_msg = s; + error = true; + dirty_condition.notify_all(); +} + + +bool Session::timed_out(void) +{ + return time(NULL) - last_access > time_out; +} + diff --git a/extras/anytermd/src/Session.hh b/extras/anytermd/src/Session.hh new file mode 100644 index 0000000000..030e918b85 --- /dev/null +++ b/extras/anytermd/src/Session.hh @@ -0,0 +1,99 @@ +// daemon/Session.hh +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2005-2006 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#ifndef Session_hh +#define Session_hh + +//#define __STDC_LIMIT_MACROS +//#include + +#include "SessionId.hh" + +#include "Mutex.hh" +#include "Condition.hh" + +#include +#include +#include +#include + + +#include "Screen.hh" +#include "Terminal.hh" +#include "Activity.hh" +#include "Iconver.hh" + + +class Session: boost::noncopyable { +public: + + const SessionId id; + const int rows; + const int cols; + const int scrollback; + const int time_out; + const std::string charset; + const bool diff; + +private: + pbe::Iconver utf8_to_charset; + pbe::Iconver charset_to_ucs4; + pbe::Iconver ucs4_to_utf8; + +public: + Screen screen; + typedef pbe::Mutex<> screen_lock_t; + screen_lock_t screen_lock; + volatile bool dirty; + typedef pbe::Condition dirty_condition_t; + dirty_condition_t dirty_condition; + /*volatile*/ std::string error_msg; + volatile bool error; + ucs4_string old_html_screen; + volatile time_t last_access; + + Terminal term; + boost::scoped_ptr activity; + + typedef boost::function activityfactory_t; + Session(int r, int c, int sb, + std::string host, std::string user, std::string param, + int t, + activityfactory_t activityfactory, + std::string charset_, + bool diff_); + ~Session(); + + void touch(void); + void report_any_backend_error(void); + void send(std::string k); + std::string rcv(void); + + bool timed_out(void); + +private: + void process_output(std::string s); + void process_error(std::string s); +}; + + +#endif diff --git a/extras/anytermd/src/SessionId.cc b/extras/anytermd/src/SessionId.cc new file mode 100644 index 0000000000..17300b5224 --- /dev/null +++ b/extras/anytermd/src/SessionId.cc @@ -0,0 +1,53 @@ +// apachemod/SessionId.cc +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2005 Philip Endecott + + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#include "SessionId.hh" + +#include +#include +#include +#include + +using namespace std; + + +SessionId::SessionId() +{ + ifstream devurandom("/dev/urandom"); + devurandom.read(reinterpret_cast(&n),sizeof(n)); +} + +uint64_t SessionId::hexstr_to_uint64(string s) const +{ + stringstream ss; + ss << s; + uint64_t i; + ss >> setbase(16) >> i; + return i; +} + + +string SessionId::uint64_to_hexstr(uint64_t i) const +{ + stringstream ss; + ss << setbase(16) << i; + return ss.str(); +} + + diff --git a/extras/anytermd/src/SessionId.hh b/extras/anytermd/src/SessionId.hh new file mode 100644 index 0000000000..28fb00efb9 --- /dev/null +++ b/extras/anytermd/src/SessionId.hh @@ -0,0 +1,54 @@ +// apachemod/SessionId.hh +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2005 Philip Endecott + + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +// This file defines the 64-bit random ID used to identify sessions + +#ifndef SessionId_hh +#define SessionId_hh + +#include + +#include +#include + + +class SessionId { +public: + SessionId(void); + SessionId(uint64_t i): n(i) {} + SessionId(std::string s): n(hexstr_to_uint64(s)) {} + std::string str(void) const { return uint64_to_hexstr(n); } + bool operator==(SessionId rhs) const { return n==rhs.n; } + bool operator<(SessionId rhs) const { return n +#include +#elif defined(__OpenBSD__) +#include +#include +#elif defined(__APPLE__) +#include +#else +#include +#endif + +#include +#include +#include +#include +#include + +#include + +#include "select.hh" +#include "Exception.hh" + + +using namespace std; +using namespace pbe; + + +static std::pair open_subprocess(string command, int pty_rows, int pty_cols) +{ + struct winsize ws; + ws.ws_row=pty_rows; + ws.ws_col=pty_cols; + ws.ws_xpixel=0; + ws.ws_ypixel=0; + + int fd; + int pid = forkpty(&fd, NULL, NULL, &ws); + if (pid==-1) { + throw_ErrnoException("forkpty()"); + } + if (pid==0) { + setenv("TERM","linux",1); + struct termios t; + tcgetattr(0,&t); // Could fail, but where would we send the error? + t.c_cc[VERASE]=8; // Make ctrl-H (backspace) the erase character. + tcsetattr(0,TCSANOW,&t); // ditto. + execl("/bin/sh","/bin/sh","-c",command.c_str(),NULL); + throw_ErrnoException("execl(/bin/sh -c "+command+")"); // pointless. + } + return make_pair(fd,pid); +} + + +SubProcess::SubProcess(Activity::onOutput_t onOutput, + Activity::onError_t onError, + string command, + int pty_rows, int pty_cols): + SubProcess_base(open_subprocess(command, pty_rows, pty_cols)), + Activity(onOutput, onError, SubProcess_base::fd) +{} + + +SubProcess::~SubProcess() +{ + // We do two things to try to kill the subprocess: we close the fd, + // which really ought to kill it, and we SIGHUP it. The SIGHUP + // by itself may not be sufficient if the process is nohup or + // setuid or something. The close by itself really should be + // sufficient, but I'm keeping the SIGHUP because I'm paranoid. + // The three results that we want are (a) the process dies, + // (b) it does not become a zombie, and (c) the output processor + // thread terminates so that ~Activity can join it. For (c), + // we hope that it will get an error when reading from the fd + // and/or that it sees terminating set. There's a danger that + // we could close the fd and something else could open another + // fd with the same number, which the output processor could read. + // I hope that's not a high probability. + + terminating = true; + + fd_open = false; + try { + SubProcess_base::fd.close(); + // fd.close() can throw. + } catch (...) {} + kill(pid,SIGHUP); +} + + diff --git a/extras/anytermd/src/SubProcess.hh b/extras/anytermd/src/SubProcess.hh new file mode 100644 index 0000000000..405c429001 --- /dev/null +++ b/extras/anytermd/src/SubProcess.hh @@ -0,0 +1,59 @@ +// common/SubProcess.hh +// This file is part of AnyTerm; see http://anyterm.org/ +// (C) 2006 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#ifndef SubProcess_hh +#define SubProcess_hh + +#include "Activity.hh" + +#include + +#include "FileDescriptor.hh" + + +// This base class is just so that fd can be initialised before Activity. +struct SubProcess_base { + pbe::FileDescriptor fd; + bool fd_open; + ::pid_t pid; + SubProcess_base(std::pair fd_pid): + fd(fd_pid.first, false), fd_open(true), pid(fd_pid.second) + {} + ~SubProcess_base() { + if (fd_open) { + // Normally fd is closed by ~SubProcess; this is here in case + // e.g. Activity's ctor throws and ~SubProcess is not executed. + fd.close(); + } + } +}; + + +class SubProcess: private SubProcess_base, public Activity { + +public: + SubProcess(Activity::onOutput_t onOutput_, + Activity::onError_t onError_, + std::string command, + int pty_rows=25, int pty_cols=80); + + ~SubProcess(); +}; + + +#endif diff --git a/extras/anytermd/src/Terminal.cc b/extras/anytermd/src/Terminal.cc new file mode 100644 index 0000000000..53cc1e8914 --- /dev/null +++ b/extras/anytermd/src/Terminal.cc @@ -0,0 +1,789 @@ +// common/Terminal.cc +// This file is part of libpbe; see http://svn.chezphil.org/libpbe/ +// (C) 2006-2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#include "Terminal.hh" + +#include + +#include "Exception.hh" + +using namespace std; + + +static const bool debug_emulation = false; + + +// This terminal emulation code attempts to emulate the Linux console, as described in +// "man 4 console_codes". This has quite a lot in common with other common terminals +// (e.g. vt102, xterm etc) but there are always small differences. +// The starting point for this code is ROTE, and it's because ROTE chose to emulate the +// Linux console that this does the same. ROTE's emulation is not, however, very +// comprehensive. + +// [*] = This code is not implemented, but has been observed in use. + +// We really ought to implement an additional level of character set mode selection. SI +// and SO switch between G0 and G1 character sets, and we implement that assuming that G0 +// and G1 have their default meanings. However, ESC ( and ESC ) can be used to change +// the meaning of G0 and G1; we don't implement that (but could). This would make pstree +// -G work. + + +Terminal::Terminal(int rows, int cols, Screen& screen_): + screen(screen_), + rows_(rows), + cols_(cols), + saved_cursor_row(0), + saved_cursor_col(0), + scrolltop(0), + scrollbottom(rows-1), + charset_mode(0), + crlf_mode(false), + dirty(true), + esc_state(normal), + pc850_to_ucs4("850",UCS4_NATIVE) +{ + charset_modes[0] = cs_normal; + charset_modes[1] = cs_vt100gr; +} + + +Terminal::~Terminal() +{} + + +int Terminal::rows() const +{ + return rows_; +} + +int Terminal::cols() const +{ + return cols_; +} + + +bool Terminal::was_dirty(void) +{ + bool d = dirty; + dirty = false; + return d; +} + + +void Terminal::clip_cursor() +{ + screen.cursor_row = max(screen.cursor_row,0); + screen.cursor_row = min(screen.cursor_row,rows()-1); + screen.cursor_col = max(screen.cursor_col,0); + screen.cursor_col = min(screen.cursor_col,cols()-1); +} + + +void Terminal::cursor_line_down() +{ + screen.cursor_row++; + if (screen.cursor_row<=scrollbottom) { + return; + } + screen.scroll_down(scrolltop,scrollbottom); + screen.cursor_row = scrollbottom; +} + +void Terminal::cursor_line_up() +{ + screen.cursor_row--; + if (screen.cursor_row>=scrolltop) { + return; + } + screen.scroll_up(scrolltop,scrollbottom); + screen.cursor_row = scrolltop; +} + + +void Terminal::write_normal_char(ucs4_char c) +{ + switch (charset_modes[charset_mode]) { + case cs_vt100gr: + switch (c) { + case 'j': c=L'\u255b'; break; // lower right corner + case 'k': c=L'\u2510'; break; // upper right corner + case 'l': c=L'\u250c'; break; // upper left corner + case 'm': c=L'\u2514'; break; // lower left corner + case 'n': c=L'\u253c'; break; // crossing lines + case 'o': c=L'\u2500'; break; // horizontal line - scan 1 + case 'p': c=L'\u2500'; break; // horizontal line - scan 3 + case 'q': c=L'\u2500'; break; // horizontal line - scan 5 + case 'r': c=L'\u2500'; break; // horizontal line - scan 7 + case 's': c=L'\u2500'; break; // horizontal line - scan 9 + case 't': c=L'\u251c'; break; // left T + case 'u': c=L'\u2524'; break; // right T + case 'v': c=L'\u2534'; break; // bottom T + case 'w': c=L'\u252c'; break; // top T + case 'x': c=L'\u2502'; break; // vertical bar + } + break; + case cs_pc: { + string ch(1,c); + pc850_to_ucs4.reset(); + ucs4_string s = pc850_to_ucs4(ch); + c = s[0]; + break; + } + case cs_normal: + break; + } + + if (screen.cursor_col>=cols()) { + screen.cursor_col = 0; + cursor_line_down(); + } + screen(screen.cursor_row,screen.cursor_col) = Cell(c,current_attrs); + screen.cursor_col++; +} + + +void Terminal::carriage_return() +{ + screen.cursor_col = 0; +} + + +void Terminal::line_feed() +{ + cursor_line_down(); + if (crlf_mode) { + carriage_return(); + } +} + + +void Terminal::backspace() +{ + if (screen.cursor_col>0) { + screen.cursor_col--; + } +} + + +void Terminal::tab() +{ + do { + write_normal_char(' '); + } while (screen.cursor_col%8); +} + + +void Terminal::reset() +{ + for (int r=0; r=30 && params[i]<=37) { + current_attrs.fg = params[i]-30; + } else if (params[i]>=40 && params[i]<=47) { + current_attrs.bg = params[i]-40; + } else { + if (debug_emulation) { + cout << "Ignoring attribute " << params[i] << "\n"; + } + } + break; + } + } +} + + +void Terminal::csi_SM() +{ + // Set modes. + // Not implemented. + if (nparams!=1) { + return; + } + + switch (params[0]) { + //case 3: DECCRM mode - display control characters break; + //case 4: DECIM mode - insert mode break; + case 20: crlf_mode = true; + default: if (debug_emulation) { + cout << "Ignoring SM mode " << params[0] << "\n"; + } + } +} + + +void Terminal::csi_RM() +{ + // Reset modes. + // These modes are not implemented, so resetting them does nothing. + if (nparams!=1) { + return; + } + + switch (params[0]) { + case 3: break; // DECCRM mode - display control characters + case 4: break; // DECIM mode - insert mode + case 20: crlf_mode = false; break; + default: if (debug_emulation) { + cout << "Ignoring RM mode " << params[0] << "\n"; + } + } +} + + +void Terminal::csi_DSR() +{ + // Report status. + // Not implemented. + if (nparams!=1) { + return; + } + + switch (params[0]) { + //case 5: DSR - device status - reply ESC [ 0 n break; + //case 6: CPR - cursor position - reply ESC [ y ; x R break; + default: if (debug_emulation) { + cout << "Ignoring status report request " << params[0] << "\n"; + } + } +} + + +void Terminal::csi_ED() +{ + // Erase display. + int start_row; + int start_col; + int end_row; + int end_col; + + if (nparams>0 && params[0]==1) { + start_row = 0; + start_col = 0; + end_row = screen.cursor_row; + end_col = screen.cursor_col; + + } else if (nparams>0 && params[0]==2) { + start_row = 0; + start_col = 0; + end_row = rows()-1; + end_col = cols()-1; + + } else { + start_row = screen.cursor_row; + start_col = screen.cursor_col; + end_row = rows()-1; + end_col = cols()-1; + } + + for (int r=start_row; r<=end_row; ++r) { + for (int c=(r==start_row?start_col:0); c<=(r==end_row?end_col:cols()-1); ++c) { + screen(r,c) = Cell(' ',current_attrs); + } + } +} + + +void Terminal::csi_CUP() +{ + // Move cursor to absolute position. + // With no parameters, move to origin. + if (nparams==0) { + screen.cursor_row = 0; + screen.cursor_col = 0; + } else { + screen.cursor_row = params[0]-1; + screen.cursor_col = params[1]-1; + clip_cursor(); + } +} + +void Terminal::csi_HVP() +{ + csi_CUP(); +} + + +void Terminal::csi_CUU() +{ + // Cursor Up. + int n = params[0]; + screen.cursor_row -= n; + clip_cursor(); +} + + +void Terminal::csi_CUD() +{ + // Cursor Down. + int n = params[0]; + screen.cursor_row += n; + clip_cursor(); +} + +void Terminal::csi_VPR() +{ + csi_CUD(); +} + + +void Terminal::csi_CUF() +{ + // Cursor Forward (right). + int n = params[0]; + screen.cursor_col += n; + clip_cursor(); +} + +void Terminal::csi_HPR() +{ + csi_CUF(); +} + +void Terminal::csi_CUB() +{ + // Cursor Back (left). + int n = params[0]; + screen.cursor_col -= n; + clip_cursor(); +} + + +void Terminal::csi_CNL() +{ + // Cursor next line. + csi_CUD(); + screen.cursor_col = 0; + clip_cursor(); +} + + +void Terminal::csi_CPL() +{ + // Cursor previous line. + csi_CUU(); + screen.cursor_col = 0; + clip_cursor(); +} + + +void Terminal::csi_CHA() +{ + screen.cursor_col = params[0]-1; + clip_cursor(); +} + +void Terminal::csi_HPA() +{ + csi_HPA(); +} + + +void Terminal::csi_VPA() +{ + screen.cursor_row = params[0]-1; + clip_cursor(); +} + + +void Terminal::csi_EL() +{ + // Erase line. + int start; + int end; + + if (nparams>0 && params[0]==1) { + start = 0; + end = screen.cursor_col; + + } else if (nparams>0 && params[0]==2) { + start = 0; + end = cols()-1; + + } else { + start = screen.cursor_col; + end = cols()-1; + } + + for (int i=start; i<=end; ++i) { + screen(screen.cursor_row,i) = Cell(' ',current_attrs); + } +} + + +void Terminal::csi_ICH() +{ + // Insert blanks. + int n = params[0]; + for (int i=cols()-1; i>=screen.cursor_col+n; --i) { + screen(screen.cursor_row,i) = screen(screen.cursor_row,i-n); + } + for (int i=screen.cursor_col; inewbottom) { + return; + } + + scrolltop = newtop; + scrollbottom = newbottom; +} + + +void Terminal::csi_SAVECUR() +{ + // Save cursor position. + saved_cursor_row = screen.cursor_row; + saved_cursor_col = screen.cursor_col; +} + + +void Terminal::csi_RESTORECUR() +{ + // Restore cursor position. + screen.cursor_row = saved_cursor_row; + screen.cursor_col = saved_cursor_col; +} + + +void Terminal::csi_DECSET() +{ + if (nparams!=1) { + return; + } + + switch (params[0]) { + //case 1: Change cursor key prefix break; + //case 3: 80/132-column mode break; + //case 5: Reverse video mode break; + //case 6: Scroll-region-relative cursor addressing mode break; + //case 7: Autowrap mode break; [*] + //case 8: Autorepeat break; + //case 9: X10 mouse reporting break; + case 25: screen.cursor_visible=true; break; + //case 1000: X11 mouse reporting break; + default: if (debug_emulation) { + cout << "Ignoring DECSET " << params[0] << "\n"; + } + } +} + + +void Terminal::csi_DECRST() +{ + if (nparams!=1) { + return; + } + + switch (params[0]) { + //case 1: Change cursor key prefix break; + //case 3: 80/132-column mode break; + //case 5: Reverse video mode break; + //case 6: Scroll-region-relative cursor addressing mode break; + //case 7: Autowrap mode break; + //case 8: Autorepeat break; + //case 9: X10 mouse reporting break; + case 25: screen.cursor_visible=false; break; + //case 1000: X11 mouse reporting break; + default: if (debug_emulation) { + cout << "Ignoring DECSET " << params[0] << "\n"; + } + } +} + + +void Terminal::write_char(ucs4_char c) +{ + if (c<=31) { + switch (c) { + case '\a': /* bell */ break; + case '\b': backspace(); break; + case '\t': tab(); break; + case '\n': /* fall through */ + case '\v': /* fall through */ + case '\f': line_feed(); break; + case '\r': carriage_return(); break; + case '\x0E': charset_mode=1; break; + case '\x0F': charset_mode=0; break; + case '\x18': /* fall through */ + case '\x1A': esc_state = normal; break; + case '\x1B': esc_state = seen_esc; break; + default: if (debug_emulation) { + cout << "Ignoring ctrl character " << static_cast(c) << "\n"; break; + } + } + dirty = true; + + } else if (c==0x9b) { + // Is there a conflict between 9b==CSI and a UTF8 or ISO-8859 interpretation? + esc_state = seen_csi; nparams=0; params[0]=1; + + } else { + switch (esc_state) { + case normal: + write_normal_char(c); + dirty = true; + break; + + case seen_esc: + switch (c) { + case 'c': reset(); dirty=true; esc_state=normal; break; + case 'D': line_feed(); dirty=true; esc_state=normal; break; + case 'E': carriage_return(); dirty=true; esc_state=normal; break; + //case 'H': set_tab_stop(); esc_state=normal; break; [*] + case 'M': cursor_line_up(); dirty=true; esc_state=normal; break; + //case 'Z': dec_priv_ident(); esc_state=normal; break; // kernel returns ESC [ ? 6 c + //case '7': save_state(); esc_state=normal; break; // save cursor pos, attributes, charsets + //case '8': restore_state(); dirty=true; esc_state=normal; break; + case '[': esc_state=seen_csi; nparams=0; params[0]=1; break; + //case '%': esc_state=seen_escpercent; break; // select character set based on next char + // @=8859-1, G=8=UTF-8 + //case '#': esc_state=seen_eschash; break; // ESC # 8 = fill screen with Es + case '(': esc_state=seen_esclparen; break; // select G0 charset based on next char + case ')': esc_state=seen_escrparen; break; // select G1 charset based on next char + //case '>': numeric_keypad_mode(); esc_state=normal; break; + //case '=': application_keypad_mode(); esc_state=normal; break; + //case ']': esc_state=seen_escrbraket; break; + // ESC ] P nrrggbb = set palette; colour n (hex) + // ESC ] R = reset palette [*] + default: if (debug_emulation) { + cout << "Ignoring esc character " << c << "\n"; + } + esc_state = normal; + } + break; + + case seen_csi: /* fall through */ + case seen_csi_private: + if (c>='0' && c<='9') { + if (nparams==0) { + nparams=1; + params[0]=0; + } + params[nparams-1] = params[nparams-1] * 10 + (c-'0'); + + } else if (c==';') { + if (nparams>=nparams_max) { + return; + } + nparams++; + params[nparams-1] = 0; + + } else if (c=='?') { + esc_state = seen_csi_private; + + } else { + if (esc_state==seen_csi_private) { + switch (c) { + //case 'c': Unknown; code seen but not described in 'man console_codes' [*] + case 'h': csi_DECSET(); break; + case 'l': csi_DECRST(); break; + default: if (debug_emulation) { + cout << "Ignoring private csi character " << c << "\n"; + } + } + } else { + switch (c) { + case '@': csi_ICH(); break; + case 'A': csi_CUU(); break; + case 'B': csi_CUD(); break; + case 'C': csi_CUF(); break; + case 'D': csi_CUB(); break; + case 'E': csi_CNL(); break; + case 'F': csi_CPL(); break; + case 'G': csi_CHA(); break; + case 'H': csi_CUP(); break; + case 'J': csi_ED(); break; + case 'K': csi_EL(); break; + case 'L': csi_IL(); break; + case 'M': csi_DL(); break; + case 'P': csi_DCH(); break; + case 'X': csi_ECH(); break; + case 'a': csi_HPR(); break; + //case 'c': csi_DA(); break; // Reply ESC [ ? 6 c + case 'd': csi_VPA(); break; + case 'e': csi_VPR(); break; + case 'f': csi_HVP(); break; + //case 'g': csi_TBC(); break; // Clear tab stop [*] + case 'h': csi_SM(); break; + case 'l': csi_RM(); break; + case 'm': csi_SGR(); break; + case 'n': csi_DSR(); break; + //case 'q': csi_DECLL(); break; // Set keyboard LEDs + case 'r': csi_DECSTBM(); break; + case 's': csi_SAVECUR(); break; + case 'u': csi_RESTORECUR(); break; + case '`': csi_HPA(); break; + default: if (debug_emulation) { + cout << "Ignoring csi character " << c << "\n"; + } + } + } + dirty = true; + esc_state = normal; + + } + break; + + case seen_esclparen: /* fall through */ + case seen_escrparen: { + charset_mode_t m = cs_normal; + switch (c) { + case 'B': m = cs_normal; break; + case '0': m = cs_vt100gr; break; + case 'U': m = cs_pc; break; + } + if (esc_state == seen_esclparen) { + charset_modes[0] = m; + } else { + charset_modes[1] = m; + } + esc_state = normal; + break; + } + } + } +} + + +void Terminal::write(ucs4_string data) +{ + for (size_t i=0; i + +#include "Cell.hh" +#include "Attributes.hh" +#include "Screen.hh" +#include "unicode.hh" +#include "Iconver.hh" + + +class Terminal { + +public: + Terminal(int rows, int cols, Screen& screen_); + ~Terminal(); + + int rows(void) const; + int cols(void) const; + + Screen& screen; + + bool was_dirty(void); + + void write(ucs4_string data); + +private: + int rows_; + int cols_; + + int saved_cursor_row; + int saved_cursor_col; + + int scrolltop; + int scrollbottom; + + Attributes current_attrs; + + enum charset_mode_t { cs_normal, cs_pc, cs_vt100gr }; + charset_mode_t charset_modes[2]; + int charset_mode; + + bool crlf_mode; + + bool dirty; + + int nparams; + static const int nparams_max = 16; + int params[nparams_max]; + + enum esc_state_t { normal, seen_esc, seen_csi, seen_csi_private, seen_esclparen, seen_escrparen }; + esc_state_t esc_state; + + pbe::Iconver pc850_to_ucs4; + + void clip_cursor(); + void cursor_line_down(); + void cursor_line_up(); + void write_normal_char(ucs4_char c); + void carriage_return(); + void line_feed(); + void backspace(); + void tab(); + void reset(); + void csi_SGR(); + void csi_SM(); + void csi_RM(); + void csi_DSR(); + void csi_ED(); + void csi_CUP(); + void csi_HVP(); + void csi_CUU(); + void csi_CUD(); + void csi_VPR(); + void csi_CUF(); + void csi_HPR(); + void csi_CUB(); + void csi_CNL(); + void csi_CPL(); + void csi_CHA(); + void csi_HPA(); + void csi_VPA(); + void csi_EL(); + void csi_ICH(); + void csi_DCH(); + void csi_IL(); + void csi_DL(); + void csi_ECH(); + void csi_DECSTBM(); + void csi_SAVECUR(); + void csi_RESTORECUR(); + void csi_DECSET(); + void csi_DECRST(); + void write_char(ucs4_char c); +}; + + +#endif diff --git a/extras/anytermd/src/TrivialAuthenticator.hh b/extras/anytermd/src/TrivialAuthenticator.hh new file mode 100644 index 0000000000..68d135db26 --- /dev/null +++ b/extras/anytermd/src/TrivialAuthenticator.hh @@ -0,0 +1,39 @@ +// daemon/TrivialAuthenticator.hh +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#ifndef TrivialAuthenticator_hh +#define TrivialAuthenticator_hh + +#include "HttpAuthenticator.hh" + +class TrivialAuthenticator: public HttpAuthenticator { + +public: + + void basic_auth(std::string username, std::string password) const { + if (username=="foo" && password=="blah") { + return; + } + throw NotAuthenticated(); + } + +}; + + +#endif diff --git a/extras/anytermd/src/UrlEncodedCgiParams.cc b/extras/anytermd/src/UrlEncodedCgiParams.cc new file mode 100644 index 0000000000..b56d76eca4 --- /dev/null +++ b/extras/anytermd/src/UrlEncodedCgiParams.cc @@ -0,0 +1,105 @@ +// common/UrlEncodedCgiParams.cc +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2006 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#include "UrlEncodedCgiParams.hh" + +#include "Exception.hh" + + +#include +#include +#include +#include + +using namespace std; +using namespace pbe; +using namespace boost::lambda; + + +// Parse x-www-form-urlencoded parameters as defined in HTML 4.01 +// section 17.13.4: +// +// 1. Control names and values are escaped. Space characters are +// replaced by `+', and then reserved characters are escaped as +// described in [RFC1738], section 2.2: Non-alphanumeric characters +// are replaced by `%HH', a percent sign and two hexadecimal digits +// representing the ASCII code of the character. Line breaks are +// represented as "CR LF" pairs (i.e., `%0D%0A'). +// +// 2. The control names/values are listed in the order they appear +// in the document. The name is separated from the value by `=' and +// name/value pairs are separated from each other by `&'. + + + + +static int hexchar(char c) +{ + if (c>='0' && c<='9') { + return c-'0'; + } else if (c>='a' && c<='f') { + return c-'a'+10; + } else if (c>='A' && c<='F') { + return c-'A'+10; + } else { + throw StrException(string("Invalid hex character '")+c+"'"); + } +} + +static char decode_percent_escape(string s) +{ + return hexchar(s[1])*16 + hexchar(s[2]); +} + + +static string decode_uri_escapes(string s) +{ + string t; + for (string::size_type i=0; i name_value_pairs_t; + name_value_pairs_t name_value_pairs; + boost::split(name_value_pairs, query, _1=='&'); + + for(name_value_pairs_t::const_iterator i = name_value_pairs.begin(); + i != name_value_pairs.end(); ++i) { + + string name_value_pair = *i; + string::size_type equals_pos = name_value_pair.find('='); + if (equals_pos==name_value_pair.npos) { + throw StrException("Misformatted URL-encoded query string component '" + +name_value_pair+"' does not contain an '='"); + } + string name = name_value_pair.substr(0,equals_pos); + string value = name_value_pair.substr(equals_pos+1); + + insert(make_pair(decode_uri_escapes(name),decode_uri_escapes(value))); + } +} diff --git a/extras/anytermd/src/UrlEncodedCgiParams.hh b/extras/anytermd/src/UrlEncodedCgiParams.hh new file mode 100644 index 0000000000..14efa52882 --- /dev/null +++ b/extras/anytermd/src/UrlEncodedCgiParams.hh @@ -0,0 +1,36 @@ +// common/UrlEncodedCgiParams.hh +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2006 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#ifndef UrlEncodedCgiParams_hh +#define UrlEncodedCgiParams_hh + +#include "CgiParams.hh" + +#include + + +class UrlEncodedCgiParams: public CgiParams { + +public: + UrlEncodedCgiParams(std::string query); + +}; + + +#endif diff --git a/extras/anytermd/src/auto_CgiParams.cc b/extras/anytermd/src/auto_CgiParams.cc new file mode 100644 index 0000000000..720a8199ca --- /dev/null +++ b/extras/anytermd/src/auto_CgiParams.cc @@ -0,0 +1,42 @@ +// common/auto_CgiParams.cc +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2006 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#include "auto_CgiParams.hh" + +#include "UrlEncodedCgiParams.hh" + +#include + +using namespace std; +using namespace pbe; + + +CgiParams auto_CgiParams(HttpRequest request) +{ + if (request.method=="GET") { + return UrlEncodedCgiParams(request.query); + } else if (request.method=="POST") { + if (boost::algorithm::starts_with(request.headers["Content-Type"], + "application/x-www-form-urlencoded")) { + return UrlEncodedCgiParams(request.body); + } + } + + return CgiParams(); +} diff --git a/extras/anytermd/src/auto_CgiParams.hh b/extras/anytermd/src/auto_CgiParams.hh new file mode 100644 index 0000000000..f5033deb5d --- /dev/null +++ b/extras/anytermd/src/auto_CgiParams.hh @@ -0,0 +1,30 @@ +// common/auto_CgiParams.hh +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2006 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#ifndef auto_CgiParams_hh +#define auto_CgiParams_hh + +#include "CgiParams.hh" +#include "HttpRequest.hh" + + +CgiParams auto_CgiParams(pbe::HttpRequest request); + + +#endif diff --git a/extras/anytermd/src/config.hh b/extras/anytermd/src/config.hh new file mode 100644 index 0000000000..47c21a828f --- /dev/null +++ b/extras/anytermd/src/config.hh @@ -0,0 +1,26 @@ +// daemon/config.hh +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2005-2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#ifndef config_hh +#define config_hh + +#define ANYTERM_TIMEOUT 30 + + +#endif diff --git a/extras/anytermd/src/diff.cc b/extras/anytermd/src/diff.cc new file mode 100644 index 0000000000..582b526855 --- /dev/null +++ b/extras/anytermd/src/diff.cc @@ -0,0 +1,544 @@ +// common/diff.cc +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2005 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#include "diff.hh" + +#include +#include +using namespace std; + + +namespace DiffAlgo { + + // The algorithm used is the one described in + // "An O(ND) Difference Algorithm and Its Variations" + // by Eugene W Myers. + // (Postscript can be found on Myers' web page.) + + // O(ND) refers to N = the size of the input (the sum of the two) + // and D = the length of the resulting "edit script", i.e. the + // number of differences. The paper notes that the expected + // performance is O(N+D^2), with O(ND) being a pathological case. + // This is the complexity for the first phase of the computation in + // which the "edit script size" is found. There is a second phase + // during which the actual "edit script" is determined. This runs + // in O(N) time, but has space complexity O(D^2) (or worse O(ND) if + // the implementation is naive). The paper presents an alternative + // implementation for the second phase in section 4b which has O(N) + // space requirements. This is NOT implemented here. It is likely + // that for non-trivial applications, space complexity is likely to + // be a concern. Comparing sequences of only a few thousand items + // with signficant differences, i.e. an "edit script length" of a + // few thousand will consume megabytes, yet execution time is only + // seconds. (On the other hand, when the inputs are similar, space + // performance is good.) + + // Myers and others have proposed improvements to this algorithm, + // including one in the following paper: + // "An O(NP) Sequence Comparison Algorithm" + // Sun Wu, Udi Manber, Gene Myers + // (Postscript ditto). + + // In this case, P is the number of deletions in the "edit script", + // which is less than the size of the script D. This has NOT been + // implemented here. + + // Here is a quick overview of the algorithm: + + // The two input strings are A and B. Imagine a grid with A + // labelling the columns and B labelling the rows. There are + // additional "zeroth" rows and columns. Say A = abc and B = baba: + + // a b c + // . . . . + // b . . * . + // a . * . . + // b . . * . + // a . * . . + + // The points where the labels on the rows and columns match are + // marked specially. + + // The aim is to find a path from the top-left to the bottom-right + // of this grid in the following way: + // - Rightward horizontal moves indicate taking an element from + // sequence A. + // - Downward vertical moves indicate taking an element from + // sequence B. + // - Down-Right diagonal moves indicate taking an element common to + // A and B. + // Horizontal and vertical moves are always allowed. Diagonal moves + // are only allowed in order to reach the (specially marked) match + // points. The aim is to find a path obeying these constraints that + // has the fewest horizontal and vertical moves, and hence the most + // diagonal moves. + // In the example above, a possible solution is as follows: + + // a b c + // + . . . + // b | . * . + // a .\* . . + // b . .\*-. + // a . * . | + + // (i.e.: vertically to take b, diagonally twice to take a and b, + // horizontally to take c, and vertically to take a.) + + // The algorithm searches in a greedy fashion, that is, it is + // breadth-first when it is doing badly, but then depth-first once + // it is "on to a good thing" (i.e. a diagonal). It expands a + // "frontier" across the grid from the top-left towards the bottom + // right. Once the frontier hits the bottom-right, the problem is + // solved. + + // Giving each horizontal and vertical move a cost of one and each + // diagonal move a cost of zero, all points on the frontier during a + // particular iteration have the same cost. The variable d is used + // to refer to the costs (d is the number of differences). + + // Points on the grid can be referred to using (x,y) coordinates. + // Row/column 0 are necessarily "empty" and do not correspond to + // elements of A or B. Care is needed with off-by-one errors, since + // the sequences A and B are indexed from 0. + + // Points can also be referred to using one or other of x and y (x + // by convention) and the "diagonal index", k, defined by k=x-y. + + // The point about the diagonal index is that the frontier will + // always expand in such a way that it cuts each diagonal exactly + // once: + + // k + // 2 + // 1 / . + // 0 \ . + // -1 \ . + // -2 /\/\/ . + // / . + + // So we can record the position of the frontier by giving the + // x-coordinate for each value of k. In the code, the vector V + // records these values. + + // Thinking of "snakes and ladders", diagonals are named "snakes" + // (though surely, since they lead toward the goal, they should be + // ladders?). A "snake" is a (possibly empty or singleton) sequence + // of diagonals. + + // Once the frontier has reached the target, a second phase of the + // algorithm identifies the optimal path by studying saved copies of + // the V vector from each step of the expansion. This is the + // space-hungry step mentioned above. + + // To reduce the space-hungryness from O(ND) to O(D^2), something of + // a hack is used. The first phase is run twice. In the first run, + // nothing is stored (so memory use is moderate). At the end of + // this run, the "edit script length" is known. In the second run, + // the V vectors are stored but this knowledge is used to limit + // their size. + + + + // The Differ class is template-parameterised by the sequence type + // that it operates on. This is normally string, but if you want to + // use a different type, you should just be able to create a Differ + // object specifying a different type. vector should + // work, as long as operator= is defined on 'something'. See the + // end of the file for how Differ is used. + + template + class Differ { + + private: + const SEQ& A; // Input sequences + const SEQ& B; + const int N; // length of A + const int M; // length of B + const int max_D; + bool store; + + // Output + typename fragment_seq::Type& result; + + // Ideally V would be an array indexed from -(M+N) to (M+N) + // inclusive, but we only have zero-indexed arrays. So we use a + // zero-indexed array and apply an offset. + + const int V_size; + const int V_offset; + + typedef vector V_impl_t; + V_impl_t V_impl; + int& V ( int k ) { return V_impl[V_offset+k]; } + + + typedef vector stored_V_impls_t; + stored_V_impls_t stored_V_impls; + int stored_V ( int d, int k ) const { return (stored_V_impls[d])[V_offset+k]; } + + // This is filled in when solve() finishes. If all that is wanted + // is to know the distance between the two inputs, there is no + // need to call find_trace() at all; just read this using + // get_edit_distance(). + int edit_distance; + + + // Append an item to the result, with a tag. + // If the tag matches the tag of the current end of the result, it + // is merged with it. + void append_result ( fragment_tag tag, typename SEQ::value_type datum ) + { + if (!result.empty() && (result.back().first == tag)) { + result.back().second.push_back(datum); + } else { + result.push_back(make_pair(tag,SEQ(1,datum))); + } + } + + // Follow any snake from (k,x) to its end, and return the x + // coordinate at the end. + int follow_snake ( int k, int x ) + { + int y = x - k; + while ( (x>=0) && (x=0) && (y=0) && (x=0) && (y::Type& r, + bool s = true, int md = -1): + A(a), + B(b), + N(A.size()), + M(B.size()), + max_D((md==-1)?(M+N):(min(md,M+N))), + store(s), + result(r), + V_size(max(2*(max_D)+1,2)), // 2 allows for d=0 V(1) special case + V_offset(max_D), + V_impl(V_size) + {} + + class max_D_exceeded {}; // Exception thrown if solution has not + // been found after max_D + // frontier-expansion iterations. + + // Perform the first phase of the algorithm, expanding the + // frontier. + // This function is essentially what is described in Figure 2 of + // Myers' paper. + void solve ( void ) + { + // The normal operation is (H or V) then any diagonal then repeat. + // But this is broken if the first diagonal starts from the origin (e.g. for equal strings). + // The following is a hack that works around that: + V(1) = 0; + // But this requires that V is large enough for this extra element. + + // Loop for increasing values of D until target reached + int D = -1; + bool done=false; + while (!done) { + ++D; + if (D>max_D) { + throw max_D_exceeded(); + } + + // Scan across the width of the frontier + for ( int k = -D; k <= D; k += 2 ) { + + // Find a new x value for this point on the frontier. + // Special cases for either end. + // Otherwise, move horizontally or vertically from a neighbour. + int x; + if ( (k==-D) || ((k!=D) && (V(k-1)=N) && (y>=M) ) { + done = true; + // We could probably leave the inner loop at this point, + // but I'm not certain it's safe, and it certainly makes + // debugging harder when only some of the points have been + // updated, so don't bother. + //break; + } + } + + // Save a copy of V for use during the second phase. + if (store) { + stored_V_impls.push_back(V_impl); + } + } + + edit_distance = D; + } + + + void find_trace ( void ) + { + find_trace_r ( edit_distance, N-M ); + } + + + int get_edit_distance(void) { return edit_distance; } + }; + + + + void make_trivial_solution ( const string& A, const string& B, string_fragment_seq& result ) + { + result.push_back(make_pair(from_a,A)); + result.push_back(make_pair(from_b,B)); + } + + + void string_diff ( const string& A, const string& B, string_fragment_seq& result ) + { + // Consider time efficiency. Aim not to take more than this much + // time (arbitary units). Return a sub-optimal solution if this + // time is exceeded. + const int max_time = 1000; + + // Consider space efficiency. Aim not to use more than this much + // memory (arbitary units). Take more time or return a + // sub-optimal solution if this much memory is exceeded. + const int max_mem = 10000000; + + // Consider changing the above settings if "top" shows that the + // apache frontend request-handling processes are using more + // memory or CPU time than you would like. + + // Reducing them means that Anyterm will give up looking for an + // edit script and just send the complete new screen sooner. So + // making them too low will use more network bandwidth. On the + // other hand, for a fast local network, you might get a faster + // response with a lower max_time setting. + + + try { + + int sz = A.size() + B.size(); + // If input is small, i.e. N^2 is acceptable, we don't worry about + // space complexity. (This will take O(ND) space, but D could + // equal N.) + if ((sz*sz) d1(A,B,result,true,max_time); + d1.solve(); + d1.find_trace(); + return; + } + + // If input is larger, do a first pass to find the edit distance: + Differ d2(A,B,result,false,max_time); + d2.solve(); + + // We could now solve this with space complexity O(ND), if that + // were acceptable: + if (sz*d2.get_edit_distance() d3(A,B,result,true,d2.get_edit_distance()); + d3.solve(); + d3.find_trace(); + return; + } + + // If even O(ND) is not acceptable, we give up and return a result + // indicating no common subset: + + make_trivial_solution(A,B,result); + } + + catch (Differ::max_D_exceeded) { + make_trivial_solution(A,B,result); + } + } + + + + void make_trivial_solution ( const ucs4_string& A, const ucs4_string& B, ucs4_string_fragment_seq& result ) + { + result.push_back(make_pair(from_a,A)); + result.push_back(make_pair(from_b,B)); + } + + + void ucs4_string_diff ( const ucs4_string& A, const ucs4_string& B, ucs4_string_fragment_seq& result ) + { + // Consider time efficiency. Aim not to take more than this much + // time (arbitary units). Return a sub-optimal solution if this + // time is exceeded. + const int max_time = 300; + + // Consider space efficiency. Aim not to use more than this much + // memory (arbitary units). Take more time or return a + // sub-optimal solution if this much memory is exceeded. + const int max_mem = 10000000; + + // Consider changing the above settings if "top" shows that the + // apache frontend request-handling processes are using more + // memory or CPU time than you would like. + + // Reducing them means that Anyterm will give up looking for an + // edit script and just send the complete new screen sooner. So + // making them too low will use more network bandwidth. On the + // other hand, for a fast local network, you might get a faster + // response with a lower max_time setting. + + + try { + + int sz = A.size() + B.size(); + // If input is small, i.e. N^2 is acceptable, we don't worry about + // space complexity. (This will take O(ND) space, but D could + // equal N.) + if ((sz*sz) d1(A,B,result,true,max_time); + d1.solve(); + d1.find_trace(); + return; + } + + // If input is larger, do a first pass to find the edit distance: + Differ d2(A,B,result,false,max_time); + d2.solve(); + + // We could now solve this with space complexity O(ND), if that + // were acceptable: + if (sz*d2.get_edit_distance() d3(A,B,result,true,d2.get_edit_distance()); + d3.solve(); + d3.find_trace(); + return; + } + + // If even O(ND) is not acceptable, we give up and return a result + // indicating no common subset: + + make_trivial_solution(A,B,result); + } + + catch (Differ::max_D_exceeded) { + make_trivial_solution(A,B,result); + } + } + +}; + + diff --git a/extras/anytermd/src/diff.hh b/extras/anytermd/src/diff.hh new file mode 100644 index 0000000000..7ef6c1f891 --- /dev/null +++ b/extras/anytermd/src/diff.hh @@ -0,0 +1,89 @@ +// common/diff.hh +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2005 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#ifndef diff_hh +#define diff_hh + +#include +#include + +#include "unicode.hh" + + +namespace DiffAlgo { + + // "Diff Algorithm" i.e. the algorithm used by the diff program. + + // Unlike the diff program, this algorithm can do comparisons on + // sequences of any types, not just lines. But the template + // specialisation is hidden inside the .cc file, so if you want to + // do anything other character-by-character (string) comparisons + // you'll need to make some changes there. Don't be put off, it's + // not hard. (If you want to do line-by-line comparisons, you'll + // want to implement a line type with an efficient equality + // comparison operator, e.g. a precomputed hash. The same applies + // for other complex types.) + + // It's also possible to find a measure of similarity of two + // sequences - the "distance" between them - using this code. + // Again, you'll need to delve into the .cc file for the details. + + // Implementation details are also in the .cc file. + + // Given two input string, say "hello world" and "goodbye world", + // the algorithm finds and returns a sequence of fragments, + // indicating for each whether it was from the first string, or from + // the second string, or was common to both strings: + + // From A: "hell" + // From B: "g" "odbye" + // Common: "o" " world" + + // Here are the types that define this return format: + + + enum fragment_tag { from_a, from_b, common }; + + template + struct fragment_seq { + typedef std::list > Type; + }; + + + // String diffs: + + typedef fragment_seq::Type string_fragment_seq; + + typedef fragment_seq::Type ucs4_string_fragment_seq; + + // Here is the prototype for the diff function. It returns its + // result via an "out" parameter: + + void string_diff ( const std::string& A, const std::string& B, + string_fragment_seq& result ); + + void ucs4_string_diff ( const ucs4_string& A, const ucs4_string& B, + ucs4_string_fragment_seq& result ); + + +}; + + + +#endif diff --git a/extras/anytermd/src/editscript.cc b/extras/anytermd/src/editscript.cc new file mode 100644 index 0000000000..e6bcade9e0 --- /dev/null +++ b/extras/anytermd/src/editscript.cc @@ -0,0 +1,136 @@ +// common/editscript.cc +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2005 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#include "editscript.hh" + +#include "diff.hh" + +#include +#include + +#include + + +using namespace std; + + +static ucs4_string ucs4_int(int i) +{ + ucs4_string s; + do { + s = ucs4_string(1,(i%10)+L'0') + s; + i = i/10; + } while (i); + return s; +} + + +void simplify_editscript(const DiffAlgo::ucs4_string_fragment_seq& in, + DiffAlgo::ucs4_string_fragment_seq& out) +{ + ucs4_string a_cf; // A data to carry forward + ucs4_string b_cf; // B data to carry forward + + for ( DiffAlgo::ucs4_string_fragment_seq::const_iterator i = in.begin(); + i != in.end(); ++i ) { + switch (i->first) { + case DiffAlgo::from_a: + a_cf.append(i->second); + break; + + case DiffAlgo::from_b: + b_cf.append(i->second); + break; + + case DiffAlgo::common: + if (i->second.size() > 4) { + if (!a_cf.empty()) { + out.push_back(make_pair(DiffAlgo::from_a,a_cf)); + a_cf.clear(); + } + if (!b_cf.empty()) { + out.push_back(make_pair(DiffAlgo::from_b,b_cf)); + b_cf.clear(); + } + out.push_back(*i); + } else { + a_cf.append(i->second); + b_cf.append(i->second); + } + break; + } + } + if (!a_cf.empty()) { + out.push_back(make_pair(DiffAlgo::from_a,a_cf)); + } + if (!b_cf.empty()) { + out.push_back(make_pair(DiffAlgo::from_b,b_cf)); + } +} + + +ucs4_string make_editscript(ucs4_string o, ucs4_string n) +{ + DiffAlgo::ucs4_string_fragment_seq e; + + DiffAlgo::ucs4_string_diff(o,n,e); + + DiffAlgo::ucs4_string_fragment_seq simp_e; + simplify_editscript(e,simp_e); + + ucs4_string editscript; + ucs4_string editscript_r = L"R"; + bool any_common = false; + bool any_change = false; + + for ( DiffAlgo::ucs4_string_fragment_seq::const_iterator i = simp_e.begin(); + i != simp_e.end(); ++i ) { + unsigned int len = i->second.length(); + switch (i->first) { + case DiffAlgo::from_a: + editscript += L'd'; + editscript += ucs4_int(len); + editscript += ':'; + any_change = true; + break; + case DiffAlgo::from_b: + editscript += L'i'; + editscript += ucs4_int(len); + editscript += ':'; + editscript += i->second; + editscript_r += i->second; + any_change = true; + break; + case DiffAlgo::common: + editscript += L'k'; + editscript += ucs4_int(len); + editscript += ':'; + any_common = true; + break; + } + } + + if (!any_change) { + return L"n"; + } else if (any_common) { + return editscript; + } else { + return editscript_r; + } +} diff --git a/extras/anytermd/src/editscript.hh b/extras/anytermd/src/editscript.hh new file mode 100644 index 0000000000..4c7b4c4a9d --- /dev/null +++ b/extras/anytermd/src/editscript.hh @@ -0,0 +1,38 @@ +// common/editscript.hh +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2005 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#ifndef editscript_hh +#define editscript_hh + +#include +#include "unicode.hh" + + +// Create an edit script describing the difference between oc and +// nc. Storage for the result is allocated by this function using +// malloc() and should be freed by the caller. Syntax of edit +// script is a sequence of commands describing how to transform oc +// to nc: +// k(num): keep num characters +// d(num): delete num characters +// i(num):text insert num characters, supplied + +ucs4_string make_editscript(ucs4_string o, ucs4_string n); + +#endif diff --git a/extras/anytermd/src/expand_command.cc b/extras/anytermd/src/expand_command.cc new file mode 100644 index 0000000000..404bda3f8e --- /dev/null +++ b/extras/anytermd/src/expand_command.cc @@ -0,0 +1,47 @@ +// common/expand_command.cc +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#include + +using namespace std; + + +// Expand command string: +// %h -> remote hostname +// %u -> HTTP AUTH username +// %p -> parameter supplied from the Javascript +// %% -> % + +string expand_command(string templ, string host, string user, string param) +{ + string::size_type p = templ.find('%'); + if (p==templ.npos || p==templ.length()-1) { + return templ; + } + string v; + switch(templ[p+1]) { + case '%': v="%"; break; + case 'h': v=host; break; + case 'u': v=user; break; + case 'p': v=param; break; + default: v="?"; break; + } + + return templ.substr(0,p) + v + expand_command(templ.substr(p+2),host,user,param); +} + diff --git a/extras/anytermd/src/expand_command.hh b/extras/anytermd/src/expand_command.hh new file mode 100644 index 0000000000..d8ae97bd4f --- /dev/null +++ b/extras/anytermd/src/expand_command.hh @@ -0,0 +1,27 @@ +// common/expand_command.hh +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#include + +// Expand command string: +// %h -> remote hostname +// %u -> HTTP AUTH username +// %p -> parameter supplied from the Javascript +// %% -> % + +std::string expand_command(std::string templ, std::string host, std::string user, std::string param); diff --git a/extras/anytermd/src/html.cc b/extras/anytermd/src/html.cc new file mode 100644 index 0000000000..8ac014f7f2 --- /dev/null +++ b/extras/anytermd/src/html.cc @@ -0,0 +1,120 @@ +// common/html.cc +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2005 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#include "html.hh" + +#include + +using namespace std; + + +// Screen to HTML conversion: + +static bool gen_style(ucs4_string& h, Attributes attrs) +{ + if (attrs!=Attributes()) { + unsigned int fg = attrs.fg; + unsigned int bg = attrs.bg; + if (attrs.inverse) { + swap(fg,bg); + } + ucs4_string classes; + if (attrs.bold) { + classes += L'z'; + } + if (bg!=Attributes().bg) { + if (!classes.empty()) { + classes += L' '; + } + classes += L'a'+bg; + } + if (fg!=Attributes().fg) { + if (!classes.empty()) { + classes += L' '; + } + classes += L'i'+fg; + } + h += L""; + return true; + } + return false; +} + +static const ucs4_char* attr_end = L""; + +static const ucs4_char* cursor_start = L""; +static const ucs4_char* cursor_end = L""; + + +ucs4_string htmlify_screen(const Screen& screen) +{ + // Convert screen into HTML. + // Slightly optimised to reduce spaces at right end of lines. + + ucs4_string h; + + for (int r=-screen.scrollback(); r0 && r>0 && !cursor) { + sp++; + } else { + while (sp>0) { + h+=L'\u00A0'; + sp--; + } + if (styled && attrs!=prev_attrs) { + h+=attr_end; + } + if (c==0 || attrs!=prev_attrs) { + styled = gen_style(h,attrs); + prev_attrs=attrs; + } + if (cursor) { + h+=cursor_start; + } + switch (ch) { + case '<': h+=L"<"; break; + case '>': h+=L">"; break; + case '&': h+=L"&"; break; + case ' ': h+=L'\u00A0'; break; + default: h+=ch; break; + } + if (cursor) { + h+=cursor_end; + } + } + } + if (styled) { + h+=attr_end; + } + h+=L"
"; + } + + return h; +} + + diff --git a/extras/anytermd/src/html.hh b/extras/anytermd/src/html.hh new file mode 100644 index 0000000000..930caf2078 --- /dev/null +++ b/extras/anytermd/src/html.hh @@ -0,0 +1,38 @@ +// common/html.hh +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2005 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +// This file declares a conversion function to transform a terminal +// screen represented as an array of character cells with associated +// attributes into a string of HTML, or WML. + + +#ifndef html_hh +#define html_hh + +#include + +#include "Screen.hh" + +#include "unicode.hh" + + +ucs4_string htmlify_screen(const Screen& screen); + + +#endif diff --git a/extras/anytermd/src/main.cc b/extras/anytermd/src/main.cc new file mode 100644 index 0000000000..56855b273c --- /dev/null +++ b/extras/anytermd/src/main.cc @@ -0,0 +1,175 @@ +// daemon/main.cc +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2005-2006 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + +#include "AnytermDaemon.hh" + +#include +#include + +#include +#include +#include + +#include "Exception.hh" +#include "segv_backtrace.hh" + +using namespace pbe; +using namespace std; + + +static void usage() +{ + cerr << "Usage: anytermd [options]\n" + << "Available options:\n" + << " --help Show this help message\n" + << " -c|--command Command to run in terminal (default /bin/bash)\n" + << " -d|--device Device to connect to (e.g. serial port)\n" + << " -p|--port Port number to listen on (default 8080)\n" + << " -u|--user User to run as\n" + << " -a|--auth none|null|trivial Authorisation scheme to use (default none)\n" + << " -s|--charset Character set to use (default ASCII)\n" + << " -f|--foreground Run in foreground (by default, backgrounds itself)\n" + << " --diff Send only differences to browser (default)\n" + << " -n|--nodiff Send whole screen to browser each time\n" + << " -m|--max-sessions Maximum number of simultaneous sessions (default 20)\n" + << " --max-http-connections Maximum number of simultaneous HTTP connections (default unlimited)\n" + << " --local-only Accept connections only from localhost\n" + << " --name Name used for logging and pid file (default anytermd)\n"; +} + + +struct Options { + bool background; + short port; + string user; + string command; + string device; + string authname; + string charset; + bool diff; + int max_sessions; + int max_http_connections; + bool local_only; + string name; + + Options(): + background(true), + port(8080), + user(""), + command("/bin/bash"), + device(""), + authname("none"), + charset("ascii"), + diff(true), + max_sessions(20), + max_http_connections(0), + local_only(false), + name("anytermd") + {} +}; + + +static Options parse_command_line(int argc, char* argv[]) +{ + Options options; + + for (int i=1; i(argv[++i]); + + } else if (arg=="--user" || arg=="-u") { + options.user = argv[++i]; + + } else if (arg=="--auth" || arg=="-a") { + options.authname = argv[++i]; + + } else if (arg=="--charset" || arg=="-s") { + options.charset = argv[++i]; + + } else if (arg=="--max-sessions" || arg=="-m") { + options.max_sessions = boost::lexical_cast(argv[++i]); + + } else if (arg=="--max-http-connections") { + options.max_http_connections = boost::lexical_cast(argv[++i]); + + } else if (arg=="--name") { + options.name = argv[++i]; + + } else { + cerr << "Unrecognised option '" << arg << "'\n"; + exit(1); + } + } + + return options; +} + + +int main(int argc, char* argv[]) +{ + get_backtrace_on_segv(); + + Options options = parse_command_line(argc,argv); + + if (getuid()==0 && options.user=="") { + cerr << "Please specify a user to run as using --user.\n"; + exit(1); + } + + try { try { + + AnytermDaemon d(options.port, options.user, options.command, options.device, options.name, + options.authname, options.charset, options.diff, options.max_sessions, + options.max_http_connections, options.local_only); + d.run_as_daemon(options.background); + + } RETHROW_MISC_EXCEPTIONS } + catch (Exception& E) { + E.report(cerr); + exit(E.exit_status); + } +} diff --git a/extras/anytermd/src/mk_blob.c b/extras/anytermd/src/mk_blob.c new file mode 100644 index 0000000000..91649e7b07 --- /dev/null +++ b/extras/anytermd/src/mk_blob.c @@ -0,0 +1,49 @@ +// daemon/mk_blob.c +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2008 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#include +#include + + +int main(int argc, char* argv[]) +{ + if (argc!=2) { + fprintf(stderr,"usage: mk_blob fn\n"); + exit(1); + } + const char* fn = argv[1]; + + printf("const char _binary_%s_start[] = {\n",fn); + int l=0; + while (1) { + int c = getchar(); + if (c==EOF) { + break; + } + printf("0x%02x, ",c); + ++l; + if (l%32==0) { + printf("\n"); + } + } + printf("\n};\n"); + printf("const char _binary_%s_end[] = {};\n",fn); + exit(0); +} + diff --git a/extras/anytermd/src/slugmake.sh b/extras/anytermd/src/slugmake.sh new file mode 100755 index 0000000000..418b4742ab --- /dev/null +++ b/extras/anytermd/src/slugmake.sh @@ -0,0 +1,5 @@ +#!/bin/sh + +make CXX=arm-linux-gnu-g++ CC=arm-linux-gnu-gcc AR=arm-linux-gnu-ar \ +ROTE_CONFIG=/usr/arm-linux-gnu/local/bin/rote-config + diff --git a/extras/anytermd/src/static_content.hh b/extras/anytermd/src/static_content.hh new file mode 100644 index 0000000000..9ea4fbbec0 --- /dev/null +++ b/extras/anytermd/src/static_content.hh @@ -0,0 +1,27 @@ +// daemon/static_content.hh +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2005 Philip Endecott + +// This is version $Name$ +// (if there is no version (e.g. V0-1) mentioned in the previous line, +// this is probably a snapshot from between "official" releases.) + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#include + +bool get_static_content(std::string fn, + std::string& mime_type, std::string& content); diff --git a/extras/anytermd/src/unicode.hh b/extras/anytermd/src/unicode.hh new file mode 100644 index 0000000000..ed471e5e5b --- /dev/null +++ b/extras/anytermd/src/unicode.hh @@ -0,0 +1,41 @@ +// daemon/unicode.hh +// This file is part of Anyterm; see http://anyterm.org/ +// (C) 2007 Philip Endecott + +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, write to the Free Software +// Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + + +#ifndef unicode_hh +#define unicode_hh + +#include + +#include "endian.hh" + +#if PBE_BYTE_ORDER == PBE_LITTLE_ENDIAN +#define UCS4_NATIVE "UCS-4LE" +#elif PBE_BYTE_ORDER == PBE_BIG_ENDIAN +#define UCS4_NATIVE "UCS-4BE" +#endif + +typedef wchar_t char32_t; +typedef char32_t ucs4_char; +typedef std::basic_string ucs4_string; + +typedef char utf8_char; +typedef std::basic_string utf8_string; + + +#endif