squidanalyzer/ChangeLog

Revision history for SquidAnalyzer

5.3 - Sun Dec  1 16:36:52 CET 2013

This release fixes several bugs and adds two new pie charts about the most use
top second level domains. It is also possible to do DNS lookup of Ip addresses
inside SquidAnalyzer, see UseClientDNSName new configuration directive. This can
slow down dramatically the squid-analyzer performances but you can adjust the
DNS lookup timeout to prevent waiting slow DNS server, See DNSLookupTimeout new
configuration directive.

	- Update and fix first and second top level domain name. Thanks to Luis
	  Armando Roca Fumero for the report.
	- Add new directive DNSLookupTimeout to change the default timeout for
	  DNS lookup. Add 0.0001 second timeout when SquidAnalyzer look for a DNS
	  name and can't find a name server. Thanks to alex for the patch.
	- Add pie chart of top second level domains. Thanks to Jeetendra Poojari
	  for the feature request.
	- Fix some HTML tag issues and table ordering on Top domain hits and Top
	  url hits. Thanks to Jeetendra Poojari for the report.
	- Update INSTALL file to remove GD::Graph requirements.
	- Change underscore used to replace space in user name by the special
	  string _SPC_ so that underscore will not be wrongly replaced on HTML
	  output.
	- Fix pt_BR translation with charset to utf-8 and a few words with
	  accentuation fix. Thanks to Joseh-Henrique for the fix.
	- Allow Ip addresses on user names to be replaced by their DNS name, this
	  feature is activated by a new directive: UseClientDNSName. Thanks to
	  Pfisztner Tamas for the patch.
	- Add missing description of --no-year-stat option to documentation and
	  squid-analyzer usage.

UPDATE: you must override all your installation, Perl scripts, configuration file,
CSS and Javascript files. Backward compatibility with data files is fully preserved.
Note that if you run a Perl version < 5.8 you may need the CPAN module Time::HiRes.

5.2 - Sun Sep  1 21:17:53 CET 2013

This is release fixes lot of minor bugs and adds support to CIDR notation in
network-alias and exclude files as well as some code rewriting and options to
improve speed. New feature to show the top N users that look at an url or a
domain. Adds a new configuration directive, TopUrlUser, to control the number
of users to show in reports.

	- Fix pod2man error with perl 5.18 about non-ascii character. Thanks to
	  David Walser for the report.
	- Skip some unwanted code execution when --no-year-stat is enabled.
	- Fix graphs on domain statistics reports.
	- Fix badly ordered check on squid log file to parse. Thanks to wordelle
	  for the report.
	- Fix issue where squid-analyzer use default configuration file even if
	  a custom one is specified with the -c option. Thanks to Thibaud Aubert
	  for the report.
	- Add --no-year-stat to disable year statistics, reports will start from
	  month level only. This allow to save time during reports generation.
	- Allow composed top level domain statistics in Top Domain report, like
	  co.uk. Thanks to Thibaut Aubert for the feature request.
	- Add support to CIDR notation in network-alias file. Thanks to Thibaud
	  Aubert for the feature request.
	- Fix issue in Domain report where domain of top url was not found in
	  the domain report even if the download was bigger than those reported.
	  Thanks to Aubert Thibaud for the report.
	- Fix bug in last/first hour of a day data storage that mixed collected
	  data over the two days and stored false first and last visit time.
	- Fix translation for user and count strings.
	- Add -b | --build_date command line option to limit rebuilt to a given
	  date, the format of its value can be yyyy-mm-dd, yyyy-mm or yyyy. This
	  should be used to prevent rebuilding html files from all data files.
	- New feature to show the top N users that look at an url or a domain.
	  Add a new configuration directive, TopUrlUser, to control the number
	  of users to show. Thanks to Mr-Ed for the feature resquest.
	- Fix first and last visit that was displayed in random order. Thanks
	  to MangOuste for the report.
	- Allow CIDR notation in exclude file to skip line matching the given
	  xxx.xxx.xxx.xxx/n network. Thanks to the MangOuste for the patch.
	- Add WriteDelay configuration directive (default to 3600 - stores one
	  hour of statistics from log file) to be able to save resources on huge
	  log files. On small log file it is not a good idea to set a lower
	  value as SquidAnalyzer will spend his time to write and read from
	  disk. This is a first attempt to speedup statistic reports, especially	  with year and month, but do not expect lot of performances gain.
	- Add regex checker for alias file and improve speed for network and
	  alias replacement. Thanks to jcputter for the report.
	- Fix unwanted message display in quiet mode.
	- Call localtime as little time as possible. Thanks to ammdispose for
	  the patch.
	- Remove the use of backstick to call system command in loop to gain
	  some speed. Thanks to ammdispose for the patch.
	- Revome the call to date and iconv system command each time a HTML file
	  is genrated, call it just one time if directive Locale is defined at
	  initialisation. Thanks to ammdispose for the report.
	- Fix some print statements were outputting to STDOUT instead of STDERR.
	  Thanks to ammdispose for the patch.
	- Allow peer SIBLING_HIT to be added to HIT counters. Thanks to Maher
	  Kassem forn the patch.
	- Recode initial column sorted by default to fix pending sort issues.
	- Fix bug on ordering column with sorttable. Thanks to David Walser for
	 the report.
	- Fix support to regex with coma in user and network alias file. Regex
	  like "1stFloor 192.168.1.1\d{0,2}" in user-aliases and network-aliases	  was not permitted. Thanks to Steffen Junick for the report.
	- Fix issue with log file provided at command line (option -l) and non
	  existant LogFile in squidanalyzer.conf. Thanks to Kalin Kozhuharov for
	  the report.
	- Fix test to exclude old months and days from the statistics builder.
	  Thanks to ammdispose for the patch.
	- Fix some code comment about month obsolescence. Thanks to ammdispose
	  for the patch.
	- Change code to remove obsolete statistics to more stable code with
	  less iterations. Thanks to ammdispose for the patch.
	- Fix install of configuration files. Thanks to David Wasler for the
	  patch.

5.1 - Wed Jan 30 00:11:57 CET 2013

This release adds more graphs, fixes several major issues and allow automatic
removing of obsolete statistics. The Javascript library sorttable.js have been
updated.

	- Add squid-analyzer usage information into documentation.
	- Update INSTALL file about FreeBSD issue.
	- Add -v | --version and -d | --debug command line options.
	- Add support to automatic deletion of obsolete statistics and data.
	- Add -p | --preserve option to allow automatic removing of obsolete
	  statistic. Thanks to ammdispose for the feature request.
	- Fix documentation about OrderUrl that is limited to User detailed Urls
	  reports, not Top Url and Top domain.
	- Set default sorttable sort in descending order first.
	- Fix default order in Mime, Network, User and User details views.
	  Thanks to Wesley Bresson for the report.
	- Update sorttable.js to allow sort on page load using the new function
	  sorttable.innerSortFunction().
	- Add documentation about FreeBSD install to avoid issue with error:
	    FreeBSD: Cannot determine short module description
	  Thanks to hackson99 for the report.
	- Fix first and last visit time in the user detail view that was not
	  displayed at all.
	- Fix parser issue when squid logformat has additional field at end.
	  Thanks to nuxsmin for the patch.
	- Add pie chart on Mime type statistics.
	- Add MinPie configuration directive to set the percentage of pie graph
	  data which under item will be stored in others label.
	- Force left alignment on largest Url column.
	- Add Locale configuration directive to set the locale when retrieving
	  date from system. Thanks to Klaus Tachtler for the suggestion.
	- Fix missing Up link translation and some German translation. Thanks
	  to Klaus Tachtler for the patch.
	- Fix HTML code in user and url reports. Thanks to Nathanael Bonin for
	  the patch.
	- Fix typo in fr_FR.txt lang file. Thanks to Nathanael Bonin for the
	  patch.
	- Replace .orig extension on configuration files by .sample. Thanks to
	  David Walser for the report.
	- Update squidanalyzer.spec for RPM build. Thanks to David Walser for
	  the patch.
	- Modify sorttable.js to be able to sort column with both value and
	  percentage as numeric.
	- Remove percentage column and store the percent information directly
	  in the column between parenthesis. Thanks for Nathanael Bonin for
	  the patch.
	- Add missing class CSS style to user graphs table. Thanks to Nathanael
	  Bonin for the report.

UPGRADE: You must reinstall all. Backward compatibility is preserved on data
files, but the HTML reports have changed, so after re-installing SquidAnalyzer
you may want to run manually squid-analyzer with the --rebuild command line
option to regenerate those files.  So run the following commands:
        
        tar xzf SquidAnalyzer-5.1.tar.gz
        cd SquidAnalyzer-5.1/
        perl Makefile.PL HTMLDIR=/var/www/squidanalyzer
        make && make install
        /usr/local/bin/squid-analyzer --rebuild
        
should be the normal procedure to upgrade. Change the path to the HTML output
dir if required.


5.0 - Mon Dec 10 22:24:23 CET 2012

Thanks to the great work of Nathanael Bonin, SquidAnalyzer has now a new design.
I also replaced all GD graphs by the use of a Javascript Flotr2 graphic library,
that mean that there is no more needs of Perl modules dependancies to install
SquidAnalyzer.

	- Update documentation.
	- Add TransfertUnit configuration directive to allow changing default
	  Bytes transfert unit. Possible values are BYTES, KB, MB and GB.
	  Thanks to Nathanael Bonin for the feature request.
	- Fix date encoding with call to date+iconv or fallback to strftime().
	- Fix sorttable with missing tbody tag. Thanks to Nathanael Bonin for
	  the patch.
	- Remove some hardcoded CSS from the html code. Thanks to Nathanael
	  Bonin for the patches.
	- Fix some translation issues.
	- Improve CSS and dynamic html general design. Thanks to Nathanael Bonin
	  for the patches.
	- Fix some untranslated string. Thanks to Nathanael Bonin for the patch.
	- Fix accent within report, now SquidAnalyzer will use the CharSet
	  translation directive to set the charset following the language.
	  Default to utf-8, french, german and portuges will use iso-8859-15.
	- Fix github issue #5 where per user/ip network stats was wrong and very
	  high. Thanks to Ben Tullis for the report.
	- Removed obsolete files.
	- Fix several design issues, moved png images in a dedicated directory.
	  Thanks to Nathanael Bonin for the patches.
	- Reorder source tree and apply tree change on Perl install.
	- Removed FooterFile, no more used.
	- Remove code for graphs generated using GD::Graph libraries, replaced
	  by the Javascript library Flotr2.
	- Fix extra .0 in networks report. Thanks to Ben Tullis for the report.
	- Add the use %M to replace month by its 3 letters abbreviation into
	  DateFormat configuration directive. Thanks to pk-its for the feature
	  request.
	- Add missing german translation. Thanks to pk-its for the patch.
	- Remove space in user alias to prevent corruption of data files. Thanks
	  to Robert Becskei for the report.
	- Fix stat_user.dat format error when username have a space in its login
	  name. Thanks to Moise Michel for the report.
	- Force date/time used in report generation date to used 24h notation.
	  Thanks to Klaus Tachtler for the report
	- Fix install of squidanalyzer.conf, network-aliases, user-aliases  and
	  excluded to be copied with the .orig extention so that nones of these
	  file can be overriden at upgrade. Thanks to Klaus Tachtler for the
	  report.

UPGRADE: You must reinstall all but backward compatibility is fully preserved.
Even if backward compatibility is preserved on data files, the HTML and graph
have changed, so after re-installing SquidAnalyzer you may want to run manually
squid-analyzer with the --rebuild command line option to regenerate those files.
So run the following commands:
        
        tar xzf SquidAnalyzer-5.0.tar.gz
        cd SquidAnalyzer-5.0/
        perl Makefile.PL HTMLDIR=/var/www/squidanalyzer
        make && make install
        /usr/local/bin/squid-analyzer --rebuild
        
should be the normal procedure to upgrade. Change the path to the HTML output
dir if required.

4.4 - Tue Aug 21 21:01:43 CEST 2012

	- Add support to bzip2 compressed log file. Feature request by Edmonds
	  Namasenda.
	- Allow exclusion of URL from the report, the format of the exclude file
	  has changed, see header of the excluded file.
	- Fix some RPM spec file issue. Patch by David Walser.
	- Add german translation file. Translation by Klaus Tachtler.
	- Fix translation file path into squidanalyzer.conf.

4.3 - Fri Apr 27 21:47:12 CEST 2012

	- Fix broken statistics when an access log file run over multiple
	  years. Thanks to Jugurtha Massinissa for the report.
	- Prevent buildHtml to create directories on month/day without log.
	- Sort year's view per decreasing years.
	- Peer CD_SIBLING_HIT status will now be taken as local cache hit with
	  a new configuration directive (SiblingHit) to disable this feature.
	  Default is enable. Documentation and configuration file has been
	  updated - Thanks to Nathanael BONIN for the feature request.
	- Add spec file for RPM packaging - Thanks to David Walser for the
	  patch.
	- Fix path and comment to translation file in squidanalyzer.conf. Thanks
	  to Jugurtha Massinissa for the report.

4.2 - Sun Jan 22 16:52:27 CET 2012

	- Fix network alias issue broken in last release, now data files will
	  store the network alias instead of the default class C network. Thank
	  to Fabrice Prigent for the report.
	- Add support to multiple regex pattern to match networks aliases. See
	  network-aliases file for more information.
	- User aliases are now directly changed into data files instead of html
	  file like before.
	- Add support to multiple regex pattern to match users aliases. See
	  users-aliases file for more inforamtion.
	- Add AnonymizeLogin configuration directives to allow full anomymization
	  of all users/ip.  Thanks to Fabrice Prigent for the request.
	- Add support to squidmime logformat, original request and reply header
	  are simply omitted. Squid and squidmime native format are still the
	  only log format supported.

UPGRADE: If you use network and/or user aliases, even if I try to preserved backward
compatibility, you may want to start with new data files as these informations are
now replaced directly into the data file instead of the HTML files. Changes only
concern file SquidAnalyzer.pm so you can just override it. There's also a new
configuration directive 'AnonymizeLogin' so you may copy/paste his definition in
default squidanalyzer.conf file.

4.1 - Tue Jan 17 19:58:35 CET 2012

	- Fix data file parser issue with largest_file_size or largest_file_url
	  are empty. Thanks to David Walser for the report.
	- Add line number when reporting bad format into a data file.
	- Add missing DESTDIR variable into the allowed commande line arguments.
	  Thanks to David Walser for the report.
	- Fix error when header file are missing as they are no more used now.
	  Thanks to David Walser for the report.
	- Remove salogo.png from install script, it is now replaced by file
	  logo-squidanalyzer.png. Thanks to David Walser for the report.
	- Install of README, INSTALL and Changelog is now disabled by default,
	  you must set DOCDIR parameter to install those files in the desired
	  directory. Thanks to David Walser for the request.

4.0 - Wed Dec 28 02:06:22 CET 2011

	- Fix regex search failure on fake domain. Thanks to Michel Reischl for
	  the report.
	- Add support to squid log with client dnsname instead of ip address.
	  Thanks to Matthew Jurgens for the patch.
	- Add options --logfile (-l) and --configfile (-c) to squid-analyzer
	  command line argument.  Thanks to Matthew Jurgens for the patch.
	- Add the first and last time an url is viewed by a user on day and hour
	  views. Thanks to Franco Mazz for the feature request.
	- Allow column sorting using the Javascript sorttable library
	  available at http://www.kryogenix.org/code/browser/sorttable/.
	  Thanks to Matthew Jurgens for the patch.
	- Renamed file Changes as ChangeLog.
	- Makefile.PL is now able to do a full install by creating directories
	  and moving files.
	- Allow parsing of compressed log file. SquidAnalyzer will automaticaly
	  use zcat when the logfile extention ends with .gz
	- The cost column now is not shown when the cost price is 0.
	- Change CSS style of links.
	- Complete rewrite of report menu and calendar sot that they appears on
	  all HTML page.
	- Change License from Artistic to GPL v3.
	- Remove HeaderFile, replaced by squidanalyzer.css file.
	- Add --rebuild option to allow rebuilding of all html and graph output
	  from data files. This should be done after upgrading.
	- Add ..._REFRESH_UNMODIFIED into cache HIT statistics.

UPGRADE: Even if backward compatibility is preserved on data files, the HTML and
graph have changed, so after re-installing SquidAnalyzer you may want to run
squid-analyzer with the --rebuild command line option to regenerate those files.
So run the following commands:

	tar xzf SquidAnalyzer-4.0.tar.gz
	cd SquidAnalyzer-4.0/
	perl Makefile.PL HTMLDIR=/var/www/squidanalyzer
	make && make install
	/usr/local/bin/squid-analyzer --rebuild

should the normal procedure to upgrade. Change the path to the HTML output if
required.

3.1 - Mon Aug  6 12:48:14 CEST 2007

	- Fix illegal division by zero errors.
	- Remove extra spaces in username

3.0 Fri May  7 12:31:01 CEST 2004

	- Many thanks to Alexandre Sieira for all good ideas and reports !
	- Add localization support.
	- Add date formatting option for localisation.
	- Add French translation.
	- Add Portugal/brazilan translation.
	- Add custom HTML header an footer support.

2.1 Tue May  4 16:35:08 CEST 2004

	- Fix Url escaping problem on users pages
	- Add translation (En + Fr)

2.0 Wed Oct 15 11:40:03 CEST 2003

	- This is a complete rewrite of SquidAnalyzer.
	- Add configuration file.
	- Add Top n of url and second level domain.
	- Add alias for client ip address and/or login. 
	- Add exclusion from report of certain client ip
	  addresses, network addresses and/or auth login. 
	- Many bug fix.

1.1 Wed Feb 28 12:21:30 CET 2001

	- Fix problem with access.log file has zero size.
	- Skip log line with field lenght != 8
 
1.0 Wed Jan 24 13:47:14 CET 2001

	- original version