Commit Graph

502 Commits

Author SHA1 Message Date
DL6ER 1e8bfd33f5
Improve output
Signed-off-by: DL6ER <dl6er@dl6er.de>
2020-02-23 22:50:06 +01:00
DL6ER 8131b5961c
Add comments to the code describing the changes.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2020-02-22 15:22:29 +01:00
DL6ER 81d4531e10
Implement performant list checking routine.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2020-02-22 13:01:55 +01:00
DL6ER 050e2963c7
Remove redundant code.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2020-02-21 22:28:53 +01:00
DL6ER 3c09cd4a3a
Experimental output of matching line from shown warnings.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2020-02-21 21:47:56 +01:00
DL6ER 839fe32042
Fix issue with missing newline at the end of adlists (#3144)
* Also display non-fatal warnings during the database importing. Previously, we have only show warnings when there were also errors (errors are always fatal).

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Ensure there is always a newline on the last line.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Stickler linting

Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>

* Move sed command into subroutine to avoid code duplication.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Also unify comments.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Also unify comments.

Signed-off-by: DL6ER <dl6er@dl6er.de>

Co-authored-by: Dan Schaper <dan@glacialmagma.com>
2020-02-21 18:56:48 +00:00
DL6ER d14ee26d6a
Merge pull request #3139 from pi-hole/fix/count_before_calling_FTL
Fix wrong number of blocking domains shown on the dashboard
2020-02-17 21:32:45 +01:00
DL6ER 52398052e9
Compute number of domains (and store it in the database) BEFORE calling FTL to re-read said value.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2020-02-17 21:07:48 +01:00
DL6ER 601f9048cd
Merge pull request #3130 from pi-hole/fix/gravity_updated_timestamp
Store gravity update timestamp only after database swapping
2020-02-17 06:07:05 +01:00
DL6ER dc2fce8e1d
Store gravity update timestamp only after database swapping.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2020-02-12 19:26:25 +01:00
DL6ER 2a5cf221fa
Store number of distinct gravity domains in database after counting.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2020-02-02 23:46:33 +01:00
DL6ER 10c2dad48a Improve gravity performance (#3100)
* Gravity performance improvements.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Do not move downloaded lists into migration_backup directory.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Do not (strictly) sort domains. Random-leaf access is faster than always-last-leaf access (on average).

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Append instead of overwrite gravity_new collection list.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Rename table gravity_new to gravity_temp to clarify that this is only an intermediate table.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Add timers for each of the calls to compute intense parts. They are to be removed before this finally hits the release/v5.0 branch.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Fix legacy list files import. It currently doesn't work when the gravity database has already been updated to using the single domainlist table.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Simplify database_table_from_file(), remove all to this function for gravity lost downloads.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Update gravity.db.sql to version 10 to have newle created databases already reflect the most recent state.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Create second gravity database and swap them on success. This has a number of advantages such as instantaneous gravity updates (as seen from FTL) and always available gravity blocking. Furthermore, this saves disk space as the old database is removed on completion.

* Add timing output for the database swapping SQLite3 call.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Explicitly generate index as a separate process.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Remove time measurements.

Signed-off-by: DL6ER <dl6er@dl6er.de>
2020-01-24 09:39:13 -08:00
DL6ER 8f22203d24
Wait 30 seconds for obtaining a database lock instead of immediately failing if the database is busy.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2020-01-14 20:02:00 +01:00
DL6ER 948f4a8827
Ensure permissions and ownership of gravity.db are correctly set on each run of pihole -g. This would have prevented https://github.com/pi-hole/AdminLTE/issues/1077
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-12-16 09:55:46 +00:00
DL6ER 2cec9eaf65
Merge pull request #3033 from pi-hole/fix/duplicates_in_adlists
Remove duplicates from adlists before importing
2019-12-12 21:37:26 +01:00
DL6ER 779fe670f7
Show full URL during gravity download instead of only domain and file
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-12-12 10:29:44 +00:00
DL6ER 570a7a5c11
Use sort -u instead of uniq as it is guaranteed to be safe when doing inline file operations.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-12-12 10:17:59 +00:00
DL6ER bd1b004d94
Remove possible duplicates found in lower-quality adlists
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-12-12 10:13:51 +00:00
Adam Warner d29947ba32
optimise gravity list inserts
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
2019-12-09 22:30:41 +00:00
Adam Warner 1f03faddef
shell check recomends
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
2019-12-09 21:35:54 +00:00
DL6ER 3231e5c3ba
Address stickler requests.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-12-09 16:52:03 +00:00
DL6ER 0c5185f8ba
Also display how many unique domains we have caught in the event horizon.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-12-04 21:02:46 +00:00
DL6ER 037d52104a
New command "pihole -g -r" recreates gravity.db based on files backed up in /etc/pihole/migration_update. This is useful to restore a working version of the database when the user destroyed the original database. Also, update gravity.db to version 5 because of a fix we needed to implement.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-11-26 10:58:39 +01:00
DL6ER 756c99653e
Merge branch 'development' into new/internal-blocking 2019-10-03 12:01:27 +02:00
Adam Warner 61a40c1b43
merge devel into 4.3.2 And Resolve merge conflicts
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
2019-09-22 01:16:44 +01:00
DL6ER a8af2e1837
Store domains without sorting and unifying them first. This allows us to preserve the relationship of the individual domains to the lists they came from.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-09-04 23:14:29 +02:00
Adam Warner 8bef5dc805
remove n from -ne
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
2019-09-03 23:56:23 +01:00
Adam Warner ad41bcca5a
Remove support for adblock style lists to prevent false positives
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
2019-09-03 23:43:11 +01:00
DL6ER ca8982494b
Store timestamp when the gravity table was last updated successfully. This fixes https://github.com/pi-hole/AdminLTE/issues/989
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-09-01 14:42:07 +02:00
DL6ER 1820c2c598
Merge branch 'development' into new/whitelist-regex-support
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-08-22 14:19:51 +02:00
Andreas c53be459c6
quick fix for when dig also returns a CNAME
Signed-off-by: Andreas <ryrun@online.de>
2019-08-21 04:37:06 -07:00
DL6ER 4371c9ba03
Ensure proper permissions are set for gravity.db after creation.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-08-05 21:20:07 +02:00
Andreas 63230cb72d
quick fix for when dig also returns a CNAME
Signed-off-by: ryrun <ryrun@online.de>
2019-08-04 21:21:08 +02:00
DL6ER 5ff9052200
Review comments
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-07-09 11:41:44 +02:00
DL6ER f5121c64be
We should still add the regex lines (initially) to the regex table as the renaming will happen only after the importing.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-07-08 21:39:30 +02:00
DL6ER 054c7a2c05
Create new table + view regex_whitelist + rename old regex table to regex_blacklist. This updates the gravity.db version to 3.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-07-08 21:35:31 +02:00
DL6ER be3e198f9a
Address linting errors.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-07-07 10:46:20 +02:00
DL6ER acc50b709e
Only migrate files once (domain and adlist lists druing initial creation of gravity.db auditlog.list on database upgrade from version 1 to 2.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-07-07 10:33:08 +02:00
DL6ER 0405aaa3da
Review comments and fixing stickler complaints.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-07-06 09:32:41 +02:00
DL6ER 2fb4256f84
Rename table to "auditlist"
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-07-05 16:28:36 +02:00
DL6ER 4f4a12bb40
Upgrade database if necessary and store audit domains therein.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-07-05 14:03:57 +02:00
DL6ER 7a1b6fd205
Merge branch 'development' into new/gravity.db_domain_groups 2019-07-03 19:21:55 +02:00
DL6ER f91421418a
Rename adlists -> adlist table.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-06-30 23:21:10 +02:00
DL6ER 87c115fc86
Change implementation to use linking tables (domain_groups, adlists_groups). This allows adding domains to multiple lists conveniently.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-06-30 19:04:38 +02:00
DL6ER d5e8f1a781
Add 10 seconds timeout to the sqlite3 command that writes tables to the gravity database. This prevents gravity from failing when the database file is locked for a short time.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-06-26 18:20:36 +02:00
DL6ER cbc210b014
Add adlist_groups and mark group_id columns as NOT NULL.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-06-16 17:05:33 +02:00
Chris Crocker-White 69dba022c4 Handle an empty local.list
Handle the case of an empty local.list file which would otherwise prevent the system from starting

Change-type: patch
Signed-off-by: Chris Crocker-White <chriscw@balena.io>
2019-05-27 10:27:28 -07:00
Dan Schaper 7de6bbca61
Merge branch 'development' into release/v4.3 2019-05-18 13:18:14 -07:00
Mcat12 8e1bf6e3d4
Merge branch 'development' into new/gravitydb
# Conflicts:
#	advanced/Scripts/list.sh
#	gravity.sh
#	pihole
2019-05-12 15:20:52 -07:00
pvogt09 3f90261520 Moves chmod after every creation of ${destination} instead of end of function
Signed-off-by: pvogt09 <50047961+pvogt09@users.noreply.github.com>
2019-05-12 11:25:48 +02:00
pvogt09 225285cb97 Removes chmod for ${source} and superfluous chmod for ${destination}
Signed-off-by: pvogt09 <50047961+pvogt09@users.noreply.github.com>
2019-05-11 11:14:07 +02:00
pvogt09 a3d2a1062f Makes blocklists world readable
Signed-off-by: pvogt09 <50047961+pvogt09@users.noreply.github.com>
2019-05-10 11:13:19 +02:00
Mcat12 ec06fd0ad4
Fix incorrect adlist query when an adlist is blocked during gravity
hosts-file.net was hard-coded as the domain to check instead of the
actual domain.

Signed-off-by: Mcat12 <newtoncat12@yahoo.com>
2019-05-04 11:35:21 -07:00
DL6ER 0cfecd31d3
Invert logic to reduce nesting
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-01 22:07:14 +02:00
DL6ER 008e88b84b
Only migrate legacy list files when we create gravity.db in the same gravity run
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-01 18:04:31 +02:00
DL6ER 540c57f755
Add comment for automigrated list entries
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-01 17:51:25 +02:00
DL6ER 983117d788
Add id (primary key, autoincrement) field to lists tables
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-01 17:47:15 +02:00
DL6ER b372f808dd
Fix lint errors
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-05-01 17:06:14 +02:00
pvogt09 8a92fb24c4 Incorporates changes requested by @Mcat12
Signed-off-by: pvogt09 <50047961+pvogt09@users.noreply.github.com>
2019-05-01 11:20:26 +02:00
pvogt09 5c575e73c7 Adds world readable attribute to files created by Pi-hole to circumvent #2724
Signed-off-by: pvogt09 <50047961+pvogt09@users.noreply.github.com>
2019-04-30 22:41:12 +02:00
DL6ER a932209143
Set default for date_modified to be the current time
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-29 17:21:59 +02:00
DL6ER 4946b5907b
Move migrated files to backup directory instead of deleting them. This has the benefit that gravity.db can be recreated at any time by: 1. deleting gravity.db, 2. copying all files from the migration_backup directory back to /etc/pihole, 3. running pihole -g again.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-28 22:15:47 +02:00
Mark Drobnak 4596d9a577
Merge branch 'development' into new/gravitydb 2019-04-26 23:40:28 -04:00
DL6ER 8524aecfed
Don't set gravity database permissions
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-26 17:56:11 +02:00
DL6ER 3dcb9722d3
Remove global haveSourceUrls variable and just skip downloading and processing adlists if there aren't any to be downloaded
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-26 17:39:35 +02:00
DL6ER bd0215524b
Add vw_adlists view which returns only enabled adlists
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-26 17:36:00 +02:00
DL6ER 53e3ff2b24
Only add non-empty lines when moving file content into database tables
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-26 17:32:05 +02:00
DL6ER 0efc46260f
Use both old and new strings for detecting whether this is a Pi-hole dnsmasq config file
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-26 16:16:30 +02:00
DL6ER d94bbfca7f
Finish migrating gravity.db to using our new gravity database. This removes the --skip-download, --blacklist-only, --whitelist-only, and --wildcard-only options as it is not necessary to run gravity when modifying them from now on (gravity really only builds the gravity database and the local.list file).
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-25 11:23:06 +02:00
DL6ER 465a39427b
Source adlists from gravity database
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-25 10:46:18 +02:00
DL6ER 2664ac0efc
Improve error reporting
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-25 10:18:37 +02:00
DL6ER c1277705d8
Copy existing whitelist.txt, blacklist.txt, regex.list, and adlists.list to the database. We remove the files afterwards as the content lives in the database now
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-25 10:14:45 +02:00
DL6ER e57d0fb93c
Remove the empty templates gravity.list, white.list, black.list. The old source files whitelist.txt, blacklist.txt, and regex.list still exist and need to be removed in a follow-up commit
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-24 19:05:08 +02:00
DL6ER ee20164207
Remove detect_pihole_blocking_status() as we can modify the database at any time
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-04-24 19:01:31 +02:00
Dan Schaper 54ae9d83b0 Stickler-lints
Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>
2019-02-23 13:42:15 -08:00
DL6ER bbc392caeb
More review comments
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-02-22 22:49:02 +01:00
DL6ER 61840edb30
Fill comment field with NULL (default) instead of empty string and use a variable for the timestamp instead of periodically calling date
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-02-22 22:46:19 +01:00
DL6ER f9250d91a5
Explictly escape the escape.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-02-06 19:17:13 +01:00
DL6ER 491c828661
Improved code, ensure that gravity.sh passes shellcheck as a whole
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-02-06 19:09:09 +01:00
DL6ER 644ec36e64
Review comments
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-02-06 18:57:48 +01:00
DL6ER ec5b16ef85
Fix incomplete comment
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-02-05 19:08:08 +01:00
DL6ER dbbf21071b
Invert logic to use ENABLED instead of DISABLED. We make sure to set "enabled = 1 WHERE enabled IS NULL" in gravity_store_in_database()
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-02-05 19:05:11 +01:00
DL6ER b2f4385232
Avoid infinity loop if gravity optimization fails
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-02-03 17:05:00 +01:00
DL6ER 1f9a7d0b0c
Only run VACUUM on "pihole -g -o"
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-02-03 17:01:38 +01:00
DL6ER d5feffa117
Do not backup the database
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-02-03 16:45:19 +01:00
DL6ER fc62cf7e2f
Only run VACUUM on "full" gravity runs
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-02-03 16:45:06 +01:00
DL6ER 93f1859bab
Store regex in gravity.db as well
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-02-03 15:13:18 +01:00
DL6ER dcf0a605cf
Use views for all tables and set disabled column to false for those tables that support it
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-02-03 14:14:14 +01:00
DL6ER 8a91fe6992
Store domains in gravit tables: gravity, whitelist, blacklist
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-02-03 13:21:26 +01:00
DL6ER 09c4c88a6d
Create and fill gravity.db during pihole -g
Signed-off-by: DL6ER <dl6er@dl6er.de>
2019-02-03 13:04:31 +01:00
xch12i5 cd8a423b32 Solves RegEx issue and updates documentation.
Signed-off-by: xch12i5 <40517505+xCh12i5@users.noreply.github.com>
2019-01-20 16:09:25 +01:00
xch12i5 e02da90faa Replaces double quotes by single quotes and updates documentation.
Signed-off-by: xch12i5 <40517505+xCh12i5@users.noreply.github.com>
2019-01-19 17:52:28 +01:00
xch12i5 0de5132e2b Solves RegEx issue.
Signed-off-by: xch12i5 <40517505+xCh12i5@users.noreply.github.com>
2019-01-19 17:07:27 +01:00
DL6ER 05bd88b4d8
Update agent string: the selected string is the most abundant one on 06 Dec 2018 with a percentage of almost 10%
Signed-off-by: DL6ER <dl6er@dl6er.de>
2018-12-06 18:04:17 +01:00
Adam Warner 83f7bd66e6
Don't `mv` `preEventHorizon`, copy it.
Signed-off-by: Adam Warner <adamw@rner.email>
2018-12-05 22:29:01 +00:00
Dan Schaper 5c621fa253
Lints
Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>
2018-11-12 17:21:34 -08:00
Dan Schaper f9e71bee0a
Account for port, or lack of, in PIHOLE_DNS_1
Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>
2018-11-12 17:15:24 -08:00
Mark Drobnak 33939a30ea
Merge pull request #2347 from Gusher123/patch-1
Don't let one blocklist block an other blocklist
2018-09-30 21:24:45 -04:00
Adam Warner 0131055ec9
Submission stale, addressing @mcat12's comments
Signed-off-by: Adam Warner <adamw@rner.email>
2018-09-30 20:00:06 +01:00
DL6ER 337cc5ca18
BLOCKING -> BLOCKING_ENABLED
Signed-off-by: DL6ER <dl6er@dl6er.de>
2018-08-13 17:22:10 +02:00
DL6ER b011adc453
Use new setupVars BLOCKING variable to determine which file to update via gravity
Signed-off-by: DL6ER <dl6er@dl6er.de>
2018-08-13 14:19:59 +02:00
DL6ER 6cde066edd
Have gravity detect the presence of files like gravity.list.bck and update this file if present (assuming blocking is disabled)
Signed-off-by: DL6ER <dl6er@dl6er.de>
2018-08-13 13:43:14 +02:00
Gusher123 2feb5f22cc
Signed-off-by
This is my commit message

Signed-off-by: Gusher123 <t.siebeling@gmail.com>
2018-08-12 10:05:58 +02:00
Gusher123 03a9d3bc5b Not hardcoding dns and covering all blockingmodes
Using double brackets

Mode double brackets
2018-08-12 03:05:53 +02:00
Gusher123 e58142b4ac
Added comments 2018-08-11 15:14:45 +02:00
Gusher123 3e5c6afaee
Don't let one blocklist, block an other
If a blocklist is blocked by an other and 'pihole -g' is run, update the blocked blocklist by useing an alternate dns server (in this case 1.1.1.1) to acquire the IP adres and download the blocklist using the 'curl --resolve' (https://curl.haxx.se/docs/manpage.html#--resolve)
2018-08-11 14:33:33 +02:00
DL6ER 760dd01b50
Use grep invert-match
Signed-off-by: DL6ER <dl6er@dl6er.de>
2018-08-07 18:36:06 +02:00
DL6ER 5bf90cb77d
Use alternative logic for counting regex file entries, skipping commented out lines
Signed-off-by: DL6ER <dl6er@dl6er.de>
2018-08-07 18:02:01 +02:00
Mcat12 1a8d5fb4a9
Allow PHP to edit the regex file if installed
Fixes permissions error on the web interface when modifying the regex.

Signed-off-by: Mcat12 <newtoncat12@yahoo.com>
2018-07-28 21:57:39 -04:00
Dan Schaper dbc82cfb6a
Merge pull request #2236 from pi-hole/new/regex-lists
Use regex lists instead of wildcards for blocking
2018-07-14 06:10:44 -07:00
Mcat12 72be7ad5f0
Don't count commented out regex lines
Signed-off-by: Mcat12 <newtoncat12@yahoo.com>
2018-07-12 21:21:06 -04:00
WaLLy3K 7ddae8f2eb
Treat URLs without a protocol as HTTP
Signed off by WaLLy3K <wally3k@pi-hole.net>
2018-07-02 19:59:22 +10:00
Mcat12 0f4968d75c
Merge remote-tracking branch 'origin/development' into new/regex-lists 2018-06-29 23:12:34 -04:00
Mcat12 55175087c4
Set regex file permissions for web access
Signed-off-by: Mcat12 <newtoncat12@yahoo.com>
2018-06-28 22:52:44 -04:00
Dan Schaper d5547f5c7c
Revert "Fix error: /opt/pihole/gravity.sh: 385: Warning: command substitution: ignored null byte in input" 2018-06-24 16:06:55 -07:00
Mark Drobnak 4fc3546887
Merge pull request #2069 from pi-hole/fix/nullbyte
Fix error: /opt/pihole/gravity.sh: 385: Warning: command substitution: ignored null byte in input
2018-06-20 20:42:24 -04:00
DL6ER bc705aac03
Add automated wildcard list -> regex filter conversion
Signed-off-by: DL6ER <dl6er@dl6er.de>
2018-06-17 14:39:19 +02:00
Mark Drobnak 3149a95d6a
Fix gravity_ParseFileIntoDomains Awk comment
Signed-off-by: Mark Drobnak <newtoncat12@yahoo.com>
2018-06-03 13:33:33 -04:00
Adam Warner 5f2217a079
Fix for gravity.list not being emptied with an empty/missing adlist file.
Variable output depending on content of /existence of adlist

Signed-off-by: Adam Warner <adamw@rner.email>
2018-04-30 21:56:33 +01:00
Adam Warner 3216d65353
Merge branch 'development' into new/RemoveDefaultBlocklists 2018-04-29 13:39:17 +01:00
Adam Warner 9c29bd627e
fix user blacklisting 2018-04-21 23:55:33 +01:00
DL6ER d24b141d8b
Get rid of numBlocked file, FTLDNS can now determine number of domains thanks to the new lists format
Signed-off-by: DL6ER <dl6er@dl6er.de>
2018-04-21 23:58:42 +02:00
DL6ER bac9e005ae
Generate gravity.list and black.list in simple domain lists format for FTLDNS. Leave local.list in HOSTS format
Signed-off-by: DL6ER <dl6er@dl6er.de>
2018-04-21 23:27:36 +02:00
Dan Schaper d4d1ec2568
Merge pull request #2124 from pi-hole/development
Internal
2018-04-17 11:54:50 -07:00
Adam Warner 88d4108f92
- Add new whipstail to choose from example blocklists on new install
- remove adlists.default
- do not copy non-existent adlists.default on run of gravity

Signed-off-by: Adam Warner <adamw@rner.email>
2018-04-17 10:33:06 +01:00
Jacob Salmela 187848660c
try another fix for #2028
Signed-off-by: Jacob Salmela <jacob.salmela@pi-hole.net>
2018-03-31 16:12:29 -05:00
Jacob Salmela c3f391dc5a
maybe fixes #2028 by deleting null bytes
Signed-off-by: Jacob Salmela <jacob.salmela@pi-hole.net>
2018-03-31 14:08:48 -05:00
DL6ER ff47ab12fb
Save number of effectively blocked domains in /etc/pihole/numBlocked
Signed-off-by: DL6ER <dl6er@dl6er.de>
2018-03-30 21:01:51 +02:00
Riley Avron f390671018 Improve scheme matching to avoid inadvertent blocking
Signed-off-by: Riley Avron <riley.avron@gmail.com>
2018-03-06 22:03:56 -08:00
Riley Avron 512ec7fb87 Remove unneeded ifs in awk
Signed-off-by: Riley Avron <riley.avron@gmail.com>
2018-03-06 22:03:43 -08:00
Adam Warner 24219422df
Merge branch 'development' into FTLDNSremoveDnsmasq 2018-03-06 20:43:27 +00:00
Adam Warner ad1f210528
As we no longer add source lists to the whitelist by default, gravity falls over on new installs.
Signed-off-by: Adam Warner <adamw@rner.email>
2018-03-06 18:29:28 +00:00
Adam Warner cadab61a21
Merge branch 'development' into FTLDNS 2018-03-05 21:33:53 +00:00
DL6ER 7ed45d26c2
Remove " " at some places where they shouldn't be
Signed-off-by: DL6ER <dl6er@dl6er.de>
2018-02-26 17:26:51 +01:00
DL6ER e8ec0dc701
Use pihole-FTL resolver in pihole and gravity
Signed-off-by: DL6ER <dl6er@dl6er.de>
2018-02-25 09:34:04 +01:00
Adam Warner 6b4685b333
much more elegant to export LC_ALL at the beginning of the script!
Signed-off-by: Adam Warner <adamw@rner.email>
2018-02-21 11:33:29 +00:00
Adam Warner d4ac818a0b
Specifty `LC_ALL=C` when dealing with sorted lists.
Signed-off-by: Adam Warner <adamw@rner.email>
2018-02-21 11:23:07 +00:00
DL6ER fc5034696d
Don't auto-whitelist blocking list domains
Signed-off-by: DL6ER <dl6er@dl6er.de>
2018-02-16 19:42:33 +01:00
Dan Schaper 9f2ce0e296
Merge pull request #1910 from pi-hole/fix/BinaryFileLineInGravity
Potential fix for #1909
2018-01-28 03:23:19 -08:00
Adam Warner 55e10d8287
replace another grep -F as pointed out by @StarPicard
Signed-off-by: Adam Warner <adamw@rner.email>
2018-01-28 00:50:15 +00:00
Adam Warner f1956119fb
proposed gravity function renames
Signed-off-by: Adam Warner <adamw@rner.email>
2018-01-14 20:38:39 +00:00
Adam Warner 876170767c
Print actual domain instead of grep output of "Binary file /etc/pihole/list.preEventHorizon matches" when a domain with a unicode char is in a source list.
Signed-off-by: Adam Warner <adamw@rner.email>
2018-01-13 16:43:20 +00:00
Adam Warner 2d3906450a
add back in str local var, and change ticks to info on whitelist lines
Signed-off-by: Adam Warner <adamw@rner.email>
2017-12-18 20:14:53 +00:00
Adam Warner 5ecc46d85b
Accidentally too many spaces
Signed-off-by: Adam Warner <adamw@rner.email>
2017-12-16 14:03:12 +00:00
Adam Warner ddd6f3dbe5
Grammar changes. Lets not work out programmatically whether we need to pluralise a word. It's not needed.
Signed-off-by: Adam Warner <adamw@rner.email>
2017-12-16 13:55:52 +00:00
Adam Warner 761fd13a70
Remove quotes from the uniqDomains array to actually whitelist the sources
Signed-off-by: Adam Warner <adamw@rner.email>
2017-12-16 13:15:26 +00:00
Adam Warner c24e5cd3f6
:codacy: :codacy2: :codacy3:
Signed-off-by: Adam Warner <adamw@rner.email>
2017-12-12 12:45:25 +00:00
Adam Warner cc3035267f
Revert parsing method to that of 3.1.4. New Method is allowing illegal lists through, and that is causing issues such as blocking www.google.com. This method will just throw away invalid list entries.
Signed-off-by: Adam Warner <adamw@rner.email>
2017-12-12 12:36:09 +00:00
Jacob Salmela 80e17ab721
potentially fixes #1806 by falling back to dig if getent fails
Signed-off-by: Jacob Salmela <github@decoy.email>
2017-12-07 22:38:47 -06:00
Mcat12 8d721d086c
Modify indentation 2017-11-21 18:37:38 -05:00