webadmin/bpfire
1
0
Fork 0
mirror of https://github.com/vincentmli/bpfire.git synced 2026-04-10 02:55:55 +02:00
Code Issues Packages Projects Releases Wiki Activity

Fix bug 11558 updxlrator: use mirror mode for SHA1, filenames

Browse Source 
Most Microsoft updates now contain an SHA1 hash in the filename.
Since these files are uniquely identifiable, use mirror mode
(which creates a hash of just the filename instead of the entire URL)
to cache them. (But first check the URL cache to see if it
has been downloaded as a URL already.)

This is a HUGELY needed fix. Windows 10 updates are 5+ GB
per month, and we lose several days of bandwidth downloading
duplicates from different mirrors. Sometimes a single client
will request the same patch from multiple mirrors. That's bad.
This patch will save a ton of bandwidth, and lots of disk space.

The patch limits the SHA1 test to microsoft only, but it
could be easily expanded to other vendors if there is a need.

Signed-off-by: Justin Luth  <jluth@mail.com>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
This commit is contained in:
Justin Luth
2017-12-30 22:12:01 +03:00
committed by Michael Tremer
parent 645da4f33d
commit 329631dee9
1 changed files with 13 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
config/updxlrator/updxlrator
View File

@@ -86,6 +86,8 @@ while (<>) {
&& ($source_url !~ m@\&@)
)
{
# NOTE: check_cache will change to $mirror instead of $unique if the filename contains an SHA1 hash
# and the URL is not found in cache!
$xlrator_url = &check_cache($source_url,$hostaddr,$username,"Microsoft",$unique);
}
@@ -400,6 +402,17 @@ sub check_cache
&debuglog("Retrieving file from cache ($updsource)");
&setcachestatus("$updcachedir/$vendorid/$uuid/access.log",time);
$cacheurl="http://$netsettings{'GREEN_ADDRESS'}:$http_port/updatecache/$vendorid/$uuid/$updfile";
}
elsif (
($cfmirror == $unique) &&
($vendorid == "microsoft") &&
($source_url =~ m@.*[0-9a-f]{40}\.[^\.]+@i)
)
{
# Most Microsoft updates now have an SHA1 hash in the name. These should be treated as unique files.
# Since it wasn't found in the URL cache, switch to mirror mode and try again using just the filename.
&debuglog("SHA1: $vendorid $uuid not cached. Reprocessing as mirror $sourceurl");
$cacheurl = &check_cache($source_url,$hostaddr,$username,$vendorid,$mirror);
}
else
{