thh/newsstats
1
0
Fork 0
Code Issues 4 Pull requests Packages Projects Releases Activity

Forcibly decode headers with unencoded 8bit chars.

Browse source 
Just assume UTF-8 for the time being.
Fixes database errors with illegal characters
when writing parsed data.

Signed-off-by: Thomas Hochstein <thh@inter.net>
This commit is contained in:
Thomas Hochstein 2021-05-29 10:17:00 +02:00
parent 6deb7dbaa4
commit b99d41010d
1 changed files with 5 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
bin/parsedb.pl
View file

@ -198,6 +198,11 @@ while (my $HeadersR = $DBQuery->fetchrow_hashref) {
$Headers{$HeaderName.'_parsed'} = decode('MIME-Header',$Headers{$_})
if (exists($LegalEncodings{$Encoding}));
}
# forcibly modify headers with un-encoded 8bit data assuming utf-8
# TODO: try to guess correct enconding
elsif ($Headers{$_} =~ /[^\x00-\x7F]/) {
$Headers{$_} = decode('utf-8',$Headers{$_});
}
# extract name(s) and mail(s) from From: / Sender: / Reply-To:
# in parsed form, if available
if ($_ ne 'subject') {