mail charset incorrect in helpdesk tiket body

linvinus · 2009-07-16 11:43:13

if body charset different from utf8 for example
Content-Type: text/plain; charset="koi8-r"
then charset in tiket body in helpdesk are broken

email example

Return-Path: Denis@domain.ru
Received: from dc1.domain2.ru (LHLO dc1.domain2.ru) (10.10.1.1) by dc1.domain2.ru
with LMTP; Wed, 15 Jul 2009 17:01:13 +0400 (MSD)
Received: from localhost (localhost [127.0.0.1])
by dc1.domain2.ru (Postfix) with ESMTP id B911A76AD
for <help@domain2.ru>; Wed, 15 Jul 2009 17:01:13 +0400 (MSD)
X-Virus-Scanned: amavisd-new at domain2.ru
X-Spam-Flag: NO
X-Spam-Score: -0.222
X-Spam-Level:
X-Spam-Status: No, score=-0.222 tagged_above=-10 required=6.6
tests=[AWL=0.158, BAYES_00=-2.599, TVD_SPACE_RATIO=2.219]
Received: from dc1.domain2.ru ([127.0.0.1])
by localhost (dc1.domain2.ru [127.0.0.1]) (amavisd-new, port 10024)
with ESMTP id tNwL6G-2MaTQ for <help@domain2.ru>;
Wed, 15 Jul 2009 17:01:09 +0400 (MSD)
Received: from gw.domain.ru (gw.domain.ru [195.28.52.95])
by dc1.domain2.ru (Postfix) with ESMTP id F2DEA74E7
for <help@domain2.ru>; Wed, 15 Jul 2009 17:01:08 +0400 (MSD)
X-Virus-Scanner: This message was checked by NOD32 Antivirus system
NOD32 for Linux Mail Server.
For more information on NOD32 Antivirus System,
please, visit our website: http://www.nod32.com/.
Received: from exchub.domain.ru (exchub.domain.ru [192.168.1.25])
by gw.domain.ru (Postfix) with ESMTP id 826261021E
for <help@domain2.ru>; Wed, 15 Jul 2009 17:01:08 +0400 (MSD)
Received: from CLMAILBOX.domain.sw ([192.168.1.22]) by exchub.domain.ru
([192.168.1.25]) with mapi; Wed, 15 Jul 2009 17:01:07 +0400
From: =?koi8-r?B?68/O09TBztTJzs/XIOTFzsnT?= <Denis@domain.ru>
To: "help@domain2.ru" <help@domain2.ru>
Date: Wed, 15 Jul 2009 17:01:06 +0400
Subject: =?koi8-r?Q?=D4=C5=D3=D4?=
Thread-Topic: =?koi8-r?Q?=D4=C5=D3=D4?=
Thread-Index: AQHKBUxQFHlJ6tpd5k6D+TBUFef5fQ==
Message-ID: <31620F7EC6955B4F86513B4C0682128BEAAAE5A9AC@CLMAILBOX.domain.ru>
Accept-Language: ru-RU
Content-Language: ru-RU
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
acceptlanguage: ru-RU
Content-Type: text/plain; charset="koi8-r"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-SpamTest-Version: SMTP-Filter Version 2.0.0 [0124], KAS/Release
X-Spamtest-Info: Pass through

=C6=D9=D7=C6=D9=D8=C2=CC=C4=D8=C4=CC=

following patch for glpi 0.72 resolve this problem
it also works for Content-Type: multipart/mixed;
(patch change function get_part )

--- ./inc.old/mailgate.class.php 2009-07-15 23:15:00.000000000 +0400
+++ ./inc/mailgate.class.php 2009-07-16 13:05:07.000000000 +0400
@@ -571,13 +571,23 @@
$part_number = "1";
}
$text = imap_fetchbody($stream, $msg_number, $part_number);
- if($structure->encoding == 3) {
- return imap_base64($text);
- } else if($structure->encoding == 4) {
- return imap_qprint($text);
- } else {
- return $text;
- }
+ if($structure->encoding == 3) {
+ $text = imap_base64($text);
+ } else if($structure->encoding == 4) {
+ $text = imap_qprint($text);
+ } /*else {
+ return $text;
+ } */
+ //$text .= " this->structure->parameters".$structure->parameters." count=".count($structure->parameters);
+ if (count($structure->parameters)>0){
+ foreach ($structure->parameters as $param){
+ if ((strtoupper($param->attribute)=='CHARSET')&&function_exists('mb_convert_encoding')){
+ $text = mb_convert_encoding($text, 'utf-8',$param->value);
+ }
+ }
+ }
+
+ return $text;
}
if($structure->type == 1){ /* multipart */
$prefix="";

P.S.
i didn't find how to register on https://dev.indepnet.net/glpi/report so i report bug here.
sorry for English

Last edited by linvinus (2009-07-16 11:49:39)

MoYo · 2009-07-26 14:28:04

bug must be post here.
If you have no answer it is just because we lack of time.

You patch seems to be interesting but we need to do some tests

linvinus · 2009-07-27 09:12:32

ok, i'm understand.
there is my last patch
+ if( $structure->subtype && $structure->subtype=="HTML"){
+ $text = str_replace("\r","",$text);
+ $text = str_replace("\n","",$text);
+ }

it resolve double line spaces in mail if content text/html
because in html end line it is tag , so \n and \r we should ignore.
example:

line 1 \r\n
line 2 \r\n
result in glpi without patch
line1

line2

after patch
line1
line2

--- ./inc.old/mailgate.class.php 2009-07-15 23:15:00.000000000 +0400
+++ ./inc/mailgate.class.php 2009-07-16 13:05:07.000000000 +0400
$part_number = "1";
}
$text = imap_fetchbody($stream, $msg_number, $part_number);
- if($structure->encoding == 3) {
- return imap_base64($text);
- } else if($structure->encoding == 4) {
- return imap_qprint($text);
- } else {
- return $text;
- }
+ if($structure->encoding == 3) {
+ $text = imap_base64($text);
+ } else if($structure->encoding == 4) {
+ $text = imap_qprint($text);
+ } /*else {
+ return $text;
+ } */
+ //$text .= " this->structure->parameters".$structure->parameters." count=".count($structure->parameters);
+
+ if( $structure->subtype && $structure->subtype=="HTML"){
+ $text = str_replace("\r","",$text);
+ $text = str_replace("\n","",$text);
+ }
+
+ if (count($structure->parameters)>0){
+ foreach ($structure->parameters as $param){
+ if ((strtoupper($param->attribute)=='CHARSET')&&function_exists('mb_convert_encoding')){
+ $text = mb_convert_encoding($text, 'utf-8',$param->value);
+ }
+ }
+ }
+
+ return $text;
}
if($structure->type == 1){ /* multipart */
$prefix="";

Last edited by linvinus (2009-07-27 09:20:19)

kpcomp · 2009-09-14 19:32:02

This patch work for Outlook 2003 emails in Czech encoding (probably ISO-8859-2), but the one included in 0.72.21 did not (https://forge.indepnet.net/issues/show/1635). It looks like "tiskĂĄrny" instead of "tiskárny". Looks like untranslated double-byte UTF-8

Difference is line 590:

linvinus original:
if ((strtoupper($param->attribute)=='CHARSET') && function_exists('mb_convert_encoding')){

0.72.21 modified:
if ((strtoupper($param->attribute)=='CHARSET') && function_exists('mb_convert_encoding') && strtoupper($param->value) != 'UTF-8'){

Best, Pavel

kpcomp · 2009-09-15 10:41:58

Sorry. Too soon... Still same error.
Started after upgrade 0.72 => 0.72.21.

Pavel

MoYo · 2009-09-17 21:26:11

the patch applied on 0.72.21 is the last version of linvinus patch.
I do not understand what you want to say on your last post.

kpcomp · 2009-09-18 11:32:21

After upgrade from 0.72 to 0.72.21 it changed behavior and now
body is mess like "tiskĂĄrny" instead of "tiskárny" or "sepsĂĄnĂ"
instead of "sepsání".

Same problem for ISO-8859-2 and for UTF-8. Examples below.

Before upgrade it was different mess with different encoding error
but was "readable".

Subject seems to be OK.

Examaple email ISO-8859-2 Quoted Printable:
Content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary="----_=_NextPart_001_01CA3835.3CE11580"
X-MimeOLE: Produced By Microsoft Exchange V6.5
Subject: =?iso-8859-2?B?Y2h5Ym7pIOy5+L7o7Ln9+Ojsue396L744bn9+L7huej4uej4?=
Date: Fri, 18 Sep 2009 09:54:23 +0200
Message-ID: <7441A17B1572464590F3F529C3F1CE3702B25D@servername>
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Thread-Topic: =?iso-8859-2?B?Y2h5Ym7pIOy5+L7o7Ln9+Ojsue396L744bn9+L7huej4uej4?=
Thread-Index: Aco4NTzQieZOARfGRtSpG2aKiWOBrA==
From: "Administrator" <administrator@domain.tld>
To: <podpora@domain.tld>
X-Spam-Status: No, hits=1.90 required=3.00 tests=HTML_MESSAGE=0.00,BAYES_00=-0.11,NO_RDNS2=0.01,OTHER=2.0
version=3.2.5
X-Spam-Level: *
X-Spam-Checker-Version: SpamAssassin 3.2.5 (1.1) on canopus.kpcomp.cz

This is a multi-part message in MIME format.

------_=_NextPart_001_01CA3835.3CE11580
Content-Type: text/plain;
charset="iso-8859-2"
Content-Transfer-Encoding: quoted-printable

N=ECjak=E9 divn=E9 kodov=E1n=ED v emailu

=A9=E8=F8=BE=FD=B9=E1=EC=F8=B9=FD=BE=E9=ED=B9=E8=FD=BE=E9=ED=E1=FD=BE

Adsfjaklfhf=F9ladsfj=F9fj=F9laadsf

=CC=B9=E8=F8=BE=FD=E1=ED

=20

K"LJsdf

------_=_NextPart_001_01CA3835.3CE11580
Content-Type: text/html;
charset="iso-8859-2"
Content-Transfer-Encoding: quoted-printable

</head>

N=ECjak=E9 divn=E9 kodov=E1n=ED =
v emailu<o:p></o:p>

=A9=E8=F8=BE=FD=B9=E1=EC=F8=B9=FD=BE=E9=ED=B9=E8=FD=BE=
=E9=ED=E1=FD=BE<o:p></o:p>

Adsfjaklfhf=F9ladsfj=F9fj=F9laadsf<o:p></o:p></=
font>

=CC=B9=E8=F8=BE=FD=E1=ED<o:p></o:p></span=
>

<o:p> </o:p>

K“LJsdf<o:p></o:p>

</div>

</body>

</html>

------_=_NextPart_001_01CA3835.3CE11580--

Example mail UTF-8 Base64 encoded:
Content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary="----_=_NextPart_001_01CA3838.7B8E52C8"
X-MimeOLE: Produced By Microsoft Exchange V6.5
Subject: =?utf-8?B?d2XEjcWZxaHEjcW+xZnEjcW+w73FmcW+w73FmcOhw73FvsO9w6FnxI1mxb50dXR6?=
=?utf-8?B?dcO9?=
Date: Fri, 18 Sep 2009 10:17:37 +0200
Message-ID: <7441A17B1572464590F3F529C3F1CE3702B25F@pe2650.almika.local>
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Thread-Topic: =?utf-8?B?d2XEjcWZxaHEjcW+xZnEjcW+w73FmcW+w73FmcOhw73FvsO9w6FnxI1mxb50dXR6?=
=?utf-8?B?dcO9?=
Thread-Index: Aco4OHuJdzWIsCoaTSyZ86ASy55V/g==
From: "Administrator" <administrator@domain.tld>
To: <podpora@domain.tld>
X-Spam-Status: No, hits=1.90 required=3.00 tests=HTML_MESSAGE=0.00,BAYES_00=-0.11,NO_RDNS2=0.01,OTHER=2.0
version=3.2.5
X-Spam-Level: *
X-Spam-Checker-Version: SpamAssassin 3.2.5 (1.1) on canopus.kpcomp.cz

This is a multi-part message in MIME format.

------_=_NextPart_001_01CA3838.7B8E52C8
Content-Type: text/plain;
charset="utf-8"
Content-Transfer-Encoding: base64

xaDEm8SNxZnFvsO9w6HDrcOpPWVzw71vYnplw6FvxI3DvXhuemVzcHJnY2RmDQoNCiANCg0KR2Zo
DQoNCkhoZGhqZmhqZ2hqDQoNCkRlcMOtY8Ohxb7FocOpxZnDocW+w71kw6FtdWVpcm9lZ3QNCg0K
IA0KDQo=

------_=_NextPart_001_01CA3838.7B8E52C8
Content-Type: text/html;
charset="utf-8"
Content-Transfer-Encoding: base64

PGh0bWwgeG1sbnM6bz0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6b2ZmaWNlIiB4
bWxuczp3PSJ1cm46c2NoZW1hcy1taWNyb3NvZnQtY29tOm9mZmljZTp3b3JkIiB4bWxucz0iaHR0
cDovL3d3dy53My5vcmcvVFIvUkVDLWh0bWw0MCI+DQoNCjxoZWFkPg0KPG1ldGEgaHR0cC1lcXVp
dj1Db250ZW50LVR5cGUgY29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjxtZXRh
IG5hbWU9R2VuZXJhdG9yIGNvbnRlbnQ9Ik1pY3Jvc29mdCBXb3JkIDExIChmaWx0ZXJlZCBtZWRp
dW0pIj4NCjxzdHlsZT4NCjwhLS0NCiAvKiBTdHlsZSBEZWZpbml0aW9ucyAqLw0KIHAuTXNvTm9y
bWFsLCBsaS5Nc29Ob3JtYWwsIGRpdi5Nc29Ob3JtYWwNCgl7bWFyZ2luOjBjbTsNCgltYXJnaW4t
Ym90dG9tOi4wMDAxcHQ7DQoJZm9udC1zaXplOjEyLjBwdDsNCglmb250LWZhbWlseToiVGltZXMg
TmV3IFJvbWFuIjt9DQphOmxpbmssIHNwYW4uTXNvSHlwZXJsaW5rDQoJe2NvbG9yOmJsdWU7DQoJ
dGV4dC1kZWNvcmF0aW9uOnVuZGVybGluZTt9DQphOnZpc2l0ZWQsIHNwYW4uTXNvSHlwZXJsaW5r
Rm9sbG93ZWQNCgl7Y29sb3I6cHVycGxlOw0KCXRleHQtZGVjb3JhdGlvbjp1bmRlcmxpbmU7fQ0K
c3Bhbi5TdHlsRS1tYWlsb3ZacHJ2eTE3DQoJe21zby1zdHlsZS10eXBlOnBlcnNvbmFsLWNvbXBv
c2U7DQoJZm9udC1mYW1pbHk6QXJpYWw7DQoJY29sb3I6d2luZG93dGV4dDt9DQpAcGFnZSBTZWN0
aW9uMQ0KCXtzaXplOjU5NS4zcHQgODQxLjlwdDsNCgltYXJnaW46NzAuODVwdCA3MC44NXB0IDcw
Ljg1cHQgNzAuODVwdDt9DQpkaXYuU2VjdGlvbjENCgl7cGFnZTpTZWN0aW9uMTt9DQotLT4NCjwv
c3R5bGU+DQoNCjwvaGVhZD4NCg0KPGJvZHkgbGFuZz1DUyBsaW5rPWJsdWUgdmxpbms9cHVycGxl
Pg0KDQo8ZGl2IGNsYXNzPVNlY3Rpb24xPg0KDQo8cCBjbGFzcz1Nc29Ob3JtYWw+PGZvbnQgc2l6
ZT0yIGZhY2U9QXJpYWw+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMC4wcHQ7DQpmb250LWZhbWls
eTpBcmlhbCc+xaDEm8SNxZnFvsO9w6HDrcOpPWVzw71vYnplw6FvxI3DvXhuemVzcHJnY2RmPG86
cD48L286cD48L3NwYW4+PC9mb250PjwvcD4NCg0KPHAgY2xhc3M9TXNvTm9ybWFsPjxmb250IHNp
emU9MiBmYWNlPUFyaWFsPjxzcGFuIHN0eWxlPSdmb250LXNpemU6MTAuMHB0Ow0KZm9udC1mYW1p
bHk6QXJpYWwnPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvZm9udD48L3A+DQoNCjxwIGNsYXNz
PU1zb05vcm1hbD48Zm9udCBzaXplPTIgZmFjZT1BcmlhbD48c3BhbiBzdHlsZT0nZm9udC1zaXpl
OjEwLjBwdDsNCmZvbnQtZmFtaWx5OkFyaWFsJz5HZmg8bzpwPjwvbzpwPjwvc3Bhbj48L2ZvbnQ+
PC9wPg0KDQo8cCBjbGFzcz1Nc29Ob3JtYWw+PGZvbnQgc2l6ZT0yIGZhY2U9QXJpYWw+PHNwYW4g
c3R5bGU9J2ZvbnQtc2l6ZToxMC4wcHQ7DQpmb250LWZhbWlseTpBcmlhbCc+SGhkaGpmaGpnaGo8
bzpwPjwvbzpwPjwvc3Bhbj48L2ZvbnQ+PC9wPg0KDQo8cCBjbGFzcz1Nc29Ob3JtYWw+PGZvbnQg
c2l6ZT0yIGZhY2U9QXJpYWw+PHNwYW4gc3R5bGU9J2ZvbnQtc2l6ZToxMC4wcHQ7DQpmb250LWZh
bWlseTpBcmlhbCc+RGVwPGI+PHNwYW4gc3R5bGU9J2ZvbnQtd2VpZ2h0OmJvbGQnPsOtY8Ohxb7F
ocOpPC9zcGFuPjwvYj7FmcOhxb7DvWTDoW11ZWlyb2VndDxvOnA+PC9vOnA+PC9zcGFuPjwvZm9u
dD48L3A+DQoNCjxwIGNsYXNzPU1zb05vcm1hbD48Zm9udCBzaXplPTIgZmFjZT1BcmlhbD48c3Bh
biBzdHlsZT0nZm9udC1zaXplOjEwLjBwdDsNCmZvbnQtZmFtaWx5OkFyaWFsJz48bzpwPiZuYnNw
OzwvbzpwPjwvc3Bhbj48L2ZvbnQ+PC9wPg0KDQo8L2Rpdj4NCg0KPC9ib2R5Pg0KDQo8L2h0bWw+
DQo=

------_=_NextPart_001_01CA3838.7B8E52C8--

kpcomp · 2009-09-18 13:43:53

Those modifications of inc/mailgate.class.php solved some problems with encoding for me:

line 319
if (!empty($this->charset)&&function_exists('mb_convert_encoding'&&strtoupper($this->charset)!='HTML-ENTITIES')){

line 590
if ((strtoupper($param->attribute)=='CHARSET') && function_exists('mb_convert_encoding') && strtoupper($param->value) != 'UTF-8' && strtoupper($param->value) != 'HTML-ENTITIES'){

Still testing.

Best, Pavel

Forum GLPI-Project

Announcement

#1 2009-07-16 11:43:13

mail charset incorrect in helpdesk tiket body

#2 2009-07-26 14:28:04

Re: mail charset incorrect in helpdesk tiket body

#3 2009-07-27 09:12:32

Re: mail charset incorrect in helpdesk tiket body

#4 2009-09-14 19:32:02

Re: mail charset incorrect in helpdesk tiket body

#5 2009-09-15 10:41:58

Re: mail charset incorrect in helpdesk tiket body

#6 2009-09-17 21:26:11

Re: mail charset incorrect in helpdesk tiket body

#7 2009-09-18 11:32:21

Re: mail charset incorrect in helpdesk tiket body

#8 2009-09-18 13:43:53

Re: mail charset incorrect in helpdesk tiket body

Board footer