Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

In this Discussion

osTicket v1.10 (stable) and Maintenance Release v1.9.15 are now available! Go get it now

[resolved] URGENT: Bug - Truncated mails in v1.10

osTicket-Version     v1.10 (901e5ea) — Up to date
Server-Software     Apache/2.4.10 (Debian)
MySQL-Version     5.5.49
PHP-Version     5.6.20-0+deb8

After some complaints from customers, we noticed that osticket sometimes truncate mails.

E.g. an incoming mail with the text

Hi Mats
11.12.16
is in osticket truncated to
Hi Mats
This seems to be a very bad bug, since it only occurs in some of the incoming mails.

Here is the full email including header for testing (ip/mail/server name changed):
Return-Path: 
X-Original-To: receiver@example.org
Delivered-To: archiv@example.org
Received: from localhost (ourmailserver [127.0.0.1])
by ourmailserver (Postfix) with ESMTP id 5E211281F202;
Sat, 31 Dec 2016 07:39:58 +0100 (CET)
X-Virus-Scanned: Debian amavisd-new at
ourmailserver
Received: from ourmailserver ([127.0.0.1])
by localhost (ourmailserver [127.0.0.1]) (amavisd-new, port 10024)
with ESMTP id b4_pai8SrIfZ; Sat, 31 Dec 2016 07:39:57 +0100 (CET)
Received: from mout.gmx.net (unknown [212.227.17.20])
(using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(No client certificate requested)
by ourmailserver (Postfix) with ESMTPS id 5426D281F201
for ; Sat, 31 Dec 2016 07:39:57 +0100 (CET)
Received: from [84.152.xxx.xxx] by xxx (via HTTP);
Sat, 31 Dec 2016 07:39:57 +0100
MIME-Version: 1.0
Message-ID:
From: "name"
To: receiver@example.org
Subject: Bestellung
Content-Type: text/html; charset=UTF-8
Date: Sat, 31 Dec 2016 07:39:57 +0100
X-Provags-ID: V03:K0:qQDi030ak0umJ01NFhumMK/qsi3/M/vE314je8FkgLy
s+5Q0yIgyIzgYIq53o5ofVVj9kJEe7+0V6OcGVpY3SxR/Yeo7J
Pn0OC/XULGLAQYdlXe01aPWZKrJ0tPCvsKtnw7A69LqnfpMTj+
DhqOtyW4DCdpRiiWr9nSY79yn58M43nMOByTlCow/fLiEQfKqo
1+fwW8qOUACiBTQ62TJqD3rur7ls8WZsHicloIoP83rbEUgX5r
9JkOTXbB/PD7CusdOPKCucKXKGVAfSe2wB50aPKvJSuyoWEbMY BE9IjU=
X-UI-Out-Filterresults: notjunk:1;V01:K0:3Oy14AThXNs=:3qAJ7Z51jWc7oESnj1J+EK
A0o9mKLa6rLwJZuXPq6UtYASqzsKL2s2EsbGIb2PMKnxz3Cdj24Pn2sJMGoPeWpyiZ9hG2sKE
jf4PMuIuyfH/3krEQRQUyVbgWcQqyt+4ADOjuVlvqjh0krp6rcRvv058GmsUq//i3uc6GH5Ud
geYRo1zUPSHytV24/RWF4eJAcJt08kIxukqSE5hngqWByH11pEMv2pPgeEzq9QDa42pn75HQM
rI2nT2fRS1fhwHBYJMfAaG/82sw6QfGn/7nRTui/PhXaQ8p8feHV6yhKj6i6ScgmSTgrtn5xg
T9RxQNX3cs3hzZHt5PwWQuCLBc2IMnaUOKRGJonUC0u+gIDU+asI3rWpULrbFwdM3OwEaZYQ0
9wKqAsG4mamikfplXtwvX7kxKBAPgNmKAmsituOXM5M4Htnjnas86eKY99wbnO+K5HIk9MBNB
pxOBT52Nr5Nw7kd0lj+mZlv2tbxsPBtEcjJpkND59TvzX8X388Yk

Hi Mats

11.12.16

--

Diese Nachricht wurde von meinem Android Mobiltelefon mit GMX Mail gesendet.
Here as image if this forum strips the html tags:
image

Comments

  • Maybe you have used an unusual separator tag:


    The only other reason I know why tickets get truncated is a four byte character (for example an emoji):

  • Thanks for help.

    > Maybe you have used an unusual separator tag:

    There is no separator tag defined (field is empty).

    Does anyone can point me, how I can test incoming mails without sending test mails (e.g. in a php file set $incoming_mail = [here email header + body]? This would make troubleshooting easier.
  • "Field is empty" sounds to me that space could be in use as a separator tag.

    Maybe you should define a tag to see if the problem disappears..
  • As a side note: I pinged the Devs about this yesterday but failed to update the thread.
    They are trying to replicate this report currently and I expect to hear back from them regarding it.
  • Thanks ntozier.

    I have found two other incoming mails causing this trouble. Those mails are all from web.de accounts (web.de is a German freemail provider). They all have a X-Provags-ID and a X-UI-Out-Filterresults tag in the mail header, maybe this is causing the trouble?

    If I can provide any other information to help to get this fixed ASAP, please let me know.

  • How could you set/save an empty separator tag?
    In my installations (1.9.x/1.10) the gui from osticket prevents that and an error is displayed (Reply separator is required to strip quoted reply).

    The only way I can do this is to change it in the database without the gui.

    Btw: I don't know how to test this without sending test mails..

  • Can you send me the original email in a DM? I piped the email text on this page to osTicket and it created a ticket just fine, so I need the original to test it definitively. Thank you.
  • edited January 5
    @mfelber you pipe it to osTicket! Put the email file (.text or whatever) into the main osTicket directory and in command line you $ php api/pipe.php < example.text and it should create a ticket in the helpdesk! Pretty cool stuff!
  • Never mind, I found the issue and I'm making a Pull Request now. The issue was with our sanitation method. When we sanitize incoming emails we don't catch tags like '<head/>' leaving us with a bad format and a truncated email. Thank you for the find (very unique)!
  • edited January 5
    wait... since when is '<head/>' a legitimate HTML(5) tag?

    Tags that do not have closing tags like meta or br sure... but head has a closing tag.  I do not think that head/ is a legitimate HTML code.  So there wouldn't really be a reason to look for it.
  • If you're interested https://github.com/osTicket/osTicket/pull/3600 is the pull request. You can pull it and test it if you'd like!
  • Your fix works like a charm. A BIG THANK YOU for all helping to find a fix so fast!
  • Not a problem! Thank you for reporting the find!
This discussion has been closed.