Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

In this Discussion

osTicket v1.10 (stable) and Maintenance Release v1.9.15 are now available! Go get it now

Problems with german umlauts

Hello,

yesterday evening I search hours about hours for a troubleticket system. Your system was the best trade off between to much and enough features ;-).

Unfortunately I have problems with german umlauts. Every piped ticket/mail in an iso format is truncated at the first "umlaut".

This is a sniped of the mail header ... have you any ideas?

To: [email]xxx@xasdgadsg.de[/email]
Subject: test 123
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-15
Content-Transfer-Encoding: 8bit

Sehr geehrte Frau Weber,

Vielen Dank für Ihre Bestellung!

Der fehlende Artikel Engelbild - Kinder & Brücke ist leider nicht mehr
lieferbar, dieser Artikel ist ausverkauft.

Es wäre schön, wenn Sie sich in Kürze noch einmal bei uns melden könnten,
wie wir mit Ihrer Bestellung verbleiben dürfen, oder sogar gemeinsam eine
Alternative finden.

Mit freundlichen Grüßen,
asdgasd sdfh sdf hfsdhsd
Thanks
Michael
«1

Comments

  • Hello again,

    I tried 3 big german freemailer and always I got the same problem (like descripted above).
    Always wenn E-mails use NO utf-8 encoding there is this problem
    -> the tool is great but needless without a overwork of the parser ....

    Thanks
    Michael
  • Hello,

    no I made some changes in the class.pop3.php and have no problems with "german umlauts".

    I changed the following functions:

        function mime_encode($text,$charset=null,$enc='utf-8') {
    $encodings=array('UTF-8','WINDOWS-1251', 'ISO-8859-1', 'ISO-8859-15', 'WINDOWS-1252');

    if($charset){
    $charset = strtoupper($charset);
    $text = iconv($charset, $enc.'//IGNORE', $text);
    }else{
    $sourceEncoding = mb_detect_encoding($text, $encodings);
    $text = iconv($sourceEncoding, $enc . '//IGNORE', $text);
    }

    return $text;
    }


        function getPart($mid,$mimeType,$encoding=false,$struct=null,$partNumber=false){

    if(!$struct)
    $struct=imap_fetchstructure($this->mbox, $mid);

    //Match the mime type.
    if($struct && strcasecmp($mimeType,$this->getMimeType($struct))==0){
    $partNumber=$partNumber?$partNumber:1;
    if(($text=imap_fetchbody($this->mbox, $mid, $partNumber))){
    if($struct->encoding==3 or $struct->encoding==4){
    $text=$this->decode($struct->encoding,$text);
    }

    if($struct->parameters[0] && !strcasecmp($struct->parameters[0]->attribute,'CHARSET') && strcasecmp($struct->parameters[0]->value,'US-ASCII')) {
    $charset=trim($struct->parameters[0]->value);
    $text=$this->mime_encode($text,$charset,$encoding);
    }else{
    $text=$this->mime_encode($text, '', $encoding);
    }

    return $text;
    }
    }
    //Do recursive search
    if($struct && $struct->parts){
    while(list($i, $substruct) = each($struct->parts)) {
    if($partNumber)
    $prefix = $partNumber . '.';
    if(($text=$this->getPart($mid,$mimeType,$encoding,$substruct,$prefix.($i+1))))
    return $text;
    }
    }
    //No luck.

    return false;
    }


    Michael
  • Hi,

    I want to try this one. Do I integrate the text marked red into the file?

    Because I can't find this text (underneath) in class.pop3.php:

    if($charset){
    $charset = strtoupper($charset);
    $text = iconv($charset, $enc.'//IGNORE', $text);
    }else{
  • Hi,

    just replace the whole functions with this version:

    //Conver text to desired encoding..defaults to utf8
    function mime_encode($text, $charset=null, $enc='utf-8') {
    $encodings = array('UTF-8','WINDOWS-1251', 'WINDOWS-1252', 'ISO-8859-1', 'ISO-8859-15', 'koi8-r');

    if($charset){
    $charset = strtoupper($charset);
    $text = iconv($charset, $enc.'//IGNORE', $text);
    }else{
    $sourceEncoding = mb_detect_encoding($text, $encodings);
    $text = iconv($sourceEncoding, $enc . '//IGNORE', $text);
    }

    return $text;
    }
    Currently I have no further problems with mails. even russian mails work ;-).

    Michael
  • Yes, i understand, but I can't figure out what section to overwrite?

    I looks like this for me:

    //Conver text to desired encoding..defaults to utf8
    function mime_encode($text,$charset=null,$enc='uft-8') { //Thank in part to afterburner

    $encodings=array('UTF-8','WINDOWS-1251', 'ISO-8859-5', 'ISO-8859-1');

    if(function_exists("iconv") and $text) {
    if($charset)
    return iconv($charset,$enc.'//IGNORE',$text);
    elseif(function_exists("mb_detect_encoding"))
    return iconv(mb_detect_encoding($text,$encodings),$enc,$text);
    }
    return imap_utf8($text);
    }



    Should I just overwrite this section with this part underneath?:

    //Conver text to desired encoding..defaults to utf8
    function mime_encode($text, $charset=null, $enc='utf-8') {
    $encodings = array('UTF-8','WINDOWS-1251', 'WINDOWS-1252', 'ISO-8859-1', 'ISO-8859-15', 'koi8-r');

    if($charset){
    $charset = strtoupper($charset);
    $text = iconv($charset, $enc.'//IGNORE', $text);
    }else{
    $sourceEncoding = mb_detect_encoding($text, $encodings);
    $text = iconv($sourceEncoding, $enc . '//IGNORE', $text);
    }

    return $text;
    }
  • I did try doing the above, but still gets ? instead of Å, Ä and Ö..
  • Please note that Å, Ä and Ö are showing if customer opens a ticket, but not if sending an email to us (forwarded to osTicket by POP3)..
  • This is how it looks..

    image
  • This is strange. So the first time the email is sent the characters are replaced with question marks and the second time not?

    I assume you have changed the function mime_encode() like michael said? I wonder if michael has this issue also, but he seems to have no problem anymore.
  • Hi,

    Yes, exactly.

    Å, Ä and Ö was replaced with a questionsmark the second time to BEFORE, but not any longer when customers reply to the message and leaving the ticket id in the subject line.

    I did try to do what Michael wrote but I didn't understand everything..what should I overwrite? Everything is normal in the coding now, nothing changed.
  • Hi again,

    Noticed today that when receiving replies by email, some emails have Å, Ä and Ö replaced with a questionmark, and some shows just like they should.. Wrote earlier that it did work as it should, but now it doesn't sometimes.

    1. Customer sends an email to us (ÅÄÖ always gets replaced with a ?)
    2. We reply to the ticket in osTicket (ÅÄÖ are showing in the ticket)
    3. Customer replies to our reply (Sometimes are ÅÄÖ showing like they should, but sometimes replaced with a ?)

    Need to get this fixed.. Would appreciate all help I can get..
  • Perhaps you need to try Michael's solution first.

    From what I understand there are two functions that you will need to change.

    1) Replace the complete function "mime_encode()" in the file ./include/class.pop3.php with:
    //Conver text to desired encoding..defaults to utf8
    function mime_encode($text, $charset=null, $enc='utf-8') {
    $encodings = array('UTF-8','WINDOWS-1251', 'WINDOWS-1252', 'ISO-8859-1', 'ISO-8859-15', 'koi8-r');

    if($charset){
    $charset = strtoupper($charset);
    $text = iconv($charset, $enc.'//IGNORE', $text);
    }else{
    $sourceEncoding = mb_detect_encoding($text, $encodings);
    $text = iconv($sourceEncoding, $enc . '//IGNORE', $text);
    }

    return $text;
    }


    2) Replace the complete function "getPart()" also in the file ./include/class.pop3.php with:
        function getPart($mid,$mimeType,$encoding=false,$struct=null,$partNumber=false){ 

    if(!$struct)
    $struct=imap_fetchstructure($this->mbox, $mid);

    //Match the mime type.
    if($struct && strcasecmp($mimeType,$this->getMimeType($struct))==0){
    $partNumber=$partNumber?$partNumber:1;
    if(($text=imap_fetchbody($this->mbox, $mid, $partNumber))){
    if($struct->encoding==3 or $struct->encoding==4){
    $text=$this->decode($struct->encoding,$text);
    }

    if($struct->parameters[0] && !strcasecmp($struct->parameters[0]->attribute,'CHARSET') && strcasecmp($struct->parameters[0]->value,'US-ASCII')) {
    $charset=trim($struct->parameters[0]->value);
    $text=$this->mime_encode($text,$charset,$encoding);
    }else{
    $text=$this->mime_encode($text, '', $encoding);
    }

    return $text;
    }
    }
    //Do recursive search
    if($struct && $struct->parts){
    while(list($i, $substruct) = each($struct->parts)) {
    if($partNumber)
    $prefix = $partNumber . '.';
    if(($text=$this->getPart($mid,$mimeType,$encoding,$substruct,$prefix.($i+1))))
    return $text;
    }
    }
    //No luck.

    return false;
    }
  • Hello,

    attached you can find my last Version of the class.pop3.php. Currently I have no further problems with popular charsets in germany. All "special chars" are transformed/converted correctly.

    Michael
    class.pop3.txt
    12K
  • Hi,

    How do I do to download the attachment?

    Will try the things above, and I truly hope that's the solution :)
  • I have now made the changes above, and Å, Ä and Ö are showing like they should! :)

    BUT in the subject-line they still don't work if we receives emails from customers (it works great if we opens a ticket from admin). ÅÄÖ = A?A?O?

    Any suggestions?
  • @michael: thanks for helping out Alex and perhaps others. I approved your attachment, but it's not tested.

    @Alex: the subject could be a different part. Perhaps michael has a solution for this as well. Otherwise, perhaps you can have look yourself?
  • Yes, thanks Michael for your help.

    My coding-skills are not good at all, sorry. I'm improving but I don't know how to fix the subject-line at all.. It says A?A?O? instead of ÅÄÖ.
  • alexn;4465 said:
    Yes, thanks Michael for your help.

    My coding-skills are not good at all, sorry. I'm improving but I don't know how to fix the subject-line at all.. It says A?A?O? instead of ÅÄÖ.
    Hi,

    very importend for the subject is, that "special chars" are correcly encoded ...

    Example:
    Subject: neuer =?ISO-8859-1?Q?G=E4stebucheintrag?=

    Otherwise osticket is not able to transform correclty to utf-8. Could you post the subject line of the header?

    Michael
  • In what file can I find the "subject"-line?

    Header.inc.php <- no "subject"-line there
  • In class.pop3.php there are a few subject-lines:

    1.
    $header=array(
    'from' =>array('name' =>@$sender->personal,'email' =>strtolower($sender->mailbox).'@'.$sender->host),
    'subject'=>@$headerinfo->subject,);
    return $header;
    }

    2.
    $var['subject']=$mailinfo['subject']?imap_utf8($mailinfo['subject']):'[No Subject]';

    Or is it in another file?
  • Hi,

    sorry for the misunderstanding. I mean the Subject Line in the E-Mail Header. You can view this in your current e-mail client.

    Michael
  • Sorry, but I don't really understand what you mean.

    Do you want me to post/create a ticket with the subject-line you wrote? Or do you want a print screen how it looks in the mail client? (the subject-line)

    Sorry
  • alexn;4471 said:
    Sorry, but I don't really understand what you mean.

    Do you want me to post/create a ticket with the subject-line you wrote? Or do you want a print screen how it looks in the mail client? (the subject-line)

    Sorry
    Hi,

    every email has a header with additional informations. Send yourself an E-Mail with the sweden chars and "view" the header in your Mail Client.

    Date: Wed, 17 Sep 2008 07:06:52 +0000
    To: [email]mmmichael@xxxxx.de[/email]
    From: "osTicket Forums"
    Auto-Submitted: auto-generated
    Message-ID: <20080917070641.5312cc255807@osticket.com>
    MIME-Version: 1.0
    Content-Type: text/plain; charset="ISO-8859-1"
    Content-Transfer-Encoding: 8bit
    X-Priority: 3
    Michael
  • In addition to Michael's answer above:
    with Mail Client he doesn't mean osTicket, but Outlook or something like that.
  • Is this what you mean? Email sent FROM a hotmail-address TO the email-address using osTicket.

    From: "A:A1"
    To: "xxxx.se [Support]"
    Subject: =?iso-8859-1?B?5eT2?=
    Date: Thu, 2 Oct 2008 22:28:43 +0200
    MIME-Version: 1.0
    Content-Type: multipart/alternative;
    boundary="----=_NextPart_000_01B2_01C924DE.3A75AD20"
    X-Priority: 3
    X-MSMail-Priority: Normal
    Importance: Normal
    X-Mailer: Microsoft Windows Live Mail 12.0.1606
    X-MimeOLE: Produced By Microsoft MimeOLE V12.0.1606


    Subject: åäö
    Message: åäö

    Message was shown correctly in osTicket, but the subject-line was replaced with A?A?O?
  • Hi,

    yes, this is what I looked for. I have no problems with an quoted-printable decoding. An Test Message from gmx.de (a very popular freemailer in germany) was successfull.

    Subject: =?iso-8859-1?Q?=E5=E4=F6?=

    You (hotmail) has a base64 encoding, maybe this could be the problem??

    Subject: =?iso-8859-1?B?5eT2?=

    Michael
  • Some of our customers are using gmail, some hotmail etc. etc. But subject line always replaces Å Ä Ö with ?.

    I can't figure out why..?
  • Suggestions?
  • Has anyone solved this for the Subject and Name entries?

    I get this:
    Incorrect string value: '\xCC\x88he' for column 'subject' at row 1
    Incorrect string value: '\xCC\x88he' for column 'name' at row 1

    For subjects and names.

    I am already using mmmichael's modified pop3.class.php for the message body being displayed.
  • Hi,

    sorry, I have only time for a short statement:

    With my modified class.pop3.php I have absolutly no problems with russian/german/swedish/... special chars.

    Maybe all of your problems have there origin in missing php modules (iconv, imap)?

    Michael
Sign In or Register to comment.