Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

In this Discussion

osTicket v1.10 (stable) and Maintenance Release v1.9.15 are now available! Go get it now

Problems with german umlauts

2»

Comments

  • @mmmichael:

    Another thing I noticed was that with your modified file, all incoming tickets via email have this before the actual email body:

    html::


    And occasionally a few carriage returns, but not always.

    Have you ever heard of something similar?

    Can you provide a list with you php extensions, for me to compare?
  • SentryRaven;5544 said:

    Can you provide a list with you php extensions, for me to compare?
    Hi,

    I use PHP Version 5.2.6 with imap Support (IMAP c-Client Version: 2004; SSL Support: enabled) and iconv (iconv support enabled; iconv implementation: glibc; iconv library version: 1.11). Iconv seems to be very important, because it is the interface to character conversion facilities.

    Michael
  • iconv implementation 	\"libiconv\"
    iconv library version 1.9


    iconv.input_encoding	ISO-8859-1	ISO-8859-1
    iconv.internal_encoding ISO-8859-1 ISO-8859-1
    iconv.output_encoding ISO-8859-1 ISO-8859-1


    PHP 5.2.3
    IMAP the same as yours

    It works all well for Umlauts in the Message body, although I am still looking to see where the html:: and the empty lines come from in my ticket system, but I still receive DB Errors from Umlauts in the Subject and FROM: Name.

    [INSERT INTO ost_ticket SET created=NOW() ,ticketID=245763,dept_id=1,priority_id=2,email='mail@*snip*',name='Adam *snip*',subject='Test 2 Kühe',phone='',ip_address='*snip*',source='Email'] - Incorrect string value: '\xCC\x88he' for column 'subject' at row 1
  • Ok... converted my DB to utf8_bin, from latin_swedish.... seems to have done the trick, no more DB errors since then... but.... umlauts are not displayed at th what they should be... ü looks more like a... cant even describe...

    and another thing is bothering me:

    html::


    .hmmessage P
    {
    margin:0px;
    padding:0px
    }
    body.hmmessage
    {
    font-size: 10pt;
    font-family:Verdana
    }
  • I swear, I am not making this stuff up:
    [INSERT INTO ost_ticket_message SET created=NOW() ,ticket_id=6,message='html::\n\n\n.hmmessage P\n{\nmargin:0px;\npadding:0px\n}\nbody.hmmessage\n{\nfont-size: 10pt;\nfont-family:Verdana\n}\n\n\n\nVid Söderarm utanför Stockholm finns stormvindar på 25 meter per sekund, och det har även uppnåtts vindbyar med orkanstyrka på 32 meter per sekund, säger Sten Laurin vid SMHI.\n�\nSå... åk inte ut med bilen!Beställ bläck före 19 för leverans nästa vardag inkClub\n',headers='',source='Email',ip_address='***'] - Incorrect string value: '\xA0\x0AS\xC3\xA5....' for column 'message' at row 1
  • It seems, the issue is solved on my install now, I am not quite sure what I changed to make it work.

    I know I used part of the original functions and part of mmmichael's functions, so I will just upload the class.pop3.php for you to try for yourself if this helps.

    Summary of what I did:

    1) Change collation of entire DB to UTF8_bin
    2) Edit class.pop3.php, especially functions mime_encode() and getBody().
    I left the part that I replaced commented out.
    class.pop3.txt
    12K
  • mmmichael;4459 said:
    Hello,

    attached you can find my last Version of the class.pop3.php. Currently I have no further problems with popular charsets in germany. All "special chars" are transformed/converted correctly.

    Michael
    Hello,

    please consider, that this version was just for Testing. Please use the attached version (removes a security Problem because the previous version has no strip_tags at html Mails.

    Michael
    class.pop3.txt
    12K
  • This modified class.pop3.php-file seems to work fine, however I am seeing som problems when the file seemingly tries to convert the subject line to upper text? In the subject line norwegian ÆØÅ shows up correctly but æøå dont, and I noticed the strtoupper-function in the file. Is strtoupper nescessary?

    Regards
  • Great

    Yes, thank you :) Works fine for polish letters !
  • How can i get this to work on the newest version of OsTicket? There the class.pop3.php isnt in use.
  • Would be interested in a solution for the newer v1.6 RC5 version where the class.pop3.php is not in use any more?

    Clients get the Emails where special chars like ÄÖÜ are shown crypted like described in previous posts.

    Has anybody worked this out so far?
  • Hi Guys,

    i'm pretty sure that the following is not the best solution, but after trying for several hours now (too bad, i'm not a PHP programmer and i have to use a lot of try & error), i am happy with the second best solution now...

    First, open the include/class.mailfetch.php and find the following:
          
    //Generic decoder - mirrors imap_utf8
    function mime_decode($text) {

    $a = imap_mime_header_decode($text);
    $str = '';
    foreach ($a as $k => $part)
    $str.= $part->text;

    return $str?$str:imap_utf8($text);
    }


    Replace the Function with the following:

        
    //Generic decoder - mirrors imap_utf8
    function mime_decode($text) {

    $a = imap_utf8($text);
    $str = '';
    foreach ($a as $k => $part)
    $str.= $part->text;

    return $str?$str:imap_utf8($text);
    }


    That is solving the incoming mails, at least for me.

    Now for outgoing, go to include/class.email.php and look for the following (Line 135):
        
    function send($to,$subject,$message,$attachment=null) {
    global $cfg;

    //Get SMTP info IF enabled!
    $smtp=array();
    ...



    I've added 3 lines to that like followed (after the global $cfg definition):
        
    function send($to,$subject,$message,$attachment=null) {
    global $cfg;

    // the following 3 lines replace the german umlauts
    $umlaute = array('ä', 'ö', 'ü', 'ß', 'Ä', 'Ö', 'Ü');
    $ersetze = array('ae', 'oe', 'ue', 'ss', 'Ae', 'Oe', 'Ue');
    $subject = str_replace($umlaute, $ersetze, $subject);

    //Get SMTP info IF enabled!
    $smtp=array();
    ...


    This one converts all german umlauts in the Subject line with "ae", "oe" etc., this is not the best solution but it works. Even with someone answers to the ticket via email.

    Hopefully there will be a correct implementation of that whole thing in a upcoming version.
  • Thanks for Dominox.

    with russian letters its working fine but new trule was detected:

    if SUBJECT contain letters in more that 1 language, f.e. RU and EN, in control panel shown only russian (uppercase only). English letters is missing.

    Any ideas about it?
  • thanks to dominox.

    Problem was solved.
  • Small addition

    Thanks for your solution, dominox, it helped me with the incoming email. I have found that there is a proper solution for outgoing email.

    The problem turns out to be that the implementation of the PEAR mime-class in class.email.php is not containing the 'head_charset' variable, and thus the system falls back to the default value, which is ISO-8859-1. This is simply fixed by adding 'head_charset' => 'utf-8', in the list at lines 175-179 in class.email.php.

    The new list should looks like this:

    $options=array('head_encoding' => 'quoted-printable',
    'text_encoding' => 'quoted-printable',
    'html_encoding' => 'base64',
    'head_charset' => 'utf-8',
    'html_charset' => 'utf-8',
    'text_charset' => 'utf-8');

    I have reported this bug to the osTicket team.
  • Some versions of PHP have a bug where imap_utf8() returns the string in capital letters, I guess this is why the mime_decode function exists. I have changed it so that it actually work, with help from a comment on the php help page for imap_utf8().

    I have added a zip file with the changes needed for getting both incoming and outgoing email headers to work properly.
    osTicket-1.60-email_header_fix.zip
    9K
  • I found a very easy solution to this problem by simply setting AddDefaultCharset to utf-8 in my (Apache) virtual host config.

    The global setting in /etc/apache2/conf.d/charset on this particular server is AddDefaultCharset ISO-8859-1, so because I need that setting for other applications I only changed it in the virtual host scope.

    If you don't have access to your virtual host config you might still be able to set this via .htaccess if your hosting provider allows it.
  • I have tried dominox, tcvitan and openstream bugfixes, but without luck :(

    All incomming emails are truncated on the first german umlaut.

    What else can I check to get this working? Thanks in advance.
  • I have worked on that issue several days now. I have read a lot of posts, made some bug fixes, but it does not work at all. In some cases special chars (like german umlauts) work correct, but unfortunately not in all.


    I used a popular german email provider (web.de) to send the following Text:
    Umlaute Test: ä Ö ü und auch ß
    Sending Text-Email with special chars works perfekt. Received:
    Umlaute Test: ä Ö ü und auch ß
    Part of the email header:
    MIME-Version: 1.0
    Content-Type: text/plain; charset=UTF-8
    Content-Transfer-Encoding: quoted-printable
    Sending Design-Email with special chars does not work. Received:
    Umlaute Test:
    Part of the email header:
    MIME-Version: 1.0
    Content-Type: text/html; charset=UTF-8
    Content-Transfer-Encoding: 7bit
    Any idea where to fix this issue?
  • First of all, have you checked your mysql collation? The recommended setting should be selected to utf8_general_ci, I wouldn't change this to an alternative collation unless you really know what you're doing and are prepared to respond to some immediate collation issues. You may not get them depending on how the data was initially stored in mysql.

    Option 2, check encoding in the language pack, you will see a charset variable that posts to the HTML header, this should be set to UTF-8

    If you're still having difficulty, as a last resort and "quick fix" you can go hunting for each piece of code that outputs to a page and use utf8_encode() or utf8_decode() depending on your database collation and the input of the data to the database - this will take a lot of time and a lot of reading, it is definitely not recommended but is a last resort. Unfortunately with 1 field (subject) I have resorted to this for now to ensure our support site is professional to our customers while I find a better solution.

    I hope that helps.
  • Nachdem ich das letzte mal aufgegeben hatte, heute dann ein paar Stunden Debugging und nun geht es (ist allerdings spät geworden ;-()

    Alle Tipps aus dem Thread umgesetzt. Bei Html Emails mit Umlauten wurde aber immer noch die Meldung abgeschnitten.

    Letztendlich habe ich in der class.format.php dann noch den Fehler gefunden.

    Suche:
        function striptags($string) {
    return strip_tags(html_entity_decode($string)); //strip all tags ...no mercy!
    }


    Ersetze:
        function striptags($string) {
    return strip_tags(html_entity_decode($string, ENT_COMPAT, 'UTF-8')); //strip all tags ...no mercy!
    }


    siehe auch http://www.php.net/manual/de/function.html-entity-decode.php#102474

    Hoffe das spart jemanden einige Stunden/Tage.
  • I understand this is an old post.
    Currently the chinese text in subject line will be missing if someone used chinese.
    And dominox's solution fixed it.
    Thanks !

    [QUOTE=dominox;13595]Hi Guys,

    i'm pretty sure that the following is not the best solution, but after trying for several hours now (too bad, i'm not a PHP programmer and i have to use a lot of try & error), i am happy with the second best solution now...

    First, open the include/class.mailfetch.php and find the following:
          
    //Generic decoder - mirrors imap_utf8
    function mime_decode($text) {

    $a = imap_mime_header_decode($text);
    $str = '';
    foreach ($a as $k => $part)
    $str.= $part->text;

    return $str?$str:imap_utf8($text);
    }


    Replace the Function with the following:

        
    //Generic decoder - mirrors imap_utf8
    function mime_decode($text) {

    $a = imap_utf8($text);
    $str = '';
    foreach ($a as $k => $part)
    $str.= $part->text;

    return $str?$str:imap_utf8($text);
    }


    That is solving the incoming mails, at least for me.
Sign In or Register to comment.