Email archiving with special characters (smileys) fails

[cperrot]

Issue

A email containing a smiley hex combination (\xF0\x9F\x98\x8A\x0D\x0A)
😊 fails with Internal Server Error 500

MySQL error 1366: Incorrect string value: '\xF0\x9F\x98\x8A\x0D\x0A...' for column 'description' at row 1

Expected Behavior

Not failing

Actual Behavior

Failing

Possible Fix

Either sanity check the content or delete non compatible symbols

Steps to Reproduce

1. Send yourself a Smiley
2. Archive the email to contact

Context

Logs

Your Environment

• Outlook Plugin Version used: 3.0.25.2
• Outlook Client Version used: 2016
• SuiteCRM Version used: 7.11.1
• Host (e.g. PC, Server): Ubuntu 18.04

[cperrot]

This looks promising
https://stackoverflow.com/questions/20411440/incorrect-string-value-xf0-x9f-x8e-xb6-xf0-x9f-mysql

[simon-brooke]

Can you check your version number, please? We have not yet issued 3.0.25.2, we're only up to 3.0.18.0. Obviously if you're on your own branch you're on your own, but I have seen this bug recently and if my memory serves we've fixed it.

[cperrot]

I pulled the latest version from the master branch and compiled it myself just to make sure the problem wasn't fixed in the latest master version.

[simon-brooke]

Fair enough. So the version number when pulled was 3.0.18.0? Or 3.0.17.0?

The number auto-increments each time you do a release build, so that probably accounts for the difference.

I know we've seen this recently, but I can't currently find it in my notes. From memory, please check whether your MySQL database is configured to accept UTF.

[cperrot]

Yes, my database is configured to use utf8-general-ci. I pulled the version 3.0.18.
I even tried to convert the descrition field in contacts and emails_text into utf8mb4.
It worked for the contact.description field but the email_text.description field still threw an error.

[simon-brooke]

OK. I know we've had that one, and I thought I had it fixed, but obviously not. I'll put it on the list of things to do. If you could put a sample of an emoticon that you know for certain doesn't save correctly for you into this thread, I'll check that I can reproduce your error.

…

On Tue, 26 Feb 2019 at 10:10, cperrot ***@***.***> wrote: Yes, my database is configured to use utf8-general-ci. I pulled the version 3.0.18. I even tried to convert the descrition field in contacts and emails_text into utf8mb4. It worked for the contact.description field but the email_text.description field still threw an error. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#97 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFPvW_KgmUDYK4Al3b8HWeXMf4lPHAtqks5vRQgLgaJpZM4bN-FO> .

-- Simon Brooke :: http://www.journeyman.cc/~simon/ ;; Stultus in monte

[simon-brooke]

Scratch that, I see you've already given me one. I'll test it and let you know.

[simon-brooke]

OK, can confirm I can reproduce your error. It's on my TODO list, but I can't promise when.

[cperrot]

Do you have an ETA on this issue?

[sasha2002]

salesagility/SuiteCRM#6447
The problem persist and no ETA as i see, an is Low priority.....
SugarCRM v9 fixed problem with smile.

[sasha2002]

Nearly a year and nothing .....
Can someone from SalesAgility say when it will be fixed?
Or maybe when fix from SugarCRM can be imported in suiteCRM ?