This morning, I got an email from one of the people I’m hosting a website for. They’re unable to create a newsletter.
So, their website is running WordPress, and they use a plugin to send newsletters. When trying to create a new newsletter, all it said was “Database error”.
I’ve located the failing code in includes/store.php. The failing code is
$r = $wpdb->insert($table, $data);
if ($r === false) {
$this->logger->fatal($wpdb->last_error);
die('Database error.');
}
There was no $wpdb->last_error set, so it’s not quite obvious what was causing the error. To find out what was going on, I had to dive into the WordPress $wpdb class. The insert() method calls the _insert_replace_helper() method. In this method, a call to the process_field() method is made. This call failed. To be specific, the failing code:
$converted_data = $this->strip_invalid_text( $data );
if ( $data !== $converted_data ) {
return false;
}
The strip_invalid_text() method was added in WordPress 4.2.0. This method strips invalid characters for a specified character set. So… this check was never performed with earlier versions of WordPress…. so… let’s have a look at the cause of the trouble.
The cause of the trouble, the newsletter template, or, more specifically, the template for the plain text version of this template. Rather then the default This email requires a modern e-mail reader. You can view the email online here, I’ve created a plain text output for this template. Part of this template was the following function:
function tse_newsletter_formattext($text){
$text = strip_tags($text);
$text = html_entity_decode($text);
$text = preg_replace('/&#(\d+);/me',"chr(\\1)",$text); #decimal notation
$text = preg_replace('/&#x([a-f0-9]+);/mei',"chr(0x\\1)",$text); #hex notation
$text = str_replace('…','...',$text);
$text = wordwrap($text,60,"\r\n",true);
return $text;
}
And this little function was the cause of the trouble. More specifically, the output of html_entitity_decode(), produced from the input . Character set issues, indeed. When called like this, running on PHP 5.3.3, it returns the decoded entities in ISO-8859-1 encoding. Specifically for this would result in 0xA0. The WordPress Database class however, was expecting UTF-8 encoding, and therefore considering this being invalid data. The decoding of html entities specifying codepoints (preg_replace lines) are probably also causing issues as they assume single-byte encodings.