encoding problematic chars in rss feed?

Member Posts : 11 Join date : 2011-04-17

Hi All

im creating an rss feed (or should i say trying to create) but the problem is that it wont validate the data is being pulled from an api and then inserted into my db and from there im pulling it out to create the feed.

here is the validation result

the problematic urls are displayed like so
/cread.php?v=1270&t=79524&p=http://www.toysrus.co.uk/index.jsf?fh_eds=%EF%BF%BD&fh_location=//brus/en_GB/categories%3C{brus_brus}/itemtype=product/categories%3C{brus_brus_babiesrus_brutravelpushchairs}/categories%3C{brus_brus_babiesrus_brutravelpushchairs_brutravelpushchairstravelsy

its the %EF%BF%BD which is the problem and appears as a black diamond with a question mark inside it when i hover over the text link?

ive tried adding htmlspecialchars/decode to the link ive also tried adding mysql_query("SET NAMES 'utf8'"); top my script but nothing seems to sort this out.

here is the code to insert the data into my db

Code:: <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="robots" content="noindex">
<title>Kernow Connect - Discount Updater.</title>
<?php

$user_name = "";
$password = "";
$database = "";
$server = "";

$db_handle = mysql_connect($server, $user_name, $password);
$db_found = mysql_select_db($database, $db_handle);
mysql_query("SET NAMES 'utf8'");

define('API', 'PS');
require_once('../shopwindow.client/constants.inc.php');
require_once('../shopwindow.client/classes/class.ClientFactory.php');
date_default_timezone_set('Europe/London');
$date = date("Y.m.d.G.i");
echo "<h1>$date</h1>";

?>
</head>

<body>

<?php
mysql_query("TRUNCATE TABLE `tbl_codes`") or die(mysql_error());

$storeID = array(9,65,157,172,282,323,380,427,483,547,550,686,782,854,896,911,980,1043);

$oClient = ClientFactory::getClient();
$merchantcolumns = array(sStrapline,sDescription,sLogoUrl,sDisplayUrl,sClickThroughUrl,oDiscountCode);
$multiplemerchantwithcodes = array("sColumnToReturn" => $merchantcolumns, "iAdult" => 1);
$oResponse= $oClient->call('getMerchant', $multiplemerchantwithcodes);

foreach($oResponse->oMerchant as $details)
{
$iId = $details->iId;
$iId2 = "AW".$iId;
$smsName =$details->sName;
$sStrapline = $details->sStrapline;
$sDescription = $details->sDescription;
$sLogoUrl = $details->sLogoUrl;
$sDisplayUrl = $details->sDisplayUrl;
$sClickThroughUrl = $details->sClickThroughUrl;

#for merchants with multi codes
if(in_array($iId, $storeID))
{
$arraycheck1 = $details->oDiscountCode;

if(!empty($arraycheck1) and is_array($arraycheck1))
{
echo '<p><h3>Vouchers for '.$smsName.'</h3>';

foreach($details->oDiscountCode as $key=>$details2)
{
$sCode = "";
$sCode = $details2->sCode;
$sDescription = $details2->sDescription;
$sDescription = str_replace("'","",$sDescription);
$sDescription = str_replace("´","",$sDescription);
$sUrl = htmlspecialchars_decode($details2->sUrl);
$sEndDate = $details2->sEndDate;

echo "<p><a href='$sUrl'>".$sCode."<br />".$sDescription."</a><br /></p>";

if ($sUrl == '')
{
$sUrl = $sClickThroughUrl;
}

#if(!mysql_query("SELECT * FROM tbl_codes WHERE codeID = '$sCode'"))
#{
$insert_codes = "INSERT INTO
tbl_codes (code, codeLink, storeID, details, datestamp, expireDate)
VALUES ('$sCode', '$sUrl', '$iId2', '$sDescription', '$date', '$sEndDate')";
#print $insert_codes;
$sql = mysql_query($insert_codes)or die(mysql_error());
#}
}
}

#for merchants with single codes
elseif(!empty($arraycheck1) and !is_array($arraycheck1))
{
echo '<p><h3>Vouchers for '.$smsName.'</h3>';

$sCode = $details->oDiscountCode->sCode;
$sDescription = $details->oDiscountCode->sDescription;
$sDescription = str_replace("'","",$sDescription);
$sDescription = str_replace("´","",$sDescription);
$sUrl = htmlspecialchars_decode($details->oDiscountCode->sUrl);
$sEndDate = $details2->sEndDate;

echo "<p>".$sCode."<br />".$sDescription."<br /></p>";

if ($sUrl==''){
$sUrl = $sClickThroughUrl;
}
#if(!mysql_query("SELECT * FROM tbl_codes WHERE codeID = '$sCode'"))
#{
$insert_codes = "INSERT INTO
tbl_codes (code, codeLink, storeID, details, datestamp, expireDate)
VALUES ('$sCode', '$sUrl', '$iId2', '$sDescription', '$date', '$sEndDate')";
#print $insert_codes;
$sql = mysql_query($insert_codes)or die(mysql_error());
#}
}

}}
?>
</body>
</html>

its the $sUrl which is the problem

then the code to create my rss is

Code:: <?php header("Content-Type: application/xml"); ?>
<?php echo '<?xml version="1.0" encoding="UTF-8"?>';?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" >
<channel>
<title>Kernow Connect Discount Codes</title>
<description>The Latest Discount Codes From Your Favourite Retail Stores</description>
<link>http://www.kernow-connect.com/discount.php</link>
<atom:link href="http://www.kernow-connect.com/rss_feeds/discount_rss.php" rel="self" type="application/rss+xml" />
<?php
include_once("../dbinfo.php");
mysql_query("SET NAMES 'utf8'");

$create_feeds = "SELECT * FROM tbl_codes ORDER BY expireDate ASC";
$query = mysql_query($create_feeds)or die(mysql_error());
while($row = mysql_fetch_array($query))
{
date_default_timezone_set('GMT');
#$description = htmlentities($row['details']);
$description = htmlspecialchars($row['details']);

$getstorename = "SELECT storeName FROM stores WHERE storeID = '".$row['storeID']."'";
$sql = mysql_query($getstorename) or die (mysql_error());
$store = mysql_fetch_array($sql);
$name = $store['storeName'];
$link = $store['storeLink'];?>
   <item>
   <title><?php echo $row['code']?></title>
   <description><?php echo $description." from $name";?></description>
   <pubDate><?php echo date("D, d M Y H:i:s T", strtotime($row['datestamp']))?></pubDate>
   <link><?php echo htmlspecialchars($row['codeLink'])?></link>
   </item>
   <?php

}?>

</channel>
</rss>

any help would be great
thanks

Member Posts : 5 Join date : 2011-04-14

Krushh wrote:

Hi All

im creating an rss feed (or should i say trying to create) but the problem is that it wont validate the data is being pulled from an api and then inserted into my db and from there im pulling it out to create the feed.

here is the validation result

the problematic urls are displayed like so
/cread.php?v=1270&t=79524&p=http://www.toysrus.co.uk/index.jsf?fh_eds=%EF%BF%BD&fh_location=//brus/en_GB/categories%3C{brus_brus}/itemtype=product/categories%3C{brus_brus_babiesrus_brutravelpushchairs}/categories%3C{brus_brus_babiesrus_brutravelpushchairs_brutravelpushchairstravelsy

its the %EF%BF%BD which is the problem and appears as a black diamond with a question mark inside it when i hover over the text link?

ive tried adding htmlspecialchars/decode to the link ive also tried adding mysql_query("SET NAMES 'utf8'"); top my script but nothing seems to sort this out.

here is the code to insert the data into my db

Code:: <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="robots" content="noindex">
<title>Kernow Connect - Discount Updater.</title>
<?php

$user_name = "";
$password = "";
$database = "";
$server = "";

$db_handle = mysql_connect($server, $user_name, $password);
$db_found = mysql_select_db($database, $db_handle);
mysql_query("SET NAMES 'utf8'");

define('API', 'PS');
require_once('../shopwindow.client/constants.inc.php');
require_once('../shopwindow.client/classes/class.ClientFactory.php');
date_default_timezone_set('Europe/London');
$date = date("Y.m.d.G.i");
echo "<h1>$date</h1>";

?>
</head>

<body>

<?php
mysql_query("TRUNCATE TABLE `tbl_codes`") or die(mysql_error());

$storeID = array(9,65,157,172,282,323,380,427,483,547,550,686,782,854,896,911,980,1043);

$oClient = ClientFactory::getClient();
$merchantcolumns = array(sStrapline,sDescription,sLogoUrl,sDisplayUrl,sClickThroughUrl,oDiscountCode);
$multiplemerchantwithcodes = array("sColumnToReturn" => $merchantcolumns, "iAdult" => 1);
$oResponse= $oClient->call('getMerchant', $multiplemerchantwithcodes);

foreach($oResponse->oMerchant as $details)
{
$iId = $details->iId;
$iId2 = "AW".$iId;
$smsName =$details->sName;
$sStrapline = $details->sStrapline;
$sDescription = $details->sDescription;
$sLogoUrl = $details->sLogoUrl;
$sDisplayUrl = $details->sDisplayUrl;
$sClickThroughUrl = $details->sClickThroughUrl;

#for merchants with multi codes
if(in_array($iId, $storeID))
{
$arraycheck1 = $details->oDiscountCode;

if(!empty($arraycheck1) and is_array($arraycheck1))
{
echo '<p><h3>Vouchers for '.$smsName.'</h3>';

foreach($details->oDiscountCode as $key=>$details2)
{
$sCode = "";
$sCode = $details2->sCode;
$sDescription = $details2->sDescription;
$sDescription = str_replace("'","",$sDescription);
$sDescription = str_replace("´","",$sDescription);
$sUrl = htmlspecialchars_decode($details2->sUrl);
$sEndDate = $details2->sEndDate;

echo "<p><a href='$sUrl'>".$sCode."
".$sDescription."</a>
</p>";

if ($sUrl == '')
{
$sUrl = $sClickThroughUrl;
}

#if(!mysql_query("SELECT * FROM tbl_codes WHERE codeID = '$sCode'"))
#{
$insert_codes = "INSERT INTO
tbl_codes (code, codeLink, storeID, details, datestamp, expireDate)
VALUES ('$sCode', '$sUrl', '$iId2', '$sDescription', '$date', '$sEndDate')";
#print $insert_codes;
$sql = mysql_query($insert_codes)or die(mysql_error());
#}
}
}

#for merchants with single codes
elseif(!empty($arraycheck1) and !is_array($arraycheck1))
{
echo '<p><h3>Vouchers for '.$smsName.'</h3>';

$sCode = $details->oDiscountCode->sCode;
$sDescription = $details->oDiscountCode->sDescription;
$sDescription = str_replace("'","",$sDescription);
$sDescription = str_replace("´","",$sDescription);
$sUrl = htmlspecialchars_decode($details->oDiscountCode->sUrl);
$sEndDate = $details2->sEndDate;

echo "<p>".$sCode."
".$sDescription."
</p>";

if ($sUrl==''){
$sUrl = $sClickThroughUrl;
}
#if(!mysql_query("SELECT * FROM tbl_codes WHERE codeID = '$sCode'"))
#{
$insert_codes = "INSERT INTO
tbl_codes (code, codeLink, storeID, details, datestamp, expireDate)
VALUES ('$sCode', '$sUrl', '$iId2', '$sDescription', '$date', '$sEndDate')";
#print $insert_codes;
$sql = mysql_query($insert_codes)or die(mysql_error());
#}
}

}}
?>
</body>
</html>

its the $sUrl which is the problem

then the code to create my rss is

Code:: <?php header("Content-Type: application/xml"); ?>
<?php echo '<?xml version="1.0" encoding="UTF-8"?>';?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" >
<channel>
<title>Kernow Connect Discount Codes</title>
<description>The Latest Discount Codes From Your Favourite Retail Stores</description>
<link>http://www.kernow-connect.com/discount.php</link>
<atom:link href="http://www.kernow-connect.com/rss_feeds/discount_rss.php" rel="self" type="application/rss+xml" />
<?php
include_once("../dbinfo.php");
mysql_query("SET NAMES 'utf8'");

$create_feeds = "SELECT * FROM tbl_codes ORDER BY expireDate ASC";
$query = mysql_query($create_feeds)or die(mysql_error());
while($row = mysql_fetch_array($query))
{
date_default_timezone_set('GMT');
#$description = htmlentities($row['details']);
$description = htmlspecialchars($row['details']);

$getstorename = "SELECT storeName FROM stores WHERE storeID = '".$row['storeID']."'";
$sql = mysql_query($getstorename) or die (mysql_error());
$store = mysql_fetch_array($sql);
$name = $store['storeName'];
$link = $store['storeLink'];?>
   <item>
   <title><?php echo $row['code']?></title>
   <description><?php echo $description." from $name";?></description>
   <pubDate><?php echo date("D, d M Y H:i:s T", strtotime($row['datestamp']))?></pubDate>
   <link><?php echo htmlspecialchars($row['codeLink'])?></link>
   </item>
   <?php

}?>

</channel>
</rss>

any help would be great
thanks

try also this:

http://www.php.net/manual/en/function.urldecode.php
http://www.php.net/manual/en/function.urlencode.php
http://www.php.net/manual/en/function.rawurldecode.php
http://www.php.net/manual/en/function.rawurlencode.php

best regards

Member Posts : 11 Join date : 2011-04-17

thanks for the reply mate,

unfortunately none of them solve the problem

Member Posts : 5 Join date : 2011-04-14

Krushh wrote:: thanks for the reply mate,

unfortunately none of them solve the problem

can you post what you tried?

probably this part is relvant:

Code:: <link><?php echo htmlspecialchars($row['codeLink'])?></link>

and change it like this:

Code:: <?php
$alink = htmlspecialchars($row['codeLink']);
print $row['codeLink'] . "<br/>";
print $alink . "<br/>";
?>
<link><?php echo $alink; ?></link>

best regards

Member Posts : 11 Join date : 2011-04-17

hi mate,

ok i was applying the urlencode's to the script which updated the db and not the one to create the feed

ive added the above to the rss creation code like so

Code:: <?php header("Content-Type: application/xml"); ?>
<?php echo '<?xml version="1.0" encoding="UTF-8"?>';?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" >
<channel>
<title>Kernow Connect Discount Codes</title>
<description>The Latest Discount Codes From Your Favourite Retail Stores</description>
<link>http://www.kernow-connect.com/discount.php</link>
<atom:link href="http://www.kernow-connect.com/rss_feeds/discount_rss.php" rel="self" type="application/rss+xml" />
<?php
include_once("../dbinfo.php");
mysql_query("SET NAMES 'utf8'");

$create_feeds = "SELECT * FROM tbl_codes ORDER BY expireDate ASC";
$query = mysql_query($create_feeds)or die(mysql_error());
while($row = mysql_fetch_array($query))
{
date_default_timezone_set('GMT');
#$description = htmlentities($row['details']);
$description = htmlspecialchars($row['details']);

$getstorename = "SELECT storeName FROM stores WHERE storeID = '".$row['storeID']."'";
$sql = mysql_query($getstorename) or die (mysql_error());
$store = mysql_fetch_array($sql);
$name = $store['storeName'];
$link = $store['storeLink'];
$alink = htmlspecialchars($row['codeLink']);
print $row['codeLink'] . "<br/>";
print $alink . "<br/>";?>
<item>
<title><?php echo $row['code']?></title>
<description><?php echo $description." from $name";?></description>
<pubDate><?php echo date("D, d M Y H:i:s T", strtotime($row['datestamp']))?></pubDate>
<link><?php echo $alink?></link>
</item>
<?php
}?>

and it doesent like the prints.

but the print statments are are printing out (for one of the problematic urls)

Code:: http://www.awin1.com/cread.php?v=1270&t=79524&p=http://www.toysrus.co.uk/index.jsf?fh_eds=%EF%BF%BD&fh_location=//brus/en_GB/categories%3C{brus_brus}/itemtype=product/categories%3C{brus_brus_babiesrus_brufeeding}/categories%3C{brus_brus_babiesrus_brufeeding_brufeedinghighchairsboosters}&fh_start_index=

Code:: http://www.awin1.com/cread.php?v=1270&t=79524&p=http://www.toysrus.co.uk/index.jsf?fh_eds=%EF%BF%BD&fh_location=//brus/en_GB/categories%3C{brus_brus}/itemtype=product/categories%3C{brus_brus_babiesrus_brufeeding}/categories%3C{brus_brus_babiesrus_brufeeding_brufeedinghighchairsboosters}&fh_start_index=

and after removing the print commands the feed still doesnt validate but the diamond with the question mark is replaced with the "%EF%BF%BD"

cheers

Member Posts : 5 Join date : 2011-04-14

replace this line:

Code:: $alink = htmlspecialchars($row['codeLink']);

with this one:

Code:: $alink = rowurldecode($row['codeLink']);

and check again,

best regards

Member Posts : 11 Join date : 2011-04-17

i assume u meant rawurldecode?

that returns
XML parsing error: <unknown>:11:46: not well-formed (invalid token)

on the first item

cheers