libcurl and httpHeaders

BlitzMax Forums/Brucey's Modules/libcurl and httpHeaders

siread

(Posted 2011) [#1]

I am trying to check whether a name exists in my player database using libcurl to connect to a php page. This works fine until I throw some foreign characters into the mix. Both the names Simon Read and José Hernández are in the database but José is not found.

I believe the problem lies in setting the charset to utf-8 but I'm struggling to work out how to do this. If I uncomment the httpHeader line it causing problems posting the form. What am I doing wrong?

Import bah.libcurl

CheckPlayerExists("Simon Read")
CheckPlayerExists("José Hernández")

Function CheckPlayerExists:Int(name:String)	
	Local url:String = "http://newstarsoccer.newstargames.com/gamephp/checkexists.php"
	
	Local form:TCurlFormData = TCurlFormData.Create()
	form.addContent("name", name)

	Local curl:TCurlEasy = TCurlEasy.Create()
'	curl.httpHeader(["Content-type: text/html; charset=utf-8"])
	curl.setWriteString()
	curl.setOptInt(CURLOPT_CONNECTTIMEOUT, 4)
	curl.setOptInt(CURLOPT_TIMEOUT, 4)
	curl.setOptString(CURLOPT_URL, url)
	curl.httpPost(form)
	
	Local res:Int = curl.perform()
	
	If res
		Print CurlError(res)
		curl.Cleanup()
		Return -1
	Else
		Local str:String = curl.toString()
		curl.Cleanup()
		
		Print str
		
		If str.Contains("playerexists")
			Return 1
		ElseIf str.Contains("playerdoesnotexist")
			Return 0
		EndIf
	End If
	
	Return -1
End Function

Last edited 2011

SLotman

(Posted 2011) [#2]

Maybe you should use URL escape codes making it something like:

CheckPlayerExists("Jos%E9%20Hern%E1ndez")

Just guessing...

Edit: if that helps out, check this page for all info on URL encoding

Last edited 2011

siread

(Posted 2011) [#3]

That didn't work either for some reason. Seems php urldecodes things different to how I encoded it. This php function seems to do the trick though:

function fixEncoding($in_str) 
	{ 
		if(mb_detect_encoding($in_str) == "UTF-8" && mb_check_encoding($in_str,"UTF-8")) 
			return $in_str; 
		else 
			return utf8_encode($in_str); 
	}

Brucey

(Posted 2011) [#4]

If you are going to use UTF8 encoding, you need to ensure the string you are sending is UTF8 encoded. The module only does a basic CString conversion.

You should be able to do something like :

Local b:Byte Ptr = name.ToUTF8String()
name = String.FromCString(b)
MemFree(b)

form.addContent("name", name)

I wonder if it's worth adding a flag for UTF8 support which will do UTF8 string conversions automagically when it is set...