Search This Blog

Monday, November 30, 2009

Yahoo News Search_Part2(FetcherThread)

In the last post we saw how layout file and activity class of our News search application look like; as you might remember our activity class uses a Handler to communicate with the other thread which is responsible to fetch the result of search using Yahoo WebServices. I used Yahoo Java SDK in this case,despite the fact that it has been designed and implemented for Java SE & EE users and you could argue that it's not suitable for Mobile environment which is a totally sensible critisim. however WE ARE JUST ANDROID STUDENTS, i mean we
should take it easy at this stage...
Any way you can see our FetcherThread Class below, when you start a FetcherThread it will wait untill startOperation() method is called, this method has a parameter which repesents search criteria, this method is called by our activity whenever user presses the search button.



package com.news.search;

import java.io.IOException;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.Map;
import javax.xml.parsers.ParserConfigurationException;
import javax.xml.parsers.SAXParser;
import javax.xml.parsers.SAXParserFactory;

import org.xml.sax.InputSource;
import org.xml.sax.SAXException;
import org.xml.sax.XMLReader;

import android.os.Bundle;
import android.os.Handler;
import android.os.Message;

import com.yahoo.rest.RestClient;
import com.yahoo.rest.RestException;
import com.yahoo.search.NewsSearchRequest;
import com.yahoo.search.NewsSearchResult;
import com.yahoo.xml.XmlParser;
import com.yahoo.search.xmlparser.XmlParserNewsSearchResults;

public class FetcherThread extends Thread {

public volatile XmlParserNewsSearchResults result = new XmlParserNewsSearchResults();
public volatile boolean lastRequest_finished = true;


private boolean anyRequest = false;
private boolean stopFlag = false;
private NewsSearchRequest request = new NewsSearchRequest();
private Handler callback;



public FetcherThread(Handler handler){
this.callback = handler;
}

public void run(){

request.getParameters().put("appid", "javasdktest");
// request.setResults(15);


do {
while(!this.anyRequest){

try{
synchronized (this) {
this.wait();
}
}catch(InterruptedException exp){
////Just Nothing
}

}

try{

Map results = executeAndParse(request.getRequestUrl(), request.getParameters());
Thread.sleep(4000); ///Just to simulate any possible real world delay
result.parse(results);

}catch(Exception exp){
exp.printStackTrace();

}


Bundle content = new Bundle();
content.putSerializable("result",result.listResults());
Message msg = new Message();
msg.setData(content);
this.callback.sendMessage(msg);

this.anyRequest = false;
this.lastRequest_finished = true;

}while(!this.stopFlag);

}

public synchronized void startOperation(String str){

this.request.setQuery(str);
this.anyRequest = true;
this.lastRequest_finished = false;
this.notifyAll();

}

public void cancelThread(){
this.stopFlag = true;
}


private Map executeAndParse(String serviceUrl, Map parameters) throws Exception {
XmlParser xmlParser = null;

try {
SAXParser parser = SAXParserFactory.newInstance().newSAXParser();
xmlParser = new XmlParser();

XMLReader reader = parser.getXMLReader();
reader.setContentHandler(xmlParser);
reader.parse(new InputSource(RestClient.call(serviceUrl, parameters)));

}
catch (ParserConfigurationException e) {
throw e;
}
catch (SAXException e) {
throw new SAXException("Error parsing XML response", e);
}
catch (RestException ye) {
throw ye;
}

return xmlParser.getRoot();
}


}



And before i forget it... we know our appilaction will be intracting with internet so we need to announce to the system that we're gonna use network and take the appropriate permission to do that, it's pretty easy, we just gotta add following tag to Android Manifest file :


<uses-permission android:name="android.permission.INTERNET" />


You can see here how this thread is iteracting with our activity using Handler, we send a request to this thread then it starts calling Web Service and extracts information, store them in an array and send the result array back to our activity using Handler.sendMessage() method, the message will be delivered to handleMessage() method when appropriate.
OK...it seems that everything is alright,especially because i tested almost the same code on Desktop Envirnoment and every thing looked pretty good, although i didn't need to do that since Yahoo SDK is tested regularly and there are heaps of people using it out there...
but if you run this application you're gonna be surprised since there will be a wierd bug there, the map that is returned by executeAndParse() method is always gonna be empty....YES...I KNOW.. Why? that one nearly killed me to figure it out and you wouldn't believe this...
let's see how it works...this is the RestClient.Call() method implementation extracted from Yahoo SDK :


public static InputStream call(String serviceUrl, Map parameters) throws IOException, RestException {
StringBuffer urlString = new StringBuffer(serviceUrl);
String query = RestClient.buildQueryString(parameters);

HttpURLConnection conn;
if((urlString.length() + query.length() + 1) > MAX_URI_LENGTH_FOR_GET) {
// Request is too big, do a POST
URL url = new URL(urlString.toString());
conn = (HttpURLConnection) url.openConnection();
conn.setRequestProperty("User-Agent", USER_AGENT_STRING);

conn.setRequestMethod("POST");
conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
conn.setDoOutput(true);
conn.getOutputStream().write(query.getBytes());
}
else {
// Request is small enough it should fit in a GET
if(query.length() > 0) {
urlString.append("?").append(query);
}

URL url = new URL(urlString.toString());
conn = (HttpURLConnection) url.openConnection();
conn.setRequestProperty("User-Agent", USER_AGENT_STRING);

conn.setRequestMethod("GET");
}


int responseCode = conn.getResponseCode();
if (HttpURLConnection.HTTP_OK != responseCode) {
ByteArrayOutputStream errorBuffer = new ByteArrayOutputStream();

int read;
byte[] readBuffer = new byte[ERROR_READ_BUFFER_SIZE];
InputStream errorStream = conn.getErrorStream();
while (-1 != (read = errorStream.read(readBuffer))) {
errorBuffer.write(readBuffer, 0, read);
}

throw new RestException("Request failed, HTTP " + responseCode + ": " + conn.getResponseMessage(), errorBuffer.toByteArray());
}

return conn.getInputStream();
}



an appropriate query is made in this method and Web Service get called, then an InputStream containing XML will be returned, then what we do with that inputStream is to parse the content using a SAX parser...
for SAX Parsing we usually need a Handler, we have used XmlParser class which is the default SAX handler of Yahoo SDK, here is implemtation of three main methods of XmlParser class which extracted from Yahoo SDK :


public void startElement(String uri, String localName, String qName, Attributes attributes) throws SAXException {
String parentPath;
if(typeStack.size() < 0) {
parentPath = (String) typeStack.peek();
}
else {
parentPath = "";
}
typeStack.push(parentPath + "/" + qName);

Map newMap = new HashMap();
for (int i = 0; i < attributes.getLength(); i++) {
newMap.put(attributes.getQName(i), attributes.getValue(i));
}

Map top = (Map) stack.peek();
Object obj = top.get(qName);
if (obj == null) {
top.put(qName, newMap);
}
else if (obj instanceof Map) {
List newList = new LinkedList();
newList.add(obj);
newList.add(newMap);
top.put(qName, newList);
}
else if (obj instanceof List) {
((List) obj).add(newMap);
}

stack.push(newMap);

}


public void endElement(String uri, String localName, String qName) throws SAXException {
stack.pop();
typeStack.pop();

}

public void characters(char ch[], int start, int length) throws SAXException {
String current = (String) ((Map) stack.peek()).get("value");
if(current != null) {
current += new String(ch, start, length);
}
else {
current = new String(ch, start, length);
}

((Map) stack.peek()).put("value", current);
}



(You might be thinking that those methods have been implemented a bit weird, yes
..because they have been implemented to handle all Yahoo Web Services and not
only News Web Service).
I bet you will be surprised if i tell you that problem is here. actually i had assumed if we have got Java SE APIs in android it would mean that you can expect them to work exactly like it does in Java SE, but apparently it is not correct, take a second look at startElement() method , we normally use qName to get tag names just like it's being used here, but the value of qName is an empty String in android environment!! all we need to do is to replace all accurances of qName with localName and it will be fine then, so our new implementation will be
like this:


public void startElement(String uri, String localName, String qName, Attributes attributes) throws SAXException {
String parentPath;
if(typeStack.size() < 0) {
parentPath = (String) typeStack.peek();
}
else {
parentPath = "";
}
typeStack.push(parentPath + "/" + localName);

Map newMap = new HashMap();
for (int i = 0; i < attributes.getLength(); i++) {
newMap.put(attributes.getLocalName(i), attributes.getValue(i));
}

Map top = (Map) stack.peek();
Object obj = top.get(localName);
if (obj == null) {
top.put(localName, newMap);
}
else if (obj instanceof Map) {
List newList = new LinkedList();
newList.add(obj);
newList.add(newMap);
top.put(localName, newList);
}
else if (obj instanceof List) {
((List) obj).add(newMap);
}

stack.push(newMap);

}



although it took a long time to me to find out what was wrong, it was worthwhile since i will be more carefull about how i think some java APIs should work from my exprience and how they actually work within android environment.
i mean it would not hurt to have a second look at API documentation before using them, even though you have had some exprience working with them before.
XML Parsing was not addressed in details in our application, obviously because i used Yahoo SDK to make things easier, but if you got interested in XML Parsing you can find a beautiful document here, besides i'm probabaly gonna change this application to see how we can use Pull Parser instead of SAX Parser.










No comments: