Monkey Metrics.

Monkey Archive Forums/Monkey Discussion/Monkey Metrics.

Paul - Taiphoz(Posted 2014) [#1]



AdamRedwoods(Posted 2014) [#2]
cool.
can you break the posts down into topics? maybe find out what the most commonly asked beginner question is? (my guess is it's multi-dimensional arrays)


Paul - Taiphoz(Posted 2014) [#3]
Nah I just parsed the user profile page's and gathered and totaled all the stats, I could count number of users who have no posts, or no stars or no apps, you know find the slackers lol..

I wanted to parse it and generate a pie chart to show the distribution of users per country but the country is stored as a number and I dont have access to the enum list for it so no idea what each country should be.

The code is rough and kinda slow given what it's doing, and the escape for finding the end of the users is a search for error which the page displays but it seems some users have been deleted causing this to show prematurely so I really need a better catch for that, it's a shame we don't have threading it would really speed this up.

Ideally I would have liked to have done this and made it a html5 app that displays various metrics but html5 is not currently supported by the stream module.

Still interesting numbers tho :)

Oh yeah I was planning to make a User class so I could deal with the data in a more efficient and manageable manner but never got round to it, to impatient and wanted to see if it worked so just threw in some prints.


Import mojo
Import brl.httprequest
Import brl.tcpstream

Const MONKEY_HOST:String	= "www.monkey-x.com"
Const MONKEY_PORT:Int		= 80
Const MONKEY_PHP:String	= "/Account/showuser.php"


'Global UserList:list<profile> = New List<profile>
Global TotalUsers:Int=0
Global TotalPosts:Int=0
Global TotalStars:Int=0
Global TotalApps:Int=0
Global TotalAppsPlayable:Int=0
Global Complete:Bool=False
Global UserID:Int =1


Global Timer:Int=Millisecs()
Global Delay:Int = 200

Function Main()
	New ParseApp
End Function

Class ParseApp Extends App

	Method OnCreate()
		SetUpdateRate(60)
	End Method
	
	Method OnUpdate:Int()



		If Millisecs() > Timer+Delay And Complete=False
		
			Self.ReadHtml(String(UserID))
			UserID+=1					
			Timer=Millisecs()
			
		Endif
		
		If Complete = True And UserID>2118 'added a rough check seems user 700 got deleted and it produces and error escaping the search.
			
			Print("Total Posts :"+TotalPosts)
			Print("Total Stars :"+TotalStars)
			Print("Total Apps :"+TotalApps)
			Print("Total Play Apps :"+TotalAppsPlayable)
			Print("Parsing User :"+UserID)
			
			Repeat
				If KeyHit(KEY_SPACE) Then Error("")
			Forever
			
			Error("Done")
		Endif
		
		Return 0
	End
	
	Method OnRender:Int()
		Cls
		
			DrawText("Total Posts :"+TotalPosts,10,10)
			DrawText("Total Stars :"+TotalStars,10,30)
			DrawText("Total Apps :"+TotalApps,10,50)
			DrawText("Total Play Apps :"+TotalAppsPlayable,10,70)
			DrawText("Parsing User :"+UserID,10,90)
		
		Return 0
	End
	
	Method ReadHtml:TcpStream( opt:String )
		Local ReaderStream:TcpStream = New TcpStream
	
		If Not ReaderStream.Connect( MONKEY_HOST, MONKEY_PORT )
			Return Null
		Endif
	
		ReaderStream.WriteLine "GET "+MONKEY_PHP+"?id="+opt+" HTTP/1.0"
		ReaderStream.WriteLine "HOST:"+MONKEY_HOST
		ReaderStream.WriteLine ""

		Local line:String
		Repeat
			line = ReaderStream.ReadLine()

#rem
				<dt>Country</dt>
				<dd>178</dd> ''' emum ??.

#end
			
				If line.Find("<dt>Posts:</dt>")<>-1
					line = ReaderStream.ReadLine()
					line=line.Trim()
					line=line[4..line.Length()-5]
					TotalPosts+=Int(line)
				Endif

				If line.Find("<dt>Country</dt>")<>-1
					line = ReaderStream.ReadLine()
					line=line.Trim()
					line=line[4..line.Length()-5]
					'TotalStars+=line
				Endif	

				If line.Find("<dt>Stars:</dt>")<>-1
					line = ReaderStream.ReadLine()
					line=line.Trim()
					line=line[4..line.Length()-5]
					TotalStars+=Int(line)
				Endif	

				If line.Find(" >Apps:</a></dt>")<>-1
					line = ReaderStream.ReadLine()
					line=line.Trim()
					line=line[4..line.Length()-5]
					TotalApps+=Int(line)
				Endif	
				
				If line.Find(">Playable Apps:</a></dt>")<>-1
					line = ReaderStream.ReadLine()
					line=line.Trim()
					line=line[4..line.Length()-5]
					TotalAppsPlayable+=Int(line)
					line="</html>" 'get out of the current loop we dont need the rest of the html.
				Endif					
				
				If line.Find("<h4>Error</h4>")<>-1 Then
					line="</html>"
					Complete=True
				Endif	
			'Print ">"+line
		Until line="</html>"

		
		Return ReaderStream
	End
		
End Class



Class profile
	Field country:String
	Field posts:Int
	Field stars:Int
	Field apps:Int
	Field playableApps:Int
	
	Method New()
		UserList.AddLast(Self)
	End Method
	
End Class



Gerry Quinn(Posted 2014) [#4]
There are a lot more actual apps than those listed here though.


Paul - Taiphoz(Posted 2014) [#5]
355 unless the parser some how missed some users out, exactly how is the number not accurate ? were you adding the two app counts together ?


mjcamerer(Posted 2014) [#6]
I think he means some people may have released apps but not listed them here. I know I have 6 apps released, but I have only added 1 here on this site.


Gerry Quinn(Posted 2014) [#7]
Ditto, have released about 6 but only added 2 or 3.


Paul - Taiphoz(Posted 2014) [#8]
well yeah clearly it cant parse peoples hard drives or app stores lol, so of course its only counting this sites stats :D