Thursday 18 April 2013

Chrome Extension SWF Page Requests for Edgeworld Meldown value (1)

So the most difficult part of working with the Edgeworld API was the fact you needed the meltdown value to initiate the session with Edgeworld in order to access the data.  I learned that in addition to that you needed to have an active session with a particular sector in order to download data from that sector even if the meltdown value was the same.  Finally, even if you were actively using the meltdown value to download data it would eventually time out. So I set of on a path to try to capture this value the answer was Chrome Extensions.  Now I assume you can do the same thing with other browsers but all I will focus on right now is Google Chrome.

You can browse through the Google Chrome Extension library on your own but I will work you through
chrome.devtools.network.onRequestFinished.addListener()
for the purposes of this post.

This post will be discussed in 2 parts.  This first part will be setting up the files required.  In part two I will try to explain what is happening and then we will install the extension.

So what you will need for this tutorial is to create a new folder called EdgeworldExtension.  Within that folder create 4 text files called devtools.html, devtools.js, background.jsmanifest.json.

devtools.html

<html>
  <head>
    <script src="devtools.js"></script>
  </head>
  <body>
  </body>
</html>

The purpose of this file seems to be just to call the javascript file so I am unsure of why it is required but at this time it is required so just build it.

devtools.js

function sendNote(myValue) {
   chrome.runtime.sendMessage({greeting: myValue}, function(response) {
     console.log(response.farewell);
   });
}

chrome.devtools.network.onRequestFinished.addListener(
    function(mydata) {
  var requestURL = mydata.request.url;
  var myRequest = requestURL.toString();
  
  if(requestURL.indexOf("meltdown") >-1)
  {
   var start=requestURL.indexOf("meltdown") + 8;
   var meltdownParam = requestURL.substring(start,requestURL.length);
   var end=meltdownParam.indexOf("&");
   var meltdown = meltdownParam.substring(1,end);

   console.log("Meltdown found");
   console.log(meltdown);
   sendNote(meltdown);
   
  }
 });


This is a content script file it is the bulk of the code, however, not all API calls can be utilized in this context. This is why we need the background page below. If you get the error
Uncaught TypeError: Cannot call method 'set' of undefined
it is most likely because you are trying to make an API call that is only available in the background pages.

function setMyCookie(myValue) {
 chrome.cookies.set({ url: "http://gamerprogress.com/", name: "edgeworldMeltdown", value: myValue });
}

chrome.runtime.onMessage.addListener(
  function(request, sender, sendResponse) {
    console.log(request.greeting);
    sendResponse({farewell: "meltdown received - cookie set"});
 
 setMyCookie(request.greeting);
  });


This file is a background page that has access to the chrome.cookies.set that is not available in the content scripts as described above.


manifest.json


{
  "name": "EW Meltdown Grabber",
  "version": "1.0.2",
  "manifest_version": 2,
  "description": "Grabs the metldown value out of EW to be used in API calls.",
  "devtools_page": "devtools.html",
 "background": { "scripts": ["background.js"] },
 "permissions": [
    "tabs",
    "http://*/*",
    "https://*/*",
 "cookies"
  ]
}


This is the file that sets the permissions for this extension as well as the title and the description.

Wednesday 27 March 2013

Using C# to Get and Manipulate JSON

In my previous post I showed you how to see the JSON data that your browser was requesting from the Edgeworld server.  In this post I am going to show you how to mimic that request using C# and Visual Studio 2012.

When you look at the URL of the page request it has 2 basic parts
  • The root URL (http://sector15.c1.galactic.wonderhill.com/api/alliances/428295)
  • The URL parameters (_session_id, meltdown, reactor, user_id)
I do not know what the Edgeworld servers do with all of these parameters but I have discovered that the URL parameter of meltdown has a key value.  This value changes every time you log in to Edgeworld and it will time out once you log off.  This means that before you run a program to mimic the page request you need to log in to Edgeworld and use your developer tools to get this value and use it in your code.  The long number at the end of the string is the alliance ID.  So when you get this working for your alliance use your developer tools to find other alliance ID and track your enemies progress.

Step 1: Mimic the page request

The following code will mimic the page request.  I have created a few variables to make it easier to change out the meltdown value as mentioned above.

using System.IO;
using System.Net;

string fileContents;
string meltdown = "7463c45526b77360ffcb8d5e7f109c9d6a68048f";
string pagerequest = "http://sector15.c1.galactic.wonderhill.com/api/alliances/428295?%5Fsession%5Fid=48f319f393d1bca6ef3bed661658734a&meltdown=" + meltdown + "&reactor=4d545bca4d1d48b4d8f05685cc684a000849fb1c&user%5Fid=3608004";

WebRequest request = WebRequest.Create(@pagerequest);
WebResponse response = request.GetResponse();
Stream data = response.GetResponseStream();
string html = String.Empty;
using (StreamReader sr = new StreamReader(data))
{
    fileContents = sr.ReadToEnd();
}

For C# to use WebRequest you need to add using System.Net to the top of your program.  In the same way if you want to use Stream you need to add using System.IO.  This code will mimic the page request and save the contents of the request into a variable called fileContents.

Step 2: Parse the JSON data

Now that we have the JSON data saved into a variable it is time to parse through the data and convert it to a data structure that we can easily reference.

There are many different libraries and ways of doing this but the one I found the easiest to implement was JavaScriptSerializer.

using System.Web.Script.Serialization;

JavaScriptSerializer jsonSerializer = new JavaScriptSerializer();
PlayerData alliance = jsonSerializer.Deserialize<PlayerData>(fileContents);

To be able to use the JavaScriptSerializer you need to add a reference to your project.  To do this you need to go to Project > Add Reference. The reference that we are looking for is System.Web.Extensions and it can be found int Assemblies>Framework. Once the reference is added we need to add using System.Web.Script.Serialization to the top of our program.

The JSON data that comes from the web server is nested data that looks something like this:

{"response":
 {"timestamp":1364414863,"alliance":
  {"id":428295,"name":"ThisGameTooExpensive","description":"","members":
   [{"id":1828109,"name":"Door2Death","level":269}]
  }
 }
}

All the data is nested under response and, in this example, it has 2 variables (timestamp,alliance).  The alliance information is nested under alliance.  Then a variable under alliance is members which then nests all of the members of this particular alliance.  In order to parse through this we have to create a class to describe each nested level.  Below are four classes that parse out this data.

public class PlayerData
{
    public DataItem response { get; set; }
}
public class DataItem
{
    public int timestamp { get; set; }
    public Alliance alliance { get; set; }
    public List<Members> members { get; set; }
}
public class Alliance
{
    public int id { get; set; }
    public string name { get; set; }
}
public class Members
{
    public int id { get; set; }
    public string name { get; set; }
}

In the actual data that you will download through this method there will be more variables under each nested group.  You can simply add the variable names and the data will be added to your final data structure.

Step 3: Working with the new data structure

Now that the data is parsed into variables how do we work with it?  The advantage of Visual Studio in this case is that it will prompt you as you type out the variable structure starting with alliance once you have the above code in your program.

So if we wanted to print out the name of the alliance and then list all of the members and their respective levels to the console window we would use the code below:

Console.WriteLine(alliance.response.alliance.name);
Console.WriteLine();

for (int i = 0; i < alliance.response.members.Count;  i++)
{
    Console.WriteLine(alliance.response.members[i].id + " " + alliance.response.members[i].name);
}
Console.Read();

You can refer to a simple value like the alliance name because there is only one name or you can loop through and array of values like the list of members.  Once you understand these ideas you can then explore other things to do with the values.  Just displaying them in the console window isn't that helpful as you can see them in the alliance page.  However, if you were to save these values into an excel file and then save the values again in a week you can start tracking the progress of your alliance over time.

Thanks for reading. I have also demonstrated these ideas in the video below and the entire program is written out below the video.





using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net;
using System.Text;
using System.Threading.Tasks;
using System.Web.Script.Serialization;

namespace JSON_tutorial
{
    class Program
    {
        static void Main(string[] args)
        {
            string fileContents;
            string meltdown = "7463c45526b77360ffcb8d5e7f109c9d6a68048f";
            string pagerequest = "http://sector15.c1.galactic.wonderhill.com/api/alliances/428295?%5Fsession%5Fid=48f319f393d1bca6ef3bed661658734a&meltdown=" + meltdown + "&reactor=4d545bca4d1d48b4d8f05685cc684a000849fb1c&user%5Fid=3608004";

            WebRequest request = WebRequest.Create(@pagerequest);
            WebResponse response = request.GetResponse();
            Stream data = response.GetResponseStream();
            string html = String.Empty;
            using (StreamReader sr = new StreamReader(data))
            {
                fileContents = sr.ReadToEnd();
            }
            JavaScriptSerializer jsonSerializer = new JavaScriptSerializer();
            PlayerData alliance = jsonSerializer.Deserialize<PlayerData>(fileContents);

            Console.WriteLine(alliance.response.alliance.name);
            Console.WriteLine();
            for (int i = 0; i < alliance.response.members.Count;  i++)
            {
                Console.WriteLine(alliance.response.members[i].id + " " + alliance.response.members[i].name);
            }
            Console.Read();
        }
    }
    public class PlayerData
    {
        public DataItem response { get; set; }
    }
    public class DataItem
    {
        public int timestamp { get; set; }
        public Alliance alliance { get; set; }
        public List<Members> members { get; set; }
    }
    public class Alliance
    {
        public int id { get; set; }
        public string name { get; set; }
    }
    public class Members
    {
        public int id { get; set; }
        public string name { get; set; }
    }
}


Wednesday 20 March 2013

Edgeworld API

Edgeworld is multi-player on-line game that was originally played through Facebook and Google+.  It has since found a home on the developers website Kabam.com and is now played through a variety of platforms. One of the frustrations of the game is that they make it difficult to track your own progress or the progress of your alliance.  They have implemented a few alliance tracking tools but have since removed them from the game. I have been working to try and figure out the Edgeworld API so that I can download and store alliance data so that I can track my players level of activity.

I am a self taught techie so I may not have all the correct terms for the things that I have learned.  I would appreciate constructive criticism of what I am doing so that we can learn together.  Edgeworld doesn't publish an API but I am trying to figure out ways around it.  I will share the things that I have learned here.

Step1:

How can I get the data out of the game?

Edgeworld similar to many web based applications uses JSON formatted data. Below is a small example of this data.


You can get this data by examining how your browser makes requests.  Each browser does this a little bit different so I will try to explain how it works with each major browser. In all cases load up Edgeworld to the main base screen.

Google Chrome

Click on the 3 line icon on the far right side of the top of the screen and then select Tools > Developer Tools ( or use the F12 key).  This will open a new window, ensure "Network" is selected.  Now go back to your base in Edgeworld and select the Alliance button.  This sends a request to the server to get all the Alliance information.  Watch what happens in the Developer Tools window. You will see all the page requests that the browser is making.  Find the one that starts with a number and looks something like this "#####?%5Fsession%5fID=###" and click on that request. It should open up to show the details.  What you want to do at this stage is to copy the entire page request and then paste it into another tab in your browser.  The full page request URL should look something like this:

The pound signs will be numbers that correspond to your alliance ID and you userID. If you paste this into another browser window you should get something similar to the JSON code that I posted above.

FireFox

For Firefox I have downloaded the Firebug plugin which I would strongly recommend for anyone who is interested in this work. If you have the Firebug plugin installed then you can just press F12 to launch it. Select and enable the "Net" tab. Now go back to your base in Edgeworld and select the Alliance button.  This sends a request to the server to get all the Alliance information.  Watch what happens in the Firebug window. You will see all the page requests that the browser is making. Find the one that starts with a number and looks something like this "GET #####?%5Fsession%5fID=###" and right-click on that request and select."copy location".  Paste the location into another browser tab and you will see the JSON code similar to what is posted above.

Internet Explorer

Click on the Tools gear icon and select Developer Tools select "Network".  Click the Start Capturing button and then go back to your base in Edgeworld and select the Alliance button.  This sends a request to the server to get all the Alliance information.  Watch what happens in the Developer Tools window. You will see all the page requests that the browser is making. Find the one that starts with a number and looks something like this "GET #####?%5Fsession%5fID=###" and right-click on that request and select."copy location".  Paste the location into another browser tab and you will see the JSON code similar to what is posted above.


Depending on how the browser is set up it may display the JSON file or it might prompt you to download the text file.  Either way you will have learned how to download the data out of Edgeworld.

In my next post I will show you how to use C# to make the page request and save the data to your computer.  That will build us up to what can we do with this data now that we have it.

Thanks for reading.  I have also demonstrated these ideas in the video below.  Please leave comments tell me what you are interested in.