Visualize Tweets on a Map Using Java and Gephi

8 comments

Some people are asking me how to visualize tweets related to a given topic on a map. In this tutorial you will learn how to do it using Java (to collect the tweets) and Gephi (to visualize them).

Phase 1 - Collecting the tweets


Here I will show you a sample code based on the Twitter4J libray (here more details on how to use the library) to collect geolocated tweets from the Twitter API. If you want to use another language to collect the tweets, or you already have a dataset of tweets to visualize, skip to the next phase. The important thing is that the collected tweets has latitude and longitude information.

The following code shows how to collect geolocated tweets (in this case related to the topic "Obama").


 int LIMIT= 5000; //the number of retrieved tweets
ConfigurationBuilder cb;
Twitter twitter;
cb = new ConfigurationBuilder();
cb.setOAuthConsumerKey("***");
cb.setOAuthConsumerSecret("***");
 cb.setOAuthAccessToken("***");
cb.setOAuthAccessTokenSecret("***");
twitter = new TwitterFactory(cb.build()).getInstance();
 Query query = new Query("Obama");
query.setCount(100);

 try {
int count=0;
QueryResult r;
 
  do {
r = twitter.search(query);
ArrayList ts= (ArrayList) r.getTweets();
 

  for (int i = 0; i < ts.size() && count < LIMIT; i++) {

   Status tweet = ts.get(i);

    if (tweet.getGeoLocation() != null) {
count++;
     
     //add to a csv
     //for example three attributes:
     //tweet.getId()
     //tweet.getGeoLocation().getLatitude();
     //tweet.getGeoLocation().getLongitude();
 }
}
} while ((query = r.nextQuery()) != null && count < LIMIT);
}
catch (TwitterException te) {
System.out.println("Couldn't connect: " + te);
}
}


After this phase you should have a .csv file like this:


Phase 2 - Visualizing the tweets



To visualize the tweets you will need Gephi (a graph visualization tool available here).

  1. Open Gephi.
  2. Download the GeoLayout plugin: Tools -> Plugins -> Available plugins -> Select "GeoLayout"  -> Install. Then restart Gephi.
  3. Open a new project: File -> New Project
  4. Open the data table: Window -> Data Table
  5. Click on Import Spreadsheet and choose the separator you used and "Nodes Table" (See Figure).

  6. Click on "Next" and choose "Double" as the type of the "latitude" and "longitude" attributes and "String" for the ID.
  7. Go back to the "Overview" tab, you should see your graph which is still not laid out (See Figure).
  8. On the left, in the "Layout" tab select "Geo Layout" and the Latitude and Longitude attributes. Then click on Run. You should have obtained something like this:

  9. In the Data Laboratory you can set the size and color of all the nodes to obtain your desired result. Then you can switch to the "Preview" view and set the opacity of the nodes and the background of the canvas, obtaining the following result:

8 comments :

  1. i read this tutorial from facebook and give it a try, but i can't seems make it work. i use twitter4j-3.0.3 and processing 1.5.1.

    i already put twitter4j(core) file in a folder named 'code', run the code and i got

    ...cannot find anything named "status"...

    if you would mind giving me insight what might i miss in the process.

    thanks before

    ReplyDelete
    Replies
    1. That's because I decided to call it "tweet" instead of "status" to make it more clear but then I forgot to modify one assignment. Now I fixed the code, tell me if you are able to run it :)

      Delete
    2. sorry for late reply, just read it. thanks for the response

      Delete
  2. This comment has been removed by the author.

    ReplyDelete
    Replies
    1. You have to call the two columns containing coordinates "lat" and "lon", try and tell me if this works :)

      Delete
    2. Sorry, I deleted my comment just before you responded! I realised my mistake - I had one column of combined coordinates, not two! Sorry.

      I do have another wee problem. I can now see the dots on the map - but how did you get the image of the world in the background?

      Delete
    3. There is no map in the background, but since I plotted many points the boundaries are evident. If you want to put a map in the background you can export your graph as a .svg using Gephi and then use a software like Adobe Illustrator or inkscape to add the map.

      Delete
    4. i keep getting expecting EOF, found } ...but i can't seen to figure out where the missing bracket is! i just copied the above code for now, my twitter keys are all placed correctly as well.

      Delete