Sunday, 6 July 2014

Delphi REST client to retrieve flickr statistics

It's been a while since I last posted an article here, and today I'm coming with an exciting application I have been working on lately called "flickr photo stats". I'm a keen photographer and I like posting all my pictures in Flickr as it's a great platform for sharing and publishing your work. You can find my profile here:
What's most important is that Flickr provides an API which allows you to request information about your pictures. The Flickr API is available for non-commercial use by outside developers and we can request an API key for our particular use. To use the Flickr API, you need to have an application key that can be requested here: The App garden. As mentioned before this application key is for personal use and I had to request mine and explain the intentions for its use as I only wanted to get some values from my photos and keep track of them.


What I'm trying to achieve?

If I open one of the pictures, I can see that every picture contains 3 main values regarding statistics:
  - Views: Number of views that the picture receives.
  - Favourites: Number of likes that the picture receives.
  - Comments: Number of comments that the picture receives.
In previous versions of Flickr, they offered a professional profile in where users could see the global stats for the entire portfolio. Now with the new version, they only show the values for individual pictures. Many other platforms like ViewBug or 500px show the total number of views and other interesting information that could be useful when building your portfolio and study the data to see what people are more interested in seeing in your profile. I spent a while going through some forums in Flickr, trying to see if someone had created an additional service or application to display the information of the profile, but after weeks of research, nothing original came up.

Then I started doing my own research as to see how difficult or easy it would be to request the information from the API and when going through the definition of it I saw that they provide a REST request/response. A smile was drawn to my face :) when I immediately thought that I could build a REST client with Delphi and get the information I needed easily.

I'm on the phase of testing Delphi XE6 so I developed the application with it. :)


My solution

The requirement is to get as much information regarding the number of views, comments and likes from your portfolio. As the profile could have many pictures, this application only focuses in the photos you want to track e.g. the number of views overtime of one of your photos or counting the total views, likes and comments for the photos you add into the app list.

My current solution is really simple and effective after the results of my testing. Here you can see a screenshot of the application:
On the left, a list of photos containing the ID, title, number of views, likes, comments and the last time the application update the counters.
On the right a view of one of the pictures with the number of views (depending on the selection) over time. The application allows a batch update where it will request an update all the counters. Every day the system creates a new entry under a particular photo, so we can keep track of the overtime. Current example is just showing the data from the last 3 days.
The application stores all the details in an internal repository in XML format (see figure below) and it can be loaded/saved any time. The app is still under development so there could be bugs but it works fine.
The REST client uses a IdHTTP component with a IdSSLIOHandlerSocketOpenSSL handler as Flickr uses https
The main API methods used by the app are:
  • flickr.photos.getFavorites (Returns the list of people who have favorited a given photo)
    • Arguments:
      • api_key (Required) Your API application key.
      • photo_id (Required). The ID of the photo to fetch the favoriters list for
  • flickr.photos.getInfo (Get information about a photo. The calling user must have permission to view the photo)
    • Arguments:
      • api_key (Required) Your API application key.
      • photo_id (Required). The ID of the photo to fetch the favoriters list for
Example response:
<photo id="1253576" secret="81b96be690" server="1" farm="1" page="1" pages="3" perpage="10" total="1">
  <person nsid="33939862@N00" username="Dementation" favedate="1166689690" />
</photo>

Once we get the values from the REST response, we can use a XMLDocument component to read the values we need:
procedure TfrmFlickr.RequestInformation_REST_Flickr(id: string);
var
  Item, itemExisting: TListItem;
  response: string;
  iXMLRootNode, iXMLRootNode2, iXMLRootNode3, iXMLRootNode4: IXMLNode;
  views, title, likes, comments: string;
  stat: IStat;
  photo, existing: IPhoto;
begin
  response := IdHTTP1.Get(TFlickrRest.new().getInfo(apikey.text, id));
  XMLDocument1.LoadFromXML(response);
  iXMLRootNode := XMLDocument1.ChildNodes.first; 
  iXMLRootNode2 := iXMLRootNode.NextSibling; 
  iXMLRootNode3 := iXMLRootNode2.ChildNodes.first; 
  views := iXMLRootNode3.attributes['views'];
  iXMLRootNode4 := iXMLRootNode3.ChildNodes.first; 
  while iXMLRootNode4 <> nil do
  begin
    if iXMLRootNode4.NodeName = 'title' then
      title := iXMLRootNode4.NodeValue;
    if iXMLRootNode4.NodeName = 'comments' then
      comments := iXMLRootNode4.NodeValue;
    iXMLRootNode4 := iXMLRootNode4.NextSibling;
  end;

  response := IdHTTP1.Get(TFlickrRest.new().getFavorites(apikey.text, id));
  XMLDocument1.LoadFromXML(response);
  iXMLRootNode := XMLDocument1.ChildNodes.first; 
  iXMLRootNode2 := iXMLRootNode.NextSibling;
  iXMLRootNode3 := iXMLRootNode2.ChildNodes.first;
  likes := iXMLRootNode3.attributes['total'];
end;


How to start 

Once you have your application key sorted out, you can check that your API key works by running the test echo that Flickr service API provides:

An example response is as follows:

<rsp stat="ok">
  <method>flickr.test.echo</method>
  <api_key>9e727ec8b24c10431e696a26d7191800</api_key>
  <format>rest</format>
  <auth_token>72157645540005814-7ab42556416acf10</auth_token>
  <api_sig>f92d4327bd54c967db3cd2a41ce8a718</api_sig>
</rsp>

From here we will use the api_key value and place it into the apy key textbox in the app:

To know what's your photo_id, you just need to open your picture on Flickr and you will get the ID on the URL:

Once we have the photo_id we can add it into the list and the application will add a new entry. If we click on the checkbox associated to the entry, the graph on the right will be displayed with the values (current and stored). The graph is dynamically build so you can add/remove as many as you want. A selection of views/likes/comments is possible under the graph so every time a selection is made, the graph is generated displaying one of those selected values.
At the end of the main list there is a label that tallies up all the values from the list which gives you a great indication about how your pictures are doing.

Source code and binaries can be found here:
Enjoy the reading and do not hesitate to add your comments and thoughts about it. There are still many things to fix and enhance but at least I can get some meaningful values and track items easily without having to explore each single picture on Flickr.
A better example is shown in the figure below after adding 275 photos into the app:
I can easily see that those pictures have reached more than 200k views in total which gives me a good indication of the amount of visits the profile gets. As I run the app every day (to keep track of everything), I can see that the number of views are going up.

Update 20/07/2014.
A second version of the up is ready and it just enhances the way information is displayed:


New version here:

Jordi

Sunday, 3 November 2013

My first Android App with Delphi XE5

A lot has been going on since Embarcadero released Delphi XE4. I must say I was astonished with what they achieved in such a short time with that version. Just being able to develop your own mobile applications using Delphi was something amazing. Even though I found that version quite cumbersome as it was not trivial the way to deploy your app into an iOS system I gave it a try with no much success (I needed a virtual machine for the Mac device and had to borrow someone's iPad to publish the app, but I'm not a Mac person). So, what I was really waiting for was their latest release Delphi XE5 with Android compatibility. The most interesting part: how simple it is to create a project and deploy it into your android device. In less than a minute you can have your "hello world!" app in your phone with just few clicks. 
Delphi XE5 comes with the Android SDK 22.0.1 32 bit (at least the one that I got when installing Delphi) and you can see all the configuration paths for that particular SDK under Tools -> SDK Manager:

The first step is to enable your device to accept USB Debugging. To achieve this, you can follow the steps from the following link which related the way you have to enable developer mode for Samsung Galaxy Note 3. I have got a Samsung Galaxy Mega, so the same rule applies:

Once done, you will see new options in your phone:

Enable "Still awake" and "USB debugging". Once those are enabled, plug your phone into the computer using the USB cable. Then, the phone will ask you if you want to allow USB debugging: Accept it and let's go to Delphi UI.

Now in Delphi XE5, I will create a very simple project. So, go to File -> New -> FireMonkey Mobile Application.

As my new device is not in the list of available devices, I just choose something that more or less has the same dimensions. In this case as my phablet is a 6.3 inches, I choose the WSVGA 7" option:

Once this is done, I just want to create a simple "hello world" app which consists of displaying a label and when a button is pressed displaying a message. A very simple application but without changing the way I think (I'm building a Delphi application).
unit Unit1;

interface

uses
  System.SysUtils, System.Types, System.UITypes, System.Classes, System.Variants,
  FMX.Types, FMX.Controls, FMX.Forms, FMX.Graphics, FMX.Dialogs, FMX.StdCtrls;

type
  TForm1 = class(TForm)
    Label1: TLabel;
    Button1: TButton;
    procedure Button1Click(Sender: TObject);
  private
    { Private declarations }
  public
    { Public declarations }
  end;

var
  Form1: TForm1;

implementation

{$R *.fmx}

procedure TForm1.Button1Click(Sender: TObject);
begin
  showMessage('Delphi XE5 Rocks');
end;

As you can see, I'm coding in Delphi. Once this is done, If I have followed the previous steps correctly, my phone should appear under targets for that particular SDK:
Now I just need to Save my project and Run it so it can be automatically deployed in my phone. Once done, I'll be able to see the app in my phone and perform the operations it was designed to do:
Here it is. As you can see really simple steps and with a great outcome: A Delphi App in your most loved android device. Now it's time to explore and do some more research about what you can do here.
Jordi

Related links:

Sunday, 24 February 2013

Using Delphi code coverage

I couldn't find a good self-explanatory example for the usage of Delphi-code-coverage tool so here you will find a simple example from top to bottom. The idea is really simple; I have a set of units which I want to know whether they are covered by my unit tests or not. Even though there is a lot of information about the project, I couldn't find a really good example explaining all the details about How to properly set up the project, folders, flags, etc. I have used the Delphi-code-coverage-wizard which is great to generate the initial structure (batch file + list of paths + units) but I went manual from there as I couldn't get any output.
I must say that both projects are really good and that's why I'm giving them an opportunity as I want to test the coverage of my projects under Delphi XE.
First step is to correctly identify where your source code is and your unit tests are. This step is really important as to correctly set-up the code coverage project. All my testing code is under my repository and available here: Delphi-repositoty. For this example I'm using a very simple example where I have a unique unit test which tries to cover two units (thundax.AbstractFactory.pas and thundax.Prototype.pas).

My unit test project is called thundax.DesignPatternsTest.exe which has 2 tests:
Now I want to know if those two unit tests cover my previously defined units (thundax.AbstractFactory.pas and thundax.Prototype.pas).

Download Delphi-Code-Coverage and leave it under your bin\ folder or where your test.exe application is (in my case thundax.DesignPatternsTest.exe).
Create two text files called dcov_paths.lst and dcov_units.lst. You can use Delphi-code-coverage-wizard to set up the initial structure, but I would rather go manual as I have more control of the execution.

- The first file (dcov_paths.lst) will contain the path to your source code:
C:\ThundaxDelphiRepository\thundax-delphi-personal-repository\src\Design Patterns\

- The second file (dcov_units.lst) will contain the list of unit tests you want to check whether they cover your code or not.
thundax.Prototype
thundax.AbstractFactory

Your projects needs to have debug information generating a detailed Map file:

Now we can create a small batch file (dcov_execute.bat) or run the following command from the command line tool:
CodeCoverage.exe -e "thundax.DesignPatternsTest.exe" -m "thundax.DesignPatternsTest.map" -ife -uf dcov_units.lst -spf dcov_paths.lst -od ".\" -lt -html -xml -emma -meta

This is the configuration that works for me, where:
  • -e identifies the executable to run (in this case the Unit test framework).
  • -m identifies the map file (the map file contains all the debugging symbols).
  • -ife include file extension (if you are using generics.collections use this option).
  • -uf list of your units (source code, not your unit tests code).
  • -spf list of source directories.
  • -od Output directory (".\" gets  the current).
  • -lt generates log file. (Delphi-Code-Coverage-Debug.txt).
  • -html Html output
  • -xml Xml output
  • -emma EMMA output
  • -meta Generate separate meta and coverage files when generating emma output - 'coverage.em' and 'coverage.ec' will be generated for meta data and coverage data. (Needs -emma as well). These are the important files to run EMMA.
Once the line has been run, you should get the correct output files:
  • CodeCoverage_summary.html (Html output)
  • CodeCoverage_summary.xml (xml output)
  • coverage.es (EMMA output)
  • coverage.em (EMMA output)
  • coverage.ec (EMMA output)
  • Delphi-Code-Coverage-Debug.txt 
  • thundax.AbstractFactory(thundax.AbstractFactory.pas).html (Your report)
  • thundax.Prototype(thundax.Prototype.pas).html (Your report)

If you open the summary you will see the total of lines and %age covered by your test:

And you can inspect the files by doing click on the links:
As you can see, one of the units is only 69% covered. Once we inspect the file we can see that there are lines (highlighted in blue) which are not covered by my unit tests and the green ones which are.

If you want to use EMMA, is as easy as this (once the code coverage tool has outputted the .ec and .em files):

java -cp emma.jar emma report -r html -in coverage.em,coverage.ec

Emma component will generate a coverage folder with the following html file:

The entire example can be found in my Delphi repository.

I hope you find this useful and start using it.
Jordi