Data Extraction

March 2009 Previous month Next month

Smart Measures

Posted by Rob Derstadt Mar 26, 2009

       Reports that contain "smart measures" will show more columns available in the report definition than are really valid for an actual request. This is an important distinction. Reports that use smart measures (which are typically only those with visitor counts, buyers or recent campaigns) will show that there are 5 visitor measures available (one each for day, week, month, etc). However, when they request report data they will only get ONE of those back, based on the period they provide. For example, if they specify period=2009m01, they will only get the smart measure for monthly visitor count.This makes sense since a monthly count would not be valid for a daily period. 

If you're in the WebTrends UI you can find a report GUID (for use in building a rest query) by looking at the javascript tag information that displays when you hover over a report in the left-nav. In this case, I am hovering over the link to the report "Same Visit Campaign Names". The browser will display the javascript that is executed when this link is clicked in the lower left-hand corner of the window (circled below). The tag displayed contains "dynamictables<table GUID><Chapter>" You can view the properties of this link to pull this text out and then copy only the GUID portion, which is simply the text between "dynamictables" and "Chapter". In this case it is "aK1I9lMxvL5". This comes in handy if you're viewing a report and want to quickly pull up the data using the web services.

 

dx.bmp

 

 

 

dx2.bmp

As many of you know, awhile back the folks over at Codeplex developed a WT reporting data as a widget on their site...it's a good example of leveraging and sharing analytics data:

codeplexwebtrendswidget.png

 

They develop some cool tools at Codeplex, including a  JSON viewer that's simple but effective.  This is a standalone tool, and a visualizer inside Visual Studio, and they went the extra mile and modified the viewer to work inside Fiddler too.  Bonus points for that move.  Here's how it looks inside Fiddler:

codeplexfiddler.png

Here is a quick example of using Ruby to call WebTrends Web Services:

 

  def query(account, username, password, query)
   
     hostname = "ws.webtrends.com"
    
     https =  Net::HTTP.new(hostname, Net::HTTP.https_default_port)
     https.use_ssl = true
     https.ca_file = "/usr/share/curl/curl-ca-bundle.crt"


     resp = https.start { |http|
      
       request = Net::HTTP::Get.new("/beta/ReportService.svc/" + query)
     
       request.basic_auth account + '\\' + username, password
           
        res = http.request(request)

 

        case res

 

        when Net::HTTPSuccess, Net::HTTPRedirection
         
          return  res.body
       
        else
       
          res.error!
       
        end
       
      }
      
  end

 

The method call would look something like this:

 

@foo = query("My Account", "My User Name", "My Password", "profile/?format=json")

Today I'm going to cover material around connecting to WebTrends new REST based Web Services.  To start off, let's cover the context behind the methods of making calls to these services.

 

There are two specific ways in which to connect to REST based Web Services, asynchronously and synchronously.  Depending on the type of application, tool, or other mechanism you are connecting to the services with you will want to use one or the other or might be using one or the other.

 

When making an asynchronous call against a service the application, or to be specific the calling thread, will initiate the call but continue executing without waiting for a response.  When the response is received an event fires and the data returned can then be handled in some way.  For client service applications, server based applications, or anything that needs to return control to the application or UI thread, this is the method to use.  This provides a smoother and more continuous application flow when an end user is accessing services.

 

A synchronous call against a service executes on the actual application thread itself, and holding that thread until a response returns.  This can make a UI appear to hang or freeze.  Often this is only used if there is no way to return data via a post event.  One scenario that would need to do this would be to render a page, or display, in its entirety before returning it to the user.  This is a common scenario on the web.  AJAX solves this to some degree, but often one still needs to render the entire initial page, and thus, must wait for the data to return.

 

Best Practice

 

The best practice is to use asynchronous calls.  Only use synchronous calls for stand alone processing or non-user viewed feeds.

 

A Simple Example

 

Here is a C# asynchronous call being made against our WebTrends Web Services.  Keep in mind, I'm providing these as examples, I do NOT suggest writing tests with Thread.Wait calls in them.

 

The example also shows how to setup your WebTrends Credentials for authenticating to our WebTrends Web Services.

 

    [TestClass]
    public class GetRestServices
    {
        private XDocument doc;

 

        private void svc_DownloadStringCompleted(object sender, DownloadStringCompletedEventArgs e)
        {
            doc = XDocument.Parse(e.Result);
        }

 

        [TestMethod]
        public void TestAsynchronousCallWithSecurity()
        {
            const string baseUri = "https://ws.webtrends.com/beta/reportservice/profile/?format=xml";

 

            var svc = new WebClient();
            svc.Credentials = new NetworkCredential("yourWebTrendsAccount\WebTrendsUserName", "yourSuperSecretPassword");
            svc.DownloadStringCompleted += svc_DownloadStringCompleted;
            svc.DownloadStringAsync(new Uri(baseUri));

 

            Thread.Sleep(3000);
            Assert.IsNotNull(doc);
            var docTest = new XDocument();
            Assert.IsInstanceOfType(doc, docTest.GetType());
        }
    }

 

Hope those are helpful. 

Web Services & REST

Posted by Adron Hall Mar 11, 2009

This is my hello world blog post for the Developer Network.  Over time I will be posting many blog posts, primarily focused on development of and around the Web Services & using REST Web Servies in general.  One of the first topics I'll cover is using authenticating programmatically to the WebTrends Web Services.  I'll follow up that post in short order with more REST specific and data specific pieces.  The primary language I'll be using for examples is C#, from different perspectives in the .NET Platform.  With this post though, I'm not going to touch on the services technical aspects, but instead on the context and reason for their existence.

 

REST Services in the Web

 

REST stands for Representational State Transfer.  REST services are really more than mere Web Services, as inherent to the acronym.  REST started as a disertation by Roy Fielding as pointed out in the Wikipedia Article on REST Services.  Give that a read, as I'm going to skip ahead and cover some of the other context points of REST.

 

Even with the massive push from enterprises (and a few large software companies) for the SOAP Web Services, the REST Web Services have really taken over the web.  A lot of this has to do with the simple, straight forward, and less overhead involved with REST based transfers.  As the web has grown, SOAP became less viable and REST increased in viability.  In this course of events we decided to go with a REST based services offering.

 

One of the main cocepts around REST is that of resources.  Almost everything with the REST ideal is embodied around a global identifier, which is a URI, pointing to a particular resource.  These resources are then manipulated by standard HTTP interfaces.  In our case, we provide primarily resources at this point in read only XML, JSON, and Excel friendly XML, without manipulation.  In the future we'll exand to provide other capabilities to manipulate these resources.

 

Some of the key elements of REST have enabled us to move quickly and efficiently to build out the needed infrastructure.  In addition several of the key elements enable even further scalability and capabilities.  Some of these are the abstraction of application state and functionality into resources.  This absolves the application developer of the complexity of SOAP Web Services while increasing server response time & load.  Another great benifit of the REST approach is that it depends less on vendor software and specific technology stacks, in other words .NET, Java, PHP, Ruby on Rails, Adobe Flex or anything else can be used to connect without needing or utilizing any dependent stack components.

 

What I've covered here are a few quick points on REST services, links for more information, and a few basic reasons why we've chosen REST for these service offerings.  Keep reading and we'll soon have more information on authenticating, retrieveing, and other programmatic use of the WebTrends Web Services.

We will keep this blog post updated with the latest known issues for our DX API. Please check here first, before submitting a bug to Webtrends if you are having problem.

 

 

The beta API was last updated on May 21st, 2009. Enhancements include:

 

  • Calculated measures will no longer display in report definitions. Currently, these are only available through the Analytics user interface.
  • The report "category" has been added to the report listing. This will make it easier to obtain a list of reports matching a specific report "category". If a report has no defined category the attribute will not be displayed.
  • In some cases a report was enabled for trending, but the report meta-data did not accurately reflect this. There was a bug in the logic that determined if trending was enabled. This has been resolved.
  • The ability to filter based of a discrete set of measure identifiers was not working. This was a bug and has been resolved.
  • The profile summary report, when returned in Excel format, contained invalid XML what was preventing it from rendering properly. This has been resolved.

 

Beta API Known Issues

 

  • Totals will not be displayed in report data. Client applications can easily aggregate their own data or compute new measures (i.e. a "calculated measure").
  • Dimension attributes for multi-dimensional reports are not currently supported.These will be added in a future release.
  • Some dimension "names" do not match those shown in the UI. This is a translation problem and will also be addressed in a future release.
  • Smart Measures, if available, will show up in report definitions. However, the actual measures displayed is dependent upon the time period specified when performaing a query. If you are requesting specific measures you need to be sure to include the correct smart measure for the time period desired. For example, if you are specifying a daily time period (e.g. 2008m02d24) then you should only specify the smart measure that applies to a daily time period. If not, an error will result.
  • Trending performance is "slow". In some cases it may be possible to improve trending by performing individual requests and calculating the trend on the client.

    In order to leverage the new Webtrends REST web services, you need to configure a user in your Webtrends account properly.  We describe this in our Getting Started document.  I wanted to add a little more information for your reference.

     

    Any user can be setup to use the Web Services functionality.  If you are using the Web Services for pulling data into Excel for a static or one-time report, it's fine to use your existing user account.

     

    However, if you are planning on leveraging the Web Services data on a regular basis, or via an automated program or application, you'll be much better off by creating a new user account for that purpose.

     

    For example, you might define a new user called "WSUser" that you only plan on using for pulling Web Services data.  This is the best practice we've adopted within WebTrends for our internal consumption of WebTrends data.

     

    Questions?  Comments?

    Filter Blog

    By date:
    By tag: