Data Extraction

3 Posts tagged with the services tag

Recently I have been working a lot with the Webtrends DX web services here internally.  Our consulting & services group has been working with various technologies to connect, retrieve data, and provide that data in informational reports to clients and partners.  In the course of working on these various projects I have added some functionality to the Webtrends DX web services SDK and also made some fairly solid in roads on the practices used to develop applications for Excel, Outlook, and other Microsoft Office Applications.  Here is a break down of the progress I have made over the last few weeks.  I hope it is helpful in everyone's efforts to extend, report, and provide analytics data & knowledge to their respective organizations!

 

Webtrends DX SDK - The SDK provides dramatically simplified standard programmatic access to reports, profile listings, report listings, and a mapper object to provide the reports in various object formats.  Recently I have added the ability for the mapper object to take a standard dimensional report object and provide the report as a standard ADO.NET DataTable.  This dramatically simplifies a lot of the functions of sorting, finding, or specifying various parts of the report data or manipulating it for presentation.  In addition this is a more familiar object format for ETL using SSIS or even other ETL tools  out on the market.  The following is an example of the code used to retrieve a report with the SDK and then do a conversion into a standard DataTable Object.

 

User webtrendsUser =
    new User
        {
            AccountName = Resources.Options.Default.Account,
            UserName = Resources.Options.Default.Password,
            Password = EncryptionHelper.Decrypt(Resources.Options.Default.Password)
        };
 
var report = ReportFactory.CreateDimensionalReport("reportIdGoesHere", "profileIdGoesHere", null, null, null,
                                                   null, null, true, webtrendsUser);
Mapper mapper = new Mapper(report);
DataTable dataTable = mapper.MapReportDefinitionToDataTable();

 

In the code above, I have created a Webtrends User object and assigned the appropriate parameters.  In this case I am using the Resources (Options.Settings) available to Windows Application to store and retrieve those settings.  This way the user will not have to enter them every time they want to run the report.

 

I then have the report object being pulled from the ReportFactory Object via the CreateDimensionalReport factory method.  The parameters that are actually needed are the report ID, the profile ID, and the Webtrends User Object.  The null values are other various parameters that can be set but do not particularly need to be set.

 

Then I create a Mapper Object and pass it the report object via the constructor.  This object requires a report object upon instantiation.  I setup the object this way since there always needs to be a report object before any manipulations or mappings can be made.

 

The last step is to declare the DataTable Object, then pull the data table from the builder method MapReportDefinitionToDataTable().

 

For the SDK assembly and code check out the Document Outlining the offering.  In addition over the next couple of weeks I will be posting more information regarding how to use the SDK, best practices, and code snippets for accessing the Webtrends DX Web Services.  There will be additional blog entries and definitely some documentation coming soon (I'm working on it!).  But in the meantime, if you have any questions, comments, or ideas on how we should extend or modify the SDK please let us know here on the Webtrends Developer Forums!

 

I'll have some more entries real soon about best practices, how to code against Outlook or Excel, and more.  So subscribe and stay up to speed on everything Webtrends!  Thanks for reading.

0 Comments Permalink

I did a blog entry a while back in regards to doing asynchronous web requests against REST based web services, which is what the Webtrends Web Services are built against.  This entry I'll show how to setup a synchronous web request for data against the services.

 

Benifits:

  • The request is made and in a top down fashion the response is returned with the respective string or stream object data needed.
  • Easier to test versus asynchronous web requests.

 

Disadvantages:

  • The request is made, and will pause or "freeze" while waiting for the response.  In a Windows Forms, WPF, Silverlight (or Flash, etc) client application this will cause the user interface to become unresponsive while waiting for the response.

 

So how do we make a synchronous call?  First create a HttpWebRequest object and instantiate it.

 

            var req = (HttpWebRequest)WebRequest.Create(@"https://ws.webtrends.com/beta/ReportService/profiles/?format=json");

 

Next you'll want to setup credentials.

 

            req.Credentials = new NetworkCredential(@"YourWebTrendsAccount\YouUserName", "YourPassword");

Now make sure when the request is made request the server to use compression.  This is entirely transparent and only needed at this single stage of the request.

 

            req.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip,deflate");

 

Finally set the method of request and other peripheral properties.

 

            req.Method = "Get";

            req.AllowAutoRedirect = true;

 

Once this is done you're ready to submit the request.

 

                WebResponse res = req.GetResponse();

 

To retrieve a simple string object feed through to the end of a the stream object response that is available via the WebResponse object.  This can also prospectively be done with the actualy HttpWebRequest object as it has a GetResponseStream() method.  I've segmented it here for example.

 

                string jsonResponse = (new StreamReader(res.GetResponseStream()).ReadToEnd());

 

The following is the complete code with exception handling.

 

            // Create a request

            var req = (HttpWebRequest)WebRequest.Create(@"https://ws.webtrends.com/beta/ReportService/profiles/?format=json");


            req.Credentials = new NetworkCredential(@"YourWebTrendsAccount\YouUserName", "YourPassword");

            // Add compression request.  IIS will return a compressed data result.

            req.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip,deflate");

            req.Method = "Get";

            req.AllowAutoRedirect = true;


            // Read the response

            try

            {

                WebResponse res = req.GetResponse();

                string jsonResponse = (new StreamReader(res.GetResponseStream()).ReadToEnd());


            }

            catch (WebException exw)

            {

                WebResponse response = exw.Response;

                string error = (new StreamReader(response.GetResponseStream())).ReadToEnd();

            }

 

Hope that helps!  Leave a comment or question if anything comes up while coding!

 

In a following entry the team will be providing more examples in consumption of these results programmaticaly, but as mentioned by Developer_Donut's entry we'll be releasing a software SDK soon.  This SDK will drammatically simplify and provide guidance on how to access and to utilize the web services.

0 Comments Permalink

The REST web services documentation has been updated.  New sections include:

 

  • How-to section: invoking with Ruby, getting report GUIDs
  • Best practices section: connecting asynchronously, compressing data

 

You can find the documentation from the Developer Site home page, REST API under the Resources area in the upper-right.

 

We'll be updating the documentation frequently, so please provide your feedback via the Developer Network.  Feel free to comment on this blog post or start a new discussion.

0 Comments Permalink