0 Replies Latest reply: Mar 2, 2012 9:47 AM by Michael Powe RSS

Determining size of dataset

Michael Powe Newbie



A big advantage to using ADO/ODBC to pull data is that you can determine up front how large the dataset is going to be.  With REST, you might make a "reasonable" request for data that turns out to be retrieving a million rows.  However, you won't know that until Excel crashes or your beard turns white waiting for the Web Query to complete.  From the data retrieval side, the data store is completely opaque.  I just had a case where a month of data for a report was around 44,000 rows and two months later, a month of data was a million rows.  (Yeah, who set that table limit to 'unlimited'?)  This is like walking into a door.


This is particularly significant when retrieving multidimensional datasets, because you can't filter those datasets on subdimensions.


I would like to suggest as a feature, a method for returning the table size or row count of the proposed data pull.  Beyond allowing me to retire with my sanity intact, this has useful functionality when determining how to process the data once it has been returned by the server. 


Of course, a search method that would search the entire record and not merely the first dimension, would also be extremely useful.