Idea

spring-it avatar image
spring-it Posted ·

Call Log totalPages Element Not Present

I am attempting to pull all of our existing call-logs for a day, but we have more than 1000 logs that I need to pull. I have been using the totalPages element to iterate through all the pages. All other calls that I have used have this element (extensions, phone-number, presence, message-store, etc.). However, it appears that the call-log request does not return a totalPages element, making it difficult to accomplish this. I could continue to get the next page until I get no records, but this would require an extra call each time, and simply is not as clean of a soluton. This would also be inconsistent to what I am currently doing for all other calls.

Could you please let me know if I am missing something, or if there is a reason why this element does not exist for this call? This is very frustrating to run into seemingly random inconsistencies in your API.

Thanks for your time.
topic-default
1 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Tyler Liu avatar image
Tyler Liu Posted ·
Confirmed and I agree it's inconsistent. We will switch this topic type to be an "idea" instead of a problem so it can act as a suggestion for product management.

Workarounds:

1. check 'navigation.nextPage' element. If it doesn't exist, you don't need to issue an extra call
2. specify 'perPage' query parameter to a large number. Then you might be able to get all the call logs with merely one call.
1 comment
1 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Thanks for the reply!

It sounds like I will need to use the nextPage element, as all calls do not contain the totalPages/totalElements elements, and this wont be added to those that don't contain them from Anton's reply below. This is unfortunate, but a reality. Your second solution may work for some, but assuming that you'll never have more than 1000 records returned does not sound like a good idea.

Anyway, thanks again!
0 Likes 0 · ·
anton-nikitin avatar image
anton-nikitin Posted ·
It is not a bug or something we wish to fix going forward. totalPages and totalElements cannot be calculated efficiently for call logs since it will require fetching potentially unlimited number of elements in advance (just to calculate count) and has dramatic impact on server performance. So clients should work with dataset only using prev/next navigation which is pretty standard way to iterate through large datasets. 

I do not fully understand the issue. If a client needs to get all records, it will need to iterate anyway through all pages and the number of requests cannot be optimized. "perPage" parameter can be set to "max" value to work with maximum allowed page size.
3 comments
1 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Thanks for the reply!

I see what you're saying. However, while I do not understand how your data is stored on the back end, in any traditional RDBMS (SQL Server, MySQL, etc.) getting a count of all elements requires substantially less processing then retrieving all of the records. If your data is stored in this way, it should be really easy, and fast to get a count of all of the records. Also, there should always be a date range for the logs that are being requested, and even if there was no date, I believe that you do not keep call logs indefinitely, so there will always be a cutoff for the amount of call logs available to pull. So saying that there is a potential to fetch an unlimited number of records is not right.

As far as I can tell, the max number of elements allowed on one page is 1000. This seems like a lot, but we have more than 1000 messages for any given location/day on a very regular basis. 

The reason that I chose to use the totalPages element to iterate through all pages was because I assumed (falsely) that all requests/results would be consistent, thus returning the totalPages for any and all calls. Secondly, using a specific number for iterations is a more concrete, predictable way to get all record. Checking for the nextPage element each and every time just requires a bit more processing.
0 Likes 0 · ·
Actually, checking for nextPage element shouldn't be any more expensive.

Here is a recursive example using the JS SDK to fetch all call logs. It expects some environment variables to be set (which are prefixed with process.env.[ENV_VAR_NAME].

    function getCallLogsPage() {
                   
        return platform
            .get('/account/~/call-log', {
                page: page,
                perPage: process.env.LOGS_PER_PAGE, // MAX is 1000
                dateFrom: process.env.DATE_FROM,
                dateTo: process.env.DATE_TO,
                view: process.env.VIEW
            })  
            .then(function (response) {
                console.log(response.json());
                // Sanity check
                if( response ) { 
                    console.log("************** SHOULD BE A SUCCESSFUL 200 REQUEST - NOTHING TO SEE HERE...MOVE ALONG PLEASE ***************");
                    if(timeoutObject) {
                        clearTimeout(timeoutObject);
                    }   
                    retryAfter = 0;
                    var data = response.json();
                    //console.log("************** THE NUMBER OF CALL LOGS THIS REQUEST CYCLE ARE : ***************", data.records.length);
                    callLogs = callLogs.concat(data.records);
                    if (data.navigation.nextPage) {
                        page++;
                        return getCallLogsPage(); // this will be chained
                    } else {
                        return callLogs; // this is the finally resolved thing
                    }
                }
            });
    } // end of getCallLogsPage()
    /*
     Loop until you capture all extensions, and then continue processing data
     */
    return getCallLogsPage()
    .then(<NEXT_FUNCTION_IN_PROGRAM>)
    .catch(function (e) {
            console.error(e);
            throw e;
        }
    });
0 Likes 0 · ·
Thanks for your reply. I have my app updated to use the nextPage element. Thanks!
0 Likes 0 · ·
spring-it avatar image
spring-it Posted ·
I do believe that the API should be updated to return the totalPages/totalElements element for the call_logs, and any other calls that may be missing this. It does not makes sense to omit this from the one call, creating a discrepancy and preventing your users from being able to consistently use your API. It will only continue to cause your paying customers headache.

However, I do appreciate the quick reply and information.
1 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Joe Ramos avatar image
Joe Ramos Posted ·

I know this has been a while, but as I read through the above. I find it very out of the ordinary to not include totalPages. I will have to agree that having the date range as well as the fact that data has a cutoff is interesting. We too have a large regular call volume. Using the next page from the example is more costly in the use of it. Eitherway, I am glad I read this as I was trying to figure out why it was broken.

1 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Write a Comment

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Your Opinion Counts

Share your great idea, or help out by voting for other people's ideas.