Concept

Performance analytics information

When the performance analytics is configured within your application, you can execute one or multiple webservice calls to your application. During these calls, the webservice will automatically create an Excel sheet within the configured FileName location (see Calculation). The different calls to your application and the formulas that are executed, will be saved within this Excel sheet. After an application reset, a new Excel sheet will be created and it will start again with call 1. Make sure to create a backup of the Excel sheet before resetting the application, if you wish to review this Excel sheet later.

Within the created Excel sheet, different information about your application and formulas is available. This Excel sheet contains the following information:

• Item ID; This column contains the ID of the Futurama formula.
• callnr; This is the number of the request that is send to the webservice. After the webservice is started, the PerformanceAnalytics will start with callnr 1 for the first request, then use callnr 2 for the second request, etc. With this value the performance for 1 specific call can be viewed. After an application reset, the performance analysis will again start with callnr 1.
• # of Calls; Within this column the total calls to this formula is shown.
• # of Calls from cache; When a formula is called multiple times, some of these calls will be used from the cache instead of executing this formula. Within this column the number of calls that are used from the cache is shown. The number of calls above minus the number of calls from the cache is the number of times the formula is calculated.
• # ticks excluding children; This column contains the time per formula. To improve the performance, the formulas with the highest number of ticks are the most interesting. A tick is the smallest unit of time recognized by a device. In absolute sense a tick doesn't tell you much, but in relative sense it can be used to compare the time needed to execute the different formulas.
• location; This column contains the location of this formula within your Futurama document. The location and the item ID can both be used to find the correct formula within your document.

Interpretation of the performance analytics

Performance of the different formulas

With the performance analytics it is possible to determine which formulas are taking the highest calculation time. These formulas are the interested ones to examine as they possibly can be improved in order to reduce the overall response time of your webservice call. You can sort the column '# ticks excluding children' to get the formulas that are the slowest. As pointed, it is best to first look at the formulas that take the longest in order to get the biggest improvement. When the performance analysis is turned on, your application will be much slower. Keep this in mind when looking at the performance of your application. It is important to compare the performance with the performance of the other formulas within the same session. The comparison of the total request time with a non-performance analytics session doesn't give a good view on your performance, because the performance analytics will make it slower.

Caching of the formulas

Within an application you can use caching to make sure fixed formulas aren't executed multiple times. See this page to see how to turn on caching. When using caching, the first request after an application reset isn't interesting. This request will normally be executed during the warm-up of your application and will be used to cache the different fixed formulas. With the performance analytics you should look from the second request and forward, because these requests will use the formulas that are cached from the first request. From the second request forward it is possible to look at the different formulas that should be cached, to check if the caching is done correct. Within the column "# of Calls from cache" it is possible to see how much of the total calls to this formula is used from the cache.

Tips to improve the performance

There are different changes that can be made to improve the performance of your application. Some of these changes will be explained below with tips how to speed up the performance of your application.

Use of allowed ranges

When using an argument within a formula, it is possible to set an allowed range for this argument. See the Formula page how to use allowed ranges in an argument. When using allowed ranges, the calculation will be executed faster when using CalculationMode Compile. Make sure that all arguments within your application have an allowed range if this is possible.

Use arguments of type long

It is possible to use different types of arguments. If you wish to speed up your application, it is best to use arguments of the type long. With arguments of type long it is for instance possible to use allowed ranges (see the tip above).

Avoid the use of dynamic xpaths

Within a XmlField it is possible to use a dynamic xpaths. However, the use of dynamic xpaths can be bad for the performance of your application, so it is best to avoid dynamic xpaths if possible. When validating your document, you may get a message when using dynamic xpaths. You could change these dynamic xpaths to fixed xpaths to improve the performance. See the XmlField page for more information.

Use of caching

When executing the same formula multiple times, it is a good idea to use caching. With caching turned on, the formula will be executed the first time and the result will be stored in the cache. The next time the same answer of this formula is needed, the formula won't be executed again, but the result will be used from the cache. This way the formula won't be calculated every time and the performance of your application will be better. See the Cache page how to enable caching within your application. With the performance analysis you can make sure that the caching is done correctly by checking the total number of calls and the number of calls from cache.

Example

Below an example of the usage of a performance analysis sheet.

Download and install

Take the next steps to set up the example webservice:

• install Futurama Webservice and create an application in IIS
• download the Futurama document and the example performance analysis sheet
• install the Futurama document in the folder of your webservice application
• rename the default.config to web.config
• make sure cache is enabled
• define the rootPath of the mapping element (for example C:\inetpub\wwwroot\FuturamaPerformance\)
• define at the calculation element the location where the performance analysis sheet is saved

SOAP UI

For example SOAP UI can be used to test your webservice. To use SOAP UI for these tests take the next steps:

• open SOAP UI
• select File / New SOAP Project
• choose a Project Name
• define the initial WSDL. This is http://servername/futuramainstallationfolder/service.svc?wsdl&folder=performance&document=performancetest.xml&operation=performance_test&operation=performance_test (for example http://localhost/futuramaperformance/service.svc?wsdl&folder=performance&document=performancetest.xml&operation=performance_test&operation=performance_test)
• open Request 1
• choose an age (for example 45) and execute

Futurama Document

Open the Futurama Document. This document contains a (simplified) calculation of a capital at age 70. The next is important:

• input of the webservice is an age for a person at this moment;
• the output of the webservice is a 50% percentile capital at age 70;
• the calculation is based on 10.000 scenarios, so the 50% percentile is taken from 10.000 capitals at age 70 based on each of the 10.000 scenarios
• there are two matrices of 10.000 * 60 used: one for the inflation and one for the rate of return for assets

Performance analysis sheet

In the download a performance analysis sheet is included based on two successive requests to the webservice. In the first request age of 65 years is used, and in the second request age of 45 years. Feel free to try the webservice by yourself to see the performance analysis sheet that is created. Below a brief explanation of the attached performance analaysis sheet is given:

• in column B the call number is given. As two calls succesive calls were executed in this column two different call numbers are given;
• some rows are highlighted. For these rows a more detailed explanation is given below;
• all the Item ID's that are given for call number 2 are also given for call number 1. However there are a couple of Item ID's that are used in call 1, but not anymore in call 2. The rows with these Item ID's are highlighted in blue. Let's investigate one of these rows: Item ID = 35. Check this ID in the Futurama Document (/PerformanceTest/Inflation/Vector). In the Dependants Window you see that this formula has one Dependant: /PerformanceTest/Inflation/Inflation (with ID=36). In the second call you see (the green highlighted row) that 200.000 calls to this ID are executed, and that all these calls are from cache. That means that the result of ID=36 is retrieved from cache, and that the value of the Determinant(s) of this ID is not needed. This means that no calls are executed to ID = 35 and therefore this ID is not included in the performance analysis sheet;
• to explain the concept of caching check the orange highlighted rows. In these rows the calls to the formula /PerformanceTest/RateOfReturnAssets/RateOfReturnAssets are given. In the Futurama document you will see that the dependant of this formula is /PerformanceTest/RateOfReturnAssets/RateOfReturnAssets (year, scenario). In this formula for each future year 10.000 scenarios will be calculated. So for each year 10.000 calls are executed to the matrix /PerformanceTest/RateOfReturnAssets/RateOfReturnAssets. In request 1 the age of 65 years was used. This implied 70 - 65 = 5 future years. And this means 10.000 * 5 = 50.000 calls. This number you find back in column C. After the first of these 50.000 calls the matrix is in cache, so for the remaining 49.999 calls the value can be retrieved from this cache.
  
  In the second orange highlighted row you will see the number of calls of the second request. In the second request the age of 45 years is used. That implies 70 - 45 = 25 future years. However, in the first request already the first 5 futurama years were calculated. So only 25 - 5 = 20 future years have to be calculated. This means 20 * 10.000 = 200.000 calls. As in request 1 this matrix is already cached all the 200.000 calls are from cache.

Finally

Above some basis knowledge is given for the way caching in Futurama works. When formulas become more complex it will be more difficult to understand how the exact number of calls and calls from cache are calculated. But this is not necessary to improve the performance of your model. The most important is to both look at the formulas with the highest amount of ticks (as investigating this can potentially lead to an improvement of the performance of your model) and to look at the number of calls that are cached. How more calls are cached the better the performance of your model will be. Note that caching will increase memory usage. Especially when using more evolved models pay attention to the memory usage of your application and check whether this remains below a critital server memory level.

Feedback

If you have any questions about this subject or if you want to provide us feedback please send us an e-mail.

Updated: 2023-07-25