Applicable Versions: 4.05 onwards
System.IO.OutOfmemory error messages with a dashboard that returns a lot of data a consequence of a 32 bit process performing various simultaneous functions on a large dataset. The dashboard is doing a lot more than Excel with the dataset. In addition to displaying the dataset as Excel does, for each object the dashboard needs to perform complex summary and calculations against the dataset.
The longer a process is running, the more fragmented the available memory space it can possibly address becomes. Any 32bit process can only ever address 4GB of address space. Each time it processes your dataset it needs to find a single continuous data slot for the entire dataset. As the available memory space gets more fragmented, it eventually becomes impossible to find the required space and hence the error message.
In version 5, a lot of work to reduce the amount of times the data was duplicated to a bare minimum, and ensure that any large portions of memory were re-used so as to reduce fragmentation. Version 6 of the Dashboard can run as 64 bit, which will allow for datasets many times larger, but this is only a solution for machines running 64 bit, and when embedded only if your application is also running 64bit.
In the meantime, there are a number of things you can do to reduce the likelihood of running into this problem which are set out below;
- Keep the amount of data to a minimum by using summary queries to summarise the data on the AppServer before it is returned to the dashboard wherever possible.
- Only return the columns you are actually going to use in the dashboard with your query. It is not the number of rows that is important, but the number of data elements. If your dataset had only 3 or 4 columns, you may well be able to process well over a million records. String columns are particularly expensive, taking 2 bytes for every character in every string returned (other data types generally only requiring 1 or 2 bytes for every data element).
- Similarly keep the number of calculated columns defined in the dashboard to a minimum. Each calculated column takes up memory for itself, but also requires memory for processing.
- Keep the number of objects on a single dashboard to a minimum. Think about using multiple dashboards rather than multiple tabs. Each object requires its own processing. Treeviews are particularly expensive, given the complex multi-level summarizing they require.
- If you are embedding the dashboard in your own application, be aware that the dashboard control shares the same process as your application. As such, if your application is not efficient in its use of memory it will exacerbate the problem.