Archival functionality in Crow Canyon Application

Applies To: SharePoint On-Premises 2013, 2016, 2019, SharePoint Server Subscription Edition

Description

This article describes the archival feature settings in Crow Canyon Service Request Application. Archival feature moves the old closed Tickets and related list items (like Tasks, Emails etc.) to corresponding ‘archived’ lists. This is usually done to have better performance by keeping only the required items in Tickets list. It also helps in managing the SharePoint list items threshold limit. Archival process works based on the “Request Status” and “Closed Date” column values in the Tickets list items.

Note: Archival process deletes items from ‘main’ lists and moves the data to corresponding ‘archived’ lists. Please read the archival feature configurations carefully to avoid any data loss due to the deletion of items in main lists.

Archival Settings

Service Request Application has archival feature to move old Tickets to a separate list so that your list of active Tickets doesn’t become too large. The old Tickets are put into the ‘Archived Tickets’ list and are fully available in the site for reference and reporting etc.

‘Archival Settings’ list in your site has the required configuration settings to manage the archival process. This includes the lists used in archival and criteria to archive (like archiving after specified number of days once the Ticket has been closed).

Tickets list items data is copied into corresponding columns in the Archived Tickets list. Archival feature also moves corresponding related items like associated IT Support Tasks, Email Tickets, and Approver Tasks.

We can access Archival Settings under Site Contents. There should only be one item for the Archival Settings list. Do not add another item in this list as this may disrupt the archival process, instead edit the existing item to make the required changes.

The Archival Settings are provided with the default values as shown below:

Note:

Before enabling archival feature, ensure that both “Tickets” and “Archived Tickets” lists has the same set of columns with same data type, internal name and display name. Ensure the same for related lists as well. Missing columns in archival lists can cause data loss during archival process
Whenever any change is made to any column in Tickets list, make the same change in “Archived Tickets” list as well. This consistency should be maintained in Tickets and Archived Tickets list to prevent any data loss during archival process
Whenever any list names are changed, update the Archival Settings accordingly
Archival configurations include below mentioned columns. Please create if any of these columns are not present in ‘Archival Settings’ list in your site
Please create the columns with same Internal Names as mentioned in below table. Display name can be changed after creating the column (by editing the column settings). Please ignore if the column already exists in the list

Archival Settings List Column Details

S. No.	Column Internal Name	Column Display Name	Type of Column	Description
1	Title	Title	Single line of Text	Default Value: Archival Settings Note: Do not change the value in this column.
2	SourceTicketListName	Source Ticket List Name	Single line of Text	Name of main ‘Tickets’ list from which items will be moved to archived list.
3	TargetTicketListName	Target Ticket List Name	Single line of Text	Name of Archived Tickets list to which old Ticket items will be moved.
4	SourceEmailListName	Source Related List Names	Multiple lines of text – Plain Text	Tickets list has associated lists like ‘Email Tickets’, ‘IT Support Tasks’ etc. Each of these lists should have a lookup column to Tickets list. Specify name of each of these lists that has to be archived in a separate line in this column.
5	TargetEmailListName	Target Related List Names	Multiple lines of text – Plain Text	Similar to above, there will be associated lists for ‘Archived Tickets’ list having lookup column to ‘Archived Tickets’ list. Items from above lists will be archived to these lists and linked to the archived Ticket item. Ensure that these are in same order as source list names above. Specify each list name in a separate line.
6	TicketIDLookupInTargetList	Ticket ID Lookup Field Name in Target List	Multiple lines of text – Plain Text	These are the names of lookup columns to Tickets/Archived Tickets list in associated lists specified in above two columns.
7	ArchiveCompletedItemsMoreThan	Archive Completed Items More Than	Number (Default to zero decimals)	Set this value to the number of days to keep the Ticket in the main Tickets List after it has been closed. The Ticket will be archived after this many days. With default configuration, Tickets with ‘Closed’ status value are archived if the ‘Closed Date’ column has a date which is less than Today minus the number of days configured in this column.
8	IncludeModifiedDate	Include Modified Date	Yes/No Default Value: No	If this is set to “Yes”, process will look at the last date on which the item was modified and archive the item if it has not been modified for more than days mentioned in ‘ArchiveCompletedItemsMoreThan’ column. Note that this is in addition to other criteria based on the status and closed date. This means that archival of the item will go ahead even it it is not ‘Closed’, provided it has not been modified for required number of days. This setting can be used to archive ‘inactive’ Tickets though they have not yet been closed.
9	ExecutionMode	Execution Mode	Choice Options: Test Production	– In ‘Test’ mode, archival is done by creating items in corresponding archival lists. But the source items are not deleted. This mode can be used to verify that data is being moved correctly from source to target lists. – In ‘Production’ mode, source items are deleted (non-recoverable) after archival. This should be enabled after verifying the data of moved items in archival lists.
10	ProcessItemIds	Process Item Ids	Multiple lines of text – Plain Text	If any Ticket Item Ids are specified in this column, then archival is run only for these items (instead of the state and date based criteria). This is considered only when ‘ExecutionMode’ is set to ‘Test’. In ‘Production’ mode this setting is ignored and normal archival process runs. This can be used to verify archival of few selected items before doing it for all eligible items. E.g.: 1025,1026 (comma separated SharePoint generated Id values) If this column value is blank, then archival is run for items as per the state and date criteria. Normally this column will have no value, only during initial setup and testing some item Ids can be specified for data verification.
11	IgnoreColumns	Ignore Columns	Multiple lines of text – Plain Text	Specify the columns (internal names) that should be excluded from archival process for the Tickets list. Data from these columns will not be moved to archived list. Specify each column name in separate line. E.g.: TicketCr CalculatedColumn Note: This settings is only valid for Tickets list and not associated lists
12	ArchiveStatus	Archive Status	Multiple lines of text – Plain Text	By default Tickets with ‘Closed’ status are considered for archiving. In this column additional status values can be specified. For example, if ‘Resolved’ items too should be archived then that can be specified in this column. Specify each column name in separate line.
13	ArchiveDateColumn	Archive Date Column	Single line of Text	If instead of ‘Closed Date’, archival should happen based on the value of some other date column in the Ticket item, then specify the internal name of that column. For example, if archival should happen based on ‘Resolved Date’ or ‘Created Date’ of the Ticket, we can specify the internal name of corresponding column here.

Archival Logs

Archival process stores the process log in a list in site. This can be used to monitor the archival process and identify any issues. Logs are stored in ‘CCSArchivalLogs’ list, this will be available from the Site Contents.

CCSArchivalLogs list is created automatically when the Archival Timer job runs for the first time. This requires latest version of Crow Canyon Timer Job solution in the SharePoint server.

For each run of archival timer job, an entry gets created in this list with Start Time, End Time, Status, Number of items found eligible for archiving, Number of items successfully archived, items that failed along with detail etc.

Sample log item:

If the number of items to archive exceeds 100, a separate log entry is created for each set of 100 items.

Scenarios in which archival process will stop processing

Archival process has built in monitoring and stops in following scenarios. Once the reason for stoppage is corrected, archival will resume normal operation from next run onward.

This can be monitored from archival log list described above:

If there are more than 5 column mismatches between the source and Target lists. This is done to prevent data loss as columns that are missing in target (or not having correct data type) will not get the data from corresponding source columns during archival.
If there is an error for more than 3 items continuously during the archival process. This indicates some error in configuration or environment.
If error occurred for more than 10 items during the archival process. Large number of errors indicate some environment issue.
If current time is approaching the business hours (as per site regional time zone settings) then process is stopped and it resumes in next cycle. Since archival process is resource intensive, it is preferable to run it in non-business hours. Note that this is temporary stoppage that auto-resumes from next run and doesn’t require any user intervention.

Sample configuration for Status and Date Column Change

If the business process requires considering a different status value or date column for archival then this can be done by changing the archival settings. More than one status value can be considered for archival and instead of ‘Closed Date’ a different column can be specified for considering the age of Tickets for archival. Examples of changes in ‘Archival Settings’ list:

1. If ‘Resolved’ status is added as one of the Ticket status values and Tickets with this status should also be archived in addition to Tickets with ‘Closed’ status, then add this status to the ‘Archive Status’ column as shown below:

2. If the requirement is to archive all Tickets created before a certain date as a one time operation, add all possible status values to the ‘Archive Status‘ column and modify the ‘Archive Date Column’ value to ‘Created’. And specify the number of days after Created in ‘ArchiveCompletedItemsMoreThan‘ column. On next run, archival job will archive all matching Tickets. Once this is done revert the archival settings to normal values: