Snapshotting objects in Salesforce with apex

A common issue that we have is a need to see information about Cases when it is created.  We do this to do some analysis about how a case changes (primarily to verify how good our automated tools are working).  To achieve this, we made a generic snapshot object that will store a JSON version of our data.  We chose JSON for it’s portability and it’s ability to dump into other systems.

The Object

To start out we’ll need a place to put this data, so we created the object with the following fields. Download object.

  • JSON_Data_{0-9}__cRequired – These are several LongTextAreas that stores json data
  • Object_Name__cRequired – This is the name of the object that was snapshotted
  • NameRequired – An auto number, just used for identification
  • Case__c – Optional – This is used for our case specific snapshot to link a snapshot back to a specific case

Apex Class

/**
* Utility methods for use with the Object Snapshot object
*
* @author Patrick Connelly
*/

public with sharing class ObjectSnapshotUtils {
	public static String JSON_DELIMITER = '[delimiter]';
	public static Integer JSON_FIELD_SIZE = 32768 - (JSON_DELIMITER.length() * 2);
	public static Integer JSON_FIELD_COUNT = 10;

	public static final String JSON_FIELD_TEMPLATE = 'JSON_Data_{0}__c';

	public static Boolean SPARSE_JSON = true;

	public static final String SNAPSHOT_VERSION = '20120625';
	public static final String SNAPSHOT_VERSION_LABEL = 'snapshot_version';
	public static final String SNAPSHOT_SUCCESS_LABEL = 'snapshot_success';
	public static final String SNAPSHOT_MESSAGE_LABEL = 'snapshot_message';
	
	public static final String MSG_JSON_TO_LARGE = 'JSON data too large to store';

	public static final Map<String, Object> SNAPSHOT_INFO = new Map<String, Object> {
		SNAPSHOT_VERSION_LABEL => SNAPSHOT_VERSION,
		SNAPSHOT_SUCCESS_LABEL => true,
		SNAPSHOT_MESSAGE_LABEL => ''
	};

	public static final Map<String, Object> SNAPSHOT_INFO_JSON_TO_LARGE = new Map<String, Object> {
		SNAPSHOT_VERSION_LABEL => SNAPSHOT_VERSION,
		SNAPSHOT_SUCCESS_LABEL => false,
		SNAPSHOT_MESSAGE_LABEL => MSG_JSON_TO_LARGE
	};

	// These fields should never be added into the field map
	public static final Map<String, Set<String>> FIELD_BLACKLIST_MAP = new Map<String, Set<String>>{
		'Case' => new Set<String>{
			'lastvieweddate',
			'lastreferenceddate'
		}
	};

	private static Map<String, Map<String, Object>> objectDescription = new Map<String, Map<String, Schema.sObjectField>>();

	/**
	* Gets a map of field name to their values
	*
	* NOTE: Before calling this method, make sure that objectDescription has been populated
	*       for the called object.  getSnapshot handles this, but if you are going to call
	*       this directly, make sure it's populated.
	*
	* @param obj The sObject to build the map from
	* @return A map of field name to value
	*/
	private static Map<String, Object> getMapOfAllFields(String objName, sObject obj) {
		Map<String, Object> result = new Map<String, Object>();

		Set<String> fieldNames = objectDescription.get(objName).keySet();
		if (FIELD_BLACKLIST_MAP.containsKey(objName)) {
			fieldNames.removeAll(FIELD_BLACKLIST_MAP.get(objName));
		}

		for (String fieldName: fieldNames) {
			if (
				!SPARSE_JSON || (
					obj.get(fieldName) != null &&
					String.valueOf(obj.get(fieldName)).trim() != ''
				)
			) {
				result.put(fieldName, obj.get(fieldName));
			}
		}

		return result;
	}

	/**
	* Adds the delimiter to the start and end of the string
	*
	* NOTE: This is done because SFDC trims whitespace from the start/end of
	*       All fields
	*
	* @param data The string
	* @return The appended data
	*/
	public static String appendDelimiter(String data) {
		return JSON_DELIMITER + data + JSON_DELIMITER;
	}

	/**
	* Converts a string of json data into an object snapshot
	*
	* @param jsonData The json data
	* @return The object snapshot
	*/
	private static ObjectSnapshot__c jsonToSnapshot(Map<String, Object> fieldMap) {
		ObjectSnapshot__c result = new ObjectSnapshot__c();

		String jsonData = JSON.serialize(fieldMap);

		// Figure out if we have enough room in our fields for all the json data
		Integer numberOfFieldsRequired = (Integer)(Math.floor(jsonData.length() / JSON_FIELD_SIZE));
		if (numberOfFieldsRequired >= JSON_FIELD_COUNT) {
			fieldMap = SNAPSHOT_INFO_JSON_TO_LARGE;
			jsonData = JSON.serialize(fieldMap);
		}

		for (Integer lowerBound = 0; lowerBound < jsonData.length(); lowerBound += JSON_FIELD_SIZE) {
			Integer upperBound = ((lowerBound + JSON_FIELD_SIZE) > jsonData.length()) ? jsonData.length() : lowerBound + JSON_FIELD_SIZE;
			Integer index = (Integer)(Math.floor(lowerBound / JSON_FIELD_SIZE));
			String fieldName = String.format(JSON_FIELD_TEMPLATE, new List<String>{String.valueOf(index)});
			String field = appendDelimiter(jsonData.subString(lowerBound, upperBound));
			result.put(fieldName, field);
		}

		return result;
	}

	/**
	* Removes the delimiter from the start and end of the string
	*
	* @param data The data
	* @return The stipped down data
	*/
	public static String removeDelimiter(String data) {
		if (data == null) {
			return data;
		}

		return data.removeStart(JSON_DELIMITER).removeEnd(JSON_DELIMITER);
	}

	/**
	* Converts a Object Snapshot object to a json string
	*
	* @param snapshot The snapshot to convert
	* @return The json data
	*/
	public static String snapshotToJson(ObjectSnapshot__c snapshot) {
		List<String> JSONDataList = new List<String>();

		for (Integer i = 0; i < ObjectSnapshotUtils.JSON_FIELD_COUNT; i += 1) {
			String fieldName = String.format(ObjectSnapshotUtils.JSON_FIELD_TEMPLATE, new List<String>{String.valueOf(i)});
			String field = (String)(snapshot.get(fieldName));
			field = removeDelimiter(field);
			JSONDataList.add(field);
		}

		return String.join(JSONDataList, '');
	}

	/**
	* Gets the snapshot object of an sObject
	*
	* @param obj The sObject to snapshot
	* @return The snapshot
	*/
	public static ObjectSnapshot__c getSnapshot(sObject obj) {
		ObjectSnapshot__c result = new ObjectSnapshot__c();
		Schema.DescribeSObjectResult describeResult = obj.getSobjectType().getDescribe();
		String objName = describeResult.getName();

		// Doing this to reduce the number of field queries we make so we don't hit the limit of 100
		if (!objectDescription.containsKey(objName)) {
			objectDescription.put(objName, describeResult.fields.getMap());
		}

		Map<String, Object> fieldMap = getMapOfAllFields(objName, obj);
		fieldMap.putAll(SNAPSHOT_INFO);
		result = jsonToSnapshot(fieldMap);

		result.Object_Name__c = objName;
		return result;
	}

	/**
	* Creates and inserts the snapshots of a case
	*
	* @param cases A list of cases to snapshot
	*/
	public static void createSnapshots(List<Case> cases) {
		List<ObjectSnapshot__c> snapshots = new List<ObjectSnapshot__c>();

		for (Case newCase: cases) {
			ObjectSnapshot__c snapshot = ObjectSnapshotUtils.getSnapshot(newCase);
			snapshot.Case__c = newCase.Id;
			snapshots.add(snapshot);
		}

		if (!snapshots.isEmpty()) {
			insert snapshots;
		}
	}
}

Let’s breakdown this class and explain what is happening.

  • Static Variables
    • JSON_DELIMITER – Because Salesforce strips whitespace from the end of textarea fields, we need to surround all of our split JSON data with a delimiter to force Salesforce to honor the data as we hand it in.
    • JSON_FIELD_SIZE – This is the size of our LongTextArea, minus the length of the delimiter twice (since we’re wrapping it)
    • JSON_FIELD_COUNT – This is the number of JSON_Data_*__c fields we have on our ObjectSnapshot__c object
    • JSON_FIELD_TEMPLATE – This is a String.format template to convert to the JSON_Data_*__c field we are storing the data in
    • SPARSE_JSON – If the JSON data should be trimmed prior to insertion
    • SNAPSHOT_VERSION – The “version” of the snapshot utils.  Used for auditing and stored with the snapshot
    • SNAPSHOT_*_LABEL – Text used to generate the map for the SNAPSHOT_INFO*
    • MSG_JSON_TO_LARGE – The message used if the JSON data exceeds the JSON_FIELD_COUNT multiplied by the JSON_FIELD_SIZE
    • SNAPSHOT_INFO – A map of data used to identify a successful snapshot.  This data is added to the snapshot for identification
    • SNAPSHOT_INFO_JSON_TO_LARGE – A map of data used to identify a unsuccessful snapshot due to the resultant data being to large to store in a single ObjectSnapshot__c object.
    • FIELD_BLACKLIST_MAP – This map of a set of strings is used to blacklist certain fields that should NEVER be fetched or attempted to be snapshotted.  For example, Case.lasevieweddate is a field that will throw a DML exception if the code attempts to do a .get(…) on that field.  Because of this, any items in the FIELD_BLACKLIST_MAP.get(objectName) will be removed from the objectDescription results.
  • Methods
    • getMapOfAllFields – This gets a map of all of the fields (minus the blacklisted fields) for a given object.  NOTE: This should not be called directly unless the objectDescription variable is already populated.  The getSnapshot method handles this request.
    • appendDelimiter – This appends the JSON_DELIMITER variable to the start and end of a string
    • jsonToSnapshot – This converts a map of String to Objects to ObjectSnapshot__c object
    • removeDelimiter – This removes the JSON_DELIMITER variable from the start and end of a string
    • snapshotToJSON – This converts an ObjectSnapshot__c object to JSON
    • getSnapshot – This takes an sObject and converts it to an ObjectSnapshot__c
    • createSnapshots – This takes a list of cases and converts them to an ObjectSnapshot__c and inserts them.

To create your own snapshots for a non Case object, just look at the implementation of createSnapshots(List<Case> cases) and tailor it towards your object.  Since the getSnapshot method uses Schema.DescribeSOjbectResult methods, it should be a matter of converting and then enriching the ObjectSnapshot__c object with any additional information you desire.  For example, we added a Case__c field that the Case’s Id is stored into, linking it back to the original object.

Usage

Using this is very simple after creating the createSnapshots method.  The following trigger calls our ObjectSnapshotUtils and tells it to create a snapshot for all of the new Cases.

trigger CaseTrigger on Case (after insert) {
	ObjectSnapshotUtils.createSnapshots(Trigger.new);
}

Future Plans

There are a couple of features I’d like to add to this to make it easier for others to consume. (In no particular order)

  • Better OO methods for this class.  Making it so you can extend a snapshot method and then do your enrichment (or overload an enrichment method).  This would allow you to do something like CaseSnapshotUtils implements ObjectSnapshotUtils and then call CaseSnapshotUtils instead
  • Better error handling JSON data being too long. Like putting more statistics in the SNAPSHOT_INFO_JSON_TO_LARGE map that gets inserted with the snapshot
This entry was posted in Development, Salesforce and tagged , , , , , , . Bookmark the permalink.
  • Kirill Yunussov

    Do you know if there is a way to schedule native SF Snapshots using apex? I want to make a snapshot run daily for 10 days the first month of every quarter. I haven’t got to figuring out the cron expression for this yet, would like to first know if I can even use it.

  • Raj Bhatt

    Hi Patrick, I’m really interested in trying this out! How would you suggest writing a test class for this?

  • Testing this is complicated and very dependent on your objects and fields that you are looking to snapshot. I would recommend that you test it like any other class [1]. The key will be to recombine the data in the snapshot object and deserialize the data back into either a map of strings or back into the source object. The compare it against the expected results.

    [1] http://blog.deadlypenguin.com/blog/testing/strategies/