Incorporating Unstructured Data with EmeraldVision7th May 2014
by Craig Kelly
Requirements for great analytics sometimes reach beyond traditional relational data sources. In some cases, you might want to include valuable insights that are contained within unstructured data from social media, text, parsed logs, or emails. This information may, for example, supplement traditional sales or CRM related data, or be used to capture events or alerts for further insights.
In my example below, the customer wanted to obtain names and contact information within specific CRM related emails. This information was desired to help identify new sales leads for their Sales Reps to follow up with. Leveraging CloudConnect to populate EmeraldVision, I used a straight forward process that includes the following four simple steps to extract specific data elements from emails that were being received to a service account:
- Read in the email content
- Filter on email sender, subject, or other identifying attribute
- Parse the required data
- Deliver to the data warehouse
Within the Email Reader component, you simply provide the relevant username and password information, and then select which metadata objects you want to extract:
Next, you define what specific emails you want to include in your model. In my example below, I included a filter based off of specific email subject text:
From there, I used a Reformat object, and incorporated a little Regex to parse out names and contact information from the body of the email. In my example, I looked for specific text tags that existed in my emails:
And then applied the following code to retrieve the text that followed the identified tags:
After setting up these steps, I simply pushed the resulting metadata to my model, and quickly built out the required reports within EmeraldVision. The customer had these reports emailed on a set schedule to all of their sales reps on a regular basis with no manual effort required.
Be on the lookout to learn more about how easy it is to incorporate nontraditional data into your analytics!