As we talked about in our recent Dear Data blogs, our Analytics team uses data in innovative ways that bring value to healthcare organizations and the patients they serve, value like improving clinic operations, increasing quality of care, decreasing accounts receivable, and more. Today we are going to focus on a recent interesting data challenge and the value it provided the organization.

Situation

We were working with UCSF’s Medical Education Team to gather insight into their education program and curriculum. They were trying to answer questions like:

• Where in the curriculum do we cover opioid addiction?
• How much do students learn about domestic violence?
• When are students exposed to the coursework, and is the coursework clustered in certain areas of the curriculum?

Answering these types of questions facilitates meeting regulatory requirements, but also allows the team to better plan going forward.

Solution

Through talking with the team, we identified the need to better search across their existing data set, by creating a “google-like” search within a Tableau application. To do this, we faced two main challenges:

• Determine and execute on the best way to bring data into Tableau from Ilios, their curriculum data source (https://www.iliosproject.org/)
• Mimic “google-like” search using Boolean search operators to filter Tableau results

Challenge #1: Getting the Data

Tableau users will appreciate its ability to connect to a variety of data sources with little more than the click of a button. However, an API-based data source like Ilios presents a bit more of a challenge. To overcome this hurdle, we took advantage of Tableau’s Web Data Connector or WDC (https://tableau.github.io/webdataconnector/), which provides a JavaScript interface for querying web APIs to produce Tableau extracts.

WDC is created and maintained by Tableau and integrates well with the product, but unlike standard database connections, implementing with the WDC isn’t exactly plug-and-play. Among the considerations while developing for the Ilios API with WDC were:

• Authentication and credential management
• Iterative extraction for pulling megabytes of data over HTTP a few thousand rows at a time
• Cascading filters guaranteeing that only correlated data was extracted across the many Ilios API endpoints
• Joining a dozen or so entities into a singular data model
• Throttling to ensure the API owners didn’t kick us off their network J

Challenges notwithstanding, 1100 lines of JavaScript later, we had a fully-functional API-driven data source. Data successfully extracted, it was time to tackle search.

Challenge #2: ‘Google-like’ Searching

Searching sounds simple, right? It is really about creating relationships between various data elements and determining, what parameters and Boolean logic to apply.  You are right, the Tableau Boolean logic was not the tough part.  The tough part was making it run quickly!  Our initial iteration took over 30 minutes to return results – imagine typing something into Google and having to wait for 30(!!) minutes to see the results.  Our second attempt brought the time down to 5 minutes, something that we thought was a pretty good improvement from 30 minutes, but we knew we could do better!

The finished product returns search results anywhere from 10-60 seconds – still not quite like Google but much closer.  The optimal Tableau configuration for performance involved creating a second search data source that was a slimmed down version of the primary data source.  The search data source contained only those fields that were used to match the keyword Boolean search and was used as a filter throughout the application.  If you haven’t already, check out the Tableau Desktop’s built in Performance Monitoring tool (https://onlinehelp.tableau.com/current/pro/desktop/en-us/perf_record_create_desktop.htm ), as analyzing this was key to optimizing workbook performance.

Cheers to New Challenges

While some customers need a dashboard and the ability to quickly see key performance indicators, others need the ability to ask questions of their data.  The value derived from the data customers have is use case specific and that is where the fun really starts.  We love exploring new data sets, new questions and challenges. Do you have a challenge you would like to share with us?