The Semantic Data Model in Big Data
One of the keys to taking unstructured data—audio, video, images, unstructured text, events, tweets, wikis, forums and blogs—and extracting useful data from it is to create a semantic data model as a layer that sits on top of your data stores and helps you make sense of everything. Hadoop Big Data Training in Chennai
“Traditionally, the way in which we’ve done that and the way in which the industry has done that is we’ll take extractions of that data from however many different places and build a repository and produce reports off that repository. That’s a time-consuming process and not an extremely flexible one. Every time you make a change, you have to go back and change the data repository.”
International Business Machines
To make that process more efficient, State Street set out to establish a semantic layer that allows data to stay where it is, but provides additional descriptive information about it.
Our customers may call the same thing by two different names. Semantic technology has the ability to indicate those things are in fact the same thing. For instance, someone might call IBM ‘IBM’ or ‘International Business Machines’ or ‘IBM Corporation’ or some other variation. They really are the same thing. By showing that equivalence within the semantic layer, you can indicate they’re the same thing.” Hadoop Big Data Training in Chennai
“If we’re trying to pull together a risk profile for all of the exposures we have to a particular entity or geography or whatever, that information is kept in lots of different places. Numerical information in databases, unstructured information in documents or spreadsheets. We see that providing a semantic description for these various sources of risk information means we can quickly pull together a consolidated risk profile or an ad hoc request. One of the other benefits that we see is that semantic technology, unlike a lot of other things, doesn’t mean we have to go back and redo all of our legacy systems and database definitions. It lays on top of that, so it’s much less disruptive than another type of technology that would require us to go to a clean slate. We can do it incrementally. Once we’ve provided a semantic definition for one of these sources, we can add on other definitions from other sources without having to go back and redo the first one.”
Programmer or DBA
State Street has approached the semantic data model by building a set of tools to help end users—generally a business person rather than a programmer or DBA—do the description.
“In most cases that’s not a programmer or DBA, that’s a business person. The business person, in describing the data, knows what that data is. They know what this reference information is supposed to connote. Using the tool, they can translate that into a semantic definition and in turn use that and combine it with some other definitions to produce, say, a risk report or the onboarding of a new customer. For years we’ve talked about being able to blur the line that exists between IT and the business and having business be able to have tools where they can more clearly express requirements. This is a step in that direction. It’s not full business process management, but it’s certainly a step in getting there.”
You Must be Like