The X12 HIPAA transaction set is used across the healthcare industry to transmit claim, enrollment and payment information. Given the importance and ubiquity of these EDI files, you might assume that translating them from ANSI to a relational database format would be well-supported with a range of options.
In practice, a task as common as parsing a claim or encounter and storing it in a database can quickly escalate into a significant problem.
One solution we’ve seen involves archiving a snapshot of the EDI file using filestream storage. This can satisfy some retention requirements, but provides little in terms of fine-grained tracking or analytic capabilities.
A more complete approach is to parse the X12 file into its discrete elements and store them in a relational database. The ideal solution captures the full extent of the EDI transactions while also applying a reasonable leveling of flattening to keep in the number of table joins under control.
Yesterday, I attended a seminar covering the Oracle BI landscape. My aim was to come out of this session with a clearer idea of how Oracle products correspond to the Microsoft BI stack. My impression going into this seminar was that Oracle had many, many applications bundled under the BI umbrella. Nevertheless, I was surprised by the sheer number of options available. Practically every piece of the MS BI Stack has at least two parallel products on the Oracle side, in some cases many more. My second impression was that Oracle has done some very credible work to integrate the vast number of applications they’ve developed alongside their Siebel and Hyperion purchases.
Here’s how Oracle visualizes their BI offering:
This post is intended to introduce Analysis Services 2005/2008 foundational concepts.
SQL Server Analysis Services is a component included in the Microsoft SQL Server product, and its use is fully covered under the same license (which applies to Integration Services and Reporting Services as well). Like the database engine, SSAS has a range of features stratified by Express, Workgroup, Standard Edition and Enterprise Editions.
SSAS is an OLAP, multi-dimensional database. While a relational data warehouse can also be described as OLAP, products such SSAS, Cognos and Essbase have fundamental differences. These OLAP servers:
· Aggregate data from a variety of sources into a compressed format optimized for query response
· Emphasize end-user navigation with modeling capabilities such as hierarchies, or drill-paths
· Employ MDX as the standard query language for retrieving result sets
· Use XMLA as the standard communication mechanism between clients and servers
Two big pieces of news on the Microsoft BI front:
Performance Point Server is being dismantled. After the next service pack, MS will cease develop of the product as its own entity. The Monitoring & Analytics capabilities will be bundled into the enterprise version of SharePoint. The Planning application looks to be dead in the water, however.Although PPS was complicated and unwieldly, it had potential and an interesting vision. I don’t see the failure of any part of the MS BI stack as a positive thing. The official press release and blog reactions are here.
While Microsoft had previously described a major-minor release schedule as the model for SQL Server going forward, the next version sounds like an interesting departure from that. Code-named “Kilimanjaro”, the next SQL Server release has been described as minor and BI-focused.Two components described thus far are “Madison”…
Once tables grow into the millions of records, they become candidates for partitioning. Table partitioning offers many benefits, particularly in warehouse environments. Since data is split into smaller units of storage, backups can target filegroups with a higher rate of change. Systems with multiple CPUs see improved query performance as partitioned data leads to greater parallelism. Perhaps most significant is the ability to swap in huge amounts of data by partition switching, an operation that is practically instantaneous.
Tables can be partitioned horizontally or vertically. With vertical partitioning, columns are split out into separate physical tables. This post focuses on horizontally partitioned tables, which take advantage of new constructs Microsoft added in the 2005 release – partition functions and partition schemes. Table partitioning is an Enterprise Edition only feature in SQL Server 2005 and 2008.
Data warehousing is a big subject. This overview is intended to cover some of the most representative issues on a high level: the nature of OLAP systems, star schemas, facts and dimensions, and differing perspectives (Inmon vs. Kimball) on warehouse design.
OLTP vs. OLAP
OLTP systems are the operational databases supporting applications. They are highly normalized, and focused on CRUD operations.
OLAP databases are usually arranged in star schemas and are built for speed in retrieving aggregated data.