So you’ve finished writing your new XSLT file using your favorite XML/XSLT editing tool. Great. Now for the next problem, you’ve been given the task to transform thousands of XML documents using your new XSLT file. Great… There’s no way we can transform all of these documents by hand. At this point we should be thinking about automating this process in the fastest and most efficient way possible. This scenario may sound familiar to some of us. Luckily, .Net provides us with such options.
For this example we’ll be serializing an XML file and transforming it using .Net’s XSLT processor. The process is very simple, and requires very little code.
Load the XML File into memory using the XmlReader class.
var reader = XmlReader.Create(xmlFileLocation);
Create a StreamWriter object which will be used to write the transformation to a file.
var textWriter = new StreamWriter(outputFileLocation);
We are going to revisit the issue of dealing with Slowly Changing Dimensions in a data warehouse. We have seen a demonstration of using the SCD transformation that is available in SQL Server Integration Services (SSIS); however, this is not always the best option for processing an SCD. When the volume of rows you’re dealing with is substantial, this creates a significant, and usually unacceptable, performance hit. Another valid reason why you may choose not to implement the SSIS transformation is flexibility. The SSIS transformation requires your begin and end dates to be datetime data types, which could prove to be inconvenient when your SCD needs to tie back to a date and/or time dimension.
One alternative we are going to exhibit is using a SQL Server stored procedure. This example demonstrates the implementation of a Type 2 SCD, preserving the…
Yesterday, Google launched their first web browser application, named Chrome.
And here’s an interesting cartoon that describes Google’s take on web browser.
What’s interesting is the idea of Process vs Thread.
Google’s Chrome uses Process for individual window, tab, and even plug-in.
In contrast, Firefox, the popular open source web browser, utilizes multiple threads for managing its windows, tabs and even plug-ins.
I think some of you remember that Internet Explorer has always been using Process for windows management. Any new windows are launched via new processes, and people did not like this idea, I guess. Firefox, instead, created one process application.
Of course, Internet Explorer 7 uses threads, but only on the tabs.
So, google’s take on web browser application is to use all process and no thread. I find this quite interesting, because as we all know, process takes more memory space than thread does….
I was recently working on an excel pivot table report that pulls its data from a SharePoint list. One of the business requirements was to be able to sort, filter, and/or group by a “Week Ending” date. The list has a column called Action Date which could be any day/date. The solution turned out to be quite simple.
Create a new column in the list called, for example, “Week Ending.” Make this column a calculated column with the following formula:
=TEXT([Action Date]+(7-WEEKDAY([Action Date])),”mm/dd/yyyy”)
Replace [Action Date] with the column off which you are basing the week ending date.
This particular formula assumes that Saturday (7) is the week ending date. Adjust accordingly.
Moving data from one repository to the next may sound like a trivial task, and in the sense of the normal mechanical approach it very well may be. What could be simpler than INSERT INTO Here SELECT * FROM There? Not much, it would seem, until your data source is something other than a SQL Server table or your data volume is so heavy that such a simple statement chokes, or worse yet, affects the transactional system’s performance. What can you do when your data volumes are large enough that you need to consider faster approaches than manual SQL Statements?
Fortuitously, Microsoft has provided us with the BCP utility. BCP is a command line executable that is optimized to move data very quickly from one place to another. BCP is flexible enough to allow transformations of data in the operation, as…
This Saturday was the first Hartford Code Camp. I presented on SpringFramework.NET specifically regarding Inversion of Control (IOC).
The presentation is in PowerPoint and discusses some of the basic concepts that need to be understood prior to understanding Spring.NET.
The source code is what was demonstrated during the presentation. It contains two solution folders.
TightlyCoupledClasses – Example of normal C# development where classes are tightly coupled together with manual instantiation.
LooselyCoupledClasses – Similar code loosely coupled using Spring.NET IOC. Includes various NUnit tests utilizing mock objects showing the ability to isolate tests with manual injection of mock objects.
If you have any question feel free to post questions or comments here or email me directly at firstname.lastname@example.org.
Reflection has long been a point of contention among the development community.On the one hand you have the elegance purists that insist that Reflection is the panacea that ensures we write boilerplate once and only once, and on the other you have the performance purists who insist that Reflection is too slow to be considered for a real enterprise solution.
So is either side right? Well, IMHO the truth is that it depends. Let me start by saying, I find myself torn. I really sort of feel like I am a staunch advocate of both camps if that is possible. I love finding a way to avoid writing and maintaining repetitive boilerplate code. Furthermore, I love designing frameworks that service whole domains in an abstract way. On the other hand, I love hearing a user say, “Gosh, that was fast….
This weekend was the first Hartford area Microsoft Code Camp. It was very successful with well over 150 attendees and over 22 presenters. We presented the following topics:
Lee Rodgers – Spring.NET & IOC
Reddy Kadasani – Silverlight – Lighting up the Web
Presentation content and demo content will be posted to this site in the next few days, so please check back.
Addition information about the code camp can be found at the Conneticut .NET user group website (http://ctdotnet.org).
In this post, we will demonstrate the Slowly Changing Dimension (SCD) transformation that is available for use in a Data Flow task within SQL Server Integration Services (SSIS). We’ll walk through an example of how to use it, explaining the terms as we go. In addition, I’ll demonstrate some “extra effort” kind of modifications that can be made for a more thorough usage. After that, we’ll discuss the pros and cons of using it at all. And we’ll wrap up with some tips on how to tune the transformation for better performance.
Data warehousing is a big subject. This overview is intended to cover some of the most representative issues on a high level: the nature of OLAP systems, star schemas, facts and dimensions, and differing perspectives (Inmon vs. Kimball) on warehouse design.
OLTP vs. OLAP
OLTP systems are the operational databases supporting applications. They are highly normalized, and focused on CRUD operations.
OLAP databases are usually arranged in star schemas and are built for speed in retrieving aggregated data.