Tallan's Technology Blog

Tallan's Top Technologists Share Their Thoughts on Today's Technology Challenges

Introducing the T-Connect EDI File Splitter

Kevin Morillo

A common trend that we have been seeing recently is the similarities in the pain points that companies are often faced with.  One of those such pain points is the difficulty in quickly receiving and processing large EDI files having a file size over 10 MB.  In one instance, there was a need to receive a HIPAA EDI 834 Enrollment file totaling 1.3 GB, containing roughly 800,000 enrollments.  The already powerful BizTalk Server in a 1-1 architecture – 1 BizTalk Server, 1 SQL Server, was having difficulty processing the file.  The file would take several hours to process, often taking up several gigabytes of drive space to attempt to disassemble the file using the EdiDisassembler pipeline.

Benefits

Our solution to this problem was in the creation of the T-Connect EDI File Splitter.  The EDI File Splitter is a Microsoft BizTalk Server pipeline component that enables the rapid splitting of large EDI files within BizTalk Server.  These split files are delivered to an user configurable folder location.  Some of the benefits of this solution include:

  • Splitting large files up into smaller size files improves processing time significantly
  • Easy configuration settings promotes flexibility
  • Deploy easily as a BizTalk Pipeline Component and use with any supported BizTalk adapter
    • Split file by record count allows you to specify the individual file record count as they are split from the larger EDI file
    • Delay split file timer allows you to control and throttle your own delivery of individual files as they are split
    • Split file location allows you to specify where the individual split files will be delivered
  • Process files independently by reconstructing the ISA and GS envelope for each split file
  • Updates ST segment counts to reflect new split file record counts
  • Supports HIPAA X12 EDI Transaction Sets
  • Supports Supply Chain/Order Processing/Transportation X12 EDI Transaction Sets

The tool is able to process files very quickly.  Below are some general speed metrics for a few configurations for an 834 Enrollment file.  All other file types will have similar, if not the same performance.

Record Count File Size Records per split file Total split time
40,000 60 MB 500 10 seconds
40,000 60 MB 1000 10 seconds
800,000 1.3 GB 1000 3.75 minutes

Installation and Setup

The following steps can be used to install and configure the T-Connect EDI File Splitter into an existing BizTalk app

  1. Unzip the installation package after downloading the installation package from the following page
  2. The Install.bat file should then be executed to install the package to the default directory “C:\Program Files (x86)\EDISplitter”  This directory can be altered by opening up the Install.bat file and editing the following line
    • set InstallPath=C:\Program Files (x86)\EDISplitter
    • 1
  3. The bat file will uninstall any previous versions, configure the license, and install the latest version of the T-Connect EDI File Splitter.
  4. The installation will consist of the pipeline components and a sample receive port within the default BizTalk Application 1 application.
    • 2
  5. Below is an example of the configuration options within the T-Connect EDI Splitter pipeline component.
    • 3
  6. By default, the license file is installed to the documents folder of the currently logged in user.  The file can be moved to any location as long as the LicenseFilePath configuration option within the pipeline component settings.  The settings that should be configured by the user are specified below:
Property name Description Default value Required
EdiTransactionType Type of EdiTransaction  is defined herePossible values are : X12_00501_834, X12_00501_837_P, X12_00501_837_I, X12_00501_837_D, X12_00501_835, X12_00501_820, X12_00501_278, X12_00501_270, X12_00501_271, X12_00501_276, X12_00501_277 Empty Yes
OutFolder Specifies the local directory or file share path output file delivery. e.g: C:\Archive Empty Yes
ReleaseAfterXSeconds Number of seconds to delay before the next EDI file written into output folder 0 No
SegmentTerminator Edi segment terminator ~ Yes
SplitAfterOccurances Number of records in an EDI file before it written to out folder 1 Yes

Information

We have rolled this product into a simple to install package which can be downloaded from the following page.  A 30 day full functionality trial is included in the download.  If you would like to learn more about the T-Connect EDI File Splitter, or to purchase a license, please contact us.