Open Business Objects for EDI
v3.6.0

com.americancoders.edi
Class Tokenizer

java.lang.Object
  |
  +--com.americancoders.edi.Tokenizer
All Implemented Interfaces:
ITokenizer
Direct Known Subclasses:
ACHTokenizer, EDIFactTokenizer, TradacomsTokenizer, X12Tokenizer

public abstract class Tokenizer
extends java.lang.Object
implements ITokenizer

class to assist in tokenizing input transaction sets
x12 field seperator uses 3rd byte of input string
x12 segment separator uses 16th field + 1 byte field, if it sees a cr character then checks for a lf character and then assumes a \\n character
EDIFact uses different control positions. see header segment.

OBOE - Open Business Objects for EDI

Part of the OBOE Basic Extended Package
An EDI and XML Translator Written In Java
Copyright 1998-2009 - American Coders, LTD - Raleigh NC USA
All rights reserved
American Coders, Ltd
P. O. Box 97462
Raleigh, NC 27624 USA
1-919-846-2014
http://www.americancoders.com

Version:
3.6.0
Author:
Joe McVerry - American Coders, Ltd.

Field Summary
protected  java.lang.String currentDataElement
          String current data element value
protected  java.lang.String currentSegID
           
protected  com.americancoders.edi.IDataTokenizer currentTokenizer
          String current tokenizer
protected  java.lang.String currentTokenStringToken
          String current token
protected  int dataelementPos
          String position within
protected  boolean dataElementReady
          boolean has it been processed yet?
protected  com.americancoders.edi.DocumentErrors dErr
           
protected  java.lang.String escapeCharacters
          allow token characters in text
protected  char repeatChar
          token the breaks up repeating elements
protected  com.americancoders.edi.SegmentContainer requestingContainer
           
protected  int segmentPos
          String position within
protected  java.lang.StringBuffer separators
          all the tokens separator
protected  char[] tokenGroups
          String simple array for helper routine
protected  java.lang.String tokenSeperatorCharacter
          Characeter that breaks up fields
protected  com.americancoders.edi.IDataTokenizer transactionSetTokenizer
          StringTokenizer data contents
 
Constructor Summary
Tokenizer(com.americancoders.edi.DocumentErrors inDErr)
          builds the parsing object for a transaction set
 
Method Summary
 int countDataElements()
          returns a count of dataelements in segment
 java.lang.String getCurrentDataElement()
          returns the currently parsed data element.
 java.lang.String getCurrentTokenStringToken()
          returns the segment string // used by edifact envelope parser
 java.lang.String getDataElementAt(int pos)
          returns the data element parsed at a particular postion
 int getDataElementPos()
          returns the position of the current dataelement being tokenized.
 java.lang.String getElementSeparator()
          returns the element delimiter
 java.lang.String getEscapeCharacters()
          returns the escape character string
 int getInputByteCount()
          returns the byte positin in the input stream of the current token
 com.americancoders.edi.SegmentContainer getLastSegmentContainer()
          returns the last segment container that made a getNextSegment request
 java.lang.String getNextDataElement()
          returns the next data element to be parsed
 com.americancoders.edi.IDataTokenizer getNextSegment(com.americancoders.edi.SegmentContainer inSegContainer)
          returns the next segment to be parsed
 java.lang.String getRepeaterCharacter()
          returns the repeater delimiter
 com.americancoders.edi.DocumentErrors getReportError()
          gets the DocumentErrors object
 java.lang.String getRestOfSegment()
          helper routine for edifact UNA segment
 java.lang.String getSegmentID()
          returns what is suppose to be the id of the segment being parsed.
 int getSegmentPos()
          returns the position (line number?)
 int getSegmentPosition()
          returns the relative segment position
 java.lang.String getSeparators()
          return the separator characters that were used to parse the document
 java.lang.String getVersion()
          returns the version id in the header segments presently only works for X12 field I11.
 boolean hasMoreDataElements()
          indicates if there are more data elements to parse
 boolean hasMoreSegments()
          indicates if there are more segments to parse
 boolean isThereADataElement()
          indicates if there is data element to parse
 com.americancoders.edi.IDataTokenizer makeSubfieldTokenizer()
          returns a subfield tokeinzer
 void reportError(java.lang.String inDescription)
          adds an error description to the OBOEParserErrors object
 void reportError(java.lang.String inDescription, com.americancoders.edi.SegmentContainer inRequestingContainer, java.lang.String inCode, java.lang.Object inObject, int inLevel)
          adds an error description to the OBOEParserErrors object
 void reportError(java.lang.String inDescription, java.lang.String inCode)
          adds an error description to the OBOEParserErrors object
 com.americancoders.edi.IDataTokenizer resetSegment()
          resets the current segment to the beginning - an inverse look-ahead method
 void setLastSegmentContainer(com.americancoders.edi.SegmentContainer inSC)
          sets the last segment container that made a getNextSegment request
 java.lang.String toString()
          returns the current token string.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Field Detail

transactionSetTokenizer

protected com.americancoders.edi.IDataTokenizer transactionSetTokenizer
StringTokenizer data contents


currentTokenStringToken

protected java.lang.String currentTokenStringToken
String current token


currentTokenizer

protected com.americancoders.edi.IDataTokenizer currentTokenizer
String current tokenizer


currentDataElement

protected java.lang.String currentDataElement
String current data element value


dataElementReady

protected boolean dataElementReady
boolean has it been processed yet?


segmentPos

protected int segmentPos
String position within


dataelementPos

protected int dataelementPos
String position within


tokenSeperatorCharacter

protected java.lang.String tokenSeperatorCharacter
Characeter that breaks up fields


tokenGroups

protected char[] tokenGroups
String simple array for helper routine


escapeCharacters

protected java.lang.String escapeCharacters
allow token characters in text


separators

protected java.lang.StringBuffer separators
all the tokens separator


requestingContainer

protected com.americancoders.edi.SegmentContainer requestingContainer

dErr

protected com.americancoders.edi.DocumentErrors dErr

currentSegID

protected java.lang.String currentSegID

repeatChar

protected char repeatChar
token the breaks up repeating elements

Constructor Detail

Tokenizer

public Tokenizer(com.americancoders.edi.DocumentErrors inDErr)
          throws OBOEException
builds the parsing object for a transaction set

Parameters:
inDErr - DocumentError
Throws:
OBOEException - invalid token most likely
Method Detail

getSeparators

public java.lang.String getSeparators()
return the separator characters that were used to parse the document

Returns:
String

getVersion

public java.lang.String getVersion()
returns the version id in the header segments presently only works for X12 field I11.

Returns:
String

getNextSegment

public com.americancoders.edi.IDataTokenizer getNextSegment(com.americancoders.edi.SegmentContainer inSegContainer)
returns the next segment to be parsed

Specified by:
getNextSegment in interface ITokenizer
Parameters:
inSegContainer - SegmentContainer requesting the next segment. Used for error handling and recovery.
Returns:
IDataTokenizer the next segment

getRestOfSegment

public java.lang.String getRestOfSegment()
helper routine for edifact UNA segment

Returns:
String

hasMoreSegments

public boolean hasMoreSegments()
indicates if there are more segments to parse

Specified by:
hasMoreSegments in interface ITokenizer
Returns:
boolean more to parse?

resetSegment

public com.americancoders.edi.IDataTokenizer resetSegment()
resets the current segment to the beginning - an inverse look-ahead method

Specified by:
resetSegment in interface ITokenizer
Returns:
IDataTokenizer the next segment

getSegmentPos

public int getSegmentPos()
returns the position (line number?) of the current segment being tokenized. value is relative to zero

Specified by:
getSegmentPos in interface ITokenizer
Returns:
int the current segment position (line number)

countDataElements

public int countDataElements()
returns a count of dataelements in segment

Specified by:
countDataElements in interface ITokenizer
Returns:
int the number of data elements

getCurrentDataElement

public java.lang.String getCurrentDataElement()
returns the currently parsed data element. It does not return a data element object but a value.

Specified by:
getCurrentDataElement in interface ITokenizer
Returns:
String current value of data element

getNextDataElement

public java.lang.String getNextDataElement()
returns the next data element to be parsed

Specified by:
getNextDataElement in interface ITokenizer
Returns:
String current value of data element

getDataElementAt

public java.lang.String getDataElementAt(int pos)
returns the data element parsed at a particular postion

Specified by:
getDataElementAt in interface ITokenizer
Parameters:
pos - int
Returns:
String current value of data element

hasMoreDataElements

public boolean hasMoreDataElements()
indicates if there are more data elements to parse

Specified by:
hasMoreDataElements in interface ITokenizer
Returns:
boolean more to parse?

isThereADataElement

public boolean isThereADataElement()
indicates if there is data element to parse

Specified by:
isThereADataElement in interface ITokenizer
Returns:
boolean data element ready?

getDataElementPos

public int getDataElementPos()
returns the position of the current dataelement being tokenized. Value is relative to zero.

Specified by:
getDataElementPos in interface ITokenizer
Returns:
int the current datalement position

makeSubfieldTokenizer

public com.americancoders.edi.IDataTokenizer makeSubfieldTokenizer()
returns a subfield tokeinzer

Specified by:
makeSubfieldTokenizer in interface ITokenizer
Returns:
IDataTokenizer

getCurrentTokenStringToken

public java.lang.String getCurrentTokenStringToken()
returns the segment string // used by edifact envelope parser

Returns:
String

reportError

public void reportError(java.lang.String inDescription)
adds an error description to the OBOEParserErrors object

Specified by:
reportError in interface ITokenizer
Parameters:
inDescription - description of the error,

reportError

public void reportError(java.lang.String inDescription,
                        com.americancoders.edi.SegmentContainer inRequestingContainer,
                        java.lang.String inCode,
                        java.lang.Object inObject,
                        int inLevel)
adds an error description to the OBOEParserErrors object

Specified by:
reportError in interface ITokenizer
Parameters:
inDescription - description of the error,
inRequestingContainer - segment, table...
inCode - err code
inObject - object reporting the error
inLevel - int error level

reportError

public void reportError(java.lang.String inDescription,
                        java.lang.String inCode)
adds an error description to the OBOEParserErrors object

Parameters:
inDescription - description of the error,
inCode - err code

getReportError

public com.americancoders.edi.DocumentErrors getReportError()
gets the DocumentErrors object

Returns:
DocumentErrors object

setLastSegmentContainer

public void setLastSegmentContainer(com.americancoders.edi.SegmentContainer inSC)
sets the last segment container that made a getNextSegment request

Parameters:
inSC - a SegmentContainer

getLastSegmentContainer

public com.americancoders.edi.SegmentContainer getLastSegmentContainer()
returns the last segment container that made a getNextSegment request

Returns:
SegmentContainer, maybe null

getEscapeCharacters

public java.lang.String getEscapeCharacters()
returns the escape character string

Specified by:
getEscapeCharacters in interface ITokenizer
Returns:
String

getSegmentID

public java.lang.String getSegmentID()
returns what is suppose to be the id of the segment being parsed.

Returns:
String

getRepeaterCharacter

public java.lang.String getRepeaterCharacter()
Description copied from interface: ITokenizer
returns the repeater delimiter

Specified by:
getRepeaterCharacter in interface ITokenizer
Returns:
String
See Also:
ITokenizer.getRepeaterCharacter()

getElementSeparator

public java.lang.String getElementSeparator()
Description copied from interface: ITokenizer
returns the element delimiter

Specified by:
getElementSeparator in interface ITokenizer
Returns:
String
See Also:
ITokenizer.getElementSeparator()

getInputByteCount

public int getInputByteCount()
returns the byte positin in the input stream of the current token

Specified by:
getInputByteCount in interface ITokenizer
Returns:
int

getSegmentPosition

public int getSegmentPosition()
returns the relative segment position

Specified by:
getSegmentPosition in interface ITokenizer
Returns:
int

toString

public java.lang.String toString()
returns the current token string.

Specified by:
toString in interface ITokenizer
Overrides:
toString in class java.lang.Object

Open Business Objects for EDI
v3.6.0


OBOE is a trademark or registered trademark of American Coders, Ltd. in the US and other countries.
Copyright 1999-2004 American Coders, Ltd.
POB 97462
Raleigh NC 27624 USA
All Rights Reserved.