Parsing AS2505/8583 Messages

Foreword

Previously I briefly touched on the AS2805 standards, and now I have an implementation of the parsing of these messages

The full code of this post is available here : https://github.com/Arthurvdmerwe/AS2805_Python_Implementation

I have a C# implementation of this as well here:

https://github.com/Arthurvdmerwe/AS2805

The code is thanks to the following author: http://www.vulcanno.com.br/python/ISO8583.html

He has a brilliant implementation of all the structures of the 8583 format, which goes hand in hand with AS2805

First off, the AS2805 standard includes a range of specifications, from Key handling settlement. This is not a post to explain all the details, as the key and settlement handling is a processor specific implementation.

AS2805 is extremely similar to the ISO8583 specification, and most of it can be taken directly out of this specification.

AS2805 Financial transaction card originated messages — Interchange message specifications is the International Organization for Standardization standard for systems that exchange electronic transactions made by cardholders using payment cards. It has three parts:

• Part 1: Messages, data elements and code values
• Part 2: Application and registration procedures for Institution Identification Codes (IIC)
• Part 3: Maintenance procedures for messages, data elements and code values

Introduction

A card-based transaction typically travels from a transaction acquiring device, such as a point-of-sale terminal or an automated teller machine (ATM), through a series of networks, to a card issuing system for authorization against the card holder’s account. The transaction data contains information derived from the card (e.g., the account number), the terminal (e.g., the merchant number), the transaction (e.g., the amount), together with other data which may be generated dynamically or added by intervening systems. The card issuing system will either authorize or decline the transaction and generate a response message which must be delivered back to the terminal within a predefined time period.

AS2805 defines a message format and a communication flow so that different systems can exchange these transaction requests and responses. The vast majority of transactions made at ATMs use AS2805 at some point in the communication chain, as do transactions made when a customer uses a card to make a payment in a store (EFTPOS). In particular, both the MasterCard andVisa networks base their authorization communications on the ISO 8583 standard, as do many other institutions and networks. AS2805 has no routing information, so is sometimes used with aTPDU header.

Cardholder-originated transactions include purchase, withdrawal, deposit, refund, reversal, balance inquiry, payments and inter-account transfers. AS2805 also defines system-to-system messages for secure key exchanges, reconciliation of totals, and other administrative purposes.

Although AS2805 defines a common standard, it is not typically used directly by systems or networks. It defines many standard fields (data elements) which remain the same in all systems or networks, and leaves a few additional fields for passing network specific details. These fields are used by each network to adapt the standard for its own use with custom fields and custom usages.

The placements of fields in different versions of the standard varies;

An AS2805 message is made of the following parts:

• Message type indicator (MTI)
• One or more bitmaps, indicating which data elements are present
• Data elements, the fields of the message

 

Message type indicator

This is a 4 digit numeric field which classifies the high level function of the message. A message type indicator includes the ISO 8583 version, the Message Class, the Message Function and the Message Origin, each described briefly in the following sections. The following example (MTI 0110) lists what each digit indicates:

  0xxx -> version of AS2805 
  x1xx -> class of the Message (Authorization Message)
  xx1x -> function of the Message (Request Response)
  xxx0 -> who began the communication (Acquirer)

AS2805 version

Position one of the MTI specifies the versions of the AS2805 standard which is being used to transmit the message.

Position	Meaning
0xxx	ISO 8583-1:1987 version
1xxx	ISO 8583-2:1993 version
2xxx	ISO 8583-1:2003 version
3xxx	Reserved for ISO use
4xxx	Reserved for ISO use
5xxx	Reserved for ISO use
6xxx	Reserved for ISO use
7xxx	Reserved for ISO use
8xxx	Reserved for National use
9xxx	Reserved for Private use

Message class

Position two of the MTI specifies the overall purpose of the message.

Position	Meaning	Usage
x1xx	Authorization Message	Determine if funds are available, get an approval but do not post to account for reconciliation, Dual Message System (DMS), awaits file exchange for posting to account
x2xx	Financial Messages	Determine if funds are available, get an approval and post directly to the account, Single Message System (SMS), no file exchange after this
x3xx	File Actions Message	Used for hot-card, TMS and other exchanges
x4xx	Reversal Message	Reverses the action of a previous authorization
x5xx	Reconciliation Message	Transmits settlement information message
x6xx	Administrative Message	Transmits administrative advice. Often used for failure messages (e.g. message reject or failure to apply)
x7xx	Fee Collection Messages
x8xx	Network Management Message	Used for secure key exchange, logon, echo test and other network functions
x9xx	Reserved by ISO

Message function

Position three of the MTI specifies the message function which defines how the message should flow within the system. Requests are end-to-end messages (e.g., from acquirer to issuer and back with timeouts and automatic reversals in place), while advices are point-to-point messages (e.g., from terminal to acquirer, from acquirer to network, from network to issuer, with transmission guaranteed over each link, but not necessarily immediately).

Position	Meaning
xx0x	Request
xx1x	Request Response
xx2x	Advice
xx3x	Advice Response
xx4x	Notification
xx8x	Response acknowledgment
xx9x	Negative acknowledgment

Message origin

Position four of the MTI defines the location of the message source within the payment chain.

Position	Meaning
xxx0	Acquirer
xxx1	Acquirer Repeat
xxx2	Issuer
xxx3	Issuer Repeat
xxx4	Other
xxx5	Other Repeat

Examples

Bearing each of the above four positions in mind, an MTI will completely specify what a message should do, and how it is to be transmitted around the network. Unfortunately, not all AS2805 implementations interpret the meaning of an MTI in the same way. However, a few MTIs are relatively standard:

MTI	Meaning	Usage
0100	Authorization request	Request from a point-of-sale terminal for authorization for a cardholder purchase
0110	Issuer Response	Issuer response to a point-of-sale terminal for authorization for a cardholder purchase
0120	Authorization Advice	When the Point of Sale device breaks down and you have to sign a voucher
0121	Authorisation Advice Repeat	If the advice times out
0130	Issuer Response to Authorization Advice	Confirmation of receipt of authorization advice
0200	Acquirer Financial Request	Request for funds, typically from an ATM or pinned point-of-sale device
0210	Issuer Response to Financial Request	Issuer response to request for funds
0220	Acquirer Financial Advice	e.g. Checkout at a hotel. Used to complete transaction initiated with authorization request
0221	Acquirer Financial Advice repeat	If the advice times out
0230	Issuer Response to Financial Advice	Confirmation of receipt of financial advice
0400	Acquirer Reversal Request	Reverses a transaction
0420	Acquirer Reversal Advice	Advises that a reversal has taken place
0421	Acquirer Reversal Advice Repeat Message	If the reversal times out
0430	Issuer Reversal Response	Confirmation of receipt of reversal advice
0800	Network Management Request	Echo test, logon, log off etc.
0810	Network Management Response	Echo test, logon, log off etc.
0820	Network Management Advice	Keychange

Bitmaps

Within AS2805, a bitmap is a field or subfield within a message which indicates which other data elements or data element subfields may be present elsewhere in a message.

A message will contain at least one bitmap, called the Primary Bitmap which indicates which of Data Elements 1 to 64 are present. A secondary bitmap may also be present, generally as data element one and indicates which of data elements 65 to 128 are present. Similarly, a tertiary, or third, bitmap can be used to indicate the presence or absence of fields 129 to 192, although these data elements are rarely used.

The bitmap may be transmitted as 8 bytes of binary data, or as 16 hexadecimal characters 0-9, A-F in the ASCII or EBCDIC character sets.

A field is present only when the specific bit in the bitmap is true. For example, byte ’82x is binary ‘1000 0010’ which means fields 1 and 7 are present in the message and fields 2, 3, 4, 5, 6, and 8 are not present.

---

Examples —–

Bitmap	Defines presence of
4210001102C04804	Fields 2, 7, 12, 28, 32, 39, 41, 42, 50, 53, 62
7234054128C28805	Fields 2, 3, 4, 7, 11, 12, 14, 22, 24, 26, 32, 35, 37, 41, 42, 47, 49, 53, 62, 64
8000000000000001	Fields 1, 64
0000000000000003 (secondary bitmap)	Fields 127, 128

Explanation of Bitmap (8 BYTE Primary Bitmap = 64 Bit) field 4210001102C04804
BYTE1 : 01000010 = 42x (counting from the left, the second and seventh bits are 1, indicating that fields 2 and 7 are present)
BYTE2 : 00010000 = 10x (field 12 is present)
BYTE3 : 00000000 = 00x (no fields present)
BYTE4 : 00010001 = 11x (fields 28 and 32 are present)
BYTE5 : 00000010 = 02x (field 39 is present)
BYTE6 : 11000000 = C0x (fields 41 and 42 are present)
BYTE7 : 01001000 = 48x (fields 50 and 53 are present)
BYTE8 : 00000100 = 04x (field 62 is present)

0________10________20________30________40________50________60__64
1234567890123456789012345678901234567890123456789012345678901234  n-th bit
0100001000010000000000000001000100000010110000000100100000000100  bit map

Fields present in the above variable length message record:
2-7-12-28-32-39-41-42-50-53-62

Data elements

Data elements are the individual fields carrying the transaction information. There are up to 128 data elements specified in the original AS2805 standard, and up to 192 data elements in later releases.

While each data element has a specified meaning and format, the standard also includes some general purpose data elements and system- or country-specific data elements which vary enormously in use and form from implementation to implementation.

Each data element is described in a standard format which defines the permitted content of the field (numeric, binary, etc.) and the field length (variable or fixed), according to the following table:

Abbreviation	Meaning
a	Alpha, including blanks
n	Numeric values only
s	Special characters only
an	Alphanumeric
as	Alpha & special characters only
ns	Numeric and special characters only
ans	Alphabetic, numeric and special characters.
b	Binary data
z	Tracks 2 and 3 code set as defined in ISO/IEC 7813 and ISO/IEC 4909 respectively
. or .. or …	variable field length indicator, each . indicating a digit.
x or xx or xxx	fixed length of field or maximum length in the case of variable length fields.

Additionally, each field may be either fixed or variable length. If variable, the length of the field will be preceded by a length indicator.

Type	Meaning
Fixed	no field length used
LLVAR or (..xx)	Where LL < 100, means two leading digits LL specify the field length of field VAR
LLLVAR or (…xxx)	Where LLL < 1000, means three leading digits LLL specify the field length of field VAR
LL and LLL are hex or ASCII. A VAR field can be compressed or ASCII depending of the data element type.	LL can be 1 or 2 bytes. For example, if compressed as one hex byte, ’27x means there are 27 VAR bytes to follow. If ASCII, the two bytes ’32x, ’37x mean there are 27 bytes to follow. 3 digit field length LLL uses 2 bytes with a leading ‘0’ nibble if compressed, or 3 bytes if ASCII. The format of a VAR data element depends on the data element type. If numeric it will be compressed, e.g. 87456 will be represented by 3 hex bytes ‘087456x. If ASCII then one byte for each digit or character is used, e.g. ’38x, ’37x, ’34x, ’35x, ’36x.

AS2805-defined data elements

Data element	Type	Usage
1	b 64	Bit map (b 128 if secondary is present and b 192 if tertiary is present)
2	n ..19	Primary account number (PAN)
3	n 6	Processing code
4	n 12	Amount, transaction
5	n 12	Amount, settlement
6	n 12	Amount, cardholder billing
7	n 10	Transmission date & time
8	n 8	Amount, cardholder billing fee
9	n 8	Conversion rate, settlement
10	n 8	Conversion rate, cardholder billing
11	n 6	Systems trace audit number
12	n 6	Time, local transaction (hhmmss)
13	n 4	Date, local transaction (MMDD)
14	n 4	Date, expiration
15	n 4	Date, settlement
16	n 4	Date, conversion
17	n 4	Date, capture
18	n 4	Merchant type
19	n 3	Acquiring institution country code
20	n 3	PAN extended, country code
21	n 3	Forwarding institution. country code
22	n 3	Point of service entry mode
23	n 3	Application PAN number
24	n 3	Function code (ISO 8583:1993)/Network International identifier (NII)
25	n 2	Point of service condition code
26	n 2	Point of service capture code
27	n 1	Authorizing identification response length
28	n 8	Amount, transaction fee
29	n 8	Amount, settlement fee
30	n 8	Amount, transaction processing fee
31	n 8	Amount, settlement processing fee
32	n ..11	Acquiring institution identification code
33	n ..11	Forwarding institution identification code
34	n ..28	Primary account number, extended
35	z ..37	Track 2 data
36	n …104	Track 3 data
37	an 12	Retrieval reference number
38	an 6	Authorization identification response
39	an 2	Response code
40	an 3	Service restriction code
41	ans 16	Card acceptor terminal identification
42	ans 15	Card acceptor identification code
43	ans 40	Card acceptor name/location (1-23 address 24-36 city 37-38 state 39-40 country)
44	an ..25	Additional response data
45	an ..76	Track 1 data
46	an …999	Additional data – ISO
47	an …999	Additional data – national
48	an …999	Additional data – private
49	an 3	Currency code, transaction
50	an 3	Currency code, settlement
51	an 3	Currency code, cardholder billing
52	b 64	Personal identification number data
53	n 18	Security related control information
54	an …120	Additional amounts
55	ans …999	Reserved ISO
56	ans …999	Reserved ISO
57	ans …999	Reserved national
58	ans …999	Reserved national
59	ans …999	Reserved for national use
60	an .7	Advice/reason code (private reserved)
61	ans …999	Reserved private
62	ans …999	Reserved private
63	ans …999	Reserved private
64	b 16	Message authentication code (MAC)
65	b 64	*Bit indicator of tertiary bitmap only*, tertiary bitmap data follows secondary in message stream.
66	n 1	Settlement code
67	n 2	Extended payment code
68	n 3	Receiving institution country code
69	n 3	Settlement institution country code
70	n 3	Network management information code
71	n 4	Message number
72	ans …999	Data record (ISO 8583:1993)/n 4 Message number, last(?)
73	n 6	Date, action
74	n 10	Credits, number
75	n 10	Credits, reversal number
76	n 10	Debits, number
77	n 10	Debits, reversal number
78	n 10	Transfer number
79	n 10	Transfer, reversal number
80	n 10	Inquiries number
81	n 10	Authorizations, number
82	n 12	Credits, processing fee amount
83	n 12	Credits, transaction fee amount
84	n 12	Debits, processing fee amount
85	n 12	Debits, transaction fee amount
86	n 15	Credits, amount
87	n 15	Credits, reversal amount
88	n 15	Debits, amount
89	n 15	Debits, reversal amount
90	n 42	Original data elements
91	an 1	File update code
92	n 2	File security code
93	n 5	Response indicator
94	an 7	Service indicator
95	an 42	Replacement amounts
96	an 8	Message security code
97	n 16	Amount, net settlement
98	ans 25	Payee
99	n ..11	Settlement institution identification code
100	n ..11	Receiving institution identification code
101	ans 17	File name
102	ans ..28	Account identification 1
103	ans ..28	Account identification 2
104	ans …100	Transaction description
105	ans …999	Reserved for ISO use
106	ans …999	Reserved for ISO use
107	ans …999	Reserved for ISO use
108	ans …999	Reserved for ISO use
109	ans …999	Reserved for ISO use
110	ans …999	Reserved for ISO use
111	ans …999	Reserved for ISO use
112	ans …999	Reserved for national use
113	n ..11	Authorizing agent institution id code
114	ans …999	Reserved for national use
115	ans …999	Reserved for national use
116	ans …999	Reserved for national use
117	ans …999	Reserved for national use
118	ans …999	Reserved for national use
119	ans …999	Reserved for national use
120	ans …999	Reserved for private use
121	ans …999	Reserved for private use
122	ans …999	Reserved for private use
123	ans …999	Reserved for private use
124	ans …255	Info text
125	ans ..50	Network management information
126	ans …999	Issuer trace id
127	ans …999	Reserved for private use
128	b 16	Message authentication code

Implementation and understanding the Code

The first step to understanding the packing and unpacking of the message fields would be to create a class that describes then entire structure.

First we create a class and create a dictionary that describes the message formats:

 #2805 contants
 _DEF = {}
 # Every _DEF has:
 # _DEF[N] = [X, Y, Z, W, K]
 # N = bitnumber
 # X = smallStr representation of the bit meanning
 # Y = large str representation
 # Z = length indicator of the bit (F, LL, LLL, LLLL, LLLLL, LLLLLL)
 # W = size of the information that N need to has
 # K = type os values a, an, ans, n, xn, b
 _DEF[1] = ['BM', 'Bit Map Extended', 'F', 8, 'b']
 _DEF[2] = ['2', 'Primary Account Number (PAN)', 'LL', 19, 'n']
 _DEF[3] = ['3', 'Processing Code', 'F', 6, 'n']
 _DEF[4] = ['4', 'Amount Transaction', 'F', 12, 'n']
 _DEF[5] = ['5', 'Amount Settlement', 'F', 12, 'n']
 _DEF[7] = ['7', 'Transmission Date and Time', 'F', 10, 'n']
 _DEF[9] = ['9', 'Conversion Rate, Settlement', 'F', 8, 'n']
 _DEF[10] = ['10', 'Conversion Rate, Cardholder Billing', 'F', 8, 'n']
 _DEF[11] = ['11', 'Systems Trace Audit Number', 'F', 6, 'n']
 _DEF[12] = ['12', 'Time, Local Transaction', 'F', 6, 'n']
 _DEF[13] = ['13', 'Date, Local Transaction', 'F', 4, 'n']
 _DEF[14] = ['14', 'Date, Expiration', 'F', 4, 'n']
 _DEF[15] = ['15', 'Date, Settlement', 'F', 4, 'n']
 _DEF[16] = ['16', 'Date, Conversion', 'F', 4, 'n']
 _DEF[18] = ['18', 'Merchant Type', 'F', 4, 'n']
 _DEF[22] = ['22', 'POS Entry Mode', 'F', 3, 'n']
 _DEF[23] = ['23', 'Card Sequence Number', 'F', 3, 'n']
 _DEF[25] = ['25', 'POS Condition Code', 'F', 2, 'n']
 _DEF[28] = ['28', 'Amount, Transaction Fee', 'F', 9, 'xn']
 _DEF[32] = ['32', 'Acquiring Institution ID Code', 'LL', 11, 'n']
 _DEF[33] = ['33', 'Forwarding Institution ID Code', 'LL', 11, 'n']
 _DEF[35] = ['35', 'Track 2 Data', 'LL', 37, 'an']
 _DEF[37] = ['37', 'Retrieval Reference Number', 'F', 12, 'an']
 _DEF[38] = ['38', 'Authorization ID Response', 'F', 6, 'an']
 _DEF[39] = ['39', 'Response Code', 'F', 2, 'an']
 _DEF[41] = ['41', 'Card Acceptor Terminal ID', 'F', 8, 'ans']
 _DEF[42] = ['42', 'Card Acceptor ID Code', 'F', 15, 'ans']
 _DEF[43] = ['43', 'Card Acceptor Name Location', 'F', 40, 'asn']
 _DEF[44] = ['44', 'Additional Response Data', 'LL', 25, 'ans']
 _DEF[47] = ['47', 'Additional Data National', 'LLL', 999, 'ans']
 _DEF[48] = ['48', 'Additional Data Private', 'LLL', 999, 'ans']
 _DEF[49] = ['49', 'Currency Code, Transaction', 'F', 3, 'n']
 _DEF[50] = ['50', 'Currency Code, Settlement', 'F', 3, 'n']
 _DEF[51] = ['51', 'Currency Code, Billing', 'F', 3, 'n']
 _DEF[52] = ['52', 'PIN Data', 'F', 8, 'b']
 _DEF[53] = ['53', 'Security Related Control Information', 'F', 48, 'b']
 _DEF[55] = ['55', 'ICC Data', 'LLL', 999, 'b']
 _DEF[57] = ['57', 'Amount Cash', 'F', 12, 'n']
 _DEF[58] = ['58', 'Ledger Balance', 'F', 12, 'n']
 _DEF[59] = ['59', 'Account Balance', 'F', 12, 'n']
 _DEF[64] = ['64', 'Message Authentication Code', 'F', 8, 'b']
 _DEF[66] = ['66', 'Settlement Code', 'F', 1, 'n']
 _DEF[70] = ['70', 'Network Management Information Code', 'F', 3, 'n']
 _DEF[74] = ['74', 'Credits, Number', 'F', 10, 'n']
 _DEF[75] = ['75', 'Credits, Reversal Number', 'F', 10, 'n']
 _DEF[76] = ['76', 'Debits, Number', 'F', 10, 'n']
 _DEF[77] = ['77', 'Debits, Reversal Number', 'F', 10, 'n']
 _DEF[78] = ['78', 'Transfer, Number', 'F', 10, 'n']
 _DEF[79] = ['79', 'Transfer, Reversal Number', 'F', 10, 'n']
 _DEF[80] = ['80', 'Inquiries, Number', 'F', 10, 'n']
 _DEF[81] = ['81', 'Authorizations, Number', 'F', 10, 'n']
 _DEF[83] = ['83', 'Credits, Transaction Fee Amount', 'F', 12, 'n']
 _DEF[85] = ['85', 'Debits, Transaction Fee Amount', 'F', 12, 'n']
 _DEF[86] = ['86', 'Credits, Amount', 'F', 16, 'n']
 _DEF[87] = ['87', 'Credits, Reversal Amount', 'F', 16, 'n']
 _DEF[88] = ['88', 'Debits, Amount', 'F', 16, 'n']
 _DEF[89] = ['89', 'Debits, Reversal Amount', 'F', 16, 'n']
 _DEF[90] = ['90', 'Original Data Elements', 'F', 42, 'n']
 _DEF[97] = ['97', 'Amount, Net Settlement', 'F', 17, 'xn']
 _DEF[99] = ['99', 'Settlement Institution ID Code', 'LL', 11, 'n']
 _DEF[100] = ['100', 'Receiving Institution ID Code', 'LL', 11, 'n']
 _DEF[112] = ['112', 'Key Management Data', 'LLL', 999, 'b']
 _DEF[118] = ['118', 'Cash Total Number', 'LLL', 10, 'n']
 _DEF[119] = ['119', 'Cash Total Amount', 'LLL', 10, 'n']
 _DEF[128] = ['128', 'MAC Extended', 'F', 8, 'b']

To write this structure should not be that difficult and should come directly out of the specification provided my your institution.

As for the infamous bitmaps we are going to describe a structure for them as well, with an array to track the bit positions.

 #Attributes
 # Bits to be set 00000000 -> _BIT_POSITION_1 ... _BIT_POSITION_8
 _BIT_POSITION_1 = 128 # 10 00 00 00
 _BIT_POSITION_2 = 64 # 01 00 00 00
 _BIT_POSITION_3 = 32 # 00 10 00 00
 _BIT_POSITION_4 = 16 # 00 01 00 00
 _BIT_POSITION_5 = 8 # 00 00 10 00
 _BIT_POSITION_6 = 4 # 00 00 01 00
 _BIT_POSITION_7 = 2 # 00 00 00 10
 _BIT_POSITION_8 = 1 # 00 00 00 01

 #Array to translate bit to position
 _TMP = [0, _BIT_POSITION_8, _BIT_POSITION_1, _BIT_POSITION_2, _BIT_POSITION_3, _BIT_POSITION_4, _BIT_POSITION_5,
 _BIT_POSITION_6, _BIT_POSITION_7]
 _EMPTY_VALUE = 0

Now we simply add a heap of helper functions from  http://www.vulcanno.com.br/python/ISO8583.html to fill and decode the data structure.

I have created a client and a server as part of the source code, so you can test your own implementation of the AS2805 protocol

Simple as pie!