Typical Cryptography in AS2805 Explained

Key Management conforms to AS 2805 part 6.1.

KEK Establishment

Each interchange node contains an Interchange Send Key Encrypting Key (KEKs) and an Interchange Receive Key Encrypting Key (KEKr). The Interchange Send KEK is the same key as the Interchange Receive KEK in the partnering node, similarly the Interchange Receive KEK is the same as the Interchange Send KEK in the partnering node.

The Interchange Key Encrypting Keys are used to encipher and decipher the session keys when they are transmitted between the nodes and in the proof of end points process.

Interchange Key Encrypting Keys is statistically unique and shall be changed, at a minimum, once every two years.


Node A Node B
KEkr = KEKs

Session Keys

Each node keeps four sets of session keys, two send sets and two receive sets.

Each set of session keys consists of two keys, MAC Key, PIN Protect Key. Each session key is 128-bits long and stored in a secure manner.

The send session key sets are generated by the sending node and numbered “1” or “2”. The send session key sets are then forwarded to the receiving node to be used as the receive session key sets.

The receive session key sets are received in a 0820 Network Management Advice message with bit ‘070’ equal to 101 from the sending node. The set number of either “1” or “2” contained in bit 53 indicates the receive session key set used by the receiving node to verify the MAC, decipher the data and translate or verify the PIN.

One set of send session keys is used at a time and all Transactions sent from the sending node will generate the MAC and encipher the PIN, if present, using the MAC Generator Key and PIN Protect Key, respectively, from the same send session key set. The send session key set used is indicated by bit 53 (contains “1” or “2”) in each message. Session Keys must be statistically unique and replaced, at a minimum, once every hour or on every 256 Transactions, whichever occurs first.



Node A Node B
Send Session Keys Set 1 Receive Session Keys Set 1
MAC Key (KMACs1) = MAC Verification Key (KMACr1)
PIN Protect Key (KPEs1) = PIN Protect Key (KPEr1)
Send Session Keys Set 2 Receive Session Keys Set 2
MAC Key (KMACs2) = MAC Verification Key (KMACr2)
PIN Protect Key (KPEs2) = PIN Protect Key (KPEr2)
Receive Session Keys Set 1 Send Session Keys Set 1
MAC Verification Key (KMACr1) = MAC Key (KMACs1)
PIN Protect Key (KPEr1) = PIN Protect Key (KPEs1)
Receive Session Keys Set 2 Send Session Keys Set 2
MAC Verification Key (KMACr2) = MAC Key (KMACs2)
PIN Protect Key (KPEr2) = PIN Protect Key (KPEs2)


When enciphered for transmission, each session key type will use a unique variant of the Key Enciphering Key in accordance with AS 2805 part 6.1 request response (logon) from the other before starting any other message exchange. When ready to logon, a party should attempt to logon and continue to attempt to logon until a successful response has been received. Upon receipt of an unsolicited logon (i.e. receiving a logon message when in an assumed logged on state) or a message with a response code indicating an irrecoverable error, a party should send an immediate logoff message and attempts to logon should be made as soon as possible. All logon response messages should be inspected to ensure that the response code indicates a successful logon

Changing Session keys

While one set of send session keys is being used, the other send session key set is randomly generated by the sending node and their KVCs generated, the keys are then enciphered under the Interchange Send KEK and transmitted to the receiving node in a 0820 Network Management Advice message.

When a 0820 message is received by the receiving node, the session keys are deciphered using the Interchange Receive KEK. These deciphered keys are set up as the set of receive keys specified by the set number contained in bit 53 of the 0820 message. The Key Verification Codes (KVCs) are calculated by the receiving node and transmitted to the sending node in bit 48 of the 0830 message.

When the 0830 Network Management Advice response message is received at the node initiating the key change, the KVCs contained in the 0830 message are validated. If the KVCs are correct, the new send session key set can be used immediately. If the KVCs are invalid, new send session key set must be generated and the whole process is repeated.


Sign off

Either node may terminate the transmission of financial messages by sending a Sign Off Advice. A Sign Off is accomplished by the transmission of a 0820 Network Management Advice Message with a NMIC (Bit 70) equal to ‘002’.


Key change during normal processing

A session key change can occur at any time; each node independently initiates the change of their send keys. The sender will advise their sending session keys to the receiver using a 0820 Network Management Advice message with a NMIC equal to ‘101’ indicating key change. Once a valid response (0830 message) is received and the KVCs confirmed, the new keys can be used.

Thales 9000 with AS2805 Interchange & RSA EFTPOS Commands.

Interchange Cryptographic Keys 

Interchange keys are used to protect financial transactions initiated at Acquirer eftpos / ATM Terminals while in transit to the Issuer institution. Interchange keys may be either:

(a) PIN encrypting keys – used to protect the customer PIN from the point of origin to the point of authorisation. PIN encrypting keys are a specific instance of session keys;

(b) Session keys – used to secure, validate and protect the financial message. Session keys can be further qualified into those used in the terminal to Acquirer environment (terminal session keys) or on node to node links (interchange session keys);

(c) Key Encrypting Keys (KEK) – used to protect other keys (e.g. session keys) during exchange; or

(d) Transport Keys – used to protect keys (e.g. KEKs) during transport to the partner institution.

Cryptographic Algorithms 

DEA3 and DEA2 are the only approved algorithms for the protection of interchange information (full details of these algorithms may be found in the Australian standard AS 2805 part 5).

DEA3 keys are 128 bits in length (effectively 112 bits) and are generally referred to as triple DES or 3DES keys (the corresponding encryption algorithm is specified in AS 2805 part 5.4). Triple DES may also be acceptably implemented using a key length of 192 bits (effectively 168 bits).

DEA3 with a key length of 128 bits and DEA2 with key lengths equal to, or greater than 2048 bits are the minimum acceptable requirements for the effective protection of interchange information at the time of the issuance of this document.

In accordance with AS 2805 part 3, DEA3 must be used for PIN encipherment.

 Interchange Links 

For all Interchange Links, Issuers and Acquirers must ensure that:

(a) Security for Transactions processed over that Interchange Link complies with AS2805 Part 6;

(b) Message formats comply with AS2805 Part 2;

(c) Security of transactions from terminal to Acquirer and from Acquirer to Issuer complies with AS2805 Part 6;

(d) PIN security and encryption complies with AS2805 Parts 3 and 5.4;

(e) Key management practices comply with AS2805 Part 6.1;

In each case and as more particularly set out in Part 8:

(a) Message Authentication must apply to all Interchange Links;

(b) The Message Authentication Code (MAC) must be calculated using, as a minimum, a DEA 3 (128-bit) key, Triple DES and an algorithm conforming to AS2805 Part 4; and

(c) all interchange PIN and MAC cryptographic functions must be performed within a Tamper-responsive SCM

The Actual process using an Thales 9000 HSM (CECS Approved)

Now what we are clear on the actual requirements of CECS and APCA, lets  attempt to do this using  a Thales 9000.

Generate a Sponsor RSA key pair

This command is the first step as would be required to do this for all terminal commands.

  • This is done my using the HSM EI host Command, from the HSM base manual.
    • The input is the length of the RSA key set required,  and the length go the public key modulus.
  • The Public Key Verification Code should now be generated. This is done using the HSM H2 Command from the Australian Standards Support Manual.

The Public Key and the PVC are sent to your Interchange Partner via different paths, as per their direction. (lets call this OUR-Key and OUR-PVC)

Your Interchange partner will now do the same process and provide you with a Public Key and a PVC. (lets call this THEIR-Key and THEIR-PVC)

When we receive this Public Key from our Interchange Partner, the following should happen:

  • The PVC for the Key should be generated using the HSM H2 Command from the Australian Standards Support Manual.
  • The MAC for the Key should be generated using the HSM EO command from the HSM Base Manual.

We now have public keys exchanged and have them ready for use!!

Our Database should be looking like this:


Now we have the Public keys exchanged and ready for use, we can generate our KEKs & send to Interchange Partner, and receive our KEKr from Interchange Partner;

  • To send our KEKs we will use the H4 command from the Australian Standards support manual.
  • To receive our KEKr we will use the H6 command from the Australian Standards support manual.

Once these are decrypted and stored in our key database we can generate and exchange our session MAC and PIN keys.

    • To generate and store our send keys we use the OI command from the Australian Standards support manual.
    • To receive and store our receive keys we use the OK command from the Australian Standards support manual.

Now we have all the keys in place we can start to process transactions.

    • To generate the MAC on a message there are a number of commands available, however as we are using the AS2805 standards we always recommend our customers use the C2 command from the Australian Standards support manual. This provides all the options required for the Australian environment.

Similarly to verify the MAC on a message there are a number of commands available, however as we are using the AS2805 standards we always recommend our customers use the C4 command from the Australian Standards support manual. This provides all the options required for the Australian environment.

Terminal Commands

Terminal Manufacturer will be injecting into the PINpads their Manufacturer Public Key. The MPK will be transmitted to SPONSOR securely. The MPK validity should be checked by verifying the PVC, this is achieved by generating a Public Key Verification Code This is done using the H2 command from the Australian Standards support manual. And the two values compared.

  • We also need to generate a PPASN, this is achieved using the AS2805 PK command.
  • The host will now send the SPK to the PINpad, the PINpad will now generate the KI (also known as KTI), and send to the host. This is recovered using the AS2805 host H8 command, which also returns the KCA, the KCA is encrypted under the LMK and the KTI.
  • Now we have the MPK and have verified it is genuine, we now need to generate a MAC for the Public Key, this is achieved using the Host EO command, this is used in subsequent processing. Note: this command is only available when the HSM is in Authorised State. We can now recover the PINpad Public from the MSK. This is achieved using the AS2805 H0 host command.
  • KCA is now used to create the TMK1 and TMK2 (also known as KEK1 & KEK2). These are generated using the C0 command.
  • Now we have the TMK’s in place we can use the TMK update commands.

Updating the Keys

  • When updating only TMK1 the AS2805 OU command is used.
  • When updating both TMK1 and TMK2 then the OW command is used.

Now we have the TMK’s in place and able to be updated, we can generate the Session Keys to be used for the PIN, MAC & optional encryption keys if required.

This is achieved using the AS2805 PI command. The PI command will generate the PIN, MAC, and optional Encryption keys.

  • Now we can have the session keys in place we can Decrypt the data, verify the MAC & verify the pin. The decrypt data & verify MAC steps depend on how it has been handled by the terminal. Has the terminal done the MAC first then encrypted the required data or has the terminal encrypted the data & then done the MAC. We have assumed that the Encrypt was done first.
  • Verify the MAC’s on the transactions from the terminal using the AS2805 C4.
  • Once the MAC has been verified we can then decrypt the required data with the AS2805 host command PW.
  • Now we have the required decrypted data you will need to either verify the PIN or Translate the PIN, to translate the PIN assuming the transaction is a debit card transaction. This is achieved using the AS2805 PO host command. To verify the PIN will use one of the following F0 or F2.

If you have translated the PIN we can form the message and generate a MAC for the message to be sent to Interchange Partner, this is achieved using the C2 command as detailed above in the Interchange messages.

The biggest problem we see with this are around the KEKs & KEKr is people get them around the wrong way. Your KEKs becomes the remote KEKr & vice versa. The AS2805 commands are designed to swap them over automatically. 

The other gotcha is we split the terminal side & the interchange side of the HSM, TMK (terminal master key) is like a KEK (ZMK (Zone master key)) but used on the terminal side of the network where a ZMK (KEKs & KEKr) is used for interchange side of the network.

 easy as Pie!

Thales Key Exchange Examples and Troubleshooting

Judging from the searches done to locate this blog, it’s clear many of us share the following opinion: although Thales (formerly RACAL) is a market leader with its 7000 and 8000 series of HSM devices, their documentation falls painfully short in two areas: there are NO COMMAND EXAMPLES (!!!) in the manuals (an appalling omission); and the troubleshooting assistance is also distressingly thin. As a result, we had to lean heavily on our local Thales distributor for guidance on how commands really get pieced together. And, man, is it ever esoteric….see my earlier posts regarding the ‘KSN Descriptor’ for evidence of that. Moreover, our distributor provided us with some outstanding troubleshooting support to get us through the all important FA/FB key exhange. So, I’ll post our experience here in hopes that others can benefit from it.

The FA/FB is the command exchange to “Translate a ZPK from ZMK to LMK Encryption.” We had this working internally with some simulated stuff, but once we tried this command ‘for real’ with our Debit/EBT gateway partner , we consistently received parity errors back from our Thales 8000 HSM (i.e., the FB returned with some result code != 0).

There are three possible issues / resolution paths to explore in these situations:

Your switching partner has employed an Atalla HSM and you’ve not taken it into account. When a Thales/RACAL HSM ‘talks’ to an Atalla, your box commands must specify an Atalla Variant.

Your switching partner didn’t specify a Key Scheme in its ZPK creation, and the default is X9.17 (‘X’). If you specify the RACAL Scheme (‘U’) in your ‘FA’ and the ZPK under ZMK provided to the box was created via the X9.17 scheme, you’ll get a parity error.

You created the ZMK cryptogram internally using one key scheme and now are trying to employ it in the FA command specifying (inadvertantly) the other variant scheme.

You want to try to resolve each of these in turn.

The “test solution” path for Item #1 is as follows…

Let’s assume your FA command is constructed like so:


[NOTE: I’ve obscured the ZMK Cryptogram here for blogging purposes only. The actual value is a 32-position hexidecimal string.]

If you discover that the other side is using an Atalla, you need to specify an “Atalla Variant,” which you do by specifying a ‘1’ at the end of the ‘FA’ command string:


The “test solution” path for Item #2 is as follows…

We find some endpoint partners have no idea which Key Scheme (X9.17 or Racal Native) they employed to create their keys. So, you may have to experiment. This FA command string says that the ZMK and ZPK were created using the RACAL native scheme (‘U’):


To specify that the ZPK was created using the X9.17 scheme, you’d do the following:


The “test solution” path for Item #3 is as follows…

When you created your ZMK (probably during a key ceremony involving a reconstitution of key parts provided by your endpoint gateway), you specify (or otherwise let default) your ZMK Key Scheme. Again, this can be either the RACAL Native Scheme (‘U’) or X9.17 (‘X’). [I believe the default is X9.17.] For example, if you created the ZMK using the ‘X’ approach, and then submitted an ‘FA’ command that looks like this:


…it’s gonna fail. You absolutely must maintain consistency from the creation of the ZMK cryptogram through its subsequent usage. So, the command would change to look like this:


[NOTE: I’m not advocating ‘X’ over ‘U’ here…just showing you an example. In a recent concluded project, we assumed the incoming ZPK was of the RACAL native variety, but it arrived from the remote partner as created under the X9.17 scheme. Thing is, the gateway team had no idea they had done that and could not articulate the difference. So, be prepared to experiment with every permutation of what I’ve described herein before ‘unlocking’ the solution.]

No comments

Testing DUKPT

As an acquirer, you can validate that your PIN translation command is working correctly even if you haven’t yet established connectivity to your Debit/EBT endpoint (or if you’ve established connectivity but don’t yet have your test key parts).

Typically when we’re pressed into this situation, this is what we do if we’re working with the Key-Up II HSM from Ian Donnelly Systems (‘IDS’):

For simplicity’s sake, we use a weak double-length key part.

We enter it as a single part (again, at this point, we want to keep things simple).
Typically, we use 0123456789ABCDEFFEDCBA9876543210.

We take the Cryptogram obtained out of that exercise and we use that as our Key Exchange Key (“KEK”) input.

Now, in my reading of the Thales manuals, performing this exercise with the Thales device is quite a bit more complicated.

[I’ll use Thales language here so we’re all on the same page…]

GOAL: What you need to test PIN translation is a ZPK encrypted under your LMK. [Refer to my previous post for details on the CI/CJ PIN transalation request/response.]

Note specifically where I describe “PIN Length” on the CJ command output I say:

“Although this field is not used to build the 0200 message formatted for the processor gateway, a value like ‘04’ or ‘05’ here is a pretty good indication that the translation occurred successfully.”

So, we can validate the workability of this command without a working endpoint or ‘real’ key parts from them, but – in order to do so – we need to construct a valid ZPK under LMK cryptogram. This value will allow you to create a completely valid CI command, assuming that:

You’ve got one or more valid BDK cryptograms loaded in your encryption database.

You can generate PIN-enabled Debit purchases from a test point-of-sale location.

You have addressibility from your test server to the Thales.

STEPS: My reading of the Thales manual is that you’ll need to execute the following steps (this is sort of ugly – numbers refer to specific sections in the Operations and Installation manual):

6.1 to create at least two 32-positions ZMK components (or, alternatively, 6.3 if you want to use components of your own chosing).

6.4 to get a 32-position ZMK under LMK.

7.1 to generate random ZPK (uses 6.4 ZMK under LMK output as input here) and encrypt under ZMK and LMK. The goal is that second cryptogram that comes back, the one oddly named “ZPK encrypted for bank” in the Thales document. That’s the ZPK under the LMK that you need for your tests.

Obtaining this information should allow you to validate the critical part of your encryption infrastructure prior to getting connectivity to your Debit/EBT gateway partner. Of course, once formal inter-system testing is established you still need to transport the encrypted PIN block in an 0200 request to your endpoint. But doing the exercise I describe above here first will allow you to eliminate any self-doubt you may have about your internal PIN translation activities prior to getting an external party involved in the testing

No comments

Parsing AS2505/8583 Messages


Previously I briefly touched on the AS2805 standards, and now I have an implementation of the parsing of these messages

The full code of this post is available here : https://github.com/Arthurvdmerwe/AS2805_Python_Implementation

I have a C# implementation of this as well here:


The code is thanks to the following author: http://www.vulcanno.com.br/python/ISO8583.html

He has a brilliant implementation of all the structures of the 8583 format, which goes hand in hand with AS2805

First off, the AS2805 standard includes a range of specifications, from Key handling settlement. This is not a post to explain all the details, as the key and settlement handling is a processor specific implementation.

AS2805 is extremely similar to the ISO8583 specification, and most of it can be taken directly out of this specification.

AS2805 Financial transaction card originated messages — Interchange message specifications is the International Organization for Standardization standard for systems that exchange electronic transactions made by cardholders using payment cards. It has three parts:

  • Part 1: Messages, data elements and code values
  • Part 2: Application and registration procedures for Institution Identification Codes (IIC)
  • Part 3: Maintenance procedures for messages, data elements and code values


A card-based transaction typically travels from a transaction acquiring device, such as a point-of-sale terminal or an automated teller machine (ATM), through a series of networks, to a card issuing system for authorization against the card holder’s account. The transaction data contains information derived from the card (e.g., the account number), the terminal (e.g., the merchant number), the transaction (e.g., the amount), together with other data which may be generated dynamically or added by intervening systems. The card issuing system will either authorize or decline the transaction and generate a response message which must be delivered back to the terminal within a predefined time period.

AS2805 defines a message format and a communication flow so that different systems can exchange these transaction requests and responses. The vast majority of transactions made at ATMs use AS2805 at some point in the communication chain, as do transactions made when a customer uses a card to make a payment in a store (EFTPOS). In particular, both the MasterCard andVisa networks base their authorization communications on the ISO 8583 standard, as do many other institutions and networks. AS2805 has no routing information, so is sometimes used with aTPDU header.

Cardholder-originated transactions include purchase, withdrawal, deposit, refund, reversal, balance inquiry, payments and inter-account transfers. AS2805 also defines system-to-system messages for secure key exchanges, reconciliation of totals, and other administrative purposes.

Although AS2805 defines a common standard, it is not typically used directly by systems or networks. It defines many standard fields (data elements) which remain the same in all systems or networks, and leaves a few additional fields for passing network specific details. These fields are used by each network to adapt the standard for its own use with custom fields and custom usages.

The placements of fields in different versions of the standard varies;

An AS2805 message is made of the following parts:

  • Message type indicator (MTI)
  • One or more bitmaps, indicating which data elements are present
  • Data elements, the fields of the message


Message type indicator

This is a 4 digit numeric field which classifies the high level function of the message. A message type indicator includes the ISO 8583 version, the Message Class, the Message Function and the Message Origin, each described briefly in the following sections. The following example (MTI 0110) lists what each digit indicates:

  0xxx -> version of AS2805 
  x1xx -> class of the Message (Authorization Message)
  xx1x -> function of the Message (Request Response)
  xxx0 -> who began the communication (Acquirer)

AS2805 version

Position one of the MTI specifies the versions of the AS2805 standard which is being used to transmit the message.

Position Meaning
0xxx ISO 8583-1:1987 version
1xxx ISO 8583-2:1993 version
2xxx ISO 8583-1:2003 version
3xxx Reserved for ISO use
4xxx Reserved for ISO use
5xxx Reserved for ISO use
6xxx Reserved for ISO use
7xxx Reserved for ISO use
8xxx Reserved for National use
9xxx Reserved for Private use

Message class

Position two of the MTI specifies the overall purpose of the message.

Position Meaning Usage
x1xx Authorization Message Determine if funds are available, get an approval but do not post to account for reconciliation, Dual Message System (DMS), awaits file exchange for posting to account
x2xx Financial Messages Determine if funds are available, get an approval and post directly to the account, Single Message System (SMS), no file exchange after this
x3xx File Actions Message Used for hot-card, TMS and other exchanges
x4xx Reversal Message Reverses the action of a previous authorization
x5xx Reconciliation Message Transmits settlement information message
x6xx Administrative Message Transmits administrative advice. Often used for failure messages (e.g. message reject or failure to apply)
x7xx Fee Collection Messages
x8xx Network Management Message Used for secure key exchange, logon, echo test and other network functions
x9xx Reserved by ISO

Message function

Position three of the MTI specifies the message function which defines how the message should flow within the system. Requests are end-to-end messages (e.g., from acquirer to issuer and back with timeouts and automatic reversals in place), while advices are point-to-point messages (e.g., from terminal to acquirer, from acquirer to network, from network to issuer, with transmission guaranteed over each link, but not necessarily immediately).

Position Meaning
xx0x Request
xx1x Request Response
xx2x Advice
xx3x Advice Response
xx4x Notification
xx8x Response acknowledgment
xx9x Negative acknowledgment

Message origin

Position four of the MTI defines the location of the message source within the payment chain.

Position Meaning
xxx0 Acquirer
xxx1 Acquirer Repeat
xxx2 Issuer
xxx3 Issuer Repeat
xxx4 Other
xxx5 Other Repeat


Bearing each of the above four positions in mind, an MTI will completely specify what a message should do, and how it is to be transmitted around the network. Unfortunately, not all AS2805 implementations interpret the meaning of an MTI in the same way. However, a few MTIs are relatively standard:

MTI Meaning Usage
0100 Authorization request Request from a point-of-sale terminal for authorization for a cardholder purchase
0110 Issuer Response Issuer response to a point-of-sale terminal for authorization for a cardholder purchase
0120 Authorization Advice When the Point of Sale device breaks down and you have to sign a voucher
0121 Authorisation Advice Repeat If the advice times out
0130 Issuer Response to Authorization Advice Confirmation of receipt of authorization advice
0200 Acquirer Financial Request Request for funds, typically from an ATM or pinned point-of-sale device
0210 Issuer Response to Financial Request Issuer response to request for funds
0220 Acquirer Financial Advice e.g. Checkout at a hotel. Used to complete transaction initiated with authorization request
0221 Acquirer Financial Advice repeat If the advice times out
0230 Issuer Response to Financial Advice Confirmation of receipt of financial advice
0400 Acquirer Reversal Request Reverses a transaction
0420 Acquirer Reversal Advice Advises that a reversal has taken place
0421 Acquirer Reversal Advice Repeat Message If the reversal times out
0430 Issuer Reversal Response Confirmation of receipt of reversal advice
0800 Network Management Request Echo test, logon, log off etc.
0810 Network Management Response Echo test, logon, log off etc.
0820 Network Management Advice Keychange


Within AS2805, a bitmap is a field or subfield within a message which indicates which other data elements or data element subfields may be present elsewhere in a message.

A message will contain at least one bitmap, called the Primary Bitmap which indicates which of Data Elements 1 to 64 are present. A secondary bitmap may also be present, generally as data element one and indicates which of data elements 65 to 128 are present. Similarly, a tertiary, or third, bitmap can be used to indicate the presence or absence of fields 129 to 192, although these data elements are rarely used.

The bitmap may be transmitted as 8 bytes of binary data, or as 16 hexadecimal characters 0-9, A-F in the ASCII or EBCDIC character sets.

A field is present only when the specific bit in the bitmap is true. For example, byte ’82x is binary ‘1000 0010’ which means fields 1 and 7 are present in the message and fields 2, 3, 4, 5, 6, and 8 are not present.

Examples —–

Bitmap Defines presence of
4210001102C04804 Fields 2, 7, 12, 28, 32, 39, 41, 42, 50, 53, 62
7234054128C28805 Fields 2, 3, 4, 7, 11, 12, 14, 22, 24, 26, 32, 35, 37, 41, 42, 47, 49, 53, 62, 64
8000000000000001 Fields 1, 64
(secondary bitmap)
Fields 127, 128

Explanation of Bitmap (8 BYTE Primary Bitmap = 64 Bit) field 4210001102C04804
BYTE1 : 01000010 = 42x (counting from the left, the second and seventh bits are 1, indicating that fields 2 and 7 are present)
BYTE2 : 00010000 = 10x (field 12 is present)
BYTE3 : 00000000 = 00x (no fields present)
BYTE4 : 00010001 = 11x (fields 28 and 32 are present)
BYTE5 : 00000010 = 02x (field 39 is present)
BYTE6 : 11000000 = C0x (fields 41 and 42 are present)
BYTE7 : 01001000 = 48x (fields 50 and 53 are present)
BYTE8 : 00000100 = 04x (field 62 is present)

1234567890123456789012345678901234567890123456789012345678901234  n-th bit
0100001000010000000000000001000100000010110000000100100000000100  bit map

Fields present in the above variable length message record:

Data elements

Data elements are the individual fields carrying the transaction information. There are up to 128 data elements specified in the original AS2805 standard, and up to 192 data elements in later releases.

While each data element has a specified meaning and format, the standard also includes some general purpose data elements and system- or country-specific data elements which vary enormously in use and form from implementation to implementation.

Each data element is described in a standard format which defines the permitted content of the field (numeric, binary, etc.) and the field length (variable or fixed), according to the following table:

Abbreviation Meaning
a Alpha, including blanks
n Numeric values only
s Special characters only
an Alphanumeric
as Alpha & special characters only
ns Numeric and special characters only
ans Alphabetic, numeric and special characters.
b Binary data
z Tracks 2 and 3 code set as defined in ISO/IEC 7813 and ISO/IEC 4909 respectively
. or .. or … variable field length indicator, each . indicating a digit.
x or xx or xxx fixed length of field or maximum length in the case of variable length fields.

Additionally, each field may be either fixed or variable length. If variable, the length of the field will be preceded by a length indicator.

Type Meaning
Fixed no field length used
LLVAR or (..xx) Where LL < 100, means two leading digits LL specify the field length of field VAR
LLLVAR or (…xxx) Where LLL < 1000, means three leading digits LLL specify the field length of field VAR
LL and LLL are hex or ASCII. A VAR field can be compressed or ASCII depending of the data element type. LL can be 1 or 2 bytes. For example, if compressed as one hex byte, ’27x means there are 27 VAR bytes to follow. If ASCII, the two bytes ’32x, ’37x mean there are 27 bytes to follow. 3 digit field length LLL uses 2 bytes with a leading ‘0’ nibble if compressed, or 3 bytes if ASCII. The format of a VAR data element depends on the data element type. If numeric it will be compressed, e.g. 87456 will be represented by 3 hex bytes ‘087456x. If ASCII then one byte for each digit or character is used, e.g. ’38x, ’37x, ’34x, ’35x, ’36x.
AS2805-defined data elements
Data element Type Usage
1 b 64 Bit map (b 128 if secondary is present and b 192 if tertiary is present)
2 n ..19 Primary account number (PAN)
3 n 6 Processing code
4 n 12 Amount, transaction
5 n 12 Amount, settlement
6 n 12 Amount, cardholder billing
7 n 10 Transmission date & time
8 n 8 Amount, cardholder billing fee
9 n 8 Conversion rate, settlement
10 n 8 Conversion rate, cardholder billing
11 n 6 Systems trace audit number
12 n 6 Time, local transaction (hhmmss)
13 n 4 Date, local transaction (MMDD)
14 n 4 Date, expiration
15 n 4 Date, settlement
16 n 4 Date, conversion
17 n 4 Date, capture
18 n 4 Merchant type
19 n 3 Acquiring institution country code
20 n 3 PAN extended, country code
21 n 3 Forwarding institution. country code
22 n 3 Point of service entry mode
23 n 3 Application PAN number
24 n 3 Function code (ISO 8583:1993)/Network International identifier (NII)
25 n 2 Point of service condition code
26 n 2 Point of service capture code
27 n 1 Authorizing identification response length
28 n 8 Amount, transaction fee
29 n 8 Amount, settlement fee
30 n 8 Amount, transaction processing fee
31 n 8 Amount, settlement processing fee
32 n ..11 Acquiring institution identification code
33 n ..11 Forwarding institution identification code
34 n ..28 Primary account number, extended
35 z ..37 Track 2 data
36 n …104 Track 3 data
37 an 12 Retrieval reference number
38 an 6 Authorization identification response
39 an 2 Response code
40 an 3 Service restriction code
41 ans 16 Card acceptor terminal identification
42 ans 15 Card acceptor identification code
43 ans 40 Card acceptor name/location (1-23 address 24-36 city 37-38 state 39-40 country)
44 an ..25 Additional response data
45 an ..76 Track 1 data
46 an …999 Additional data – ISO
47 an …999 Additional data – national
48 an …999 Additional data – private
49 an 3 Currency code, transaction
50 an 3 Currency code, settlement
51 an 3 Currency code, cardholder billing
52 b 64 Personal identification number data
53 n 18 Security related control information
54 an …120 Additional amounts
55 ans …999 Reserved ISO
56 ans …999 Reserved ISO
57 ans …999 Reserved national
58 ans …999 Reserved national
59 ans …999 Reserved for national use
60 an .7 Advice/reason code (private reserved)
61 ans …999 Reserved private
62 ans …999 Reserved private
63 ans …999 Reserved private
64 b 16 Message authentication code (MAC)
65 b 64 *Bit indicator of tertiary bitmap only*, tertiary bitmap data follows secondary in message stream.
66 n 1 Settlement code
67 n 2 Extended payment code
68 n 3 Receiving institution country code
69 n 3 Settlement institution country code
70 n 3 Network management information code
71 n 4 Message number
72 ans …999 Data record (ISO 8583:1993)/n 4 Message number, last(?)
73 n 6 Date, action
74 n 10 Credits, number
75 n 10 Credits, reversal number
76 n 10 Debits, number
77 n 10 Debits, reversal number
78 n 10 Transfer number
79 n 10 Transfer, reversal number
80 n 10 Inquiries number
81 n 10 Authorizations, number
82 n 12 Credits, processing fee amount
83 n 12 Credits, transaction fee amount
84 n 12 Debits, processing fee amount
85 n 12 Debits, transaction fee amount
86 n 15 Credits, amount
87 n 15 Credits, reversal amount
88 n 15 Debits, amount
89 n 15 Debits, reversal amount
90 n 42 Original data elements
91 an 1 File update code
92 n 2 File security code
93 n 5 Response indicator
94 an 7 Service indicator
95 an 42 Replacement amounts
96 an 8 Message security code
97 n 16 Amount, net settlement
98 ans 25 Payee
99 n ..11 Settlement institution identification code
100 n ..11 Receiving institution identification code
101 ans 17 File name
102 ans ..28 Account identification 1
103 ans ..28 Account identification 2
104 ans …100 Transaction description
105 ans …999 Reserved for ISO use
106 ans …999 Reserved for ISO use
107 ans …999 Reserved for ISO use
108 ans …999 Reserved for ISO use
109 ans …999 Reserved for ISO use
110 ans …999 Reserved for ISO use
111 ans …999 Reserved for ISO use
112 ans …999 Reserved for national use
113 n ..11 Authorizing agent institution id code
114 ans …999 Reserved for national use
115 ans …999 Reserved for national use
116 ans …999 Reserved for national use
117 ans …999 Reserved for national use
118 ans …999 Reserved for national use
119 ans …999 Reserved for national use
120 ans …999 Reserved for private use
121 ans …999 Reserved for private use
122 ans …999 Reserved for private use
123 ans …999 Reserved for private use
124 ans …255 Info text
125 ans ..50 Network management information
126 ans …999 Issuer trace id
127 ans …999 Reserved for private use
128 b 16 Message authentication code

Implementation and understanding the Code

The first step to understanding the packing and unpacking of the message fields would be to create a class that describes then entire structure.

First we create a class and create a dictionary that describes the message formats:

 #2805 contants
 _DEF = {}
 # Every _DEF has:
 # _DEF[N] = [X, Y, Z, W, K]
 # N = bitnumber
 # X = smallStr representation of the bit meanning
 # Y = large str representation
 # Z = length indicator of the bit (F, LL, LLL, LLLL, LLLLL, LLLLLL)
 # W = size of the information that N need to has
 # K = type os values a, an, ans, n, xn, b
 _DEF[1] = ['BM', 'Bit Map Extended', 'F', 8, 'b']
 _DEF[2] = ['2', 'Primary Account Number (PAN)', 'LL', 19, 'n']
 _DEF[3] = ['3', 'Processing Code', 'F', 6, 'n']
 _DEF[4] = ['4', 'Amount Transaction', 'F', 12, 'n']
 _DEF[5] = ['5', 'Amount Settlement', 'F', 12, 'n']
 _DEF[7] = ['7', 'Transmission Date and Time', 'F', 10, 'n']
 _DEF[9] = ['9', 'Conversion Rate, Settlement', 'F', 8, 'n']
 _DEF[10] = ['10', 'Conversion Rate, Cardholder Billing', 'F', 8, 'n']
 _DEF[11] = ['11', 'Systems Trace Audit Number', 'F', 6, 'n']
 _DEF[12] = ['12', 'Time, Local Transaction', 'F', 6, 'n']
 _DEF[13] = ['13', 'Date, Local Transaction', 'F', 4, 'n']
 _DEF[14] = ['14', 'Date, Expiration', 'F', 4, 'n']
 _DEF[15] = ['15', 'Date, Settlement', 'F', 4, 'n']
 _DEF[16] = ['16', 'Date, Conversion', 'F', 4, 'n']
 _DEF[18] = ['18', 'Merchant Type', 'F', 4, 'n']
 _DEF[22] = ['22', 'POS Entry Mode', 'F', 3, 'n']
 _DEF[23] = ['23', 'Card Sequence Number', 'F', 3, 'n']
 _DEF[25] = ['25', 'POS Condition Code', 'F', 2, 'n']
 _DEF[28] = ['28', 'Amount, Transaction Fee', 'F', 9, 'xn']
 _DEF[32] = ['32', 'Acquiring Institution ID Code', 'LL', 11, 'n']
 _DEF[33] = ['33', 'Forwarding Institution ID Code', 'LL', 11, 'n']
 _DEF[35] = ['35', 'Track 2 Data', 'LL', 37, 'an']
 _DEF[37] = ['37', 'Retrieval Reference Number', 'F', 12, 'an']
 _DEF[38] = ['38', 'Authorization ID Response', 'F', 6, 'an']
 _DEF[39] = ['39', 'Response Code', 'F', 2, 'an']
 _DEF[41] = ['41', 'Card Acceptor Terminal ID', 'F', 8, 'ans']
 _DEF[42] = ['42', 'Card Acceptor ID Code', 'F', 15, 'ans']
 _DEF[43] = ['43', 'Card Acceptor Name Location', 'F', 40, 'asn']
 _DEF[44] = ['44', 'Additional Response Data', 'LL', 25, 'ans']
 _DEF[47] = ['47', 'Additional Data National', 'LLL', 999, 'ans']
 _DEF[48] = ['48', 'Additional Data Private', 'LLL', 999, 'ans']
 _DEF[49] = ['49', 'Currency Code, Transaction', 'F', 3, 'n']
 _DEF[50] = ['50', 'Currency Code, Settlement', 'F', 3, 'n']
 _DEF[51] = ['51', 'Currency Code, Billing', 'F', 3, 'n']
 _DEF[52] = ['52', 'PIN Data', 'F', 8, 'b']
 _DEF[53] = ['53', 'Security Related Control Information', 'F', 48, 'b']
 _DEF[55] = ['55', 'ICC Data', 'LLL', 999, 'b']
 _DEF[57] = ['57', 'Amount Cash', 'F', 12, 'n']
 _DEF[58] = ['58', 'Ledger Balance', 'F', 12, 'n']
 _DEF[59] = ['59', 'Account Balance', 'F', 12, 'n']
 _DEF[64] = ['64', 'Message Authentication Code', 'F', 8, 'b']
 _DEF[66] = ['66', 'Settlement Code', 'F', 1, 'n']
 _DEF[70] = ['70', 'Network Management Information Code', 'F', 3, 'n']
 _DEF[74] = ['74', 'Credits, Number', 'F', 10, 'n']
 _DEF[75] = ['75', 'Credits, Reversal Number', 'F', 10, 'n']
 _DEF[76] = ['76', 'Debits, Number', 'F', 10, 'n']
 _DEF[77] = ['77', 'Debits, Reversal Number', 'F', 10, 'n']
 _DEF[78] = ['78', 'Transfer, Number', 'F', 10, 'n']
 _DEF[79] = ['79', 'Transfer, Reversal Number', 'F', 10, 'n']
 _DEF[80] = ['80', 'Inquiries, Number', 'F', 10, 'n']
 _DEF[81] = ['81', 'Authorizations, Number', 'F', 10, 'n']
 _DEF[83] = ['83', 'Credits, Transaction Fee Amount', 'F', 12, 'n']
 _DEF[85] = ['85', 'Debits, Transaction Fee Amount', 'F', 12, 'n']
 _DEF[86] = ['86', 'Credits, Amount', 'F', 16, 'n']
 _DEF[87] = ['87', 'Credits, Reversal Amount', 'F', 16, 'n']
 _DEF[88] = ['88', 'Debits, Amount', 'F', 16, 'n']
 _DEF[89] = ['89', 'Debits, Reversal Amount', 'F', 16, 'n']
 _DEF[90] = ['90', 'Original Data Elements', 'F', 42, 'n']
 _DEF[97] = ['97', 'Amount, Net Settlement', 'F', 17, 'xn']
 _DEF[99] = ['99', 'Settlement Institution ID Code', 'LL', 11, 'n']
 _DEF[100] = ['100', 'Receiving Institution ID Code', 'LL', 11, 'n']
 _DEF[112] = ['112', 'Key Management Data', 'LLL', 999, 'b']
 _DEF[118] = ['118', 'Cash Total Number', 'LLL', 10, 'n']
 _DEF[119] = ['119', 'Cash Total Amount', 'LLL', 10, 'n']
 _DEF[128] = ['128', 'MAC Extended', 'F', 8, 'b']

To write this structure should not be that difficult and should come directly out of the specification provided my your institution.

As for the infamous bitmaps we are going to describe a structure for them as well, with an array to track the bit positions.

 # Bits to be set 00000000 -> _BIT_POSITION_1 ... _BIT_POSITION_8
 _BIT_POSITION_1 = 128 # 10 00 00 00
 _BIT_POSITION_2 = 64 # 01 00 00 00
 _BIT_POSITION_3 = 32 # 00 10 00 00
 _BIT_POSITION_4 = 16 # 00 01 00 00
 _BIT_POSITION_5 = 8 # 00 00 10 00
 _BIT_POSITION_6 = 4 # 00 00 01 00
 _BIT_POSITION_7 = 2 # 00 00 00 10
 _BIT_POSITION_8 = 1 # 00 00 00 01

 #Array to translate bit to position

Now we simply add a heap of helper functions from  http://www.vulcanno.com.br/python/ISO8583.html to fill and decode the data structure.

I have created a client and a server as part of the source code, so you can test your own implementation of the AS2805 protocol

Simple as pie!



Dynamic Key Exchange Models

Dynamic Key Exchange Models

I’ve had a number of people ask me recently about how to implement Dynamic Key Exchange models.  Specifically, I’m talking here about ISO8583-based financial payment gateways.  This post pertains to situations where you’re acting either as the Card Issuer (in which case you’re receiving payment transaction requests from the gateway) or the transaction acquirer (in which case you’re sending payment transaction requests to the gateway in order that they route it for appropriate authorization decisioning).

There’s some terminology to square away first:

Local Master Key (‘LMK’) – This is the key you store in the HSM in order to encrypt and do software-based storage of the current Working Keys (and Base Derivation Keys if you’re using DUKPT).  Also called the Master File Key (‘MFK’)

Zone PIN Key (ZPK’) – The ZPK is what’s used to encrypt the PIN blocks that traverse the wires between institutions.  Also referred to as the ‘Working Key.’  This is the key that the Dynamic Key Exchange is acting upon.  You’re obligated to change the Working Key at agreed-upon intervals (I typically advocate every 12 hours).

Zone Master Key (‘ZMK’) –  Think of the ZMK as the key transportation vehicle.  It’s the key that the two parties use to encrypt and exchange new ZPKs.  This key is established via a key ceremony.  You keep a copy of the ZMK encrypted under the LMK in a file somewhere (you’ll see how it’s used here further down this post).  Also called the Key Exchange Key (‘KEK’).

  1. From the moment you start planning discussions with the gateway, establish RIGHT AWAY that you want field-by-field level specifics of how the Dynamic Key Exchange is to be performed.  It’ll be within the context of some  Network Message Exchange (e.g., 0800/0810), but that’s not granular enough – you need to know the thing down to the field-content level.
  2. Scour the documentation you’ve been provided to see if those details are in there.  I’ve done two different gateway projects recently, and in both cases the Key Exchange details were notably absent from the doc.  But, that doc exists somewhere within the gateway institution.  Track it down.  Get your hands on it.
  3. Knowledge of the Key Exchange model is – by design – not widespread throughout the gateway provider’s project personnel.  Insist on getting their expert in on at least one of the planning calls.  Make note of this person’s name and contact details.  Establish that information channel.  This is a critically important point to your success.

At a high level, there are two models:

  • You request a new ZPK from the gateway, and they provide it in the response.  [I call this the ‘Pull‘ model (for obvious reasons – you pull the key from them).]
  • The gateway sends you a new ZPK and you respond with a message indicating success or failure.  [This, by contrast is the ‘Push‘ model.]

Your implementation will be one of those.

Now, I’ll provide two examples, one push, one pull.

The sequence of events is:

  1. The gateway sends us a new ZPK (under ZMK) in an 0800 (MTI) Network Request.
  2. We obtain the ZMK (under LMK) from our files.
  3. We use the cryptograms from Steps 1 and 2 to create the appropriate command to the Key-Up (here, a ’12’)
  4. We get the response from the Key-Up (the ’13’) and validate that the Check Digits match those provided by the Issuer.
  5. Assuming the check in Step 4 is okay, we store the result (the ZPK under LMK) as the new Working Key.
  6. We send an 0810 (MTI) Network Response back to the Issuer (Note that Field 39 on our response is ’00’ – indicating success).

There’s so much detail here worthy of comment.  I’ll touch on a few things (these are the types of detail you want to bring to the surface in your reviews):

  • This gateway uses ‘162’ in Field 70 to tip to us that it’s a Key Exchange in play.
  • Note how we have to pluck the incoming cryptogram out of the esoteric morass of Field 123.
  • We have to construct an equally cryptic Field 123 on our response.


Here is the pull model:

  1. We request a new key from the Gateway in an 0800.
  2. The new key (ZPK under ZMK) comes back in an 0810.
  3. We fire off an ‘FA’ to the Thales 8000.
  4. We get the ‘FB’ back and validate the check digits.
  5. If okay, we store the result (the ZPK under LMK) as the new Working Key.

Now, since we’re the initiator here we have to have a way to determine when to trigger the exchange request.  We do that through a channel Logon Manager.

You get the idea, I hope!  Nail down all those details in order to maximize your chances of success.  Otherwise, feel free to beat your head against a wall, because that’s what will happen if you don’t get this information.

Doing PIN Translation with DUKPT

On PIN-enabled Debit/EBT transactions sent in from an acquirer’s point-of-sale location, your payment switch application must perform a PIN translation, typically transforming an incoming DUKPT PIN block from the POS device-initiated request into a outgoing Triple DES-encrypted PIN block that makes use of an established Zone PIN Key (“ZPK”) which would have been previously established via a dynamic key exchange with your Debit/EBT gateway provider.

[The remainder of this example assumes you’re using a Thales (formerly Racal) Hardware Security Module (“HSM”)….] Using strict Thales parlance, this variant of a PIN translation request is a request to “translate a PIN from *BDK encryption to interchange key encryption.”  This topic is covered in Section 27.2 (page 2-185) of the Thales reference document entitled “Host Security Module RG7000 Programmer’s Manual” (Thales reference number 1270A514 Issue 5). The CI/CJ exchange should be handled as follows: — CI — Message header – You can use as you see fit. Value is echoed back in CJ.  Note that the length is constant and must be configured in HSM by administrator. Command code – CI BDK – The Base Derivation Key “in play” for this transaction.  In my installations we’ve set this up as follows…

  • Selected the “1A + 32H” option, where the ‘1A’ value should be set to ‘U’
  • Configured such that the first six positions of the KSN represent the “key name” of the BDK injected into the PIN Pad at the transaction origination point (an acquirer can use a number of BDKs in their terminal population).

ZPK – Your current ZPK Cryptogram (obtained dynamically via a key exchange with your Debit/EBT  gateway partner) and stored under your Local Master Key (“LMK”).  In my installations, we’ve used the “1A + 32H” option, where the ‘1A’ value should be set to ‘U’. KSN Descriptor – This value is a bit esoteric and refers directly to the make-up of the KSN which follows.  So to understand the descriptor, it’s first necessary to talk a bit about the KSN (the next field in the CI command layout).  Here’s a typical KSN implementation where the acquirer has chosen a 16-position scheme:

  • Positions 1 – 6: The name of the BDK injected into this device
  • Positions 7 – 11:  The device ID
  • Positions 12 – 16: The transaction counter

[Note that the KSN implementation has to be in synch between the PIN pad and your host-side implementations in order for this to work.] The ‘rules’ for a KSN construction are as follows (reading from left to right in the KSN): a. The ‘base derivation key identifier,’ which is mandatory and five to nine (Hex) positions in length. b. A ‘sub-key identifier,’ which Thales says is ‘optional’ but in practice is ‘reserved for future use’ (and therefore always set to zero). c. A ‘device identifier’ (mandatory), which is two to five Hex digits in length. d. A ‘transaction counter’ (mandatory) which essentially is the part “left over”. So, in the example here, the client with a 6, 0, 5, 5 implementation. With this information in hand, the KSN Descriptor (a three-position value) is better described as XYZ, where: X = base derivation key identifier length Y = sub-key identifier key length (will be zero) Z = device identifier length So, in this context, the ‘605’ submitted in my example is better visualized. ‘605’ says that the 16-digit KSN consists of a 6-position BDK ID, a 0-position sub-key, a 5-position device ID, **AND** (what’s remaining basically) a 5-position transaction counter. [NOTE: Remember that this post applies *specifically* to the Thales/RACAL implementation of PIN translation] Now, with this informatation in hand, we can introduce the next field, the KSN itself… KSN – Using the layout from the descriptor, a typical KSN at this acquirer might be 123456000A8001D4 where: ‘123456’ is the BDK indentifier; ‘000A8’ is the Device ID; and ‘001D4’ is the transaction counter. The BDK name embedded in a particular KSN string must find a match within your BDK cryptogram list (which you need to keep loaded into your payment switch’s encryption database).  If a match is not found in the encryption database, then set your Internal Result Code to “Invalid BDK” and end the transaction.  If found, the value you retrieve goes into the BDK field (as described above). Source encrypted block – The PIN block plucked from the POS device request (this is a 16H value; no ‘1A’ indicator is required). Destination PIN block – In my installations, we typically use ANSI format, so we set this to ‘01’ to signify ANSI format code Account Number – Right-most 12 positions of the PAN excluding the check digit Typically, that is the END of the required CI request message (remainder of the fields in the Thales spec are not mandatory). — CJ — Message header – Echoed back from CI usage. Response code – CJ Error Code – Only ‘00’ should be accepted as an exchange that “worked.” PIN length – Although this field is not used to build the 0200 message formatted for your Debit/EBT gateway, a value like ‘04’ or ‘05’ here are a pretty good indication that the translation occurred successfully. Encrypted PIN – The PIN block that will be used to build the 0200 message formatted for your Debit/EBT gateway (this is a 16H value; no ‘1A’ indicator is required). Destination PIN block – Echoed back from the device as ’01’ format code Typically, that is the END of the response message (remainder of list in the vendor spec would only be present if they were provided in the CI command request)

No comments

AS2805 Standards for EFT

Australia Standards 2805 (AS2805) is the standard for Electronic Funds Transfer (EFT) and Payments in Australia and New Zealand. AS2805 is also used for some implementations in South Africa and SE Asian.

AS2805 is owned by Australia Standards and was developed by various voluntary working groups within Committee IT/5. The implementation of AS2805 standards across all industries is clearly defined by the Australian Payments Clearing Association (APCA) as part of the Consumer Electronic Clearing System (CECS) and detailed in the CECS Manual.

Contrary to popular belief AS2805 is not a rename of the ISO8583 standard in the Australia Standards numbering system, as is the case with most international standards.

ISO8583 was first published in 1987, while AS2805 was published two years earlier in 1985, after a lengthy period of draft and review in Australia, New Zealand and South Africa. ISO8583 consists of three (3) parts:

  • Part 1: Messages, Data Elements and Code Values
  • Part 2: Application and Registration Procedures for Institution Identification Codes (IIC)
  • Part 3: Maintenance Procedures for Messages, Data Elements and Code Values

All three (3) parts of ISO8583 are concentrated on only message formats between devices (EFTPOS and ATM) and an acquiring host. ISO8583 can be seen as a small subset of the AS2805 standard and there is no clear guide for uniform implementation as is the case with CECS. AS2805 on the other hand consist of at least thirty three (33) separate published parts and covers general EFT topics such as:

  • Card Management & Authorisation
  • Card Detail Updating
  • PIN Management
  • Key Management and Security
  • Message Authentication
  • Privacy and Data Encryption
  • Communications
  • Message Structure between Devices and Acquiring Host
  • Message Structure between Hosts
  • File Transfers

The thirty three (33) AS2805 standards published so far are the following:

2805.1 Part 1: Communications
2805.2 Part 2: Message Structure, format and content
2805.3.1 Part 3.1: PIN Management and Security – General
2805.3.2 Part 3.2: PIN Management and Security – Offline
2805.4.1 Part 4.1: Message Authentication – Mechanisms Using a Block Cipher
2805.4.2 Part 4.2: Message Authentication – Mechanisms Using a Hash Function
2805.5.1 Part 5.1: Ciphers – Data Encipherment Algorithm 1 (DEA 1)
2805.5.2 Part 5.2: Ciphers – Modes of Operation for an n-bit block cipher algorithm
2805.5.3 Part 5.3: Ciphers – Data Encipherment Algorithm 2 (DEA 2)
2805.5.4 Part 5.4: Ciphers – Data Encipherment Algorithm 3 (DEA 3) & related techniques
2805.6.1.1 Part 6.1.1: Key Management – Principles
2805.6.1.2 Part 6.1.2: Key Management – Symmetric Ciphers, their Key Management & Life Cycle
2805.6.1.4 Part 6.1.4: Key Management – Asymmetric Cryptosystems – Key Management & Life Cycle
2805.6.2 Part 6.2: Key Management – Transaction keys
2805.6.3 Part 6.3: Key Management – Session Keys – Node to Node
2805.6.4 Part 6.4: Key Management – Session Keys – Terminal to Acquirer
2805.6.5.1 Part 6.5.1: Key Management – TCU Initialisation – Principles
2805.6.5.2 Part 6.5.2: Key Management – TCU Initialisation – Symmetric
2805.6.5.3 Part 6.5.3: Key Management – TCU Initialisation – Asymmetric
2805.6.6 Part 6.6: Key Management – Session Keys – Node to Node with KEK Replacement
2805.9 Part 9: Privacy of Communications
2805.10.1 Part 10.1: File Transfer Integrity Validation
2805.10.2 Part 10.2: Secure File Transfer (Retail)
2805.11 Part 11: Card Parameter Table
2805.12.1 Part 12.1: Message Content – Structure and Format
2805.12.2 Part 12.2: Message Content – Codes
2805.12.3 Part 12.3: Message Content – Maintenance of Codes
2805.13.1 Part 13.1: Secure Hash Functions – General
2805.13.2 Part 13.2: Secure Hash Functions – MD5
2805.13.3 Part 13.3: Secure Hash Functions – SHA-1
2805.14.1 Part 14.1: Secure Cryptographic Devices (Retail) – Concepts, Requirements and Evaluation Methods
2805.14.2 Part 14.2: Secure Cryptographic Devices (Retail) – Security Compliance Checklist for Devices used in Financial Transactions
2805.16 Part 16: Merchant Category Codes

The AS2805 standard also provides three (3) published Handbooks related to the AS2805 standard:

HB 127 EFT – Implementing Message Content Standards – Conversion Handbook
HB 128 EFT – Implementing Message Content Standards – Terminal Handbook
HB 129 EFT – Implementing Message Content Standards – Interchange Handbook

There are a number of guideline white papers available to assist the implementation of EFT related functionality such as:

  • Card Management & Production
  • EFTPOS/POS Software Management
  • EFTPOS and POS Product Management
  • Software and Configuration File Downloading
  • Retail Electronic Data Exchange (EDT) that covers price downloads, ordering and statistics
  • Retail Automation
  • Terminal Management
  • Merchant Management
  • Cashier Management
  • Fraud Monitoring and Management

Trace your ATM Transactions


Generally when entrepreneurs decide to become ATM deployers, they do not have sufficient knowledge about ATM protocols and specifications. This is not needed as there are switching providers that can switch their ATM’s  transactions and provide them with adequate reporting.

Following this approach generally leaves them with a massive gap in terms of managing their terminals and merchants  correctly. ATM switching providers have the ability to decode the terminal status messages in real time and determine for example is a terminal has ran out of cash, if hardware is busy failing or even if a processing bank has gone offline.

This allows switching providers to have the upper hand over the smaller ATM deployers.

In this post I will show you how to develop a middleware where this pro active approach can be followed even with small ATM deployers. You will be able to see live cash levels of your terminals and monitor ATM transactions.


ATM Languages

ATM’s  and EFTPOS devices speak different languages and each terminal manufacturer might have their own custom implementation. Terminal developers  normally follow a few types of language implementations: Triton / NDC+ / AS2805 / ISO8583. As all of these will run on TCP using some sort of control protocol (VISAII / ACK Controlled) we need to decode the control elements of the protocol as well.

I will be using the Standard Triton Protocol over VISAII to demonstrate the ability to trace transactional protocols.

Most ATM’s have an TCP/IP setting that will enable you to point the ATM to an IP Address and port. Writing an Server component to listen to this is of course as easy as pie.

Conversion to ASCII

It is an easy approach to write a server application and listen on a post for incoming transactions but we will quickly notice that these message are encoded. This encoding it an easy hurdle (as long as there is no SSL component on the ATM)

StringToAscii Class – The readable encoding

The first class I’m going to create is an class that can convert encoded strings to readable ascii, this need the be based on the control characters specified in your protocol specification.

Each Hex encoded string needs to be mapped to the format you require. (<ETX> / <STX>) and a simple function can provide the conversion.

controls_dic = {
 0: '<NUL>', 1: '<SOH>', 2: '<STX>', 3: '<ETX>', 4: '<EOT>', 5: '<ENQ>', 6: '<ACK>',
 7: '<BEL>', 8: '<BS>' , 9: '<HT>' , 10: '<LF>' , 11: '<VT>' , 12: '<FF>' , 13: '<CR>' ,
14: '<SO>' , 15: '<SI>' , 16: '<DLE>', 17: '<DC1>', 18: '<DC2>', 19: '<DC3>',
20: '<DC4>', 21: '<NAK>', 22: '<SYN>', 23: '<ETB>', 24: '<CAN>', 25: '<EM>',
26: '<SUB>', 27: '<ESC>', 28: '<FS>' , 29: '<GS>' , 30: '<RS>' , 31: '<US>'

def Str2Ascii(s):
 r = ''
 for c in s:
 if ord(c) < 32:
 r = r + controls_dic[ord(c)]
 elif ord(c) >= 127:
 r = r + '<%d>' % ord(c)
 r = r + c
 return r

The reverse conversion is slightly more complex, but in essence its just a reverse function of the above.

reverse_dic5 = { 
'<NUL>': 0, '<SOH>': 1, '<STX>': 2, '<ETX>': 3, '<EOT>': 4, '<ENQ>': 5, '<ACK>': 6,
'<BEL>': 7, '<DLE>': 16, '<DC1>': 17, '<DC2>': 18, '<DC3>': 19,
'<DC4>': 20, '<NAK>': 21, '<SYN>': 22, '<ETB>': 23, '<CAN>': 24, 
'<SUB>': 26, '<ESC>': 27

reverse_dic4 = {
'<BS>' : 8, '<HT>' : 9, '<LF>' : 10, '<VT>' : 11, '<FF>' : 12, '<CR>' : 13,
'<SO>' : 14, '<SI>' : 15, '<EM>' : 25, '<FS>' : 28, '<GS>' : 29, '<RS>' : 30, '<US>' : 31

reverse_dicN = {
 '<130>':130, '<140>':140, '<150>':150, '<160>':160, '<170>':170, '<180>':180, '<190>':190, '<200>':200, '<210>':210, '<220>':220, '<230>':230, '<240>':240, '<250>':250, 
 '<131>':131, '<141>':141, '<151>':151, '<161>':161, '<171>':171, '<181>':181, '<191>':191, '<201>':201, '<211>':211, '<221>':221, '<231>':231, '<241>':241, '<251>':251, 
 '<132>':132, '<142>':142, '<152>':152, '<162>':162, '<172>':172, '<182>':182, '<192>':192, '<202>':202, '<212>':212, '<222>':222, '<232>':232, '<242>':242, '<252>':252, 
 '<133>':133, '<143>':143, '<153>':153, '<163>':163, '<173>':173, '<183>':183, '<193>':193, '<203>':203, '<213>':213, '<223>':223, '<233>':233, '<243>':243, '<253>':253, 
 '<134>':134, '<144>':144, '<154>':154, '<164>':164, '<174>':174, '<184>':184, '<194>':194, '<204>':204, '<214>':214, '<224>':224, '<234>':234, '<244>':244, '<254>':254, 
 '<135>':135, '<145>':145, '<155>':155, '<165>':165, '<175>':175, '<185>':185, '<195>':195, '<205>':205, '<215>':215, '<225>':225, '<235>':235, '<245>':245, '<255>':255, 
 '<136>':136, '<146>':146, '<156>':156, '<166>':166, '<176>':176, '<186>':186, '<196>':196, '<206>':206, '<216>':216, '<226>':226, '<236>':236, '<246>':246, 
'<127>' : 127, '<137>':137, '<147>':147, '<157>':157, '<167>':167, '<177>':177, '<187>':187, '<197>':197, '<207>':207, '<217>':217, '<227>':227, '<237>':237, '<247>':247, 
'<128>' : 128, '<138>':138, '<148>':148, '<158>':158, '<168>':168, '<178>':178, '<188>':188, '<198>':198, '<208>':208, '<218>':218, '<228>':228, '<238>':238, '<248>':248, 
'<129>' : 129, '<139>':139, '<149>':149, '<159>':159, '<169>':169, '<179>':179, '<189>':189, '<199>':199, '<209>':209, '<219>':219, '<229>':229, '<239>':239, '<249>':249 

def Ascii2Str(s):
 r = ''
 i = 0
 l = len(s)
 while i < l:
 if s[i] == '<':
 x = reverse_dic5[s[i:i+5]]
 r = r + chr(x)
 i = i + 5
 except KeyError:
 x = reverse_dic4[s[i:i+4]]
 r = r + chr(x)
 i = i + 4
 except KeyError:
 x = reverse_dicN[s[i:i+5]]
 r = r + chr(x)
 i = i + 5
 except KeyError:
 r = r + s[i]
 i = i + 1 
 r = r + s[i]
 i = i + 1
 return r

We can now pass a simulated transaction to the function and test the output and see if the class is correctly implemented acceding to the Triton Standard Protocol.


message = '05 02 39 56 44 44 39 30 30 32 32 30 30 30 30 30 32 1C 31 31 1C 30 34 30 36 1C 34 32 31 36 34 36 30 30 30 30 30 30 30 30 30 38 3D 31 34 31 32 31 30 31 31 31 38 38 35 32 34 39 31 32 33 34 35 1C 30 30 30 30 32 30 30 30 1C 30 30 30 30 30 30 30 30 1C 38 36 32 37 42 38 36 33 34 37 41 30 33 37 37 35 1C 1C 1C 1C 5E 35 45 36 32 20 44 35 37 37 1C 03 6E '.replace(" ", "").decode("hex")


<ENQ><STX>9U00009D       <FS>11<FS>0406<FS>4216460000000008=14121011188524912345<FS>00002000<FS>00000000<FS>8627B86347A03775<FS><FS><FS><FS>^5E62 D577<ETX>

Reversing this back to an encoded string we just need to pass it to the ASCIIToString Function of course.

Tracing Data

Now that we have all the field from the request from the ATM, the next step is to create a server component that can run and pass this transactional information to our processing party.

Creating a Server is not part of this post, but all source code is available in the gitbub repository. But the basic structure should be that you should create a incoming and sending socket, and poll for connections using a daemon thread.  Passing ALL information to the processing party, but saving the requests to the database.

Port Listening for ATM connections

 self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

 self.sock.bind(('', int(listening_port)))


Port forwarding to Processor with SSL

fwd = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

 ssl_sock = ssl.wrap_socket(fwd)
 ssl_sock.connect((session.destination_ip, int(session.destination_port)))
 self.log.info('SSL Handshake Completed')

 self.log.debug('%s: Connected to switch %s:%s' % (
 session.SessionUUID, session.destination_ip, session.destination_port,))
 except socket.timeout:
 session.error = "Timeout connecting to switch"
 self.log.critical('%s: EE Cant Connect to %s:%s : %s' % (
 session.SessionUUID, session.destination_ip, session.destination_port, session.error,))
 except socket.sslerror:
 session.error = "SSL Error connecting to TNS"
 self.log.critical('%s: EE Cant Connect to %s:%s : %s' % (
 session.SessionUUID, session.destination_ip, session.destination_port, session.error,))
 session.error = "Unknown Error connecting to switch"
 self.log.exception('%s: EE Unknown Error Connecting to %s:%s : %s' % (
 session.SessionUUID, session.destination_ip, session.destination_port, session.error,))
 RequestThread(session, newsock, ssl_sock).start()
 ResponseThread(session, ssl_sock, newsock).start()

When a specific Transaction type is received from the ATM you can dissect the readable ascii and save the request to database, when the corresponding session has a response then you can of course save the response information with the request.

In my code example I provide the implementation of request  and response saving as based on the Triton Specification. If you are using NDC+ or ISO 8583 then this will dramatically change.


The security aspect of this project should be implemented in the raw socket components (SSL Sockets), if there is a need for SSL certificate validation and ACL’s then it should not be difficult to add this to the required class.

PCI DSS require us not to store raw Personal Account numbers in the database so we should in fact use a hash function.

A simple method for doing this is the following:

return pan[:6] + ("*" * (len(pan)-10)) + pan[-4:]

Final Product

The final product is a python implementation of a Transactional Middleware. Allot of changes will be required to make the project work for your environment, but here are basic instructions to make it work for you.

  1. Get a clean Linux installation
  2. Install the LAMP Stack
  3. run the SQL file in the project
  4. copy the project directory to the server
  5. app_get all the project dependencies. (pymysql ect.)
  6. change the config file to point to your database and your switching provider (Middleware.ini)
  7. create a screen session (command: screen -S TransactionServer )
  8. start the Server in the screen session (sudo python pyMiddlewareServer foreground)

When starting it up you should see the following:

2014-05-24 17:53:38,177 MyDaemon INFO run() Start
2014-05-24 17:53:38,180 Pinhole INFO Redirecting: localhost:9100 -> xxx.xxx.xxx.xx:xxxxx
2014-05-24 17:53:38,263 QueueLogger INFO Connected to Database <_mysql.connection open to '' at 100837020>
2014-05-24 17:53:45,704 Pinhole INFO 8850226b-e318-11e3-8a9a-28cfe91efd87: NN New Session from ('', 49660)
2014-05-24 17:53:45,817 Pinhole INFO SSL Handshake Completed
2014-05-24 17:53:45,832 Request INFO 8850226b-e318-11e3-8a9a-28cfe91efd87: > AtmID=[9VDD90022000002], Type=[Authorization from Cheque Account], Operation=[CW]
2014-05-24 17:53:45,832 Request INFO 8850226b-e318-11e3-8a9a-28cfe91efd87: > PAN=[216460000000008]
2014-05-24 17:53:46,072 Request INFO 8850226b-e318-11e3-8a9a-28cfe91efd87: > AtmID=[9VDD90022000002], Type=[Authorization from Cheque Account], Operation=[CW]
2014-05-24 17:53:46,072 Request INFO 8850226b-e318-11e3-8a9a-28cfe91efd87: > PAN=[216460000000008]

Source code: https://github.com/Arthurvdmerwe/ATM-Transaction-Trace.git