Hiperspace.ISO20022.canm 1.0.4

Hiperspace.ISO20022

Hiperspace Domain Specific Database for ISO20022 messages sent via SWIFT and other Financial Institutions, and stored for historical, investigation and analysis purposes in a Hiperspace store.

All XSD Validation rules are included via the Validation property that can be applied using a Horizon filter.

NB Horizon validation rules are optional, to allow the same model to be used for durable valid storage, but also for logging purposes

The main Hiperspace.ISO20022 package aggregates all document types within the ISO20022 namespace, but for specific subject areas it is better to specific packages that only contain the elements that are needed for those document types

This initial release does not merge the duplicate definitions for common elements between subject areas

Domain Specific Database

This component provides a Domain specific database, as the structure can only be changed by amending the .hilang schemas and regenerating the model. Applications can manipulate the elements and query them using LINQ expressions (C#/F#/VB.net), using SQL (especially for Parquet export) or ODATA (BI), GPU Graph search, or Cube drilldown (using Hiperspace.DB)

The Types documentation can be found of GitHub.

Showcase

ISO20022 is a showcase for the architecture of Hiperspace:

• Elements can be serialized from XML documents to naturally compressed information that does not need to carry meta-data tags in the store
• As XML Documents are stored, they are split into constituent parts, depending on the type of element:
  • Entity : Document is stored with any one of its constituents as its key, since it is only an envelope, no value is stored
  • Entity : GroupHeader is stored by its key MsgID with other attributes stored as value
  • Entity : CustomerCreditTransferInitiation is stored with the key GrpHdr which is a reference to a GroupHeader
  • Segment : CreditTransferTransaction is a segment of CustomerCreditTransferInitiation with the additional key PmtId of type PaymentIdentification
  • Value : values can be any structure of elements with Lists and Sets. Values can contain other values and entity / segment / aspect transparently by value
• Key can be any structure that unambiguously serialized (including List but not Set) to 8 MiB of data
• Value can be any structure that can be serialized to 2 GiB of data
• Indexes are created automatically when an entity is referenced with an extent reference that is not a key
• Additional indexes can be explicitly added with the @AlternateIndex property

a sample XML document

<?xml version="1.0" encoding="UTF-8" ?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:pain.001.001.12">
    <CstmrCdtTrfInitn>
        <GrpHdr>
            <MsgId>Message-Id</MsgId>
            <CreDtTm>2024-05-10T16:10:02.017+00:00</CreDtTm>
            <NbOfTxs>1</NbOfTxs>
            <CtrlSum>510.24</CtrlSum>
            <InitgPty>
                <Id>
                	<OrgId>
                		<Othr>
                			<Id>Client-Id</Id>
                		</Othr>
                	</OrgId>
                </Id>
            </InitgPty>
        </GrpHdr>
        <PmtInf>
            <PmtInfId>Batch-Id</PmtInfId>
            <PmtMtd>TRF</PmtMtd>
            <ReqdExctnDt>YYYY-MM-DD</ReqdExctnDt>
            <Dbtr>
                <Nm>Debtor Account Holder Name</Nm>
            </Dbtr>
            <DbtrAcct>
                <Id>
                	<Othr>
                		<Id>Debtor Account Id</Id>
                	</Othr>
                </Id>
            </DbtrAcct>
            <DbtrAgt>
                <FinInstnId>
                	<BIC>BANK BIC</BIC>
                </FinInstnId>
            </DbtrAgt>
            <CdtTrfTxInf>
                <PmtId>
                	<EndToEndId>End-to-End-Id</EndToEndId>
                </PmtId>
                <Amt>
                	<InstdAmt Ccy="USD">510.24</InstdAmt>
                </Amt>
                <CdtrAgt>
                	<FinInstnId>
                		<BIC>BANK BIC</BIC>
                	</FinInstnId>
                </CdtrAgt>
                <Cdtr>
                	<Nm>Creditor Account Holder Name</Nm>
                </Cdtr>
                <CdtrAcct>
                	<Id>
                		<Othr>
                			<Id>Creditor Account ID</Id>
                		</Othr>
                	</Id>
                </CdtrAcct>
            </CdtTrfTxInf>
        </PmtInf>
    </CstmrCdtTrfInitn>
</Document>

can be loaded to Hiperspace with

var document = xmlSerialiser.Deserialize(stringReader) as Document;
document.Should().NotBeNull();
pain.Documents.BindAll(document);

and queried using LINQ (Language Integrated Query) in C# / F# / VB.net, or remotely using Hiperspace.SQL

var query = from d in read.Documents               // documents set
            from pay in d.CstmrCdtTrfInitn.PmtInf  // implicit join
            from trans in pay.CdtTrfTxInf          // implicit join
            where d.CstmrCdtTrfInitn.GrpHdr.MsgId == "Message-Id"
            select new
            { 
                d.CstmrCdtTrfInitn.GrpHdr.CtrlSum, 
                trans.Cdtr, 
                trans.CdtrAcct, 
                pay.Dbtr,
                pay.DbtrAcct,
                trans.Amt.Value.InstdAmt.Value.Ccy,
                trans.Amt.Value.InstdAmt.Value.Value
            };

// or directly              
var alter = from tran in read.CreditTransferTransaction61s
            where tran.owner.owner.GrpHdr.MsgId == "Message-Id"
            select new
            {
                tran.owner.owner.GrpHdr.CtrlSum,
                tran.Cdtr,
                tran.CdtrAcct,
                tran.owner.Dbtr,
                tran.owner.DbtrAcct,
                tran.Amt.Value.InstdAmt.Value.Ccy,
                tran.Amt.Value.InstdAmt.Value.Value
            };

The second query is preferable since it is a single key/value lookup in Hiperspace, but the first is better if XML/JSON is needed

Hiperspace LINQ and SQL automatically insert null coalescing logic so that

• A query of a document containing a CustomerPaymentReversal will not fail if the document does not have a CstmrCdtTrfInitn.GrpHdr.MsgId
• a query of trans.Amt.Value.InstdAmt.Value.Ccy will not fail if a different kind of payment is included

Hiperspace.SQL enables standard SQL to query hiperspace directly and output the result as a data-frame or Parquet file

the following SQL is translated into a cached query-plan that internally uses LINQ expressions to retrieve data.

    SELECT tran.Cdtr.Value.Nm
         , tran.CdtrAcct.Id.Value.Othr.Value.Id
         , tran.Amt.Value.InstdAmt.Value.Ccy
         , tran.Amt.Value.InstdAmt.Value.Value
    FROM   CreditTransferTransaction61s AS tran;

HiperSpace.SQL does not require the data to be flattened to a table prior to the query tran.Cdtr.Value.Nm returns text, but tran.Cdtr.Value could be selected instead and would return the JSON equivilent JSON structure