Extracting data from Shopify Part 1

"Once upon a time I was asked to provide assistance with extracting data from Shopify."

Background

Our customer company XYZ is based in US would like to use Advanced ETL Processor to connect to shopping platforms, such as Shopify, Big Commerce, Amazon and Volusion.

Extracting data from Shopify Part 2

Loading ShopiFy data into the database.  

Introduction

In the previous post we discussed extracting data from ShopiFy. Once data is extracted usually there is need to load it into the database. The data is extracted in XML format. I personally never understood obsession with XML. It seems to me that a lot of time organizations follow current trends and forget about common sense.

Consider following example:
 

<CustomerOrderMessage>
<OrderNumber>1</OrderNumber>
</CustomerOrderMessage>

The actual data here is just one byte the rest is of it just metatada. To process it we need more hardware which is good for hardware vendors. Some times XML is so complicated so you have to be a rocket scientist to understand it. That leads to errors and drives the cost up.
We have a wiki page dedicated to this topic: Advantages and disadvantages of working with XML
Simplicity is the answer.

Here is ShopiFy orders XML file example (part of it)

<?xml version="1.0" encoding="UTF-8"?>
<orders type="array">
  <order>
    <id type="integer">3961189377</id>
    <email></email>
    <closed-at type="dateTime" nil="true"/>
    <created-at type="dateTime">2016-09-13T10:17:35-04:00</created-at>
    <updated-at type="dateTime">2016-10-03T14:24:15-04:00</updated-at>
    <number type="integer">4</number>
    <note>order for team</note>
    <token>8dfabd2504c936bb5da4bd00a406f6ba</token>
    <gateway>bogus</gateway>
    <test type="boolean">true</test>

The file is very complex and it has a hierarchical structure.

What we want to do is to convert it more readable format using XSLT