Over the last decade the amount of data that systems and devices generate has increased significantly. Because of this increase there is a significant driver for organisations to understand their own operations and ecosystems better. To create value and tap into the promise of this new data landscape, organisations need to evolve their methodologies and technologies.
The establishment of the Cloud as the preferred location for compute has been a game changer especially with regards to data analytics and insights. The Cloud provides a comprehensive set of data technologies that can store, transform, process, analyze, and visualize data in a secure way. The Cloud model has brought an important change to traditional on-premises hosting models in that cloud requires no upfront capital investment. Instead, an organization provisions services in the cloud and pays only for what it uses. This brings data analytics and insights, which was traditionally an expensive and investment heavy capability, to all organisations.
The traditional approach to data analytics has been to extract raw data from a structured or unstructured data pool and migrate it to a staging data repository. Because the data source might have a different structure than the target destination, the data from the source schema then is transformed to the destination schema. Following the transformation of the data it is then loaded into the data warehouse. These steps form a process called extract, transform, and load (ETL). This approach has several disadvantages, the biggest of which is the assumption that the required data model is understood upfront. An assumption which has been prevalent in computing and has since been disproven through the rise and adoption of Agile as an execution methodology.
Tangent prescribes an alternative approach, which is to extract, load, and transform (ELT). In ELT, the data is immediately extracted and loaded into a large data repository such as a Cloud Data Lake. This change in process reduces the resource contention on source systems. Data engineers can begin transforming the data as soon as the load is complete. ELT has more architectural flexibility to support multiple transformations without any loss of data due to early transformations in the data pipeline. For example, how the marketing department needs to transform the data can be different than how the operations department needs to transform that same data.
Through the utilization of the power of the cloud Tangent deliver next generation ELT architectures which allow for an agile execution empowering the analytics function to meet the evolving requirements of the modern organisation.