Data Pipelines Using Azure Data Factory

Leveraging cloud services makes us easy to do integration part of on-premise data sources with cloud services, ETL (Extract, Transform, and Load) has been a part of every digital transformation project that we undertake as…

Apache Spark Do’s and Don’ts

One of the main tools for every data engineer tools box would be apache-spark, knowing how to use it effectively and efficiently will pay him dividends in long run. There are many different tools in…

B-Tree on Big Data

One can say that with a strong conviction that most of the commercial success of MySQL comes from the InnoDB engine. Oracle acquired InnoDB several years before MySQL!I believe that Oracle acquired not only the…